Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged nginx

Software Engineering

nginx buildpack – installed

It’s possible to add a custom buildpack to Stackato or Helion Development Platform so that it’s available to all applications. When using an installed buildpack it is not necessary to include a manifest or identify the buildpack. Instead it will be selected by the detect script in the buildpack. All files are on the cloud controller node which eliminates download time and bandwidth.

Package the buildpack

Prepare your buildpack for installation by adding all files to a zip file (of any name). The bin folder should be in the root of the zip file with any related files in their relative locations. Normally this will happen as expected if the buildpack is cloned and then within the buildpack directory zip is run, as shown.

vagrant@vagrant-ubuntu-trusty-64:~$ git clone -b precompiled https://github.com/dwatrous/buildpack-nginx.git
Cloning into 'buildpack-nginx'...
remote: Counting objects: 156, done.
remote: Compressing objects: 100% (99/99), done.
remote: Total 156 (delta 44), reused 142 (delta 32)
Receiving objects: 100% (156/156), 1.80 MiB | 2.64 MiB/s, done.
Resolving deltas: 100% (44/44), done.
Checking connectivity... done.
vagrant@vagrant-ubuntu-trusty-64:~$ cd buildpack-nginx/
vagrant@vagrant-ubuntu-trusty-64:~/buildpack-nginx$ zip -r buildpack-nginx.zip *
  adding: bin/ (stored 0%)
  adding: bin/detect (deflated 19%)
  adding: bin/compile (deflated 57%)
  adding: bin/release (deflated 2%)
  adding: bin/launch.sh (deflated 54%)
  adding: README.md (deflated 44%)
  adding: vendor/ (stored 0%)
  adding: vendor/nginx.tar.gz (deflated 4%)
vagrant@vagrant-ubuntu-trusty-64:~/buildpack-nginx$ cp buildpack-nginx.zip /vagrant/

It may be obvious that the commands above were run in a vagrant built Ubuntu box. Buildpack scripts must have the executable bit set, which doesn’t carry over when the zip file is created on windows. To accommodate this I created a vagrant box, installed git and cloned and zipped. I then copied the zip file into the vagrant folder for easy access.

The README.md isn’t necessary for the buildpack, obviously, but it doesn’t interfere either. Any unrelated files that are in the buildpack can be removed before creating the buildpack zip file, but they won’t have any impact if they are left.

Install the buildpack

Adding the buildpack to Stackato or Helion is very straight forward. Use the stackato or helion client to call create-buildpack and provide the name and zip file, as shown.

C:\Users\watrous\Documents\buildpack>stackato create-buildpack daniel-static-nginx buildpack-nginx.zip
Creating new buildpack daniel-static-nginx ... OK
Uploading buildpack bits ...  OK
OK

All installed buildpacks can be listed using the stackato client with the buildpacks command.

C:\Users\watrous\Documents\buildpack>stackato buildpacks
+---+---------------------+---------------------+---------+--------+
| # | Name                | Filename            | Enabled | Locked |
+---+---------------------+---------------------+---------+--------+
| 1 | java                | java.zip            | yes     | no     |
| 2 | ruby                | ruby.zip            | yes     | no     |
| 3 | nodejs              | nodejs.zip          | yes     | no     |
| 4 | python              | python.zip          | yes     | no     |
| 5 | go                  | go.zip              | yes     | no     |
| 6 | clojure             | clojure.zip         | yes     | no     |
| 7 | scala               | scala.zip           | yes     | no     |
| 8 | perl                | perl.zip            | yes     | no     |
| 9 | daniel-static-nginx | buildpack-nginx.zip | yes     | no     |
+---+---------------------+---------------------+---------+--------+

Use the buildpack

In the case of this buildpack, the detect script is looking for an index file with and extension of either html or htm. To deploy using this buildpack, just make sure that a file matching that description is in the root of your application files. There is no need to include a manifest.

Software Engineering

nginx buildpack – pre-compiled

Previously I detailed the process to create a buildpack for Stackato or Helion, including reconfiguring the buildpack to be self-contained. In both previous examples, the compile script performed a configure and make on source to build the binaries for the application. Since the configure->make process is often slow, this can be done once and the binaries added to the git repository for the buildpack.

Pre-compile nginx

The first step is to build nginx to run in the docker container for Stackato or Helion. These steps were previously in the compile script, but now they need to be run independently. After buliding, package up the necessary files and include that file in the git repository in place of the source file. Here are the commands I ran to accomplish this.

# create a docker image for the build process
docker run -i -t stackato/stack-alsek:latest /bin/bash
# download, configure and compile
wget -e use_proxy=yes http://nginx.org/download/nginx-1.6.2.tar.gz
tar xzf nginx-1.6.2.tar.gz
cd nginx-1.6.2
./configure
make
# rearrange and package up the files
cd
mkdir -p nginx/bin
mkdir -p nginx/conf
mkdir -p nginx/logs
cp nginx-1.6.2/objs/nginx nginx/bin/
cp nginx-1.6.2/conf/nginx.conf  nginx/conf/
cp nginx-1.6.2/conf/mime.types  nginx/conf/
tar czvf nginx.tar.gz nginx
# copy packaged file where I can get it to add to git repository
scp nginx.tar.gz user@myhost.domain.com:~/

After you get this new packaged file in place, the structure would look like this.

buildpack-files-precompiled

Updated compile script

The compile script is quite different. Here’s what I ended up with.

#!/usr/bin/env bash
# bin/compile <build-dir> <cache-dir>
 
shopt -s dotglob    # enables commands like 'mv *' to see hidden files
set -e              # exit immediately if any command fails (non-zero status)
 
# create local variables pointing to key paths
app_files_dir=$1
cache_dir=$2
buildpack_dir=$(cd $(dirname $0) && cd .. && pwd)
 
# unpackage nginx
mkdir -p $cache_dir
cp $buildpack_dir/vendor/nginx.tar.gz $cache_dir
tar xzf $cache_dir/nginx.tar.gz -C $cache_dir
 
# move applicaiton files into public directory
mkdir -p $cache_dir/public
mv $app_files_dir/* $cache_dir/public/
 
# put everything in place for droplet creation
mv $buildpack_dir/bin/launch.sh $app_files_dir/
mv $cache_dir/public $app_files_dir/
mv $cache_dir/nginx $app_files_dir/
 
# ensure manifest not in public directory
if [ -f $cache_dir/public/manifest.yml ]; then rm $cache_dir/public/manifest.yml; fi
if [ -f $cache_dir/public/stackato.yml ]; then rm $cache_dir/public/stackato.yml; fi

Now rather than build nginx it is just unpackaged right into place.

Staging Performance

Pre-compiling can save a lot of time when staging. For example, in this case the following times were observed when staging under each scenario.

Time to stage build from source

[stackato[dea_ng]] 2014-10-22T15:42:34.000Z: Completed uploading droplet
[staging]          2014-10-22T15:41:16.000Z:

That’s a total of 1:18 to build from source as part of the staging process.

Time to stage pre-compiled

[stackato[dea_ng]] 2014-10-22T16:33:38.000Z: Completed uploading droplet
[staging]          2014-10-22T16:33:32.000Z:

The total time to stage is 0:06 when the binary resources are precompiled. That’s 92.3% faster staging. That can be a significant advantage when staging applications which employ a very heavy compile process to establish the runtime environment.

Considerations

It could be argued that a pre-compiled binary is more likely to have a conflict at deploy time. In many deployment scenarios I would be tempted to agree. However, since the deployment environment is a Docker container and will presumably be identical to the container used to compile the binary, this risk is relatively low.

One question is whether a git repository is an appropriate place to store binary files. If this is a concern, the binary files can be stored on a file server, CDN or any location which can be accessed by the staging Docker container. This will keep the git repository clean with text only and can also provide a file system collection of binary artifacts that are available independent of git.

Software Engineering

nginx buildpack – offline

I previously documented the process to create a buildpack for nginx to use with Stackato or Helion Dev Platform. In that buildpack example, the compile script would download the nginx source using wget. In some cases, the time, bandwidth or access required to download external resources may be undesirable. In those cases the buildpack can be adjusted to work offline by adding the external resources to the git repository. The new structure would look like this.

buildpack-files-offline

Updated compile script

The only bulidpack related file that would need to change to accommodate this is the compile script. The affected section is download and build nginx.

# download and build nginx
mkdir -p $cache_dir
cp $buildpack_dir/vendor/nginx-1.6.2.tar.gz $cache_dir
tar xzf $cache_dir/nginx-1.6.2.tar.gz -C $cache_dir
cd $cache_dir/nginx-1.6.2
./configure
make

Notice that now the source is copied from the vendor directory in the buildpack instead downloaded using wget.

Using buildpack branches

In this case I created a branch of my existing buildpack to make these changes. By adding #branchname, it was possible to identify that branch with a in the manifest.yml. In this case my branch name was offline.

---
applications:
- name: static-test
  buildpack: https://github.com/dwatrous/buildpack-nginx.git#offline
Software Engineering

nginx buildpack – realtime

CloudFoundry accommodates buildpacks which define a deployment environment. A buildpack is distinct from an application and provides everything the application needs to run, including web server, language runtime, libraries, etc. The most basic structure for a buildpack requires three files inside a directory named bin.

buildpack-files

The buildpack files discussed in this post can be cloned or forked at https://github.com/dwatrous/buildpack-nginx

Some quick points about these buildpack files

  • All three files must be executable via bash
  • Can be shell scripts or any language that can be invoked using bash
  • Explicit use of paths recommended

detect

The detect script is intended to examine application files to determine whether it can accommodate the application. If it finds evidence that it can provide an environment to run an application, it should echo a message and exit with a status of zero. Otherwise it should exit with a non-zero status. In this example, the detect script looks for an index file with the extension html or htm.

#!/usr/bin/env bash
 
if [[ ( -f $1/index.html || -f $1/index.htm ) ]]
then
  echo "Static" && exit 0
else
  exit 1
fi

compile

The compile script is responsible to gather, build and position everything exactly as it needs to be in order to create the droplet that will be used to deploy the application. The second line of the script below shows that cloudfoundry calls the compile script and passes in paths to build-dir and cache-dir.

Develop and test the compile script

All of the commands in this file will run on a docker instance created specifically to perform the staging operation. It’s possible to develop the compile script inside the same docker container that will be used to stage your applicationi.

#!/usr/bin/env bash
# bin/compile <build-dir> <cache-dir>
 
shopt -s dotglob    # enables commands like 'mv *' to see hidden files
set -e              # exit immediately if any command fails (non-zero status)
 
# create local variables pointing to key paths
app_files_dir=$1
cache_dir=$2
buildpack_dir=$(cd $(dirname $0) && cd .. && pwd)
 
# download and build nginx
mkdir -p $cache_dir
cd $cache_dir
wget http://nginx.org/download/nginx-1.6.2.tar.gz
tar xzf nginx-1.6.2.tar.gz
cd nginx-1.6.2
./configure
make
 
# create hierarchy with only needed files
mkdir -p $cache_dir/nginx/bin
mkdir -p $cache_dir/nginx/conf
mkdir -p $cache_dir/nginx/logs
cp $cache_dir/nginx-1.6.2/objs/nginx $cache_dir/nginx/bin/nginx
cp $cache_dir/nginx-1.6.2/conf/nginx.conf $cache_dir/nginx/conf/nginx.conf
cp $cache_dir/nginx-1.6.2/conf/mime.types $cache_dir/nginx/conf/mime.types
 
# move applicaiton files into public directory
mkdir -p $cache_dir/public
mv $app_files_dir/* $cache_dir/public/
# copy nginx error template
cp $cache_dir/nginx-1.6.2/html/50x.html $cache_dir/public/50x.html
 
# put everything in place for droplet creation
mv $buildpack_dir/bin/launch.sh $app_files_dir/
mv $cache_dir/public $app_files_dir/
mv $cache_dir/nginx $app_files_dir/
 
# ensure manifest not in public directory
if [ -f $cache_dir/public/manifest.yml ]; then rm $cache_dir/public/manifest.yml; fi
if [ -f $cache_dir/public/stackato.yml ]; then rm $cache_dir/public/stackato.yml; fi

Notice that after nginx has been compiled, the desired files, such as the nginx binary and configuration file, must be explicitly copied into place where they will be included when the droplet is packaged up. In this case I put the application files in a sub-directory called public and the nginx binary, conf and logs in a sub-directory called nginx. A shell script, launch.sh, is also copied into the root of the application directory, which will be explained in a minute.

release

The output of the release script is a YAML file, but it’s important to understand that the release file itself must be an executable script. The key detail in the release file is the line that indicates how to start the web server process. In some cases that may require several commands, which would require them to be encapsulated in another script, such as the launch.sh script shown below.

#!/usr/bin/env bash
 
cat <<YAML
---
default_process_types:
  web: sh launch.sh
YAML

launch.sh

The launch.sh script creates a configuration file for nginx that includes the PORT and HOME directory for this specific docker instance. It then starts nginx as the local user.

#!/usr/bin/env bash
 
# create nginx conf file with PORT and HOME directory from cloudfoundry environment variables
mv $HOME/nginx/conf/nginx.conf $HOME/nginx/conf/nginx.conf.original
sed "s|\(^\s*listen\s*\)80|\1$PORT|" $HOME/nginx/conf/nginx.conf.original > $HOME/nginx/conf/nginx.conf
sed -i "s|\(^\s*root\s*\)html|\1$HOME/public|" $HOME/nginx/conf/nginx.conf
 
# start nginx web server
$HOME/nginx/bin/nginx -c $HOME/nginx/conf/nginx.conf -p $HOME/nginx

Using a buildpack

There are two ways to use a buildpack: either install it on the cloud controller node and let the detect script select it or store it in a GIT repository and provide the GIT URL in manifest.yml. The GIT repository must be publicly readable and accessible from the docker instance where staging will occur (proxy issues may interfere with this).

To use this buildpack, in an empty directory, create two files:

manifest.yml

---
applications:
- name: static-test
  memory: 40M
  disk: 200M
  instances: 1
  buildpack: https://github.com/dwatrous/buildpack-nginx.git

index.html

<h1>Hello World!</h1>

From within that directory, target, login and push your app (can be done locally or in the cloud).

helion-static-nginx-app-deployed

Resources

https://developer.ibm.com/answers/questions/15623/buildpack-compilation-step-failed/
http://blog.lesc.se/2011/11/how-to-change-file-premissions-in-git.html

Software Engineering

Nginx in Docker for Stackato buildpacks

I’m about to write a few articles about creating buildpacks for Stackato, which is a derivative of CloudFoundry and the technology behind Helion Development Platform. The approach for deploying nginx in docker as part of a buildpack differs from the approach I published previously. There are a few reasons for this:

  • Stackato discourages root access to the docker image
  • All services will run as the stackato user
  • The PORT to which services must bind is assigned and in the free range (no root required)

Get a host setup to run docker

The easiest way to follow along with this tutorial is to deploy stackato somewhere like hpcloud.com. The resulting host will have the docker image you need to follow along below.

Manually prepare a VM to run docker containers

You can also use Vagrant to spin up a virtual host where you can run these commands.

vagrant init ubuntu/trusty64

Modify the Vagrantfile to contain this line

  config.vm.provision :shell, path: "bootstrap.sh"

Then create a the bootstrap.sh file based on the details below.

bootstrap.sh

#!/usr/bin/env bash
 
# set proxy variables
#export http_proxy=http://proxy.example.com:8080
#export https_proxy=https://proxy.example.com:8080
 
# bootstrap ansible for convenience on the control box
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
apt-get update
apt-get -y install lxc-docker
 
# need to add proxy specifically to docker config since it doesn't pick them up from the environment
#sed -i '$a export http_proxy=http://proxy.example.com:8080' /etc/default/docker
#sed -i '$a export https_proxy=https://proxy.example.com:8080' /etc/default/docker
 
# enable non-root use by vagrant user
groupadd docker
gpasswd -a vagrant docker
 
# restart to enable proxy
service docker restart
# give docker a few seconds to start up
sleep 2s
 
# load in the stackato/stack-alsek image (can take a while)
docker load < /vagrant/stack-alsek.tar

Get the stackato docker image

Before the last line in the above bootstrap.sh script will work, it’s necessary to place the docker image for Stackato in the same directory as the Vagrantfile. Unfortunately the Stackato docker image is not published independently, which makes it more complicated to get. One way to do this is to deploy stackato locally and grab a copy of the image with this command.

docker save stackato/stack-alsek > stack-alsek.tar

You might want to save a copy of stack-alsek.tar to save time in the future. I’m not sure if it can be published (legally), but you’ll want to update this image with each release of Stackato anyway.

Launch a new docker instance using the Stackato image

You should now have three files in the directory where you did ‘vagrant init’.

-rw-rw-rw-   1 user     group          4998 Oct  9 14:01 Vagrantfile
-rw-rw-rw-   1 user     group           971 Oct  9 14:23 bootstrap.sh
-rw-rw-rw-   1 user     group    1757431808 Oct  9 14:02 stack-alsek.tar

At this point you should be ready to create a new VM and spin up a docker instance. First tell Vagrant to build the virtual server.

vagrant up

Next, log in to your server and create the docker container.

vagrant@vagrant-ubuntu-trusty-64:~$ docker run -i -t stackato/stack-alsek:latest /bin/bash
root@33ad737d42cf:/#

Build and configure your services

Once you have a system setup and can create docker instances based on the Stackato image, you’re ready to craft your buildpack compile script. One of the first things I do is install the w3m browser so I can test my setup. In this example, I’m just going to build and test nginx. The same process could be used to build any number of steps into a compile script. It may be necessary to manage http_proxy, https_proxy and no_proxy environment variables for both root and stackato users while completing the steps below.

apt-get -y install w3m

Since everything in a stackato DEA is expected to run as the stackato user, we’ll switch to that user and move into the home directory

root@33ad737d42cf:/# su stackato
stackato@33ad737d42cf:/$ cd
stackato@33ad737d42cf:~$ pwd
/home/stackato/

Next I’m going to grab the source for nginx and configure and make.

wget -e use_proxy=yes http://nginx.org/download/nginx-1.6.2.tar.gz
tar xzf nginx-1.6.2.tar.gz
cd nginx-1.6.2
./configure
make

By this point nginx has been built successfully and we’re in the nginx source directory. Next I want to update the configuration file to use a non-privileged port. For now I’ll use 8080, but Stackato will assign the actual PORT when deployed.

mv conf/nginx.conf conf/nginx.conf.original
sed 's/\(^\s*listen\s*\)80/\1 8080/' conf/nginx.conf.original > conf/nginx.conf

I also need to make sure that there is a logs directory available for nginx error logs on startup.

mkdir logs

It’s time to test the nginx build, which we can do with the command below. A message should be displayed saying the test was successful.

stackato@33ad737d42cf:~/nginx-1.6.2$ ./objs/nginx -t -c conf/nginx.conf -p /home/stackato/nginx-1.6.2
nginx: the configuration file /home/stackato/nginx-1.6.2/conf/nginx.conf syntax is ok
nginx: configuration file /home/stackato/nginx-1.6.2/conf/nginx.conf test is successful

With the setup and configuration validated, it’s time to start nginx and verify that it works.

./objs/nginx -c conf/nginx.conf -p /home/stackato/nginx-1.6.2

At this point it should be possible to load the nginx welcome page using the command below.

w3m http://localhost:8080

nginx-docker

Next steps

If an application requires other resources, this same process can be followed to build, integrate and test them within a running docker container based on the stackato image.

Software Engineering

Use Docker to Build a LEMP Stack (Buildfile)

I’ve been reviewing Docker recently. As part of that review, I decided to build a LEMP stack in Docker. I use Vagrant to create an environment in which to run Docker. For this experiment I chose to create Buildfiles to create the Docker container images. I’ll be discussing the following files in this post.

Vagrantfile
bootstrap.sh
mysql/Dockerfile
mysql/mysqlpwdseed
nginx/Dockerfile
nginx/default
nginx/wall.php

Download the Docker LEMP files as a zip (docker-lemp.zip).

Spin up the Host System

I start with Vagrant to spin up a host system for my Docker containers. To do this I use the following files.

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :
 
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
 
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.network "public_network"
  config.vm.provider "virtualbox" do |v|
    v.name = "docker LEMP"
    v.cpus = 1
    v.memory = 512
  end
  config.vm.provision :shell, path: "bootstrap.sh"
 
end

I start with Ubuntu and keep the size small. I create a public network to make it easier to test my LEMP setup later on.

bootstrap.sh

#!/usr/bin/env bash
 
# set proxy variables
#export http_proxy=http://proxy.example.com:8080
#export https_proxy=https://proxy.example.com:8080
 
# bootstrap ansible for convenience on the control box
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
apt-get update
apt-get -y install lxc-docker
 
# need to add proxy specifically to docker config since it doesn't pick them up from the environment
#sed -i '$a export http_proxy=http://proxy.example.com:8080' /etc/default/docker
#sed -i '$a export https_proxy=https://proxy.example.com:8080' /etc/default/docker
 
# enable non-root use by vagrant user
groupadd docker
gpasswd -a vagrant docker
 
# restart to enable proxy
service docker restart

I’m working in a proxied environment, so I need to provide proxy details to the Vagrant host for subsequent Docker installation steps. Unfortunately Docker doesn’t key off the typical environment variables for proxy, so I have to define them explicitly in the docker configuration. This second proxy configuration allows Docker to download images from DockerHub. Finally I create a docker group and add the vagrant user to it so I don’t need sudo for docker commands.

At this point, it’s super easy to play around with Docker and you might enjoy going through the Docker User Guide. We’re ready to move on to create Docker container images for our LEMP stack.

Build the LEMP Stack

This LEMP stack will be split across two containers, one for MySQL and the other for Nginx+PHP. I build on Ubuntu as a base, which may increase the total size of the image in exchange for the ease of using Ubuntu.

MySQL Container

We’ll start with MySQL. Here’s the Dockerfile.

Dockerfile

# LEMP stack as a docker container
FROM ubuntu:14.04
MAINTAINER Daniel Watrous <email>
#ENV http_proxy http://proxy.example.com:8080
#ENV https_proxy https://proxy.example.com:8080
 
RUN apt-get update
RUN apt-get -y upgrade
# seed database password
COPY mysqlpwdseed /root/mysqlpwdseed
RUN debconf-set-selections /root/mysqlpwdseed
 
RUN apt-get -y install mysql-server
 
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
 
RUN /usr/sbin/mysqld & \
    sleep 10s &&\
    echo "GRANT ALL ON *.* TO admin@'%' IDENTIFIED BY 'secret' WITH GRANT OPTION; FLUSH PRIVILEGES" | mysql -u root --password=secret &&\
    echo "create database test" | mysql -u root --password=secret
 
# persistence: http://txt.fliglio.com/2013/11/creating-a-mysql-docker-container/ 
 
EXPOSE 3306
 
CMD ["/usr/bin/mysqld_safe"]

Notice that I declare my proxy servers for the third time here. This one is so that the container can access package downloads. I then seed the root database passwords and proceed to install and configure MySQL. Before running CMD, I expose port 3306. Remember that this port will be exposed on the private network between Docker containers and is only accessible to linked containers when you start it the way I show below. Here’s the mysqlpwdseed file.

mysqlpwdseed

mysql-server mysql-server/root_password password secret
mysql-server mysql-server/root_password_again password secret

If you downloaded the zip file above and ran vagrant in the resulting directory, you should have the files above available in /vagrant/mysql. The following commands will build and start the MySQL container.

cd /vagrant/mysql
docker build -t "local/mysql:v1" .
docker images
docker run -d --name mysql local/mysql:v1
docker ps

At this point you should show a local image for MySQL and a running mysql container, as shown below.

docker-mysql-container

Nginx Container

Now we’ll build the Nginx container with PHP baked in.

Dockerfile

# LEMP stack as a docker container
FROM ubuntu:14.04
MAINTAINER Daniel Watrous <email>
ENV http_proxy http://proxy.example.com:8080
ENV https_proxy https://proxy.example.com:8080
 
# install nginx
RUN apt-get update
RUN apt-get -y upgrade
RUN apt-get -y install nginx
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
RUN mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak
COPY default /etc/nginx/sites-available/default
 
# install PHP
RUN apt-get -y install php5-fpm php5-mysql
RUN sed -i s/\;cgi\.fix_pathinfo\s*\=\s*1/cgi.fix_pathinfo\=0/ /etc/php5/fpm/php.ini
 
# prepare php test scripts
RUN echo "<?php phpinfo(); ?>" > /usr/share/nginx/html/info.php
ADD wall.php /usr/share/nginx/html/wall.php
 
# add volumes for debug and file manipulation
VOLUME ["/var/log/", "/usr/share/nginx/html/"]
 
EXPOSE 80
 
CMD service php5-fpm start && nginx

After setting the proxy, I install Nginx and update the configuration file. I then install PHP5 and update the php.ini file. I then use two different methods to create php files. If you’re deploying an actual application to a Docker container, you may not end up using either of these, but instead install a script that will grab your application from git or subversion.

Next I define two volumes. You’ll see shortly that this makes it straightforward to view logs and manage code for debug. Finally I expose port 80 for web traffic and the CMD references two commands using && to make sure both PHP and Nginx are running in the container.

default

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;
 
        root /usr/share/nginx/html;
        index index.php index.html index.htm;
 
        server_name localhost;
 
        location / {
                try_files $uri $uri/ =404;
        }
 
        error_page 404 /404.html;
 
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
                root /usr/share/nginx/html;
        }
 
        location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
        }
}

This is my default Nginx configuration file.

wall.php

<?php
 
// database credentials (defined in group_vars/all)
$dbname = "test";
$dbuser = "admin";
$dbpass = "secret";
$dbhost = "mysql";
 
// query templates
$create_table = "CREATE TABLE IF NOT EXISTS `wall` (
   `id` int(11) unsigned NOT NULL auto_increment,
   `title` varchar(255) NOT NULL default '',
   `content` text NOT NULL default '',
   PRIMARY KEY  (`id`)
   ) ENGINE=MyISAM  DEFAULT CHARSET=utf8";
$select_wall = 'SELECT * FROM wall';
 
// Connect to and select database
$link = mysql_connect($dbhost, $dbuser, $dbpass)
    or die('Could not connect: ' . mysql_error());
echo "Connected successfully\n<br />\n";
mysql_select_db($dbname) or die('Could not select database');
 
// create table
$result = mysql_query($create_table) or die('Create Table failed: ' . mysql_error());
 
// handle new wall posts
if (isset($_POST["title"])) {
    $result = mysql_query("insert into wall (title, content) values ('".$_POST["title"]."', '".$_POST["content"]."')") or die('Create Table failed: ' . mysql_error());
}
 
// Performing SQL query
$result = mysql_query($select_wall) or die('Query failed: ' . mysql_error());
 
// Printing results in HTML
echo "<table>\n";
while ($line = mysql_fetch_array($result, MYSQL_ASSOC)) {
    echo "\t<tr>\n";
    foreach ($line as $col_value) {
        echo "\t\t<td>$col_value</td>\n";
    }
    echo "\t</tr>\n";
}
echo "</table>\n";
 
// Free resultset
mysql_free_result($result);
 
// Closing connection
mysql_close($link);
?>
 
<form method="post">
Title: <input type="text" name="title"><br />
Message: <textarea name="content"></textarea><br />
<input type="submit" value="Post to wall">
</form>

Wall is meant to be a simple example that shows both PHP and MySQL are working. The commands below will build this new image and start it as a linked container to the already running MySQL container.

cd /vagrant/nginx
docker build -t "local/nginx:v1" .
docker images
docker run -d -p 80:80 --link mysql:mysql --name nginx local/nginx:v1
docker ps

So that now your Docker environment looks like the image below with an image for MySQL and one for Nginx and two processes running that are linked together.

docker-nginx-container

As shown in the image above, we have mapped port 80 on the container to port 80 on the host system. This means we can discover the IP address of our host (remember the public network in the vagrant file) and load the web page.

docker-php-wall-browser

Working with Running Containers

True to the single responsibility container design of Docker, these running containers are only running their respective service(s). That means they aren’t running SSH (and they shouldn’t be). So how do we interact with them, such as viewing the log files on the Nginx server or connecting to MySQL? We use Linked containers that leverage the shared volumes or exposed ports.

Connect to MySQL

To connect to MySQL, create a new container from the same MySQL image and link it to the already running MySQL. Start that container with /bin/bash. You can then use the mysql binaries to connect. Notice that I identify the host as ‘mysql’. That’s because when I linked the containers, Docker added an alias in the /etc/hosts file of the new container that mapped the private Docker network IP to ‘mysql’.

docker run -i -t --link mysql:mysql local/mysql:v1 /bin/bash
mysql -u admin -h mysql test --password=secret

docker-mysql-linked

View Ngnix Logs

It’s just as easy to view the Nginx logs. In this case I start up a plain Ubuntu container using the –volumes-from and indicate the running nginx container. From bash I can easily navigate to the Nginx logs or the html directory.

docker run -i -t --volumes-from nginx ubuntu /bin/bash

docker-nginx-view-logs

References

https://docs.docker.com/userguide/dockerlinks/

Software Engineering

Build a Multi-server LEMP stack using Ansible

My objective in this post is to explore the use of Ansible to configure a multi-server LEMP stack. This builds on the preliminary work I did demonstrating how to use Vagrant to create an environment to run Ansible. You can follow this entire example on any Windows (or Linux) host.

Ansible only runs on Linux hosts, not Windows. As a result, I needed to provision one Linux host to act as Ansible controller. One aspect of Ansible that I wanted to explore is the ability to manage multiple hosts with different configurations. For this experiment, I provision two more Linux hosts, one to act as a database host and the other to function as an Nginx/PHP server for a complete LEMP stack. I created the diagram below to illustrate my setup.

vagrant-ansible-lemp

There are two primary artifact categories for this experiement:

  • Vagrantfile to provision each host
  • Ansible playbook related files

Since there were more than a few Ansible playbook files, I chose to create a github repository rather than provide all the code here. You can clone/fork the files to run this experiment here:

https://github.com/dwatrous/vagrant-ansible-lemp

Explanation

Here is a list of the files you’ll find in that repository.

  • Vagrantfile
  • control.sh
  • lemp/group_vars/all
  • lemp/hosts
  • lemp/roles/common/handlers/main.yml
  • lemp/roles/common/tasks/main.yml
  • lemp/roles/database/handlers/main.yml
  • lemp/roles/database/tasks/main.yml
  • lemp/roles/web/handlers/main.yml
  • lemp/roles/web/tasks/main.yml
  • lemp/roles/web/templates/default
  • lemp/roles/web/templates/wall.php
  • lemp/site.yml

I do use a bootstrap shell script, control.sh, with Vagrant for the Ansible control server. It is necessary to install Ansible on the control server, but since Ansible doesn’t require an agent, there’s no need to bootstrap the other servers.

Playbook files

For each Ansible defined role there are three artifact categories.

  • handlers
  • tasks
  • templates

Handlers are named tasks that can be called or notified when Ansible detects other events. These are commonly used to trigger service restarts when configuration files change, as an example.

Tasks are the meat of the playbook. This lists out the steps to put a system into a desired state, including installing software, copying templates, registering and calling handlers, etc.

Configuration files, such as the nginx ‘default’ configuration in this case, can be stored in the templates folder and copied to the host using a task. Templates are helpful when a desired configuration differs significantly from a system default, this can be easier than updating individual lines in a file one at a time using lineinfile. The Ansible playbook files are in the following directory.

/vagrant/lemp

The site.yml file ties it all together by associating host groups with roles. You run the playbook like this.

ansible-playbook -i hosts site.yml

The example wall.php script should be accessible locally using the port 80->8080 mapping as http://127.0.0.1:8080/wall.php or over port 80 on the external IP assigned to the web host. Here’s what you can expect to see.

ansible-wall-example

Resources

I used the ansible examples repository on Github while putting this together. You may find it useful. For the specifics of installing LEMP on Ubuntu, I followed my Vagrant tutorial.

Software Engineering

Using Vagrant to build a LEMP stack

I may have just fallen in love with the tool Vagrant. Vagrant makes it possible to quickly create a virtual environment for development. It is different than cloning or snapshots in that it uses minimal base OSes and provides a provisioning mechanism to setup and configure the environment exactly the way you want for development. I love this for a few reasons:

  • All developers work in the exact same environment
  • Developers can get a new environment up in minutes
  • Developers don’t need to be experts at setting up the environment.
  • System details can be versioned and stored alongside code

This short tutorial below demonstrates how easy it is to build a LEMP stack using Vagrant.

Install VirtualBox

Vagrant is not a virtualization tool. Instead vagrant will leverage an existing provider of virtual compute resources, either local or remote. For example, Vagrant can be used to create a virtual environment on Amazon Web Services or locally using a tool like VirtualBox. For this tutorial, we’ll use VirtualBox. You can download and install VirtualBox from the official website.

https://www.virtualbox.org/

Install Vagrant

Next, we install Vagrant. Downloads are freely available on their website.

http://www.vagrantup.com/

For the remainder of this tutorial, I’m going to assume that you’ve been through the getting started training and are somewhat familiar with Vagrant.

Accommodate SSH Keys

UPDATE 6/26/2015: Vagrant introduced the unfortunate feature of producing a random key for each new VM as the default behavior. It’s possible to restore the original functionality (described below) and use the insecure key with the config.ssh.insert_key = false setting in a Vagrantfile.

Until (if ever) Vagrant defaults to using the insecure key, a system wide work around is to add a Vagrantfile to the local .vagrant.d folder which will add set this setting for all VMs (see Load Order and Merging), unless otherwise overridden. The Vagrant file can be as simple as this:

# -*- mode: ruby -*-
# vi: set ft=ruby :
 
VAGRANTFILE_API_VERSION = "2"
 
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.ssh.insert_key = false
 
end

Vagrant creates an SSH key which it installs on guest hosts by default. This can be a huge time saver since it prevents the need for passwords. Since I use PuTTY on windows, I needed to convert the SSH key and save a PuTTY session to accommodate connections. Use PuTTYgen to do this.

  1. Open PuTTYgen
  2. Click “Load”
  3. Navigate to the file C:\Users\watrous\.vagrant.d\insecure_private_key

PuTTYgen shows a dialog saying that the import was successful and displays the details of the key, as shown here:

import-vagrant-ssh-key-puttygen

Click “Save private key”. You will be prompted about saving the key without a passphrase, which in this case is fine, since it’s just for local development. If you end up using Vagrant to create public instances, such as using Amazon Web Services, you should use a more secure connection method. Give the key a unique name, like C:\Users\watrous\.vagrant.d\insecure_private_key-putty.ppk and save.

Finally, create a saved PuTTY session to connect to new Vagrant instances. Here are some of my PuTTY settings:

putty-session-vagrant-settings-1

putty-session-vagrant-settings-auth

The username may change if you choose a different base OS image from the vagrant cloud, but the settings shown above should work fine for this tutorial.

Get Ready to ‘vagrant up’

Create a directory where you can store the files Vagrant needs to spin up your environment. I’ll refer to this directory as VAGRANT_ENV.

To build a LEMP stack we need a few things. First is a Vagrantfile file where we identify the base OS, or box, ports, etc. This is a text file that follows Ruby language conventions. Create the file VAGRANT_ENV/Vagrantfile with the following contents:

# -*- mode: ruby -*-
# vi: set ft=ruby :
 
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
 
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # All Vagrant configuration is done here. The most common configuration
  # options are documented and commented below. For a complete reference,
  # please see the online documentation at vagrantup.com.
 
  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "ubuntu/trusty64"
  config.vm.provision :shell, path: "bootstrap.sh"
  config.vm.network :forwarded_port, host: 4567, guest: 80
  config.ssh.shell = "bash -c 'BASH_ENV=/etc/profile exec bash'"
end

This file chooses a 64 bit trusty version of Ubuntu, forwards port 4567 on the host machine to port 80 on the guest machine and identifies a bootstrap shell script, which I show next.

Create VAGRANT_ENV/bootstrap.sh with the following contents:

#!/usr/bin/env bash
 
#accommodate proxy environments
#export http_proxy=http://proxy.company.com:8080
#export https_proxy=https://proxy.company.com:8080
apt-get -y update
apt-get -y install nginx
debconf-set-selections <<< 'mysql-server mysql-server/root_password password secret'
debconf-set-selections <<< 'mysql-server mysql-server/root_password_again password secret'
apt-get -y install mysql-server
#mysql_install_db
#mysql_secure_installation
apt-get -y install php5-fpm php5-mysql
sed -i s/\;cgi\.fix_pathinfo\s*\=\s*1/cgi.fix_pathinfo\=0/ /etc/php5/fpm/php.ini
service php5-fpm restart
mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak
cp /vagrant/default /etc/nginx/sites-available/default
service nginx restart
echo "<?php phpinfo(); ?>" > /usr/share/nginx/html/info.php

This script executes a sequence of commands from the shell as root after provisioning the new server. This script must run without requiring user input. It also should accommodate any configuration changes and restarts necessary to get your environment ready to use.

More sophisticated tools like Ansible, Chef and Puppet can also be used.

you may have noticed that the above script expects a modified version of nginx’s default configuration. Create the file VAGRANT_ENV/default with the following contents:

server {
	listen 80 default_server;
	listen [::]:80 default_server ipv6only=on;
 
	root /usr/share/nginx/html;
	index index.php index.html index.htm;
 
	server_name localhost;
 
	location / {
		try_files $uri $uri/ =404;
	}
 
	error_page 404 /404.html;
 
	error_page 500 502 503 504 /50x.html;
	location = /50x.html {
		root /usr/share/nginx/html;
	}
 
	location ~ \.php$ {
		fastcgi_split_path_info ^(.+\.php)(/.+)$;
		fastcgi_pass unix:/var/run/php5-fpm.sock;
		fastcgi_index index.php;
		include fastcgi_params;
	}
}

vagrant up

Now it’s time to run ‘vagrant up‘. To do this, open a console window and navigate to your VAGRANT_ENV directory, then run ‘vagrant up’.

vagrant-up-console

If this is the first time you have run ‘vagrant up’, it may take a few minutes to download the ‘box’. Once it’s done, you should be ready to visit your PHP page rendered by nginx on a local virtual machine created and configured by Vagrant:

http://127.0.0.1:4567/info.php