nginx buildpack – pre-compiled
Previously I detailed the process to create a buildpack for Stackato or Helion, including reconfiguring the buildpack to be self-contained. In both previous examples, the compile script performed a configure and make on source to build the binaries for the application. Since the configure->make process is often slow, this can be done once and the binaries added to the git repository for the buildpack.
The first step is to build nginx to run in the docker container for Stackato or Helion. These steps were previously in the compile script, but now they need to be run independently. After buliding, package up the necessary files and include that file in the git repository in place of the source file. Here are the commands I ran to accomplish this.
# create a docker image for the build process docker run -i -t stackato/stack-alsek:latest /bin/bash # download, configure and compile wget -e use_proxy=yes http://nginx.org/download/nginx-1.6.2.tar.gz tar xzf nginx-1.6.2.tar.gz cd nginx-1.6.2 ./configure make # rearrange and package up the files cd mkdir -p nginx/bin mkdir -p nginx/conf mkdir -p nginx/logs cp nginx-1.6.2/objs/nginx nginx/bin/ cp nginx-1.6.2/conf/nginx.conf nginx/conf/ cp nginx-1.6.2/conf/mime.types nginx/conf/ tar czvf nginx.tar.gz nginx # copy packaged file where I can get it to add to git repository scp nginx.tar.gz email@example.com:~/
After you get this new packaged file in place, the structure would look like this.
Updated compile script
The compile script is quite different. Here’s what I ended up with.
#!/usr/bin/env bash # bin/compile <build-dir> <cache-dir> shopt -s dotglob # enables commands like 'mv *' to see hidden files set -e # exit immediately if any command fails (non-zero status) # create local variables pointing to key paths app_files_dir=$1 cache_dir=$2 buildpack_dir=$(cd $(dirname $0) && cd .. && pwd) # unpackage nginx mkdir -p $cache_dir cp $buildpack_dir/vendor/nginx.tar.gz $cache_dir tar xzf $cache_dir/nginx.tar.gz -C $cache_dir # move applicaiton files into public directory mkdir -p $cache_dir/public mv $app_files_dir/* $cache_dir/public/ # put everything in place for droplet creation mv $buildpack_dir/bin/launch.sh $app_files_dir/ mv $cache_dir/public $app_files_dir/ mv $cache_dir/nginx $app_files_dir/ # ensure manifest not in public directory if [ -f $cache_dir/public/manifest.yml ]; then rm $cache_dir/public/manifest.yml; fi if [ -f $cache_dir/public/stackato.yml ]; then rm $cache_dir/public/stackato.yml; fi
Now rather than build nginx it is just unpackaged right into place.
Pre-compiling can save a lot of time when staging. For example, in this case the following times were observed when staging under each scenario.
Time to stage build from source
[stackato[dea_ng]] 2014-10-22T15:42:34.000Z: Completed uploading droplet [staging] 2014-10-22T15:41:16.000Z:
That’s a total of 1:18 to build from source as part of the staging process.
Time to stage pre-compiled
[stackato[dea_ng]] 2014-10-22T16:33:38.000Z: Completed uploading droplet [staging] 2014-10-22T16:33:32.000Z:
The total time to stage is 0:06 when the binary resources are precompiled. That’s 92.3% faster staging. That can be a significant advantage when staging applications which employ a very heavy compile process to establish the runtime environment.
It could be argued that a pre-compiled binary is more likely to have a conflict at deploy time. In many deployment scenarios I would be tempted to agree. However, since the deployment environment is a Docker container and will presumably be identical to the container used to compile the binary, this risk is relatively low.
One question is whether a git repository is an appropriate place to store binary files. If this is a concern, the binary files can be stored on a file server, CDN or any location which can be accessed by the staging Docker container. This will keep the git repository clean with text only and can also provide a file system collection of binary artifacts that are available independent of git.