Custom Buildpacks

Page last updated: October 27, 2015

Buildpacks are a convenient way of packaging framework and/or runtime support for your application.

For example, by default Cloud Foundry does not support Haskell. Using a buildpack for Haskell allows you to add support for it at the deployment stage. When you push an application written using Haskell, the required buildpack is automatically installed on the Cloud Foundry Droplet Execution Agent (DEA) where the application will run.

Note: A common development practice for custom buildpacks is to fork existing buildpacks and sync subsequent patches from upstream. To merge upstream patches to your custom buildpack, use the approach that Github recommends for syncing a fork.

Custom Buildpacks

The structure of a buildpack is straightforward. A buildpack repository contains three main scripts, situated in a folder named bin.

bin/detect

The detect script is used to determine whether or not to apply the buildpack to an application. The script is called with one argument, the build directory for the application. It returns an exit code of 0 if the application can be supported by the buildpack. If the script does return 0, it should also print a framework name to STDOUT.

Shown below is an example detect script written in Ruby that checks for a Ruby application based on the existence of a Gemfile:


#!/usr/bin/env ruby

gemfile_path = File.join ARGV[0], "Gemfile"

if File.exist?(gemfile_path)
  puts "Ruby"
  exit 0
else
  exit 1
end

Note that cf CLI displays the returned STDOUT “framework name” when printing the details of an application. Buildpack authors are therefore encouraged to include additional details in this returned framework name, such as buildpack versioning information, and a detailed list of configured frameworks and their associated versions. The following is an example of the detailed information returned by the Java buildpack:

java-buildpack=v3.0-https://github.com/cloudfoundry/java-buildpack.git#3bd15e1 open-jdk-jre=1.8.0_45 spring-auto-reconfiguration=1.7.0_RELEASE tomcat-access-logging-support=2.4.0_RELEASE tomcat-instance=8.0.21 tomcat-lifecycle-support=2.4.0_RELEASE ...

bin/compile

The compile script builds the droplet that will be run by the DEA and will therefore contain all the components necessary to run the application.

The script is run with two arguments, the build directory for the application and the cache directory, which is a location the buildpack can use to store assets during the build process.

During execution of this script all output sent to STDOUT will be relayed via the cf CLI back to the end user. The generally accepted pattern for this is to break out this functionality in to a ‘language_pack’. A good example of this can be seen at https://github.com/cloudfoundry/heroku-buildpack-ruby/blob/master/lib/language_pack/ruby.rb

A simple compile script is shown below:


#!/usr/bin/env ruby

#sync output

$stdout.sync = true

build_path = ARGV[0]
cache_path = ARGV[1]

install_ruby

private

def install_ruby
  puts "Installing Ruby"

  # !!! build tasks go here !!!
  # download ruby to cache_path
  # install ruby
end

bin/release

The release script provides feedback metadata back to Cloud Foundry indicating how the application should be executed. The script is run with one argument, the build location of the application. The script must generate a YAML file in the following format:

  config_vars:
    name: value
  default_process_types:
    web: commandLine

Where config_vars is an optional set of environment variables that will be defined in the environment in which the application is executed. default_process_types indicates the type of application being run and the command line used to start it. At this time only web type of applications are supported.

The following example shows what a Rack application’s release script might return:


config_vars:
  RACK_ENV: production
default_process_types:
  web: bundle exec rackup config.ru -p $PORT

Packaging Custom Buildpacks

Partially or completely disconnected environments

Cloud Foundry buildpacks work with limited or no internet connectivity. A Cloud Foundry operator can enable the same support in your custom buildpack by using the Buildpack Packager application.

Using buildpack-packager

  1. Create a manifest.yml in your buildpack.
  2. Run the packager in cached mode: buildpack-packager cached

The packager will add (almost) everything in your buildpack directory into a zip file. It will exclude anything marked for exclusion in your manifest.

In cached mode, the packager will download and add dependencies as described in the manifest.

Option Flags

--force-download

By default, buildpack-packager stores the dependencies that it downloads while building a cached buildpack in a local cache at ~/.buildpack-packager. This is in order to avoid re-downloading them when repackaging similar buildpacks. Running buildpack-packager cached with the --force-download option will force the packager to download dependencies from the s3 host and ignore the local cache. When packaging an uncached buildpack, --force-download does nothing.

--use-custom-manifest

If you would like to include a different manifest file in your packaged buildpack, you may call buildpack-packager with the --use-custom-manifest PATH/TO/MANIFEST.YML option. buildpack-packager will generate a buildpack with the specified manifest. If you are building a cached buildpack, buildpack-packager will vendor dependencies from the specified manifest as well.

You can find more documentation at the buildpack-packager Github repo.

Uploading the buildpack

Once you have packaged your buildpack using buildpack-packager, you can upload the resulting .zip file to your instance of Cloud Foundry:

cf create-buildpack BUILDPACK_NAME BUILDPACK_ZIP_PATH POSITION

You can find more documentation at the buildpack admin guide.

Deploying With a Custom Buildpack

Once a custom buildpack has been created and pushed to a public git repository, the git URL can be passed via the cf CLI when pushing an application.

For example, for a buildpack that has been pushed to Github:

$ cf push my-new-app -b git://github.com/johndoe/my-buildpack.git

Alternatively, you can use a private git repository, with https and username/password authentication, as follows:

$ cf push my-new-app -b https://username:[email protected]/johndoe/my-buildpack.git

By default, Cloud Foundry uses the master branch of the buildpack’s git repository. You can specify a different branch using the git url as shown in the following example:

$ cf push my-new-app -b https://username:[email protected]/johndoe/my-buildpack.git#my-branch-name

The application will then be deployed to Cloud Foundry, and the buildpack will be cloned from the repository and applied to the application (provided that the detect script returns 0).