Droplet Execution Agent

Page last updated: August 6, 2015

A Droplet Execution Agent (DEA) performs the following key functions:

Manage Warden containers: The DEA stages applications and runs applications in Warden containers.
Stage applications: When a new application or a new version of an application is pushed to Cloud Foundry, the Cloud Controller selects a DEA from the pool of available DEAs to stage the application. The DEA uses the appropriate buildpack to stage the application. The result of this process is a droplet.
Run droplets: A DEA manages the lifecycle of each application instance running in it, starting and stopping droplets upon request of the Cloud Controller. The DEA monitors the state of a started application instance, and periodically broadcasts application state messages over NATS for consumption by the HM9000.

Directory Server

When the DEA receives requests for directories and files, it redirects them to the Directory Server URL. The URL is signed by the DEA, and the Directory Server checks the validity of the URL with the DEA before serving it.

Configurable Directory Server behaviors are controlled by keys in the DEA configuration file, dea.yml, described below in DEA Configuration.

The Directory Server is written in Go and can be found in the go/ directory of the DEA source code repository. It is a replacement for the older directory server that was embedded in the DEA itself.

DEA Health Checks

A DEA periodically checks the health of the applications running in it.

If a URL is mapped to an application, the DEA attempts to connect to the port assigned to the application. If the application port is accepting connections, the DEA considers that application state to be “Running.” If there is no URL mapped to the application, the DEA checks the system process table for the application process PID. If the PID exists, the DEA considers that application state to be “Running.”

The DEA also checks for a AppState object for the application.

Usage

You can run the dea executable at the command line by passing the path to the DEA configuration file:

$ bin/dea config/dea.yml

DEA Configuration

DEA behavior is configured in the dea.yml file.

The following is a partial list of the keys that the DEA reads from the YAML file:

logging: A Steno configuration
nats_uri: A URI of the form nats://host:port that the DEA uses to connect to NATS
warden_socket: The path to a UNIX domain socket that the DEA uses to communicate to a Warden server.

A sample dea.yml file follows:

Note: See https://github.com/cloudfoundry/dea_ng/blob/master/lib/dea/config.rb for optional configuration keys.


# See src/lib/dea/config.rb for optional config values.

# Base directory for dea. Application directories, DEA temp files, etc. are all relative to this.
base_dir: /tmp/dea_ng

logging:
  level: debug

resources:
  memory_mb: 2048
  memory_overcommit_factor: 2
  disk_mb: 2048
  disk_overcommit_factor: 2

nats_uri: nats://localhost:4222/

pid_filename: /tmp/dea_ng.pid

warden_socket: /tmp/warden.sock

evacuation_delay_secs: 10

index: 0

staging:
  enabled: true
  platform_config:
    cache: /var/vcap/data/stager/package_cache/ruby
  environment:
    PATH: /usr/local/ruby/bin
    BUILDPACK_CACHE: /var/vcap/packages/buildpack_cache
  memory_limit_mb: 1024
  disk_limit_mb: 2048
  max_staging_duration: 900 # 15 minutes

dea_ruby: /usr/bin/ruby

# For Go-based directory server
directory_server:
  v1_port: 4385
  v2_port: 5678
  file_api_port: 1234
  streaming_timeout: 10
  logging:
    level: info

stacks:
  - lucid64

# Hook scripts for droplet start/stop
# hooks:
#   before_start: path/to/script
#   after_start: path/to/script
#   before_stop: path/to/script
#   after_stop: path/to/script

Running the DEA Standalone

When you contribute to the DEA, we recommend that you run it as a standalone component. Follow the instructions below for running the DEA as a standalone component on Vagrant 1.1x.

Refer to the Vagrant documentation for instructions on installing Vagrant.

$ git clone http://github.com/cloudfoundry/dea_ng
$ bundle install

# check that your version of vagrant is 1.1 or greater
$ vagrant --version

# create your test VM
$ rake test_vm

Failure of the rake test_vm step followed by an error similar to “undefined method ‘configure’ for Vagrant” or “found character that cannot start any token while scanning for the next token” can result from using an unsupported version of Vagrant. Ensure you have installed Vagrant version 1.1 or higher.

# initialize the test VM
$ vagrant up

# shell into the VM
$ vagrant ssh

# start warden
$ cd /warden/warden
$ bundle install
$ rvmsudo bundle exec rake warden:start[config/test_vm.yml] > /tmp/warden.log &

# start the DEA's dependencies
$ cd /vagrant
$ bundle install
$ git submodule update --init
$ foreman start > /tmp/foreman.log &

# run the dea tests
$ bundle exec rspec

Staging

See the How Applications Are Staged topic for a description of the application staging flow that the DEA uses.

A DEA publishes the following staging messages on NATS:

staging.advertise: Stagers broadcast their capacity/capability.
staging.locate: Stagers respond to any message on this subject with a staging.advertise message. CC uses this to bootstrap.
staging.UUID.start: Stagers respond to requests on this subject to stage applications.
staging: Stagers, in a queue group, respond to requests to stage an application. Note: This is an old protocol.

Logs

Steno handles the DEA logging. DEA logs to STDOUT by default. You can configure the DEA to log to a file, a syslog server, or both. The logging level specifies the verbosity of the logs, for example warn, info, or debug.