Deploying a Nozzle to the Loggregator Firehose
- Step 1: Download Your Cloud Foundry BOSH Manifest
- Step 2: Add UAA client
- Step 3: Redeploy Cloud Foundry
- Step 4: Clone example release
- Step 5: Prepare the nozzle manifest
- Step 6: Select the new manifest as the deployment manifest for the nozzle
- Step 7: Create a BOSH release for the nozzle
- Step 8: Upload the BOSH release
- Step 9: Deploy the nozzle
- Step 10: View the nozzle output
Page last updated: November 25, 2015
This topic will walk through the procedure for deploying a ‘nozzle’ application to the Cloud Foundry Loggregator Firehose. The Loggregator team has prepared an example nozzle application for use with this tutorial.
The procedure described here will deploy this example nozzle to the Firehose of a Cloud Foundry deployed locally, with BOSH Lite. This is generally the easiest point of entry into Cloud Foundry operations.
Prerequisites:
- BOSH CLI installed locally.
- Spiff installed locally and added to your shell’s load path. Spiff on GitHub.
- A BOSH Lite deployed locally using VirtualBox. See BOSH lite on GitHub.
- A working Cloud Foundry, including Loggregator, deployed with your local BOSH Lite. This will serve as our source of data. See Deploying Cloud Foundry using BOSH Lite, or use the
provion_cfscript included in the BOSH Lite release.
Note: Even using the automated provision_cf script, Deploying Cloud Foundry can take up to several hours, depending on your internet bandwith.
Step 1: Download Your Cloud Foundry BOSH Manifest
Run
bosh deploymentsto identify the name of the current BOSH deployment:$ bosh deployments +-----------------+--------------------------+-------------------------------------------------+ | Name | Release(s) | Stemcell(s) | +-----------------+--------------------------+-------------------------------------------------+ | cf-example | cf-mysql/10 | bosh-vsphere-esxi-ubuntu-trusty-go_agent/2690.3 | | | cf/183.2 | | +-----------------+--------------------------+-------------------------------------------------+
Run
bosh download manifest DEPLOYMENT-NAME LOCAL-SAVE-NAMEto download and save the current BOSH deployment manifest. You need this manifest to locate information about your databases. ReplaceDEPLOYMENT-NAMEwith the name of the current BOSH deployment. For this procedure, usecf.ymlas theLOCAL-SAVE-NAME.$ bosh download manifest cf-example cf.yml Deployment manifest saved to `cf.yml'
Step 2: Add UAA client
You must authorize the example nozzle as a UAA client for your CF deployment. To do this, add an entry for the example nozzle as client for uaa under properties key in your CF deployment manifest. You must enter the example nozzle object in the right location in the manifest, and with the correct indentation, as described below.
Deployment manifests have the .yml extension, and are therefore YAML files. Visit YAML on the web to learn about YAML syntax.
- Find the
propertieskey that is at 0 indentation (left aligned). - Under that
propertieskey, finduaaat the next level of indentation. - Under that
uaakey, find theclientskey at the next level of indentation. Enter properties for the
example-nozzleat the next level of indentation, exactly as below. The...in the figure below indicate that other properties may populate the manifest at each level in the hierarchy.properties: ... uaa: ... clients: ... example-nozzle: access-token-validity: 1209600 authorized-grant-types: authorization_code,client_credentials,refresh_token override: true secret: example-nozzle scope: openid,oauth.approvals,doppler.firehose authorities: oauth.login,doppler.firehoseSave the manifest file
cf.yml.
Step 3: Redeploy Cloud Foundry
Select
cf.ymlas the manifest for your deployment.$ bosh deployment cf.yml Deployment set to `/Users/example_user/workspace/bosh-lite/cf.yml'
Deploy your Cloud Foundry with BOSH.
$ bosh deploy Acting as user 'admin' on deployment 'cf-warden' on 'Bosh Lite Director' Getting deployment properties from director...
Detecting deployment changes \---------------------------- Releases No changes Compilation No changes Update No changes Resource pools No changes Disk pools No changes Networks No changes Jobs No changes Properties uaa clients example-nozzle + access-token-validity: 1209600 + authorized-grant-types: authorization_code,client_credentials,refresh_token + override: true + secret: example-nozzle + scope: openid,oauth.approvals,doppler.firehose + authorities: oauth.login,doppler.firehose Meta No changes Please review all changes carefully Deploying --------- Are you sure you want to deploy? (type 'yes' to continue):
Step 4: Clone example release
The Loggregator team has prepared an example nozzle application for use with this tutorial.
Run
git cloneto clone the main release repository from GitHub.$ git clone [email protected]:cloudfoundry-incubator/example-nozzle-release.git Cloning into 'example-nozzle-release'...
Run
git submodule update --init --recursiveto update all the included submodules.$ git submodule update --init --recursive Submodule 'src/github.com/cloudfoundry-incubator/example-nozzle' ([email protected]:cloudfoundry-incubator/example-nozzle.git) registered for path 'src/github.com/cloudfoundry-incubator/example-nozzle' Submodule 'src/github.com/cloudfoundry-incubator/uaago' ([email protected]:cloudfoundry-incubator/uaago.git) registered for path 'src/github.com/cloudfoundry-incubator/uaago' ... Cloning into 'src/github.com/cloudfoundry-incubator/example-nozzle'... ...
cdinto theexample-releasedirectory.$ cd example-nozzle-release
Step 5: Prepare the nozzle manifest
There are two options for preparing the nozzle deployment manifest:
(Quicker option) Our engineers have included a script to prepare the manifest in a single step. To use this option simply run the following in the terminal:
$ scripts/make_manifest_spiff_bosh_lite
(Recommended, in order to fully understand the deployment procedure) Complete the following steps:
Inside
example-nozzle-release, change into thetemplatesdirectory.$ cd templates
There are two yaml files inside:template.ymlis template for the manifest we will use to deploy the nozzle.bosh-lite-stub.ymlcontains the values that will populate the missing information in the manifest template to complete our deployment manifest.
Make a
tmpdirectory to house our compiled manifest.Use spiff to compile a deployment manifest from the template and stub, and save this manifest.
$ spiff merge templates/template.yml templates/bosh-lite-stub.yml > tmp/manifest_bosh_lite.yml
Get your BOSH director UUID by running:
$ bosh status --uuid
In the compiled nozzle deployment manifest, find the
director_uuidproperty and replace its value ofPLACEHOLDER-DIRECTOR-UUIDwith your BOSH director UUID.
compilation:
cloud_properties:
name: default
network: example-nozzle-net
reuse_compilation_vms: true
workers: 1
director_uuid: PLACEHOLDER-DIRECTOR-UUID
Step 6: Select the new manifest as the deployment manifest for the nozzle
$ bosh deployment tmp/manifest_bosh_lite.yml Deployment set to `/Users/example_user/workspace/example-nozzle-release/templates/tmp/manifest_bosh_lite.yml'
Step 7: Create a BOSH release for the nozzle
Run the following terminal command, using the name example-nozzle, which matches the UAA client created earlier in the CF deployment manifest:
$ bosh create release --name example-nozzle Syncing blobs... ...
Step 8: Upload the BOSH release
Run the following terminal command:
$ bosh upload release Acting as user 'admin' on 'Bosh Lite Director'
Copying packages ---------------- example-nozzle golang1.4
Copying jobs ------------ example-nozzle
Generated /var/folders/4n/qs1rjbmd1c5gfb78m3_06j6r0000gn/T/d20151009-71219-17a5m49/d20151009-71219-rts928/release.tgz Release size: 59.2M
Verifying release... ... Release info ------------ Name: nozzle-test Version: 0+dev.2
Packages - example-nozzle (b0944f95eb5a332e9be2adfb4db1bc88f9755894) - golang1.4 (b68dc9557ef296cb21e577c31ba97e2584a5154b)
Jobs - example-nozzle (112e01c6ee91e8b268a42239e58e8e18e0360f58)
License - none
Uploading release
Step 9: Deploy the nozzle
Run the following terminal command:
$ bosh deploy Acting as user 'admin' on deployment 'example-nozzle-lite' on 'Bosh Lite Director' Getting deployment properties from director... Unable to get properties list from director, trying without it... Cannot get current deployment information from director, possibly a new deployment Please review all changes carefully
Deploying --------- Are you sure you want to deploy? (type 'yes' to continue):
Step 10: View the nozzle output
Congratulations! You have successfully deployed a Loggregator firehose nozzle.
The example nozzle simply dumps all of the data coming from the firehose to its log files. To view this data, SSH into the example-nozzle VM and examine the logs.
Run
bosh sshto access the nozzle VM at the IP configured in the nozzle’s manifest template stub./templates/bosh-lite-stub.yml.bosh ssh example-nozzle --strict_host_key_checking no Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.19.0-25-generic x86_64) Documentation: https://help.ubuntu.com/ Last login: Wed Sep 23 21:29:50 2015 from 192.168.50.1
Use the
catcommand to output thestdoutlog file.$ cat /var/vcap/sys/log/example-nozzle/example-nozzle.stdout.log ===== Streaming Firehose (will only succeed if you have admin credentials) origin:"DopplerServer" eventType:ValueMetric timestamp:1443046217700750747 deployment:"cf-warden" job:"doppler_z1" index:"0" ip:"10.244.0.142" valueMetric: