Using Log Management Services

Page last updated: December 11, 2015

This page assumes you are using cf CLI v6.

This topic describes how to drain logs from Cloud Foundry to a third party log management service.

Cloud Foundry aggregates logs for all instances of your applications as well as for requests made to your applications through internal components of Cloud Foundry. For example, when the Cloud Foundry Router forwards a request to an application, the Router records that event in the log stream for that app. Run the following command to access the log stream for an app in the terminal:

cf logs YOUR-APP-NAME

If you want to persist more than the limited amount of logging information that Cloud Foundry can buffer, drain these logs to a log management service.

For more information about the systems responsible for log aggregation and streaming in Cloud Foundry, see Application Logging in Cloud Foundry.

Using Services from the Cloud Foundry Marketplace

The Cloud Foundry marketplace offers several log management services. To use one of these services, create an instance of the service and bind it to your application. See the following topics for more information:

Note: Not all marketplace services support syslog drains. Some services implement an integration with Cloud Foundry that enables automated streaming of application syslogs. If you are interested in building Services for Cloud Foundry and making them available to end users, see the Custom Services documentation.

Using Other Services

If a compatible log management service is not available in your Cloud Foundry marketplace, use the following procedure to stream your logs to a service of your choice.

Note: Only use this procedure if the service that you want to use is not covered in the Service-Specific Instructions for Streaming Application Logs topic.

Step 1: Configure the Log Management Service

Complete the following steps to set up a communication channel between the log management service and your Cloud Foundry deployment:

  1. Obtain the external IP addresses that your Cloud Foundry administrator assigns to outbound traffic.

  2. Provide these IP addresses to the log management service. The specific steps to configure a third-party log management service depend on the service. See the Service-Specific Instructions for Streaming Application Logs topic for more information.

  3. Whitelist these IP addresses to ensure unrestricted log routing to your log management service.

  4. Record the syslog URL provided by the third-party service. Third-party services typically provide a syslog URL to use as an endpoint for incoming log data. You use this syslog URL in Step 2: Create a User-provided Service Instance.

    Cloud Foundry uses the syslog URL to route messages to the service. The syslog URL has a scheme of syslog, syslog-tls, or https, and can include a port number. For example:

    syslog://logs.example.com:1234

    Note: Cloud Foundry does not support using syslog-tls with self-signed certificates. If you are running your own syslog server and want to use syslog-tls, you must have an SSL certificate signed by a well-known certificate authority.

Step 2: Create a User-provided Service Instance

Create a user-provided service instance using the cf CLI create-user-provided-service command with the -l flag and the syslog URL that you obtained in Step 1: Configure the Log Management Service. The -l flag configures the syslog drain.

$ cf create-user-provided-service SERVICE-INSTANCE -l SYSLOG-URL

Refer to User-Provided Service Instances for more information.

Step 3: Bind the Service Instance

You have two options for binding the service instance to an application:

  • Run cf push with a manifest. The services block in the manifest must specify the service instance that you want to bind.
  • Run cf bind-service. After a short delay, logs begin to flow automatically.

Refer to Binding Applications to Service Instances for more information.

Step 4: Verify Logs are Draining

To verify that logs are draining correctly to a third-party log management service:

  1. Take actions that produce log messages, such as making requests of your app.

  2. Compare the logs displayed in the CLI against those displayed by the log management service.

For example, if your application serves web pages, you can send HTTP requests to the application. In Cloud Foundry, these generate Router log messages, which you can view in the CLI. Your third-party log management service should display corresponding messages.

Note: For security reasons, Cloud Foundry applications do not respond to ping. You cannot use ping to generate log entries.