Installation

The Confluent Platform is available in a variety of formats. Ensure you have the Requirements, then select one of these options:

If you’re an application developer, you can include Confluent dependencies in your project via a repository:

Confluent does not currently support Windows. Windows users can download and use the zip and tar archives, but will have to run the jar files directly rather than use the wrapper scripts in the bin/ directory.

Requirements

The only requirement is Oracle Java >= 1.7. Java installation varies by platform, so please check the JRE installation requirements for your platform. Before installing the Confluent Platform, double check your Java version:

java -version

Various installation options are listed in the sections below. Select the approach you prefer and follow the given instructions to install the Confluent Platform.

zip and tar archives

These archives contain the jars, driver scripts, and configuration files in plain zip and tar archives. These should be used on OS X and as a simple way to evaluate the platform.

Start by downloading one of the archives. The complete list of downloads can be found on http://confluent.io/downloads/. For example, OS X users may choose http://packages.confluent.io/archive/2.0/confluent-2.0.1-2.11.7.tar.gz. After downloading, you can verify the integrity of the package by checking the the SHA-1 or SHA-256 signature of the package against the <file>.sha1.txt or <file>.sha256.txt provided in the same directory as the package.

Next extract the contents of the archive. For zip files, use a GUI to extract the contents or run this command in a terminal:

unzip confluent-2.0.1-2.11.7.zip

For tar files run this command:

tar xzf confluent-2.0.1-2.11.7.tar.gz

Next, try the Quickstart to test out the Confluent Platform locally.

deb packages via apt

The apt repositories provide packages for Debian-based Linux distributions such as Debian and Ubuntu.

First install Confluent’s public key, which is used to sign the packages in the apt repository.

wget -qO - http://packages.confluent.io/deb/2.0/archive.key | sudo apt-key add -

Add the repository to your /etc/apt/sources.list:

sudo add-apt-repository "deb http://packages.confluent.io/deb/2.0 stable main"

Run apt-get update and install the Confluent Platform:

sudo apt-get update && sudo apt-get install confluent-platform-2.11.7

The number at the end of the package name specifies the Scala version. Currently supported versions are 2.10.5 and 2.11.7. Individual components of the Confluent Platform are also available as standlone packages. See the Available Packages section for a listing of packages.

Next, try the Quickstart to test out the Confluent Platform locally.

rpm packages via yum

The yum repositories provide packages for RHEL, CentOS, and Fedora-based distributions.

First install Confluent’s public key, which is used to sign packages in the yum repository.

sudo rpm --import http://packages.confluent.io/rpm/2.0/archive.key

Add the repository to your /etc/yum.repos.d/ directory in a file named confluent.repo:

[confluent-2.0]
name=Confluent repository for 2.0.x packages
baseurl=http://packages.confluent.io/rpm/2.0
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/2.0/archive.key
enabled=1

The repository is now ready for use.

Note for users of RHEL 6.x and similar distributions:

Because the librdkafka RPM is currently not yet compatible with older Linux distributions based on RHEL 6.x, you cannot use the confluent-platform-2.11.7 package to install the entire Confluent Platform in a single command as shown below. Instead, please install the desired software packages such as confluent-kafka-2.11.7 or confluent-schema-registry individually. If you want to install librdkafka, the current workaround is to install it from the zip and tar archives.

You can install the entire Confluent Platform with:

sudo yum install confluent-platform-2.11.7

The number at the end of the package name specifies the Scala version. Currently supported versions are 2.10.5 and 2.11.7. Individual components of the Confluent Platform are also available as standlone packages. See the Available Packages section for a listing of packages.

Next, try the Quickstart to test out the Confluent Platform locally.

deb/rpm packages with installer script

Debian and RPM packages are also provided in self-contained archives with installers. These archives are available for Debian packages:

wget http://packages.confluent.io/archive/2.0/confluent-2.0.1-2.11.7-deb.tar.gz

and RPM packages:

wget http://packages.confluent.io/archive/2.0/confluent-2.0.1-2.11.7-rpm.tar.gz

Before running the installer script, install Confluent’s public key for Debian-based systems:

wget -qO - http://packages.confluent.io/deb/2.0/archive.key | sudo apt-key add -

or for RPM-based systems:

sudo rpm --import http://packages.confluent.io/rpm/2.0/archive.key

Next extract the contents and run the installer, demonstrated here with the Debian archive:

tar xzf confluent-2.0.1-2.11.7-deb.tar.gz
cd confluent-2.0.1
./install.sh

The script also provide an uninstall option:

./install.sh --uninstall

This only removes the contents of the package. Any data generated by running the services will be left in place.

Next, try the Quickstart to test out the Confluent Platform locally.

jars via maven repository

All jars included in the packages are also available in a Maven repository. Here’s a sample POM file showing how to add the repository:

<repositories>
  ...
  <repository>
    <id>confluent</id>
    <url>http://packages.confluent.io/maven/</url>
  </repository>
  ...
</repositories>

With the repository added, you can add dependencies on artifacts. For example, to use Confluent’s serializers that integrate with the rest of the Confluent Platform you could include the following in your pom.xml:

<dependencies>
  ...
  <dependency>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-avro-serializer</artifactId>
    <version>2.0.1</version>
  </dependency>
  ...
</dependencies>

The Confluent Platform’s Maven repository includes compiled versions of Kafka. To reference the Kafka version 0.9.0.1 that is included with CP 2.0.1, use the following in your pom.xml:

<dependencies>
  ...
  <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
    <!-- For CP 2.0.1 -->
    <version>0.9.0.1-cp1</version>
  </dependency>
  ...
</dependencies>

Confluent always contributes patches back to the open source project, but these may differ from the Apache artifacts when Confluent Platform and Apache Kafka releases do not align. In the case they do differ, we append the suffix -cpX to the version identifier of the CP version (with X being a digit) while keeping the group and artifact IDs identical in order to distinguish these from the Apache artifacts. For example, to reference the Kafka version 0.9.0.1 that is included with Confluent Platform 2.0, use the following in your pom.xml.

<dependencies>
  ...
  <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
    <!-- For CP 2.0.1 -->
    <version>0.9.0.1-cp1</version>
  </dependency>
  ...
</dependencies>

Available Packages

The Confluent Platform is available for several versions of Scala. The package names include the Scala version they were compiled for:

confluent-platform-<scala_version>

where scala_version can be one of:

  • 2.10.5
  • 2.11.7

If you choose to install via an apt or yum repository, you can install individual components instead of the complete Confluent Platform. Use these packages when you only need one component on your server, e.g. on production servers. The following are packages corresponding to each of the Confluent Platform components:

  • confluent-kafka-<scala_version>
  • confluent-kafka-connect-hdfs
  • confluent-kafka-connect-jdbc
  • confluent-schema-registry
  • confluent-kafka-rest
  • confluent-camus
  • librdkafka (C/C++ Kafka client):
    • deb:
      • librdkafka1 – library
      • librdkafka-dev – development headers and library
      • librdkafka1-dbg – debugging symbols
    • rpm:
      • librdkafka1 – library
      • librdkafka-devel – development headers and library
      • librdkafka1-debuginfo – debugging symbols
      • librdkafka – source rpm

The following packages are also installed as dependencies:

  • confluent-rest-utils
  • confluent-common