Installation

The Confluent Platform is available in a variety of formats. Ensure you have the Requirements, then select one of these options:

If you’re an application developer, you can include Confluent dependencies in your project via a repository:

Confluent does not currently support Windows. Windows users can download and use the zip and tar archives, but will have to run the jar files directly rather than use the wrapper scripts in the bin/ directory.

Docker images for the Confluent Platform are currently available on DockerHub. Alternatively, the source files for the images are available on Github if you would prefer to extend and/or rebuild the images and upload them to your own DockerHub repository. If you are interested in deploying with Docker, please refer to our full Docker image documentation.

Requirements

The only requirement is Oracle Java >= 1.7. Java installation varies by platform, so please check the JRE installation requirements for your platform. Before installing the Confluent Platform, double check your Java version:

$ java -version

Various installation options are listed in the sections below. Select the approach you prefer and follow the given instructions to install the Confluent Platform.

ZIP and TAR archives

These archives contain the jars, driver scripts, and configuration files in plain zip and tar archives. These should be used on OS X and as a simple way to evaluate the platform.

Start by downloading one of the archives. The complete list of downloads can be found on http://confluent.io/downloads/. For example, OS X users may choose http://packages.confluent.io/archive/3.0/confluent-3.0.1-2.11.tar.gz. After downloading, you can verify the integrity of the package by checking the the SHA-1 or SHA-256 signature of the package against the <file>.sha1.txt or <file>.sha256.txt provided in the same directory as the package.

Next extract the contents of the archive. For zip files, use a GUI to extract the contents or run this command in a terminal:

$ unzip confluent-3.0.1-2.11.zip

For tar files run this command:

$ tar xzf confluent-3.0.1-2.11.tar.gz

Next, try the Quickstart to test out the Confluent Platform locally.

DEB packages via apt

The apt repositories provide packages for Debian-based Linux distributions such as Debian and Ubuntu.

First install Confluent’s public key, which is used to sign the packages in the apt repository.

$ wget -qO - http://packages.confluent.io/deb/3.0/archive.key | sudo apt-key add -

Add the repository to your /etc/apt/sources.list:

$ sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.0 stable main"

Run apt-get update and install the Confluent Platform:

$ sudo apt-get update && sudo apt-get install confluent-platform-2.11

The number at the end of the package name specifies the Scala version. Currently supported versions are 2.11 (recommended) and 2.10. Individual components of the Confluent Platform are also available as standalone packages. See the Available Packages section for a listing of packages.

Next, try the Quickstart to test out the Confluent Platform locally.

RPM packages via yum

The yum repositories provide packages for RHEL, CentOS, and Fedora-based distributions.

First install Confluent’s public key, which is used to sign packages in the yum repository.

$ sudo rpm --import http://packages.confluent.io/rpm/3.0/archive.key

Add the repository to your /etc/yum.repos.d/ directory in a file named confluent.repo.

If you are using RHEL/Centos/Oracle 6

[Confluent.dist]
name=Confluent repository (dist)
baseurl=http://packages.confluent.io/rpm/3.0/6
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1

[Confluent]
name=Confluent repository
baseurl=http://packages.confluent.io/rpm/3.0
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1

If you are using RHEL/Centos/Oracle 7

[Confluent.dist]
name=Confluent repository (dist)
baseurl=http://packages.confluent.io/rpm/3.0/7
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1

[Confluent]
name=Confluent repository
baseurl=http://packages.confluent.io/rpm/3.0
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1

It is recommended to clear the yum caches before proceeding:

$ sudo yum clean all

The repository is now ready for use.

You can install the entire Confluent Platform with:

$ sudo yum install confluent-platform-2.11

The number at the end of the package name specifies the Scala version. Currently supported versions are 2.11 (recommended) and 2.10. Individual components of the Confluent Platform are also available as standalone packages. See the Available Packages section for a listing of packages.

Next, try the Quickstart to test out the Confluent Platform locally.

DEB/RPM packages with installer script

Debian and RPM packages are also provided in self-contained archives with installers. These archives are available for Debian packages:

$ wget http://packages.confluent.io/archive/3.0/confluent-3.0.1-2.11-deb.tar.gz

and RPM packages:

$ wget http://packages.confluent.io/archive/3.0/confluent-3.0.1-2.11-rpm.tar.gz

Before running the installer script, install Confluent’s public key for Debian-based systems:

$ wget -qO - http://packages.confluent.io/deb/3.0/archive.key | sudo apt-key add -

or for RPM-based systems:

$ sudo rpm --import http://packages.confluent.io/rpm/3.0/archive.key

Next extract the contents and run the installer, demonstrated here with the Debian archive:

$ tar xzf confluent-3.0.1-2.11-deb.tar.gz
$ cd confluent-3.0.1
$ ./install.sh

The script also provide an uninstall option:

$ ./install.sh --uninstall

This only removes the contents of the package. Any data generated by running the services will be left in place.

Next, try the Quickstart to test out the Confluent Platform locally.

Available Packages

Confluent Platform ships with two groups of packages:

  1. The platform packages of CP (confluent-platform-<scala_version>), which are umbrella packages to install the complete Confluent Platform, i.e. all its individual packages. See next point.
  2. The individual packages of the Confluent Platform such as confluent-kafka-<scala_version> and confluent-schema-registry.

Here, the platform packages (confluent-platform-<scala_version>) as well as the individual Kafka packages (confluent-kafka-<scala_version>) – and only these – are available in two variants, named by the respective Scala version of Apache Kafka that is included in the packages.

Note

Why Scala, and why different Scala versions? Apache Kafka is implemented in Scala and built for multiple versions of Scala. However, the Scala version only matters if you are actually using Scala for implementing your applications and you want a Kafka version built for the same Scala version you use. Otherwise any version should work, with 2.11 being recommended.

In CP 3.0.1, scala_version can be one of:

  • 2.11 (recommended) – giving confluent-platform-2.11 and confluent-kafka-2.11
  • 2.10 – giving confluent-platform-2.10 and confluent-kafka-2.10

If you choose to install via an apt or yum repository, you can install individual components instead of the complete Confluent Platform. Use these packages when you only need one component on your server, e.g. on production servers. The following are packages corresponding to each of the Confluent Platform components:

  • confluent-platform-<scala_version> – umbrella package that installs the complete Confluent Platform, i.e. all the individual packages mentioned below

  • confluent-platform-oss-<scala version> – umbrella package that installs only the open source Confluent Platform, i.e. all the individual packages mentioned below except confluent-control-center and confluent-support-metrics.

  • confluent-kafka-<scala_version> – Apache Kafka, including Kafka Streams, Kafka Connect framework, and Kafka Java client

  • confluent-control-center – GUI-based system for managing and monitoring Apache Kafka

  • confluent-kafka-connect-hdfs – Hadoop HDFS Connector for Kafka Connect

  • confluent-kafka-connect-jdbc – JDBC Connector for Kafka Connect

  • confluent-schema-registry – Schema Registry

  • confluent-kafka-rest – HTTP REST Proxy for Apache Kafka

  • confluent-support-metrics – Part of “Proactive Support” - improves Confluent’s support for the platform by collecting and reporting support metrics (“Metrics”) to Confluent

  • confluent-camus – Camus (deprecated)

  • librdkafka – C/C++ Kafka client

    • DEB packages

      • librdkafka1 – library
      • librdkafka-dev – development headers and library
      • librdkafka1-dbg – debugging symbols

    • RPM packages

      • librdkafka1 – library
      • librdkafka-devel – development headers and library
      • librdkafka1-debuginfo – debugging symbols
      • librdkafka – source rpm

    • Source packages - included in zip and tar archives under src/ directory

  • avro – C/C++ Avro serialization library

    • DEB packages

      • libavro-c1 – C library and source
      • libavro-cpp1 – C++ library and source
      • libavro-c-dev – C development headers and library
      • libavro-cpp-dev – C++ development headers and library

    • RPM packages

      • avro-c – C library and source
      • avro-cpp – C++ library and source
      • avro-c-tools – command line tools
      • avro-c-devel – C development headers and library
      • avro-cpp-devel – C++ development headers and library
      • avro-c-debuginfo – C debugging symbols
      • avro-cpp-debuginfo – C++ debugging symbols

    • Source packages - included in zip and tar archives under src/ directory

  • confluent-libserdes – C/C++ Avro Serialization with Schema Registry support

    • DEB packages

      • confluent-libserdes-c1 – library and source
      • confluent-libserdes-dev – development headers and library

    • RPM packages

      • confluent-libserdes – library and source
      • confluent-libserdes-devel – development headers and library
      • confluent-libserdes-debuginfo – debugging symbols

    • Source packages - included in zip and tar archives under src/ directory

  • confluent-kafka-python – Python client library

    • Python package – available on PyPI
    • Source packages – included in zip and tar archives under src/ directory

The following packages may also be installed automatically as dependencies:

  • confluent-common
  • confluent-rest-utils

Maven repository for jars

All jars included in the packages are also available in the Confluent maven repository. Here’s a sample POM file showing how to add this repository:

<repositories>

  <repository>
    <id>confluent</id>
    <url>http://packages.confluent.io/maven/</url>
  </repository>

  <!-- further repository entries here -->

</repositories>

The Confluent maven repository includes compiled versions of Kafka. To reference the Kafka version 0.10.0.1 that is included with CP 3.0.1, use the following in your pom.xml:

<dependencies>

  <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
    <!-- For CP 3.0.1 -->
    <version>0.10.0.1-cp1</version>
  </dependency>

  <!-- further dependency entries here -->

</dependencies>

Note

Version names of Kafka in Apache vs. Kafka in Confluent Platform: Confluent always contributes patches back to the Apache Kafka open source project. However, the exact versions (and version names) being included in Confluent Platform may differ from the Apache artifacts when Confluent Platform and Apache Kafka releases do not align. In the case they do differ, we keep the groupId and artifactId identical but append the suffix -cpX to the version identifier of the CP version (with X being a digit) in order to distinguish these from the Apache artifacts.

For example, to reference the Kafka version 0.9.0.1 that is included with Confluent Platform 2.0.1, you would use the following snippet in your pom.xml:

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka_2.11</artifactId>
  <!-- For CP 2.0.1 -->
  <version>0.9.0.1-cp1</version>
</dependency>

Further artifacts are available. For example, to use Confluent’s serializers that integrate with the rest of the Confluent Platform you would include the following in your pom.xml:

<dependencies>

  <dependency>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-avro-serializer</artifactId>
    <!-- For CP 3.0.1 -->
    <version>3.0.1</version>
  </dependency>

  <!-- further dependency entries here -->

</dependencies>