Product
KSQL
Learn about KSQL, Streaming SQL for Apache Kafka and what it can do.
Play
Confluent Platform
Open Source
Connectors
Enterprise
Confluent Control Center
Auto Data Balancer
Replicator
Connectors
Cloud
KSQL
Subscription
Services
Solutions
New
Get an overview of an Apache Kafka-centric microservices architecture.
Play
Industry Solutions
Microservices
Internet of Things
Developers
Kafka Summit
Get early-bird pricing for Kafka Summit London, April 23-24, 2018.
Register Now
What is Kafka?
Resources
Online Talks
Community
Training
Kafka Summit
Blog
Docs
Download
Getting Started
What is the Confluent Platform?
What is included in the Confluent Platform?
Overview
Confluent Open Source
Confluent Enterprise
Confluent Proactive Support
Migrating an existing Kafka deployment
Migrating developer tools
Migrating to Control Center
Confluent Platform Quick Start
Installing and Upgrading
Install Confluent Platform
System Requirements
Installation Formats
Packages
Platform Packages
Component Packages
Installing using Docker
Docker Quick Start
Docker Configuration
Docker Advanced Tutorials
Clients
Maven repository for JARs
C/C++
Python
Go
.NET
Migrating from Open Source to Enterprise
Upgrade
Preparation
Step-by-step Guide
Supported Versions and Interoperability
Confluent Platform Versions
Operating Systems
Confluent Control Center
Java
Scala
Component Security
Clients
Connectors
Cross-Component Compatibility
Docker and Orchestration Tools
Command Line Interface
Installing and Configuring the CLI
Installing the CLI
Configuring the CLI
Command Reference
confluent acl
confluent config
confluent current
confluent destroy
confluent help
confluent list
confluent load
confluent log
confluent start
confluent status
confluent stop
confluent top
confluent unload
Available Commands
Environment Variables
Confluent Platform 4.0.0 Release Notes
Highlights
Enterprise Features
Open Source Features
Previous releases
Confluent Platform 3.3.1 Release Notes
Confluent Platform 3.3.0 Release Notes
Confluent Platform 3.2.2 Release Notes
Confluent Platform 3.2.1 Release Notes
Confluent Platform 3.2.0 Release Notes
Confluent Platform 3.1.2 Release Notes
Confluent Platform 3.1.1 Release Notes
Confluent Platform 3.1.0 Release Notes
Confluent Platform 3.0.1 Release Notes
Confluent Platform 3.0.0 Release Notes
Confluent Platform 2.0.1 Release Notes
Confluent Platform 2.0.0 Release Notes
How to Download
Questions?
Operations
Kafka Operations
Production Deployment
Hardware
JVM
Important Configuration Options
File descriptors and mmap
Multi-node Configuration
Post Deployment
Changing configs dynamically
Logging
Admin operations
Rolling Restart
Performance Tips
Software Updates
Backup and Restoration
Auto Data Balancing
Requirements
Quickstart
Licensing
Configuration Options
Monitoring Kafka
Server Metrics
Producer Metrics
New Consumer Metrics
Old Consumer Metrics
Confluent Metrics Reporter
Installation
Configuration
Configuration Options
Verification
Confluent Control Center
Control Center Quick Start
Configure Kafka and Kafka Connect
Configure and start Control Center
Setup stream monitoring
Control Center Concepts
Motivation
Time windows and metrics
Latency
Installing and Upgrading Control Center
Installing Control Center
Control Center System Requirements
Configuring Control Center
Installing Control Center Interceptors
Installing Control Center on Apache Kafka
Troubleshooting Control Center
Upgrading Control Center
Control Center User Interface
System Health
Stream Monitoring
Kafka Connect
Alerts
Clusters
Topic Management
Security
Configuring SSL
Configuring SASL
UI Authentication
UI HTTPS
Authorization with Kafka ACLS
Changelog
Version 4.0.0
Version 3.3.1
Version 3.3.0
Version 3.2.2
Version 3.2.1
Version 3.2.0
Version 3.1.0
Version 3.0.1
Multi Data-Center Deployment
Replicator Quick Start
Start the destination cluster
Start the origin cluster
Create a topic
Configure and run Replicator
Installing and Configuring Replicator
Install and Configure Kafka Connect Cluster for Replicator
Configure and run a Confluent Replicator on the Connect Cluster
Tuning and Monitoring Replicator
Sizing Replicator Cluster
Getting More Throughput From Replicator Tasks
Monitoring Replicator
Configuration Options
Confluent Platform
Source Topics
Source Data Conversion
Source Zookeeper
Source Kafka
Source Kafka: Security
Source Kafka: Consumer
Destination Topics
Destination Zookeeper
Destination Data Conversion
Apache Kafka’s Mirror Maker
Comparing Mirror Maker to Confluent Replicator
Example of Mirror Maker Use
Schema Registry Operations
Production Deployment
Hardware
Memory
CPUs
Disks
Network
JVM
Important Configuration Options
Don’t Touch These Settings!
Kafka & ZooKeeper
Migration from Zookeeper master election to Kafka master election
Backup and Restore
Monitoring
Global Metrics
Per-Endpoint Metrics
Endpoints
Kafka REST Proxy Operations
Production Deployment
Hardware
JVM
Deployment
Important Configuration Options
Don’t Touch These Settings!
Post Deployment
Monitoring
Global Metrics
Per-Endpoint Metrics
Endpoints
Docker Operations
Monitoring
Using JMX
Configuring logging
log4j Log Levels
Component Names
Mounting External Volumes
Data Volumes for Kafka & ZooKeeper
Security: Data Volumes for Configuring Secrets
Configuring Connect with External jars
Security
Prefixes for Configuring Security
Encryption with SSL
SSL Overview
Creating SSL Keys and Certificates
Brokers
Clients
ZooKeeper
Kafka Connect
Confluent Replicator
Confluent Control Center
Confluent Metrics Reporter
Confluent Monitoring Interceptors
Schema Registry
REST Proxy
SSL Logging
Encryption and Authentication with SSL
SSL Overview
Creating SSL Keys and Certificates
Brokers
Clients
ZooKeeper
Kafka Connect
Confluent Replicator
Confluent Control Center
Confluent Metrics Reporter
Confluent Monitoring Interceptors
Schema Registry
REST Proxy
SSL Logging
Authentication with SASL
SASL Overview
GSSAPI
SCRAM
PLAIN
JAAS configurations
Enabling multiple SASL mechanisms
Modifying SASL mechanisms in a Running Cluster
Authorization and ACLs
Overview
Command Line Interface
Adding Security to a Running Cluster
Migrating Brokers and Clients
Migrating ZooKeeper
Migrating the ZooKeeper ensemble
Kafka Security & the Confluent Platform
Other Observations
Streams Security
Required ACL setting for secure Kafka clusters
Security example
Confluent Security Plugins
Installation
Docker Security
Kafka Connect
Introduction
Requirements
Kafka Connect Quick Start
Goal
What we will do
Start the services
Read File Data with Connect
Write File Data with Connect
Concepts
Connectors
Tasks
Workers
Converters
Transforms
Installing and Configuring Kafka Connect
Getting Started
Planning for Installation
Installing Plugins
Running Workers
Configuring Workers
Upgrading Kafka Connect Workers
Managing Connectors
Using Bundled Connectors
Configuring Connectors
Managing Running Connectors
Using Community Connectors
Upgrading a Connector Plugin
Bundled Connectors
Confluent JDBC Connector
Confluent HDFS Connector
Confluent S3 Connector
Confluent Elasticsearch Connector
Confluent Kafka Replicator
Kafka FileStream Connectors
Security
Encryption
Authentication
Separate principals
ACL Considerations
Architecture & Internals
Motivation
Architecture
Connector Developer Guide
Core Concepts and APIs
Developing a Simple Connector
Dynamic Input/Output Partitions
Configuration Validation
Working with Schemas
Schema Evolution
Testing
Packaging
FAQ
How do I change the output data format of a SinkConnector?
Why does a connector configuration update trigger a task rebalance?
Why should I use distributed mode instead of standalone?
Do I need to write custom code to use Kafka Connect?
Is the Schema Registry a required service to run Kafka Connect?
How can I use plain JSON data with Connect?
Does source connector X support output format Y?
Why is CPU usage high for my Connect worker when no connectors have been deployed?
Can connect sink connectors read data written by other clients, e.g. a custom client?
After testing a connector in standalone mode, restarting it doesn’t write the data again?
Can I use a newer version of connect with older brokers?
Reference
Javadocs
REST Interface
All Worker Configs
Camus Operations
Important Configuration Options
ZooKeeper Operations
ZooKeeper Production Deployment
Stable version
Hardware
JVM
Important Configuration Options
Monitoring
Multi-node Setup
Post Deployment
Development
Kafka Streams
Introduction
The Kafka Streams API in a Nutshell
Use Case Examples
A Closer Look
Requirements
Apache Kafka
Confluent
Kafka Streams Quick Start
Purpose
Start the Kafka cluster
Prepare the topics and the input data
Process the input data with Kafka Streams
Inspect the output data
Stop the Kafka cluster
Next steps
Kafka Streams Demo Application
Requirements
Running the Kafka Music demo application
Running further Confluent demo applications for the Kafka Streams API
Appendix
Concepts
Kafka 101
Stream
Stream Processing Application
Processor Topology
Stream Processor
Stateful Stream Processing
Duality of Streams and Tables
KStream
KTable
GlobalKTable
Time
Aggregations
Joins
Windowing
Interactive Queries
Architecture
Processor Topology
Parallelism Model
State
Memory management
Fault Tolerance
Processing Guarantees
Flow Control with Timestamps
Backpressure
Code Examples
Getting started examples
Security examples
Interactive queries examples
End-to-end application examples
Developer Guide
Writing a Streams Application
Configuring a Streams Application
Streams DSL
Processor API
Data Types and Serialization
Interactive Queries
Memory Management
Running Streams Applications
Managing Streams Application Topics
Streams Security
Application Reset Tool
Operations
Capacity planning and sizing
Monitoring your application
Upgrade Guide
Upgrading from CP 3.3.x (Kafka 0.11.0.x-cp1) to CP 4.0.0 (Kafka 1.0.0-cp1)
Upgrading older Kafka Streams applications to CP 4.0.0
FAQ
General
Scalability
Processing
Failure and exception handling
Interactive Queries
Security
Troubleshooting and debugging
Javadocs
KSQL
Download and Learn More
Kafka Clients
Kafka Consumers
Concepts
Configuration
Initialization
Basic Usage
Detailed Examples
Administration
Kafka Producers
Concepts
Configuration
Examples
Confluent’s JMS Client for Apache Kafka
Overview
Examples
Requirements
Installation
JMS 1.1 Compatibility
Features
Unsupported Features
Developing JMS Clients
Threading
API Docs
Connector Developer Guide
Core Concepts and APIs
Connectors and Tasks
Partitions and Records
Dynamic Connectors
Developing a Simple Connector
Connector Example
Task Example - Source Task
Sink Tasks
Resuming from Previous Offsets
Dynamic Input/Output Partitions
Configuration Validation
Working with Schemas
Schema Evolution
Testing
Packaging
Creating an Archive
Creating an Uber JAR
Schema Registry
Introduction
Quickstart
Installation
Deployment
Development
Requirements
Contribute
License
Changelog
Version 3.4.0
Version 3.3.0
Version 3.2.1
Version 3.2.0
Version 3.1.1
Version 3.1.0
Version 3.0.1
Version 3.0.0
Version 2.0.1
Version 2.0.0
API Reference
Overview
Schemas
Subjects
Compatibility
Config
Configuration Options
Design Overview
Schema ID Allocation
Kafka Backend
Single Master Architecture
Schema Registry Operations
Production Deployment
Monitoring
Multi-DC Setup
Overview
Kafka Election
Zookeeper Election
Security Overview
Kafka Store
ZooKeeper
Upgrade from HTTP to HTTPS for REST API
Serializer and Formatter
Serializer
Formatter
Wire Format
Schema Deletion Guidelines
Maven Plugin
schema-registry:download
schema-registry:test-compatibility
schema-registry:register
Docker Developer Guide
Image Design Overview
The Bootup Process
Configuration
Preflight Checks
Launching the Process
Development Guidelines
Setup
Building the Images
Running Tests
Make Targets
Extending the Docker Images
Prerequisites
Adding Connectors to the Kafka Connect Image
Examples
Utility Scripts
Docker Utility Belt (dub)
Confluent Platform Utility Belt (cub)
Client Properties
References
Data Serialization and Evolution
Introduction
Avro
Defining an Avro Schema
Schema Evolution
Backward Compatibility
Forward Compatibility
Full Compatibility
Confluent Schema Registry
Kafka REST Proxy
Kafka REST Proxy
Quickstart
Features
Installation
Deployment
Development
Requirements
Contribute
License
Changelog
Version 3.3.0
Version 3.2.2
Version 3.2.1
Version 3.2.0
Version 3.1.2
Version 3.1.1
Version 3.1.0
Version 3.0.0
Version 2.0.1
Version 2.0.0
API Reference
Overview
API v2
API v1
Configuration Options
Security Configuration Options
Interceptor Configuration Options
Kafka REST Operations
Production Deployment
Monitoring
Security Overview
Camus
Camus
Key Features
Quickstart
Installation
Deployment
Development
Requirements
Contribute
License
Changelog
Version 4.0.0
Version 3.3.1
Version 3.3.0
Version 3.2.2
Version 3.2.1
Version 3.2.0
Version 3.1.2
Version 3.1.1
Version 3.1.0
Version 3.0.0
Version 2.0.1
Version 2.0.0
Configuration Options
Schema Registry Configuration
Camus Job Configuration
Kafka Configuration
Example Configuration
Design
Operations
Important Configuration Options
Application Development
Native Clients with Serializers
Java
C/C++
REST Proxy
Confluent Proactive Support
What is Proactive Support?
How it works
Which metadata is collected?
Version Collector (default)
Confluent Support Metrics (add-on package)
Which metadata and data is not being collected?
Installing Support Metrics
DEB Packages via apt
RPM Packages via yum
Enabling or disabling the Metrics feature
Recommended Proactive Support configuration settings for licensed Confluent customers
Proactive Support configuration settings
Network ports used by Proactive Support
Sharing Proactive Support Metadata with Confluent manually
Privacy Policy
Version 4.0.0
Docs
»
Index
Index
Please report any inaccuracies on this page or suggest an edit.
v.--
