Kafka Streams

This section describes Kafka Streams, a component of open source Apache Kafka. Kafka Streams is a powerful, easy-to-use library for building highly scalable, fault-tolerant, distributed stream processing applications on top of Apache Kafka.

Contents:

• Introduction
  • Kafka Streams – The Power without the Weight
  • A closer look
• Requirements
• Quickstart
  • Goal of this quickstart
  • What we want to do
  • Start the Kafka cluster
  • Prepare the input data
  • Process the input data with Kafka Streams
  • Inspect the output data
  • Stop the Kafka cluster
  • Where to go from here
• Concepts
  • Kafka 101
  • Stream
  • Stream Processing Application
  • Processor Topology
  • Stream Processor
  • Time
  • Stateful Stream Processing
  • Duality of Streams and Tables
  • KStream (record stream)
  • KTable (changelog stream)
  • Windowing
  • Joins
  • Aggregations
• Architecture
  • Processor Topology
  • Parallelism Model
    • Stream Partitions and Tasks
    • Threading Model
    • Example
  • State
  • Fault Tolerance
  • Processing Guarantees
  • Flow Control with Timestamps
  • Backpressure
• Developer Guide
  • Code examples
    • Streams application examples in Apache Kafka
    • Streams application examples provided by Confluent
  • Configuring a Kafka Streams application
    • Overview
    • Required configuration parameters
    • Optional configuration parameters
    • Non-Streams configuration parameters
  • Writing a Kafka Streams application
    • Overview
    • Libraries and maven artifacts
    • Using Kafka Streams within your application code
    • Processor API
    • Kafka Streams DSL
  • Running a Kafka Streams application
    • Starting a Kafka Streams application
    • Elastic scaling of your application
  • Data types and serialization
    • Overview
    • Configuring default serializers/deserializers (serdes)
    • Overriding default serializers/deserializers (serdes)
    • Available serializers/deserializers (serdes)
    • Implementing custom serializers/deserializers (serdes)
  • Security
    • Overview
    • Security example
  • Application Reset Tool
    • Overview
    • User topics vs. internal topics
    • Scope
    • Usage
    • Example
• Upgrade Guide
  • Upgrading from CP 3.0.0 (Kafka 0.10.0.0-cp1) to CP 3.0.1 (Kafka 0.10.0.1-cp1)
    • Compatibility
    • Upgrading your Kafka Streams applications to CP 3.0.1
    • Full upgrade workflow
• FAQ
  • General
    • Is Kafka Streams a project separate from Apache Kafka?
    • Is Kafka Streams a proprietary library of Confluent?
    • Do Kafka Streams applications run inside the Kafka brokers?
    • What are the system dependencies of Kafka Streams?
    • How do I migrate my Kafka Streams applications CP 3.0.0 applications to CP 3.0.1
    • Can I use Kafka Streams with Kafka clusters running 0.9, 0.8, or 0.7?
  • Troubleshooting and debugging
    • Get more meaningful Java stacktraces?
    • Invalid Timestamp Exception
• Javadocs