Configuration Options for the rebalancer tool

The config file path is passed via the --config-file option and the following configs are supported. There are no required configs to try the tool although confluent.rebalancer.license is required after 30 days.

confluent.rebalancer.license

Confluent will issue a license key to each subscriber. The license key will be a short snippet of text that you can copy and paste. Without the license key, you can use the Confluent Rebalancer for a 30-day trial period. If you are a subscriber and don’t have a license key, please contact Confluent Support at support@confluent.io.

  • Type: string
  • Valid Values: Confluent Platform license
  • Importance: high
confluent.rebalancer.metrics.topic

The topic where the metrics reporter publishes its metrics to

  • Type: string
  • Default: “_confluent-metrics”
  • Importance: low

The consumer used to retrieve metrics data can be configured by prefixing the consumer config name with confluent.rebalancer.metrics.. For example, to configure the consumer to use SSL, one would set confluent.rebalancer.metrics.security.protocol=SSL.

Configuration Options for the metrics publisher

These configs should be added to the server.properties of each broker that is part of the Kafka cluster to be rebalanced. Only the first two configs below are required although confluent.metrics.reporter.topic.replicas should be changed if there are less than 3 brokers in the Kafka metrics cluster. The other configs allow one to tune the publisher for additional performance and reliability.

confluent.metrics.reporter.bootstrap.servers

Bootstrap servers for the Kafka cluster where metrics will be published to.

  • Type: string
  • Importance: high
confluent.metrics.reporter.zookeeper.connect

The Zookeeper URL for the Kafka cluster used for publishing metrics. The metrics cluster may be different from the cluster(s) whose metrics are being collected. Several production Kafka clusters can publish to a single metrics cluster, for example.

  • Type: string
  • Importance: high
confluent.metrics.reporter.publish.ms

The metrics reporter will publish new metrics to the metrics topic in intervals defined by this setting. This means that the rebalancer may compute a plan based on partition sizes that are stale by this duration. The default of 15 seconds is a reasonable value for production environments and it typically does not need to be changed.

  • Type: long
  • Default: 15000
  • Importance: medium
confluent.metrics.reporter.topic

Topic on which metrics data will be written.

  • Type: string
  • Default: “_confluent-metrics”
  • Importance: medium
confluent.metrics.reporter.topic.partitions

Number of partitions in the metric topic.

  • Type: int
  • Default: 10
  • Importance: medium
confluent.metrics.reporter.topic.replicas

Number of replicas in the metric topic. It should not be higher than the number of brokers in the Kafka cluster.

  • Type: int
  • Default: 3
  • Importance: medium
confluent.metrics.reporter.topic.retention.bytes

Retention bytes for the metrics topic.

  • Type: long
  • Default: -1
  • Importance: medium
confluent.metrics.reporter.topic.retention.ms

Retention time for the metrics topic.

  • Type: long
  • Default: 14400000
  • Importance: medium
confluent.metrics.reporter.topic.roll.ms

Log rolling time for the metrics topic.

  • Type: long
  • Default: 14400000
  • Importance: medium
confluent.metrics.reporter.whitelist

Regex matching the yammer metric mbean name or Kafka metric name.

  • Type: string
  • Default: ”.*MaxLag.*|kafka.log:type=Log,name=Size.*”
  • Importance: medium