Configuration Options for the rebalancer tool¶
The config file path is passed via the --config-file
option and the following configs are supported. There are no required configs
to try the tool although confluent.rebalancer.license
is required after 30 days.
confluent.rebalancer.license
Confluent will issue a license key to each subscriber. The license key will be a short snippet of text that you can copy and paste. Without the license key, you can use the Confluent Rebalancer for a 30-day trial period. If you are a subscriber and don’t have a license key, please contact Confluent Support at support@confluent.io.
- Type: string
- Valid Values: Confluent Platform license
- Importance: high
confluent.rebalancer.metrics.topic
The topic where the metrics reporter publishes its metrics to
- Type: string
- Default: “_confluent-metrics”
- Importance: low
The consumer used to retrieve metrics data can be configured by prefixing the consumer config name with confluent.rebalancer.metrics.
. For example, to
configure the consumer to use SSL, one would set confluent.rebalancer.metrics.security.protocol=SSL
.
Configuration Options for the metrics publisher¶
These configs should be added to the server.properties
of each broker that is part of the Kafka cluster to be rebalanced. Only the first two configs below
are required although confluent.metrics.reporter.topic.replicas
should be changed if there are less than 3 brokers in the Kafka metrics cluster. The
other configs allow one to tune the publisher for additional performance and reliability.
confluent.metrics.reporter.bootstrap.servers
Bootstrap servers for the Kafka cluster where metrics will be published to.
- Type: string
- Importance: high
confluent.metrics.reporter.zookeeper.connect
The Zookeeper URL for the Kafka cluster used for publishing metrics. The metrics cluster may be different from the cluster(s) whose metrics are being collected. Several production Kafka clusters can publish to a single metrics cluster, for example.
- Type: string
- Importance: high
confluent.metrics.reporter.publish.ms
The metrics reporter will publish new metrics to the metrics topic in intervals defined by this setting. This means that the rebalancer may compute a plan based on partition sizes that are stale by this duration. The default of 15 seconds is a reasonable value for production environments and it typically does not need to be changed.
- Type: long
- Default: 15000
- Importance: medium
confluent.metrics.reporter.topic
Topic on which metrics data will be written.
- Type: string
- Default: “_confluent-metrics”
- Importance: medium
confluent.metrics.reporter.topic.partitions
Number of partitions in the metric topic.
- Type: int
- Default: 10
- Importance: medium
confluent.metrics.reporter.topic.replicas
Number of replicas in the metric topic. It should not be higher than the number of brokers in the Kafka cluster.
- Type: int
- Default: 3
- Importance: medium
confluent.metrics.reporter.topic.retention.bytes
Retention bytes for the metrics topic.
- Type: long
- Default: -1
- Importance: medium
confluent.metrics.reporter.topic.retention.ms
Retention time for the metrics topic.
- Type: long
- Default: 14400000
- Importance: medium
confluent.metrics.reporter.topic.roll.ms
Log rolling time for the metrics topic.
- Type: long
- Default: 14400000
- Importance: medium
confluent.metrics.reporter.whitelist
Regex matching the yammer metric mbean name or Kafka metric name.
- Type: string
- Default: ”.*MaxLag.*|kafka.log:type=Log,name=Size.*”
- Importance: medium