Upgrade Apache Kafka

Upgrade each Apache Kafka node one at a time.

You can stop each Kafka broker and upgrade the component without downtime if you have enough replication for your topic.

You can stop one Kafka broker at a time, upgrade the Kafka binaries to HDP 2.6, and start the broker, before stopping the next broker.

	Note
If you are willing to accept downtime, you can take all of the brokers down, update the code, and restart all of the brokers. By default, the brokers start with the new protocol. You can bump the protocol version and restart, at any time after the brokers are upgraded.

Note

If you are willing to accept downtime, you can take all of the brokers down, update the code, and restart all of the brokers. By default, the brokers start with the new protocol.

You can bump the protocol version and restart, at any time after the brokers are upgraded.

The following steps are specific to upgrading to HDP 2.6 from a version lower than HDP 2.5. Follow these steps prior to upgrading Kafka:

Add the following properties to the server.properties file on all brokers:
```
inter.broker.protocol.version=CURRENT_KAFKA_VERSION
log.message.format.version=CURRENT_KAFKA_VERSION
```
Examples of CURRENT_KAFKA_VERSION are 0.8.2.0, 0.9.0.0 or 0.10.0.0.
Upgrade the brokers one at a time: shut down the broker, update the code, and restart it.
Once the entire cluster has been upgraded, bump the protocol version by setting inter.broker.protocol.version to 0.10.1.0.
If your previous message format is 0.10.0, change log.message.format.version to 0.10.1 (this is a no-op as the message format is the same for both 0.10.0 and 0.10.1). If your previous message format version is lower than 0.10.0, do not change log.message.format.version yet - this parameter should only change once all consumers have been upgraded to 0.10.0.0 or later.
Restart the brokers one at a time for the new protocol version to take effect.
If log.message.format.version is lower than 0.10.0, wait until all consumers have been upgraded to 0.10.0 or later. Once all consumers are upgraded to 0.10.0, change log.message.format.version to 0.10.1 on each broker, and restart each broker one at a time.

Follow these steps to upgrade Kafka:

Shut down the current Kafka daemon, switch to the new version, and start the daemon:

su - kafka -c "/usr/hdp/current/kafka-broker/bin/kafka stop"
hdp-select set kafka-broker 2.4.4.0-2633
su - kafka -c "usr/hdp/current/kafka-broker/bin/kafka start"

To verify that the Kafka daemon joined the cluster, create a topic and submit it to Kafka. Send a test message for that topic, and then validate that it was received by a consumer.
If the upgrade process fails, follow the steps in "Downgrading Kafka" to return to your previous version of Kafka.

​Upgrade Apache Kafka

Upgrade Apache Kafka