Kafka Monthly Digest – March 2021 – IBM Developer

This is the 38th edition of Kafka Monthly Digest!This edition covers what happened with Apache Kafka The community in March 2021.

See last month’s digest Kafka Monthly Digest: February 2020..



On February 23, Konstantine Karantasis volunteered for a major 3.0.0 release. This will be the next release after 2.8.So Determined by the community, 3.x series can run Kafka with or without ZooKeeper, ZooKeeper support will be completely removed only in 4.0.

The next phase of 3.0.0 is the KIP freeze. This is currently targeted for early June.Please refer to Release plan See the wiki for more information.


John Roesler unveiled RC0 on March 30th Voting in progress.

Kafka 2.8.0 introduces the option to run Kafka without ZooKeeper in Early Access, This is not for production..The name of this mode is KRaft, For Kafka Raft. Currently there is no migration path and many features are still disabled, but if you want to try it on a test cluster prior to 3.0.0, see. KRaft README..


On March 18th, we released the first release candidate for 2.7.1. Shortly thereafter, a Kafka Streams blocker issue was identified (KAFKA-12508). On March 26th, we released our second release candidate, RC1. Voting Currently in progress.


The 2.6.2 release process has continued. Sophie Blee-Goldman created RC0, the first release candidate on March 12th. KAFKA-12508 It also influenced this release. New release candidates should be available in the coming days.


Last month, the community submitted 12 KIP (KIP-717 to KIP-728), and these are my eye-catchers.

These four KIPs are in preparation for Kafka 3.0. Since 3.0 is a major release, the project may change the default behavior of some features and make backward incompatible changes.

  • KIP-720: MirrorMaker v1 abolished.. MirrorMaker 2 has been available for over a year now (Kafka 2.4.0, December 2019) and aims to completely replace its predecessor. Therefore, the community is considering deprecating MirrorMaker 1 for Kafka 3.0 before removing it altogether in the next major release.

  • KIP-722: Enable connector client override by default.. Kafka Connect client overrides were added in Kafka 2.3. This allows the connector to be used by the Connect runtime to assign different IDs (and ACLs) to different connectors and to fine-tune these clients for specific connectors (consumers, producers, and administrators). ) Can be customized. However, in very special cases, this feature is left disabled by default as it can cause backward compatibility issues. This KIP suggests changing this setting to enable all client overrides (set) connector.client.config.override.policy To all) By default.

  • KIP-724: End of support for message formats v0 and v1..Kafka uses Message format Write and save the record. Since 0.11 (released in 2017), Kafka has been using message format v2. Brokers still support the old format and can translate messages as needed. This proposal aims to improve broker maintainability by removing support for older formats (v0 and v1) in 3.0 as all modern clients are already using the latest format. ..

  • KIP-726: Makes CooperativeStickyAssignor the default assigner..Consumers use Transferor Calculates the partition allocation for members in a consumer group. Kafka currently offers four assigners. range, Round Robin, Sticky And Cooperative Sticky.. Cooperative Sticky was introduced in Kafka 2.4 and offers many improvements beyond the current default Range. This KIP suggests using Cooperative Sticky by default.

This section describes the releases of some community projects. This includes only open source projects.

  • strimzi-kafka-operator 0.22: Strimzi is a Kubernetes operator for running Kafka. This new version improves the management of Connect clusters and connectors. We will also introduce a new API version. v1beta2, To prepare for the adoption of Kubernetes 1.22.
  • Azkarra Streams 0.9: Azkarra is a framework that helps you build streaming microservices using KafkaStreams. RocksDB can be tweaked in this release. The UI dashboard has also been updated to improve the user experience and support Kafka 2.7.


Get started with Kafka

IBM Event Streams for Cloud is an Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today. Kafka Monthly Digest – March 2021 – IBM Developer

Back to top button