Chapter 3. Features
Features introduced in Streams for Apache Kafka 3.0.
Streams for Apache Kafka 3.0 on RHEL is based on Apache Kafka 4.0.
To view all the enhancements and bugs that are resolved in this release, see the Streams for Apache Kafka Jira project.
3.1. Kafka 4.0 support
Streams for Apache Kafka now supports and uses Apache Kafka version 4.0. Only Kafka distributions built by Red Hat are supported.
For upgrade instructions, see Using Streams for Apache Kafka on RHEL in KRaft mode.
Refer to the Kafka 4.0 Release Notes for additional information.
Kafka 3.9.x is supported only for the purpose of upgrading to Streams for Apache Kafka 3.0. We recommend that you perform a rolling update to use the new binaries.
Support for ZooKeeper ends
Kafka 4.0 operates only in KRaft mode, where Kafka runs without ZooKeeper by utilizing the Raft protocol. Kafka 3.9 was the final version to support ZooKeeper. Consequently, Streams for Apache Kafka 2.9.x (LTS) is the last version compatible with Kafka clusters using ZooKeeper.
Migrate ZooKeeper-based Kafka clusters
To upgrade to Streams for Apache Kafka 3.0 or later, you must first migrate your Kafka clusters to KRaft mode.
KRaft mode limitations
For Kafka 3.8 and earlier, the controller quorums (which replace Zookeeper) were of fixed size (static). Dynamic controller quorums were introduced in Kafka 3.9.
Migration between Kafka’s static and dynamic controller quorums is not currently supported. For more information, see KAFKA-16538.
Streams for Apache Kafka 3.0 on RHEL supports static and dynamic controller quorums, with dynamic quorums recommended for new deployments.
3.2. Streams for Apache Kafka
3.2.1. Full adoption of KRaft mode
In line with the removal of ZooKeeper support in Apache Kafka 4.0, this release fully adopts KRaft mode (Kafka Raft metadata mode) for managing Kafka metadata. Streams for Apache Kafka no longer supports ZooKeeper-based deployments.
The move to KRaft simplifies deployment, streamlines management of Kafka clusters, and aligns with the Kafka 4.0 architecture.
Make sure that Kafka clusters are using KRaft before upgrading to Streams for Apache Kafka 3.0 or later.
For more information, see Migrating your Kafka cluster to KRaft mode.
3.2.2. MirrorMaker 1 removed
Support for MirrorMaker 1 has been removed. From Streams for Apache Kafka 3.0, only MirrorMaker 2 is supported.
Make sure to migrate to MirrorMaker 2 before before upgrading to Streams for Apache Kafka 3.0 or later.
To replicate the behavior of MirrorMaker 1, configure MirrorMaker 2 with the IdentityReplicationPolicy property.
3.2.3. Tiered storage
Tiered storage moves from early access and is now suitable for production environments.
Tiered storage introduces a flexible approach to managing Kafka data whereby log segments are moved to a separate storage system. Tiered storage requires an implementation of Kafka’s RemoteStorageManager interface to handle communication between Kafka and the remote storage system, which is enabled through configuration of the Kafka resource.
Streams for Apache Kafka includes plugin libraries from the open-source tiered-storage-for-apache-kafka project from Aiven for AWS S3 and file system storage by default in Kafka v4.0 and later. Using these plugins you can configure tiered storage using the following types of remote storage:
- AWS S3 remote storage
- File system remote storage
Streams for Apache Kafka supports a set of optional tuning properties for the tiered storage plugin. These properties can help optimize storage performance, throughput, and resource usage depending on your environment.
3.2.4. Metrics Reporter (Tech Preview)
The Streams for Apache Kafka Metrics Reporter is introduced as a Tech Preview. The reporter exposes Kafka metrics directly over HTTP in a Prometheus-compatible format. It integrates with Kafka brokers, clients, Kafka Connect, MirrorMaker 2, and Kafka Streams applications.
3.3. Kafka Bridge
3.3.1. Metrics Reporter (Tech Preview)
The Tech Preview for the Metrics Reporter can also be enabled on the Kafka Bridge by setting the bridge.metrics option to strimziMetricsReporter.
3.3.2. Support for custom Prometheus JMX Exporter configuration
Set bridge.metrics.exporter.config.path=/<path_to>/my-exporter-config.yml to use your custom configuration.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}