Release Notes for Streams for Apache Kafka 2.7 on RHEL

Streams for Apache Kafka now supports and uses Apache Kafka version 3.7.0. Only Kafka distributions built by Red Hat are supported.

For upgrade instructions, see the instructions for Streams for Apache Kafka and Kafka upgrades in the following guides:

Refer to the Kafka 3.7.0 Release Notes for additional information.

Kafka 3.6.x is supported only for the purpose of upgrading to Streams for Apache Kafka 2.7. We recommend that you perform a rolling update to use the new binaries.

KRaft mode in Streams for Apache Kafka is a technology preview, with some limitations, but this release introduces a number of new features that support KRaft. To support using KRaft, a new using guide is available: Using Streams for Apache Kafka on RHEL in KRaft mode.

If you are using ZooKeeper for metadata management in your Kafka cluster, you can now migrate to using Kafka in KRaft mode.

During the migration, you do the following:

See Migrating to KRaft mode.

KRaft to KRaft upgrades are now supported. You update the installation files, then configure and restart all Kafka nodes. You then upgrade the KRaft-based Kafka cluster to a newer supported KRaft metadata version.

./bin/kafka-features.sh --bootstrap-server <broker_host>:<port> upgrade --metadata 3.7

./bin/kafka-features.sh --bootstrap-server <broker_host>:<port> upgrade --metadata 3.7

See Upgrading KRaft-based Kafka clusters.

RHEL 7 is no longer supported. The decision was made due to known incompatibility issues.

For an overview of the enhancements introduced with Kafka 3.7.0, refer to the Kafka 3.7.0 Release Notes.

When performing producer operations, POST requests must provide Content-Type headers specifying the embedded data format of the messages produced. Previously, JSON and binary were the supported formats for record and key values. It’s now possible to also use text format.

KRaft mode is available as a technology preview.

Currently, the KRaft mode in Streams for Apache Kafka has the following limitations:

Use the technology preview of the Kafka Static Quota plugin to set throughput and storage limits on brokers in your Kafka cluster. You can set a byte-rate threshold and storage quotas to put limits on the clients interacting with your brokers.

client.quota.callback.class= io.strimzi.kafka.quotas.StaticQuotaCallback
client.quota.callback.static.produce= 1000000
client.quota.callback.static.fetch= 1000000
client.quota.callback.static.storage.soft= 400000000000
client.quota.callback.static.storage.hard= 500000000000
client.quota.callback.static.storage.check-interval= 5

client.quota.callback.class= io.strimzi.kafka.quotas.StaticQuotaCallback
client.quota.callback.static.produce= 1000000
client.quota.callback.static.fetch= 1000000
client.quota.callback.static.storage.soft= 400000000000
client.quota.callback.static.storage.hard= 500000000000
client.quota.callback.static.storage.check-interval= 5

See Setting limits on brokers using the Kafka Static Quota plugin.

Support for Java 11 is deprecated in Kafka 3.7.0 and Streams for Apache Kafka 2.7.0. Java 11 will be unsupported for all Streams for Apache Kafka components, including clients, in the future.

Streams for Apache Kafka supports Java 17. Use Java 17 when developing new applications. Plan to migrate any applications that currently use Java 11 to 17.

Identity replication policy is a feature used with MirrorMaker 2 to override the automatic renaming of remote topics. Instead of prepending the name with the source cluster’s name, the topic retains its original name. This setting is particularly useful for active/passive backups and data migration scenarios.

To implement an identity replication policy, you must specify a replication policy class (replication.policy.class) in the MirrorMaker 2 configuration. Previously, you could specify the io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy class included with the Streams for Apache Kafka mirror-maker-2-extensions component. However, this component is now deprecated and will be removed in the future. Therefore, it is recommended to update your implementation to use Kafka’s own replication policy class (org.apache.kafka.connect.mirror.IdentityReplicationPolicy).

See Using Streams for Apache Kafka with MirrorMaker 2.

Kafka MirrorMaker replicates data between two or more active Kafka clusters, within or across data centers. Kafka MirrorMaker 1 was deprecated in Kafka 3.0.0 and will be removed in Kafka 4.0.0. MirrorMaker 2 will be the only version available. MirrorMaker 2 is based on the Kafka Connect framework, connectors managing the transfer of data between clusters.

As a result, MirrorMaker 1 has also been deprecated in Streams for Apache Kafka as well. If you are using MirrorMaker 1 (referred to as just MirrorMaker in the Streams for Apache Kafka documentation), use MirrorMaker 2 with the IdentityReplicationPolicy class. MirrorMaker 2 renames topics replicated to a target cluster. IdentityReplicationPolicy configuration overrides the automatic renaming. Use it to produce the same active/passive unidirectional replication as MirrorMaker 1.

See Using Streams for Apache Kafka with MirrorMaker 2.

The following Kafka Bridge span attributes are deprecated with replacements shown where applicable:

Kafka Bridge uses OpenTelemetry for distributed tracing. The changes are inline with changes to OpenTelemetry semantic conventions. The attributes will be removed in a future release of the Kafka Bridge

There are known incompatibility issues when using RHEL 7 with Kafka 3.7. As a result, RHEL 7 is no longer supported. The issues arise due to an outdated GCC (GNU Compiler Collection) version on RHEL 7, which is incompatible with the version of the RocksDB JNI library (org.rocksdb:rocksdbjni:7.9.2) required by Kafka 3.7.

RocksDB JNI version 7.9.2 requires a newer version of the GCC and the associated libstdc++ library than what is available on RHEL 7. Snappy compression and Kafka Streams, which depend on RocksDB, will not function correctly on RHEL 7 due to these outdated libraries.

When running Streams for Apache Kafka in FIPS mode with JMX authentication enabled, clients may fail authentication. To work around this issue, do not enable JMX authentication while running in FIPS mode. We are investigating the issue and working to resolve it in a future release.

The following platforms are tested for Streams for Apache Kafka 2.7 running with Kafka on the version of Red Hat Enterprise Linux (RHEL) stated.

Platforms are tested with Open JDK 11 and 17, though Java 11 is deprecated in Streams for Apache Kafka 2.7.0. The IBM JDK is supported but not regularly tested against during each release. Oracle JDK 11 is not supported.

To check which versions of RHEL are approved by the National Institute of Standards and Technology (NIST), see the Cryptographic Module Validation Program on the NIST website.

Only client libraries built by Red Hat are supported for Streams for Apache Kafka. Currently, Streams for Apache Kafka only provides a Java client library.

Clients are tested with Open JDK 11 and 17.

In Streams for Apache Kafka, only the following components released directly from the Apache Software Foundation are supported:

See also, Chapter 10, Supported integration with Red Hat products.

Streams for Apache Kafka has been tested with block storage and is compatible with the XFS and ext4 file systems, both of which are commonly used with Kafka. File storage options, such as NFS, are not compatible.

Streams for Apache Kafka supports OAuth 2.0 token-based authorization through Red Hat build of Keycloak Authorization Services, providing centralized management of security policies and permissions.

Revised on 2024-07-08 11:23:30 UTC