Release Notes for AMQ Streams 2.4 on RHEL

AMQ Streams now supports Apache Kafka version 3.4.0.

AMQ Streams uses Kafka 3.4.0. Only Kafka distributions built by Red Hat are supported.

For upgrade instructions, see AMQ Streams and Kafka upgrades.

Refer to the Kafka 3.4.0 Release Notes for additional information.

For more information on supported versions, see the AMQ Streams Component Details.

Kafka 3.4.0 uses ZooKeeper version 3.6.3, which is the same version as Kafka 3.3.x. We recommend that you perform a rolling update to use the new binaries.

AMQ Streams 2.4 is enabled to run on IBM Z and LinuxONE s390x architecture.

IBM Z and LinuxONE is tested for AMQ Streams running with Kafka and Open JDK 11, on Red Hat Enterprise Linux 8 and later.

AMQ Streams 2.4 is enabled to run on IBM Power ppc64le architecture.

IBM Power is tested for AMQ Streams running with Kafka and Open JDK 11, on Red Hat Enterprise Linux 8 and later.

For an overview of the enhancements introduced with Kafka 3.4.0, refer to the Kafka 3.4.0 Release Notes.

You can now use configuration to control HTTP requests to an OAuth 2.0 authorization server.

If you are creating a listener for your Kafka brokers that uses OAuth 2.0 authentication or authorization, you can add the following properties to the listener configuration:

sasl.enabled.mechanisms=OAUTHBEARER
listeners=CLIENT://0.0.0.0:9092
listener.security.protocol.map=CLIENT:SASL_PLAINTEXT
listener.name.client.sasl.enabled.mechanisms=OAUTHBEARER
sasl.mechanism.inter.broker.protocol=OAUTHBEARER
inter.broker.listener.name=CLIENT
listener.name.client.oauthbearer.sasl.server.callback.handler.class=io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler
listener.name.client.oauthbearer.sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required ;
listener.name.client.oauthbearer.sasl.login.callback.handler.class=io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler
listener.name.client.oauthbearer.sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
  # ...
  oauth.token.endpoint.uri="https://AUTH-SERVER-ADDRESS/auth/realms/REALM-NAME/protocol/openid-connect/token" \
  oauth.custom.claim.check="@.custom == 'custom-value'" \
  oauth.scope="SCOPE" \
  oauth.check.audience="true" \
  oauth.audience="AUDIENCE" \
  oauth.valid.issuer.uri="https://https://AUTH-SERVER-ADDRESS/auth/REALM-NAME" \
  oauth.client.id="kafka-broker" \
  oauth.client.secret="kafka-broker-secret" \
  oauth.connect.timeout.seconds=60 \
  oauth.read.timeout.seconds=60 \
  oauth.http.retries=2 \
  oauth.http.retry.pause.millis=300 \
  oauth.groups.claim="$.groups" \
  oauth.groups.claim.delimiter="," ;

See the following:

OpenTelemetry for distributed tracing is available as a technology preview. You can use OpenTelemetry with a specified tracing system. OpenTelemetry is replacing OpenTracing for distributed tracing. Support for OpenTracing is deprecated.

By Default, OpenTelemetry uses the OTLP (OpenTelemetry Protocol) exporter for tracing. AMQ Streams with OpenTelemetry is distributed for use with the Jaeger exporter, but you can specify other tracing systems supported by OpenTelemetry. AMQ Streams plans to migrate to using OpenTelemetry with the OTLP exporter by default, and is phasing out support for the Jaeger exporter.

See Introducing distributed tracing.

Apache Kafka is in the process of phasing out the need for ZooKeeper. You can now try deploying a Kafka cluster in KRaft (Kafka Raft metadata) mode without ZooKeeper as a technology preview.

Currently, the KRaft mode in AMQ Streams has the following major limitations:

See Running Kafka in KRaft mode.

Use the Kafka Static Quota plugin to set throughput and storage limits on brokers in your Kafka cluster. You can set a byte-rate threshold and storage quotas to put limits on the clients interacting with your brokers.

client.quota.callback.class= io.strimzi.kafka.quotas.StaticQuotaCallback
client.quota.callback.static.produce= 1000000
client.quota.callback.static.fetch= 1000000
client.quota.callback.static.storage.soft= 400000000000
client.quota.callback.static.storage.hard= 500000000000
client.quota.callback.static.storage.check-interval= 5

See Setting limits on brokers using the Kafka Static Quota plugin.

Support for Java 8 was deprecated in AMQ Streams 2.0.0. Support for Java 8 is removed in AMQ Streams 2.4.0. This applies to all AMQ Streams components, including clients.

AMQ Streams supports Java 11. Use Java 11 when developing new applications. Plan to migrate any applications that currently use Java 8 to Java 11.

If you want to continue using Java 8 for the time being, AMQ Streams 2.2 provides Long Term Support (LTS). For information on the LTS terms and dates, see the AMQ Streams LTS Support Policy.

Support for OpenTracing is deprecated.

The Jaeger clients are now retired and the OpenTracing project archived. As such, we cannot guarantee their support for future Kafka versions. We are introducing a new tracing implementation based on the OpenTelemetry project.

Identity replication policy is used with MirrorMaker 2 to override the automatic renaming of remote topics. Instead of prepending the name with the name of the source cluster, the topic retains its original name. This optional setting is useful for active/passive backups and data migration.

To implement an identity replication policy, you specify a replication policy class (replication.policy.class) in the MirrorMaker 2 configuration. The AMQ Streams mirror-maker-2-extensions component, which includes the io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy class, is now deprecated and will be removed in the future. Therefore, it is recommended to update your implementation to use Kafka’s own replication policy class (org.apache.kafka.connect.mirror.IdentityReplicationPolicy).

Kafka MirrorMaker replicates data between two or more active Kafka clusters, within or across data centers. Kafka MirrorMaker 1 was deprecated in Kafka 3.0.0 and will be removed in Kafka 4.0.0. MirrorMaker 2 will be the only version available. MirrorMaker 2 is based on the Kafka Connect framework, connectors managing the transfer of data between clusters.

As a result, MirrorMaker 1 has also been deprecated in AMQ Streams as well. If you are using MirrorMaker 1 (referred to as just MirrorMaker in the AMQ Streams documentation), use MirrorMaker 2 with the IdentityReplicationPolicy class. MirrorMaker 2 renames topics replicated to a target cluster. IdentityReplicationPolicy configuration overrides the automatic renaming. Use it to produce the same active/passive unidirectional replication as MirrorMaker 1.

See Using AMQ Streams with MirrorMaker 2.

When running AMQ Streams in FIPS mode with JMX authentication enabled, clients may fail authentication. To work around this issue, do not enable JMX authentication while running in FIPS mode. We are investigating the issue and working to resolve it in a future release.