Chapter 4. Enhancements

The AMQ Streams 2.5.x release supports Kafka 3.5.0. Upgrading to the 2.5.2 patch release incorporates the updates and improvements from Kafka 3.5.2.

For an overview of the enhancements introduced with Kafka 3.5.x, refer to the Kafka 3.5.0, Kafka 3.5.1, and Kafka 3.5.2 Release Notes.

The UseStrimziPodSets feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled.

StrimziPodSet resources are now used to manage pods instead of StatefulSet resources. This means that AMQ Streams handles the creation and management of pods instead of OpenShift, providing more control over the functionality.

See UseStrimziPodSets feature gate and Feature gate releases.

To deploy a Kafka cluster in KRaft mode, you must now enable the UseStrimziPodSets and KafkaNodePools feature gates. KRaft mode is supported only by using KafkaNodePool resources to manage the configuration of Kafka nodes.

For more information, see the following:

KeycloakRBACAuthorizer, the Red Hat Single Sign-On authorizer provided with AMQ Streams, has been replaced with the KeycloakAuthorizer. The new authorizer is compatible with using AMQ Streams with ZooKeeper cluster management or in KRaft mode. As with the previous authorizer, to be able to use the Red Hat Single Sign-On REST endpoints for Authorization Services provided by Red Hat Single Sign-On, you configure KeycloakAuthorizer on the Kafka broker. KeycloakRBACAuthorizer can still be used when using AMQ Streams with ZooKeeper cluster management, but you should migrate to the new authorizer.

You can now use additional configuration to manage OAuth 2.0 grants from the authorization server.

If you are using Red Hat Single Sign-On for OAuth 2.0 authorization, you can add the following properties to the authorization configuration of your Kafka brokers:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
    authorization:
      type: keycloak
      tokenEndpointUri: <https://<auth_server_-_address>/auth/realms/external/protocol/openid-connect/token>
      clientId: kafka
      # ...
      grantsMaxIdleSeconds: 300
      grantsGcPeriodSeconds: 300
      grantsAlwaysLatest: false
    #...

See Configuring OAuth 2.0 authorization support.

To use OAuth 2.0 authentication in a Kafka cluster, you specify listener configuration in the Kafka custom resource with the authentication method oauth. When configuring the listener properties, it is now possible to use a JsonPath query to extract a username from the authorization server being used. You can use a JsonPath query to specify username extraction options in your listener for the userNameClaim and fallbackUserNameClaim properties. This allows you to extract a username from a token by accessing a specific value within a nested data structure. For example, you might have a username that is contained within a user info data structure within a JSON token data structure.

The following example shows how JsonPath queries are used with the properties when configuring token validation using an introspection endpoint.

- name: external
  port: 9094
  type: loadbalancer
  tls: true
  authentication:
    type: oauth
    validIssuerUri: <https://<auth-server-address>/auth/realms/external>
    introspectionEndpointUri: <https://<auth-server-address>/auth/realms/external/protocol/openid-connect/token/introspect>
    clientId: kafka-broker
    clientSecret:
      secretName: my-cluster-oauth
      key: clientSecret
    userNameClaim: "['user.info'].['user.id']" 

    maxSecondsWithoutReauthentication: 3600
    fallbackUserNameClaim: "['client.info'].['client.id']" 

    fallbackUserNamePrefix: client-account-
    # ...

See Configuring OAuth 2.0 support for Kafka brokers.

Support for Kafka Exporter deployment configuration introduces new properties to exclude specified topics and consumer groups from the metrics extracted from Kafka brokers.

You can use the following properties in the Kafka Exporter specification:

In the following example configuration, the two properties exclude topics and consumer groups that start with the prefix excluded-.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  # ...
  kafkaExporter:
    image: my-registry.io/my-org/my-exporter-cluster:latest
    groupRegex: ".*"
    topicRegex: ".*"
    groupExcludeRegex: "^excluded-.*"
    topicExcludeRegex: "^excluded-.*"
# ...

See KafkaExporterSpec schema reference.

The latest release of the Kafka Bridge introduces the following changes:

See Using the AMQ Streams Kafka Bridge.