Chapter 2. Enhancements

For an overview of the enhancements introduced with Kafka 3.3.0 and 3.3.1, refer to the Kafka 3.3.0 and Kafka 3.3.1 Release Notes.

The status of the KafkaConnector custom resource now shows NotReady if the connector or any associated tasks are reporting as FAILED. Previously, the custom resource would show READY even when the connector or a task had failed.

The ControlPlaneListener feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled.

With ControlPlaneListener enabled, the connections between the Kafka controller and brokers use an internal control plane listener on port 9090.

See ControlPlaneListener feature gate.

The ServiceAccountPatching feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled.

With ServiceAccountPatching enabled, the Cluster Operator always reconciles service accounts and updates them when needed. For example, when you change service account labels or annotations using the template property of a custom resource, the operator automatically updates them on the existing service account resources.

See ServiceAccountPatching feature gate.

The UseStrimziPodSets feature gate moves to a beta level of maturity, and is now enabled by default. This means StrimziPodSets are used by default instead of StatefulSets.

The feature gate controls a resource for managing pods called StrimziPodSet. AMQ Streams handles the creation and management of pods instead of OpenShift. Using StrimziPodSets instead of StatefulSets provides more control over the functionality.

See UseStrimziPodSets feature gate and Feature gate releases.

Rack awareness for Kafka Bridge pods is now supported. Use the KafkaBridge custom resource to configure rack awareness. You can configure a Kafka Bridge pod to be aware of the rack in which it runs. A rack can represent an availability zone, data center, or an actual rack in your data center.

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaBridge
# ...
spec:
  # ...
  rack:
    topologyKey: topology.kubernetes.io/zone
  # ...

See Rack schema reference.

Security context defines constraints on pods and containers. OpenShift uses built-in security context constraints (SCCs) to control permissions. SCCs are the settings and strategies that control the security features a pod has access to. You can also create and manage your own SCCs.

The optional STRIMZI_POD_SECURITY_PROVIDER_CLASS environment variable for the Cluster Operator provides security context configuration for pods and containers.

See Applying security context to AMQ Streams pods and containers.

After the Cluster Operator restarts a Kafka pod in an OpenShift cluster, it emits an OpenShift event into the pod’s namespace explaining why the pod restarted. For help in understanding cluster behavior, you can check the reason for a restart event from the command line. When checking restart events from the command line, you can also specify a reason or other field-selector options to filter the events returned.

The following example returns restart events that were triggered due to an error.

oc -n kafka get events --field-selector reportingController=strimzi.io/cluster-operator,reason=PodForceRestartOnError

See Finding information on Kafka restarts.

A new User Operator environment variable called STRIMZI_KAFKA_ADMIN_CLIENT_CONFIGURATION can pass additional configuration to the Kafka Admin client. The Kafka Admin client helps manage brokers and topics. You can now tune the Kafka Admin client without rebuilding the User Operator. For example, you can use it to pass SASL configuration or tune timeouts.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: strimzi-user-operator
  labels:
    app: strimzi
spec:
  # ...
  template:
    # ...
    spec:
      # ...
      containers:
        - name: strimzi-user-operator
          # ...
          env:
            - name: STRIMZI_KAFKA_ADMIN_CLIENT_CONFIGURATION
              value: |
                default.api.timeout.ms=120000
                request.timeout.ms=60000

See Deploying the standalone User Operator.

New Cruise Control configuration options allow you to specify overrides that set the network capacity and CPU limits for each Kafka broker. You can use these options when brokers are running on nodes with heterogeneous network or CPU resources.

Override capacity limits can be set for the following broker resources:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  # ...
  cruiseControl:
    # ...
    brokerCapacity:
      cpu: "1"
      inboundNetwork: 10000KiB/s
      outboundNetwork: 10000KiB/s
      overrides:
      - brokers: [0]
        cpu: "2.755"
        inboundNetwork: 20000KiB/s
        outboundNetwork: 20000KiB/s
      - brokers: [1, 2]
        cpu: 3000m
        inboundNetwork: 30000KiB/s
        outboundNetwork: 30000KiB/s

See Capacity overrides.

You can now configure Kafka clients to use the OAuth password grants mechanism for interaction with Kafka brokers.

security.protocol=SASL_SSL
sasl.mechanism=OAUTHBEARER
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
  oauth.token.endpoint.uri="<token_endpoint_url>" \
  oauth.client.id="<client_id>" \ 1
  oauth.client.secret="<client_secret>" \ 2
  oauth.password.grant.username="<username>" \ 3
  oauth.password.grant.password="<password>" \ 4
  oauth.scope="<scope>" \
  oauth.audience="<audience>" ;
  # ...

See Configuring Kafka Java clients to use OAuth 2.0.

You can now collect metrics specific to oauth authentication and opa or keycloak authorization. You do this by setting the enableMetrics property to true in the listener configuration of the Kafka resource. For example, set enableMetrics to true in spec.kafka.listeners.authentication and spec.kafka.authorization. Similarly, you can enable metrics for oauth authentication in the KafkaBridge, KafkaConnect, KafkaMirrorMaker, and KafkaMirrorMaker2 custom resources.

See Introducing metrics.