Chapter 3. New features and enhancements

To upgrade to Streams for Apache Kafka 3.0 or later, you must first migrate your Kafka clusters to KRaft mode.

Dynamic controller quorums are not currently supported in Streams for Apache Kafka on OpenShift, as the related Kafka feature was only recently resolved. For more information, see KAFKA-16538.

To maintain compatibility with existing KRaft-based deployments, Streams for Apache Kafka on OpenShift uses only static controller quorums. This applies to new and existing clusters, regardless of whether you’re deploying, migrating, or maintaining them.

Support for dynamic controller quorums is expected in a future Streams for Apache Kafka release.

As a technology preview, the Metrics Reporter exposes Kafka metrics directly over HTTP in a Prometheus-compatible format.

You can now monitor partition rebalance progress using status information in the KafkaRebalance resource. Progress details are stored in a ConfigMap, accessible with oc get configmaps, and include:

The ServerSideApplyPhase1 feature gate (disabled by default) adds Server Side Apply (SSA) support for ConfigMap, Ingress, PVC, Service, and ServiceAccount resources.

You can now configure Kafka Connect so that connectors are automatically mounted as Kubernetes Image Volumes. By defining the connector plugins using the .spec.plugins property of the KafkaConnect custom resource, Strimzi automatically mounts them into the Kafka Connect deployment.

The strimzi.io/node-pools and strimzi.io/kraft annotations are not required anymore and will be ignored if set.

The following properties are now configurable for Kafka resources:

The STRIMZI_IGNORED_USERS_PATTERN environmental variable and a regex expression allows configuration of users for which any existing ACLs, Quotas and SCRAM-SHa credentials will be ignored. This option is useful when you want to configure the User Operator to ignore users that are managed through another mechanism.

The KafkaClientAuthenticationCustom schema supports custom client authentication. Custom client authentication allows you to use any type of Kafka-supported authentication mechanism.

The state of custom resources can now be monitored using kube-state-metrics (KSM). Example configuration is provided in examples/metrics/kube-state-metrics.

A distinction is now made between changes to the cluster-wide and broker-specific properties applied dynamically. This related to properties configured in the Kafka and KafkaNodePool resources.

Support for configuring a pod disruption budget (PDB) has been extended to EntityOperator, CruiseControl and KafkaExporter resources through the PodDisruptionBudgetTemplate schema.

The Kafka Admin API is now used to to get the list of registered brokers. The KafkaStatus schema registeredNodeIds property is no longer required and is deprecated.

Refine connector restart behavior with the includeTasks and onlyFailed parameters, which both default to false.

Additional OAuth configuration options have been added for OAuth 2 authentication on the listener and the client. On the listener clientGrantType has been added. On the client grantType has been added.

Grant type is used when requesting a token from the authorization server.

JsonTemplateLayout is now supported when configuring log appenders in Log4j2.

Kafka Connect now uses KubernetesSecretConfigProvider to load PEM-format truststore and keystore certificates directly from Kubernetes secrets.

Kafka Bridge now includes a validation_errors field in the error response JSON when a schema validation error occurs. This field is part of the Error OpenAPI component and is omitted for other error types.

A new Authorization filter is available. The filter applies Kafka-style topic authorization to client requests, starting with metadata.

A new SASL Inspection filter is available. The filter extracts the authenticated principal from a successful SASL exchange and makes it available to other filters in the chain.

Supported mechanisms:

A new Azure Key Vault Key Management Service (KMS) implementation is available for record encryption. This allows users to store and manage encryption keys in Azure Key Vault.

Filters can now map topic IDs to topic names using the new topicNames lookup, improving compatibility with topic-ID–based traffic.

Virtual clusters can now define a subjectBuilder to control how client identities are derived from mTLS certificates.

Record encryption now supports Azure Key Vault for key storage and retrieval.

The proxy now assigns a session ID that links downstream and upstream connections, improving traceability in logs.

The operator can now automatically detect the plain listener bootstrap address from a Strimzi Kafka resource.

New metrics report the number of active downstream/upstream connections handled by the proxy.

New metrics report how long the proxy applies back pressure on client connections.

The proxy now rejects configurations that contain unexpected or misspelled fields.

The operator now reports more detailed load balancer status information for virtual Kafka clusters.

As a technology preview, the console adds configuration and display options for Kafka Connect and its connectors. Connect clusters can be associated with one or more Kafka clusters, allowing users to view connected clusters, connectors, and their configurations.

The console now supports automatic configuration of KafkaUser custom resources when connecting to Kafka clusters secured with mutual TLS authentication.

OpenID Connect (OIDC) scopes are now configurable. The groups scope is no longer a fixed requirement for the identity provider.

Security rule configurations now support the use of regular expressions in resourceNames definitions.

The console now displays the actual platform and version (for example, OpenShift 4.18.5) on the home page. An About modal provides application version details to assist with support.

When a user signs out, the console automatically terminates the session with the identity provider if an end_session_endpoint is available through OIDC discovery.

Console UI controls are now automatically enabled or disabled based on user authorization configurations.

The console supports light, dark, and automatic color themes. With the auto option, the console inherits the host operating system’s color preference.

The console now supports topic metrics emitted by the Streams for Apache Kafka Metrics Reporter.

All dates in the console are displayed in ISO format, using the user’s local time zone except where UTC times are explicitly shown in the topic message browser.

When reconciliation is paused and the Kafka version in .status is unavailable, the console now displays the desired Kafka version.

If a user’s OAuth2 or OIDC token expires and cannot be refreshed, the console now automatically signs the user out and redirects them to the identity provider.