Chapter 7. Enabling AMQ Streams feature gates

The ControlPlaneListener feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled. With ControlPlaneListener enabled, the connections between the Kafka controller and brokers use an internal control plane listener on port 9090. Replication of data between brokers, as well as internal connections from AMQ Streams operators, Cruise Control, or the Kafka Exporter use the replication listener on port 9091.

The ServiceAccountPatching feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled. With ServiceAccountPatching enabled, the Cluster Operator always reconciles service accounts and updates them when needed. For example, when you change service account labels or annotations using the template property of a custom resource, the operator automatically updates them on the existing service account resources.

The UseStrimziPodSets feature gate has moved to GA, which means it is now permanently enabled and cannot be disabled. Support for StatefulSets has been removed and AMQ Streams is now always using StrimziPodSets to manage Kafka and ZooKeeper pods.

The UseKRaft feature gate has a default state of disabled.

The UseKRaft feature gate deploys the Kafka cluster in the KRaft (Kafka Raft metadata) mode without ZooKeeper. ZooKeeper and KRaft are mechanisms used to manage metadata and coordinate operations in Kafka clusters. KRaft mode eliminates the need for an external coordination service like ZooKeeper. In KRaft mode, Kafka nodes take on the roles of brokers, controllers, or both. They collectively manage the metadata, which is replicated across partitions. Controllers are responsible for coordinating operations and maintaining the cluster’s state.

This feature gate is currently intended only for development and testing.

Enabling the UseKRaft feature gate requires the KafkaNodePools feature gate to be enabled as well. To deploy a Kafka cluster in KRaft mode, you must use the KafkaNodePool resources. For more details and examples, see Section 6.3.2, “(Preview) Deploying Kafka node pools”. The Kafka custom resource using KRaft mode must also have the annotation strimzi.io/kraft: enabled.

When the UseKRaft feature gate is enabled and such annotation is set, the Kafka cluster is deployed without ZooKeeper. The .spec.zookeeper properties in the Kafka custom resource are ignored, but still need to be present. The UseKRaft feature gate provides an API that configures Kafka cluster nodes and their roles. The API is still in development and is expected to change before the KRaft mode is production-ready.

Currently, the KRaft mode in AMQ Streams has the following major limitations:

The StableConnectIdentities feature gate has a default state of enabled.

The StableConnectIdentities feature gate uses StrimziPodSet resources to manage Kafka Connect and Kafka MirrorMaker 2 pods instead of using OpenShift Deployment resources. StrimziPodSets give the pods stable names and stable addresses, which do not change during rolling upgrades. This helps to minimize the number of rebalances of connector tasks.

The KafkaNodePools feature gate has a default state of disabled.

The KafkaNodePools feature gate introduces a new KafkaNodePool custom resource that enables the configuration of different pools of Apache Kafka nodes.

A node pool refers to a distinct group of Kafka nodes within a Kafka cluster. Each pool has its own unique configuration, which includes mandatory settings such as the number of replicas, storage configuration, and a list of assigned roles. You can assign the controller role, broker role, or both roles to all nodes in the pool in the .spec.roles field. When used with a ZooKeeper-based Apache Kafka cluster, it must be set to the broker role. When used with the UseKRaft feature gate, it can be set to broker, controller, or both.

In addition, a node pool can have its own configuration of resource requests and limits, Java JVM options, and resource templates. Configuration options not set in the KafkaNodePool resource are inherited from the Kafka custom resource.

The KafkaNodePool resources use a strimzi.io/cluster label to indicate to which Kafka cluster they belong. The label must be set to the name of the Kafka custom resource.

Examples of the KafkaNodePool resources can be found in the example configuration files provided by AMQ Streams.

The UnidirectionalTopicOperator feature gate has a default state of disabled.

The UnidirectionalTopicOperator feature gate introduces a unidirectional topic management mode for creating Kafka topics using the KafkaTopic resource. Unidirectional mode is compatible with using KRaft for cluster management. With unidirectional mode, you create Kafka topics using the KafkaTopic resource, which are then managed by the Topic Operator. Any configuration changes to a topic outside the KafkaTopic resource are reverted. For more information on topic management, see Section 9.1, “Topic management modes”.

Feature gates have three stages of maturity:

Alpha stage features might be experimental or unstable, subject to change, or not sufficiently tested for production use. Beta stage features are well tested and their functionality is not likely to change. GA stage features are stable and should not change in the future. Alpha and beta stage features are removed if they do not prove to be useful.

If a feature gate is enabled, you may need to disable it before upgrading or downgrading from a specific AMQ Streams version (or first upgrade / downgrade to a version of AMQ Streams where it can be disabled). The following table shows which feature gates you need to disable when upgrading or downgrading AMQ Streams versions.