Chapter 7. Feature gates
Streams for Apache Kafka operators use feature gates to enable or disable specific features and functions. Enabling a feature gate alters the behavior of the associated operator, introducing the corresponding feature to your Streams for Apache Kafka deployment.
The purpose of feature gates is to facilitate the trial and testing of a feature before it is fully adopted. The state (enabled or disabled) of a feature gate may vary by default, depending on its maturity level.
As a feature gate graduates and reaches General Availability (GA), it transitions to an enabled state by default and becomes a permanent part of the Streams for Apache Kafka deployment. A feature gate at the GA stage cannot be disabled.
7.1. Graduated feature gates (GA)
Graduated feature gates have reached General Availability (GA) and are permanently enabled features.
7.1.1. ControlPlaneListener feature gate
The ControlPlaneListener
feature gate separates listeners for data replication and coordination:
- Connections between the Kafka controller and brokers use an internal control plane listener on port 9090.
- Replication of data between brokers, as well as internal connections from Streams for Apache Kafka operators, Cruise Control, or the Kafka Exporter use a replication listener on port 9091.
With the ControlPlaneListener
feature gate permanently enabled, direct upgrades or downgrades between Streams for Apache Kafka 1.7 and earlier and Streams for Apache Kafka 2.3 and newer are not possible. You must first upgrade or downgrade through one of the Streams for Apache Kafka versions in-between, disable the ControlPlaneListener
feature gate, and then downgrade or upgrade (with the feature gate enabled) to the target version.
7.1.2. ServiceAccountPatching feature gate
The ServiceAccountPatching
feature gate ensures that the Cluster Operator always reconciles service accounts and updates them when needed. For example, when you change service account labels or annotations using the template
property of a custom resource, the operator automatically updates them on the existing service account resources.
7.1.3. UseStrimziPodSets feature gate
The UseStrimziPodSets
feature gate introduced the StrimziPodSet
custom resource for managing Kafka and ZooKeeper pods, replacing the use of OpenShift StatefulSet
resources.
With the UseStrimziPodSets
feature gate permanently enabled, direct downgrades from Streams for Apache Kafka 2.5 and newer to Streams for Apache Kafka 2.0 or earlier are not possible. You must first downgrade through one of the Streams for Apache Kafka versions in-between, disable the UseStrimziPodSets
feature gate, and then downgrade to Streams for Apache Kafka 2.0 or earlier.
7.1.4. StableConnectIdentities feature gate
The StableConnectIdentities
feature gate introduced the StrimziPodSet
custom resource for managing Kafka Connect and Kafka MirrorMaker 2 pods, replacing the use of OpenShift Deployment
resources.
StrimziPodSet
resources give the pods stable names and stable addresses, which do not change during rolling upgrades, replacing the use of OpenShift Deployment
resources.
With the StableConnectIdentities
feature gate permanently enabled, direct downgrades from Streams for Apache Kafka 2.7 and newer to Streams for Apache Kafka 2.3 or earlier are not possible. You must first downgrade through one of the Streams for Apache Kafka versions in-between, disable the StableConnectIdentities
feature gate, and then downgrade to Streams for Apache Kafka 2.3 or earlier.
7.2. Stable feature gates (Beta)
Stable feature gates have reached a beta level of maturity, and are generally enabled by default for all users. Stable feature gates are production-ready, but they can still be disabled.
7.2.1. UseKRaft feature gate
The UseKRaft
feature gate has a default state of enabled.
The UseKRaft
feature gate deploys a Kafka cluster in KRaft (Kafka Raft metadata) mode without ZooKeeper. ZooKeeper and KRaft are mechanisms used to manage metadata and coordinate operations in Kafka clusters. KRaft mode eliminates the need for an external coordination service like ZooKeeper. In KRaft mode, Kafka nodes take on the roles of brokers, controllers, or both. They collectively manage the metadata, which is replicated across partitions. Controllers are responsible for coordinating operations and maintaining the cluster’s state.
Using the UseKRaft
feature gate requires the KafkaNodePools
feature gate to be enabled as well. To deploy a Kafka cluster in KRaft mode, you must use the KafkaNodePool
resources. For more details and examples, see Section 6.3.1, “Deploying a Kafka cluster with node pools”. The Kafka
custom resource using KRaft mode must also have the annotation strimzi.io/kraft="enabled"
.
Currently, the KRaft mode in Streams for Apache Kafka has the following major limitations:
-
Only the Unidirectional Topic Operator is supported in KRaft mode. The Bidirectional Topic Operator is not supported and when the
UnidirectionalTopicOperator
feature gate is disabled, thespec.entityOperator.topicOperator
property must be removed from theKafka
custom resource. -
JBOD storage is not supported. The
type: jbod
storage can be used, but the JBOD array can contain only one disk. - Scaling of KRaft controller-only nodes up or down is not supported.
- Unregistering Kafka nodes removed from the Kafka cluster.
Disabling the UseKRaft feature gate
To disable the UseKRaft
feature gate, specify -UseKRaft
in the STRIMZI_FEATURE_GATES
environment variable in the Cluster Operator configuration.
7.2.2. KafkaNodePools feature gate
The KafkaNodePools
feature gate has a default state of enabled.
The KafkaNodePools
feature gate introduces a new KafkaNodePool
custom resource that enables the configuration of different pools of Apache Kafka nodes.
A node pool refers to a distinct group of Kafka nodes within a Kafka cluster. Each pool has its own unique configuration, which includes mandatory settings such as the number of replicas, storage configuration, and a list of assigned roles. You can assign the controller role, broker role, or both roles to all nodes in the pool in the .spec.roles
field. When used with a ZooKeeper-based Apache Kafka cluster, it must be set to the broker
role. When used with the UseKRaft
feature gate, it can be set to broker
, controller
, or both.
In addition, a node pool can have its own configuration of resource requests and limits, Java JVM options, and resource templates. Configuration options not set in the KafkaNodePool
resource are inherited from the Kafka
custom resource.
The KafkaNodePool
resources use a strimzi.io/cluster
label to indicate to which Kafka cluster they belong. The label must be set to the name of the Kafka
custom resource.
Examples of the KafkaNodePool
resources can be found in the example configuration files provided by Streams for Apache Kafka.
Disabling the KafkaNodePools feature gate
To disable the KafkaNodePools
feature gate, specify -KafkaNodePools
in the STRIMZI_FEATURE_GATES
environment variable in the Cluster Operator configuration. The Kafka
custom resource using the node pools must also have the annotation strimzi.io/node-pools: enabled
.
Downgrading from KafkaNodePools
If your cluster already uses KafkaNodePool
custom resources, and you wish to downgrade to an older version of Streams for Apache Kafka that does not support them or with the KafkaNodePools
feature gate disabled, you must first migrate from KafkaNodePool
custom resources to managing Kafka nodes using only Kafka
custom resources.
7.2.3. UnidirectionalTopicOperator feature gate
The UnidirectionalTopicOperator
feature gate has a default state of enabled.
The UnidirectionalTopicOperator
feature gate introduces a unidirectional topic management mode for creating Kafka topics using the KafkaTopic
resource. Unidirectional mode is compatible with using KRaft for cluster management. With unidirectional mode, you create Kafka topics using the KafkaTopic
resource, which are then managed by the Topic Operator. Any configuration changes to a topic outside the KafkaTopic
resource are reverted. For more information on topic management, see Section 10.1, “Topic management modes”.
Disabling the UnidirectionalTopicOperator
feature gate
To disable the UnidirectionalTopicOperator
feature gate, specify -UnidirectionalTopicOperator
in the STRIMZI_FEATURE_GATES
environment variable in the Cluster Operator configuration.
7.3. Early access feature gates (Alpha)
Early access feature gates have not yet reached the beta stage, and are disabled by default. An early access feature gate provides an opportunity for assessment before its functionality is permanently incorporated into Streams for Apache Kafka.
Currently, there are no feature gates in alpha stage.
7.4. Enabling feature gates
To modify a feature gate’s default state, use the STRIMZI_FEATURE_GATES
environment variable in the operator’s configuration. You can modify multiple feature gates using this single environment variable. Specify a comma-separated list of feature gate names and prefixes. A +
prefix enables the feature gate and a -
prefix disables it.
Example feature gate configuration that enables FeatureGate1
and disables FeatureGate2
env: - name: STRIMZI_FEATURE_GATES value: +FeatureGate1,-FeatureGate2
7.5. Feature gate releases
Feature gates have three stages of maturity:
- Alpha — typically disabled by default
- Beta — typically enabled by default
- General Availability (GA) — typically always enabled
Alpha stage features might be experimental or unstable, subject to change, or not sufficiently tested for production use. Beta stage features are well tested and their functionality is not likely to change. GA stage features are stable and should not change in the future. Alpha and beta stage features are removed if they do not prove to be useful.
-
The
ControlPlaneListener
feature gate moved to GA stage in Streams for Apache Kafka 2.3. It is now permanently enabled and cannot be disabled. -
The
ServiceAccountPatching
feature gate moved to GA stage in Streams for Apache Kafka 2.3. It is now permanently enabled and cannot be disabled. -
The
UseStrimziPodSets
feature gate moved to GA stage in Streams for Apache Kafka 2.5 and the support for StatefulSets is completely removed. It is now permanently enabled and cannot be disabled. -
The
StableConnectIdentities
feature gate moved to GA stage in Streams for Apache Kafka 2.7. It is now permanently enabled and cannot be disabled. -
The
UseKRaft
feature gate is in beta stage and is enabled by default. -
The
KafkaNodePools
feature gate is in beta stage and is enabled by default. -
The
UnidirectionalTopicOperator
feature gate is in beta stage and is enabled by default.
Feature gates might be removed when they reach GA. This means that the feature was incorporated into the Streams for Apache Kafka core features and can no longer be disabled.
Feature gate | Alpha | Beta | GA |
---|---|---|---|
| 1.8 | 2.0 | 2.3 |
| 1.8 | 2.0 | 2.3 |
| 2.1 | 2.3 | 2.5 |
| 2.2 | 2.7 | - |
| 2.4 | 2.6 | 2.7 |
| 2.5 | 2.7 | - |
| 2.5 | 2.7 | - |
If a feature gate is enabled, you may need to disable it before upgrading or downgrading from a specific Streams for Apache Kafka version (or first upgrade / downgrade to a version of Streams for Apache Kafka where it can be disabled). The following table shows which feature gates you need to disable when upgrading or downgrading Streams for Apache Kafka versions.
Disable Feature gate | Upgrading from Streams for Apache Kafka version | Downgrading to Streams for Apache Kafka version |
---|---|---|
| 1.7 and earlier | 1.7 and earlier |
| - | 2.0 and earlier |
| - | 2.3 and earlier |