Chapter 7. Feature gates
Streams for Apache Kafka operators use feature gates to enable or disable specific features and functions. Enabling a feature gate alters the behavior of the associated operator, introducing the corresponding feature to your Streams for Apache Kafka deployment.
The purpose of feature gates is to facilitate the trial and testing of a feature before it is fully adopted. The state (enabled or disabled) of a feature gate may vary by default, depending on its maturity level.
As a feature gate graduates and reaches General Availability (GA), it transitions to an enabled state by default and becomes a permanent part of the Streams for Apache Kafka deployment. A feature gate at the GA stage cannot be disabled.
7.1. Graduated feature gates (GA) Copy linkLink copied to clipboard!
Graduated feature gates have reached General Availability (GA) and are permanently enabled features.
7.1.1. ControlPlaneListener feature gate Copy linkLink copied to clipboard!
The ControlPlaneListener feature gate separates listeners for data replication and coordination:
- Connections between the Kafka controller and brokers use an internal control plane listener on port 9090.
- Replication of data between brokers, as well as internal connections from Streams for Apache Kafka operators, Cruise Control, or the Kafka Exporter use a replication listener on port 9091.
With the ControlPlaneListener feature gate permanently enabled, direct upgrades or downgrades between Streams for Apache Kafka 1.7 and earlier and Streams for Apache Kafka 2.3 and newer are not possible. You must first upgrade or downgrade through one of the Streams for Apache Kafka versions in-between, disable the ControlPlaneListener feature gate, and then downgrade or upgrade (with the feature gate enabled) to the target version.
7.1.2. ServiceAccountPatching feature gate Copy linkLink copied to clipboard!
The ServiceAccountPatching feature gate ensures that the Cluster Operator always reconciles service accounts and updates them when needed. For example, when you change service account labels or annotations using the template property of a custom resource, the operator automatically updates them on the existing service account resources.
7.1.3. UseStrimziPodSets feature gate Copy linkLink copied to clipboard!
The UseStrimziPodSets feature gate introduced the StrimziPodSet custom resource for managing Kafka and ZooKeeper pods, replacing the use of OpenShift StatefulSet resources.
With the UseStrimziPodSets feature gate permanently enabled, direct downgrades from Streams for Apache Kafka 2.5 and newer to Streams for Apache Kafka 2.0 or earlier are not possible. You must first downgrade through one of the Streams for Apache Kafka versions in-between, disable the UseStrimziPodSets feature gate, and then downgrade to Streams for Apache Kafka 2.0 or earlier.
7.1.4. StableConnectIdentities feature gate Copy linkLink copied to clipboard!
The StableConnectIdentities feature gate introduced the StrimziPodSet custom resource for managing Kafka Connect and Kafka MirrorMaker 2 pods, replacing the use of OpenShift Deployment resources.
StrimziPodSet resources give the pods stable names and stable addresses, which do not change during rolling upgrades, replacing the use of OpenShift Deployment resources.
With the StableConnectIdentities feature gate permanently enabled, direct downgrades from Streams for Apache Kafka 2.7 and newer to Streams for Apache Kafka 2.3 or earlier are not possible. You must first downgrade through one of the Streams for Apache Kafka versions in-between, disable the StableConnectIdentities feature gate, and then downgrade to Streams for Apache Kafka 2.3 or earlier.
7.2. Stable feature gates (Beta) Copy linkLink copied to clipboard!
Stable feature gates have reached a beta level of maturity, and are generally enabled by default for all users. Stable feature gates are production-ready, but they can still be disabled.
7.2.1. UseKRaft feature gate Copy linkLink copied to clipboard!
The UseKRaft feature gate has a default state of enabled.
The UseKRaft feature gate deploys a Kafka cluster in KRaft (Kafka Raft metadata) mode without ZooKeeper. ZooKeeper and KRaft are mechanisms used to manage metadata and coordinate operations in Kafka clusters. KRaft mode eliminates the need for an external coordination service like ZooKeeper. In KRaft mode, Kafka nodes take on the roles of brokers, controllers, or both. They collectively manage the metadata, which is replicated across partitions. Controllers are responsible for coordinating operations and maintaining the cluster’s state.
Using the UseKRaft feature gate requires the KafkaNodePools feature gate to be enabled as well. To deploy a Kafka cluster in KRaft mode, you must use the KafkaNodePool resources. For more details and examples, see Section 6.3.1, “Deploying a Kafka cluster with node pools”. The Kafka custom resource using KRaft mode must also have the annotation strimzi.io/kraft="enabled".
Currently, the KRaft mode in Streams for Apache Kafka has the following major limitations:
-
Only the Unidirectional Topic Operator is supported in KRaft mode. The Bidirectional Topic Operator is not supported and when the
UnidirectionalTopicOperatorfeature gate is disabled, thespec.entityOperator.topicOperatorproperty must be removed from theKafkacustom resource. -
JBOD storage is not supported. The
type: jbodstorage can be used, but the JBOD array can contain only one disk. - Scaling of KRaft controller-only nodes up or down is not supported.
- Unregistering Kafka nodes removed from the Kafka cluster.
Disabling the UseKRaft feature gate
To disable the UseKRaft feature gate, specify -UseKRaft in the STRIMZI_FEATURE_GATES environment variable in the Cluster Operator configuration.
7.2.2. KafkaNodePools feature gate Copy linkLink copied to clipboard!
The KafkaNodePools feature gate has a default state of enabled.
The KafkaNodePools feature gate introduces a new KafkaNodePool custom resource that enables the configuration of different pools of Apache Kafka nodes.
A node pool refers to a distinct group of Kafka nodes within a Kafka cluster. Each pool has its own unique configuration, which includes mandatory settings such as the number of replicas, storage configuration, and a list of assigned roles. You can assign the controller role, broker role, or both roles to all nodes in the pool in the .spec.roles field. When used with a ZooKeeper-based Apache Kafka cluster, it must be set to the broker role. When used with the UseKRaft feature gate, it can be set to broker, controller, or both.
In addition, a node pool can have its own configuration of resource requests and limits, Java JVM options, and resource templates. Configuration options not set in the KafkaNodePool resource are inherited from the Kafka custom resource.
The KafkaNodePool resources use a strimzi.io/cluster label to indicate to which Kafka cluster they belong. The label must be set to the name of the Kafka custom resource.
Examples of the KafkaNodePool resources can be found in the example configuration files provided by Streams for Apache Kafka.
Disabling the KafkaNodePools feature gate
To disable the KafkaNodePools feature gate, specify -KafkaNodePools in the STRIMZI_FEATURE_GATES environment variable in the Cluster Operator configuration. The Kafka custom resource using the node pools must also have the annotation strimzi.io/node-pools: enabled.
Downgrading from KafkaNodePools
If your cluster already uses KafkaNodePool custom resources, and you wish to downgrade to an older version of Streams for Apache Kafka that does not support them or with the KafkaNodePools feature gate disabled, you must first migrate from KafkaNodePool custom resources to managing Kafka nodes using only Kafka custom resources.
7.2.3. UnidirectionalTopicOperator feature gate Copy linkLink copied to clipboard!
The UnidirectionalTopicOperator feature gate has a default state of enabled.
The UnidirectionalTopicOperator feature gate introduces a unidirectional topic management mode for creating Kafka topics using the KafkaTopic resource. Unidirectional mode is compatible with using KRaft for cluster management. With unidirectional mode, you create Kafka topics using the KafkaTopic resource, which are then managed by the Topic Operator. Any configuration changes to a topic outside the KafkaTopic resource are reverted. For more information on topic management, see Section 10.1, “Topic management modes”.
Disabling the UnidirectionalTopicOperator feature gate
To disable the UnidirectionalTopicOperator feature gate, specify -UnidirectionalTopicOperator in the STRIMZI_FEATURE_GATES environment variable in the Cluster Operator configuration.
7.3. Early access feature gates (Alpha) Copy linkLink copied to clipboard!
Early access feature gates have not yet reached the beta stage, and are disabled by default. An early access feature gate provides an opportunity for assessment before its functionality is permanently incorporated into Streams for Apache Kafka.
Currently, there are no feature gates in alpha stage.
7.4. Enabling feature gates Copy linkLink copied to clipboard!
To modify a feature gate’s default state, use the STRIMZI_FEATURE_GATES environment variable in the operator’s configuration. You can modify multiple feature gates using this single environment variable. Specify a comma-separated list of feature gate names and prefixes. A + prefix enables the feature gate and a - prefix disables it.
Example feature gate configuration that enables FeatureGate1 and disables FeatureGate2
env:
- name: STRIMZI_FEATURE_GATES
value: +FeatureGate1,-FeatureGate2
env:
- name: STRIMZI_FEATURE_GATES
value: +FeatureGate1,-FeatureGate2
7.5. Feature gate releases Copy linkLink copied to clipboard!
Feature gates have three stages of maturity:
- Alpha — typically disabled by default
- Beta — typically enabled by default
- General Availability (GA) — typically always enabled
Alpha stage features might be experimental or unstable, subject to change, or not sufficiently tested for production use. Beta stage features are well tested and their functionality is not likely to change. GA stage features are stable and should not change in the future. Alpha and beta stage features are removed if they do not prove to be useful.
-
The
ControlPlaneListenerfeature gate moved to GA stage in Streams for Apache Kafka 2.3. It is now permanently enabled and cannot be disabled. -
The
ServiceAccountPatchingfeature gate moved to GA stage in Streams for Apache Kafka 2.3. It is now permanently enabled and cannot be disabled. -
The
UseStrimziPodSetsfeature gate moved to GA stage in Streams for Apache Kafka 2.5 and the support for StatefulSets is completely removed. It is now permanently enabled and cannot be disabled. -
The
StableConnectIdentitiesfeature gate moved to GA stage in Streams for Apache Kafka 2.7. It is now permanently enabled and cannot be disabled. -
The
UseKRaftfeature gate is in beta stage and is enabled by default. -
The
KafkaNodePoolsfeature gate is in beta stage and is enabled by default. -
The
UnidirectionalTopicOperatorfeature gate is in beta stage and is enabled by default.
Feature gates might be removed when they reach GA. This means that the feature was incorporated into the Streams for Apache Kafka core features and can no longer be disabled.
| Feature gate | Alpha | Beta | GA |
|---|---|---|---|
|
| 1.8 | 2.0 | 2.3 |
|
| 1.8 | 2.0 | 2.3 |
|
| 2.1 | 2.3 | 2.5 |
|
| 2.2 | 2.7 | - |
|
| 2.4 | 2.6 | 2.7 |
|
| 2.5 | 2.7 | - |
|
| 2.5 | 2.7 | - |
If a feature gate is enabled, you may need to disable it before upgrading or downgrading from a specific Streams for Apache Kafka version (or first upgrade / downgrade to a version of Streams for Apache Kafka where it can be disabled). The following table shows which feature gates you need to disable when upgrading or downgrading Streams for Apache Kafka versions.
| Disable Feature gate | Upgrading from Streams for Apache Kafka version | Downgrading to Streams for Apache Kafka version |
|---|---|---|
|
| 1.7 and earlier | 1.7 and earlier |
|
| - | 2.0 and earlier |
|
| - | 2.3 and earlier |