Red Hat Summit Red Hat SummitSuporteConsoleDesenvolvedoresComeçar um testeContato

Este conteúdo não está disponível no idioma selecionado.

Chapter 2. Using Kafka in KRaft mode

KRaft (Kafka Raft metadata) mode replaces Kafka’s dependency on ZooKeeper for cluster management. KRaft mode simplifies the deployment and management of Kafka clusters by bringing metadata management and coordination of clusters into Kafka.

Kafka in KRaft mode is designed to offer enhanced reliability, scalability, and throughput. Metadata operations become more efficient as they are directly integrated. And by removing the need to maintain a ZooKeeper cluster, there’s also a reduction in the operational and security overhead.

To deploy a Kafka cluster in KRaft mode, you must use Kafka and KafkaNodePool custom resources. The Kafka resource using KRaft mode must also have the annotations strimzi.io/kraft: enabled and strimzi.io/node-pools: enabled. For more details and examples, see Section 8.2.1, “Deploying a Kafka cluster in KRaft mode”.

Through node pool configuration using KafkaNodePool resources, nodes are assigned the role of broker, controller, or both:

  • Controller nodes operate in the control plane to manage cluster metadata and the state of the cluster using a Raft-based consensus protocol.
  • Broker nodes operate in the data plane to manage the streaming of messages, receiving and storing data in topic partitions.
  • Dual-role nodes fulfill the responsibilities of controllers and brokers.

Controllers use a metadata log, stored as a single-partition topic (__cluster_metadata) on every node, which records the state of the cluster. When requests are made to change the cluster configuration, an active (lead) controller manages updates to the metadata log, and follower controllers replicate these updates. The metadata log stores information on brokers, replicas, topics, and partitions, including the state of in-sync replicas and partition leadership. Kafka uses this metadata to coordinate changes and manage the cluster effectively.

Broker nodes act as observers, storing the metadata log passively to stay up-to-date with the cluster’s state. Each node fetches updates to the log independently. If you are using JBOD storage, you can change the volume that stores the metadata log.

Note

The KRaft metadata version used in the Kafka cluster must be supported by the Kafka version in use. Both versions are managed through the Kafka resource configuration. For more information, see Section 10.2, “Configuring Kafka”.

In the following example, a Kafka cluster comprises a quorum of controller and broker nodes for fault tolerance and high availability.

Figure 2.1. Example cluster with separate broker and controller nodes

KRaft quorums for broker and controller

In a typical production environment, use dedicated broker and controller nodes. However, you might want to use nodes in a dual-role configuration for development or testing.

You can use a combination of nodes that combine roles with nodes that perform a single role. In the following example, three nodes perform a dual role and two nodes act only as brokers.

Figure 2.2. Example cluster with dual-role nodes and dedicated broker nodes

KRaft cluster with nodes that combine roles

2.1. KRaft limitations
Copiar o link

KRaft limitations primarily relate to controller scaling, which impacts cluster operations.

2.1.1. Controller scaling
Copiar o link

KRaft mode supports two types of controller quorums:

  • Static controller quorums
    In this mode, the number of controllers is fixed, and scaling requires downtime.
  • Dynamic controller quorums
    This mode enables dynamic scaling of controllers without downtime. New controllers join as observers, replicate the metadata log, and eventually become eligible to join the quorum. If a controller being removed is the active controller, it will step down from the quorum only after the new quorum is confirmed.

Scaling is useful not only for adding or removing controllers, but supports the following operations:

  • Renaming a node pool, which involves adding a new node pool with the desired name and deleting the old one.
  • Changing non-JBOD storage, which requires creating a new node pool with the updated storage configuration and removing the old one.

Dynamic controller quorums provide the flexibility to make these operations significantly easier to perform.

2.1.2. Limitations with static controller quorums
Copiar o link

Migration between static and dynamic controller quorums is not currently supported by Apache Kafka, though it is expected to be introduced in a future release. As a result, Streams for Apache Kafka uses static controller quorums for all deployments, including new installations. All pre-existing KRaft-based Apache Kafka clusters that use static controller quorums must continue using them. To ensure compatibility with existing KRaft-based clusters, Streams for Apache Kafka continues to use static controller quorums as well.

This limitation means dynamic scaling of controller quorums cannot be used to support the following:

  • Adding or removing node pools with controller roles
  • Adding the controller role to an existing node pool
  • Removing the controller role from an existing node pool
  • Scaling a node pool with the controller role
  • Renaming a node pool with the controller role

Static controller quorums also limit operations that require scaling. For example, changing the storage type for a node pool with a controller role is not possible because it involves scaling the controller quorum. For non-JBOD storage, creating a new node pool with the desired storage type, adding it to the cluster, and removing the old one would require scaling operations, which are not supported. In some cases, workarounds are possible. For instance, when modifying node pool roles to combine controller and broker functions, you can add broker roles to controller nodes instead of adding controller roles to broker nodes to avoid controller scaling. However, this approach would require reassigning more data, which may temporarily affect cluster performance.

Once migration is possible, Streams for Apache Kafka plans to assess introducing support for dynamic quorums.

Red Hat logoGithubredditYoutubeTwitter

Aprender

Experimente, compre e venda

Comunidades

Sobre a documentação da Red Hat

Ajudamos os usuários da Red Hat a inovar e atingir seus objetivos com nossos produtos e serviços com conteúdo em que podem confiar. Explore nossas atualizações recentes.

Tornando o open source mais inclusivo

A Red Hat está comprometida em substituir a linguagem problemática em nosso código, documentação e propriedades da web. Para mais detalhes veja o Blog da Red Hat.

Sobre a Red Hat

Fornecemos soluções robustas que facilitam o trabalho das empresas em plataformas e ambientes, desde o data center principal até a borda da rede.

Theme

© 2026 Red HatVoltar ao topo