Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 17. Scaling clusters by adding or removing brokers

Scaling Kafka clusters by adding brokers can increase the performance and reliability of the cluster. Adding more brokers increases available resources, allowing the cluster to handle larger workloads and process more messages. It can also improve fault tolerance by providing more replicas and backups. Conversely, removing underutilized brokers can reduce resource consumption and improve efficiency. Scaling must be done carefully to avoid disruption or data loss. By redistributing partitions across all brokers in the cluster, the resource utilization of each broker is reduced, which can increase the overall throughput of the cluster.

Note

To increase the throughput of a Kafka topic, you can increase the number of partitions for that topic. This allows the load of the topic to be shared between different brokers in the cluster. However, if every broker is constrained by a specific resource (such as I/O), adding more partitions will not increase the throughput. In this case, you need to add more brokers to the cluster.

Adjusting the Kafka.spec.kafka.replicas configuration affects the number of brokers in the cluster that act as replicas. The actual replication factor for topics is determined by settings for the default.replication.factor and min.insync.replicas, and the number of available brokers. For example, a replication factor of 3 means that each partition of a topic is replicated across three brokers, ensuring fault tolerance in the event of a broker failure.

Example replica configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    replicas: 3
    # ...
  config:
      # ...
      default.replication.factor: 3
      min.insync.replicas: 2
 # ...

When adding brokers through the Kafka configuration, node IDs start at 0 (zero) and the Cluster Operator assigns the next lowest ID to a new node. The broker removal process starts from the broker pod with the highest ID in the cluster.

If you are managing nodes in the cluster using the the preview of the node pools feature, you adjust the KafkaNodePool.spec.replicas configuration to change the number of nodes in the node pool. Additionally, when scaling existing clusters with node pools, you can assign node IDs for the scaling operations.

When you add add or remove brokers, Kafka does not automatically reassign partitions. The best way to do this is using Cruise Control. You can use Cruise Control’s add-brokers and remove-brokers modes when scaling a cluster up or down.

  • Use the add-brokers mode after scaling up a Kafka cluster to move partition replicas from existing brokers to the newly added brokers.
  • Use the remove-brokers mode before scaling down a Kafka cluster to move partition replicas off the brokers that are going to be removed.

17.1. Skipping checks on scale-down operations
Copy link

By default, AMQ Streams performs a check to ensure that there are no partition replicas on a broker before initiating a scale-down operation on a Kafka cluster. If replicas are found, cluster operations are blocked to prevent potential data loss. To resume cluster operations, no replicas must be left on the broker before trying to scale it down again.

However, there may be scenarios where you want to bypass this blocking mechanism. Disabling the check might be necessary on busy clusters, for example, because new topics keep generating replicas for the broker. This situation can indefinitely block cluster operations, even when brokers are nearly empty. Overriding the blocking mechanism in this way has an impact: the presence of topics on the broker being scaled down will likely cause a reconciliation failure for the Kafka cluster.

You can bypass the blocking mechanism by annotating the Kafka resource for the Kafka cluster. Annotate the resource by setting the strimzi.io/skip-broker-scaledown-check annotation to true:

Adding the annotation to skip checks on scale-down operations

oc annotate Kafka my-kafka-cluster strimzi.io/skip-broker-scaledown-check="true"

This annotation instructs AMQ Streams to skip the scale-down check. Replace my-kafka-cluster with the name of your specific Kafka resource.

To restore the check for scale-down operations, remove the annotation:

Removing the annotation to skip checks on scale-down operations

oc annotate Kafka my-kafka-cluster strimzi.io/skip-broker-scaledown-check-

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top