Red Hat SummitChapter 3. Technology Previews
Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete; therefore, Red Hat does not recommend implementing any Technology Preview features in production environments. This Technology Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about support scope, see Technology Preview Features Support Scope.
3.1. Cluster rebalancing with Cruise Control
Cruise Control stays in Technology Preview in this release, with some new enhancements.
You can now deploy Cruise Control to your AMQ Streams cluster and use it to rebalance the Kafka cluster using optimization goals - predefined constraints on CPU, disk, and network load. In a balanced Kafka cluster, the workload is more evenly distributed across the broker pods.
Cruise Control is configured and deployed as part of a Kafka resource. Example YAML configuration files for Cruise Control are provided in examples/cruise-control/.
When Cruise Control is deployed, you can use KafkaRebalance custom resources to:
- Generate optimization proposals from multiple optimization goals
- Rebalance a Kafka cluster based on an optimization proposal
Other Cruise Control features are not currently supported, including anomaly detection, notifications, write-your-own goals, and changing the topic replication factor.
See Cruise Control for cluster rebalancing.
3.1.1. Enhancements to the Technology Preview
The following enhancements have been added to the initial Technology Preview of cluster rebalancing with Cruise Control.
Rebalance performance tuning
Five new performance tuning options allow you to control how cluster rebalances are executed and reduce their performance impact.
For each batch of partition reassignment commands that comprise a cluster rebalance, you can configure the following:
- Maximum concurrent partition movements per broker (the default is 5)
- Maximum concurrent intra-broker partition movements (the default is 2)
- Maximum concurrent leader movements (the default is 1000)
- Bandwidth in bytes-per-second to assign to partition reassignment (the default is no limit)
-
Replica movement strategy (the default is
BaseReplicaMovementStrategy)
Previously, AMQ Streams inherited these options from Cruise Control, so their default values could not be adjusted.
You can set performance tuning options for the Cruise Control server, individual rebalances, or both.
-
For the Cruise Control server, set options in the
Kafkacustom resource, underspec.cruiseControl.config. -
For a cluster rebalance, set options in the
specproperty of theKafkaRebalancecustom resource.
See Rebalance performance tuning overview.
Exclude topics from optimization proposals
You can now exclude one or more topics from an optimization proposal. Those topics are not included in the calculation of partition replica and partition leadership movements for the cluster rebalance.
To exclude topics, specify a regular expression matching the topic names in the KafkaRebalance custom resource, in the spec.excludedTopicsRegex property.
When viewing a generated optimization proposal, the excludedTopics property shows you the topics that were excluded.
See Rebalance performance tuning overview.
CPU capacity goal support
Rebalancing a Kafka cluster based on CPU capacity is now supported through the following configurations:
-
The
CpuCapacityGoaloptimization goal -
The
cpuUtilizationcapacity limit
The CPU capacity goal prevents the CPU utilization of each broker from exceeding a maximum percentage threshold. The default threshold is set as 100% of CPU capacity per broker.
To reduce the percentage threshold, configure the cpuUtilization capacity limit in the Kafka custom resource. Capacity limits apply to all brokers.
CPU capacity is preset as a hard goal in Cruise Control. Therefore, it is inherited from Cruise Control as a hard goal, unless you override the preset hard goals in the hard.goals property in Kafka.spec.cruiseControl.config.
Example configuration for CPU capacity goal in a KafkaRebalance custom resource
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaRebalance
metadata:
name: my-rebalance
labels:
strimzi.io/cluster: amq-streams-cluster
spec:
goals:
- CpuCapacityGoal
- DiskCapacityGoal
#...Example configuration for percentage CPU utilization capacity limit
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: amq-streams-cluster
spec:
# ...
cruiseControl:
# ...
brokerCapacity:
cpuUtilization: 85
disk: 100Gi
# ...
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}