이 콘텐츠는 선택한 언어로 제공되지 않습니다.
Chapter 1. Kafka tuning overview
Fine-tuning the performance of your Kafka deployment involves optimizing various configuration properties according to your specific requirements. This section provides an introduction to common configuration options available for Kafka brokers, producers, and consumers.
While a minimum set of configurations is necessary for Kafka to function, Kafka properties allow for extensive adjustments. Through configuration properties, you can enhance latency, throughput, and overall efficiency, ensuring that your Kafka deployment meets the demands of your applications.
For effective tuning, take a methodical approach. Begin by analyzing relevant metrics to identify potential bottlenecks or areas for improvement. Adjust configuration parameters iteratively, monitoring the impact on performance metrics, and then refine your settings accordingly.
For more information about Apache Kafka configuration properties, see the Apache Kafka documentation.
The guidance provided here offers a starting point for tuning your Kafka deployment. Finding the optimal configuration depends on factors such as workload, infrastructure, and performance objectives.
1.1. Mapping properties and values 링크 복사링크가 클립보드에 복사되었습니다!
How you specify configuration properties depends on the type of deployment. If you deployed Streams for Apache Kafka on OCP, you can use the Kafka resource to add configuration for Kafka brokers through the config property. With Streams for Apache Kafka on RHEL, you add the configuration to a properties file as environment variables.
When you add config properties to custom resources, you use a colon (':') to map the property and value.
Example configuration in a custom resource
num.partitions:1
num.partitions:1
When you add the properties as environment variables, you use an equal sign ('=') to map the property and value.
Example configuration as an environment variable
num.partitions=1
num.partitions=1
Some examples in this guide may show resource configuration specifically for Streams for Apache Kafka on OpenShift. However, the properties presented are equally applicable as environment variables when using Streams for Apache Kafka on RHEL.
1.2. Tools that help with tuning 링크 복사링크가 클립보드에 복사되었습니다!
The following tools help with Kafka tuning:
- Cruise Control generates optimization proposals that you can use to assess and implement a cluster rebalance
- Strimzi Quotas plugin sets limits on brokers
- Rack configuration spreads broker partitions across racks and allows consumers to fetch data from the nearest replica