第 9 章 Configuring a deployment
Configure and manage a Streams for Apache Kafka deployment to your precise needs using Streams for Apache Kafka custom resources. Streams for Apache Kafka provides example custom resources with each release, allowing you to configure and create instances of supported Kafka components. Fine-tune your deployment by configuring custom resources to include additional features according to your specific requirements. For specific areas of configuration, namely metrics, logging, and external configuration for Kafka Connect connectors, you can also use ConfigMap resources. By using a ConfigMap resource to incorporate configuration, you centralize maintenance. You can also use configuration providers to load configuration from external sources, which we recommend for supplying the credentials for Kafka Connect connector configuration.
Use custom resources to configure and create instances of the following components:
- Kafka clusters
- Kafka Connect clusters
- Kafka MirrorMaker
- Kafka Bridge
- Cruise Control
You can also use custom resource configuration to manage your instances or modify your deployment to introduce additional features. This might include configuration that supports the following:
- Specifying node pools
- Securing client access to Kafka brokers
- Accessing Kafka brokers from outside the cluster
- Creating topics
- Creating users (clients)
- Controlling feature gates
- Changing logging frequency
- Allocating resource limits and requests
- Introducing features, such as Streams for Apache Kafka Drain Cleaner, Cruise Control, or distributed tracing.
The Streams for Apache Kafka Custom Resource API Reference describes the properties you can use in your configuration.
Labels applied to a custom resource are also applied to the OpenShift resources making up its cluster. This provides a convenient mechanism for resources to be labeled as required.
Applying changes to a custom resource configuration file
You add configuration to a custom resource using spec properties. After adding the configuration, you can use oc to apply the changes to a custom resource configuration file:
oc apply -f <kafka_configuration_file>
9.1. Using example configuration files 复制链接链接已复制到粘贴板!
Further enhance your deployment by incorporating additional supported configuration. Example configuration files are provided with the downloadable release artifacts from the Streams for Apache Kafka software downloads page.
The example files include only the essential properties and values for custom resources by default. You can download and apply the examples using the oc command-line tool. The examples can serve as a starting point when building your own Kafka component configuration for deployment.
If you installed Streams for Apache Kafka using the Operator, you can still download the example files and use them to upload configuration.
The release artifacts include an examples directory that contains the configuration examples.
Example configuration files provided with Streams for Apache Kafka
examples
├── user
├── topic
├── security
│ ├── tls-auth
│ ├── scram-sha-512-auth
│ └── keycloak-authorization
├── mirror-maker
├── metrics
├── kafka
│ └── nodepools
├── cruise-control
├── connect
└── bridge
- 1
KafkaUsercustom resource configuration, which is managed by the User Operator.- 2
KafkaTopiccustom resource configuration, which is managed by Topic Operator.- 3
- Authentication and authorization configuration for Kafka components. Includes example configuration for TLS and SCRAM-SHA-512 authentication. The Red Hat Single Sign-On example includes
Kafkacustom resource configuration and a Red Hat Single Sign-On realm specification. You can use the example to try Red Hat Single Sign-On authorization services. There is also an example with enabledoauthauthentication andkeycloakauthorization metrics. - 4
Kafkacustom resource configuration for a deployment of Mirror Maker. Includes example configuration for replication policy and synchronization frequency.- 5
- Metrics configuration, including Prometheus installation and Grafana dashboard files.
- 6
Kafkacustom resource configuration for a deployment of Kafka. Includes example configuration for an ephemeral or persistent single or multi-node deployment.- 7
KafkaNodePoolconfiguration for Kafka nodes in a Kafka cluster. Includes example configuration for nodes in clusters that use KRaft (Kafka Raft metadata) mode or ZooKeeper.- 8
Kafkacustom resource with a deployment configuration for Cruise Control. IncludesKafkaRebalancecustom resources to generate optimization proposals from Cruise Control, with example configurations to use the default or user optimization goals.- 9
KafkaConnectandKafkaConnectorcustom resource configuration for a deployment of Kafka Connect. Includes example configurations for a single or multi-node deployment.- 10
KafkaBridgecustom resource configuration for a deployment of Kafka Bridge.