Ce contenu n'est pas disponible dans la langue sélectionnée.
Chapter 120. KafkaMirrorMakerConsumerSpec schema reference
Used in: KafkaMirrorMakerSpec
Full list of KafkaMirrorMakerConsumerSpec schema properties
Configures a MirrorMaker consumer.
120.1. numStreams
Use the consumer.numStreams property to configure the number of streams for the consumer.
You can increase the throughput in mirroring topics by increasing the number of consumer threads. Consumer threads belong to the consumer group specified for Kafka MirrorMaker. Topic partitions are assigned across the consumer threads, which consume messages in parallel.
120.2. offsetCommitInterval
Use the consumer.offsetCommitInterval property to configure an offset auto-commit interval for the consumer.
You can specify the regular time interval at which an offset is committed after Kafka MirrorMaker has consumed data from the source Kafka cluster. The time interval is set in milliseconds, with a default value of 60,000.
120.3. config
Use the consumer.config properties to configure Kafka options for the consumer as keys.
The values can be one of the following JSON types:
- String
- Number
- Boolean
Exceptions
You can specify and configure the options listed in the Apache Kafka configuration documentation for consumers.
However, Streams for Apache Kafka takes care of configuring and managing options related to the following, which cannot be changed:
- Kafka cluster bootstrap address
- Security (encryption, authentication, and authorization)
- Consumer group identifier
- Interceptors
Properties with the following prefixes cannot be set:
-
bootstrap.servers -
group.id -
interceptor.classes -
sasl. -
security. -
ssl.
If the config property contains an option that cannot be changed, it is disregarded, and a warning message is logged to the Cluster Operator log file. All other supported options are forwarded to MirrorMaker, including the following exceptions to the options configured by Streams for Apache Kafka:
-
Any
sslconfiguration for supported TLS versions and cipher suites
The Cluster Operator does not validate keys or values in the config object provided. If an invalid configuration is provided, the MirrorMaker cluster might not start or might become unstable. In this case, fix the configuration so that the Cluster Operator can roll out the new configuration to all MirrorMaker nodes.
120.4. groupId
Use the consumer.groupId property to configure a consumer group identifier for the consumer.
Kafka MirrorMaker uses a Kafka consumer to consume messages, behaving like any other Kafka consumer client. Messages consumed from the source Kafka cluster are mirrored to a target Kafka cluster. A group identifier is required, as the consumer needs to be part of a consumer group for the assignment of partitions.
120.5. KafkaMirrorMakerConsumerSpec schema properties
| Property | Property type | Description |
|---|---|---|
| numStreams | integer | Specifies the number of consumer stream threads to create. |
| offsetCommitInterval | integer | Specifies the offset auto-commit interval in ms. Default value is 60000. |
| bootstrapServers | string | A list of host:port pairs for establishing the initial connection to the Kafka cluster. |
| groupId | string | A unique string that identifies the consumer group this consumer belongs to. |
| authentication |
| Authentication configuration for connecting to the cluster. |
| config | map | The MirrorMaker consumer config. Properties with the following prefixes cannot be set: ssl., bootstrap.servers, group.id, sasl., security., interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols). |
| tls | TLS configuration for connecting MirrorMaker to the cluster. |
