10.4. Configuring Kafka topics
Use the properties of the KafkaTopic resource to configure Kafka topics. Changes made to topic configuration in the KafkaTopic are propagated to Kafka.
You can use oc apply to create or modify topics, and oc delete to delete existing topics.
For example:
-
oc apply -f <topic_config_file> -
oc delete KafkaTopic <topic_name>
To be able to delete topics, delete.topic.enable must be set to true (default) in the spec.kafka.config of the Kafka resource.
This procedure shows how to create a topic with 10 partitions and 2 replicas.
The procedure is the same for the unidirectional and bidirectional modes of topic management.
Before you begin
The KafkaTopic resource does not allow the following changes:
-
Renaming the topic defined in
spec.topicName. A mismatch betweenspec.topicNameandstatus.topicNamewill be detected. -
Decreasing the number of partitions using
spec.partitions(not supported by Kafka). -
Modifying the number of replicas specified in
spec.replicas.
Increasing spec.partitions for topics with keys will alter the partitioning of records, which can cause issues, especially when the topic uses semantic partitioning.
Prerequisites
- A running Kafka cluster configured with a Kafka broker listener using mTLS authentication and TLS encryption.
- A running Topic Operator (typically deployed with the Entity Operator).
-
For deleting a topic,
delete.topic.enable=true(default) in thespec.kafka.configof theKafkaresource.
Procedure
Configure the
KafkaTopicresource.Example Kafka topic configuration
apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: name: my-topic-1 labels: strimzi.io/cluster: my-cluster spec: partitions: 10 replicas: 2提示When modifying a topic, you can get the current version of the resource using
oc get kafkatopic my-topic-1 -o yaml.Create the
KafkaTopicresource in OpenShift.oc apply -f <topic_config_file>Wait for the ready status of the topic to change to
True:oc get kafkatopics -o wide -w -n <namespace>Kafka topic status
NAME CLUSTER PARTITIONS REPLICATION FACTOR READY my-topic-1 my-cluster 10 3 True my-topic-2 my-cluster 10 3 my-topic-3 my-cluster 10 3 TrueTopic creation is successful when the
READYoutput showsTrue.If the
READYcolumn stays blank, get more details on the status from the resource YAML or from the Topic Operator logs.Status messages provide details on the reason for the current status.
oc get kafkatopics my-topic-2 -o yamlDetails on a topic with a
NotReadystatus# ... status: conditions: - lastTransitionTime: "2022-06-13T10:14:43.351550Z" message: Number of partitions cannot be decreased reason: PartitionDecreaseException status: "True" type: NotReadyIn this example, the reason the topic is not ready is because the original number of partitions was reduced in the
KafkaTopicconfiguration. Kafka does not support this.After resetting the topic configuration, the status shows the topic is ready.
oc get kafkatopics my-topic-2 -o wide -w -n <namespace>Status update of the topic
NAME CLUSTER PARTITIONS REPLICATION FACTOR READY my-topic-2 my-cluster 10 3 TrueFetching the details shows no messages
oc get kafkatopics my-topic-2 -o yamlDetails on a topic with a
READYstatus# ... status: conditions: - lastTransitionTime: '2022-06-13T10:15:03.761084Z' status: 'True' type: Ready