Chapter 5. Using the Topic Operator
5.1. Topic Operator usage recommendations
When working with topics, be consistent and always operate on either KafkaTopic resources or topics directly. Avoid routinely switching between both methods for a given topic.
Use topic names that reflect the nature of the topic, and remember that names cannot be changed later.
If creating a topic in Kafka, use a name that is a valid OpenShift resource name, otherwise the Topic Operator will need to create the corresponding KafkaTopic with a name that conforms to the OpenShift rules.
Recommendations for identifiers and names in OpenShift are outlined in Identifiers and Names in OpenShift community article.
Kafka topic naming conventions
Kafka and OpenShift impose their own validation rules for the naming of topics in Kafka and KafkaTopic.metadata.name respectively. There are valid names for each which are invalid in the other.
Using the spec.topicName property, it is possible to create a valid topic in Kafka with a name that would be invalid for the KafkaTopic in OpenShift.
The spec.topicName property inherits Kafka naming validation rules:
- The name must not be longer than 249 characters.
-
Valid characters for Kafka topics are ASCII alphanumerics,
.,_, and-. -
The name cannot be
.or.., though.can be used in a name, such asexampleTopic.or.exampleTopic.
spec.topicName must not be changed.
For example:
kind: KafkaTopic metadata: name: topic-name-1 spec: topicName: topicName-1 # Upper case is invalid in OpenShift # ...
cannot be changed to
kind: KafkaTopic metadata: name: topic-name-1 spec: topicName: name-2 # ...
Some Kafka client applications, such as Kafka Streams, can create topics in Kafka programmatically. If those topics have names that are invalid OpenShift resource names, the Topic Operator gives them valid names based on the Kafka names. Invalid characters are replaced and a hash is appended to the name.
5.2. Creating a topic
This procedure describes how to create a Kafka topic using a KafkaTopic OpenShift resource.
Prerequisites
- A running Kafka cluster.
- A running Topic Operator (typically deployed with the Entity Operator).
Procedure
Prepare a file containing the
KafkaTopicto be createdAn example
KafkaTopicapiVersion: kafka.strimzi.io/v1beta1 kind: KafkaTopic metadata: name: orders labels: strimzi.io/cluster: my-cluster spec: partitions: 10 replicas: 2NoteIt is recommended that the topic name given is a valid OpenShift resource name, as it is then not necessary to set the
KafkaTopic.spec.topicNameproperty. TheKafkaTopic.spec.topicNamecannot be changed after creation.NoteThe
KafkaTopic.spec.partitionscannot be decreased.Create the
KafkaTopicresource in OpenShift.This can be done using
oc apply:oc apply -f your-file
Additional resources
-
For more information about the schema for
KafkaTopics, seeKafkaTopicschema reference. - For more information about deploying a Kafka cluster using the Cluster Operator, see Section 2.3, “Cluster Operator”.
- For more information about deploying the Topic Operator using the Cluster Operator, see Section 2.9.2, “Deploying the Topic Operator using the Cluster Operator”.
- For more information about deploying the standalone Topic Operator, see Section 4.2.6, “Deploying the standalone Topic Operator”.
5.3. Changing a topic
This procedure describes how to change the configuration of an existing Kafka topic by using a KafkaTopic OpenShift resource.
Prerequisites
- A running Kafka cluster.
- A running Topic Operator (typically deployed with the Entity Operator).
-
An existing
KafkaTopicto be changed.
Procedure
Prepare a file containing the desired
KafkaTopicAn example
KafkaTopicapiVersion: kafka.strimzi.io/v1beta1 kind: KafkaTopic metadata: name: orders labels: strimzi.io/cluster: my-cluster spec: partitions: 16 replicas: 2TipYou can get the current version of the resource using
oc get kafkatopic orders -o yaml.NoteChanging topic names using the
KafkaTopic.spec.topicNamevariable and decreasing partition size using theKafkaTopic.spec.partitionsvariable is not supported by Kafka.CautionIncreasing
spec.partitionsfor topics with keys will change how records are partitioned, which can be particularly problematic when the topic uses semantic partitioning.Update the
KafkaTopicresource in OpenShift.This can be done using
oc apply:oc apply -f your-file
Additional resources
-
For more information about the schema for
KafkaTopics, seeKafkaTopicschema reference. - For more information about deploying a Kafka cluster, see Section 2.3, “Cluster Operator”.
- For more information about deploying the Topic Operator using the Cluster Operator, see Section 2.9.2, “Deploying the Topic Operator using the Cluster Operator”.
- For more information about creating a topic using the Topic Operator, see Section 5.2, “Creating a topic”.
5.4. Deleting a topic
This procedure describes how to delete a Kafka topic using a KafkaTopic OpenShift resource.
Prerequisites
- A running Kafka cluster.
- A running Topic Operator (typically deployed with the Entity Operator).
-
An existing
KafkaTopicto be deleted. -
delete.topic.enable=true(default)
The delete.topic.enable property must be set to true in Kafka.spec.kafka.config. Otherwise, the steps outlined here will delete the KafkaTopic resource, but the Kafka topic and its data will remain. After reconciliation by the Topic Operator, the custom resource is then recreated.
Procedure
Delete the
KafkaTopicresource in OpenShift.This can be done using
oc delete:oc delete kafkatopic your-topic-name
Additional resources
- For more information about deploying a Kafka cluster using the Cluster Operator, see Section 2.3, “Cluster Operator”.
- For more information about deploying the Topic Operator using the Cluster Operator, see Section 2.9.2, “Deploying the Topic Operator using the Cluster Operator”.
- For more information about creating a topic using the Topic Operator, see Section 5.2, “Creating a topic”.
