Chapter 1. Deployment overview

CRDs require a one-time installation in a cluster to define the schemas used to instantiate and manage Streams for Apache Kafka-specific resources.

After a new custom resource type is added to your cluster by installing a CRD, you can create instances of the resource based on its specification.

Depending on the cluster setup, installation typically requires cluster admin privileges.

A CRD defines a new kind of resource, such as kind:Kafka, within an OpenShift cluster.

The Kubernetes API server allows custom resources to be created based on the kind and understands from the CRD how to validate and store the custom resource when it is added to the OpenShift cluster.

Each Streams for Apache Kafka-specific custom resource conforms to the schema defined by the CRD for the resource’s kind. The custom resources for Streams for Apache Kafka components have common configuration properties, which are defined under spec.

To understand the relationship between a CRD and a custom resource, let’s look at a sample of the CRD for a Kafka topic.

apiVersion: kafka.strimzi.io/v1beta2
kind: CustomResourceDefinition
metadata: 
  name: kafkatopics.kafka.strimzi.io
  labels:
    app: strimzi
spec: 
  group: kafka.strimzi.io
  versions:
    v1beta2
  scope: Namespaced
  names:
    # ...
    singular: kafkatopic
    plural: kafkatopics
    shortNames:
    - kt 
  additionalPrinterColumns: 
      # ...
  subresources:
    status: {} 
  validation: 
    openAPIV3Schema:
      properties:
        spec:
          type: object
          properties:
            partitions:
              type: integer
              minimum: 1
            replicas:
              type: integer
              minimum: 1
              maximum: 32767
      # ...

apiVersion: kafka.strimzi.io/v1beta2
kind: CustomResourceDefinition
metadata: 

  name: kafkatopics.kafka.strimzi.io
  labels:
    app: strimzi
spec: 

  group: kafka.strimzi.io
  versions:
    v1beta2
  scope: Namespaced
  names:
    # ...
    singular: kafkatopic
    plural: kafkatopics
    shortNames:
    - kt 

  additionalPrinterColumns: 

      # ...
  subresources:
    status: {} 

  validation: 

    openAPIV3Schema:
      properties:
        spec:
          type: object
          properties:
            partitions:
              type: integer
              minimum: 1
            replicas:
              type: integer
              minimum: 1
              maximum: 32767
      # ...

Here is a corresponding example of a KafkaTopic custom resource.

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic 
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: my-cluster 
spec: 
  partitions: 1
  replicas: 1
  config:
    retention.ms: 7200000
    segment.bytes: 1073741824
status:
  conditions: 
    lastTransitionTime: "2019-08-20T11:37:00.706Z"
    status: "True"
    type: Ready
  observedGeneration: 1
  / ...

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic 

metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: my-cluster 

spec: 

  partitions: 1
  replicas: 1
  config:
    retention.ms: 7200000
    segment.bytes: 1073741824
status:
  conditions: 

    lastTransitionTime: "2019-08-20T11:37:00.706Z"
    status: "True"
    type: Ready
  observedGeneration: 1
  / ...

Custom resources can be applied to a cluster through the platform CLI. When the custom resource is created, it uses the same validation as the built-in resources of the Kubernetes API.

After a KafkaTopic custom resource is created, the Topic Operator is notified and corresponding Kafka topics are created in Streams for Apache Kafka.

You can use oc commands to retrieve information and perform other operations on Streams for Apache Kafka custom resources. Use oc commands, such as get, describe, edit, or delete, to perform operations on resource types. For example, oc get kafkatopics retrieves a list of all Kafka topics and oc get kafkas retrieves all deployed Kafka clusters.

When referencing resource types, you can use both singular and plural names: oc get kafkas gets the same results as oc get kafka.

You can also use the short name of the resource. Learning short names can save you time when managing Streams for Apache Kafka. The short name for Kafka is k, so you can also run oc get k to list all Kafka clusters.

oc get k

NAME         READY   METADATA STATE   WARNINGS
my-cluster   True    KRaft

oc get k

NAME         READY   METADATA STATE   WARNINGS
my-cluster   True    KRaft

oc get strimzi

NAME                                                   PODS   READY PODS   CURRENT PODS   AGE
strimzipodset.core.strimzi.io/my-cluster-brokers       3      3            3              6h11m
strimzipodset.core.strimzi.io/my-cluster-controllers   3      3            3              6h11m

NAME                                         DESIRED REPLICAS   ROLES            NODEIDS
kafkanodepool.kafka.strimzi.io/brokers       3                  ["broker"]       [3,4,5]
kafkanodepool.kafka.strimzi.io/controllers   3                  ["controller"]   [0,1,2]

NAME                                READY   METADATA STATE   WARNINGS
kafka.kafka.strimzi.io/my-cluster   True    KRaft

NAME                                   PARTITIONS REPLICATION FACTOR
kafkatopic.kafka.strimzi.io/kafka-apps 3          3

NAME                                   AUTHENTICATION AUTHORIZATION
kafkauser.kafka.strimzi.io/my-user     tls            simple

oc get strimzi

NAME                                                   PODS   READY PODS   CURRENT PODS   AGE
strimzipodset.core.strimzi.io/my-cluster-brokers       3      3            3              6h11m
strimzipodset.core.strimzi.io/my-cluster-controllers   3      3            3              6h11m

NAME                                         DESIRED REPLICAS   ROLES            NODEIDS
kafkanodepool.kafka.strimzi.io/brokers       3                  ["broker"]       [3,4,5]
kafkanodepool.kafka.strimzi.io/controllers   3                  ["controller"]   [0,1,2]

NAME                                READY   METADATA STATE   WARNINGS
kafka.kafka.strimzi.io/my-cluster   True    KRaft

NAME                                   PARTITIONS REPLICATION FACTOR
kafkatopic.kafka.strimzi.io/kafka-apps 3          3

NAME                                   AUTHENTICATION AUTHORIZATION
kafkauser.kafka.strimzi.io/my-user     tls            simple

oc get strimzi -o name

strimzipodset.core.strimzi.io/my-cluster-brokers
strimzipodset.core.strimzi.io/my-cluster-controllers
kafkanodepool.kafka.strimzi.io/brokers
kafkanodepool.kafka.strimzi.io/controllers
kafka.kafka.strimzi.io/my-cluster
kafkatopic.kafka.strimzi.io/kafka-apps
kafkauser.kafka.strimzi.io/my-user

oc get strimzi -o name

strimzipodset.core.strimzi.io/my-cluster-brokers
strimzipodset.core.strimzi.io/my-cluster-controllers
kafkanodepool.kafka.strimzi.io/brokers
kafkanodepool.kafka.strimzi.io/controllers
kafka.kafka.strimzi.io/my-cluster
kafkatopic.kafka.strimzi.io/kafka-apps
kafkauser.kafka.strimzi.io/my-user

oc delete $(oc get strimzi -o name)

strimzipodset.core.strimzi.io "my-cluster-brokers" deleted
strimzipodset.core.strimzi.io "my-cluster-controllers" deleted
kafkanodepool.kafka.strimzi.io "brokers" deleted
kafkanodepool.kafka.strimzi.io "controllers" deleted
kafka.kafka.strimzi.io "my-cluster" deleted
kafkatopic.kafka.strimzi.io "kafka-apps" deleted
kafkauser.kafka.strimzi.io "my-user" deleted

oc delete $(oc get strimzi -o name)

strimzipodset.core.strimzi.io "my-cluster-brokers" deleted
strimzipodset.core.strimzi.io "my-cluster-controllers" deleted
kafkanodepool.kafka.strimzi.io "brokers" deleted
kafkanodepool.kafka.strimzi.io "controllers" deleted
kafka.kafka.strimzi.io "my-cluster" deleted
kafkatopic.kafka.strimzi.io "kafka-apps" deleted
kafkauser.kafka.strimzi.io "my-user" deleted

oc get kafka my-cluster -o=jsonpath='{.status.listeners[?(@.name=="tls")].bootstrapServers}{"\n"}'

my-cluster-kafka-bootstrap.myproject.svc:9093

oc get kafka my-cluster -o=jsonpath='{.status.listeners[?(@.name=="tls")].bootstrapServers}{"\n"}'

my-cluster-kafka-bootstrap.myproject.svc:9093

Status properties provide status information for certain custom resources.

The following table lists the custom resources that provide status information (when deployed) and the schemas that define the status properties.

For more information on the schemas, see the Streams for Apache Kafka Custom Resource API Reference.

The status property of a resource provides information on the state of the resource. The status.conditions and status.observedGeneration properties are common to all resources.

The status properties also provide resource-specific information. For example, KafkaStatus provides information on listener addresses, and the ID of the Kafka cluster.

KafkaStatus also provides information on the Kafka and Streams for Apache Kafka versions being used. You can check the values of operatorLastSuccessfulVersion and kafkaVersion to determine whether an upgrade of Streams for Apache Kafka or Kafka has completed

Streams for Apache Kafka creates and maintains the status of custom resources, periodically evaluating the current state of the custom resource and updating its status accordingly. When performing an update on a custom resource using oc edit, for example, its status is not editable. Moreover, changing the status would not affect the configuration of the Kafka cluster.

Here we see the status properties for a Kafka custom resource.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
spec:
  # ...
status:
  clusterId: XP9FP2P-RByvEy0W4cOEUA 
  conditions: 
    - lastTransitionTime: '2023-01-20T17:56:29.396588Z'
      status: 'True'
      type: Ready 
  kafkaMetadataState: KRaft 
  kafkaVersion: 4.1.0 
  kafkaNodePools: 
    - name: broker
    - name: controller
  listeners: 
    - addresses:
        - host: my-cluster-kafka-bootstrap.prm-project.svc
          port: 9092
      bootstrapServers: 'my-cluster-kafka-bootstrap.prm-project.svc:9092'
      name: plain
    - addresses:
        - host: my-cluster-kafka-bootstrap.prm-project.svc
          port: 9093
      bootstrapServers: 'my-cluster-kafka-bootstrap.prm-project.svc:9093'
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: tls
    - addresses:
        - host: >-
            2054284155.us-east-2.elb.amazonaws.com
          port: 9095
      bootstrapServers: >-
        2054284155.us-east-2.elb.amazonaws.com:9095
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: external3
    - addresses:
        - host: ip-10-0-172-202.us-east-2.compute.internal
          port: 31644
      bootstrapServers: 'ip-10-0-172-202.us-east-2.compute.internal:31644'
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: external4
  observedGeneration: 3 
  operatorLastSuccessfulVersion: 3.1 

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
spec:
  # ...
status:
  clusterId: XP9FP2P-RByvEy0W4cOEUA 

  conditions: 

    - lastTransitionTime: '2023-01-20T17:56:29.396588Z'
      status: 'True'
      type: Ready 

  kafkaMetadataState: KRaft 

  kafkaVersion: 4.1.0 

  kafkaNodePools: 

    - name: broker
    - name: controller
  listeners: 

    - addresses:
        - host: my-cluster-kafka-bootstrap.prm-project.svc
          port: 9092
      bootstrapServers: 'my-cluster-kafka-bootstrap.prm-project.svc:9092'
      name: plain
    - addresses:
        - host: my-cluster-kafka-bootstrap.prm-project.svc
          port: 9093
      bootstrapServers: 'my-cluster-kafka-bootstrap.prm-project.svc:9093'
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: tls
    - addresses:
        - host: >-
            2054284155.us-east-2.elb.amazonaws.com
          port: 9095
      bootstrapServers: >-
        2054284155.us-east-2.elb.amazonaws.com:9095
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: external3
    - addresses:
        - host: ip-10-0-172-202.us-east-2.compute.internal
          port: 31644
      bootstrapServers: 'ip-10-0-172-202.us-east-2.compute.internal:31644'
      certificates:
        - |
          -----BEGIN CERTIFICATE-----

          -----END CERTIFICATE-----
      name: external4
  observedGeneration: 3 

  operatorLastSuccessfulVersion: 3.1 

Use oc with the status subresource of a custom resource to retrieve information about the resource.

Operators watch and manage Kafka resources within defined OpenShift namespaces. The namespace scope of where each operator can watch these resources differs.

You can choose the namespace scope for the Cluster Operator. The Topic Operator and the User Operator can only watch a single Kafka cluster in a namespace. And they can only be connected to a single Kafka cluster.

The Cluster Operator creates and manages role-based access control (RBAC) resources for Streams for Apache Kafka components that need access to OpenShift resources.

For the Cluster Operator to function, it needs permission within the OpenShift cluster to interact with Kafka resources, such as Kafka and KafkaConnect, as well as managed resources like ConfigMap, Pod, Deployment, and Service.

Permission is specified through the following OpenShift RBAC resources:

The StrimziPodSet custom resource is used by Streams for Apache Kafka to create and manage Kafka, Kafka Connect, and MirrorMaker 2 pods.

You must not create, update, or delete StrimziPodSet resources. The StrimziPodSet custom resource is used internally and resources are managed solely by the Cluster Operator. As a consequence, the Cluster Operator must be running properly to avoid the possibility of pods not starting and Kafka clusters not being available.

The Cluster Operator ensures that only one operation runs at a time for each cluster by using locks. If another operation attempts to start while a lock is held, it waits until the current operation completes.

Operations such as cluster creation, rolling updates, scaling down, and scaling up are managed by the Cluster Operator.

If acquiring a lock takes longer than the configured timeout (STRIMZI_OPERATION_TIMEOUT_MS), a DEBUG message is logged:

DEBUG AbstractOperator:406 - Reconciliation #55(timer) Kafka(myproject/my-cluster): Failed to acquire lock lock::myproject::Kafka::my-cluster within 10000ms.

DEBUG AbstractOperator:406 - Reconciliation #55(timer) Kafka(myproject/my-cluster): Failed to acquire lock lock::myproject::Kafka::my-cluster within 10000ms.

Timed-out operations are retried during the next periodic reconciliation in intervals defined by STRIMZI_FULL_RECONCILIATION_INTERVAL_MS (by default 120 seconds).

If an INFO message continues to appear with the same same reconciliation number, it might indicate a lock release error:

INFO  AbstractOperator:399 - Reconciliation #1(watch) Kafka(myproject/my-cluster): Reconciliation is in progress

INFO  AbstractOperator:399 - Reconciliation #1(watch) Kafka(myproject/my-cluster): Reconciliation is in progress

Restarting the Cluster Operator can resolve such issues.

When running in the FIPS mode, SCRAM-SHA-512 passwords need to be at least 32 characters long. From Streams for Apache Kafka 2.3, the default password length in Streams for Apache Kafka User Operator is set to 32 characters as well. If you have a Kafka cluster with custom configuration that uses a password length that is less than 32 characters, you need to update your configuration. If you have any users with passwords shorter than 32 characters, you need to regenerate a password with the required length. You can do that, for example, by deleting the user secret and waiting for the User Operator to create a new password with the appropriate length.