红帽全球峰会1.2. Streams for Apache Kafka operators
Streams for Apache Kafka operators are purpose-built with specialist operational knowledge to effectively manage Kafka on OpenShift. Each operator performs a distinct function.
- Cluster Operator
- The Cluster Operator handles the deployment and management of Apache Kafka clusters on OpenShift. It automates the setup of Kafka brokers, and other Kafka components and resources.
- Topic Operator
- The Topic Operator manages the creation, configuration, and deletion of topics within Kafka clusters.
- User Operator
- The User Operator manages Kafka users that require access to Kafka brokers.
When you deploy Streams for Apache Kafka, you first deploy the Cluster Operator. The Cluster Operator is then ready to handle the deployment of Kafka. You can also deploy the Topic Operator and User Operator using the Cluster Operator (recommended) or as standalone operators. You would use a standalone operator with a Kafka cluster that is not managed by the Cluster Operator.
The Topic Operator and User Operator are part of the Entity Operator. The Cluster Operator can deploy one or both operators based on the Entity Operator configuration.
To deploy the standalone operators, you need to set environment variables to connect to a Kafka cluster. These environment variables do not need to be set if you are deploying the operators using the Cluster Operator as they will be set by the Cluster Operator.
Operators watch and manage Streams for Apache Kafka resources in OpenShift namespaces. The Cluster Operator can watch a single namespace, multiple namespaces, or all namespaces in an OpenShift cluster. The Topic Operator and User Operator can watch a single namespace.
-
The Cluster Operator watches for
Kafkaresources -
The Topic Operator watches for
KafkaTopicresources -
The User Operator watches for
KafkaUserresources
The Topic Operator and the User Operator can only watch a single Kafka cluster in a namespace. And they can only be connected to a single Kafka cluster.
If multiple Topic Operators watch the same namespace, name collisions and topic deletion can occur. This is because each Kafka cluster uses Kafka topics that have the same name (such as __consumer_offsets). Make sure that only one Topic Operator watches a given namespace.
When using multiple User Operators with a single namespace, a user with a given username can exist in more than one Kafka cluster.
If you deploy the Topic Operator and User Operator using the Cluster Operator, they watch the Kafka cluster deployed by the Cluster Operator by default. You can also specify a namespace using watchedNamespace in the operator configuration.
For a standalone deployment of each operator, you specify a namespace and connection to the Kafka cluster to watch in the configuration.
1.2.2. Managing RBAC resources
The Cluster Operator creates and manages role-based access control (RBAC) resources for Streams for Apache Kafka components that need access to OpenShift resources.
For the Cluster Operator to function, it needs permission within the OpenShift cluster to interact with Kafka resources, such as Kafka and KafkaConnect, as well as managed resources like ConfigMap, Pod, Deployment, and Service.
Permission is specified through the following OpenShift RBAC resources:
-
ServiceAccount -
RoleandClusterRole -
RoleBindingandClusterRoleBinding
The Cluster Operator runs under a service account called strimzi-cluster-operator. It is assigned cluster roles that give it permission to create the RBAC resources for Streams for Apache Kafka components. Role bindings associate the cluster roles with the service account.
OpenShift prevents components operating under one ServiceAccount from granting another ServiceAccount privileges that the granting ServiceAccount does not have. Because the Cluster Operator creates the RoleBinding and ClusterRoleBinding RBAC resources needed by the resources it manages, it requires a role that gives it the same privileges.
The following sections describe the RBAC resources required by the Cluster Operator.
1.2.2.2. ClusterRole resources
The Cluster Operator uses ClusterRole resources to provide the necessary access to resources. Depending on the OpenShift cluster setup, a cluster administrator might be needed to create the cluster roles.
Cluster administrator rights are only needed for the creation of ClusterRole resources. The Cluster Operator will not run under a cluster admin account.
The RBAC resources follow the principle of least privilege and contain only those privileges needed by the Cluster Operator to operate the cluster of the Kafka component.
All cluster roles are required by the Cluster Operator in order to delegate privileges.
| Name | Description |
|---|---|
|
| Access rights for namespace-scoped resources used by the Cluster Operator to deploy and manage the operands. |
|
| Access rights for cluster-scoped resources used by the Cluster Operator to deploy and manage the operands. |
|
| Access rights used by the Cluster Operator for leader election. |
|
| Access rights used by the Cluster Operator to watch and manage the Streams for Apache Kafka custom resources. |
|
| Access rights to allow Kafka brokers to get the topology labels from OpenShift worker nodes when rack-awareness is used. |
|
| Access rights used by the Topic and User Operators to manage Kafka users and topics. |
|
| Access rights to allow Kafka Connect, MirrorMaker (1 and 2), and Kafka Bridge to get the topology labels from OpenShift worker nodes when rack-awareness is used. |
1.2.2.3. ClusterRoleBinding resources
The Cluster Operator uses ClusterRoleBinding and RoleBinding resources to associate its ClusterRole with its ServiceAccount. Cluster role bindings are required by cluster roles containing cluster-scoped resources.
| Name | Description |
|---|---|
|
|
Grants the Cluster Operator the rights from the |
|
|
Grants the Cluster Operator the rights from the |
|
|
Grants the Cluster Operator the rights from the |
| Name | Description |
|---|---|
|
|
Grants the Cluster Operator the rights from the |
|
|
Grants the Cluster Operator the rights from the |
|
|
Grants the Cluster Operator the rights from the |
|
|
Grants the Cluster Operator the rights from the |
1.2.2.4. ServiceAccount resources
The Cluster Operator runs using the strimzi-cluster-operator ServiceAccount. This service account grants it the privileges it requires to manage the operands. The Cluster Operator creates additional ClusterRoleBinding and RoleBinding resources to delegate some of these RBAC rights to the operands.
Each of the operands uses its own service account created by the Cluster Operator. This allows the Cluster Operator to follow the principle of least privilege and give the operands only the access rights that are really need.
| Name | Used by |
|---|---|
|
| ZooKeeper pods |
|
| Kafka broker pods |
|
| Entity Operator |
|
| Cruise Control pods |
|
| Kafka Exporter pods |
|
| Kafka Connect pods |
|
| MirrorMaker pods |
|
| MirrorMaker 2 pods |
|
| Kafka Bridge pods |
1.2.3. Managing pod resources
The StrimziPodSet custom resource is used by Streams for Apache Kafka to create and manage Kafka, Kafka Connect, and MirrorMaker 2 pods. If you are using ZooKeeper, ZooKeeper pods are also created and managed using StrimziPodSet resources.
You must not create, update, or delete StrimziPodSet resources. The StrimziPodSet custom resource is used internally and resources are managed solely by the Cluster Operator. As a consequence, the Cluster Operator must be running properly to avoid the possibility of pods not starting and Kafka clusters not being available.
OpenShift Deployment resources are used for creating and managing the pods of other components: Kafka Bridge, Kafka Exporter, Cruise Control, (deprecated) MirrorMaker 1, User Operator and Topic Operator.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}