红帽全球峰会17.2. Secrets generated by the operators
The Cluster Operator automatically sets up and renews TLS certificates to enable encryption and authentication within a cluster. It also sets up other TLS certificates if you want to enable encryption or mTLS authentication between Kafka brokers and clients.
Secrets are created when custom resources are deployed, such as Kafka and KafkaUser. Streams for Apache Kafka uses these secrets to store private and public key certificates for Kafka clusters, clients, and users. The secrets are used for establishing TLS encrypted connections between Kafka brokers, and between brokers and clients. They are also used for mTLS authentication.
Cluster and clients secrets are always pairs: one contains the public key and one contains the private key.
- Cluster secret
- A cluster secret contains the cluster CA to sign Kafka broker certificates. Connecting clients use the certificate to establish a TLS encrypted connection with a Kafka cluster. The certificate verifies broker identity.
- Client secret
- A client secret contains the clients CA for a user to sign its own client certificate. This allows mutual authentication against the Kafka cluster. The broker validates a client’s identity through the certificate.
- User secret
- A user secret contains a private key and certificate. The secret is created and signed by the clients CA when a new user is created. The key and certificate are used to authenticate and authorize the user when accessing the cluster.
You can provide Kafka listener certificates for TLS listeners or external listeners that have TLS encryption enabled. Use Kafka listener certificates to incorporate the security infrastructure you already have in place.
The secrets created by Streams for Apache Kafka provide private keys and certificates in PEM (Privacy Enhanced Mail) and PKCS #12 (Public-Key Cryptography Standards) formats. PEM and PKCS #12 are OpenSSL-generated key formats for TLS communications using the SSL protocol.
You can configure mutual TLS (mTLS) authentication that uses the credentials contained in the secrets generated for a Kafka cluster and user.
To set up mTLS, you must first do the following:
When you deploy a Kafka cluster, a <cluster_name>-cluster-ca-cert secret is created with public key to verify the cluster. You use the public key to configure a truststore for the client.
When you create a KafkaUser, a <kafka_user_name> secret is created with the keys and certificates to verify the user (client). Use these credentials to configure a keystore for the client.
With the Kafka cluster and client set up to use mTLS, you extract credentials from the secrets and add them to your client configuration.
- PEM keys and certificates
For PEM, you add the following to your client configuration:
- Truststore
-
ca.crtfrom the<cluster_name>-cluster-ca-certsecret, which is the CA certificate for the cluster.
-
- Keystore
-
user.crtfrom the<kafka_user_name>secret, which is the public certificate of the user. -
user.keyfrom the<kafka_user_name>secret, which is the private key of the user.
-
- PKCS #12 keys and certificates
For PKCS #12, you add the following to your client configuration:
- Truststore
-
ca.p12from the<cluster_name>-cluster-ca-certsecret, which is the CA certificate for the cluster. -
ca.passwordfrom the<cluster_name>-cluster-ca-certsecret, which is the password to access the public cluster CA certificate.
-
- Keystore
-
user.p12from the<kafka_user_name>secret, which is the public key certificate of the user. -
user.passwordfrom the<kafka_user_name>secret, which is the password to access the public key certificate of the Kafka user.
-
PKCS #12 is supported by Java, so you can add the values of the certificates directly to your Java client configuration. You can also reference the certificates from a secure storage location. With PEM files, you must add the certificates directly to the client configuration in single-line format. Choose a format that’s suitable for establishing TLS connections between your Kafka cluster and client. Use PKCS #12 if you are unfamiliar with PEM.
All keys are 2048 bits in size and, by default, are valid for 365 days from the initial generation. You can change the validity period.
17.2.2. Secrets generated by the Cluster Operator
The Cluster Operator generates the following certificates, which are saved as secrets in the OpenShift cluster. Streams for Apache Kafka uses these secrets by default.
The cluster CA and clients CA have separate secrets for the private key and public key.
<cluster_name>-cluster-ca- Contains the private key of the cluster CA. Streams for Apache Kafka and Kafka components use the private key to sign server certificates.
<cluster_name>-cluster-ca-cert- Contains the public key of the cluster CA. Kafka clients use the public key to verify the identity of the Kafka brokers they are connecting to with TLS server authentication.
<cluster_name>-clients-ca- Contains the private key of the clients CA. Kafka clients use the private key to sign new user certificates for mTLS authentication when connecting to Kafka brokers.
<cluster_name>-clients-ca-cert- Contains the public key of the clients CA. Kafka brokers use the public key to verify the identity of clients accessing the Kafka brokers when mTLS authentication is used.
Secrets for communication between Streams for Apache Kafka components contain a private key and a public key certificate signed by the cluster CA.
<cluster_name>-kafka-brokers- Contains the private and public keys for Kafka brokers.
<cluster_name>-zookeeper-nodes- Contains the private and public keys for ZooKeeper nodes.
<cluster_name>-cluster-operator-certs- Contains the private and public keys for encrypting communication between the Cluster Operator and Kafka or ZooKeeper.
<cluster_name>-entity-topic-operator-certs- Contains the private and public keys for encrypting communication between the Topic Operator and Kafka or ZooKeeper.
<cluster_name>-entity-user-operator-certs- Contains the private and public keys for encrypting communication between the User Operator and Kafka or ZooKeeper.
<cluster_name>-cruise-control-certs- Contains the private and public keys for encrypting communication between Cruise Control and Kafka or ZooKeeper.
<cluster_name>-kafka-exporter-certs- Contains the private and public keys for encrypting communication between Kafka Exporter and Kafka or ZooKeeper.
You can provide your own server certificates and private keys to connect to Kafka brokers using Kafka listener certificates rather than certificates signed by the cluster CA.
17.2.3. Cluster CA secrets
Cluster CA secrets are managed by the Cluster Operator in a Kafka cluster.
Only the <cluster_name>-cluster-ca-cert secret is required by clients. All other cluster secrets are accessed by Streams for Apache Kafka components. You can enforce this using OpenShift role-based access controls, if necessary.
The CA certificates in <cluster_name>-cluster-ca-cert must be trusted by Kafka client applications so that they validate the Kafka broker certificates when connecting to Kafka brokers over TLS.
| Field | Description |
|---|---|
|
| The current private key for the cluster CA. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
| The current certificate for the cluster CA. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for a Kafka broker pod <num>. Signed by a current or former cluster CA private key in |
|
|
Private key for a Kafka broker pod |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for ZooKeeper node <num>. Signed by a current or former cluster CA private key in |
|
|
Private key for ZooKeeper pod |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for mTLS communication between the Cluster Operator and Kafka or ZooKeeper. Signed by a current or former cluster CA private key in |
|
| Private key for mTLS communication between the Cluster Operator and Kafka or ZooKeeper. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for mTLS communication between the Topic Operator and Kafka or ZooKeeper. Signed by a current or former cluster CA private key in |
|
| Private key for mTLS communication between the Topic Operator and Kafka or ZooKeeper. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for mTLS communication between the User Operator and Kafka or ZooKeeper. Signed by a current or former cluster CA private key in |
|
| Private key for mTLS communication between the User Operator and Kafka or ZooKeeper. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for mTLS communication between Cruise Control and Kafka or ZooKeeper. Signed by a current or former cluster CA private key in |
|
| Private key for mTLS communication between the Cruise Control and Kafka or ZooKeeper. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
|
Certificate for mTLS communication between Kafka Exporter and Kafka or ZooKeeper. Signed by a current or former cluster CA private key in |
|
| Private key for mTLS communication between the Kafka Exporter and Kafka or ZooKeeper. |
17.2.4. Clients CA secrets
Clients CA secrets are managed by the Cluster Operator in a Kafka cluster.
The certificates in <cluster_name>-clients-ca-cert are those which the Kafka brokers trust.
The <cluster_name>-clients-ca secret is used to sign the certificates of client applications. This secret must be accessible to the Streams for Apache Kafka components and for administrative access if you are intending to issue application certificates without using the User Operator. You can enforce this using OpenShift role-based access controls, if necessary.
| Field | Description |
|---|---|
|
| The current private key for the clients CA. |
| Field | Description |
|---|---|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. |
|
| The current certificate for the clients CA. |
17.2.5. User secrets generated by the User Operator
User secrets are managed by the User Operator.
When a user is created using the User Operator, a secret is generated using the name of the user.
| Secret name | Field within secret | Description |
|---|---|---|
|
|
| PKCS #12 store for storing certificates and keys. |
|
| Password for protecting the PKCS #12 store. | |
|
| Certificate for the user, signed by the clients CA | |
|
| Private key for the user |
By configuring the clusterCaCert template property in the Kafka custom resource, you can add custom labels and annotations to the Cluster CA secrets created by the Cluster Operator. Labels and annotations are useful for identifying objects and adding contextual information. You configure template properties in Streams for Apache Kafka custom resources.
Example template customization to add labels and annotations to secrets
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
# ...
template:
clusterCaCert:
metadata:
labels:
label1: value1
label2: value2
annotations:
annotation1: value1
annotation2: value2
# ...17.2.7. Disabling ownerReference in the CA secrets
By default, the cluster and clients CA secrets are created with an ownerReference property that is set to the Kafka custom resource. This means that, when the Kafka custom resource is deleted, the CA secrets are also deleted (garbage collected) by OpenShift.
If you want to reuse the CA for a new cluster, you can disable the ownerReference by setting the generateSecretOwnerReference property for the cluster and clients CA secrets to false in the Kafka configuration. When the ownerReference is disabled, CA secrets are not deleted by OpenShift when the corresponding Kafka custom resource is deleted.
Example Kafka configuration with disabled ownerReference for cluster and clients CAs
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
# ...
spec:
# ...
clusterCa:
generateSecretOwnerReference: false
clientsCa:
generateSecretOwnerReference: false
# ...
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}