Chapter 3. Loading configuration values from external sources

Procedure

1. Create a ConfigMap or Secret that contains the configuration properties.

   In this example, a ConfigMap object named my-connector-configuration contains connector properties:

   Example ConfigMap with connector properties

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: my-connector-configuration
   data:
     option1: value1
     option2: value2

   Copy to Clipboard Toggle word wrap

2. Specify the OpenShift Configuration Provider in the Kafka Connect configuration.

   The specification shown here can support loading values from secrets and ConfigMaps.

   Example Kafka Connect configuration to enable the OpenShift Configuration Provider

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnect
   metadata:
     name: my-connect
     annotations:
       strimzi.io/use-connector-resources: "true"
   spec:
     # ...
     config:
       # ...
       config.providers: secrets,configmaps 

   1

   
       config.providers.secrets.class: io.strimzi.kafka.KubernetesSecretConfigProvider 

   2

   
       config.providers.configmaps.class: io.strimzi.kafka.KubernetesConfigMapConfigProvider 

   3

   
     # ...

   Copy to Clipboard Toggle word wrap

   1

   The alias for the configuration provider is used to define other configuration parameters. The provider parameters use the alias from config.providers, taking the form config.providers.${alias}.class.

   2

   KubernetesSecretConfigProvider provides values from secrets.

   3

   KubernetesConfigMapConfigProvider provides values from config maps.

3. Create or update the resource to enable the provider.

   oc apply -f <kafka_connect_configuration_file>

   Copy to Clipboard Toggle word wrap

4. Create a role that permits access to the values in the external config map.

   Example role to access values from a config map

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: connector-configuration-role
   rules:
   - apiGroups: [""]
     resources: ["configmaps"]
     resourceNames: ["my-connector-configuration"]
     verbs: ["get"]
   # ...

   Copy to Clipboard Toggle word wrap

   The rule gives the role permission to access the my-connector-configuration config map.

5. Create a role binding to permit access to the namespace that contains the config map.

   Example role binding to access the namespace that contains the config map

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: connector-configuration-role-binding
   subjects:
   - kind: ServiceAccount
     name: my-connect-connect
     namespace: my-project
   roleRef:
     kind: Role
     name: connector-configuration-role
     apiGroup: rbac.authorization.k8s.io
   # ...

   Copy to Clipboard Toggle word wrap

   The role binding gives the role permission to access the my-project namespace.

   The service account must be the same one used by the Kafka Connect deployment. The service account name format is <cluster_name>-connect, where <cluster_name> is the name of the KafkaConnect custom resource.

6. Reference the config map in the connector configuration.

   Example connector configuration referencing the config map

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnector
   metadata:
     name: my-connector
     labels:
       strimzi.io/cluster: my-connect
   spec:
     # ...
     config:
       option: ${configmaps:my-project/my-connector-configuration:option1}
       # ...
   # ...

   Copy to Clipboard Toggle word wrap

   Placeholders for the property values in the config map are referenced in the connector configuration. The placeholder structure is configmaps:<path_and_file_name>:<property>. KubernetesConfigMapConfigProvider reads and extracts the option1 property value from the external config map.

Procedure

1. Specify the Environment Variables Configuration Provider in the Kafka Connect configuration.

   Define environment variables using the externalConfiguration property.

   Example Kafka Connect configuration to enable the Environment Variables Configuration Provider

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnect
   metadata:
     name: my-connect
     annotations:
       strimzi.io/use-connector-resources: "true"
   spec:
     # ...
     config:
       # ...
       config.providers: env 

   1

   
       config.providers.env.class: io.strimzi.kafka.EnvVarConfigProvider 

   2

   
     # ...
     externalConfiguration:
       env:
         - name: DB_PASSWORD 

   3

   
           valueFrom:
             secretKeyRef:
               name: db-creds 

   4

   
               key: dbPassword 

   5

   
     # ...

   Copy to Clipboard Toggle word wrap

   1

   The alias for the configuration provider is used to define other configuration parameters. The provider parameters use the alias from config.providers, taking the form config.providers.${alias}.class.

   2

   EnvVarConfigProvider provides values from environment variables.

   3

   The DB_PASSWORD environment variable takes a password value from a secret.

   4

   The name of the secret containing the predefined password.

   5

   The key for the password stored inside the secret.

2. Create or update the resource to enable the provider.

   oc apply -f <kafka_connect_configuration_file>

   Copy to Clipboard Toggle word wrap

3. Reference the environment variable in the connector configuration.

   Example connector configuration referencing the environment variable

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnector
   metadata:
     name: my-connector
     labels:
       strimzi.io/cluster: my-connect
   spec:
     # ...
     config:
       option: ${env:DB_PASSWORD}
       # ...
   # ...

   Copy to Clipboard Toggle word wrap