Chapter 5. Configuring secrets for Alertmanager

Procedure

1. Edit the ConfigMap object.

   1. Edit the user-workload-monitoring-config config map in the openshift-user-workload-monitoring project:

      $ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config

   2. Add a secrets: section under data/config.yaml/alertmanager/secrets with the following configuration:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: user-workload-monitoring-config
        namespace: openshift-user-workload-monitoring
      data:
        config.yaml: |
          alertmanager:
            secrets: 1
            - <secret_name_1> 2
            - <secret_name_2>

      1

      This section contains the secrets to be mounted into Alertmanager. The secrets must be located within the same namespace as the Alertmanager object.

      2

      The name of the Secret object that contains authentication credentials for the receiver. If you add multiple secrets, place each one on a new line.

      The following sample config map settings configure Alertmanager to use two Secret objects named test-secret and test-secret-api-token:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: user-workload-monitoring-config
        namespace: openshift-user-workload-monitoring
      data:
        config.yaml: |
          alertmanager:
            enabled: true
            secrets:
            - test-secret
            - test-api-receiver-token

2. Save the file to apply the changes. The new configuration is applied automatically.

Procedure

1. Edit the ConfigMap object:

   1. Edit the user-workload-monitoring-config ConfigMap object in the openshift-user-workload-monitoring project:

      $ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config

   2. Define a map of labels you want to add for every metric under data/config.yaml:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: user-workload-monitoring-config
        namespace: openshift-user-workload-monitoring
      data:
        config.yaml: |
          prometheus:
            externalLabels:
              <key>: <value> 1

      1

      Substitute <key>: <value> with a map of key-value pairs where <key> is a unique name for the new label and <value> is its value.

      Warning
      • Do not use prometheus or prometheus_replica as key names, because they are reserved and will be overwritten.
      • Do not use cluster or managed_cluster as key names. Using them can cause issues where you are unable to see data in the developer dashboards.
      Note

      In the openshift-user-workload-monitoring project, Prometheus handles metrics and Thanos Ruler handles alerting and recording rules. Setting externalLabels for prometheus in the user-workload-monitoring-config ConfigMap object will only configure external labels for metrics and not for any rules.

      For example, to add metadata about the region and environment to all time series and alerts related to user-defined projects, use the following example:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: user-workload-monitoring-config
        namespace: openshift-user-workload-monitoring
      data:
        config.yaml: |
          prometheus:
            externalLabels:
              region: eu
              environment: prod

   3. Save the file to apply the changes. The pods affected by the new configuration are automatically redeployed.