Chapter 10. Configuring secrets for Alertmanager

Procedure

1. Create a YAML file for alert routing. The example in this procedure uses a file called example-app-alert-routing.yaml.

2. Add an AlertmanagerConfig YAML definition to the file. For example:

   apiVersion: monitoring.coreos.com/v1beta1
   kind: AlertmanagerConfig
   metadata:
     name: example-routing
     namespace: ns1
   spec:
     route:
       receiver: default
       groupBy: [job]
     receivers:
     - name: default
       webhookConfigs:
       - url: https://example.org/post

3. Save the file.

4. Apply the resource to the cluster:

   $ oc apply -f example-app-alert-routing.yaml

   The configuration is automatically applied to the Alertmanager pods.

Procedure

1. Print the currently active Alertmanager configuration into the file alertmanager.yaml:

   $ oc -n openshift-user-workload-monitoring get secret alertmanager-user-workload --template='{{ index .data "alertmanager.yaml" }}' | base64 --decode > alertmanager.yaml

2. Edit the configuration in alertmanager.yaml:

   global:
     http_config:
       proxy_from_environment: true 1
   route:
     receiver: Default
     group_by:
     - name: Default
     routes:
     - matchers:
       - "service = prometheus-example-monitor" 2
       receiver: <receiver> 3
   receivers:
   - name: Default
   - name: <receiver>
     <receiver_configuration> 4

   1

   If you configured an HTTP cluster-wide proxy, set the proxy_from_environment parameter to true to enable proxying for all alert receivers.

   2

   Specify labels to match your alerts. This example targets all alerts that have the service="prometheus-example-monitor" label.

   3

   Specify the name of the receiver to use for the alerts group.

   4

   Specify the receiver configuration.

3. Apply the new configuration in the file:

   $ oc -n openshift-user-workload-monitoring create secret generic alertmanager-user-workload --from-file=alertmanager.yaml --dry-run=client -o=yaml |  oc -n openshift-user-workload-monitoring replace secret --filename=-