Chapter 8. Enabling alert routing for user-defined projects
In Red Hat OpenShift Service on AWS, a dedicated-admin can enable alert routing for user-defined projects. This process consists of two general steps:
- Enable alert routing for user-defined projects to use a separate Alertmanager instance.
- Grant users permission to configure alert routing for user-defined projects.
After you complete these steps, developers and other users can configure custom alerts and alert routing for their user-defined projects.
8.1. Understanding alert routing for user-defined projects
As a dedicated-admin, you can enable alert routing for user-defined projects. With this feature, you can allow users with the alert-routing-edit role to configure alert notification routing and receivers for user-defined projects. These notifications are routed by an Alertmanager instance dedicated to user-defined monitoring.
Users can then create and configure user-defined alert routing by creating or editing the AlertmanagerConfig objects for their user-defined projects without the help of an administrator.
After a user has defined alert routing for a user-defined project, user-defined alert notifications are routed to the alertmanager-user-workload pods in the openshift-user-workload-monitoring namespace.
The following are limitations of alert routing for user-defined projects:
-
For user-defined alerting rules, user-defined routing is scoped to the namespace in which the resource is defined. For example, a routing configuration in namespace
ns1only applies toPrometheusRulesresources in the same namespace. -
When a namespace is excluded from user-defined monitoring,
AlertmanagerConfigresources in the namespace cease to be part of the Alertmanager configuration.
8.2. Enabling a separate Alertmanager instance for user-defined alert routing
In Red Hat OpenShift Service on AWS, you may want to deploy a dedicated Alertmanager instance for user-defined projects, which provides user-defined alerts separate from default platform alerts. In these cases, you can optionally enable a separate instance of Alertmanager to send alerts for user-defined projects only.
Prerequisites
-
You have access to the cluster as a user with the
dedicated-adminrole. -
The
user-workload-monitoring-configConfigMapobject exists. This object is created by default when the cluster is created. -
You have installed the OpenShift CLI (
oc).
Procedure
Edit the
user-workload-monitoring-configConfigMapobject:$ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config
Add
enabled: trueandenableAlertmanagerConfig: truein thealertmanagersection underdata/config.yaml:apiVersion: v1 kind: ConfigMap metadata: name: user-workload-monitoring-config namespace: openshift-user-workload-monitoring data: config.yaml: | alertmanager: enabled: true 1 enableAlertmanagerConfig: true 2- 1
- Set the
enabledvalue totrueto enable a dedicated instance of the Alertmanager for user-defined projects in a cluster. Set the value tofalseor omit the key entirely to disable the Alertmanager for user-defined projects. If you set this value tofalseor if the key is omitted, user-defined alerts are routed to the default platform Alertmanager instance. - 2
- Set the
enableAlertmanagerConfigvalue totrueto enable users to define their own alert routing configurations withAlertmanagerConfigobjects.
- Save the file to apply the changes. The dedicated instance of Alertmanager for user-defined projects starts automatically.
Verification
Verify that the
alert-manager-user-workloadpods are running:# oc -n openshift-user-workload-monitoring get pods
Example output
NAME READY STATUS RESTARTS AGE alertmanager-user-workload-0 6/6 Running 0 38s alertmanager-user-workload-1 6/6 Running 0 38s ...
8.3. Granting users permission to configure alert routing for user-defined projects
You can grant users permission to configure alert routing for user-defined projects.
Prerequisites
-
You have access to the cluster as a user with the
dedicated-adminrole. -
The
user-workload-monitoring-configConfigMapobject exists. This object is created by default when the cluster is created. - The user account that you are assigning the role to already exists.
-
You have installed the OpenShift CLI (
oc).
Procedure
Assign the
alert-routing-editcluster role to a user in the user-defined project:$ oc -n <namespace> adm policy add-role-to-user alert-routing-edit <user> 1- 1
- For
<namespace>, substitute the namespace for the user-defined project, such asns1. For<user>, substitute the username for the account to which you want to assign the role.
Additional resources
