Chapter 12. Alerts and Notifications
Alerts are current problems and critical conditions that occur in the system and notified to the user. The Grafana monitoring platform generates alerts based on severity levels.
You can configure alerts via SMTP and SNMP protocols. SMTP configuration will send email alerts to users that have email notifications enabled. SNMPv3 configuration will send SNMP trap alerts to the Alerts notifications drawer of the Web Administration environment.
12.1. Types of Alerts
The alerts triggered by the dashboard are classified in the following categories:
- Status alerts : Alerts arising when a cluster resource undergoes a change of state. For example, Healthy to Unhealthy.
- Utilization alerts: Alerts arising after a cluster resource exceed the set threshold and after it reverts to the normal state. For example, when the Host CPU utilization is breached, an alert is triggered notifying the user about the event.
12.2. List of Alerts
The list of Web Administration alerts are given in the tables below.
Status Alerts
Alert | System Resource(s) |
---|---|
volume status | Volume and Cluster |
volume state | Volume and Cluster |
brick status | Volume, Host, and Cluster |
peer status | Cluster |
rebalance status | Volume and Cluster |
Geo-Replication status | Cluster |
quorum of volume lost | Volume and Cluster |
quorum of volume regained | Volume and Cluster |
svc connected | Cluster |
svc disconnected | Cluster |
minimum number of bricks not up in EC subvolume | Volume and Cluster |
minimum number of bricks up in EC subvolume | Volume and Cluster |
afr quorum met for subvolume | Volume and Cluster |
afr quorum fail for subvolume | Volume and Cluster |
afr subvolume up | Volume and Cluster |
afr subvolume down | Volume and Cluster |
Utilization Alerts
Alert | System Resource |
---|---|
cpu utilization | Host |
memory utilization | host |
swap utilization | host |
volume utilization | Volume and Cluster |
brick utilization | Volume and Cluster |
12.3. Alerts Notifications Drawer
Alerts drawer is a notification indicator embedded in the Web Administration interface to display the system wide alerts.
Accessing Alerts Drawer
To access the Alerts drawer, log in the Web Administration interface. In the default landing interface, locate and click on the interactive bell icon on the header bar at the top right-hand side.
- The drawer is opened displaying the number of alerts generated.
To filter alerts, click on the status icon at the right.
12.4. SMTP Notifications Configuration
Tendrl-ansible installs and configures tendrl-notifier. After the tendrl-notifier file is configured, configure SMTP email notifications:
-
Open the
/etc/tendrl/notifier/email.conf.yaml
file. Update the parameters:
email_id = <The sender email id> email_smtp_server = <The smtp server> email_smtp_port = <The smtp port>
If the SMTP server supports only authenticated email, follow the template in the
/etc/tendrl/notifier/email_auth.conf.yaml
file and accordingly enable the following:auth = <ssl/tls> email_pass = <password corresponding to email_id for authenticating to smtp server>
Restart the tendrl-notifier service:
systemctl restart tendrl-notifier
12.5. SNMPv3 Notification Configuration
Configure SNMP
Tendrl-ansible installs and configures tendrl-notifier. After the tendrl-notifier file is configured, configure SNMPv3 trap notifications:
Open the
tendrl-notifier
configuration file:# cat /etc/tendrl/notifier/snmp.conf.yaml
Update the parameters in the file for v3 trap alerts:
For v3_endpoint: # For more hosts you can add more entry with endpoint2, endpoint3, etc endpoint1: # Name or IP address of the remote SNMP host. host_ip: <Receiving machine ip> # Name of the user on the host that connects to the agent. username: <Username of receiver> # Enables the agent to receive packets from the host. auth_key: <md5 password> # The private user password priv_key: <des password> # For v2_endpoint: # For more hosts you can add more entry with endpoint2, endpoint3, etc endpoint1: # Name or IP address of the remote SNMP host. host_ip: <Receiving machine ip> community: <community name>
Restart the tendrl-notifier service:
systemctl restart tendrl-notifier