Chapter 5. AlertingRule [monitoring.openshift.io/v1]
- Description
- AlertingRule represents a set of user-defined Prometheus rule groups containing alerting rules. This resource is the supported method for cluster admins to create alerts based on metrics recorded by the platform monitoring stack in OpenShift, i.e. the Prometheus instance deployed to the openshift-monitoring namespace. You might use this to create custom alerting rules not shipped with OpenShift based on metrics from components such as the node_exporter, which provides machine-level metrics such as CPU usage, or kube-state-metrics, which provides metrics on Kubernetes usage. The API is mostly compatible with the upstream PrometheusRule type from the prometheus-operator. The primary difference being that recording rules are not allowed here — only alerting rules. For each AlertingRule resource created, a corresponding PrometheusRule will be created in the openshift-monitoring namespace. OpenShift requires admins to use the AlertingRule resource rather than the upstream type in order to allow better OpenShift specific defaulting and validation, while not modifying the upstream APIs directly. You can find upstream API documentation for PrometheusRule resources here: https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md Compatibility level 1: Stable within a major release for a minimum of 12 months or 3 minor releases (whichever is longer).
- Type
-
object
- Required
-
spec
-
5.1. Specification
Property | Type | Description |
---|---|---|
|
| APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
|
| Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
| Standard object’s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata | |
|
| spec describes the desired state of this AlertingRule object. |
|
| status describes the current state of this AlertOverrides object. |
5.1.1. .spec
- Description
- spec describes the desired state of this AlertingRule object.
- Type
-
object
- Required
-
groups
-
Property | Type | Description |
---|---|---|
|
| groups is a list of grouped alerting rules. Rule groups are the unit at which Prometheus parallelizes rule processing. All rules in a single group share a configured evaluation interval. All rules in the group will be processed together on this interval, sequentially, and all rules will be processed. It’s common to group related alerting rules into a single AlertingRule resources, and within that resource, closely related alerts, or simply alerts with the same interval, into individual groups. You are also free to create AlertingRule resources with only a single rule group, but be aware that this can have a performance impact on Prometheus if the group is extremely large or has very complex query expressions to evaluate. Spreading very complex rules across multiple groups to allow them to be processed in parallel is also a common use-case. |
|
| RuleGroup is a list of sequentially evaluated alerting rules. |
5.1.2. .spec.groups
- Description
- groups is a list of grouped alerting rules. Rule groups are the unit at which Prometheus parallelizes rule processing. All rules in a single group share a configured evaluation interval. All rules in the group will be processed together on this interval, sequentially, and all rules will be processed. It’s common to group related alerting rules into a single AlertingRule resources, and within that resource, closely related alerts, or simply alerts with the same interval, into individual groups. You are also free to create AlertingRule resources with only a single rule group, but be aware that this can have a performance impact on Prometheus if the group is extremely large or has very complex query expressions to evaluate. Spreading very complex rules across multiple groups to allow them to be processed in parallel is also a common use-case.
- Type
-
array
5.1.3. .spec.groups[]
- Description
- RuleGroup is a list of sequentially evaluated alerting rules.
- Type
-
object
- Required
-
name
-
rules
-
Property | Type | Description |
---|---|---|
|
| interval is how often rules in the group are evaluated. If not specified, it defaults to the global.evaluation_interval configured in Prometheus, which itself defaults to 30 seconds. You can check if this value has been modified from the default on your cluster by inspecting the platform Prometheus configuration: The relevant field in that resource is: spec.evaluationInterval |
|
| name is the name of the group. |
|
| rules is a list of sequentially evaluated alerting rules. Prometheus may process rule groups in parallel, but rules within a single group are always processed sequentially, and all rules are processed. |
|
| Rule describes an alerting rule. See Prometheus documentation: - https://www.prometheus.io/docs/prometheus/latest/configuration/alerting_rules |
5.1.4. .spec.groups[].rules
- Description
- rules is a list of sequentially evaluated alerting rules. Prometheus may process rule groups in parallel, but rules within a single group are always processed sequentially, and all rules are processed.
- Type
-
array
5.1.5. .spec.groups[].rules[]
- Description
- Rule describes an alerting rule. See Prometheus documentation: - https://www.prometheus.io/docs/prometheus/latest/configuration/alerting_rules
- Type
-
object
- Required
-
alert
-
expr
-
Property | Type | Description |
---|---|---|
|
| alert is the name of the alert. Must be a valid label value, i.e. may contain any Unicode character. |
|
| annotations to add to each alert. These are values that can be used to store longer additional information that you won’t query on, such as alert descriptions or runbook links. |
|
| expr is the PromQL expression to evaluate. Every evaluation cycle this is evaluated at the current time, and all resultant time series become pending or firing alerts. This is most often a string representing a PromQL expression, e.g.: mapi_current_pending_csr > mapi_max_pending_csr In rare cases this could be a simple integer, e.g. a simple "1" if the intent is to create an alert that is always firing. This is sometimes used to create an always-firing "Watchdog" alert in order to ensure the alerting pipeline is functional. |
|
| for is the time period after which alerts are considered firing after first returning results. Alerts which have not yet fired for long enough are considered pending. |
|
|
labels to add or overwrite for each alert. The results of the PromQL expression for the alert will result in an existing set of labels for the alert, after evaluating the expression, for any label specified here with the same name as a label in that set, the label here wins and overwrites the previous value. These should typically be short identifying values that may be useful to query against. A common example is the alert severity, where one sets |
5.1.6. .status
- Description
- status describes the current state of this AlertOverrides object.
- Type
-
object
Property | Type | Description |
---|---|---|
|
| observedGeneration is the last generation change you’ve dealt with. |
|
| prometheusRule is the generated PrometheusRule for this AlertingRule. Each AlertingRule instance results in a generated PrometheusRule object in the same namespace, which is always the openshift-monitoring namespace. |
5.1.7. .status.prometheusRule
- Description
- prometheusRule is the generated PrometheusRule for this AlertingRule. Each AlertingRule instance results in a generated PrometheusRule object in the same namespace, which is always the openshift-monitoring namespace.
- Type
-
object
- Required
-
name
-
Property | Type | Description |
---|---|---|
|
| name of the referenced PrometheusRule. |
5.2. API endpoints
The following API endpoints are available:
/apis/monitoring.openshift.io/v1/alertingrules
-
GET
: list objects of kind AlertingRule
-
/apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules
-
DELETE
: delete collection of AlertingRule -
GET
: list objects of kind AlertingRule -
POST
: create an AlertingRule
-
/apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}
-
DELETE
: delete an AlertingRule -
GET
: read the specified AlertingRule -
PATCH
: partially update the specified AlertingRule -
PUT
: replace the specified AlertingRule
-
/apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}/status
-
GET
: read status of the specified AlertingRule -
PATCH
: partially update status of the specified AlertingRule -
PUT
: replace status of the specified AlertingRule
-
5.2.1. /apis/monitoring.openshift.io/v1/alertingrules
- HTTP method
-
GET
- Description
- list objects of kind AlertingRule
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
5.2.2. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules
- HTTP method
-
DELETE
- Description
- delete collection of AlertingRule
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
GET
- Description
- list objects of kind AlertingRule
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
POST
- Description
- create an AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
|
| fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. |
Parameter | Type | Description |
---|---|---|
|
|
HTTP code | Reponse body |
---|---|
200 - OK |
|
201 - Created |
|
202 - Accepted |
|
401 - Unauthorized | Empty |
5.2.3. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}
Parameter | Type | Description |
---|---|---|
|
| name of the AlertingRule |
- HTTP method
-
DELETE
- Description
- delete an AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
HTTP code | Reponse body |
---|---|
200 - OK |
|
202 - Accepted |
|
401 - Unauthorized | Empty |
- HTTP method
-
GET
- Description
- read the specified AlertingRule
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
PATCH
- Description
- partially update the specified AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
|
| fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. |
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
PUT
- Description
- replace the specified AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
|
| fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. |
Parameter | Type | Description |
---|---|---|
|
|
HTTP code | Reponse body |
---|---|
200 - OK |
|
201 - Created |
|
401 - Unauthorized | Empty |
5.2.4. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}/status
Parameter | Type | Description |
---|---|---|
|
| name of the AlertingRule |
- HTTP method
-
GET
- Description
- read status of the specified AlertingRule
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
PATCH
- Description
- partially update status of the specified AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
|
| fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. |
HTTP code | Reponse body |
---|---|
200 - OK |
|
401 - Unauthorized | Empty |
- HTTP method
-
PUT
- Description
- replace status of the specified AlertingRule
Parameter | Type | Description |
---|---|---|
|
| When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed |
|
| fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. |
Parameter | Type | Description |
---|---|---|
|
|
HTTP code | Reponse body |
---|---|
200 - OK |
|
201 - Created |
|
401 - Unauthorized | Empty |