Chapter 9. Exposing 3scale APIcast Metrics to Prometheus
For this release of 3scale, Prometheus installation and configuration are not supported. Optionally, you can use the community version of Prometheus to visualize metrics and alerts for APIcast-managed API services.
9.1. About Prometheus
Prometheus is an open-source systems monitoring toolkit that you can use to monitor 3scale APIcast services deployed in the Red Hat OpenShift environment.
If you want to monitor your services with Prometheus, your services must expose a Prometheus endpoint. This endpoint is an HTTP interface that exposes a list of metrics and the current value of the metrics. Prometheus periodically scrapes these target-defined endpoints and writes the collected data into its database.
9.1.1. Prometheus queries
In the Prometheus UI, you can write queries in Prometheus Query Language (PromQL) to extract metric information. With PromQL, you can select and aggregate time series data in real time.
For example, you can use the following query to select all the values that Prometheus has recorded within the last 5 minutes for all time series that have the metric name http_requests_total:
http_requests_total[5m]
You can further define or filter the results of a query by specifying a label (a key:value pair) for the metric. For example, you can use the following query to select all the values that Prometheus has recorded within the last 5 minutes for all time series that have the metric name http_requests_total and a job label set to integration:
http_requests_total{job="integration"}[5m]The result of a query can either be shown as a graph, viewed as tabular data in Prometheus’s expression browser, or consumed by external systems by using the Prometheus HTTP API. Prometheus provides a graphical view of the data. For a more robust graphical dashboard to view Prometheus metrics, Grafana is a popular choice.
You can also use the the PromQL language to configure alerts in the Prometheus alertmanager tool.
Grafana is a community-supported feature. Deploying Grafana to monitor Red Hat 3scale products is not supported with Red Hat production service level agreements (SLAs).
9.2. APIcast integration with Prometheus
APIcast integration with Prometheus is available for the following deployment options:
- Self-managed APIcast - both with 3scale Hosted or On-premises API manager.
- Embedded APIcast in 3scale On-premises.
APIcast integration with Prometheus is not available in hosted API manager and hosted APIcast.
By default, Prometheus can monitor the APIcast metrics listed in Table 9.2, “Prometheus Default Metrics for 3scale APIcast”.
9.2.1. Additional options
Optionally, if you have cluster admin access to the OpenShift cluster, you can extend the total_response_time_seconds, upstream_response_time_seconds, and upstream_status metrics to include service_id and service_system_name labels. To extend these metrics, set the APICAST_EXTENDED_METRICS OpenShift environment variable to true with this command:
oc set env dc/apicast APICAST_EXTENDED_METRICS=true
If you use the APIcast Batch policy (described in Section 4.1.2, “3scale Batcher”), Prometheus can also monitor the metrics listed in Table 9.3, “Prometheus Metrics for 3scale APIcast Batch Policy”.
If a metric has no value, Prometheus hides the metric. For example, if nginx_error_log has no errors to report, Prometheus does not display the nginx_error_log metric. The nginx_error_log metric is only visible if it has a value.
Additional resources
For information about Prometheus, refer to Prometheus: Getting Started.
9.3. OpenShift environment variables for 3scale APIcast
To configure your Prometheus instance, you can set the OpenShift environment variable described in Table 9.1, “Prometheus Environment Variables for 3scale APIcast”.
| Environment Variable | Description | Default |
|---|---|---|
|
|
A boolean value that enables additional information on Prometheus metrics. The following metrics have the
|
|
Additional resources
For information on setting environment variables, see the relevant OpenShift guides:
- OpenShift 4: Applications
- OpenShift 3.11: Developer Guide
For information about supported configurations, refer to the Red Hat 3scale API Management Supported Configurations page.
9.4. 3scale APIcast metrics exposed to Prometheus
After you set up Prometheus to monitor 3scale APIcast, by default it can monitor the metrics listed in in Table 9.2, “Prometheus Default Metrics for 3scale APIcast”.
The metrics listed in Table 9.3, “Prometheus Metrics for 3scale APIcast Batch Policy” are only available when you use the Section 4.1.2, “3scale Batcher”.
| Metric | Description | Type | Labels |
|---|---|---|---|
|
| Number of HTTP connections | gauge | state(accepted,active,handled,reading,total,waiting,writing) |
|
| APIcast errors | counter |
|
|
| Capacity of the dictionaries shared between workers | gauge |
|
|
| Free space of the dictionaries shared between workers | gauge |
|
|
| Number of errors of the Lua library that manages the metrics | counter | none |
|
| Time needed to send a response to the client (in seconds)
Note: To access the | histogram |
|
|
| Response times from upstream servers (in seconds)
Note: To access the | histogram |
|
|
| HTTP status from upstream servers
Note: To access the | counter |
|
|
| Authorize and report requests to the 3scale backend (Apisonator) | counter |
|
| Metric | Description | Type | Labels |
|---|---|---|---|
|
| Hits in the auths cache of the 3scale batching policy | counter | none |
|
| Misses in the auths cache of the 3scale batching policy | counter | none |
