Chapter 12. Accessing third-party monitoring UIs and APIs
In OpenShift Container Platform 4.10, you cannot access third-party web browser user interfaces (UIs) for the following monitoring components: Alertmanager, Thanos Ruler, and Thanos Querier. However, you can access web UIs for Grafana and Prometheus, although this access is deprecated and is planned to be removed in a future OpenShift Container Platform release. In addition, you can access web service APIs for third-party monitoring components from the command line interface (CLI).
12.1. Accessing third-party monitoring UIs
OpenShift Container Platform does not provide or support direct access to third-party web user interfaces (UIs) for the following components in the monitoring stack: Alertmanager, Thanos Ruler, and Thanos Querier. As an alternative to these third-party UIs, navigate to the Observe section of the OpenShift Container Platform web console to access metrics, alerting, metrics targets, and dashboard UIs for platform components.
Although you can access the third-party Grafana and Prometheus web UIs from the web console or the CLI, this access is deprecated and is planned to be removed in a future OpenShift Container Platform release.
12.2. Accessing third-party monitoring web service APIs
You can directly access third-party web service APIs from the command line for monitoring stack components such as Prometheus, Alertmanager, Thanos Ruler, and Thanos Querier.
The following example shows how to query the service API receivers for Alertmanager. This example requires that the associated user account be bound against the monitoring-alertmanager-edit role in the openshift-monitoring namespace and that the account has the privilege to view the route. This access only supports using a Bearer Token for authentication.
$ host=$(oc -n openshift-monitoring get route alertmanager-main -ojsonpath={.spec.host})
$ token=$(oc whoami -t)
$ curl -H "Authorization: Bearer $token" -k "https://$host/api/v2/receivers"
To access Thanos Ruler and Thanos Querier service APIs, the requesting account must have get permission on the namespaces resource, which can be done by granting the cluster-monitoring-view cluster role to the account.
12.3. Querying metrics by using the federation endpoint for Prometheus
From OpenShift Container Platform 4.10.17, you can use the federation endpoint to scrape platform and user-defined metrics from a network location outside the cluster. To do so, access the Prometheus /federate endpoint for the cluster via an OpenShift Container Platform route.
A delay in retrieving metrics data occurs when you use federation. This delay can affect the accuracy and timeliness of the scraped metrics.
Using the federation endpoint can also degrade the performance and scalability of your cluster, especially if you use the federation endpoint to retrieve large amounts of metrics data. To avoid these issues, follow these recommendations:
- Do not try to retrieve all metrics data via the federation endpoint. Query it only when you want to retrieve a limited, aggregated data set. For example, retrieving fewer than 1,000 samples for each request helps minimize the risk of performance degradation.
- Avoid querying the federation endpoint frequently. Limit queries to a maximum of one every 30 seconds.
If you need to forward large amounts of data outside the cluster, use remote write instead. For more information, see the Configuring remote write storage section.
Prerequisites
-
You have installed the OpenShift CLI (
oc). - You have obtained the host URL for the OpenShift Container Platform route.
You have access to the cluster as a user with the
cluster-monitoring-viewcluster role or have obtained a bearer token withgetpermission on thenamespacesresource.NoteYou can only use bearer token authentication to access the federation endpoint.
Procedure
Retrieve the bearer token:
$ token=`oc whoami -t`
Query metrics from the
/federateroute. The following example queriesupmetrics:$ curl -G -s -k -H "Authorization: Bearer $token" \ 'https:/<federation_host>/federate' \ 1 --data-urlencode 'match[]=up'- 1
- For <federation_host>, substitute the host URL for the federation route.
Example output
# TYPE up untyped up{apiserver="kube-apiserver",endpoint="https",instance="10.0.143.148:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035322214 up{apiserver="kube-apiserver",endpoint="https",instance="10.0.148.166:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035338597 up{apiserver="kube-apiserver",endpoint="https",instance="10.0.173.16:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035343834 ...
