Chapter 13. Accessing third-party monitoring APIs
In OpenShift Container Platform 4.11, you can access web service APIs for some third-party monitoring components from the command line interface (CLI).
13.1. Accessing third-party monitoring web service APIs
You can directly access third-party web service APIs from the command line for the following monitoring stack components: Prometheus, Alertmanager, Thanos Ruler, and Thanos Querier.
The following example commands show how to query the service API receivers for Alertmanager. This example requires that the associated user account be bound against the monitoring-alertmanager-edit role in the openshift-monitoring namespace and that the account has the privilege to view the route. This access only supports using a Bearer Token for authentication.
$ oc login -u <username> -p <password>
$ host=$(oc -n openshift-monitoring get route alertmanager-main -ojsonpath={.spec.host})$ token=$(oc whoami -t)
$ curl -H "Authorization: Bearer $token" -k "https://$host/api/v2/receivers"
To access Thanos Ruler and Thanos Querier service APIs, the requesting account must have get permission on the namespaces resource, which can be done by granting the cluster-monitoring-view cluster role to the account.
13.2. Querying metrics by using the federation endpoint for Prometheus
You can use the federation endpoint to scrape platform and user-defined metrics from a network location outside the cluster. To do so, access the Prometheus /federate endpoint for the cluster via an OpenShift Container Platform route.
A delay in retrieving metrics data occurs when you use federation. This delay can affect the accuracy and timeliness of the scraped metrics.
Using the federation endpoint can also degrade the performance and scalability of your cluster, especially if you use the federation endpoint to retrieve large amounts of metrics data. To avoid these issues, follow these recommendations:
- Do not try to retrieve all metrics data via the federation endpoint. Query it only when you want to retrieve a limited, aggregated data set. For example, retrieving fewer than 1,000 samples for each request helps minimize the risk of performance degradation.
- Avoid querying the federation endpoint frequently. Limit queries to a maximum of one every 30 seconds.
If you need to forward large amounts of data outside the cluster, use remote write instead. For more information, see the Configuring remote write storage section.
Prerequisites
-
You have installed the OpenShift CLI (
oc). - You have obtained the host URL for the OpenShift Container Platform route.
You have access to the cluster as a user with the
cluster-monitoring-viewcluster role or have obtained a bearer token withgetpermission on thenamespacesresource.NoteYou can only use bearer token authentication to access the federation endpoint.
Procedure
Retrieve the bearer token:
$ token=`oc whoami -t`
Query metrics from the
/federateroute. The following example queriesupmetrics:$ curl -G -s -k -H "Authorization: Bearer $token" \ 'https:/<federation_host>/federate' \ 1 --data-urlencode 'match[]=up'- 1
- For <federation_host>, substitute the host URL for the federation route.
Example output
# TYPE up untyped up{apiserver="kube-apiserver",endpoint="https",instance="10.0.143.148:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035322214 up{apiserver="kube-apiserver",endpoint="https",instance="10.0.148.166:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035338597 up{apiserver="kube-apiserver",endpoint="https",instance="10.0.173.16:6443",job="apiserver",namespace="default",service="kubernetes",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1657035343834 ...
