Chapter 2. Metrics and Service Mesh
2.1. Using metrics
Monitoring stack components are deployed by default in every OpenShift Container Platform installation and are managed by the Cluster Monitoring Operator (CMO). These components include Prometheus, Alertmanager, Thanos Querier, and others. The CMO also deploys the Telemeter Client, which sends a subset of data from platform Prometheus instances to Red Hat to facilitate Remote Health Monitoring for clusters.
When you have added your application to the mesh, you can monitor the in-cluster health and performance of your applications running on OpenShift Container Platform with metrics and customized alerts for CPU and memory usage, network connectivity, and other resource usage.
2.1.1. Configuring OpenShift Monitoring with Service Mesh
You can integrate Red Hat OpenShift Service Mesh with user-workload monitoring.
Prerequisites
- Red Hat OpenShift Service Mesh is installed.
- User-workload monitoring is enabled. See Enabling monitoring for user-defined projects.
Procedure
Apply a
ServiceMonitorobject to monitor the Istio control plane:Example
ServiceMonitorobjectapiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: istiod-monitor namespace: istio-system spec: targetLabels: - app selector: matchLabels: istio: pilot endpoints: - port: http-monitoring interval: 30sApply a
PodMonitorobject to collect metrics from Istio proxies:Example
PodMonitorobjectapiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: istio-proxies-monitor namespace: istio-system 1 spec: selector: matchExpressions: - key: istio-prometheus-ignore operator: DoesNotExist podMetricsEndpoints: - path: /stats/prometheus interval: 30s relabelings: - action: keep sourceLabels: ["__meta_kubernetes_pod_container_name"] regex: "istio-proxy" - action: keep sourceLabels: ["__meta_kubernetes_pod_annotationpresent_prometheus_io_scrape"] - action: replace regex: (\d+);(([A-Fa-f0-9]{1,4}::?){1,7}[A-Fa-f0-9]{1,4}) replacement: '[$2]:$1' sourceLabels: ["__meta_kubernetes_pod_annotation_prometheus_io_port","__meta_kubernetes_pod_ip"] targetLabel: "__address__" - action: replace regex: (\d+);((([0-9]+?)(\.|$)){4}) replacement: '$2:$1' sourceLabels: ["__meta_kubernetes_pod_annotation_prometheus_io_port","__meta_kubernetes_pod_ip"] targetLabel: "__address__" - action: labeldrop regex: "__meta_kubernetes_pod_label_(.+)" - sourceLabels: ["__meta_kubernetes_namespace"] action: replace targetLabel: namespace - sourceLabels: ["__meta_kubernetes_pod_name"] action: replace targetLabel: pod_name- 1
- Because OpenShift Container Platform monitoring ignores the
namespaceSelectorspec inServiceMonitorandPodMonitorobjects, you must apply thePodMonitorobject in all mesh namespaces, including the Istio control plane namespace.
-
Go to the OpenShift Console
Observe Metrics, and run the query istio_requests_total.
The Metrics implementation can take a few minutes for the query to return results.
