Chapter 2. Metrics and Service Mesh
2.1. Using metrics Copy linkLink copied to clipboard!
Monitoring stack components are deployed by default in every OpenShift Container Platform installation and are managed by the Cluster Monitoring Operator (CMO). These components include Prometheus, Alertmanager, Thanos Querier, and others. The CMO also deploys the Telemeter Client, which sends a subset of data from platform Prometheus instances to Red Hat to facilitate Remote Health Monitoring for clusters.
When you have added your application to the mesh, you can monitor the in-cluster health and performance of your applications running on OpenShift Container Platform with metrics and customized alerts for CPU and memory usage, network connectivity, and other resource usage.
When you have added your application to the mesh in ambient mode, you can monitor the Istio standard metrics of your application from the ztunnel resource and the waypoint proxies. The ztunnel also exposes a variety of DNS and debugging metrics.
Ambient mode uses two proxy layers, which results in two types of metrics for each application service. You can collect Layer 4 (L4) TCP metrics from both the ztunnel and the waypoint proxies. You can collect Layer 7 (L7) metrics, such as HTTP traffic metrics, from the waypoint proxies.
2.1.1. Configuring OpenShift Monitoring with Service Mesh Copy linkLink copied to clipboard!
You can integrate Red Hat OpenShift Service Mesh with user-workload monitoring to enable observability in your service mesh. User-workload monitoring provides access to essential built-in tools and is required to run Kiali, the dedicated console for Istio.
Prerequisites
- You have installed the Red Hat OpenShift Service Mesh Operator.
You have enabled the user-workload monitoring.
NoteYou can enable user-workload monitoring by applying the
ConfigMapchange for metrics integration. For more information, see Configuring user workload monitoring.
Procedure
Create a
ServiceMonitorresource that monitors the Istio control plane, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow To validate that the
ServiceMonitorresource is monitoring the Istio control plane, go to the OpenShift Console, navigate to ObserveMetrics, and run the query istio_requests_total. Confirm that the metrics for the Istio request are displayed.NoteThe Metrics implementation can take a few minutes for the query to return results.
2.1.2. Configuring OpenShift Monitoring with Service Mesh ambient mode Copy linkLink copied to clipboard!
You can integrate Red Hat OpenShift Service Mesh with user-workload monitoring to enable observability in your service mesh ambient mode. User-workload monitoring provides access to essential built-in tools and is required to run Kiali, the dedicated console for Istio.
Prerequisites
- You have installed the Red Hat OpenShift Service Mesh Operator.
You have enabled the user-workload monitoring.
NoteYou can enable user workload monitoring by applying the
ConfigMapchange for metrics integration. For more information, see Configuring user workload monitoring.
Procedure
Create a
Serviceresource to define a port that uses the metrics exposed by the ztunnel, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
ServiceMonitorresource that monitors the Istio control plane, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
ServiceMonitorresource that collects the ztunnel metrics, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: Deploy a waypoint proxy to enable the Layer 7 (L7) OpenShift Service Mesh features in ambient mode:
Deploy a custom waypoint proxy with additional labels and ports for the
bookinfonamespace, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow <http-monitoring>- Specifies an additional port that exposes the metrics used by the waypoint proxy.
Enroll the namespace to use the waypoint by running the following command:
oc label namespace bookinfo istio.io/use-waypoint=waypoint
$ oc label namespace bookinfo istio.io/use-waypoint=waypointCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
ServiceMonitorresource that collects the waypoint proxy metrics for thebookinfonamespace, similar to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteA waypoint proxy generates Layer 4 (L4) and L7 metrics. It scopes these statistics by Envoy proxy functions. The Envoy statistic functions are described in Envoy proxy documentation, for example,
Upstream connection,Listener,HTTP Connection Manager,TCP proxy, andRouter.
2.1.2.1. Verifying metrics in ambient mode Copy linkLink copied to clipboard!
You can verify that the metrics for your application available in the OpenShift Console.
Prerequisites
- You have deployed the Bookinfo application in ambient mode to use the following example. For more information, see Deploying the Bookinfo application in Istio ambient mode.
Procedure
-
On the OpenShift Console go to Observe
Targets. Find the status of
Metrics Targetsby searching for targets such asistiod-monitor,ztunnel-monitor, andwaypoint-monitor.waypoint-monitoris created only when the waypoint proxy is created to use Layer 7 (L7) OpenShift Service Mesh features.NoteThe
ServiceMonitorresource configuration can take a few minutes to show in theMetrics Targetsresults.Send some traffic to the Bookinfo
productpageservice for generating metrics, by running the following command:curl "http://${GATEWAY_URL}/productpage" | grep "<title>"$ curl "http://${GATEWAY_URL}/productpage" | grep "<title>"Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
On the OpenShift Console go to Observe
Metrics and run a query such as, istio_build,istio_tcp_received_bytes_total, oristio_requests_total.