Chapter 2. Metrics and Service Mesh


2.1. Using metrics

Monitoring stack components are deployed by default in every OpenShift Container Platform installation and are managed by the Cluster Monitoring Operator (CMO). These components include Prometheus, Alertmanager, Thanos Querier, and others. The CMO also deploys the Telemeter Client, which sends a subset of data from platform Prometheus instances to Red Hat to facilitate Remote Health Monitoring for clusters.

When you have added your application to the mesh, you can monitor the in-cluster health and performance of your applications running on OpenShift Container Platform with metrics and customized alerts for CPU and memory usage, network connectivity, and other resource usage.

When you have added your application to the mesh in ambient mode, you can monitor the Istio standard metrics of your application from the ztunnel resource and the waypoint proxies. The ztunnel also exposes a variety of DNS and debugging metrics.

Ambient mode uses two proxy layers, which results in two types of metrics for each application service. You can collect Layer 4 (L4) TCP metrics from both the ztunnel and the waypoint proxies. You can collect Layer 7 (L7) metrics, such as HTTP traffic metrics, from the waypoint proxies.

You can integrate Red Hat OpenShift Service Mesh with user-workload monitoring to enable observability in your service mesh. User-workload monitoring provides access to essential built-in tools and is required to run Kiali, the dedicated console for Istio.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have enabled the user-workload monitoring.

    Note

    You can enable user-workload monitoring by applying the ConfigMap change for metrics integration. For more information, see Configuring user workload monitoring.

Procedure

  1. Create a ServiceMonitor resource that monitors the Istio control plane, similar to the following example:

    apiVersion: monitoring.coreos.com/v1
    kind: ServiceMonitor
    metadata:
      name: istiod-monitor
      namespace: istio-system
    spec:
      targetLabels:
      - app
      selector:
        matchLabels:
          istio: pilot
      endpoints:
      - port: http-monitoring
        interval: 30s
    Copy to Clipboard Toggle word wrap
  2. To validate that the ServiceMonitor resource is monitoring the Istio control plane, go to the OpenShift Console, navigate to Observe Metrics, and run the query istio_requests_total. Confirm that the metrics for the Istio request are displayed.

    Note

    The Metrics implementation can take a few minutes for the query to return results.

You can integrate Red Hat OpenShift Service Mesh with user-workload monitoring to enable observability in your service mesh ambient mode. User-workload monitoring provides access to essential built-in tools and is required to run Kiali, the dedicated console for Istio.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have enabled the user-workload monitoring.

    Note

    You can enable user workload monitoring by applying the ConfigMap change for metrics integration. For more information, see Configuring user workload monitoring.

Procedure

  1. Create a Service resource to define a port that uses the metrics exposed by the ztunnel, similar to the following example:

    apiVersion: v1
    kind: Service
    metadata:
      name: ztunnel
      namespace: ztunnel
      labels:
        app: ztunnel
        service: ztunnel
    spec:
      selector:
        app: ztunnel
      ports:
        - name: http-monitoring
          protocol: TCP
          port: 15020
          targetPort: 15020
    Copy to Clipboard Toggle word wrap
  2. Create a ServiceMonitor resource that monitors the Istio control plane, similar to the following example:

    apiVersion: monitoring.coreos.com/v1
    kind: ServiceMonitor
    metadata:
      name: istiod-monitor
      namespace: istio-system
    spec:
      targetLabels:
      - app
      selector:
        matchLabels:
          istio: pilot
      endpoints:
      - port: http-monitoring
        interval: 30s
    Copy to Clipboard Toggle word wrap
  3. Create a ServiceMonitor resource that collects the ztunnel metrics, similar to the following example:

    apiVersion: monitoring.coreos.com/v1
    kind: ServiceMonitor
    metadata:
      name: ztunnel-monitor
      namespace: ztunnel
    spec:
      targetLabels:
      - app
      selector:
        matchLabels:
          service: ztunnel
      endpoints:
      - port: http-monitoring
        interval: 30s
    Copy to Clipboard Toggle word wrap
  4. Optional: Deploy a waypoint proxy to enable the Layer 7 (L7) OpenShift Service Mesh features in ambient mode:

    1. Deploy a custom waypoint proxy with additional labels and ports for the bookinfo namespace, similar to the following example:

       apiVersion: gateway.networking.k8s.io/v1
      kind: Gateway
      metadata:
        labels:
          istio.io/waypoint-for: service
          app: waypoint
          service: waypoint
        name: waypoint
        namespace: bookinfo
      spec:
        gatewayClassName: istio-waypoint
        listeners:
        - name: mesh
          port: 15008
          protocol: HBONE
        - name: http-monitoring
          protocol: TCP
          port: 15020
      Copy to Clipboard Toggle word wrap
      <http-monitoring>
      Specifies an additional port that exposes the metrics used by the waypoint proxy.
    2. Enroll the namespace to use the waypoint by running the following command:

      $ oc label namespace bookinfo istio.io/use-waypoint=waypoint
      Copy to Clipboard Toggle word wrap
    3. Create a ServiceMonitor resource that collects the waypoint proxy metrics for the bookinfo namespace, similar to the following example:

      apiVersion: monitoring.coreos.com/v1
      kind: ServiceMonitor
      metadata:
        name: waypoint-monitor
        namespace: bookinfo
      spec:
        targetLabels:
        - app
        selector:
          matchLabels:
            service: waypoint
        endpoints:
        - port: http-monitoring
          interval: 30s
      Copy to Clipboard Toggle word wrap
      Note

      A waypoint proxy generates Layer 4 (L4) and L7 metrics. It scopes these statistics by Envoy proxy functions. The Envoy statistic functions are described in Envoy proxy documentation, for example, Upstream connection, Listener, HTTP Connection Manager, TCP proxy, and Router.

2.1.2.1. Verifying metrics in ambient mode

You can verify that the metrics for your application available in the OpenShift Console.

Prerequisites

Procedure

  1. On the OpenShift Console go to Observe Targets.
  2. Find the status of Metrics Targets by searching for targets such as istiod-monitor, ztunnel-monitor, and waypoint-monitor. waypoint-monitor is created only when the waypoint proxy is created to use Layer 7 (L7) OpenShift Service Mesh features.

    Note

    The ServiceMonitor resource configuration can take a few minutes to show in the Metrics Targets results.

  3. Send some traffic to the Bookinfo productpage service for generating metrics, by running the following command:

    $ curl "http://${GATEWAY_URL}/productpage" | grep "<title>"
    Copy to Clipboard Toggle word wrap
  4. On the OpenShift Console go to Observe Metrics and run a query such as, istio_build, istio_tcp_received_bytes_total, or istio_requests_total.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat