Chapter 3. Distributed tracing and Service Mesh

Procedure

1. Navigate to the Red Hat OpenShift distributed tracing data collection Operator and install the OpenTelemetryCollector resource in the istio-system namespace, similar to the following example:

   kind: OpenTelemetryCollector
   apiVersion: opentelemetry.io/v1beta1
   metadata:
     name: otel
     namespace: istio-system
   spec:
     observability:
       metrics: {}
     deploymentUpdateStrategy: {}
     config:
       exporters:
         otlp:
           endpoint: 'tempo-sample-distributor.tempo.svc.cluster.local:4317'
           tls:
             insecure: true
       receivers:
         otlp:
           protocols:
             grpc:
               endpoint: '0.0.0.0:4317'
             http: {}
       service:
         pipelines:
           traces:
             exporters:
               - otlp
             receivers:
               - otlp

   Copy to Clipboard Toggle word wrap

2. Update the Red Hat OpenShift Service Mesh Istio custom resource (CR) to enable tracing and define the distributed tracing data collection tracing providers in your meshConfig, similar to the following example:

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
   #  ...
     name: default
   spec:
     namespace: istio-system
   #  ...
     values:
       meshConfig:
         enableTracing: true
         extensionProviders:
         - name: otel
           opentelemetry:
             port: 4317
             service: otel-collector.istio-system.svc.cluster.local 

   1

   Copy to Clipboard Toggle word wrap

   1

   The service field is the OpenTelemetry collector service in the istio-system namespace.

3. Create an Istio Telemetry resource to enable tracers defined in spec.values.meshConfig.ExtensionProviders, similar to the following example:

   apiVersion: telemetry.istio.io/v1
   kind: Telemetry
   metadata:
     name: otel-demo
     namespace: istio-system
   spec:
     tracing:
       - providers:
           - name: otel
         randomSamplingPercentage: 100

   Copy to Clipboard Toggle word wrap

   After you verify that you can see traces, lower the randomSamplingPercentage value or set it to default to reduce the number of requests.

   Note

   You can use a single Istio Telemetry resource for both the Prometheus metrics provider and a tracing provider by setting spec.metrics.overrides.disabled to false. This enables the Prometheus metrics provider. This is an optional step and you can skip it if you configured metrics through the OpenShift Cluster Monitoring method described in the previous step.

4. Create the bookinfo namespace by running the following command:

   $ oc create ns bookinfo

   Copy to Clipboard Toggle word wrap

5. Depending on the update strategy you are using, enable sidecar injection in the namespace by running the appropriate commands:

   1. If you are using the InPlace update strategy, run the following command:

      $ oc label namespace curl istio-injection=enabled

      Copy to Clipboard Toggle word wrap

   2. If you are using the RevisionBased update strategy, run the following commands:

      1. Display the revision name by running the following command:

         $ oc get istiorevisions.sailoperator.io

         Copy to Clipboard Toggle word wrap

         Example output

         NAME      TYPE    READY   STATUS    IN USE   VERSION   AGE
         default   Local   True    Healthy   True     v1.24.3   3m33s

         Copy to Clipboard Toggle word wrap

      2. Label the namespace with the revision name to enable sidecar injection by running the following command:

         $ oc label namespace curl istio.io/rev=default

         Copy to Clipboard Toggle word wrap

6. Deploy the bookinfo application in the bookinfo namespace by running the following command:

   $ oc apply -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/bookinfo/platform/kube/bookinfo.yaml -n bookinfo

   Copy to Clipboard Toggle word wrap

7. Generate traffic to the productpage pod to generate traces:

   $ oc exec -it -n bookinfo deployments/productpage-v1 -c istio-proxy -- curl localhost:9080/productpage

   Copy to Clipboard Toggle word wrap

8. Validate the integration by running the following command to see traces in the UI:

   $ oc get routes -n tempo tempo-sample-query-frontend

   Copy to Clipboard Toggle word wrap
   Note

   The OpenShift route for Jaeger UI must be created in the Tempo namespace. You can either manually create it for the tempo-sample-query-frontend service, or update the Tempo custom resource with .spec.template.queryFrontend.jaegerQuery.ingress.type: route.

Procedure

1. Navigate to the Red Hat OpenShift distributed tracing data collection Operator and install the OpenTelemetryCollector resource in the istio-system namespace, similar to the following example:

   kind: OpenTelemetryCollector
   apiVersion: opentelemetry.io/v1beta1
   metadata:
     name: otel
     namespace: istio-system
   spec:
     mode: deployment
     observability:
       metrics: {}
     deploymentUpdateStrategy: {}
     config:
       exporters:
         otlp:
           endpoint: 'tempo-sample-distributor.tempo.svc.cluster.local:4317'
           tls:
             insecure: true
       receivers:
         otlp:
           protocols:
             grpc:
               endpoint: '0.0.0.0:4317'
             http: {}
       service:
         pipelines:
           traces:
             exporters:
               - otlp
             receivers:
               - otlp

   Copy to Clipboard Toggle word wrap
   • spec.config.exporters.otlp.endpoint defines the Tempo sample distributor service in a namespace such as tempo.

2. Update the Red Hat OpenShift Service Mesh Istio custom resource (CR) to define a tracing provider in the spec.values.meshConfig field, similar to the following example:

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
   #  ...
     name: default
   spec:
     namespace: istio-system
     profile: ambient
   #  ...
     values:
       meshConfig:
         enableTracing: true
         extensionProviders:
         - name: otel
           opentelemetry:
             port: 4317
             service: otel-collector.istio-system.svc.cluster.local
       pilot:
         trustedZtunnelNamespace: ztunnel

   Copy to Clipboard Toggle word wrap
   • spec.values.meshConfig.extensionProviders.opentelemetry.service defines the OpenTelemetry collector service in the istio-system namespace.

3. Create an Istio Telemetry CR to enable the tracing provider defined in the spec.values.meshConfig.ExtensionProviders field, similar to the following example:

   apiVersion: telemetry.istio.io/v1
   kind: Telemetry
   metadata:
     name: otel-demo
     namespace: istio-system
   spec:
     tracing:
       - providers:
           - name: otel
         randomSamplingPercentage: 100

   Copy to Clipboard Toggle word wrap
   Note

   After you can see the traces, lower the randomSamplingPercentage value or set it to default to reduce the number of requests. You can also use the spec.targetRefs field to enable tracing at a gateway or a waypoint level.

4. Optional: Use a single Istio Telemetry resource for both a Prometheus metrics provider and a tracing provider by setting spec.metrics.overrides.disabled field to false. This enables the Prometheus metrics provider. You do not need this step if you have configured metrics through the OpenShift Cluster Monitoring approach described in the previous step.