Service Mesh


OpenShift Container Platform 4.13

Service Mesh installation, usage, and release notes

Red Hat OpenShift Documentation Team

Abstract

This document provides information on how to use Service Mesh in OpenShift Container Platform.

Chapter 1. Service Mesh 2.x

1.1. About OpenShift Service Mesh

Note

Because Red Hat OpenShift Service Mesh releases on a different cadence from OpenShift Container Platform and because the Red Hat OpenShift Service Mesh Operator supports deploying multiple versions of the ServiceMeshControlPlane, the Service Mesh documentation does not maintain separate documentation sets for minor versions of the product. The current documentation set applies to the most recent version of Service Mesh unless version-specific limitations are called out in a particular topic or for a particular feature.

For additional information about the Red Hat OpenShift Service Mesh life cycle and supported platforms, refer to the Platform Life Cycle Policy.

1.1.1. Introduction to Red Hat OpenShift Service Mesh

Red Hat OpenShift Service Mesh addresses a variety of problems in a microservice architecture by creating a centralized point of control in an application. It adds a transparent layer on existing distributed applications without requiring any changes to the application code.

Microservice architectures split the work of enterprise applications into modular services, which can make scaling and maintenance easier. However, as an enterprise application built on a microservice architecture grows in size and complexity, it becomes difficult to understand and manage. Service Mesh can address those architecture problems by capturing or intercepting traffic between services and can modify, redirect, or create new requests to other services.

Service Mesh, which is based on the open source Istio project, provides an easy way to create a network of deployed services that provides discovery, load balancing, service-to-service authentication, failure recovery, metrics, and monitoring. A service mesh also provides more complex operational functionality, including A/B testing, canary releases, access control, and end-to-end authentication.

1.1.2. Core features

Red Hat OpenShift Service Mesh provides a number of key capabilities uniformly across a network of services:

  • Traffic Management - Control the flow of traffic and API calls between services, make calls more reliable, and make the network more robust in the face of adverse conditions.
  • Service Identity and Security - Provide services in the mesh with a verifiable identity and provide the ability to protect service traffic as it flows over networks of varying degrees of trustworthiness.
  • Policy Enforcement - Apply organizational policy to the interaction between services, ensure access policies are enforced and resources are fairly distributed among consumers. Policy changes are made by configuring the mesh, not by changing application code.
  • Telemetry - Gain understanding of the dependencies between services and the nature and flow of traffic between them, providing the ability to quickly identify issues.

1.2. Service Mesh Release Notes

1.2.1. Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

1.2.2. New features and enhancements

This release adds improvements related to the following components and concepts.

1.2.2.1. New features Red Hat OpenShift Service Mesh version 2.5.2

This release of Red Hat OpenShift Service Mesh updates the Red Hat OpenShift Service Mesh Operator version to 2.5.2, and includes the following ServiceMeshControlPlane resource version updates: 2.5.2, 2.4.8 and 2.3.12. This release addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane.

1.2.2.1.1. Component versions for Red Hat OpenShift Service Mesh version 2.5.2
ComponentVersion

Istio

1.18.7

Envoy Proxy

1.26.8

Kiali

1.73.8

1.2.2.2. New features Red Hat OpenShift Service Mesh version 2.5.1

This release of Red Hat OpenShift Service Mesh updates the Red Hat OpenShift Service Mesh Operator version to 2.5.1, and includes the following ServiceMeshControlPlane resource version updates: 2.5.1, 2.4.7 and 2.3.11.

This release addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

1.2.2.2.1. Component versions for Red Hat OpenShift Service Mesh version 2.5.1
ComponentVersion

Istio

1.18.7

Envoy Proxy

1.26.8

Kiali

1.73.7

1.2.2.3. New features Red Hat OpenShift Service Mesh version 2.5

This release of Red Hat OpenShift Service Mesh updates the Red Hat OpenShift Service Mesh Operator version to 2.5.0, and includes the following ServiceMeshControlPlane resource version updates: 2.5.0, 2.4.6 and 2.3.10.

This release adds new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

This release ends maintenance support for OpenShift Service Mesh version 2.2. If you are using OpenShift Service Mesh version 2.2, you should update to a supported version.

1.2.2.3.1. Component versions for Red Hat OpenShift Service Mesh version 2.5
ComponentVersion

Istio

1.18.7

Envoy Proxy

1.26.8

Kiali

1.73.4

1.2.2.3.2. Istio 1.18 support

Service Mesh 2.5 is based on Istio 1.18, which brings in new features and product enhancements. While Red Hat OpenShift Service Mesh supports many Istio 1.18 features, the following exceptions should be noted:

  • Ambient mesh is not supported
  • QuickAssist Technology (QAT) PrivateKeyProvider in Istio is not supported
1.2.2.3.3. Cluster-Wide mesh migration

This release adds documentation for migrating from a multitenant mesh to a cluster-wide mesh. For more information, see the following documentation:

  • "About migrating to a cluster-wide mesh"
  • "Excluding namespaces from a cluster-wide mesh"
  • "Defining which namespaces receive sidecar injection in a cluster-wide mesh"
  • "Excluding individual pods from a cluster-wide mesh"
1.2.2.3.4. Red Hat OpenShift Service Mesh Operator on ARM-based clusters

This release provides the Red Hat OpenShift Service Mesh Operator on ARM-based clusters as a generally available feature.

1.2.2.3.5. Integration with Red Hat OpenShift distributed tracing platform (Tempo) Stack

This release introduces a generally available integration of the tracing extension provider(s). You can expose tracing data to the Red Hat OpenShift distributed tracing platform (Tempo) stack by appending a named element and the zipkin provider to the spec.meshConfig.extensionProviders specification. Then, a telemetry custom resource configures Istio proxies to collect trace spans and send them to the Tempo distributor service endpoint.

Note

Red Hat OpenShift distributed tracing platform (Tempo) Stack is not supported on IBM Z.

1.2.2.3.6. OpenShift Service Mesh Console plugin

This release introduces a generally available version of the OpenShift Service Mesh Console (OSSMC) plugin.

The OSSMC plugin is an extension to the OpenShift Console that provides visibility into your Service Mesh. With the OSSMC plugin installed, a new Service Mesh menu option is available on the navigation pane of the web console, as well as new Service Mesh tabs that enhance existing Workloads and Service console pages.

The features of the OSSMC plugin are very similar to those of the standalone Kiali Console. The OSSMC plugin does not replace the Kiali Console, and after installing the OSSMC plugin, you can still access the standalone Kiali Console.

1.2.2.3.7. Istio OpenShift Routing (IOR) default setting change

The default setting for Istio OpenShift Routing (IOR) has changed. Starting with this release, automatic routes are disabled by default for new instances of the ServiceMeshControlPlane resource.

For new instances of the ServiceMeshControlPlane resources, you can use automatic routes by setting the enabled field to true in the gateways.openshiftRoute specification of the ServiceMeshControlPlane resource.

Example ServiceMeshControlPlane resource

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  gateways:
    openshiftRoute:
      enabled: true

When updating existing instances of the ServiceMeshControlPlane resource to Red Hat OpenShift Service Mesh version 2.5, automatic routes remain enabled by default.

1.2.2.3.8. Istio proxy concurrency configuration enhancement

The concurrency parameter in the networking.istio API configures how many worker threads the Istio proxy runs.

For consistency across deployments, Istio now configures the concurrency parameter based upon the CPU limit allocated to the proxy container. For example, a limit of 2500m would set the concurrency parameter to 3. If you set the concurrency parameter to a different value, then Istio uses that value to configure how many threads the proxy runs instead of using the CPU limit.

Previously, the default setting for the parameter was 2.

1.2.2.3.9. Gateway API CRD versions
Important

OpenShift Container Platform Gateway API support is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

A new version of the Gateway API custom resource definition (CRD) is now available. Refer to the following table to determine which Gateway API version should be installed with the OpenShift Service Mesh version you are using:

Service Mesh VersionIstio VersionGateway API VersionNotes

2.5.x

1.18.x

0.6.2

Use the experimental branch because ReferenceGrand is missing in v0.6.2

2.4.x

1.16.x

0.5.1

For multitenant mesh deployment, all Gateway API CRDs must be present. Use the experimental branch.

1.2.2.4. New features Red Hat OpenShift Service Mesh version 2.4.8

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.2, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane.

1.2.2.4.1. Component versions for Red Hat OpenShift Service Mesh version 2.4.8
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.12

Kiali

1.65.11

1.2.2.5. New features Red Hat OpenShift Service Mesh version 2.4.7

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.1, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

1.2.2.5.1. Component versions for Red Hat OpenShift Service Mesh version 2.4.7
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.12

Kiali

1.65.11

1.2.2.6. New features Red Hat OpenShift Service Mesh version 2.4.6

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.0, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

1.2.2.6.1. Component versions for Red Hat OpenShift Service Mesh version 2.4.6
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.12

Kiali

1.65.11

1.2.2.7. New features Red Hat OpenShift Service Mesh version 2.4.5

This release of Red Hat OpenShift Service Mesh updates the Red Hat OpenShift Service Mesh Operator version to 2.4.5, and includes the following ServiceMeshControlPlane resource version updates: 2.4.5, 2.3.9 and 2.2.12.

This release addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later.

1.2.2.7.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4.5
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.12

Kiali

1.65.11

1.2.2.8. New features Red Hat OpenShift Service Mesh version 2.4.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.

1.2.2.8.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4.4
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.12

Jaeger

1.47.0

Kiali

1.65.10

1.2.2.9. New features Red Hat OpenShift Service Mesh version 2.4.3
  • The Red Hat OpenShift Service Mesh Operator is now available on ARM-based clusters as a Technology Preview feature.
  • The envoyExtAuthzGrpc field has been added, which is used to configure an external authorization provider using the gRPC API.
  • Common Vulnerabilities and Exposures (CVEs) have been addressed.
  • This release is supported on OpenShift Container Platform 4.10 and newer versions.
1.2.2.9.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4.3
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.10

Jaeger

1.42.0

Kiali

1.65.8

1.2.2.9.2. Red Hat OpenShift Service Mesh operator to ARM-based clusters
Important

Red Hat OpenShift Service Mesh operator to ARM based clusters is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

This release makes the Red Hat OpenShift Service Mesh Operator available on ARM-based clusters as a Technology Preview feature. Images are available for Istio, Envoy, Prometheus, Kiali, and Grafana. Images are not available for Jaeger, so Jaeger must be disabled as a Service Mesh add-on.

1.2.2.9.3. Remote Procedure Calls (gRPC) API support for external authorization configuration

This enhancement adds the envoyExtAuthzGrpc field to configure an external authorization provider using the gRPC API.

1.2.2.10. New features Red Hat OpenShift Service Mesh version 2.4.2

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.10.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4.2
ComponentVersion

Istio

1.16.7

Envoy Proxy

1.24.10

Jaeger

1.42.0

Kiali

1.65.7

1.2.2.11. New features Red Hat OpenShift Service Mesh version 2.4.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.11.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4.1
ComponentVersion

Istio

1.16.5

Envoy Proxy

1.24.8

Jaeger

1.42.0

Kiali

1.65.7

1.2.2.12. New features Red Hat OpenShift Service Mesh version 2.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.12.1. Component versions included in Red Hat OpenShift Service Mesh version 2.4
ComponentVersion

Istio

1.16.5

Envoy Proxy

1.24.8

Jaeger

1.42.0

Kiali

1.65.6

1.2.2.12.2. Cluster-wide deployments

This enhancement introduces a generally available version of cluster-wide deployments. A cluster-wide deployment contains a service mesh control plane that monitors resources for an entire cluster. The control plane uses a single query across all namespaces to monitor each Istio or Kubernetes resource that affects the mesh configuration. Reducing the number of queries the control plane performs in a cluster-wide deployment improves performance.

1.2.2.12.3. Support for discovery selectors

This enhancement introduces a generally available version of the meshConfig.discoverySelectors field, which can be used in cluster-wide deployments to limit the services the service mesh control plane can discover.

spec:
  meshConfig
    discoverySelectors:
    - matchLabels:
        env: prod
        region: us-east1
    - matchExpressions:
      - key: app
        operator: In
        values:
          - cassandra
          - spark
1.2.2.12.4. Integration with cert-manager istio-csr

With this update, Red Hat OpenShift Service Mesh integrates with the cert-manager controller and the istio-csr agent. cert-manager adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing, and using those certificates. cert-manager provides and rotates an intermediate CA certificate for Istio. Integration with istio-csr enables users to delegate signing certificate requests from Istio proxies to cert-manager. ServiceMeshControlPlane v2.4 accepts CA certificates provided by cert-manager as cacerts secret.

Note

Integration with cert-manager and istio-csr is not supported on IBM Power, IBM Z, and IBM® LinuxONE.

1.2.2.12.5. Integration with external authorization systems

This enhancement introduces a generally available method of integrating Red Hat OpenShift Service Mesh with external authorization systems by using the action: CUSTOM field of the AuthorizationPolicy resource. Use the envoyExtAuthzHttp field to delegate the access control to an external authorization system.

1.2.2.12.6. Integration with external Prometheus installation

This enhancement introduces a generally available version of the Prometheus extension provider. You can expose metrics to the OpenShift Container Platform monitoring stack or a custom Prometheus installation by setting the value of the extensionProviders field to prometheus in the spec.meshConfig specification. The telemetry object configures Istio proxies to collect traffic metrics. Service Mesh only supports the Telemetry API for Prometheus metrics.

spec:
  meshConfig:
    extensionProviders:
    - name: prometheus
      prometheus: {}
---
apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: enable-prometheus-metrics
spec:
  metrics:
  - providers:
    - name: prometheus
1.2.2.12.7. Single stack IPv6 support

This enhancement introduces generally available support for single stack IPv6 clusters, providing access to a broader range of IP addresses. Dual stack IPv4 or IPv6 cluster is not supported.

Note

Single stack IPv6 support is not available on IBM Power, IBM Z, and IBM® LinuxONE.

1.2.2.12.8. OpenShift Container Platform Gateway API support
Important

OpenShift Container Platform Gateway API support is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

This enhancement introduces an updated Technology Preview version of the OpenShift Container Platform Gateway API. By default, the OpenShift Container Platform Gateway API is disabled.

1.2.2.12.8.1. Enabling OpenShift Container Platform Gateway API

To enable the OpenShift Container Platform Gateway API, set the value of the enabled field to true in the techPreview.gatewayAPI specification of the ServiceMeshControlPlane resource.

spec:
  techPreview:
    gatewayAPI:
      enabled: true

Previously, environment variables were used to enable the Gateway API.

spec:
  runtime:
    components:
      pilot:
        container:
          env:
            PILOT_ENABLE_GATEWAY_API: "true"
            PILOT_ENABLE_GATEWAY_API_STATUS: "true"
            PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: "true"
1.2.2.12.9. Control plane deployment on infrastructure nodes

Service Mesh control plane deployment is now supported and documented on OpenShift infrastructure nodes. For more information, see the following documentation:

  • Configuring all Service Mesh control plane components to run on infrastructure nodes
  • Configuring individual Service Mesh control plane components to run on infrastructure nodes
1.2.2.12.10. Istio 1.16 support

Service Mesh 2.4 is based on Istio 1.16, which brings in new features and product enhancements. While many Istio 1.16 features are supported, the following exceptions should be noted:

  • HBONE protocol for sidecars is an experimental feature that is not supported.
  • Service Mesh on ARM64 architecture is not supported.
  • OpenTelemetry API remains a Technology Preview feature.
1.2.2.13. New features Red Hat OpenShift Service Mesh version 2.3.12

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.2, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane.

1.2.2.13.1. Component versions for Red Hat OpenShift Service Mesh version 2.3.12
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Kiali

1.57.14

1.2.2.14. New features Red Hat OpenShift Service Mesh version 2.3.11

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.1, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

1.2.2.14.1. Component versions for Red Hat OpenShift Service Mesh version 2.3.11
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Kiali

1.57.14

1.2.2.15. New features Red Hat OpenShift Service Mesh version 2.3.10

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.5.0, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.

1.2.2.15.1. Component versions for Red Hat OpenShift Service Mesh version 2.3.10
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Kiali

1.57.14

1.2.2.16. New features Red Hat OpenShift Service Mesh version 2.3.9

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.4.5, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later.

1.2.2.16.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.9
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Jaeger

1.47.0

Kiali

1.57.14

1.2.2.17. New features Red Hat OpenShift Service Mesh version 2.3.8

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.

1.2.2.17.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.8
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Jaeger

1.47.0

Kiali

1.57.13

1.2.2.18. New features Red Hat OpenShift Service Mesh version 2.3.7

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.18.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.7
ComponentVersion

Istio

1.14.6

Envoy Proxy

1.22.11

Jaeger

1.42.0

Kiali

1.57.11

1.2.2.19. New features Red Hat OpenShift Service Mesh version 2.3.6

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.19.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.6
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.11

Jaeger

1.42.0

Kiali

1.57.10

1.2.2.20. New features Red Hat OpenShift Service Mesh version 2.3.5

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.20.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.5
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.9

Jaeger

1.42.0

Kiali

1.57.10

1.2.2.21. New features Red Hat OpenShift Service Mesh version 2.3.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.21.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.4
ComponentVersion

Istio

1.14.6

Envoy Proxy

1.22.9

Jaeger

1.42.0

Kiali

1.57.9

1.2.2.22. New features Red Hat OpenShift Service Mesh version 2.3.3

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.22.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.3
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.9

Jaeger

1.42.0

Kiali

1.57.7

1.2.2.23. New features Red Hat OpenShift Service Mesh version 2.3.2

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.23.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.2
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.7

Jaeger

1.39

Kiali

1.57.6

1.2.2.24. New features Red Hat OpenShift Service Mesh version 2.3.1

This release of Red Hat OpenShift Service Mesh introduces new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.24.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3.1
ComponentVersion

Istio

1.14.5

Envoy Proxy

1.22.4

Jaeger

1.39

Kiali

1.57.5

1.2.2.25. New features Red Hat OpenShift Service Mesh version 2.3

This release of Red Hat OpenShift Service Mesh introduces new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.25.1. Component versions included in Red Hat OpenShift Service Mesh version 2.3
ComponentVersion

Istio

1.14.3

Envoy Proxy

1.22.4

Jaeger

1.38

Kiali

1.57.3

1.2.2.25.2. New Container Network Interface (CNI) DaemonSet container and ConfigMap

The openshift-operators namespace includes a new istio CNI DaemonSet istio-cni-node-v2-3 and a new ConfigMap resource, istio-cni-config-v2-3.

When upgrading to Service Mesh Control Plane 2.3, the existing istio-cni-node DaemonSet is not changed, and a new istio-cni-node-v2-3 DaemonSet is created.

This name change does not affect previous releases or any istio-cni-node CNI DaemonSet associated with a Service Mesh Control Plane deployed using a previous release.

1.2.2.25.3. Gateway injection support

This release introduces generally available support for Gateway injection. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than the sidecar Envoy proxies running alongside your service workloads. This enables the ability to customize gateway options. When using gateway injection, you must create the following resources in the namespace where you want to run your gateway proxy: Service, Deployment, Role, and RoleBinding.

1.2.2.25.4. Istio 1.14 Support

Service Mesh 2.3 is based on Istio 1.14, which brings in new features and product enhancements. While many Istio 1.14 features are supported, the following exceptions should be noted:

  • ProxyConfig API is supported with the exception of the image field.
  • Telemetry API is a Technology Preview feature.
  • SPIRE runtime is not a supported feature.
1.2.2.25.5. OpenShift Service Mesh Console
Important

OpenShift Service Mesh Console is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

This release introduces a Technology Preview version of the OpenShift Container Platform Service Mesh Console, which integrates the Kiali interface directly into the OpenShift web console. For additional information, see Introducing the OpenShift Service Mesh Console (A Technology Preview)

1.2.2.25.6. Cluster-wide deployment
Important

Cluster-wide deployment is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

This release introduces cluster-wide deployment as a Technology Preview feature. A cluster-wide deployment contains a Service Mesh Control Plane that monitors resources for an entire cluster. The control plane uses a single query across all namespaces to monitor each Istio or Kubernetes resource kind that affects the mesh configuration. In contrast, the multitenant approach uses a query per namespace for each resource kind. Reducing the number of queries the control plane performs in a cluster-wide deployment improves performance.

Note

This cluster-wide deployment documentation is only applicable for control planes deployed using SMCP v2.3. cluster-wide deployments created using SMCP v2.3 are not compatible with cluster-wide deployments created using SMCP v2.4.

1.2.2.25.6.1. Configuring cluster-wide deployment

The following example ServiceMeshControlPlane object configures a cluster-wide deployment.

To create an SMCP for cluster-wide deployment, a user must belong to the cluster-admin ClusterRole. If the SMCP is configured for cluster-wide deployment, it must be the only SMCP in the cluster. You cannot change the control plane mode from multitenant to cluster-wide (or from cluster-wide to multitenant). If a multitenant control plane already exists, delete it and create a new one.

This example configures the SMCP for cluster-wide deployment.

  apiVersion: maistra.io/v2
  kind: ServiceMeshControlPlane
  metadata:
    name: cluster-wide
    namespace: istio-system
  spec:
    version: v2.3
    techPreview:
      controlPlaneMode: ClusterScoped 1
1
Enables Istiod to monitor resources at the cluster level rather than monitor each individual namespace.

Additionally, the SMMR must also be configured for cluster-wide deployment. This example configures the SMMR for cluster-wide deployment.

  apiVersion: maistra.io/v1
  kind: ServiceMeshMemberRoll
  metadata:
    name: default
  spec:
    members:
    - '*' 1
1
Adds all namespaces to the mesh, including any namespaces you subsequently create. The following namespaces are not part of the mesh: kube, openshift, kube-* and openshift-*.
1.2.2.26. New features Red Hat OpenShift Service Mesh version 2.2.12

This release of Red Hat OpenShift Service Mesh is included with the Red Hat OpenShift Service Mesh Operator 2.4.5, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later.

1.2.2.26.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.12
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.47.0

Kiali

1.48.11

1.2.2.27. New features Red Hat OpenShift Service Mesh version 2.2.11

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.

1.2.2.27.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.11
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.47.0

Kiali

1.48.10

1.2.2.28. New features Red Hat OpenShift Service Mesh version 2.2.10

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.28.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.10
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.42.0

Kiali

1.48.8

1.2.2.29. New features Red Hat OpenShift Service Mesh version 2.2.9

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.29.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.9
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.42.0

Kiali

1.48.7

1.2.2.30. New features Red Hat OpenShift Service Mesh version 2.2.8

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.30.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.8
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.42.0

Kiali

1.48.7

1.2.2.31. New features Red Hat OpenShift Service Mesh version 2.2.7

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.

1.2.2.31.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.7
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.42.0

Kiali

1.48.6

1.2.2.32. New features Red Hat OpenShift Service Mesh version 2.2.6

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.32.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.6
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.39

Kiali

1.48.5

1.2.2.33. New features Red Hat OpenShift Service Mesh version 2.2.5

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.33.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.5
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.39

Kiali

1.48.3

1.2.2.34. New features Red Hat OpenShift Service Mesh version 2.2.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.34.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.4
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.36.14

Kiali

1.48.3

1.2.2.35. New features Red Hat OpenShift Service Mesh version 2.2.3

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.35.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.3
ComponentVersion

Istio

1.12.9

Envoy Proxy

1.20.8

Jaeger

1.36

Kiali

1.48.3

1.2.2.36. New features Red Hat OpenShift Service Mesh version 2.2.2

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.36.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.2
ComponentVersion

Istio

1.12.7

Envoy Proxy

1.20.6

Jaeger

1.36

Kiali

1.48.2-1

1.2.2.36.2. Copy route labels

With this enhancement, in addition to copying annotations, you can copy specific labels for an OpenShift route. Red Hat OpenShift Service Mesh copies all labels and annotations present in the Istio Gateway resource (with the exception of annotations starting with kubectl.kubernetes.io) into the managed OpenShift Route resource.

1.2.2.37. New features Red Hat OpenShift Service Mesh version 2.2.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.37.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.1
ComponentVersion

Istio

1.12.7

Envoy Proxy

1.20.6

Jaeger

1.34.1

Kiali

1.48.2-1

1.2.2.38. New features Red Hat OpenShift Service Mesh 2.2

This release of Red Hat OpenShift Service Mesh adds new features and enhancements, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.38.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2
ComponentVersion

Istio

1.12.7

Envoy Proxy

1.20.4

Jaeger

1.34.1

Kiali

1.48.0.16

1.2.2.38.2. WasmPlugin API

This release adds support for the WasmPlugin API and deprecates the ServiceMeshExtension API.

1.2.2.38.3. ROSA support

This release introduces service mesh support for Red Hat OpenShift on AWS (ROSA), including multi-cluster federation.

1.2.2.38.4. istio-node DaemonSet renamed

This release, the istio-node DaemonSet is renamed to istio-cni-node to match the name in upstream Istio.

1.2.2.38.5. Envoy sidecar networking changes

Istio 1.10 updated Envoy to send traffic to the application container using eth0 rather than lo by default.

1.2.2.38.6. Service Mesh Control Plane 1.1

This release marks the end of support for Service Mesh Control Planes based on Service Mesh 1.1 for all platforms.

1.2.2.38.7. Istio 1.12 Support

Service Mesh 2.2 is based on Istio 1.12, which brings in new features and product enhancements. While many Istio 1.12 features are supported, the following unsupported features should be noted:

  • AuthPolicy Dry Run is a tech preview feature.
  • gRPC Proxyless Service Mesh is a tech preview feature.
  • Telemetry API is a tech preview feature.
  • Discovery selectors is not a supported feature.
  • External control plane is not a supported feature.
  • Gateway injection is not a supported feature.
1.2.2.38.8. Kubernetes Gateway API
Important

Kubernetes Gateway API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Kubernetes Gateway API is a technology preview feature that is disabled by default. If the Kubernetes API deployment controller is disabled, you must manually deploy and link an ingress gateway to the created Gateway object.

If the Kubernetes API deployment controller is enabled, then an ingress gateway automatically deploys when a Gateway object is created.

1.2.2.38.8.1. Installing the Gateway API CRDs

The Gateway API CRDs do not come preinstalled by default on OpenShift clusters. Install the CRDs prior to enabling Gateway API support in the SMCP.

$ kubectl get crd gateways.gateway.networking.k8s.io || { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.4.0" | kubectl apply -f -; }
1.2.2.38.8.2. Enabling Kubernetes Gateway API

To enable the feature, set the following environment variables for the Istiod container in ServiceMeshControlPlane:

spec:
  runtime:
    components:
      pilot:
        container:
          env:
            PILOT_ENABLE_GATEWAY_API: "true"
            PILOT_ENABLE_GATEWAY_API_STATUS: "true"
            # and optionally, for the deployment controller
            PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: "true"

Restricting route attachment on Gateway API listeners is possible using the SameNamespace or All settings. Istio ignores usage of label selectors in listeners.allowedRoutes.namespaces and reverts to the default behavior (SameNamespace).

1.2.2.38.8.3. Manually linking an existing gateway to a Gateway resource

If the Kubernetes API deployment controller is disabled, you must manually deploy and then link an ingress gateway to the created Gateway resource.

  apiVersion: gateway.networking.k8s.io/v1alpha2
  kind: Gateway
  metadata:
    name: gateway
  spec:
    addresses:
    - value: ingress.istio-gateways.svc.cluster.local
      type: Hostname
1.2.2.39. New features Red Hat OpenShift Service Mesh 2.1.6

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.39.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.6
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.5

Jaeger

1.36

Kiali

1.36.16

1.2.2.40. New features Red Hat OpenShift Service Mesh 2.1.5.2

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.40.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.5.2
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.5

Jaeger

1.36

Kiali

1.24.17

1.2.2.41. New features Red Hat OpenShift Service Mesh 2.1.5.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.41.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.5.1
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.5

Jaeger

1.36

Kiali

1.36.13

1.2.2.42. New features Red Hat OpenShift Service Mesh 2.1.5

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.

1.2.2.42.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.5
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.36

Kiali

1.36.12-1

1.2.2.43. New features Red Hat OpenShift Service Mesh 2.1.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.43.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.4
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.30.2

Kiali

1.36.12-1

1.2.2.44. New features Red Hat OpenShift Service Mesh 2.1.3

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.44.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.3
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.30.2

Kiali

1.36.10-2

1.2.2.45. New features Red Hat OpenShift Service Mesh 2.1.2.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.45.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.2.1
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.30.2

Kiali

1.36.9

1.2.2.46. New features Red Hat OpenShift Service Mesh 2.1.2

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

With this release, the Red Hat OpenShift distributed tracing platform (Jaeger) Operator is now installed to the openshift-distributed-tracing namespace by default. Previously the default installation had been in the openshift-operator namespace.

1.2.2.46.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.2
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.30.1

Kiali

1.36.8

1.2.2.47. New features Red Hat OpenShift Service Mesh 2.1.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

This release also adds the ability to disable the automatic creation of network policies.

1.2.2.47.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.1
ComponentVersion

Istio

1.9.9

Envoy Proxy

1.17.1

Jaeger

1.24.1

Kiali

1.36.7

1.2.2.47.2. Disabling network policies

Red Hat OpenShift Service Mesh automatically creates and manages a number of NetworkPolicies resources in the Service Mesh control plane and application namespaces. This is to ensure that applications and the control plane can communicate with each other.

If you want to disable the automatic creation and management of NetworkPolicies resources, for example to enforce company security policies, you can do so. You can edit the ServiceMeshControlPlane to set the spec.security.manageNetworkPolicy setting to false

Note

When you disable spec.security.manageNetworkPolicy Red Hat OpenShift Service Mesh will not create any NetworkPolicy objects. The system administrator is responsible for managing the network and fixing any issues this might cause.

Procedure

  1. In the OpenShift Container Platform web console, click OperatorsInstalled Operators.
  2. Select the project where you installed the Service Mesh control plane, for example istio-system, from the Project menu.
  3. Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your ServiceMeshControlPlane, for example basic-install.
  4. On the Create ServiceMeshControlPlane Details page, click YAML to modify your configuration.
  5. Set the ServiceMeshControlPlane field spec.security.manageNetworkPolicy to false, as shown in this example.

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    spec:
      security:
          trust:
          manageNetworkPolicy: false
  6. Click Save.
1.2.2.48. New features and enhancements Red Hat OpenShift Service Mesh 2.1

This release of Red Hat OpenShift Service Mesh adds support for Istio 1.9.8, Envoy Proxy 1.17.1, Jaeger 1.24.1, and Kiali 1.36.5 on OpenShift Container Platform 4.6 EUS, 4.7, 4.8, 4.9, along with new features and enhancements.

1.2.2.48.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1
ComponentVersion

Istio

1.9.6

Envoy Proxy

1.17.1

Jaeger

1.24.1

Kiali

1.36.5

1.2.2.48.2. Service Mesh Federation

New Custom Resource Definitions (CRDs) have been added to support federating service meshes. Service meshes may be federated both within the same cluster or across different OpenShift clusters. These new resources include:

  • ServiceMeshPeer - Defines a federation with a separate service mesh, including gateway configuration, root trust certificate configuration, and status fields. In a pair of federated meshes, each mesh will define its own separate ServiceMeshPeer resource.
  • ExportedServiceMeshSet - Defines which services for a given ServiceMeshPeer are available for the peer mesh to import.
  • ImportedServiceSet - Defines which services for a given ServiceMeshPeer are imported from the peer mesh. These services must also be made available by the peer’s ExportedServiceMeshSet resource.

Service Mesh Federation is not supported between clusters on Red Hat OpenShift Service on AWS (ROSA), Azure Red Hat OpenShift (ARO), or OpenShift Dedicated (OSD).

1.2.2.48.3. OVN-Kubernetes Container Network Interface (CNI) generally available

The OVN-Kubernetes Container Network Interface (CNI) was previously introduced as a Technology Preview feature in Red Hat OpenShift Service Mesh 2.0.1 and is now generally available in Red Hat OpenShift Service Mesh 2.1 and 2.0.x for use on OpenShift Container Platform 4.7.32, OpenShift Container Platform 4.8.12, and OpenShift Container Platform 4.9.

1.2.2.48.4. Service Mesh WebAssembly (WASM) Extensions

The ServiceMeshExtensions Custom Resource Definition (CRD), first introduced in 2.0 as Technology Preview, is now generally available. You can use CRD to build your own plugins, but Red Hat does not provide support for the plugins you create.

Mixer has been completely removed in Service Mesh 2.1. Upgrading from a Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled. Mixer plugins will need to be ported to WebAssembly Extensions.

1.2.2.48.5. 3scale WebAssembly Adapter (WASM)

With Mixer now officially removed, OpenShift Service Mesh 2.1 does not support the 3scale mixer adapter. Before upgrading to Service Mesh 2.1, remove the Mixer-based 3scale adapter and any additional Mixer plugins. Then, manually install and configure the new 3scale WebAssembly adapter with Service Mesh 2.1+ using a ServiceMeshExtension resource.

3scale 2.11 introduces an updated Service Mesh integration based on WebAssembly.

1.2.2.48.6. Istio 1.9 Support

Service Mesh 2.1 is based on Istio 1.9, which brings in a large number of new features and product enhancements. While the majority of Istio 1.9 features are supported, the following exceptions should be noted:

  • Virtual Machine integration is not yet supported
  • Kubernetes Gateway API is not yet supported
  • Remote fetch and load of WebAssembly HTTP filters are not yet supported
  • Custom CA Integration using the Kubernetes CSR API is not yet supported
  • Request Classification for monitoring traffic is a tech preview feature
  • Integration with external authorization systems via Authorization policy’s CUSTOM action is a tech preview feature
1.2.2.48.7. Improved Service Mesh operator performance

The amount of time Red Hat OpenShift Service Mesh uses to prune old resources at the end of every ServiceMeshControlPlane reconciliation has been reduced. This results in faster ServiceMeshControlPlane deployments, and allows changes applied to existing SMCPs to take effect more quickly.

1.2.2.48.8. Kiali updates

Kiali 1.36 includes the following features and enhancements:

  • Service Mesh troubleshooting functionality

    • Control plane and gateway monitoring
    • Proxy sync statuses
    • Envoy configuration views
    • Unified view showing Envoy proxy and application logs interleaved
  • Namespace and cluster boxing to support federated service mesh views
  • New validations, wizards, and distributed tracing enhancements
1.2.2.49. New features Red Hat OpenShift Service Mesh 2.0.11.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.

1.2.2.49.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.11.1
ComponentVersion

Istio

1.6.14

Envoy Proxy

1.14.5

Jaeger

1.36

Kiali

1.24.17

1.2.2.50. New features Red Hat OpenShift Service Mesh 2.0.11

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.

1.2.2.50.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.11
ComponentVersion

Istio

1.6.14

Envoy Proxy

1.14.5

Jaeger

1.36

Kiali

1.24.16-1

1.2.2.51. New features Red Hat OpenShift Service Mesh 2.0.10

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.51.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.10
ComponentVersion

Istio

1.6.14

Envoy Proxy

1.14.5

Jaeger

1.28.0

Kiali

1.24.16-1

1.2.2.52. New features Red Hat OpenShift Service Mesh 2.0.9

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.52.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.9
ComponentVersion

Istio

1.6.14

Envoy Proxy

1.14.5

Jaeger

1.24.1

Kiali

1.24.11

1.2.2.53. New features Red Hat OpenShift Service Mesh 2.0.8

This release of Red Hat OpenShift Service Mesh addresses bug fixes.

1.2.2.54. New features Red Hat OpenShift Service Mesh 2.0.7.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs).

1.2.2.54.1. Change in how Red Hat OpenShift Service Mesh handles URI fragments

Red Hat OpenShift Service Mesh contains a remotely exploitable vulnerability, CVE-2021-39156, where an HTTP request with a fragment (a section in the end of a URI that begins with a # character) in the URI path could bypass the Istio URI path-based authorization policies. For instance, an Istio authorization policy denies requests sent to the URI path /user/profile. In the vulnerable versions, a request with URI path /user/profile#section1 bypasses the deny policy and routes to the backend (with the normalized URI path /user/profile%23section1), possibly leading to a security incident.

You are impacted by this vulnerability if you use authorization policies with DENY actions and operation.paths, or ALLOW actions and operation.notPaths.

With the mitigation, the fragment part of the request’s URI is removed before the authorization and routing. This prevents a request with a fragment in its URI from bypassing authorization policies which are based on the URI without the fragment part.

To opt-out from the new behavior in the mitigation, the fragment section in the URI will be kept. You can configure your ServiceMeshControlPlane to keep URI fragments.

Warning

Disabling the new behavior will normalize your paths as described above and is considered unsafe. Ensure that you have accommodated for this in any security policies before opting to keep URI fragments.

Example ServiceMeshControlPlane modification

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
spec:
  techPreview:
    meshConfig:
      defaultConfig:
        proxyMetadata: HTTP_STRIP_FRAGMENT_FROM_PATH_UNSAFE_IF_DISABLED: "false"

1.2.2.54.2. Required update for authorization policies

Istio generates hostnames for both the hostname itself and all matching ports. For instance, a virtual service or Gateway for a host of "httpbin.foo" generates a config matching "httpbin.foo and httpbin.foo:*". However, exact match authorization policies only match the exact string given for the hosts or notHosts fields.

Your cluster is impacted if you have AuthorizationPolicy resources using exact string comparison for the rule to determine hosts or notHosts.

You must update your authorization policy rules to use prefix match instead of exact match. For example, replacing hosts: ["httpbin.com"] with hosts: ["httpbin.com:*"] in the first AuthorizationPolicy example.

First example AuthorizationPolicy using prefix match

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: httpbin
  namespace: foo
spec:
  action: DENY
  rules:
  - from:
    - source:
        namespaces: ["dev"]
    to:
    - operation:
        hosts: [“httpbin.com”,"httpbin.com:*"]

Second example AuthorizationPolicy using prefix match

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: httpbin
  namespace: default
spec:
  action: DENY
  rules:
  - to:
    - operation:
        hosts: ["httpbin.example.com:*"]

1.2.2.55. New features Red Hat OpenShift Service Mesh 2.0.7

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.56. Red Hat OpenShift Service Mesh on Red Hat OpenShift Dedicated and Microsoft Azure Red Hat OpenShift

Red Hat OpenShift Service Mesh is now supported through Red Hat OpenShift Dedicated and Microsoft Azure Red Hat OpenShift.

1.2.2.57. New features Red Hat OpenShift Service Mesh 2.0.6

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.58. New features Red Hat OpenShift Service Mesh 2.0.5

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.59. New features Red Hat OpenShift Service Mesh 2.0.4

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

Important

There are manual steps that must be completed to address CVE-2021-29492 and CVE-2021-31920.

1.2.2.59.1. Manual updates required by CVE-2021-29492 and CVE-2021-31920

Istio contains a remotely exploitable vulnerability where an HTTP request path with multiple slashes or escaped slash characters (%2F or %5C) could potentially bypass an Istio authorization policy when path-based authorization rules are used.

For example, assume an Istio cluster administrator defines an authorization DENY policy to reject the request at path /admin. A request sent to the URL path //admin will NOT be rejected by the authorization policy.

According to RFC 3986, the path //admin with multiple slashes should technically be treated as a different path from the /admin. However, some backend services choose to normalize the URL paths by merging multiple slashes into a single slash. This can result in a bypass of the authorization policy (//admin does not match /admin), and a user can access the resource at path /admin in the backend; this would represent a security incident.

Your cluster is impacted by this vulnerability if you have authorization policies using ALLOW action + notPaths field or DENY action + paths field patterns. These patterns are vulnerable to unexpected policy bypasses.

Your cluster is NOT impacted by this vulnerability if:

  • You don’t have authorization policies.
  • Your authorization policies don’t define paths or notPaths fields.
  • Your authorization policies use ALLOW action + paths field or DENY action + notPaths field patterns. These patterns could only cause unexpected rejection instead of policy bypasses. The upgrade is optional for these cases.
Note

The Red Hat OpenShift Service Mesh configuration location for path normalization is different from the Istio configuration.

1.2.2.59.2. Updating the path normalization configuration

Istio authorization policies can be based on the URL paths in the HTTP request. Path normalization, also known as URI normalization, modifies and standardizes the incoming requests' paths so that the normalized paths can be processed in a standard way. Syntactically different paths may be equivalent after path normalization.

Istio supports the following normalization schemes on the request paths before evaluating against the authorization policies and routing the requests:

Table 1.1. Normalization schemes
OptionDescriptionExampleNotes

NONE

No normalization is done. Anything received by Envoy will be forwarded exactly as-is to any backend service.

../%2Fa../b is evaluated by the authorization policies and sent to your service.

This setting is vulnerable to CVE-2021-31920.

BASE

This is currently the option used in the default installation of Istio. This applies the normalize_path option on Envoy proxies, which follows RFC 3986 with extra normalization to convert backslashes to forward slashes.

/a/../b is normalized to /b. \da is normalized to /da.

This setting is vulnerable to CVE-2021-31920.

MERGE_SLASHES

Slashes are merged after the BASE normalization.

/a//b is normalized to /a/b.

Update to this setting to mitigate CVE-2021-31920.

DECODE_AND_MERGE_SLASHES

The strictest setting when you allow all traffic by default. This setting is recommended, with the caveat that you must thoroughly test your authorization policies routes. Percent-encoded slash and backslash characters (%2F, %2f, %5C and %5c) are decoded to / or \, before the MERGE_SLASHES normalization.

/a%2fb is normalized to /a/b.

Update to this setting to mitigate CVE-2021-31920. This setting is more secure, but also has the potential to break applications. Test your applications before deploying to production.

The normalization algorithms are conducted in the following order:

  1. Percent-decode %2F, %2f, %5C and %5c.
  2. The RFC 3986 and other normalization implemented by the normalize_path option in Envoy.
  3. Merge slashes.
Warning

While these normalization options represent recommendations from HTTP standards and common industry practices, applications may interpret a URL in any way it chooses to. When using denial policies, ensure that you understand how your application behaves.

1.2.2.59.3. Path normalization configuration examples

Ensuring Envoy normalizes request paths to match your backend services' expectations is critical to the security of your system. The following examples can be used as a reference for you to configure your system. The normalized URL paths, or the original URL paths if NONE is selected, will be:

  1. Used to check against the authorization policies.
  2. Forwarded to the backend application.
Table 1.2. Configuration examples
If your application…​Choose…​

Relies on the proxy to do normalization

BASE, MERGE_SLASHES or DECODE_AND_MERGE_SLASHES

Normalizes request paths based on RFC 3986 and does not merge slashes.

BASE

Normalizes request paths based on RFC 3986 and merges slashes, but does not decode percent-encoded slashes.

MERGE_SLASHES

Normalizes request paths based on RFC 3986, decodes percent-encoded slashes, and merges slashes.

DECODE_AND_MERGE_SLASHES

Processes request paths in a way that is incompatible with RFC 3986.

NONE

1.2.2.59.4. Configuring your SMCP for path normalization

To configure path normalization for Red Hat OpenShift Service Mesh, specify the following in your ServiceMeshControlPlane. Use the configuration examples to help determine the settings for your system.

SMCP v2 pathNormalization

spec:
  techPreview:
    global:
      pathNormalization: <option>

1.2.2.59.5. Configuring for case normalization

In some environments, it may be useful to have paths in authorization policies compared in a case insensitive manner. For example, treating https://myurl/get and https://myurl/GeT as equivalent. In those cases, you can use the EnvoyFilter shown below. This filter will change both the path used for comparison and the path presented to the application. In this example, istio-system is the name of the Service Mesh control plane project.

Save the EnvoyFilter to a file and run the following command:

$ oc create -f <myEnvoyFilterFile>
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingress-case-insensitive
  namespace: istio-system
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.lua
        typed_config:
            "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
            inlineCode: |
              function envoy_on_request(request_handle)
                local path = request_handle:headers():get(":path")
                request_handle:headers():replace(":path", string.lower(path))
              end
1.2.2.60. New features Red Hat OpenShift Service Mesh 2.0.3

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

In addition, this release has the following new features:

  • Added an option to the must-gather data collection tool that gathers information from a specified Service Mesh control plane namespace. For more information, see OSSM-351.
  • Improved performance for Service Mesh control planes with hundreds of namespaces
1.2.2.61. New features Red Hat OpenShift Service Mesh 2.0.2

This release of Red Hat OpenShift Service Mesh adds support for IBM Z and IBM Power Systems. It also addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.62. New features Red Hat OpenShift Service Mesh 2.0.1

This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.

1.2.2.63. New features Red Hat OpenShift Service Mesh 2.0

This release of Red Hat OpenShift Service Mesh adds support for Istio 1.6.5, Jaeger 1.20.0, Kiali 1.24.2, and the 3scale Istio Adapter 2.0 and OpenShift Container Platform 4.6.

In addition, this release has the following new features:

  • Simplifies installation, upgrades, and management of the Service Mesh control plane.
  • Reduces the Service Mesh control plane’s resource usage and startup time.
  • Improves performance by reducing inter-control plane communication over networking.

    • Adds support for Envoy’s Secret Discovery Service (SDS). SDS is a more secure and efficient mechanism for delivering secrets to Envoy side car proxies.
  • Removes the need to use Kubernetes Secrets, which have well known security risks.
  • Improves performance during certificate rotation, as proxies no longer require a restart to recognize new certificates.

    • Adds support for Istio’s Telemetry v2 architecture, which is built using WebAssembly extensions. This new architecture brings significant performance improvements.
    • Updates the ServiceMeshControlPlane resource to v2 with a streamlined configuration to make it easier to manage the Service Mesh Control Plane.
    • Introduces WebAssembly extensions as a Technology Preview feature.

1.2.3. Technology Preview

Some features in this release are currently in Technology Preview. These experimental features are not intended for production use.

Important

Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

1.2.4. Deprecated and removed features

Some features available in previous releases have been deprecated or removed.

Deprecated functionality is still included in OpenShift Container Platform and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.

Removed functionality no longer exists in the product.

1.2.4.1. Deprecated and removed features in Red Hat OpenShift Service Mesh 2.5

The v2.2 ServiceMeshControlPlane resource is no longer supported. Customers should update their mesh deployments to use a later version of the ServiceMeshControlPlane resource.

Support for the Red Hat OpenShift distributed tracing platform (Jaeger) Operator is deprecated. To collect trace spans, use the Red Hat OpenShift distributed tracing platform (Tempo) Stack.

Support for the OpenShift Elasticsearch Operator is deprecated.

Istio will remove support for first-party JSON Web Tokens (JWTs). Istio will still support third-Party JWTs.

1.2.4.2. Deprecated and removed features in Red Hat OpenShift Service Mesh 2.4

The v2.1 ServiceMeshControlPlane resource is no longer supported. Customers should upgrade their mesh deployments to use a later version of the ServiceMeshControlPlane resource.

Support for Istio OpenShift Routing (IOR) is deprecated and will be removed in a future release.

Support for Grafana is deprecated and will be removed in a future release.

Support for the following cipher suites, which were deprecated in Red Hat OpenShift Service Mesh 2.3, has been removed from the default list of ciphers used in TLS negotiations on both the client and server sides. Applications that require access to services requiring one of these cipher suites will fail to connect when a TLS connection is initiated from the proxy.

  • ECDHE-ECDSA-AES128-SHA
  • ECDHE-RSA-AES128-SHA
  • AES128-GCM-SHA256
  • AES128-SHA
  • ECDHE-ECDSA-AES256-SHA
  • ECDHE-RSA-AES256-SHA
  • AES256-GCM-SHA384
  • AES256-SHA
1.2.4.3. Deprecated and removed features in Red Hat OpenShift Service Mesh 2.3

Support for the following cipher suites has been deprecated. In a future release, they will be removed from the default list of ciphers used in TLS negotiations on both the client and server sides.

  • ECDHE-ECDSA-AES128-SHA
  • ECDHE-RSA-AES128-SHA
  • AES128-GCM-SHA256
  • AES128-SHA
  • ECDHE-ECDSA-AES256-SHA
  • ECDHE-RSA-AES256-SHA
  • AES256-GCM-SHA384
  • AES256-SHA

The ServiceMeshExtension API, which was deprecated in Red Hat OpenShift Service Mesh version 2.2, was removed in Red Hat OpenShift Service Mesh version 2.3. If you are using the ServiceMeshExtension API, you must migrate to the WasmPlugin API to continue using your WebAssembly extensions.

1.2.4.4. Deprecated features in Red Hat OpenShift Service Mesh 2.2

The ServiceMeshExtension API is deprecated as of release 2.2 and will be removed in a future release. While ServiceMeshExtension API is still supported in release 2.2, customers should start moving to the new WasmPlugin API.

1.2.4.5. Removed features in Red Hat OpenShift Service Mesh 2.2

This release marks the end of support for Service Mesh control planes based on Service Mesh 1.1 for all platforms.

1.2.4.6. Removed features in Red Hat OpenShift Service Mesh 2.1

In Service Mesh 2.1, the Mixer component is removed. Bug fixes and support is provided through the end of the Service Mesh 2.0 life cycle.

Upgrading from a Service Mesh 2.0.x release to 2.1 will not proceed if Mixer plugins are enabled. Mixer plugins must be ported to WebAssembly Extensions.

1.2.4.7. Deprecated features in Red Hat OpenShift Service Mesh 2.0

The Mixer component was deprecated in release 2.0 and will be removed in release 2.1. While using Mixer for implementing extensions was still supported in release 2.0, extensions should have been migrated to the new WebAssembly mechanism.

The following resource types are no longer supported in Red Hat OpenShift Service Mesh 2.0:

  • Policy (authentication.istio.io/v1alpha1) is no longer supported. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.

    • Use RequestAuthentication (security.istio.io/v1beta1)
    • Use PeerAuthentication (security.istio.io/v1beta1)
  • ServiceMeshPolicy (maistra.io/v1) is no longer supported.

    • Use RequestAuthentication or PeerAuthentication, as mentioned above, but place in the Service Mesh control plane namespace.
  • RbacConfig (rbac.istio.io/v1alpha1) is no longer supported.

    • Replaced by AuthorizationPolicy (security.istio.io/v1beta1), which encompasses behavior of RbacConfig, ServiceRole, and ServiceRoleBinding.
  • ServiceMeshRbacConfig (maistra.io/v1) is no longer supported.

    • Use AuthorizationPolicy as above, but place in Service Mesh control plane namespace.
  • ServiceRole (rbac.istio.io/v1alpha1) is no longer supported.
  • ServiceRoleBinding (rbac.istio.io/v1alpha1) is no longer supported.
  • In Kiali, the login and LDAP strategies are deprecated. A future version will introduce authentication using OpenID providers.

1.2.5. Known issues

These limitations exist in Red Hat OpenShift Service Mesh:

  • Red Hat OpenShift Service Mesh does not yet fully support IPv6. As a result, Red Hat OpenShift Service Mesh does not support dual-stack clusters.
  • Graph layout - The layout for the Kiali graph can render differently, depending on your application architecture and the data to display (number of graph nodes and their interactions). Because it is difficult if not impossible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts. To choose a different layout, you can choose a different Layout Schema from the Graph Settings menu.
  • The first time you access related services such as distributed tracing platform (Jaeger) and Grafana, from the Kiali console, you must accept the certificate and re-authenticate using your OpenShift Container Platform login credentials. This happens due to an issue with how the framework displays embedded pages in the console.
  • The Bookinfo sample application cannot be installed on IBM Power, IBM Z, and IBM® LinuxONE.
  • WebAssembly extensions are not supported on IBM Power, IBM Z, and IBM® LinuxONE.
  • LuaJIT is not supported on IBM Power, IBM Z, and IBM® LinuxONE.
  • Single stack IPv6 support is not available on IBM Power, IBM Z, and IBM® LinuxONE.
1.2.5.1. Service Mesh known issues

These are the known issues in Red Hat OpenShift Service Mesh:

  • OSSM-6267 After a data source is configured correctly in the Grafana, a data query returns authentication error. Users are not able to view data in the Istio service and Istio workload dashboards. Currently, no workaround exists for this issue.
  • OSSM-5556 Gateways are skipped when istio-system labels do not match discovery selectors.

    Workaround: Label the control plane namespace to match discovery selectors to avoid skipping the Gateway configurations.

    Example ServiceMeshControlPlane resource

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
      namespace: istio-system
    spec:
      mode: ClusterWide
      meshConfig:
        discoverySelectors:
        - matchLabels:
            istio-discovery: enabled
      gateways:
        ingress:
          enabled: true

    Then, run the following command at the command line:

    oc label namespace istio-system istio-discovery=enabled
  • OSSM-3890 Attempting to use the Gateway API in a multitenant mesh deployment generates an error message similar to the following:

    2023-05-02T15:20:42.541034Z	error	watch error in cluster Kubernetes: failed to list *v1alpha2.TLSRoute: the server could not find the requested resource (get tlsroutes.gateway.networking.k8s.io)
    2023-05-02T15:20:42.616450Z	info	kube	controller "gateway.networking.k8s.io/v1alpha2/TCPRoute" is syncing...

    To support Gateway API in a multitenant mesh deployment, all Gateway API Custom Resource Definition (CRD) files must be present in the cluster.

    In a multitenant mesh deployment, CRD scan is disabled, and Istio has no way to discover which CRDs are present in a cluster. As a result, Istio attempts to watch all supported Gateway API CRDs, but generates errors if some of those CRDs are not present.

    Service Mesh 2.3.1 and later versions support both v1alpha2 and v1beta1 CRDs. Therefore, both CRD versions must be present for a multitenant mesh deployment to support the Gateway API.

    Workaround: In the following example, the kubectl get operation installs the v1alpha2 and v1beta1 CRDs. Note the URL contains the additional experimental segment and updates any of your existing scripts accordingly:

    $ kubectl get crd gateways.gateway.networking.k8s.io ||   { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v0.5.1" | kubectl apply -f -; }
  • OSSM-2042 Deployment of SMCP named default fails. If you are creating an SMCP object, and set its version field to v2.3, the name of the object cannot be default. If the name is default, then the control plane fails to deploy, and OpenShift generates a Warning event with the following message:

    Error processing component mesh-config: error: [mesh-config/templates/telemetryv2_1.6.yaml: Internal error occurred: failed calling webhook "rev.validation.istio.io": Post "https://istiod-default.istio-system.svc:443/validate?timeout=10s": x509: certificate is valid for istiod.istio-system.svc, istiod-remote.istio-system.svc, istio-pilot.istio-system.svc, not istiod-default.istio-system.svc, mesh-config/templates/enable-mesh-permissive.yaml

  • OSSM-1655 Kiali dashboard shows error after enabling mTLS in SMCP.

    After enabling the spec.security.controlPlane.mtls setting in the SMCP, the Kiali console displays the following error message No subsets defined.

  • OSSM-1505 This issue only occurs when using the ServiceMeshExtension resource on OpenShift Container Platform 4.11. When you use ServiceMeshExtension on OpenShift Container Platform 4.11 the resource never becomes ready. If you inspect the issue using oc describe ServiceMeshExtension you will see the following error: stderr: Error creating mount namespace before pivot: function not implemented.

    Workaround: ServiceMeshExtension was deprecated in Service Mesh 2.2. Migrate from ServiceMeshExtension to the WasmPlugin resource. For more information, see Migrating from ServiceMeshExtension to WasmPlugin resources.

  • OSSM-1396 If a gateway resource contains the spec.externalIPs setting, instead of being recreated when the ServiceMeshControlPlane is updated, the gateway is removed and never recreated.
  • OSSM-1168 When service mesh resources are created as a single YAML file, the Envoy proxy sidecar is not reliably injected into pods. When the SMCP, SMMR, and Deployment resources are created individually, the deployment works as expected.
  • OSSM-1115 The concurrency field of the spec.proxy API did not propagate to the istio-proxy. The concurrency field works when set with ProxyConfig. The concurrency field specifies the number of worker threads to run. If the field is set to 0, then the number of worker threads available is equal to the number of CPU cores. If the field is not set, then the number of worker threads available defaults to 2.

    In the following example, the concurrency field is set to 0.

    apiVersion: networking.istio.io/v1beta1
    kind: ProxyConfig
    metadata:
      name: mesh-wide-concurrency
      namespace: <istiod-namespace>
    spec:
      concurrency: 0
  • OSSM-1052 When configuring a Service ExternalIP for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.

    Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).

  • OSSM-882 This applies for Service Mesh 2.1 and earlier. Namespace is in the accessible_namespace list but does not appear in Kiali UI. By default, Kiali will not show any namespaces that start with "kube" because these namespaces are typically internal-use only and not part of a mesh.

    For example, if you create a namespace called 'akube-a' and add it to the Service Mesh member roll, then the Kiali UI does not display the namespace. For defined exclusion patterns, the software excludes namespaces that start with or contain the pattern.

    Workaround: Change the Kiali Custom Resource setting so it prefixes the setting with a carat (^). For example:

    api:
      namespaces:
        exclude:
        - "^istio-operator"
        - "^kube-.*"
        - "^openshift.*"
        - "^ibm.*"
        - "^kiali-operator"
  • MAISTRA-2692 With Mixer removed, custom metrics that have been defined in Service Mesh 2.0.x cannot be used in 2.1. Custom metrics can be configured using EnvoyFilter. Red Hat is unable to support EnvoyFilter configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained.
  • MAISTRA-2648 Service mesh extensions are currently not compatible with meshes deployed on IBM Z.
  • MAISTRA-1959 Migration to 2.0 Prometheus scraping (spec.addons.prometheus.scrape set to true) does not work when mTLS is enabled. Additionally, Kiali displays extraneous graph data when mTLS is disabled.

    This problem can be addressed by excluding port 15020 from proxy configuration, for example,

    spec:
      proxy:
        networking:
          trafficControl:
            inbound:
              excludedPorts:
              - 15020
  • MAISTRA-453 If you create a new project and deploy pods immediately, sidecar injection does not occur. The operator fails to add the maistra.io/member-of before the pods are created, therefore the pods must be deleted and recreated for sidecar injection to occur.
  • MAISTRA-158 Applying multiple gateways referencing the same hostname will cause all gateways to stop functioning.
1.2.5.2. Kiali known issues
Note

New issues for Kiali should be created in the OpenShift Service Mesh project with the Component set to Kiali.

These are the known issues in Kiali:

  • KIALI-2206 When you are accessing the Kiali console for the first time, and there is no cached browser data for Kiali, the “View in Grafana” link on the Metrics tab of the Kiali Service Details page redirects to the wrong location. The only way you would encounter this issue is if you are accessing Kiali for the first time.
  • KIALI-507 Kiali does not support Internet Explorer 11. This is because the underlying frameworks do not support Internet Explorer. To access the Kiali console, use one of the two most recent versions of the Chrome, Edge, Firefox or Safari browser.

1.2.6. Fixed issues

The following issues have been resolved in the current release:

  • OSSM-6331 Previously, the smcp.general.logging.componentLevels spec accepted invalid LogLevel values, and the ServiceMeshControlPlane resource was still created. Now, the terminal shows an error message if an invalid value is used, and the control plane is not created.
  • OSSM-6290 Previously, the Project filter of the Istio Config list page did not work correctly. All istio config items were displayed from all namespaces even if you selected a specific project from the drop-down menu. Now, only the istio config items that belong to the selected project in the filter dropdown are displayed.
  • OSSM-6298 Previously, when you clicked an item reference within the OpenShift Service Mesh Console (OSSMC) plugin, the console sometimes performed multiple redirects before opening the desired page. As a result, navigating back to the previous page that was open in the console caused your web browser to open the wrong page. Now, these redirects do not occur, and clicking Back in a web browser brings you to the correct page.
  • OSSM-6299 Previously, in OpenShift Container Platform 4.15, when you clicked the Node graph menu option of any node menu within the traffic graph, the node graph was not displayed. Instead, the page refreshed with the same traffic graph. Now, clicking the Node graph menu option correctly displays the node graph.

The following issues have been resolved in previous releases:

1.2.6.1. Service Mesh fixed issues
  • OSSM-6177 Previously, when validation messages were enabled in the ServiceMeshControlPlane (SMCP), the istiod crashed continuously unless GatewayAPI support was enabled. Now, when validation messages are enabled but GatewayAPI support is not, the istiod does not continuously crash.
  • OSSM-6163 Resolves the following issues:

    • Previously, an unstable Prometheus image was included in the Service Mesh control plane (SMCP) v2.5, and users were not able to access the Prometheus dashboard. Now, in the Service Mesh operator 2.5.1, the Prometheus image has been updated.
    • Previously, in the Service Mesh control plane (SMCP), a Grafana data source was not able to set Basic authentication password automatically and users were not able to view metrics from Prometheus in Grafana mesh dashboards. Now, a Grafana data source password is configured under the secureJsonData field. Metrics are displayed correctly in dashboards.
  • OSSM-6148 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not respond when the user clicked any option in the menu of any node on the Traffic Graph page. Now, the plugin responds to the selected option in the menu by redirecting to the corresponding details page.
  • OSSM-6099 Previously, the OpenShift Service Mesh Console (OSSMC) plugin failed to load correctly in an IPv6 cluster. Now, the OSSMC plugin configuration has been modified to ensure proper loading in an IPv6 cluster.
  • OSSM-5960 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not display notification messages such as backend errors or Istio validations. Now, these notifications are displayed correctly at the top of the plugin page.
  • OSSM-5959 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not display TLS and Istio certification information in the Overview page. Now, this information is displayed correctly.
  • OSSM-5902 Previously, the OpenShift Service Mesh Console (OSSMC) plugin redirected to a "Not Found Page" error when the user clicked the Istio config health symbol on the Overview page. Now, the plugin redirects to the correct Istio config details page.
  • OSSM-5541 Previously, an Istio operator pod might keep waiting for the leader lease in some restart conditions. Now, the leader election implementation has been enhanced to avoid this issue.
  • OSSM-1397 Previously, if you removed the maistra.io/member-of label from a namespace, the Service Mesh Operator did not automatically reapply the label to the namespace. As a result, sidecar injection did not work in the namespace.

    The Operator would reapply the label to the namespace when you made changes to the ServiceMeshMember object, which triggered the reconciliation of this member object.

    Now, any change to the namespace also triggers the member object reconciliation.

  • OSSM-3647 Previously, in the Service Mesh control plane (SMCP) v2.2 (Istio 1.12), WasmPlugins were applied only to inbound listeners. Since SMCP v2.3 (Istio 1.14), WasmPlugins have been applied to inbound and outbound listeners by default, which introduced regression for users of the 3scale WasmPlugin. Now, the environment variable APPLY_WASM_PLUGINS_TO_INBOUND_ONLY is added, which allows safe migration from SMCP v2.2 to v2.3 and v2.4.

    The following setting should be added to the SMCP config:

    spec:
      runtime:
        components:
          pilot:
            container:
              env:
                APPLY_WASM_PLUGINS_TO_INBOUND_ONLY: "true"

    To ensure safe migration, perform the following steps:

    1. Set APPLY_WASM_PLUGINS_TO_INBOUND_ONLY in SMCP v2.2.
    2. Upgrade to 2.4.
    3. Set spec.match[].mode: SERVER in WasmPlugins.
    4. Remove the previously-added environment variable.
  • OSSM-4851 Previously, an error occurred in the operator deploying new pods in a namespace scoped inside the mesh when runAsGroup, runAsUser, or fsGroup parameters were nil. Now, a yaml validation has been added to avoid the nil value.
  • OSSM-3771 Previously, OpenShift routes could not be disabled for additional ingress gateways defined in a Service Mesh Control Plane (SMCP). Now, a routeConfig block can be added to each additionalIngress gateway so the creation of OpenShift routes can be enabled or disabled for each gateway.
  • OSSM-4197 Previously, if you deployed a v2.2 or v2.1 of the 'ServiceMeshControlPlane' resource, the /etc/cni/multus/net.d/ directory was not created. As a result, the istio-cni pod failed to become ready, and the istio-cni pods log contained the following message:

    $ error   Installer exits with open /host/etc/cni/multus/net.d/v2-2-istio-cni.kubeconfig.tmp.841118073: no such file or directory

    Now, if you deploy a v2.2 or v2.1 of the 'ServiceMeshControlPlane' resource, the /etc/cni/multus/net.d/ directory is created, and the istio-cni pod becomes ready.

  • OSSM-3993 Previously, Kiali only supported OpenShift OAuth via a proxy on the standard HTTPS port of 443. Now, Kiali supports OpenShift OAuth over a non-standard HTTPS port. To enable the port, you must set the spec.server.web_port field to the proxy’s non-standard HTTPS port in the Kiali CR.
  • OSSM-3936 Previously, the values for the injection_label_rev and injection_label_name attributes were hardcoded. This prevented custom configurations from taking effect in the Kiali Custom Resource Definition (CRD). Now, the attribute values are not hardcoded. You can customize the values for the injection_label_rev and injection_label_name attributes in the spec.istio_labels specification.
  • OSSM-3644 Previously, the federation egress-gateway received the wrong update of network gateway endpoints, causing extra endpoint entries. Now, the federation-egress gateway has been updated on the server side so it receives the correct network gateway endpoints.
  • OSSM-3595 Previously, the istio-cni plugin sometimes failed on RHEL because SELinux did not allow the utility iptables-restore to open files in the /tmp directory. Now, SELinux passes iptables-restore via stdin input stream instead of via a file.
  • OSSM-3586 Previously, Istio proxies were slow to start when Google Cloud Platform (GCP) metadata servers were not available. When you upgrade to Istio 1.14.6, Istio proxies start as expected on GCP, even if metadata servers are not available.
  • OSSM-3025 Istiod sometimes fails to become ready. Sometimes, when a mesh contained many member namespaces, the Istiod pod did not become ready due to a deadlock within Istiod. The deadlock is now resolved and the pod now starts as expected.
  • OSSM-2493 Default nodeSelector and tolerations in SMCP not passed to Kiali. The nodeSelector and tolerations you add to SMCP.spec.runtime.defaults are now passed to the Kiali resource.
  • OSSM-2492 Default tolerations in SMCP not passed to Jaeger. The nodeSelector and tolerations you add to SMCP.spec.runtime.defaults are now passed to the Jaeger resource.
  • OSSM-2374 If you deleted one of the ServiceMeshMember resources, then the Service Mesh operator deleted the ServiceMeshMemberRoll. While this is expected behavior when you delete the last ServiceMeshMember, the operator should not delete the ServiceMeshMemberRoll if it contains any members in addition to the one that was deleted. This issue is now fixed and the operator only deletes the ServiceMeshMemberRoll when the last ServiceMeshMember resource is deleted.
  • OSSM-2373 Error trying to get OAuth metadata when logging in. To fetch the cluster version, the system:anonymous account is used. With the cluster’s default bundled ClusterRoles and ClusterRoleBinding, the anonymous account can fetch the version correctly. If the system:anonymous account loses its privileges to fetch the cluster version, OpenShift authentication becomes unusable.

    This is fixed by using the Kiali SA to fetch the cluster version. This also allows for improved security on the cluster.

  • OSSM-2371 Despite Kiali being configured as "view-only," a user can change the proxy logging level via the Workload details' Logs tab’s kebab menu. This issue has been fixed so the options under "Set Proxy Log Level" are disabled when Kiali is configured as "view-only."
  • OSSM-2344 Restarting Istiod causes Kiali to flood CRI-O with port-forward requests. This issue occurred when Kiali could not connect to Istiod and Kiali simultaneously issued a large number of requests to istiod. Kiali now limits the number of requests it sends to istiod.
  • OSSM-2335 Dragging the mouse pointer over the Traces scatterchart plot sometimes caused the Kiali console to stop responding due to concurrent backend requests.
  • OSSM-2221 Previously, gateway injection in the ServiceMeshControlPlane namespace was not possible because the ignore-namespace label was applied to the namespace by default.

    When creating a v2.4 control plane, the namespace no longer has the ignore-namespace label applied, and gateway injection is possible.

    In the following example, the oc label command removes the ignore-namespace label from a namespace in an existing deployment:

    $ oc label namespace istio-system maistra.io/ignore-namespace-

    where:

    istio_system
    Specified the name of the ServiceMeshControlPlane namespace.
  • OSSM-2053 Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, during SMCP reconciliation, the SMMR controller removed the member namespaces from SMMR.status.configuredMembers. This caused the services in the member namespaces to become unavailable for a few moments.

    Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, the SMMR controller no longer removes the namespaces from SMMR.status.configuredMembers. Instead, the controller adds the namespaces to SMMR.status.pendingMembers to indicate that they are not up-to-date. During reconciliation, as each namespace synchronizes with the SMCP, the namespace is automatically removed from SMMR.status.pendingMembers.

  • OSSM-1962 Use EndpointSlices in federation controller. The federation controller now uses EndpointSlices, which improves scalability and performance in large deployments. The PILOT_USE_ENDPOINT_SLICE flag is enabled by default. Disabling the flag prevents use of federation deployments.
  • OSSM-1668 A new field spec.security.jwksResolverCA was added to the Version 2.1 SMCP but was missing in the 2.2.0 and 2.2.1 releases. When upgrading from an Operator version where this field was present to an Operator version that was missing this field, the .spec.security.jwksResolverCA field was not available in the SMCP.
  • OSSM-1325 istiod pod crashes and displays the following error message: fatal error: concurrent map iteration and map write.
  • OSSM-1211 Configuring Federated service meshes for failover does not work as expected.

    The Istiod pilot log displays the following error: envoy connection [C289] TLS error: 337047686:SSL routines:tls_process_server_certificate:certificate verify failed

  • OSSM-1099 The Kiali console displayed the message Sorry, there was a problem. Try a refresh or navigate to a different page.
  • OSSM-1074 Pod annotations defined in SMCP are not injected in the pods.
  • OSSM-999 Kiali retention did not work as expected. Calendar times were greyed out in the dashboard graph.
  • OSSM-797 Kiali Operator pod generates CreateContainerConfigError while installing or updating the operator.
  • OSSM-722 Namespace starting with kube is hidden from Kiali.
  • OSSM-569 There is no CPU memory limit for the Prometheus istio-proxy container. The Prometheus istio-proxy sidecar now uses the resource limits defined in spec.proxy.runtime.container.
  • OSSM-535 Support validationMessages in SMCP. The ValidationMessages field in the Service Mesh Control Plane can now be set to True. This writes a log for the status of the resources, which can be helpful when troubleshooting problems.
  • OSSM-449 VirtualService and Service causes an error "Only unique values for domains are permitted. Duplicate entry of domain."
  • OSSM-419 Namespaces with similar names will all show in Kiali namespace list, even though namespaces may not be defined in Service Mesh Member Role.
  • OSSM-296 When adding health configuration to the Kiali custom resource (CR) is it not being replicated to the Kiali configmap.
  • OSSM-291 In the Kiali console, on the Applications, Services, and Workloads pages, the "Remove Label from Filters" function is not working.
  • OSSM-289 In the Kiali console, on the Service Details pages for the 'istio-ingressgateway' and 'jaeger-query' services there are no Traces being displayed. The traces exist in Jaeger.
  • OSSM-287 In the Kiali console there are no traces being displayed on the Graph Service.
  • OSSM-285 When trying to access the Kiali console, receive the following error message "Error trying to get OAuth Metadata".

    Workaround: Restart the Kiali pod.

  • MAISTRA-2735 The resources that the Service Mesh Operator deletes when reconciling the SMCP changed in Red Hat OpenShift Service Mesh version 2.1. Previously, the Operator deleted a resource with the following labels:

    • maistra.io/owner
    • app.kubernetes.io/version

    Now, the Operator ignores resources that does not also include the app.kubernetes.io/managed-by=maistra-istio-operator label. If you create your own resources, you should not add the app.kubernetes.io/managed-by=maistra-istio-operator label to them.

  • MAISTRA-2687 Red Hat OpenShift Service Mesh 2.1 federation gateway does not send the full certificate chain when using external certificates. The Service Mesh federation egress gateway only sends the client certificate. Because the federation ingress gateway only knows about the root certificate, it cannot verify the client certificate unless you add the root certificate to the federation import ConfigMap.
  • MAISTRA-2635 Replace deprecated Kubernetes API. To remain compatible with OpenShift Container Platform 4.8, the apiextensions.k8s.io/v1beta1 API was deprecated as of Red Hat OpenShift Service Mesh 2.0.8.
  • MAISTRA-2631 The WASM feature is not working because podman is failing due to nsenter binary not being present. Red Hat OpenShift Service Mesh generates the following error message: Error: error configuring CNI network plugin exec: "nsenter": executable file not found in $PATH. The container image now contains nsenter and WASM works as expected.
  • MAISTRA-2534 When istiod attempted to fetch the JWKS for an issuer specified in a JWT rule, the issuer service responded with a 502. This prevented the proxy container from becoming ready and caused deployments to hang. The fix for the community bug has been included in the Service Mesh 2.0.7 release.
  • MAISTRA-2411 When the Operator creates a new ingress gateway using spec.gateways.additionaIngress in the ServiceMeshControlPlane, Operator is not creating a NetworkPolicy for the additional ingress gateway like it does for the default istio-ingressgateway. This is causing a 503 response from the route of the new gateway.

    Workaround: Manually create the NetworkPolicy in the istio-system namespace.

  • MAISTRA-2401 CVE-2021-3586 servicemesh-operator: NetworkPolicy resources incorrectly specified ports for ingress resources. The NetworkPolicy resources installed for Red Hat OpenShift Service Mesh did not properly specify which ports could be accessed. This allowed access to all ports on these resources from any pod. Network policies applied to the following resources are affected:

    • Galley
    • Grafana
    • Istiod
    • Jaeger
    • Kiali
    • Prometheus
    • Sidecar injector
  • MAISTRA-2378 When the cluster is configured to use OpenShift SDN with ovs-multitenant and the mesh contains a large number of namespaces (200+), the OpenShift Container Platform networking plugin is unable to configure the namespaces quickly. Service Mesh times out causing namespaces to be continuously dropped from the service mesh and then reenlisted.
  • MAISTRA-2370 Handle tombstones in listerInformer. The updated cache codebase was not handling tombstones when translating the events from the namespace caches to the aggregated cache, leading to a panic in the go routine.
  • MAISTRA-2117 Add optional ConfigMap mount to operator. The CSV now contains an optional ConfigMap volume mount, which mounts the smcp-templates ConfigMap if it exists. If the smcp-templates ConfigMap does not exist, the mounted directory is empty. When you create the ConfigMap, the directory is populated with the entries from the ConfigMap and can be referenced in SMCP.spec.profiles. No restart of the Service Mesh operator is required.

    Customers using the 2.0 operator with a modified CSV to mount the smcp-templates ConfigMap can upgrade to Red Hat OpenShift Service Mesh 2.1. After upgrading, you can continue using an existing ConfigMap, and the profiles it contains, without editing the CSV. Customers that previously used ConfigMap with a different name will either have to rename the ConfigMap or update the CSV after upgrading.

  • MAISTRA-2010 AuthorizationPolicy does not support request.regex.headers field. The validatingwebhook rejects any AuthorizationPolicy with the field, and even if you disable that, Pilot tries to validate it using the same code, and it does not work.
  • MAISTRA-1979 Migration to 2.0 The conversion webhook drops the following important fields when converting SMCP.status from v2 to v1:

    • conditions
    • components
    • observedGeneration
    • annotations

      Upgrading the operator to 2.0 might break client tools that read the SMCP status using the maistra.io/v1 version of the resource.

      This also causes the READY and STATUS columns to be empty when you run oc get servicemeshcontrolplanes.v1.maistra.io.

  • MAISTRA-1947 Technology Preview Updates to ServiceMeshExtensions are not applied.

    Workaround: Remove and recreate the ServiceMeshExtensions.

  • MAISTRA-1983 Migration to 2.0 Upgrading to 2.0.0 with an existing invalid ServiceMeshControlPlane cannot easily be repaired. The invalid items in the ServiceMeshControlPlane resource caused an unrecoverable error. The fix makes the errors recoverable. You can delete the invalid resource and replace it with a new one or edit the resource to fix the errors. For more information about editing your resource, see [Configuring the Red Hat OpenShift Service Mesh installation].
  • MAISTRA-1502 As a result of CVEs fixes in version 1.0.10, the Istio dashboards are not available from the Home Dashboard menu in Grafana. To access the Istio dashboards, click the Dashboard menu in the navigation panel and select the Manage tab.
  • MAISTRA-1399 Red Hat OpenShift Service Mesh no longer prevents you from installing unsupported CNI protocols. The supported network configurations has not changed.
  • MAISTRA-1089 Migration to 2.0 Gateways created in a non-control plane namespace are automatically deleted. After removing the gateway definition from the SMCP spec, you need to manually delete these resources.
  • MAISTRA-858 The following Envoy log messages describing deprecated options and configurations associated with Istio 1.1.x are expected:

    • [2019-06-03 07:03:28.943][19][warning][misc] [external/envoy/source/common/protobuf/utility.cc:129] Using deprecated option 'envoy.api.v2.listener.Filter.config'. This configuration will be removed from Envoy soon.
    • [2019-08-12 22:12:59.001][13][warning][misc] [external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file lds.proto. This configuration will be removed from Envoy soon.
  • MAISTRA-806 Evicted Istio Operator Pod causes mesh and CNI not to deploy.

    Workaround: If the istio-operator pod is evicted while deploying the control pane, delete the evicted istio-operator pod.

  • MAISTRA-681 When the Service Mesh control plane has many namespaces, it can lead to performance issues.
  • MAISTRA-193 Unexpected console info messages are visible when health checking is enabled for citadel.
  • Bugzilla 1821432 The toggle controls in OpenShift Container Platform Custom Resource details page does not update the CR correctly. UI Toggle controls in the Service Mesh Control Plane (SMCP) Overview page in the OpenShift Container Platform web console sometimes updates the wrong field in the resource. To update a SMCP, edit the YAML content directly or update the resource from the command line instead of clicking the toggle controls.

1.3. Upgrading Service Mesh

To access the most current features of Red Hat OpenShift Service Mesh, upgrade to the current version, 2.5.2.

1.3.1. Understanding versioning

Red Hat uses semantic versioning for product releases. Semantic Versioning is a 3-component number in the format of X.Y.Z, where:

  • X stands for a Major version. Major releases usually denote some sort of breaking change: architectural changes, API changes, schema changes, and similar major updates.
  • Y stands for a Minor version. Minor releases contain new features and functionality while maintaining backwards compatibility.
  • Z stands for a Patch version (also known as a z-stream release). Patch releases are used to addresses Common Vulnerabilities and Exposures (CVEs) and release bug fixes. New features and functionality are generally not released as part of a Patch release.
1.3.1.1. How versioning affects Service Mesh upgrades

Depending on the version of the update you are making, the upgrade process is different.

  • Patch updates - Patch upgrades are managed by the Operator Lifecycle Manager (OLM); they happen automatically when you update your Operators.
  • Minor upgrades - Minor upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the spec.version value in your ServiceMeshControlPlane resources.
  • Major upgrades - Major upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the spec.version value in your ServiceMeshControlPlane resources. Because major upgrades can contain changes that are not backwards compatible, additional manual changes might be required.
1.3.1.2. Understanding Service Mesh versions

In order to understand what version of Red Hat OpenShift Service Mesh you have deployed on your system, you need to understand how each of the component versions is managed.

  • Operator version - The most current Operator version is 2.5.2. The Operator version number only indicates the version of the currently installed Operator. Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, the version of the Operator does not determine the version of your deployed ServiceMeshControlPlane resources.

    Important

    Upgrading to the latest Operator version automatically applies patch updates, but does not automatically upgrade your Service Mesh control plane to the latest minor version.

  • ServiceMeshControlPlane version - The ServiceMeshControlPlane version determines what version of Red Hat OpenShift Service Mesh you are using. The value of the spec.version field in the ServiceMeshControlPlane resource controls the architecture and configuration settings that are used to install and deploy Red Hat OpenShift Service Mesh. When you create the Service Mesh control plane you can set the version in one of two ways:

    • To configure in the Form View, select the version from the Control Plane Version menu.
    • To configure in the YAML View, set the value for spec.version in the YAML file.

Operator Lifecycle Manager (OLM) does not manage Service Mesh control plane upgrades, so the version number for your Operator and ServiceMeshControlPlane (SMCP) may not match, unless you have manually upgraded your SMCP.

1.3.2. Upgrade considerations

The maistra.io/ label or annotation should not be used on a user-created custom resource, because it indicates that the resource was generated by and should be managed by the Red Hat OpenShift Service Mesh Operator.

Warning

During the upgrade, the Operator makes changes, including deleting or replacing files, to resources that include the following labels or annotations that indicate that the resource is managed by the Operator.

Before upgrading check for user-created custom resources that include the following labels or annotations:

  • maistra.io/ AND the app.kubernetes.io/managed-by label set to maistra-istio-operator (Red Hat OpenShift Service Mesh)
  • kiali.io/ (Kiali)
  • jaegertracing.io/ (Red Hat OpenShift distributed tracing platform (Jaeger))
  • logging.openshift.io/ (Red Hat Elasticsearch)

Before upgrading, check your user-created custom resources for labels or annotations that indicate they are Operator managed. Remove the label or annotation from custom resources that you do not want to be managed by the Operator.

When upgrading to version 2.0, the Operator only deletes resources with these labels in the same namespace as the SMCP.

When upgrading to version 2.1, the Operator deletes resources with these labels in all namespaces.

1.3.2.1. Known issues that may affect upgrade

Known issues that may affect your upgrade include:

  • When upgrading an Operator, custom configurations for Jaeger or Kiali might be reverted. Before upgrading an Operator, note any custom configuration settings for the Jaeger or Kiali objects in the Service Mesh production deployment so that you can recreate them.
  • Red Hat OpenShift Service Mesh does not support the use of EnvoyFilter configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained. If you are using Envoy Filters, and the configuration generated by Istio has changed due to the lastest version of Envoy introduced by upgrading your ServiceMeshControlPlane, that has the potential to break any EnvoyFilter you may have implemented.
  • OSSM-1505 ServiceMeshExtension does not work with OpenShift Container Platform version 4.11. Because ServiceMeshExtension has been deprecated in Red Hat OpenShift Service Mesh 2.2, this known issue will not be fixed and you must migrate your extensions to WasmPluging
  • OSSM-1396 If a gateway resource contains the spec.externalIPs setting, rather than being recreated when the ServiceMeshControlPlane is updated, the gateway is removed and never recreated.
  • OSSM-1052 When configuring a Service ExternalIP for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.

    Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).

1.3.3. Upgrading the Operators

In order to keep your Service Mesh patched with the latest security fixes, bug fixes, and software updates, you must keep your Operators updated. You initiate patch updates by upgrading your Operators.

Important

The version of the Operator does not determine the version of your service mesh. The version of your deployed Service Mesh control plane determines your version of Service Mesh.

Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, updating the Red Hat OpenShift Service Mesh Operator does not update the spec.version value of your deployed ServiceMeshControlPlane. Also note that the spec.version value is a two digit number, for example 2.2, and that patch updates, for example 2.2.1, are not reflected in the SMCP version value.

Operator Lifecycle Manager (OLM) controls the installation, upgrade, and role-based access control (RBAC) of Operators in a cluster. The OLM runs by default in OpenShift Container Platform. OLM queries for available Operators as well as upgrades for installed Operators.

Whether or not you have to take action to upgrade your Operators depends on the settings you selected when installing them. When you installed each of your Operators, you selected an Update Channel and an Approval Strategy. The combination of these two settings determine when and how your Operators are updated.

Table 1.3. Interaction of Update Channel and Approval Strategy
 Versioned channel"Stable" or "Preview" Channel

Automatic

Automatically updates the Operator for minor and patch releases for that version only. Will not automatically update to the next major version (that is, from version 2.0 to 3.0). Manual change to Operator subscription required to update to the next major version.

Automatically updates Operator for all major, minor, and patch releases.

Manual

Manual updates required for minor and patch releases for the specified version. Manual change to Operator subscription required to update to the next major version.

Manual updates required for all major, minor, and patch releases.

When you update your Red Hat OpenShift Service Mesh Operator the Operator Lifecycle Manager (OLM) removes the old Operator pod and starts a new pod. Once the new Operator pod starts, the reconciliation process checks the ServiceMeshControlPlane (SMCP), and if there are updated container images available for any of the Service Mesh control plane components, it replaces those Service Mesh control plane pods with ones that use the new container images.

When you upgrade the Kiali and Red Hat OpenShift distributed tracing platform (Jaeger) Operators, the OLM reconciliation process scans the cluster and upgrades the managed instances to the version of the new Operator. For example, if you update the Red Hat OpenShift distributed tracing platform (Jaeger) Operator from version 1.30.2 to version 1.34.1, the Operator scans for running instances of distributed tracing platform (Jaeger) and upgrades them to 1.34.1 as well.

To stay on a particular patch version of Red Hat OpenShift Service Mesh, you would need to disable automatic updates and remain on that specific version of the Operator.

For more information about upgrading Operators, refer to the Operator Lifecycle Manager documentation.

1.3.4. Upgrading the control plane

You must manually update the control plane for minor and major releases. The community Istio project recommends canary upgrades, Red Hat OpenShift Service Mesh only supports in-place upgrades. Red Hat OpenShift Service Mesh requires that you upgrade from each minor release to the next minor release in sequence. For example, you must upgrade from version 2.0 to version 2.1, and then upgrade to version 2.2. You cannot update from Red Hat OpenShift Service Mesh 2.0 to 2.2 directly.

When you upgrade the service mesh control plane, all Operator managed resources, for example gateways, are also upgraded.

Although you can deploy multiple versions of the control plane in the same cluster, Red Hat OpenShift Service Mesh does not support canary upgrades of the service mesh. That is, you can have different SCMP resources with different values for spec.version, but they cannot be managing the same mesh.

For more information about migrating your extensions, refer to Migrating from ServiceMeshExtension to WasmPlugin resources.

1.3.4.1. Upgrade changes from version 2.5 to version 2.6
1.3.4.1.1. Red Hat OpenShift distributed tracing platform (Jaeger) default setting change

This release disables Red Hat OpenShift distributed tracing platform (Jaeger) by default for new instances of the ServiceMeshControlPlane resource.

When updating existing instances of the ServiceMeshControlPlane resource to Red Hat OpenShift Service Mesh version 2.6, distributed tracing platform (Jaeger) remains enabled by default.

Red Hat OpenShift Service Mesh 2.6 is the last release that includes support for Red Hat OpenShift distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator. Both distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator will be removed in the next release. If you are currently using distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator, you must migrate to Red Hat OpenShift distributed tracing platform (Tempo) and Red Hat build of OpenTelemetry.

1.3.4.1.2. Envoy sidecar container default setting change

To enhance pod startup times, Istio now includes a startupProbe in sidecar containers by default. The pod’s readiness probes do not start until the Envoy sidecar has started.

1.3.4.2. Upgrade changes from version 2.4 to version 2.5
1.3.4.2.1. Istio OpenShift Routing (IOR) default setting change

The default setting for Istio OpenShift Routing (IOR) has changed. The setting is now disabled by default.

You can use IOR by setting the enabled field to true in the spec.gateways.openshiftRoute specification of the ServiceMeshControlPlane resource.

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  gateways:
    openshiftRoute:
      enabled: true
1.3.4.2.2. Istio proxy concurrency configuration enhancement

For consistency across deployments, Istio now configures the concurrency parameter based on the CPU limit allocated to the proxy container. For example, a limit of 2500m would set the concurrency parameter to 3. If you set the concurrency parameter to a value, Istio uses that value to configure how many threads the proxy runs instead of using the CPU limit.

Previously, the default setting for the parameter was 2.

1.3.4.3. Upgrade changes from version 2.3 to version 2.4

Upgrading the Service Mesh control plane from version 2.3 to 2.4 introduces the following behavioral changes:

  • Support for Istio OpenShift Routing (IOR) has been deprecated. IOR functionality is still enabled, but it will be removed in a future release.
  • The following cipher suites are no longer supported, and were removed from the list of ciphers used in client and server side TLS negotiations.

    • ECDHE-ECDSA-AES128-SHA
    • ECDHE-RSA-AES128-SHA
    • AES128-GCM-SHA256
    • AES128-SHA
    • ECDHE-ECDSA-AES256-SHA
    • ECDHE-RSA-AES256-SHA
    • AES256-GCM-SHA384
    • AES256-SHA

      Applications that require access to services that use one of these cipher suites will fail to connect when the proxy initiates a TLS connection.

1.3.4.4. Upgrade changes from version 2.2 to version 2.3

Upgrading the Service Mesh control plane from version 2.2 to 2.3 introduces the following behavioral changes:

  • This release requires use of the WasmPlugin API. Support for the ServiceMeshExtension API, which was deprecated in 2.2, has now been removed. If you attempt to upgrade while using the ServiceMeshExtension API, then the upgrade fails.
1.3.4.5. Upgrade changes from version 2.1 to version 2.2

Upgrading the Service Mesh control plane from version 2.1 to 2.2 introduces the following behavioral changes:

  • The istio-node DaemonSet is renamed to istio-cni-node to match the name in upstream Istio.
  • Istio 1.10 updated Envoy to send traffic to the application container using eth0 rather than lo by default.
  • This release adds support for the WasmPlugin API and deprecates the ServiceMeshExtension API.
1.3.4.6. Upgrade changes from version 2.0 to version 2.1

Upgrading the Service Mesh control plane from version 2.0 to 2.1 introduces the following architectural and behavioral changes.

Architecture changes

Mixer has been completely removed in Red Hat OpenShift Service Mesh 2.1. Upgrading from a Red Hat OpenShift Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled.

If you see the following message when upgrading from v2.0 to v2.1, update the existing Mixer type to Istiod type in the existing Control Plane spec before you update the .spec.version field:

An error occurred
admission webhook smcp.validation.maistra.io denied the request: [support for policy.type "Mixer" and policy.Mixer options have been removed in v2.1, please use another alternative, support for telemetry.type "Mixer" and telemetry.Mixer options have been removed in v2.1, please use another alternative]”

For example:

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  policy:
    type: Istiod
  telemetry:
    type: Istiod
  version: v2.5

Behavioral changes

  • AuthorizationPolicy updates:

    • With the PROXY protocol, if you’re using ipBlocks and notIpBlocks to specify remote IP addresses, update the configuration to use remoteIpBlocks and notRemoteIpBlocks instead.
    • Added support for nested JSON Web Token (JWT) claims.
  • EnvoyFilter breaking changes>

    • Must use typed_config
    • xDS v2 is no longer supported
    • Deprecated filter names
  • Older versions of proxies may report 503 status codes when receiving 1xx or 204 status codes from newer proxies.
1.3.4.7. Upgrading the Service Mesh control plane

To upgrade Red Hat OpenShift Service Mesh, you must update the version field of the Red Hat OpenShift Service Mesh ServiceMeshControlPlane v2 resource. Then, once it is configured and applied, restart the application pods to update each sidecar proxy and its configuration.

Prerequisites

  • You are running OpenShift Container Platform 4.9 or later.
  • You have the latest Red Hat OpenShift Service Mesh Operator.

Procedure

  1. Switch to the project that contains your ServiceMeshControlPlane resource. In this example, istio-system is the name of the Service Mesh control plane project.

    $ oc project istio-system
  2. Check your v2 ServiceMeshControlPlane resource configuration to verify it is valid.

    1. Run the following command to view your ServiceMeshControlPlane resource as a v2 resource.

      $ oc get smcp -o yaml
      Tip

      Back up your Service Mesh control plane configuration.

  3. Update the .spec.version field and apply the configuration.

    For example:

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
    spec:
      version: v2.5

    Alternatively, instead of using the command line, you can use the web console to edit the Service Mesh control plane. In the OpenShift Container Platform web console, click Project and select the project name you just entered.

    1. Click OperatorsInstalled Operators.
    2. Find your ServiceMeshControlPlane instance.
    3. Select YAML view and update text of the YAML file, as shown in the previous example.
    4. Click Save.
1.3.4.8. Migrating Red Hat OpenShift Service Mesh from version 1.1 to version 2.0

Upgrading from version 1.1 to 2.0 requires manual steps that migrate your workloads and application to a new instance of Red Hat OpenShift Service Mesh running the new version.

Prerequisites

  • You must upgrade to OpenShift Container Platform 4.7. before you upgrade to Red Hat OpenShift Service Mesh 2.0.
  • You must have Red Hat OpenShift Service Mesh version 2.0 operator. If you selected the automatic upgrade path, the operator automatically downloads the latest information. However, there are steps you must take to use the features in Red Hat OpenShift Service Mesh version 2.0.
1.3.4.8.1. Upgrading Red Hat OpenShift Service Mesh

To upgrade Red Hat OpenShift Service Mesh, you must create an instance of Red Hat OpenShift Service Mesh ServiceMeshControlPlane v2 resource in a new namespace. Then, once it’s configured, move your microservice applications and workloads from your old mesh to the new service mesh.

Procedure

  1. Check your v1 ServiceMeshControlPlane resource configuration to make sure it is valid.

    1. Run the following command to view your ServiceMeshControlPlane resource as a v2 resource.

      $ oc get smcp -o yaml
    2. Check the spec.techPreview.errored.message field in the output for information about any invalid fields.
    3. If there are invalid fields in your v1 resource, the resource is not reconciled and cannot be edited as a v2 resource. All updates to v2 fields will be overridden by the original v1 settings. To fix the invalid fields, you can replace, patch, or edit the v1 version of the resource. You can also delete the resource without fixing it. After the resource has been fixed, it can be reconciled, and you can to modify or view the v2 version of the resource.
    4. To fix the resource by editing a file, use oc get to retrieve the resource, edit the text file locally, and replace the resource with the file you edited.

      $ oc get smcp.v1.maistra.io <smcp_name> > smcp-resource.yaml
      #Edit the smcp-resource.yaml file.
      $ oc replace -f smcp-resource.yaml
    5. To fix the resource using patching, use oc patch.

      $ oc patch smcp.v1.maistra.io <smcp_name> --type json --patch '[{"op": "replace","path":"/spec/path/to/bad/setting","value":"corrected-value"}]'
    6. To fix the resource by editing with command line tools, use oc edit.

      $ oc edit smcp.v1.maistra.io <smcp_name>
  2. Back up your Service Mesh control plane configuration. Switch to the project that contains your ServiceMeshControlPlane resource. In this example, istio-system is the name of the Service Mesh control plane project.

    $ oc project istio-system
  3. Enter the following command to retrieve the current configuration. Your <smcp_name> is specified in the metadata of your ServiceMeshControlPlane resource, for example basic-install or full-install.

    $ oc get servicemeshcontrolplanes.v1.maistra.io <smcp_name> -o yaml > <smcp_name>.v1.yaml
  4. Convert your ServiceMeshControlPlane to a v2 control plane version that contains information about your configuration as a starting point.

    $ oc get smcp <smcp_name> -o yaml > <smcp_name>.v2.yaml
  5. Create a project. In the OpenShift Container Platform console Project menu, click New Project and enter a name for your project, istio-system-upgrade, for example. Or, you can run this command from the CLI.

    $ oc new-project istio-system-upgrade
  6. Update the metadata.namespace field in your v2 ServiceMeshControlPlane with your new project name. In this example, use istio-system-upgrade.
  7. Update the version field from 1.1 to 2.0 or remove it in your v2 ServiceMeshControlPlane.
  8. Create a ServiceMeshControlPlane in the new namespace. On the command line, run the following command to deploy the control plane with the v2 version of the ServiceMeshControlPlane that you retrieved. In this example, replace `<smcp_name.v2> `with the path to your file.

    $ oc create -n istio-system-upgrade -f <smcp_name>.v2.yaml

    Alternatively, you can use the console to create the Service Mesh control plane. In the OpenShift Container Platform web console, click Project. Then, select the project name you just entered.

    1. Click OperatorsInstalled Operators.
    2. Click Create ServiceMeshControlPlane.
    3. Select YAML view and paste text of the YAML file you retrieved into the field. Check that the apiVersion field is set to maistra.io/v2 and modify the metadata.namespace field to use the new namespace, for example istio-system-upgrade.
    4. Click Create.
1.3.4.8.2. Configuring the 2.0 ServiceMeshControlPlane

The ServiceMeshControlPlane resource has been changed for Red Hat OpenShift Service Mesh version 2.0. After you created a v2 version of the ServiceMeshControlPlane resource, modify it to take advantage of the new features and to fit your deployment. Consider the following changes to the specification and behavior of Red Hat OpenShift Service Mesh 2.0 as you’re modifying your ServiceMeshControlPlane resource. You can also refer to the Red Hat OpenShift Service Mesh 2.0 product documentation for new information to features you use. The v2 resource must be used for Red Hat OpenShift Service Mesh 2.0 installations.

1.3.4.8.2.1. Architecture changes

The architectural units used by previous versions have been replaced by Istiod. In 2.0 the Service Mesh control plane components Mixer, Pilot, Citadel, Galley, and the sidecar injector functionality have been combined into a single component, Istiod.

Although Mixer is no longer supported as a control plane component, Mixer policy and telemetry plugins are now supported through WASM extensions in Istiod. Mixer can be enabled for policy and telemetry if you need to integrate legacy Mixer plugins.

Secret Discovery Service (SDS) is used to distribute certificates and keys to sidecars directly from Istiod. In Red Hat OpenShift Service Mesh version 1.1, secrets were generated by Citadel, which were used by the proxies to retrieve their client certificates and keys.

1.3.4.8.2.2. Annotation changes

The following annotations are no longer supported in v2.0. If you are using one of these annotations, you must update your workload before moving it to a v2.0 Service Mesh control plane.

  • sidecar.maistra.io/proxyCPULimit has been replaced with sidecar.istio.io/proxyCPULimit. If you were using sidecar.maistra.io annotations on your workloads, you must modify those workloads to use sidecar.istio.io equivalents instead.
  • sidecar.maistra.io/proxyMemoryLimit has been replaced with sidecar.istio.io/proxyMemoryLimit
  • sidecar.istio.io/discoveryAddress is no longer supported. Also, the default discovery address has moved from pilot.<control_plane_namespace>.svc:15010 (or port 15011, if mtls is enabled) to istiod-<smcp_name>.<control_plane_namespace>.svc:15012.
  • The health status port is no longer configurable and is hard-coded to 15021. * If you were defining a custom status port, for example, status.sidecar.istio.io/port, you must remove the override before moving the workload to a v2.0 Service Mesh control plane. Readiness checks can still be disabled by setting the status port to 0.
  • Kubernetes Secret resources are no longer used to distribute client certificates for sidecars. Certificates are now distributed through Istiod’s SDS service. If you were relying on mounted secrets, they are longer available for workloads in v2.0 Service Mesh control planes.
1.3.4.8.2.3. Behavioral changes

Some features in Red Hat OpenShift Service Mesh 2.0 work differently than they did in previous versions.

  • The readiness port on gateways has moved from 15020 to 15021.
  • The target host visibility includes VirtualService, as well as ServiceEntry resources. It includes any restrictions applied through Sidecar resources.
  • Automatic mutual TLS is enabled by default. Proxy to proxy communication is automatically configured to use mTLS, regardless of global PeerAuthentication policies in place.
  • Secure connections are always used when proxies communicate with the Service Mesh control plane regardless of spec.security.controlPlane.mtls setting. The spec.security.controlPlane.mtls setting is only used when configuring connections for Mixer telemetry or policy.
1.3.4.8.2.4. Migration details for unsupported resources

Policy (authentication.istio.io/v1alpha1)

Policy resources must be migrated to new resource types for use with v2.0 Service Mesh control planes, PeerAuthentication and RequestAuthentication. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.

Mutual TLS

Mutual TLS enforcement is accomplished using the security.istio.io/v1beta1 PeerAuthentication resource. The legacy spec.peers.mtls.mode field maps directly to the new resource’s spec.mtls.mode field. Selection criteria has changed from specifying a service name in spec.targets[x].name to a label selector in spec.selector.matchLabels. In PeerAuthentication, the labels must match the selector on the services named in the targets list. Any port-specific settings will need to be mapped into spec.portLevelMtls.

Authentication

Additional authentication methods specified in spec.origins, must be mapped into a security.istio.io/v1beta1 RequestAuthentication resource. spec.selector.matchLabels must be configured similarly to the same field on PeerAuthentication. Configuration specific to JWT principals from spec.origins.jwt items map to similar fields in spec.rules items.

  • spec.origins[x].jwt.triggerRules specified in the Policy must be mapped into one or more security.istio.io/v1beta1 AuthorizationPolicy resources. Any spec.selector.labels must be configured similarly to the same field on RequestAuthentication.
  • spec.origins[x].jwt.triggerRules.excludedPaths must be mapped into an AuthorizationPolicy whose spec.action is set to ALLOW, with spec.rules[x].to.operation.path entries matching the excluded paths.
  • spec.origins[x].jwt.triggerRules.includedPaths must be mapped into a separate AuthorizationPolicy whose spec.action is set to ALLOW, with spec.rules[x].to.operation.path entries matching the included paths, and spec.rules.[x].from.source.requestPrincipals entries that align with the specified spec.origins[x].jwt.issuer in the Policy resource.

ServiceMeshPolicy (maistra.io/v1)

ServiceMeshPolicy was configured automatically for the Service Mesh control plane through the spec.istio.global.mtls.enabled in the v1 resource or spec.security.dataPlane.mtls in the v2 resource setting. For v2 control planes, a functionally equivalent PeerAuthentication resource is created during installation. This feature is deprecated in Red Hat OpenShift Service Mesh version 2.0

RbacConfig, ServiceRole, ServiceRoleBinding (rbac.istio.io/v1alpha1)

These resources were replaced by the security.istio.io/v1beta1 AuthorizationPolicy resource.

Mimicking RbacConfig behavior requires writing a default AuthorizationPolicy whose settings depend on the spec.mode specified in the RbacConfig.

  • When spec.mode is set to OFF, no resource is required as the default policy is ALLOW, unless an AuthorizationPolicy applies to the request.
  • When spec.mode is set to ON, set spec: {}. You must create AuthorizationPolicy policies for all services in the mesh.
  • spec.mode is set to ON_WITH_INCLUSION, must create an AuthorizationPolicy with spec: {} in each included namespace. Inclusion of individual services is not supported by AuthorizationPolicy. However, as soon as any AuthorizationPolicy is created that applies to the workloads for the service, all other requests not explicitly allowed will be denied.
  • When spec.mode is set to ON_WITH_EXCLUSION, it is not supported by AuthorizationPolicy. A global DENY policy can be created, but an AuthorizationPolicy must be created for every workload in the mesh because there is no allow-all policy that can be applied to either a namespace or a workload.

AuthorizationPolicy includes configuration for both the selector to which the configuration applies, which is similar to the function ServiceRoleBinding provides and the rules which should be applied, which is similar to the function ServiceRole provides.

ServiceMeshRbacConfig (maistra.io/v1)

This resource is replaced by using a security.istio.io/v1beta1 AuthorizationPolicy resource with an empty spec.selector in the Service Mesh control plane’s namespace. This policy will be the default authorization policy applied to all workloads in the mesh. For specific migration details, see RbacConfig above.

1.3.4.8.2.5. Mixer plugins

Mixer components are disabled by default in version 2.0. If you rely on Mixer plugins for your workload, you must configure your version 2.0 ServiceMeshControlPlane to include the Mixer components.

To enable the Mixer policy components, add the following snippet to your ServiceMeshControlPlane.

spec:
  policy:
    type: Mixer

To enable the Mixer telemetry components, add the following snippet to your ServiceMeshControlPlane.

spec:
  telemetry:
    type: Mixer

Legacy mixer plugins can also be migrated to WASM and integrated using the new ServiceMeshExtension (maistra.io/v1alpha1) custom resource.

Built-in WASM filters included in the upstream Istio distribution are not available in Red Hat OpenShift Service Mesh 2.0.

1.3.4.8.2.6. Mutual TLS changes

When using mTLS with workload specific PeerAuthentication policies, a corresponding DestinationRule is required to allow traffic if the workload policy differs from the namespace/global policy.

Auto mTLS is enabled by default, but can be disabled by setting spec.security.dataPlane.automtls to false in the ServiceMeshControlPlane resource. When disabling auto mTLS, DestinationRules may be required for proper communication between services. For example, setting PeerAuthentication to STRICT for one namespace may prevent services in other namespaces from accessing them, unless a DestinationRule configures TLS mode for the services in the namespace.

For information about mTLS, see Enabling mutual Transport Layer Security (mTLS)

1.3.4.8.2.6.1. Other mTLS Examples

To disable mTLS For productpage service in the bookinfo sample application, your Policy resource was configured the following way for Red Hat OpenShift Service Mesh v1.1.

Example Policy resource

apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
  name: productpage-mTLS-disable
  namespace: <namespace>
spec:
  targets:
  - name: productpage

To disable mTLS For productpage service in the bookinfo sample application, use the following example to configure your PeerAuthentication resource for Red Hat OpenShift Service Mesh v2.0.

Example PeerAuthentication resource

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: productpage-mTLS-disable
  namespace: <namespace>
spec:
  mtls:
    mode: DISABLE
  selector:
    matchLabels:
      # this should match the selector for the "productpage" service
      app: productpage

To enable mTLS With JWT authentication for the productpage service in the bookinfo sample application, your Policy resource was configured the following way for Red Hat OpenShift Service Mesh v1.1.

Example Policy resource

apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
  name: productpage-mTLS-with-JWT
  namespace: <namespace>
spec:
  targets:
  - name: productpage
    ports:
    - number: 9000
  peers:
  - mtls:
  origins:
  - jwt:
      issuer: "https://securetoken.google.com"
      audiences:
      - "productpage"
      jwksUri: "https://www.googleapis.com/oauth2/v1/certs"
      jwtHeaders:
      - "x-goog-iap-jwt-assertion"
      triggerRules:
      - excludedPaths:
        - exact: /health_check
  principalBinding: USE_ORIGIN

To enable mTLS With JWT authentication for the productpage service in the bookinfo sample application, use the following example to configure your PeerAuthentication resource for Red Hat OpenShift Service Mesh v2.0.

Example PeerAuthentication resource

#require mtls for productpage:9000
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: productpage-mTLS-with-JWT
  namespace: <namespace>
spec:
  selector:
    matchLabels:
      # this should match the selector for the "productpage" service
      app: productpage
  portLevelMtls:
    9000:
      mode: STRICT
---
#JWT authentication for productpage
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: productpage-mTLS-with-JWT
  namespace: <namespace>
spec:
  selector:
    matchLabels:
      # this should match the selector for the "productpage" service
      app: productpage
  jwtRules:
  - issuer: "https://securetoken.google.com"
    audiences:
    - "productpage"
    jwksUri: "https://www.googleapis.com/oauth2/v1/certs"
    fromHeaders:
    - name: "x-goog-iap-jwt-assertion"
---
#Require JWT token to access product page service from
#any client to all paths except /health_check
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: productpage-mTLS-with-JWT
  namespace: <namespace>
spec:
  action: ALLOW
  selector:
    matchLabels:
      # this should match the selector for the "productpage" service
      app: productpage
  rules:
  - to: # require JWT token to access all other paths
      - operation:
          notPaths:
          - /health_check
    from:
      - source:
          # if using principalBinding: USE_PEER in the Policy,
          # then use principals, e.g.
          # principals:
          # - “*”
          requestPrincipals:
          - “*”
  - to: # no JWT token required to access health_check
      - operation:
          paths:
          - /health_check

1.3.4.8.3. Configuration recipes

You can configure the following items with these configuration recipes.

1.3.4.8.3.1. Mutual TLS in a data plane

Mutual TLS for data plane communication is configured through spec.security.dataPlane.mtls in the ServiceMeshControlPlane resource, which is false by default.

1.3.4.8.3.2. Custom signing key

Istiod manages client certificates and private keys used by service proxies. By default, Istiod uses a self-signed certificate for signing, but you can configure a custom certificate and private key. For more information about how to configure signing keys, see Adding an external certificate authority key and certificate

1.3.4.8.3.3. Tracing

Tracing is configured in spec.tracing. Currently, the only type of tracer that is supported is Jaeger. Sampling is a scaled integer representing 0.01% increments, for example, 1 is 0.01% and 10000 is 100%. The tracing implementation and sampling rate can be specified:

spec:
  tracing:
    sampling: 100 # 1%
    type: Jaeger

Jaeger is configured in the addons section of the ServiceMeshControlPlane resource.

spec:
  addons:
    jaeger:
      name: jaeger
      install:
        storage:
          type: Memory # or Elasticsearch for production mode
          memory:
            maxTraces: 100000
          elasticsearch: # the following values only apply if storage:type:=Elasticsearch
            storage: # specific storageclass configuration for the Jaeger Elasticsearch (optional)
              size: "100G"
              storageClassName: "storageclass"
            nodeCount: 3
            redundancyPolicy: SingleRedundancy
  runtime:
    components:
      tracing.jaeger: {} # general Jaeger specific runtime configuration (optional)
      tracing.jaeger.elasticsearch: #runtime configuration for Jaeger Elasticsearch deployment (optional)
        container:
          resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
            limits:
              memory: "1Gi"

The Jaeger installation can be customized with the install field. Container configuration, such as resource limits is configured in spec.runtime.components.jaeger related fields. If a Jaeger resource matching the value of spec.addons.jaeger.name exists, the Service Mesh control plane will be configured to use the existing installation. Use an existing Jaeger resource to fully customize your Jaeger installation.

1.3.4.8.3.4. Visualization

Kiali and Grafana are configured under the addons section of the ServiceMeshControlPlane resource.

spec:
  addons:
    grafana:
      enabled: true
      install: {} # customize install
    kiali:
      enabled: true
      name: kiali
      install: {} # customize install

The Grafana and Kiali installations can be customized through their respective install fields. Container customization, such as resource limits, is configured in spec.runtime.components.kiali and spec.runtime.components.grafana. If an existing Kiali resource matching the value of name exists, the Service Mesh control plane configures the Kiali resource for use with the control plane. Some fields in the Kiali resource are overridden, such as the accessible_namespaces list, as well as the endpoints for Grafana, Prometheus, and tracing. Use an existing resource to fully customize your Kiali installation.

1.3.4.8.3.5. Resource utilization and scheduling

Resources are configured under spec.runtime.<component>. The following component names are supported.

ComponentDescriptionVersions supported

security

Citadel container

v1.0/1.1

galley

Galley container

v1.0/1.1

pilot

Pilot/Istiod container

v1.0/1.1/2.0

mixer

istio-telemetry and istio-policy containers

v1.0/1.1

mixer.policy

istio-policy container

v2.0

mixer.telemetry

istio-telemetry container

v2.0

global.oauthproxy

oauth-proxy container used with various addons

v1.0/1.1/2.0

sidecarInjectorWebhook

sidecar injector webhook container

v1.0/1.1

tracing.jaeger

general Jaeger container - not all settings may be applied. Complete customization of Jaeger installation is supported by specifying an existing Jaeger resource in the Service Mesh control plane configuration.

v1.0/1.1/2.0

tracing.jaeger.agent

settings specific to Jaeger agent

v1.0/1.1/2.0

tracing.jaeger.allInOne

settings specific to Jaeger allInOne

v1.0/1.1/2.0

tracing.jaeger.collector

settings specific to Jaeger collector

v1.0/1.1/2.0

tracing.jaeger.elasticsearch

settings specific to Jaeger elasticsearch deployment

v1.0/1.1/2.0

tracing.jaeger.query

settings specific to Jaeger query

v1.0/1.1/2.0

prometheus

prometheus container

v1.0/1.1/2.0

kiali

Kiali container - complete customization of Kiali installation is supported by specifying an existing Kiali resource in the Service Mesh control plane configuration.

v1.0/1.1/2.0

grafana

Grafana container

v1.0/1.1/2.0

3scale

3scale container

v1.0/1.1/2.0

wasmExtensions.cacher

WASM extensions cacher container

v2.0 - tech preview

Some components support resource limiting and scheduling. For more information, see Performance and scalability.

1.3.4.8.4. Next steps for migrating your applications and workloads

Move the application workload to the new mesh and remove the old instances to complete your upgrade.

1.3.5. Upgrading the data plane

Your data plane will still function after you have upgraded the control plane. But in order to apply updates to the Envoy proxy and any changes to the proxy configuration, you must restart your application pods and workloads.

1.3.5.1. Updating your applications and workloads

To complete the migration, restart all of the application pods in the mesh to upgrade the Envoy sidecar proxies and their configuration.

To perform a rolling update of a deployment use the following command:

$ oc rollout restart <deployment>

You must perform a rolling update for all applications that make up the mesh.

1.4. Understanding Service Mesh

Red Hat OpenShift Service Mesh provides a platform for behavioral insight and operational control over your networked microservices in a service mesh. With Red Hat OpenShift Service Mesh, you can connect, secure, and monitor microservices in your OpenShift Container Platform environment.

1.4.1. What is Red Hat OpenShift Service Mesh?

A service mesh is the network of microservices that make up applications in a distributed microservice architecture and the interactions between those microservices. When a Service Mesh grows in size and complexity, it can become harder to understand and manage.

Based on the open source Istio project, Red Hat OpenShift Service Mesh adds a transparent layer on existing distributed applications without requiring any changes to the service code. You add Red Hat OpenShift Service Mesh support to services by deploying a special sidecar proxy to relevant services in the mesh that intercepts all network communication between microservices. You configure and manage the Service Mesh using the Service Mesh control plane features.

Red Hat OpenShift Service Mesh gives you an easy way to create a network of deployed services that provide:

  • Discovery
  • Load balancing
  • Service-to-service authentication
  • Failure recovery
  • Metrics
  • Monitoring

Red Hat OpenShift Service Mesh also provides more complex operational functions including:

  • A/B testing
  • Canary releases
  • Access control
  • End-to-end authentication

1.4.2. Service Mesh architecture

Service mesh technology operates at the network communication level. That is, service mesh components capture or intercept traffic to and from microservices, either modifying requests, redirecting them, or creating new requests to other services.

Service Mesh architecture image

At a high level, Red Hat OpenShift Service Mesh consists of a data plane and a control plane

The data plane is a set of intelligent proxies, running alongside application containers in a pod, that intercept and control all inbound and outbound network communication between microservices in the service mesh. The data plane is implemented in such a way that it intercepts all inbound (ingress) and outbound (egress) network traffic. The Istio data plane consists of Envoy containers running along side application containers in a pod. The Envoy container acts as a proxy, controlling all network communication into and out of the pod.

  • Envoy proxies are the only Istio components that interact with data plane traffic. All incoming (ingress) and outgoing (egress) network traffic between services flows through the proxies. The Envoy proxy also collects all metrics related to services traffic within the mesh. Envoy proxies are deployed as sidecars, running in the same pod as services. Envoy proxies are also used to implement mesh gateways.

    • Sidecar proxies manage inbound and outbound communication for their workload instance.
    • Gateways are proxies operating as load balancers receiving incoming or outgoing HTTP/TCP connections. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads. You use a Gateway to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh.

      • Ingress-gateway - Also known as an Ingress Controller, the Ingress Gateway is a dedicated Envoy proxy that receives and controls traffic entering the service mesh. An Ingress Gateway allows features such as monitoring and route rules to be applied to traffic entering the cluster.
      • Egress-gateway - Also known as an egress controller, the Egress Gateway is a dedicated Envoy proxy that manages traffic leaving the service mesh. An Egress Gateway allows features such as monitoring and route rules to be applied to traffic exiting the mesh.

The control plane manages and configures the proxies that make up the data plane. It is the authoritative source for configuration, manages access control and usage policies, and collects metrics from the proxies in the service mesh.

  • The Istio control plane is composed of Istiod which consolidates several previous control plane components (Citadel, Galley, Pilot) into a single binary. Istiod provides service discovery, configuration, and certificate management. It converts high-level routing rules to Envoy configurations and propagates them to the sidecars at runtime.

    • Istiod can act as a Certificate Authority (CA), generating certificates supporting secure mTLS communication in the data plane. You can also use an external CA for this purpose.
    • Istiod is responsible for injecting sidecar proxy containers into workloads deployed to an OpenShift cluster.

Red Hat OpenShift Service Mesh uses the istio-operator to manage the installation of the control plane. An Operator is a piece of software that enables you to implement and automate common activities in your OpenShift cluster. It acts as a controller, allowing you to set or change the desired state of objects in your cluster, in this case, a Red Hat OpenShift Service Mesh installation.

Red Hat OpenShift Service Mesh also bundles the following Istio add-ons as part of the product:

  • Kiali - Kiali is the management console for Red Hat OpenShift Service Mesh. It provides dashboards, observability, and robust configuration and validation capabilities. It shows the structure of your service mesh by inferring traffic topology and displays the health of your mesh. Kiali provides detailed metrics, powerful validation, access to Grafana, and strong integration with the distributed tracing platform (Jaeger).
  • Prometheus - Red Hat OpenShift Service Mesh uses Prometheus to store telemetry information from services. Kiali depends on Prometheus to obtain metrics, health status, and mesh topology.
  • Jaeger - Red Hat OpenShift Service Mesh supports the distributed tracing platform (Jaeger). Jaeger is an open source traceability server that centralizes and displays traces associated with a single request between multiple services. Using the distributed tracing platform (Jaeger) you can monitor and troubleshoot your microservices-based distributed systems.
  • Elasticsearch - Elasticsearch is an open source, distributed, JSON-based search and analytics engine. The distributed tracing platform (Jaeger) uses Elasticsearch for persistent storage.
  • Grafana - Grafana provides mesh administrators with advanced query and metrics analysis and dashboards for Istio data. Optionally, Grafana can be used to analyze service mesh metrics.

The following Istio integrations are supported with Red Hat OpenShift Service Mesh:

  • 3scale - Istio provides an optional integration with Red Hat 3scale API Management solutions. For versions prior to 2.1, this integration was achieved via the 3scale Istio adapter. For version 2.1 and later, the 3scale integration is achieved via a WebAssembly module.

For information about how to install the 3scale adapter, refer to the 3scale Istio adapter documentation

1.4.3. Understanding Kiali

Kiali provides visibility into your service mesh by showing you the microservices in your service mesh, and how they are connected.

1.4.3.1. Kiali overview

Kiali provides observability into the Service Mesh running on OpenShift Container Platform. Kiali helps you define, validate, and observe your Istio service mesh. It helps you to understand the structure of your service mesh by inferring the topology, and also provides information about the health of your service mesh.

Kiali provides an interactive graph view of your namespace in real time that provides visibility into features like circuit breakers, request rates, latency, and even graphs of traffic flows. Kiali offers insights about components at different levels, from Applications to Services and Workloads, and can display the interactions with contextual information and charts on the selected graph node or edge. Kiali also provides the ability to validate your Istio configurations, such as gateways, destination rules, virtual services, mesh policies, and more. Kiali provides detailed metrics, and a basic Grafana integration is available for advanced queries. Distributed tracing is provided by integrating Jaeger into the Kiali console.

Kiali is installed by default as part of the Red Hat OpenShift Service Mesh.

1.4.3.2. Kiali architecture

Kiali is based on the open source Kiali project. Kiali is composed of two components: the Kiali application and the Kiali console.

  • Kiali application (back end) – This component runs in the container application platform and communicates with the service mesh components, retrieves and processes data, and exposes this data to the console. The Kiali application does not need storage. When deploying the application to a cluster, configurations are set in ConfigMaps and secrets.
  • Kiali console (front end) – The Kiali console is a web application. The Kiali application serves the Kiali console, which then queries the back end for data to present it to the user.

In addition, Kiali depends on external services and components provided by the container application platform and Istio.

  • Red Hat Service Mesh (Istio) - Istio is a Kiali requirement. Istio is the component that provides and controls the service mesh. Although Kiali and Istio can be installed separately, Kiali depends on Istio and will not work if it is not present. Kiali needs to retrieve Istio data and configurations, which are exposed through Prometheus and the cluster API.
  • Prometheus - A dedicated Prometheus instance is included as part of the Red Hat OpenShift Service Mesh installation. When Istio telemetry is enabled, metrics data are stored in Prometheus. Kiali uses this Prometheus data to determine the mesh topology, display metrics, calculate health, show possible problems, and so on. Kiali communicates directly with Prometheus and assumes the data schema used by Istio Telemetry. Prometheus is an Istio dependency and a hard dependency for Kiali, and many of Kiali’s features will not work without Prometheus.
  • Cluster API - Kiali uses the API of the OpenShift Container Platform (cluster API) to fetch and resolve service mesh configurations. Kiali queries the cluster API to retrieve, for example, definitions for namespaces, services, deployments, pods, and other entities. Kiali also makes queries to resolve relationships between the different cluster entities. The cluster API is also queried to retrieve Istio configurations like virtual services, destination rules, route rules, gateways, quotas, and so on.
  • Jaeger - Jaeger is optional, but is installed by default as part of the Red Hat OpenShift Service Mesh installation. When you install the distributed tracing platform (Jaeger) as part of the default Red Hat OpenShift Service Mesh installation, the Kiali console includes a tab to display distributed tracing data. Note that tracing data will not be available if you disable Istio’s distributed tracing feature. Also note that user must have access to the namespace where the Service Mesh control plane is installed to view tracing data.
  • Grafana - Grafana is optional, but is installed by default as part of the Red Hat OpenShift Service Mesh installation. When available, the metrics pages of Kiali display links to direct the user to the same metric in Grafana. Note that user must have access to the namespace where the Service Mesh control plane is installed to view links to the Grafana dashboard and view Grafana data.
1.4.3.3. Kiali features

The Kiali console is integrated with Red Hat Service Mesh and provides the following capabilities:

  • Health – Quickly identify issues with applications, services, or workloads.
  • Topology – Visualize how your applications, services, or workloads communicate via the Kiali graph.
  • Metrics – Predefined metrics dashboards let you chart service mesh and application performance for Go, Node.js. Quarkus, Spring Boot, Thorntail and Vert.x. You can also create your own custom dashboards.
  • Tracing – Integration with Jaeger lets you follow the path of a request through various microservices that make up an application.
  • Validations – Perform advanced validations on the most common Istio objects (Destination Rules, Service Entries, Virtual Services, and so on).
  • Configuration – Optional ability to create, update and delete Istio routing configuration using wizards or directly in the YAML editor in the Kiali Console.

1.4.4. Understanding distributed tracing

Every time a user takes an action in an application, a request is executed by the architecture that may require dozens of different services to participate to produce a response. The path of this request is a distributed transaction. The distributed tracing platform (Jaeger) lets you perform distributed tracing, which follows the path of a request through various microservices that make up an application.

Distributed tracing is a technique that is used to tie the information about different units of work together—usually executed in different processes or hosts—to understand a whole chain of events in a distributed transaction. Distributed tracing lets developers visualize call flows in large service oriented architectures. It can be invaluable in understanding serialization, parallelism, and sources of latency.

The distributed tracing platform (Jaeger) records the execution of individual requests across the whole stack of microservices, and presents them as traces. A trace is a data/execution path through the system. An end-to-end trace comprises one or more spans.

A span represents a logical unit of work that has an operation name, the start time of the operation, and the duration. Spans may be nested and ordered to model causal relationships.

1.4.4.1. Distributed tracing overview

As a service owner, you can use distributed tracing to instrument your services to gather insights into your service architecture. You can use the Red Hat OpenShift distributed tracing platform for monitoring, network profiling, and troubleshooting the interaction between components in modern, cloud-native, microservices-based applications.

With the distributed tracing platform, you can perform the following functions:

  • Monitor distributed transactions
  • Optimize performance and latency
  • Perform root cause analysis
1.4.4.2. Red Hat OpenShift distributed tracing platform architecture

Red Hat OpenShift distributed tracing platform is made up of several components that work together to collect, store, and display tracing data.

  • Red Hat OpenShift distributed tracing platform (Tempo) - This component is based on the open source Grafana Tempo project.

    • Gateway – The Gateway handles authentication, authorization, and forwarding requests to the Distributor or Query front-end service.
    • Distributor – The Distributor accepts spans in multiple formats including Jaeger, OpenTelemetry, and Zipkin. It routes spans to Ingesters by hashing the traceID and using a distributed consistent hash ring.
    • Ingester – The Ingester batches a trace into blocks, creates bloom filters and indexes, and then flushes it all to the back end.
    • Query Frontend – The Query Frontend is responsible for sharding the search space for an incoming query. The search query is then sent to the Queriers. The Query Frontend deployment exposes the Jaeger UI through the Tempo Query sidecar.
    • Querier - The Querier is responsible for finding the requested trace ID in either the Ingesters or the back-end storage. Depending on parameters, it can query the Ingesters and pull Bloom indexes from the back end to search blocks in object storage.
    • Compactor – The Compactors stream blocks to and from the back-end storage to reduce the total number of blocks.
  • Red Hat build of OpenTelemetry - This component is based on the open source OpenTelemetry project.

    • OpenTelemetry Collector - The OpenTelemetry Collector is a vendor-agnostic way to receive, process, and export telemetry data. The OpenTelemetry Collector supports open-source observability data formats, for example, Jaeger and Prometheus, sending to one or more open-source or commercial back-ends. The Collector is the default location instrumentation libraries export their telemetry data.
  • Red Hat OpenShift distributed tracing platform (Jaeger) - This component is based on the open source Jaeger project.

    • Client (Jaeger client, Tracer, Reporter, instrumented application, client libraries)- The distributed tracing platform (Jaeger) clients are language-specific implementations of the OpenTracing API. They can be used to instrument applications for distributed tracing either manually or with a variety of existing open source frameworks, such as Camel (Fuse), Spring Boot (RHOAR), MicroProfile (RHOAR/Thorntail), Wildfly (EAP), and many more, that are already integrated with OpenTracing.
    • Agent (Jaeger agent, Server Queue, Processor Workers) - The distributed tracing platform (Jaeger) agent is a network daemon that listens for spans sent over User Datagram Protocol (UDP), which it batches and sends to the Collector. The agent is meant to be placed on the same host as the instrumented application. This is typically accomplished by having a sidecar in container environments such as Kubernetes.
    • Jaeger Collector (Collector, Queue, Workers) - Similar to the Jaeger agent, the Jaeger Collector receives spans and places them in an internal queue for processing. This allows the Jaeger Collector to return immediately to the client/agent instead of waiting for the span to make its way to the storage.
    • Storage (Data Store) - Collectors require a persistent storage backend. Red Hat OpenShift distributed tracing platform (Jaeger) has a pluggable mechanism for span storage. Red Hat OpenShift distributed tracing platform (Jaeger) supports the Elasticsearch storage.
    • Query (Query Service) - Query is a service that retrieves traces from storage.
    • Ingester (Ingester Service) - Red Hat OpenShift distributed tracing platform can use Apache Kafka as a buffer between the Collector and the actual Elasticsearch backing storage. Ingester is a service that reads data from Kafka and writes to the Elasticsearch storage backend.
    • Jaeger Console – With the Red Hat OpenShift distributed tracing platform (Jaeger) user interface, you can visualize your distributed tracing data. On the Search page, you can find traces and explore details of the spans that make up an individual trace.
1.4.4.3. Red Hat OpenShift distributed tracing platform features

Red Hat OpenShift distributed tracing platform provides the following capabilities:

  • Integration with Kiali – When properly configured, you can view distributed tracing platform data from the Kiali console.
  • High scalability – The distributed tracing platform back end is designed to have no single points of failure and to scale with the business needs.
  • Distributed Context Propagation – Enables you to connect data from different components together to create a complete end-to-end trace.
  • Backwards compatibility with Zipkin – Red Hat OpenShift distributed tracing platform has APIs that enable it to be used as a drop-in replacement for Zipkin, but Red Hat is not supporting Zipkin compatibility in this release.

1.4.5. Next steps

1.5. Service mesh deployment models

Red Hat OpenShift Service Mesh supports several different deployment models that can be combined in different ways to best suit your business requirements.

In Istio, a tenant is a group of users that share common access and privileges for a set of deployed workloads. You can use tenants to provide a level of isolation between different teams. You can segregate access to different tenants using NetworkPolicies, AuthorizationPolicies, and exportTo annotations on istio.io or service resources.

1.5.1. Cluster-Wide (Single Tenant) mesh deployment model

A cluster-wide deployment contains a Service Mesh Control Plane that monitors resources for an entire cluster. Monitoring resources for an entire cluster closely resembles Istio functionality in that the control plane uses a single query across all namespaces to monitor Istio and Kubernetes resources. As a result, cluster-wide deployments decrease the number of requests sent to the API server.

Similar to Istio, a cluster-wide mesh includes namespaces with the istio-injection=enabled namespace label by default. You can change this label by modifying the spec.labelSelectors field of the ServiceMeshMemberRoll resource.

1.5.2. Multitenant deployment model

Red Hat OpenShift Service Mesh installs a ServiceMeshControlPlane that is configured for multitenancy by default. Red Hat OpenShift Service Mesh uses a multitenant Operator to manage the Service Mesh control plane lifecycle. Within a mesh, namespaces are used for tenancy.

Red Hat OpenShift Service Mesh uses ServiceMeshControlPlane resources to manage mesh installations, whose scope is limited by default to namespace that contains the resource. You use ServiceMeshMemberRoll and ServiceMeshMember resources to include additional namespaces into the mesh. A namespace can only be included in a single mesh, and multiple meshes can be installed in a single OpenShift cluster.

Typical service mesh deployments use a single Service Mesh control plane to configure communication between services in the mesh. Red Hat OpenShift Service Mesh supports “soft multitenancy”, where there is one control plane and one mesh per tenant, and there can be multiple independent control planes within the cluster. Multitenant deployments specify the projects that can access the Service Mesh and isolate the Service Mesh from other control plane instances.

The cluster administrator gets control and visibility across all the Istio control planes, while the tenant administrator only gets control over their specific Service Mesh, Kiali, and Jaeger instances.

You can grant a team permission to deploy its workloads only to a given namespace or set of namespaces. If granted the mesh-user role by the service mesh administrator, users can create a ServiceMeshMember resource to add namespaces to the ServiceMeshMemberRoll.

1.5.2.1. About migrating to a cluster-wide mesh

In a cluster-wide mesh, one ServiceMeshControlPlane (SMCP) watches all of the namespaces for an entire cluster. You can migrate an existing cluster from a multitenant mesh to a cluster-wide mesh using Red Hat OpenShift Service Mesh version 2.5 or later.

Note

If a cluster must have more than one SMCP, then you cannot migrate to a cluster-wide mesh.

By default, a cluster-wide mesh discovers all of the namespaces that comprise a cluster. However, you can configure the mesh to access a limited set of namespaces. Namespaces do not receive sidecar injection by default. You must specify which namespaces receive sidecar injection.

Similarly, you must specify which pods receive sidecar injection. Pods that exist in a namespace that receives sidecar injection do not inherit sidecar injection. Applying sidecar injection to namespaces and to pods are separate operations.

If you change the Istio version when migrating to a cluster-wide mesh, then you must restart the applications. If you use the same Istio version, the application proxies will connect to the new SMCP for the cluster-wide mesh, and work the same way they did for a multitenant mesh.

1.5.2.1.1. Including and excluding namespaces from a cluster-wide mesh by using the web console

Using the OpenShift Container Platform web console, you can add discovery selectors to the ServiceMeshControlPlane resource in a cluster-wide mesh. Discovery selectors define the namespaces that the control plane can discover. The control plane ignores any namespace that does not match one of the discovery selectors, which excludes the namespace from the mesh.

Note

If you install ingress or egress gateways in the control plane namespace, you must include the control plane namespace in the discovery selectors.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlane resource.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to OperatorsInstalled Operators.
  3. Click the Red Hat OpenShift Service Mesh Operator.
  4. Click Istio Service Mesh Control Plane.
  5. Click the name of the control plane.
  6. Click YAML.
  7. Modify the YAML file so that the spec.meshConfig field of the ServiceMeshControlPlane resource includes the discovery selector.

    Note

    When configuring namespaces that the Istiod service can discover, exclude namespaces that might contain sensitive services that should not be exposed to the rest of the mesh.

    In the following example, the Istiod service discovers any namespace that is labeled istio-discovery: enabled or any namespace that has the name bookinfo, httpbin or istio-system:

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
    spec:
      mode: ClusterWide
      meshConfig:
        discoverySelectors:
        - matchLabels:
            istio-discovery: enabled 1
        - matchExpressions:
          - key: kubernetes.io/metadata.name 2
            operator: In
            values:
            - bookinfo
            - httpbin
            - istio-system
    1
    Ensures that the mesh discovers namespaces that contain the label istio-discovery: enabled.
    2
    Ensures that the mesh discovers namespaces bookinfo, httpbin and istio-system.

    If a namespace matches any of the discovery selectors, then the mesh discovers the namespace. The mesh excludes namespaces that do not match any of the discovery selectors.

  8. Save the file.
1.5.2.1.2. Including and excluding namespaces from a cluster-wide mesh by using the CLI

Using the OpenShift Container Platform CLI, you can add discovery selectors to the ServiceMeshControlPlane resource in a cluster-wide mesh. Discovery selectors define the namespaces that the control plane can discover. The control plane ignores any namespace that does not match one of the discovery selectors, which excludes the namespace from the mesh.

Note

If you install ingress or egress gateways in the control plane namespace, you must include the control plane namespace in the discovery selectors.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlane resource.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform CLI.
  2. Open the ServiceMeshControlPlane resource as a YAML file by running the following command:

    $ oc -n istio-system edit smcp <name> 1
    1
    <name> represents the name of the ServiceMeshControlPlane resource.
  3. Modify the YAML file so that the spec.meshConfig field of the ServiceMeshControlPlane resource includes the discovery selector.

    Note

    When configuring namespaces that the Istiod service can discover, exclude namespaces that might contain sensitive services that should not be exposed to the rest of the mesh.

    In the following example, the Istiod service discovers any namespace that is labeled istio-discovery: enabled or any namespace that has the name bookinfo, httpbin or istio-system:

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
    spec:
      mode: ClusterWide
      meshConfig:
        discoverySelectors:
        - matchLabels:
            istio-discovery: enabled 1
        - matchExpressions:
          - key: kubernetes.io/metadata.name 2
            operator: In
            values:
            - bookinfo
            - httpbin
            - istio-system
    1
    Ensures that the mesh discovers namespaces that contain the label istio-discovery: enabled.
    2
    Ensures that the mesh discovers namespaces bookinfo, httpbin and istio-system.

    If a namespace matches any of the discovery selectors, then the mesh discovers the namespace. The mesh excludes namespaces that do not match any of the discovery selectors.

  4. Save the file and exit the editor.
1.5.2.1.3. Defining which namespaces receive sidecar injection in a cluster-wide mesh by using the web console

By default, the Red Hat OpenShift Service Mesh Operator uses member selectors to identify which namespaces receive sidecar injection. Namespaces that do not match the istio-injection=enabled label as defined in the ServiceMeshMemberRoll resource do not receive sidecar injection.

Note

Using discovery selectors to determine which namespaces the mesh can discover has no effect on sidecar injection. Discovering namespaces and configuring sidecar injection are separate operations.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlanae resource with the mode: ClusterWide annotation.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to OperatorsInstalled Operators.
  3. Click the Red Hat OpenShift Service Mesh Operator.
  4. Click Istio Service Mesh Member Roll.
  5. Click the ServiceMeshMemberRoll resource.
  6. Click YAML.
  7. Modify the spec.memberSelectors field in the ServiceMeshMemberRoll resource by adding a member selector that matches the inject label. The following example uses istio-injection: enabled:

    apiVersion: maistra.io/v1
    kind: ServiceMeshMemberRoll
    metadata:
      name: default
    spec:
      memberSelectors:
      - matchLabels:
          istio-injection: enabled 1
    1
    Ensures that the namespace receives sidecar injection.
  8. Save the file.
1.5.2.1.4. Defining which namespaces receive sidecar injection in a cluster-wide mesh by using the CLI

By default, the Red Hat OpenShift Service Mesh Operator uses member selectors to identify which namespaces receive sidecar injection. Namespaces that do not match the istio-injection=enabled label as defined in the ServiceMeshMemberRoll resource do not receive sidecar injection.

Note

Using discovery selectors to determine which namespaces the mesh can discover has no effect on sidecar injection. Discovering namespaces and configuring sidecar injection are separate operations.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlanae resource with the mode: ClusterWide annotation.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform CLI.
  2. Edit the ServiceMeshMemberRoll resource.

    $ oc edit smmr -n <controlplane-namespace>
  3. Modify the spec.memberSelectors field in the ServiceMeshMemberRoll resource by adding a member selector that matches the inject label. The following example uses istio-injection: enabled:

    apiVersion: maistra.io/v1
    kind: ServiceMeshMemberRoll
    metadata:
      name: default
    spec:
      memberSelectors:
      - matchLabels:
          istio-injection: enabled 1
    1
    Ensures that the namespace receives sidecar injection.
  4. Save the file and exit the editor.
1.5.2.1.5. Excluding individual pods from a cluster-wide mesh by using the web console

A pod receives sidecar injection if it has the sidecar.istio.io/inject: true annotation applied, and the pod exists in a namespace that matches either the label selector or the members list defined in the ServiceMeshMemberRoll resource.

If a pod does not have the sidecar.istio.io/inject annotation applied, it cannot receive sidecar injection.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlane resource with the mode: ClusterWide annotation.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to WorkloadsDeployments.
  3. Click the name of the deployment.
  4. Click YAML.
  5. Modify the YAML file to deploy one application that receives sidecar injection and one that does not, as shown in the following example:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          annotations:
            sidecar.istio.io/inject: 'true' 1
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:1.14.2
            ports:
            - containerPort: 80
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-without-sidecar
    spec:
      selector:
        matchLabels:
          app: nginx-without-sidecar
      template:
        metadata:
          labels:
            app: nginx-without-sidecar 2
        spec:
          containers:
          - name: nginx
            image: nginx:1.14.2
            ports:
            - containerPort: 80
    1
    This pod has the sidecar.istio.io/inject annotation applied, so it receives sidecar injection.
    2
    This pod does not have the annotation, so it does not receive sidecar injection.
  6. Save the file.
1.5.2.1.6. Excluding individual pods from a cluster-wide mesh by using the CLI

A pod receives sidecar injection if it has the sidecar.istio.io/inject: true annotation applied, and the pod exists in a namespace that matches either the label selector or the members list defined in the ServiceMeshMemberRoll resource.

If a pod does not have the sidecar.istio.io/inject annotation applied, it cannot receive sidecar injection.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You have deployed a ServiceMeshControlPlane resource with the mode: ClusterWide annotation.
  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform CLI.
  2. Edit the deployment by running the following command:

    $ oc edit deployment -n <namespace> <deploymentName>
  3. Modify the YAML file to deploy one application that receives sidecar injection and one that does not, as shown in the following example:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          annotations:
            sidecar.istio.io/inject: 'true' 1
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:1.14.2
            ports:
            - containerPort: 80
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-without-sidecar
    spec:
      selector:
        matchLabels:
          app: nginx-without-sidecar
      template:
        metadata:
          labels:
            app: nginx-without-sidecar 2
        spec:
          containers:
          - name: nginx
            image: nginx:1.14.2
            ports:
            - containerPort: 80
    1
    This pod has the sidecar.istio.io/inject annotation applied, so it receives sidecar injection.
    2
    This pod does not have the annotation, so it does not receive sidecar injection.
  4. Save the file.

1.5.3. Multimesh or federated deployment model

Federation is a deployment model that lets you share services and workloads between separate meshes managed in distinct administrative domains.

The Istio multi-cluster model requires a high level of trust between meshes and remote access to all Kubernetes API servers on which the individual meshes reside. Red Hat OpenShift Service Mesh federation takes an opinionated approach to a multi-cluster implementation of Service Mesh that assumes minimal trust between meshes.

A federated mesh is a group of meshes behaving as a single mesh. The services in each mesh can be unique services, for example a mesh adding services by importing them from another mesh, can provide additional workloads for the same services across the meshes, providing high availability, or a combination of both. All meshes that are joined into a federated mesh remain managed individually, and you must explicitly configure which services are exported to and imported from other meshes in the federation. Support functions such as certificate generation, metrics and trace collection remain local in their respective meshes.

1.6. Service Mesh and Istio differences

Red Hat OpenShift Service Mesh differs from an installation of Istio to provide additional features or to handle differences when deploying on OpenShift Container Platform.

1.6.1. Differences between Istio and Red Hat OpenShift Service Mesh

The following features are different in Service Mesh and Istio.

1.6.1.1. Command line tool

The command line tool for Red Hat OpenShift Service Mesh is oc.  Red Hat OpenShift Service Mesh does not support istioctl.

1.6.1.2. Installation and upgrades

Red Hat OpenShift Service Mesh does not support Istio installation profiles.

Red Hat OpenShift Service Mesh does not support canary upgrades of the service mesh.

1.6.1.3. Automatic injection

The upstream Istio community installation automatically injects the sidecar into pods within the projects you have labeled.

Red Hat OpenShift Service Mesh does not automatically inject the sidecar into any pods, but you must opt in to injection using an annotation without labeling projects. This method requires fewer privileges and does not conflict with other OpenShift Container Platform capabilities such as builder pods. To enable automatic injection, specify the sidecar.istio.io/inject label, or annotation, as described in the Automatic sidecar injection section.

Table 1.4. Sidecar injection label and annotation settings
 Upstream IstioRed Hat OpenShift Service Mesh

Namespace Label

supports "enabled" and "disabled"

supports "disabled"

Pod Label

supports "true" and "false"

supports "true" and "false"

Pod Annotation

supports "false" only

supports "true" and "false"

1.6.1.4. Istio Role Based Access Control features

Istio Role Based Access Control (RBAC) provides a mechanism you can use to control access to a service. You can identify subjects by user name or by specifying a set of properties and apply access controls accordingly.

The upstream Istio community installation includes options to perform exact header matches, match wildcards in headers, or check for a header containing a specific prefix or suffix.

Red Hat OpenShift Service Mesh extends the ability to match request headers by using a regular expression. Specify a property key of request.regex.headers with a regular expression.

Upstream Istio community matching request headers example

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: httpbin-usernamepolicy
spec:
  action: ALLOW
  rules:
    - when:
        - key: 'request.regex.headers[username]'
          values:
            - "allowed.*"
  selector:
    matchLabels:
      app: httpbin

1.6.1.5. OpenSSL

Red Hat OpenShift Service Mesh replaces BoringSSL with OpenSSL. OpenSSL is a software library that contains an open source implementation of the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols. The Red Hat OpenShift Service Mesh Proxy binary dynamically links the OpenSSL libraries (libssl and libcrypto) from the underlying Red Hat Enterprise Linux operating system.

1.6.1.6. External workloads

Red Hat OpenShift Service Mesh does not support external workloads, such as virtual machines running outside OpenShift on bare metal servers.

1.6.1.7. Virtual Machine Support

You can deploy virtual machines to OpenShift using OpenShift Virtualization. Then, you can apply a mesh policy, such as mTLS or AuthorizationPolicy, to these virtual machines, just like any other pod that is part of a mesh.

1.6.1.8. Component modifications
  • A maistra-version label has been added to all resources.
  • All Ingress resources have been converted to OpenShift Route resources.
  • Grafana, distributed tracing (Jaeger), and Kiali are enabled by default and exposed through OpenShift routes.
  • Godebug has been removed from all templates
  • The istio-multi ServiceAccount and ClusterRoleBinding have been removed, as well as the istio-reader ClusterRole.
1.6.1.9. Envoy filters

Red Hat OpenShift Service Mesh does not support EnvoyFilter configuration except where explicitly documented. Due to tight coupling with the underlying Envoy APIs, backward compatibility cannot be maintained. EnvoyFilter patches are very sensitive to the format of the Envoy configuration that is generated by Istio. If the configuration generated by Istio changes, it has the potential to break the application of the EnvoyFilter.

1.6.1.10. Envoy services

Red Hat OpenShift Service Mesh does not support QUIC-based services.

1.6.1.11. Istio Container Network Interface (CNI) plugin

Red Hat OpenShift Service Mesh includes CNI plugin, which provides you with an alternate way to configure application pod networking. The CNI plugin replaces the init-container network configuration eliminating the need to grant service accounts and projects access to security context constraints (SCCs) with elevated privileges.

Note

By default, Istio Container Network Interface (CNI) pods are created on all OpenShift Container Platform nodes. To exclude the creation of CNI pods in a specific node, apply the maistra.io/exclude-cni=true label to the node. Adding this label removes any previously deployed Istio CNI pods from the node.

1.6.1.12. Global mTLS settings

Red Hat OpenShift Service Mesh creates a PeerAuthentication resource that enables or disables Mutual TLS authentication (mTLS) within the mesh.

1.6.1.13. Gateways

Red Hat OpenShift Service Mesh installs ingress and egress gateways by default. You can disable gateway installation in the ServiceMeshControlPlane (SMCP) resource by using the following settings:

  • spec.gateways.enabled=false to disable both ingress and egress gateways.
  • spec.gateways.ingress.enabled=false to disable ingress gateways.
  • spec.gateways.egress.enabled=false to disable egress gateways.
Note

The Operator annotates the default gateways to indicate that they are generated by and managed by the Red Hat OpenShift Service Mesh Operator.

1.6.1.14. Multicluster configurations

Red Hat OpenShift Service Mesh support for multicluster configurations is limited to the federation of service meshes across multiple clusters.

1.6.1.15. Custom Certificate Signing Requests (CSR)

You cannot configure Red Hat OpenShift Service Mesh to process CSRs through the Kubernetes certificate authority (CA).

1.6.1.16. Routes for Istio Gateways

OpenShift routes for Istio Gateways are automatically managed in Red Hat OpenShift Service Mesh. Every time an Istio Gateway is created, updated or deleted inside the service mesh, an OpenShift route is created, updated or deleted.

A Red Hat OpenShift Service Mesh control plane component called Istio OpenShift Routing (IOR) synchronizes the gateway route. For more information, see Automatic route creation.

1.6.1.16.1. Catch-all domains

Catch-all domains ("*") are not supported. If one is found in the Gateway definition, Red Hat OpenShift Service Mesh will create the route, but will rely on OpenShift to create a default hostname. This means that the newly created route will not be a catch all ("*") route, instead it will have a hostname in the form <route-name>[-<project>].<suffix>. See the OpenShift Container Platform documentation for more information about how default hostnames work and how a cluster-admin can customize it. If you use Red Hat OpenShift Dedicated, refer to the Red Hat OpenShift Dedicated the dedicated-admin role.

1.6.1.16.2. Subdomains

Subdomains (e.g.: "*.domain.com") are supported. However this ability doesn’t come enabled by default in OpenShift Container Platform. This means that Red Hat OpenShift Service Mesh will create the route with the subdomain, but it will only be in effect if OpenShift Container Platform is configured to enable it.

1.6.1.16.3. Transport layer security

Transport Layer Security (TLS) is supported. This means that, if the Gateway contains a tls section, the OpenShift Route will be configured to support TLS.

Additional resources

1.6.2. Multitenant installations

Whereas upstream Istio takes a single tenant approach, Red Hat OpenShift Service Mesh supports multiple independent control planes within the cluster. Red Hat OpenShift Service Mesh uses a multitenant operator to manage the control plane lifecycle.

Red Hat OpenShift Service Mesh installs a multitenant control plane by default. You specify the projects that can access the Service Mesh, and isolate the Service Mesh from other control plane instances.

1.6.2.1. Multitenancy versus cluster-wide installations

The main difference between a multitenant installation and a cluster-wide installation is the scope of privileges used by istod. The components no longer use cluster-scoped Role Based Access Control (RBAC) resource ClusterRoleBinding.

Every project in the ServiceMeshMemberRoll members list will have a RoleBinding for each service account associated with the control plane deployment and each control plane deployment will only watch those member projects. Each member project has a maistra.io/member-of label added to it, where the member-of value is the project containing the control plane installation.

Red Hat OpenShift Service Mesh configures each member project to ensure network access between itself, the control plane, and other member projects. The exact configuration differs depending on how OpenShift Container Platform software-defined networking (SDN) is configured. See About OpenShift SDN for additional details.

If the OpenShift Container Platform cluster is configured to use the SDN plugin:

  • NetworkPolicy: Red Hat OpenShift Service Mesh creates a NetworkPolicy resource in each member project allowing ingress to all pods from the other members and the control plane. If you remove a member from Service Mesh, this NetworkPolicy resource is deleted from the project.

    Note

    This also restricts ingress to only member projects. If you require ingress from non-member projects, you need to create a NetworkPolicy to allow that traffic through.

  • Multitenant: Red Hat OpenShift Service Mesh joins the NetNamespace for each member project to the NetNamespace of the control plane project (the equivalent of running oc adm pod-network join-projects --to control-plane-project member-project). If you remove a member from the Service Mesh, its NetNamespace is isolated from the control plane (the equivalent of running oc adm pod-network isolate-projects member-project).
  • Subnet: No additional configuration is performed.
1.6.2.2. Cluster scoped resources

Upstream Istio has two cluster scoped resources that it relies on. The MeshPolicy and the ClusterRbacConfig. These are not compatible with a multitenant cluster and have been replaced as described below.

  • ServiceMeshPolicy replaces MeshPolicy for configuration of control-plane-wide authentication policies. This must be created in the same project as the control plane.
  • ServicemeshRbacConfig replaces ClusterRbacConfig for configuration of control-plane-wide role based access control. This must be created in the same project as the control plane.

1.6.3. Kiali and service mesh

Installing Kiali via the Service Mesh on OpenShift Container Platform differs from community Kiali installations in multiple ways. These modifications are sometimes necessary to resolve issues, provide additional features, or to handle differences when deploying on OpenShift Container Platform.

  • Kiali has been enabled by default.
  • Ingress has been enabled by default.
  • Updates have been made to the Kiali ConfigMap.
  • Updates have been made to the ClusterRole settings for Kiali.
  • Do not edit the ConfigMap, because your changes might be overwritten by the Service Mesh or Kiali Operators. Files that the Kiali Operator manages have a kiali.io/ label or annotation. Updating the Operator files should be restricted to those users with cluster-admin privileges. If you use Red Hat OpenShift Dedicated, updating the Operator files should be restricted to those users with dedicated-admin privileges.

1.6.4. Distributed tracing and service mesh

Installing the distributed tracing platform (Jaeger) with the Service Mesh on OpenShift Container Platform differs from community Jaeger installations in multiple ways. These modifications are sometimes necessary to resolve issues, provide additional features, or to handle differences when deploying on OpenShift Container Platform.

  • Distributed tracing has been enabled by default for Service Mesh.
  • Ingress has been enabled by default for Service Mesh.
  • The name for the Zipkin port name has changed to jaeger-collector-zipkin (from http)
  • Jaeger uses Elasticsearch for storage by default when you select either the production or streaming deployment option.
  • The community version of Istio provides a generic "tracing" route. Red Hat OpenShift Service Mesh uses a "jaeger" route that is installed by the Red Hat OpenShift distributed tracing platform (Jaeger) Operator and is already protected by OAuth.
  • Red Hat OpenShift Service Mesh uses a sidecar for the Envoy proxy, and Jaeger also uses a sidecar, for the Jaeger agent. These two sidecars are configured separately and should not be confused with each other. The proxy sidecar creates spans related to the pod’s ingress and egress traffic. The agent sidecar receives the spans emitted by the application and sends them to the Jaeger Collector.

1.7. Preparing to install Service Mesh

Before you can install Red Hat OpenShift Service Mesh, you must subscribe to OpenShift Container Platform and install OpenShift Container Platform in a supported configuration.

1.7.1. Prerequisites

For additional information about Red Hat OpenShift Service Mesh lifecycle and supported platforms, refer to the Support Policy.

1.7.2. Supported configurations

The following configurations are supported for the current release of Red Hat OpenShift Service Mesh.

1.7.2.1. Supported platforms

The Red Hat OpenShift Service Mesh Operator supports multiple versions of the ServiceMeshControlPlane resource. Version 2.5 Service Mesh control planes are supported on the following platform versions:

  • Red Hat OpenShift Container Platform version 4.10 or later.
  • Red Hat OpenShift Dedicated version 4.
  • Azure Red Hat OpenShift (ARO) version 4.
  • Red Hat OpenShift Service on AWS (ROSA).
1.7.2.2. Unsupported configurations

Explicitly unsupported cases include:

  • OpenShift Online is not supported for Red Hat OpenShift Service Mesh.
  • Red Hat OpenShift Service Mesh does not support the management of microservices outside the cluster where Service Mesh is running.
1.7.2.3. Supported network configurations

Red Hat OpenShift Service Mesh supports the following network configurations.

  • OpenShift-SDN
  • OVN-Kubernetes is available on all supported versions of OpenShift Container Platform.
  • Third-Party Container Network Interface (CNI) plugins that have been certified on OpenShift Container Platform and passed Service Mesh conformance testing. See Certified OpenShift CNI Plug-ins for more information.
1.7.2.4. Supported configurations for Service Mesh
  • This release of Red Hat OpenShift Service Mesh is only available on OpenShift Container Platform x86_64, IBM Z, and IBM Power.

    • IBM Z is only supported on OpenShift Container Platform 4.10 and later.
    • IBM Power is only supported on OpenShift Container Platform 4.10 and later.
  • Configurations where all Service Mesh components are contained within a single OpenShift Container Platform cluster.
  • Configurations that do not integrate external services such as virtual machines.
  • Red Hat OpenShift Service Mesh does not support EnvoyFilter configuration except where explicitly documented.
1.7.2.5. Supported configurations for Kiali
  • The Kiali console is only supported on the two most recent releases of the Google Chrome, Microsoft Edge, Mozilla Firefox, or Apple Safari browsers.
  • The openshift authentication strategy is the only supported authentication configuration when Kiali is deployed with Red Hat OpenShift Service Mesh (OSSM). The openshift strategy controls access based on the individual’s role-based access control (RBAC) roles of the OpenShift Container Platform.
1.7.2.6. Supported configurations for Distributed Tracing
  • Jaeger agent as a sidecar is the only supported configuration for Jaeger. Jaeger as a daemonset is not supported for multitenant installations or OpenShift Dedicated.
1.7.2.7. Supported WebAssembly module
  • 3scale WebAssembly is the only provided WebAssembly module. You can create custom WebAssembly modules.

1.7.3. Next steps

1.8. Installing the Operators

To install Red Hat OpenShift Service Mesh, first install the Red Hat OpenShift Service Mesh Operator and any optional Operators on OpenShift Container Platform. Then create a ServiceMeshControlPlane resource to deploy the control plane.

Note

This basic installation is configured based on the default OpenShift settings and is not designed for production use.  Use this default installation to verify your installation, and then configure your service mesh for your specific environment.

Prerequisites

The following steps show how to install a basic instance of Red Hat OpenShift Service Mesh on OpenShift Container Platform.

Important

Starting with Red Hat OpenShift Service Mesh 2.5, Red Hat OpenShift distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator are deprecated and will be removed in a future release. Red Hat will provide bug fixes and support for these features during the current release lifecycle, but this feature will no longer receive enhancements and will be removed. As an alternative to Red Hat OpenShift distributed tracing platform (Jaeger), you can use Red Hat OpenShift distributed tracing platform (Tempo) instead.

1.8.1. Service Mesh Operators overview

Red Hat OpenShift Service Mesh requires the use of the Red Hat OpenShift Service Mesh Operator which allows you to connect, secure, control, and observe the microservices that comprise your applications. You can also install other Operators to enhance your service mesh experience.

Warning

Do not install Community versions of the Operators. Community Operators are not supported.

The following Operator is required:

Red Hat OpenShift Service Mesh Operator
Allows you to connect, secure, control, and observe the microservices that comprise your applications. It also defines and monitors the ServiceMeshControlPlane resources that manage the deployment, updating, and deletion of the Service Mesh components. It is based on the open source Istio project.

The following Operators are optional:

Kiali Operator provided by Red Hat
Provides observability for your service mesh. You can view configurations, monitor traffic, and analyze traces in a single console. It is based on the open source Kiali project.
Red Hat OpenShift distributed tracing platform (Tempo)
Provides distributed tracing to monitor and troubleshoot transactions in complex distributed systems. It is based on the open source Grafana Tempo project.

The following optional Operators are deprecated:

Important

Starting with Red Hat OpenShift Service Mesh 2.5, Red Hat OpenShift distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator are deprecated and will be removed in a future release. Red Hat will provide bug fixes and support for these features during the current release lifecycle, but these features will no longer receive enhancements and will be removed. As an alternative to Red Hat OpenShift distributed tracing platform (Jaeger), you can use Red Hat OpenShift distributed tracing platform (Tempo) instead.

Red Hat OpenShift distributed tracing platform (Jaeger)
Provides distributed tracing to monitor and troubleshoot transactions in complex distributed systems. It is based on the open source Jaeger project.
OpenShift Elasticsearch Operator
Provides database storage for tracing and logging with the distributed tracing platform (Jaeger). It is based on the open source Elasticsearch project.

1.8.2. Installing the Operators

To install Red Hat OpenShift Service Mesh, you must install the Red Hat OpenShift Service Mesh Operator. Repeat the procedure for each additional Operator you want to install.

Additional Operators include:

  • Kiali Operator provided by Red Hat
  • Tempo Operator

Deprecated additional Operators include:

Important

Starting with Red Hat OpenShift Service Mesh 2.5, Red Hat OpenShift distributed tracing platform (Jaeger) and OpenShift Elasticsearch Operator are deprecated and will be removed in a future release. Red Hat will provide bug fixes and support for these features during the current release lifecycle, but this feature will no longer receive enhancements and will be removed. As an alternative to Red Hat OpenShift distributed tracing platform (Jaeger), you can use Red Hat OpenShift distributed tracing platform (Tempo) instead.

  • Red Hat OpenShift distributed tracing platform (Jaeger)
  • OpenShift Elasticsearch Operator
Note

If you have already installed the OpenShift Elasticsearch Operator as part of OpenShift Logging, you do not need to install the OpenShift Elasticsearch Operator again. The Red Hat OpenShift distributed tracing platform (Jaeger) Operator creates the Elasticsearch instance using the installed OpenShift Elasticsearch Operator.

Procedure

  1. Log in to the OpenShift Container Platform web console as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.
  2. In the OpenShift Container Platform web console, click OperatorsOperatorHub.
  3. Type the name of the Operator into the filter box and select the Red Hat version of the Operator. Community versions of the Operators are not supported.
  4. Click Install.
  5. On the Install Operator page for each Operator, accept the default settings.
  6. Click Install. Wait until the Operator installs before repeating the steps for the next Operator you want to install.

    • The Red Hat OpenShift Service Mesh Operator installs in the openshift-operators namespace and is available for all namespaces in the cluster.
    • The Kiali Operator provided by Red Hat installs in the openshift-operators namespace and is available for all namespaces in the cluster.
    • The Tempo Operator installs in the openshift-tempo-operator namespace and is available for all namespaces in the cluster.
    • The Red Hat OpenShift distributed tracing platform (Jaeger) installs in the openshift-distributed-tracing namespace and is available for all namespaces in the cluster.

      Important

      Starting with Red Hat OpenShift Service Mesh 2.5, Red Hat OpenShift distributed tracing platform (Jaeger) is deprecated and will be removed in a future release. Red Hat will provide bug fixes and support for this feature during the current release lifecycle, but this feature will no longer receive enhancements and will be removed. As an alternative to Red Hat OpenShift distributed tracing platform (Jaeger), you can use Red Hat OpenShift distributed tracing platform (Tempo) instead.

    • The OpenShift Elasticsearch Operator installs in the openshift-operators-redhat namespace and is available for all namespaces in the cluster.

      Important

      Starting with Red Hat OpenShift Service Mesh 2.5, OpenShift Elasticsearch Operator is deprecated and will be removed in a future release. Red Hat will provide bug fixes and support for this feature during the current release lifecycle, but this feature will no longer receive enhancements and will be removed.

Verification

  • After all you have installed all four Operators, click OperatorsInstalled Operators to verify that your Operators are installed.

1.8.3. Configuring the Service Mesh Operator to run on infrastructure nodes

This task should only be performed if the Service Mesh Operator runs on an infrastructure node.

If the operator will run on a worker node, skip this task.

Prerequisites

  • The Service Mesh Operator must be installed.
  • One of the nodes comprising the deployment must be an infrastructure node. For more information, see "Creating infrastructure machine sets."

Procedure

  1. List the operators installed in the namespace:

    $ oc -n openshift-operators get subscriptions
  2. Edit the Service Mesh Operator Subscription resource to specify where the operator should run:

    $ oc -n openshift-operators edit subscription <name> 1
    1
    <name> represents the name of the Subscription resource. The default name of the Subscription resource is servicemeshoperator.
  3. Add the nodeSelector and tolerations to spec.config in the Subscription resource:

    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      labels:
        operators.coreos.com/servicemeshoperator.openshift-operators: ""
      name: servicemeshoperator
      namespace: openshift-operators
    # ...
    spec:
      config:
        nodeSelector: 1
          node-role.kubernetes.io/infra: ""
        tolerations: 2
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
        - effect: NoExecute
          key: node-role.kubernetes.io/infra
          value: reserved
    1
    Ensures that the operator pod is only scheduled on an infrastructure node.
    2
    Ensures that the pod is accepted by the infrastructure node.

1.8.4. Verifying the Service Mesh Operator is running on infrastructure node

Procedure

  • Verify that the node associated with the Operator pod is an infrastructure node:

    $ oc -n openshift-operators get po -l name=istio-operator -owide

1.8.5. Next steps

  • The Red Hat OpenShift Service Mesh Operator does not create the Service Mesh custom resource definitions (CRDs) until you deploy a Service Mesh control plane. You can use the ServiceMeshControlPlane resource to install and configure the Service Mesh components. For more information, see Creating the ServiceMeshControlPlane.

1.9. Creating the ServiceMeshControlPlane

1.9.1. About ServiceMeshControlPlane

The control plane includes Istiod, Ingress and Egress Gateways, and other components, such as Kiali and Jaeger. The control plane must be deployed in a separate namespace than the Service Mesh Operators and the data plane applications and services. You can deploy a basic installation of the ServiceMeshControlPlane(SMCP) from the OpenShift Container Platform web console or the command line using the oc client tool.

Note

This basic installation is configured based on the default OpenShift Container Platform settings and is not designed for production use. Use this default installation to verify your installation, and then configure your ServiceMeshControlPlane settings for your environment.

Note

The Service Mesh documentation uses istio-system as the example project, but you can deploy the service mesh to any project.

1.9.1.1. Deploying the Service Mesh control plane from the web console

You can deploy a basic ServiceMeshControlPlane by using the web console. In this example, istio-system is the name of the Service Mesh control plane project.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator must be installed.
  • An account with the cluster-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.
  2. Create a project named istio-system.

    1. Navigate to HomeProjects.
    2. Click Create Project.
    3. In the Name field, enter istio-system. The ServiceMeshControlPlane resource must be installed in a project that is separate from your microservices and Operators.

      These steps use istio-system as an example, but you can deploy your Service Mesh control plane in any project as long as it is separate from the project that contains your services.

    4. Click Create.
  3. Navigate to OperatorsInstalled Operators.
  4. Click the Red Hat OpenShift Service Mesh Operator, then click Istio Service Mesh Control Plane.
  5. On the Istio Service Mesh Control Plane tab, click Create ServiceMeshControlPlane.

    1. Accept the default Service Mesh control plane version to take advantage of the features available in the most current version of the product. The version of the control plane determines the features available regardless of the version of the Operator.
    2. Click Create.

    The Operator creates pods, services, and Service Mesh control plane components based on your configuration parameters. You can configure ServiceMeshControlPlane settings at a later time.

Verification

  • To verify the control plane installed correctly, click the Istio Service Mesh Control Plane tab.

    1. Click the name of the new control plane.
    2. Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources the Operator created and configured.
1.9.1.2. Deploying the Service Mesh control plane using the CLI

You can deploy a basic ServiceMeshControlPlane from the command line.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator must be installed.
  • Access to the OpenShift CLI (oc).
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Create a project named istio-system.

    $ oc new-project istio-system
  2. Create a ServiceMeshControlPlane file named istio-installation.yaml using the following example. The version of the Service Mesh control plane determines the features available regardless of the version of the Operator.

    Example version 2.5 istio-installation.yaml

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
      namespace: istio-system
    spec:
      version: v2.5
      tracing:
        type: None
        sampling: 10000
      addons:
        kiali:
          enabled: true
          name: kiali
        grafana:
          enabled: true

  3. Run the following command to deploy the Service Mesh control plane, where <istio_installation.yaml> includes the full path to your file.

    $ oc create -n istio-system -f <istio_installation.yaml>
  4. To watch the progress of the pod deployment, run the following command:

    $ oc get pods -n istio-system -w

    You should see output similar to the following:

    NAME                                   READY   STATUS    RESTARTS   AGE
    grafana-b4d59bd7-mrgbr                 2/2     Running   0          65m
    istio-egressgateway-678dc97b4c-wrjkp   1/1     Running   0          108s
    istio-ingressgateway-b45c9d54d-4qg6n   1/1     Running   0          108s
    istiod-basic-55d78bbbcd-j5556          1/1     Running   0          108s
    jaeger-67c75bd6dc-jv6k6                2/2     Running   0          65m
    kiali-6476c7656c-x5msp                 1/1     Running   0          43m
    prometheus-58954b8d6b-m5std            2/2     Running   0          66m
1.9.1.3. Validating your SMCP installation with the CLI

You can validate the creation of the ServiceMeshControlPlane from the command line.

  1. Prerequisites

    • The Red Hat OpenShift Service Mesh Operator must be installed.
    • Access to the OpenShift CLI (oc).
    • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Run the following command to verify the Service Mesh control plane installation, where istio-system is the namespace where you installed the Service Mesh control plane.

    $ oc get smcp -n istio-system

    The installation has finished successfully when the STATUS column is ComponentsReady.

    NAME    READY   STATUS            PROFILES      VERSION   AGE
    basic   10/10   ComponentsReady   ["default"]   2.5.2     66m

1.9.2. About control plane components and infrastructure nodes

Infrastructure nodes provide a way to isolate infrastructure workloads for two primary purposes:

  • To prevent incurring billing costs against subscription counts
  • To separate maintenance and management of infrastructure workloads

You can configure some or all of the Service Mesh control plane components to run on infrastructure nodes.

1.9.2.1. Configuring all control plane components to run on infrastructure nodes using the web console

Perform this task if all of the components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, Ingress Gateway, and Egress Gateway, and optional applications such as Prometheus, Grafana, and Distributed Tracing.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to OperatorsInstalled Operators.
  3. Click the Red Hat OpenShift Service Mesh Operator, and then click Istio Service Mesh Control Plane.
  4. Click the name of the control plane resource. For example, basic.
  5. Click YAML.
  6. Add the nodeSelector and tolerations fields to the spec.runtime.defaults.pod specification in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
      runtime:
        defaults:
          pod:
            nodeSelector: 1
              node-role.kubernetes.io/infra: ""
            tolerations: 2
            - effect: NoSchedule
              key: node-role.kubernetes.io/infra
              value: reserved
            - effect: NoExecute
              key: node-role.kubernetes.io/infra
              value: reserved
    1
    Ensures that the ServiceMeshControlPlane pod is only scheduled on an infrastructure node.
    2
    Ensures that the pod is accepted by the infrastructure node for execution.
  7. Click Save.
  8. Click Reload.
1.9.2.2. Configuring individual control plane components to run on infrastructure nodes using the web console

Perform this task if individual components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, the Ingress Gateway, and the Egress Gateway.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Log in to the OpenShift Container Platform web console.
  2. Navigate to OperatorsInstalled Operators.
  3. Click the Red Hat OpenShift Service Mesh Operator, and then click Istio Service Mesh Control Plane.
  4. Click the name of the control plane resource. For example, basic.
  5. Click YAML.
  6. Add the nodeSelector and tolerations fields to the spec.runtime.components.pilot.pod specification in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
      runtime:
        components:
          pilot:
            pod:
              nodeSelector: 1
                node-role.kubernetes.io/infra: ""
              tolerations: 2
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
    1
    Ensures that the Istiod pod is only scheduled on an infrastructure node.
    2
    Ensures that the pod is accepted by the infrastructure node for execution.
  7. Add the nodeSelector and the tolerations fields to the spec.gateways.ingress.runtime.pod and spec.gateways.egress.runtime.pod specifications in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
      gateways:
        ingress:
          runtime:
            pod:
              nodeSelector: 1
                node-role.kubernetes.io/infra: ""
              tolerations: 2
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
        egress:
          runtime:
            pod:
              nodeSelector: 3
                node-role.kubernetes.io/infra: ""
              tolerations: 4
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
    1 3
    Ensures that the gateway pod is only scheduled on an infrastructure node
    2 4
    Ensures that the pod is accepted by the infrastructure node for execution.
  8. Click Save.
  9. Click Reload.
1.9.2.3. Configuring all control plane components to run on infrastructure nodes using the CLI

Perform this task if all of the components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, Ingress Gateway, and Egress Gateway, and optional applications such as Prometheus, Grafana, and Distributed Tracing.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Open the ServiceMeshControlPlane resource as a YAML file:

    $ oc -n istio-system edit smcp <name> 1
    1
    <name> represents the name of the ServiceMeshControlPlane resource.
  2. To run all of the Service Mesh components deployed by the ServiceMeshControlPlane on infrastructure nodes, add the nodeSelector and tolerations fields to the spec.runtime.defaults.pod spec in the ServiceMeshControlPlane resource:

    spec:
      runtime:
        defaults:
          pod:
            nodeSelector: 1
              node-role.kubernetes.io/infra: ""
            tolerations: 2
            - effect: NoSchedule
              key: node-role.kubernetes.io/infra
              value: reserved
            - effect: NoExecute
              key: node-role.kubernetes.io/infra
              value: reserved
    1
    Ensures that the SMCP pods are only scheduled on an infrastructure node.
    2
    Ensures that the pods are accepted by the infrastructure node.
1.9.2.4. Configuring individual control plane components to run on infrastructure nodes using the CLI

Perform this task if individual components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, the Ingress Gateway, and the Egress Gateway.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Open the ServiceMeshControlPlane resource as a YAML file.

    $ oc -n istio-system edit smcp <name> 1
    1
    <name> represents the name of the ServiceMeshControlPlane resource.
  2. To run the Istiod component on an infrastructure node, add the nodeSelector and the tolerations fields to the spec.runtime.components.pilot.pod spec in the ServiceMeshControlPlane resource.

    spec:
      runtime:
        components:
          pilot:
            pod:
              nodeSelector: 1
                node-role.kubernetes.io/infra: ""
              tolerations: 2
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
    1
    Ensures that the Istiod pod is only scheduled on an infrastructure node.
    2
    Ensures that the pod is accepted by the infrastructure node.
  3. To run Ingress and Egress Gateways on infrastructure nodes, add the nodeSelector and the tolerations fields to the spec.gateways.ingress.runtime.pod spec and the spec.gateways.egress.runtime.pod spec in the ServiceMeshControlPlane resource.

    spec:
      gateways:
        ingress:
          runtime:
            pod:
              nodeSelector: 1
                node-role.kubernetes.io/infra: ""
              tolerations: 2
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
        egress:
          runtime:
            pod:
              nodeSelector: 3
                node-role.kubernetes.io/infra: ""
              tolerations: 4
              - effect: NoSchedule
                key: node-role.kubernetes.io/infra
                value: reserved
              - effect: NoExecute
                key: node-role.kubernetes.io/infra
                value: reserved
    1 3
    Ensures that the gateway pod is only scheduled on an infrastructure node
    2 4
    Ensures that the pod is accepted by the infrastructure node.
1.9.2.5. Verifying the Service Mesh control plane is running on infrastructure nodes

Procedure

  • Confirm that the nodes associated with Istiod, Ingress Gateway, and Egress Gateway pods are infrastructure nodes:

    $ oc -n istio-system get pods -owide

1.9.3. About control plane and cluster-wide deployments

A cluster-wide deployment contains a Service Mesh Control Plane that monitors resources for an entire cluster. Monitoring resources for an entire cluster closely resembles Istio functionality in that the control plane uses a single query across all namespaces to monitor Istio and Kubernetes resources. As a result, cluster-wide deployments decrease the number of requests sent to the API server.

You can configure the Service Mesh Control Plane for cluster-wide deployments using either the OpenShift Container Platform web console or the CLI.

1.9.3.1. Configuring the control plane for cluster-wide deployment with the web console

You can configure the ServiceMeshControlPlane resource for cluster-wide deployment using the OpenShift Container Platform web console. In this example, istio-system is the name of the Service Mesh control plane project.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator is installed.
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Create a project named istio-system.

    1. Navigate to HomeProjects.
    2. Click Create Project.
    3. In the Name field, enter istio-system. The ServiceMeshControlPlane resource must be installed in a project that is separate from your microservices and Operators.

      These steps use istio-system as an example. You can deploy the Service Mesh control plane to any project as long as it is separate from the project that contains your services.

    4. Click Create.
  2. Navigate to OperatorsInstalled Operators.
  3. Click the Red Hat OpenShift Service Mesh Operator, then click Istio Service Mesh Control Plane.
  4. On the Istio Service Mesh Control Plane tab, click Create ServiceMeshControlPlane.
  5. Click YAML view. The version of the Service Mesh control plane determines the features available regardless of the version of the Operator.
  6. Modify the spec.mode field of the YAML file to specify ClusterWide.

    Example version 2.5 istio-installation.yaml

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
      namespace: istio-system
    spec:
      version: v2.5
      mode: ClusterWide

  7. Click Create. The Operator creates pods, services, and Service Mesh control plane components based on your configuration parameters. The operator also creates the ServiceMeshMemberRoll if it does not exist as part of the default configuration.

Verification

  • To verify that the control plane installed correctly:

    1. Click the Istio Service Mesh Control Plane tab.
    2. Click the name of the new ServiceMeshControlPlane object.
    3. Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources that the Operator created and configured.
1.9.3.2. Configuring the control plane for cluster-wide deployment with the CLI

You can configure the ServiceMeshControlPlane resource for cluster-wide deployment using the CLI. In this example, istio-system is the name of the Service Mesh control plane namespace.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator is installed.
  • You have access to the OpenShift CLI (oc).
  • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. Create a project named istio-system.

    $ oc new-project istio-system
  2. Create a ServiceMeshControlPlane file named istio-installation.yaml using the following example:

    Example version 2.5 istio-installation.yaml

    apiVersion: maistra.io/v2
    kind: ServiceMeshControlPlane
    metadata:
      name: basic
      namespace: istio-system
    spec:
      version: v2.5
      mode: ClusterWide

  3. Run the following command to deploy the Service Mesh control plane:

    $ oc create -n istio-system -f <istio_installation.yaml>

    where:

    <istio_installation.yaml>
    Specifies the full path to your file.

Verification

  1. To monitor the progress of the pod deployment, run the following command:

    $ oc get pods -n istio-system -w

    You should see output similar to the following example:

    Example output

    NAME                                   READY   STATUS    RESTARTS   AGE
    grafana-b4d59bd7-mrgbr                 2/2     Running   0          65m
    istio-egressgateway-678dc97b4c-wrjkp   1/1     Running   0          108s
    istio-ingressgateway-b45c9d54d-4qg6n   1/1     Running   0          108s
    istiod-basic-55d78bbbcd-j5556          1/1     Running   0          108s
    jaeger-67c75bd6dc-jv6k6                2/2     Running   0          65m
    kiali-6476c7656c-x5msp                 1/1     Running   0          43m
    prometheus-58954b8d6b-m5std            2/2     Running   0          66m

1.9.3.3. Customizing the member roll for a cluster-wide mesh

In cluster-wide mode, when you create the ServiceMeshControlPlane resource, the ServiceMeshMemberRoll resource is also created. You can modify the ServiceMeshMemberRoll resource after it gets created. After you modify the resource, the Service Mesh operator no longer changes it. If you modify the ServiceMeshMemberRoll resource by using the OpenShift Container Platform web console, accept the prompt to overwrite the modifications.

Alternatively, you can create a ServiceMeshMemberRoll resource before deploying the ServiceMeshControlPlane resource. When you create the ServiceMeshControlPlane resource, the Service Mesh Operator will not modify the ServiceMeshMemberRoll.

Note

The ServiceMeshMemberRoll resource name must be named default and must be created in the same project namespace as the ServiceMeshControlPlane resource.

There are two ways to add a namespace to the mesh. You can either add the namespace by specifying its name in the spec.members list, or configure a set of namespace label selectors to include or exclude namespaces based on their labels.

Note

Regardless of how members are specified in the ServiceMeshMemberRoll resource, you can also add members to the mesh by creating the ServiceMeshMember resource in each namespace.

1.9.4. Validating your SMCP installation with Kiali

You can use the Kiali console to validate your Service Mesh installation. The Kiali console offers several ways to validate your Service Mesh components are deployed and configured properly.

  1. Prerequisites

    • The Red Hat OpenShift Service Mesh Operator must be installed.
    • Access to the OpenShift CLI (oc).
    • You are logged in to OpenShift Container Platform as`cluster-admin`.

Procedure

  1. In the OpenShift Container Platform web console, navigate to NetworkingRoutes.
  2. On the Routes page, select the Service Mesh control plane project, for example istio-system, from the Namespace menu.

    The Location column displays the linked address for each route.

  3. If necessary, use the filter to find the route for the Kiali console. Click the route Location to launch the console.
  4. Click Log In With OpenShift.

    When you first log in to the Kiali Console, you see the Overview page which displays all the namespaces in your service mesh that you have permission to view. When there are multiple namespaces shown on the Overview page, Kiali shows namespaces with health or validation problems first.

    Figure 1.1. Kiali Overview page

    Kiali Overview page showing istio-system

    The tile for each namespace displays the number of labels, the Istio Config health, the number of and Applications health, and Traffic for the namespace. If you are validating the console installation and namespaces have not yet been added to the mesh, there might not be any data to display other than istio-system.

  5. Kiali has four dashboards specifically for the namespace where the Service Mesh control plane is installed. To view these dashboards, click the Options menu kebab on the tile for the control plane namespace, for example, istio-system, and select one of the following options:

    • Istio Mesh Dashboard
    • Istio Control Plane Dashboard
    • Istio Performance Dashboard
    • Istio Wasm Exetension Dashboard

      Figure 1.2. Grafana Istio Control Plane Dashboard

      Istio Control Plane Dashboard showing data for bookinfo sample project

      Kiali also installs two additional Grafana dashboards, available f