Service Mesh
Service Mesh installation, usage, and release notes
Abstract
Chapter 1. Service Mesh 2.x
1.1. About OpenShift Service Mesh
Because Red Hat OpenShift Service Mesh releases on a different cadence from OpenShift Container Platform and because the Red Hat OpenShift Service Mesh Operator supports deploying multiple versions of the ServiceMeshControlPlane
, the Service Mesh documentation does not maintain separate documentation sets for minor versions of the product. The current documentation set applies to all currently supported versions of Service Mesh unless version-specific limitations are called out in a particular topic or for a particular feature.
For additional information about the Red Hat OpenShift Service Mesh life cycle and supported platforms, refer to the Platform Life Cycle Policy.
1.1.1. Introduction to Red Hat OpenShift Service Mesh
Red Hat OpenShift Service Mesh addresses a variety of problems in a microservice architecture by creating a centralized point of control in an application. It adds a transparent layer on existing distributed applications without requiring any changes to the application code.
Microservice architectures split the work of enterprise applications into modular services, which can make scaling and maintenance easier. However, as an enterprise application built on a microservice architecture grows in size and complexity, it becomes difficult to understand and manage. Service Mesh can address those architecture problems by capturing or intercepting traffic between services and can modify, redirect, or create new requests to other services.
Service Mesh, which is based on the open source Istio project, provides an easy way to create a network of deployed services that provides discovery, load balancing, service-to-service authentication, failure recovery, metrics, and monitoring. A service mesh also provides more complex operational functionality, including A/B testing, canary releases, access control, and end-to-end authentication.
1.1.2. Core features
Red Hat OpenShift Service Mesh provides a number of key capabilities uniformly across a network of services:
- Traffic Management - Control the flow of traffic and API calls between services, make calls more reliable, and make the network more robust in the face of adverse conditions.
- Service Identity and Security - Provide services in the mesh with a verifiable identity and provide the ability to protect service traffic as it flows over networks of varying degrees of trustworthiness.
- Policy Enforcement - Apply organizational policy to the interaction between services, ensure access policies are enforced and resources are fairly distributed among consumers. Policy changes are made by configuring the mesh, not by changing application code.
- Telemetry - Gain understanding of the dependencies between services and the nature and flow of traffic between them, providing the ability to quickly identify issues.
1.2. Service Mesh Release Notes
1.2.1. Making open source more inclusive
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
1.2.2. New features and enhancements
This release adds improvements related to the following components and concepts.
1.2.2.1. New features Red Hat OpenShift Service Mesh version 2.2.3
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.1.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.3
Component | Version |
---|---|
Istio | 1.12.9 |
Envoy Proxy | 1.20.8 |
Jaeger | 1.36 |
Kiali | 1.48.3 |
1.2.2.2. New features Red Hat OpenShift Service Mesh version 2.2.2
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.2.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.2
Component | Version |
---|---|
Istio | 1.12.7 |
Envoy Proxy | 1.20.6 |
Jaeger | 1.36 |
Kiali | 1.48.2-1 |
1.2.2.2.2. Copy route labels
With this enhancement, in addition to copying annotations, you can copy specific labels for an OpenShift route. Red Hat OpenShift Service Mesh copies all labels and annotations present in the Istio Gateway resource (with the exception of annotations starting with kubectl.kubernetes.io) into the managed OpenShift Route resource.
1.2.2.3. New features Red Hat OpenShift Service Mesh version 2.2.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.3.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2.1
Component | Version |
---|---|
Istio | 1.12.7 |
Envoy Proxy | 1.20.6 |
Jaeger | 1.34.1 |
Kiali | 1.48.2-1 |
1.2.2.4. New features Red Hat OpenShift Service Mesh 2.2
This release of Red Hat OpenShift Service Mesh adds new features and enhancements, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.4.1. Component versions included in Red Hat OpenShift Service Mesh version 2.2
Component | Version |
---|---|
Istio | 1.12.7 |
Envoy Proxy | 1.20.4 |
Jaeger | 1.34.1 |
Kiali | 1.48.0.16 |
1.2.2.4.2. WasmPlugin
API
This release adds support for the WasmPlugin
API and deprecates the ServiceMeshExtention
API.
1.2.2.4.3. ROSA support
This release introduces service mesh support for Red Hat OpenShift on AWS (ROSA), including multi-cluster federation.
1.2.2.4.4. istio-node
DaemonSet renamed
This release, the istio-node
DaemonSet is renamed to istio-cni-node
to match the name in upstream Istio.
1.2.2.4.5. Envoy sidecar networking changes
Istio 1.10 updated Envoy to send traffic to the application container using eth0
rather than lo
by default.
1.2.2.4.6. Service Mesh Control Plane 1.1
This release marks the end of support for Service Mesh Control Planes based on Service Mesh 1.1 for all platforms.
1.2.2.4.7. Istio 1.12 Support
Service Mesh 2.2 is based on Istio 1.12, which brings in new features and product enhancements. While many Istio 1.12 features are supported, the following unsupported features should be noted:
- AuthPolicy Dry Run is a tech preview feature.
- gRPC Proxyless Service Mesh is a tech preview feature.
- Telemetry API is a tech preview feature.
- Discovery selectors is not a supported feature.
- External control plane is not a supported feature.
- Gateway injection is not a supported feature.
1.2.2.4.8. Kubernetes Gateway API
Kubernetes Gateway API is a technology preview feature that is disabled by default.
To enable the feature, set the following environment variables for the Istiod
container in ServiceMeshControlPlane
:
spec: runtime: components: pilot: container: env: PILOT_ENABLE_GATEWAY_API: true PILOT_ENABLE_GATEWAY_API_STATUS: true # and optionally, for the deployment controller PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: true
Restricting route attachment on Gateway API listeners is possible using the SameNamespace
or All
settings. Istio ignores usage of label selectors in listeners.allowedRoutes.namespaces
and reverts to the default behavior (SameNamespace
).
1.2.2.5. New features Red Hat OpenShift Service Mesh 2.1.5.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.5.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.5.1
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.5 |
Jaeger | 1.36 |
Kiali | 1.36.13 |
1.2.2.6. New features Red Hat OpenShift Service Mesh 2.1.5
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.6.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.5
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.36 |
Kiali | 1.36.12-1 |
1.2.2.7. New features Red Hat OpenShift Service Mesh 2.1.4
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.7.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.4
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.30.2 |
Kiali | 1.36.12-1 |
1.2.2.8. New features Red Hat OpenShift Service Mesh 2.1.3
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.8.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.3
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.30.2 |
Kiali | 1.36.10-2 |
1.2.2.9. New features Red Hat OpenShift Service Mesh 2.1.2.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.9.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.2.1
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.30.2 |
Kiali | 1.36.9 |
1.2.2.10. New features Red Hat OpenShift Service Mesh 2.1.2
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
With this release, the Red Hat OpenShift distributed tracing platform Operator is now installed to the openshift-distributed-tracing
namespace by default. Previously the default installation had been in the openshift-operator
namespace.
1.2.2.10.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.2
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.30.1 |
Kiali | 1.36.8 |
1.2.2.11. New features Red Hat OpenShift Service Mesh 2.1.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release also adds the ability to disable the automatic creation of network policies.
1.2.2.11.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1.1
Component | Version |
---|---|
Istio | 1.9.9 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.24.1 |
Kiali | 1.36.7 |
1.2.2.11.2. Disabling network policies
Red Hat OpenShift Service Mesh automatically creates and manages a number of NetworkPolicies
resources in the Service Mesh control plane and application namespaces. This is to ensure that applications and the control plane can communicate with each other.
If you want to disable the automatic creation and management of NetworkPolicies
resources, for example to enforce company security policies, you can do so. You can edit the ServiceMeshControlPlane
to set the spec.security.manageNetworkPolicy
setting to false
When you disable spec.security.manageNetworkPolicy
Red Hat OpenShift Service Mesh will not create any NetworkPolicy
objects. The system administrator is responsible for managing the network and fixing any issues this might cause.
Procedure
- In the OpenShift Container Platform web console, click Operators → Installed Operators.
-
Select the project where you installed the Service Mesh control plane, for example
istio-system
, from the Project menu. -
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your
ServiceMeshControlPlane
, for examplebasic-install
. -
On the Create ServiceMeshControlPlane Details page, click
YAML
to modify your configuration. Set the
ServiceMeshControlPlane
fieldspec.security.manageNetworkPolicy
tofalse
, as shown in this example.apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: security: trust: manageNetworkPolicy: false
- Click Save.
1.2.2.12. New features and enhancements Red Hat OpenShift Service Mesh 2.1
This release of Red Hat OpenShift Service Mesh adds support for Istio 1.9.8, Envoy Proxy 1.17.1, Jaeger 1.24.1, and Kiali 1.36.5 on OpenShift Container Platform 4.6 EUS, 4.7, 4.8, 4.9, along with new features and enhancements.
1.2.2.12.1. Component versions included in Red Hat OpenShift Service Mesh version 2.1
Component | Version |
---|---|
Istio | 1.9.6 |
Envoy Proxy | 1.17.1 |
Jaeger | 1.24.1 |
Kiali | 1.36.5 |
1.2.2.12.2. Service Mesh Federation
New Custom Resource Definitions (CRDs) have been added to support federating service meshes. Service meshes may be federated both within the same cluster or across different OpenShift clusters. These new resources include:
-
ServiceMeshPeer
- Defines a federation with a separate service mesh, including gateway configuration, root trust certificate configuration, and status fields. In a pair of federated meshes, each mesh will define its own separateServiceMeshPeer
resource. -
ExportedServiceMeshSet
- Defines which services for a givenServiceMeshPeer
are available for the peer mesh to import. -
ImportedServiceSet
- Defines which services for a givenServiceMeshPeer
are imported from the peer mesh. These services must also be made available by the peer’sExportedServiceMeshSet
resource.
Service Mesh Federation is not supported between clusters on Red Hat OpenShift Service on AWS (ROSA), Azure Red Hat OpenShift (ARO), or OpenShift Dedicated (OSD).
1.2.2.12.3. OVN-Kubernetes Container Network Interface (CNI) generally available
The OVN-Kubernetes Container Network Interface (CNI) was previously introduced as a Technology Preview feature in Red Hat OpenShift Service Mesh 2.0.1 and is now generally available in Red Hat OpenShift Service Mesh 2.1 and 2.0.x for use on OpenShift Container Platform 4.7.32, OpenShift Container Platform 4.8.12, and OpenShift Container Platform 4.9.
1.2.2.12.4. Service Mesh WebAssembly (WASM) Extensions
The ServiceMeshExtensions
Custom Resource Definition (CRD), first introduced in 2.0 as Technology Preview, is now generally available. You can use CRD to build your own plugins, but Red Hat does not provide support for the plugins you create.
Mixer has been completely removed in Service Mesh 2.1. Upgrading from a Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled. Mixer plugins will need to be ported to WebAssembly Extensions.
1.2.2.12.5. 3scale WebAssembly Adapter (WASM)
With Mixer now officially removed, OpenShift Service Mesh 2.1 does not support the 3scale mixer adapter. Before upgrading to Service Mesh 2.1, remove the Mixer-based 3scale adapter and any additional Mixer plugins. Then, manually install and configure the new 3scale WebAssembly adapter with Service Mesh 2.1+ using a ServiceMeshExtension
resource.
3scale 2.11 introduces an updated Service Mesh integration based on WebAssembly
.
1.2.2.12.6. Istio 1.9 Support
Service Mesh 2.1 is based on Istio 1.9, which brings in a large number of new features and product enhancements. While the majority of Istio 1.9 features are supported, the following exceptions should be noted:
- Virtual Machine integration is not yet supported
- Kubernetes Gateway API is not yet supported
- Remote fetch and load of WebAssembly HTTP filters are not yet supported
- Custom CA Integration using the Kubernetes CSR API is not yet supported
- Request Classification for monitoring traffic is a tech preview feature
- Integration with external authorization systems via Authorization policy’s CUSTOM action is a tech preview feature
1.2.2.12.7. Improved Service Mesh operator performance
The amount of time Red Hat OpenShift Service Mesh uses to prune old resources at the end of every ServiceMeshControlPlane
reconciliation has been reduced. This results in faster ServiceMeshControlPlane
deployments, and allows changes applied to existing SMCPs to take effect more quickly.
1.2.2.12.8. Kiali updates
Kiali 1.36 includes the following features and enhancements:
Service Mesh troubleshooting functionality
- Control plane and gateway monitoring
- Proxy sync statuses
- Envoy configuration views
- Unified view showing Envoy proxy and application logs interleaved
- Namespace and cluster boxing to support federated service mesh views
- New validations, wizards, and distributed tracing enhancements
1.2.2.13. New features Red Hat OpenShift Service Mesh 2.0.11.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.13.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.11.1
Component | Version |
---|---|
Istio | 1.6.14 |
Envoy Proxy | 1.14.5 |
Jaeger | 1.36 |
Kiali | 1.24.17 |
1.2.2.14. New features Red Hat OpenShift Service Mesh 2.0.11
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
1.2.2.14.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.11
Component | Version |
---|---|
Istio | 1.6.14 |
Envoy Proxy | 1.14.5 |
Jaeger | 1.36 |
Kiali | 1.24.16-1 |
1.2.2.15. New features Red Hat OpenShift Service Mesh 2.0.10
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.15.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.10
Component | Version |
---|---|
Istio | 1.6.14 |
Envoy Proxy | 1.14.5 |
Jaeger | 1.28.0 |
Kiali | 1.24.16-1 |
1.2.2.16. New features Red Hat OpenShift Service Mesh 2.0.9
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.16.1. Component versions included in Red Hat OpenShift Service Mesh version 2.0.9
Component | Version |
---|---|
Istio | 1.6.14 |
Envoy Proxy | 1.14.5 |
Jaeger | 1.24.1 |
Kiali | 1.24.11 |
1.2.2.17. New features Red Hat OpenShift Service Mesh 2.0.8
This release of Red Hat OpenShift Service Mesh addresses bug fixes.
1.2.2.18. New features Red Hat OpenShift Service Mesh 2.0.7.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs).
1.2.2.18.1. Change in how Red Hat OpenShift Service Mesh handles URI fragments
Red Hat OpenShift Service Mesh contains a remotely exploitable vulnerability, CVE-2021-39156, where an HTTP request with a fragment (a section in the end of a URI that begins with a # character) in the URI path could bypass the Istio URI path-based authorization policies. For instance, an Istio authorization policy denies requests sent to the URI path /user/profile
. In the vulnerable versions, a request with URI path /user/profile#section1
bypasses the deny policy and routes to the backend (with the normalized URI path /user/profile%23section1
), possibly leading to a security incident.
You are impacted by this vulnerability if you use authorization policies with DENY actions and operation.paths
, or ALLOW actions and operation.notPaths
.
With the mitigation, the fragment part of the request’s URI is removed before the authorization and routing. This prevents a request with a fragment in its URI from bypassing authorization policies which are based on the URI without the fragment part.
To opt-out from the new behavior in the mitigation, the fragment section in the URI will be kept. You can configure your ServiceMeshControlPlane
to keep URI fragments.
Disabling the new behavior will normalize your paths as described above and is considered unsafe. Ensure that you have accommodated for this in any security policies before opting to keep URI fragments.
Example ServiceMeshControlPlane
modification
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic spec: techPreview: meshConfig: defaultConfig: proxyMetadata: HTTP_STRIP_FRAGMENT_FROM_PATH_UNSAFE_IF_DISABLED: "false"
1.2.2.18.2. Required update for authorization policies
Istio generates hostnames for both the hostname itself and all matching ports. For instance, a virtual service or Gateway for a host of "httpbin.foo" generates a config matching "httpbin.foo and httpbin.foo:*". However, exact match authorization policies only match the exact string given for the hosts
or notHosts
fields.
Your cluster is impacted if you have AuthorizationPolicy
resources using exact string comparison for the rule to determine hosts or notHosts.
You must update your authorization policy rules to use prefix match instead of exact match. For example, replacing hosts: ["httpbin.com"]
with hosts: ["httpbin.com:*"]
in the first AuthorizationPolicy
example.
First example AuthorizationPolicy using prefix match
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: httpbin namespace: foo spec: action: DENY rules: - from: - source: namespaces: ["dev"] to: - operation: hosts: [“httpbin.com”,"httpbin.com:*"]
Second example AuthorizationPolicy using prefix match
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: httpbin namespace: default spec: action: DENY rules: - to: - operation: hosts: ["httpbin.example.com:*"]
1.2.2.19. New features Red Hat OpenShift Service Mesh 2.0.7
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.20. Red Hat OpenShift Service Mesh on Red Hat OpenShift Dedicated and Microsoft Azure Red Hat OpenShift
Red Hat OpenShift Service Mesh is now supported through Red Hat OpenShift Dedicated and Microsoft Azure Red Hat OpenShift.
1.2.2.21. New features Red Hat OpenShift Service Mesh 2.0.6
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.22. New features Red Hat OpenShift Service Mesh 2.0.5
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.23. New features Red Hat OpenShift Service Mesh 2.0.4
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
There are manual steps that must be completed to address CVE-2021-29492 and CVE-2021-31920.
1.2.2.23.1. Manual updates required by CVE-2021-29492 and CVE-2021-31920
Istio contains a remotely exploitable vulnerability where an HTTP request path with multiple slashes or escaped slash characters (%2F
or %5C
) could potentially bypass an Istio authorization policy when path-based authorization rules are used.
For example, assume an Istio cluster administrator defines an authorization DENY policy to reject the request at path /admin
. A request sent to the URL path //admin
will NOT be rejected by the authorization policy.
According to RFC 3986, the path //admin
with multiple slashes should technically be treated as a different path from the /admin
. However, some backend services choose to normalize the URL paths by merging multiple slashes into a single slash. This can result in a bypass of the authorization policy (//admin
does not match /admin
), and a user can access the resource at path /admin
in the backend; this would represent a security incident.
Your cluster is impacted by this vulnerability if you have authorization policies using ALLOW action + notPaths
field or DENY action + paths field
patterns. These patterns are vulnerable to unexpected policy bypasses.
Your cluster is NOT impacted by this vulnerability if:
- You don’t have authorization policies.
-
Your authorization policies don’t define
paths
ornotPaths
fields. -
Your authorization policies use
ALLOW action + paths
field orDENY action + notPaths
field patterns. These patterns could only cause unexpected rejection instead of policy bypasses. The upgrade is optional for these cases.
The Red Hat OpenShift Service Mesh configuration location for path normalization is different from the Istio configuration.
1.2.2.23.2. Updating the path normalization configuration
Istio authorization policies can be based on the URL paths in the HTTP request. Path normalization, also known as URI normalization, modifies and standardizes the incoming requests' paths so that the normalized paths can be processed in a standard way. Syntactically different paths may be equivalent after path normalization.
Istio supports the following normalization schemes on the request paths before evaluating against the authorization policies and routing the requests:
Option | Description | Example | Notes |
---|---|---|---|
| No normalization is done. Anything received by Envoy will be forwarded exactly as-is to any backend service. |
| This setting is vulnerable to CVE-2021-31920. |
|
This is currently the option used in the default installation of Istio. This applies the |
| This setting is vulnerable to CVE-2021-31920. |
| Slashes are merged after the BASE normalization. |
| Update to this setting to mitigate CVE-2021-31920. |
|
The strictest setting when you allow all traffic by default. This setting is recommended, with the caveat that you must thoroughly test your authorization policies routes. Percent-encoded slash and backslash characters ( |
| Update to this setting to mitigate CVE-2021-31920. This setting is more secure, but also has the potential to break applications. Test your applications before deploying to production. |
The normalization algorithms are conducted in the following order:
-
Percent-decode
%2F
,%2f
,%5C
and%5c
. -
The RFC 3986 and other normalization implemented by the
normalize_path
option in Envoy. - Merge slashes.
While these normalization options represent recommendations from HTTP standards and common industry practices, applications may interpret a URL in any way it chooses to. When using denial policies, ensure that you understand how your application behaves.
1.2.2.23.3. Path normalization configuration examples
Ensuring Envoy normalizes request paths to match your backend services' expectations is critical to the security of your system. The following examples can be used as a reference for you to configure your system. The normalized URL paths, or the original URL paths if NONE
is selected, will be:
- Used to check against the authorization policies.
- Forwarded to the backend application.
If your application… | Choose… |
---|---|
Relies on the proxy to do normalization |
|
Normalizes request paths based on RFC 3986 and does not merge slashes. |
|
Normalizes request paths based on RFC 3986 and merges slashes, but does not decode percent-encoded slashes. |
|
Normalizes request paths based on RFC 3986, decodes percent-encoded slashes, and merges slashes. |
|
Processes request paths in a way that is incompatible with RFC 3986. |
|
1.2.2.23.4. Configuring your SMCP for path normalization
To configure path normalization for Red Hat OpenShift Service Mesh, specify the following in your ServiceMeshControlPlane
. Use the configuration examples to help determine the settings for your system.
SMCP v2 pathNormalization
spec: techPreview: global: pathNormalization: <option>
1.2.2.23.5. Configuring for case normalization
In some environments, it may be useful to have paths in authorization policies compared in a case insensitive manner. For example, treating https://myurl/get
and https://myurl/GeT
as equivalent. In those cases, you can use the EnvoyFilter
shown below. This filter will change both the path used for comparison and the path presented to the application. In this example, istio-system
is the name of the Service Mesh control plane project.
Save the EnvoyFilter
to a file and run the following command:
$ oc create -f <myEnvoyFilterFile>
apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: ingress-case-insensitive namespace: istio-system spec: configPatches: - applyTo: HTTP_FILTER match: context: GATEWAY listener: filterChain: filter: name: "envoy.filters.network.http_connection_manager" subFilter: name: "envoy.filters.http.router" patch: operation: INSERT_BEFORE value: name: envoy.lua typed_config: "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua" inlineCode: | function envoy_on_request(request_handle) local path = request_handle:headers():get(":path") request_handle:headers():replace(":path", string.lower(path)) end
1.2.2.24. New features Red Hat OpenShift Service Mesh 2.0.3
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
In addition, this release has the following new features:
-
Added an option to the
must-gather
data collection tool that gathers information from a specified Service Mesh control plane namespace. For more information, see OSSM-351. - Improved performance for Service Mesh control planes with hundreds of namespaces
1.2.2.25. New features Red Hat OpenShift Service Mesh 2.0.2
This release of Red Hat OpenShift Service Mesh adds support for IBM Z and IBM Power Systems. It also addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.26. New features Red Hat OpenShift Service Mesh 2.0.1
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
1.2.2.27. New features Red Hat OpenShift Service Mesh 2.0
This release of Red Hat OpenShift Service Mesh adds support for Istio 1.6.5, Jaeger 1.20.0, Kiali 1.24.2, and the 3scale Istio Adapter 2.0 and OpenShift Container Platform 4.6.
In addition, this release has the following new features:
- Simplifies installation, upgrades, and management of the Service Mesh control plane.
- Reduces the Service Mesh control plane’s resource usage and startup time.
Improves performance by reducing inter-control plane communication over networking.
- Adds support for Envoy’s Secret Discovery Service (SDS). SDS is a more secure and efficient mechanism for delivering secrets to Envoy side car proxies.
- Removes the need to use Kubernetes Secrets, which have well known security risks.
Improves performance during certificate rotation, as proxies no longer require a restart to recognize new certificates.
- Adds support for Istio’s Telemetry v2 architecture, which is built using WebAssembly extensions. This new architecture brings significant performance improvements.
- Updates the ServiceMeshControlPlane resource to v2 with a streamlined configuration to make it easier to manage the Service Mesh Control Plane.
- Introduces WebAssembly extensions as a Technology Preview feature.
1.2.3. Technology Preview
Some features in this release are currently in Technology Preview. These experimental features are not intended for production use.
Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see the Technology Preview Support Scope.
1.2.3.1. Istio compatibility and support matrix
In the table, features are marked with the following statuses:
- TP: Technology Preview
- GA: General Availability
Note the following scope of support on the Red Hat Customer Portal for these features:
Feature | Istio Version | Support Status | Description |
---|---|---|---|
holdApplicationUntilProxyStarts | 1.7 | TP | Blocks application container startup until proxy is running |
DNS capture | 1.8 | GA | Enabled by default |
1.2.4. Deprecated and removed features
Some features available in previous releases have been deprecated or removed.
Deprecated functionality is still included in OpenShift Container Platform and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
Removed functionality no longer exists in the product.
1.2.4.1. Deprecated features Red Hat OpenShift Service Mesh 2.2
The ServiceMeshExtension
API is deprecated as of release 2.2 and will be removed in a future release. While ServiceMeshExtension
API is still supported in release 2.2, customers should start moving to the new WasmPlugin
API.
1.2.4.2. Removed features Red Hat OpenShift Service Mesh 2.2
This release marks the end of support for Service Mesh control planes based on Service Mesh 1.1 for all platforms.
1.2.4.3. Removed features Red Hat OpenShift Service Mesh 2.1
In Service Mesh 2.1, the Mixer component is removed. Bug fixes and support is provided through the end of the Service Mesh 2.0 life cycle.
Upgrading from a Service Mesh 2.0.x release to 2.1 will not proceed if Mixer plugins are enabled. Mixer plugins must be ported to WebAssembly Extensions.
1.2.4.4. Deprecated features Red Hat OpenShift Service Mesh 2.0
The Mixer component was deprecated in release 2.0 and will be removed in release 2.1. While using Mixer for implementing extensions was still supported in release 2.0, extensions should have been migrated to the new WebAssembly mechanism.
The following resource types are no longer supported in Red Hat OpenShift Service Mesh 2.0:
Policy
(authentication.istio.io/v1alpha1) is no longer supported. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.-
Use
RequestAuthentication
(security.istio.io/v1beta1) -
Use
PeerAuthentication
(security.istio.io/v1beta1)
-
Use
ServiceMeshPolicy
(maistra.io/v1) is no longer supported.-
Use
RequestAuthentication
orPeerAuthentication
, as mentioned above, but place in the Service Mesh control plane namespace.
-
Use
RbacConfig
(rbac.istio.io/v1alpha1) is no longer supported.-
Replaced by
AuthorizationPolicy
(security.istio.io/v1beta1), which encompasses behavior ofRbacConfig
,ServiceRole
, andServiceRoleBinding
.
-
Replaced by
ServiceMeshRbacConfig
(maistra.io/v1) is no longer supported.-
Use
AuthorizationPolicy
as above, but place in Service Mesh control plane namespace.
-
Use
-
ServiceRole
(rbac.istio.io/v1alpha1) is no longer supported. -
ServiceRoleBinding
(rbac.istio.io/v1alpha1) is no longer supported. -
In Kiali, the
login
andLDAP
strategies are deprecated. A future version will introduce authentication using OpenID providers.
1.2.5. Known issues
These limitations exist in Red Hat OpenShift Service Mesh:
- Red Hat OpenShift Service Mesh does not yet support IPv6, as it is not yet fully supported by the upstream Istio project. As a result, Red Hat OpenShift Service Mesh does not support dual-stack clusters.
- Graph layout - The layout for the Kiali graph can render differently, depending on your application architecture and the data to display (number of graph nodes and their interactions). Because it is difficult if not impossible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts. To choose a different layout, you can choose a different Layout Schema from the Graph Settings menu.
- The first time you access related services such as distributed tracing platform and Grafana, from the Kiali console, you must accept the certificate and re-authenticate using your OpenShift Container Platform login credentials. This happens due to an issue with how the framework displays embedded pages in the console.
- The Bookinfo sample application cannot be installed on IBM Z and IBM Power.
- WebAssembly extensions are not supported on IBM Z and IBM Power.
- LuaJIT is not supported on IBM Power.
1.2.5.1. Service Mesh known issues
These are the known issues in Red Hat OpenShift Service Mesh:
- Istio-14743 Due to limitations in the version of Istio that this release of Red Hat OpenShift Service Mesh is based on, there may be applications that are currently incompatible with Service Mesh. See the linked community issue for details.
OSSM-1655 Kiali dashboard shows error after enabling mTLS in
SMCP
.After enabling the
spec.security.controlPlane.mtls
setting in the SMCP, the Kiali console displays the following error messageNo subsets defined
.OSSM-1505 This issue only occurs when using the
ServiceMeshExtension
resource on OpenShift Container Platform 4.11. When you useServiceMeshExtension
on OpenShift Container Platform 4.11 the resource never becomes ready. If you inspect the issue usingoc describe ServiceMeshExtension
you will see the following error:stderr: Error creating mount namespace before pivot: function not implemented
.Workaround:
ServiceMeshExtension
was deprecated in Service Mesh 2.2. Migrate fromServiceMeshExtension
to theWasmPlugin
resource. For more information, see Migrating fromServiceMeshExtension
toWasmPlugin
resources.-
OSSM-1396 If a gateway resource contains the
spec.externalIPs
setting, instead of being recreated when theServiceMeshControlPlane
is updated, the gateway is removed and never recreated. - OSSM-1168 When service mesh resources are created as a single YAML file, the Envoy proxy sidecar is not reliably injected into pods. When the SMCP, SMMR, and Deployment resources are created individually, the deployment works as expected.
OSSM-1052 When configuring a Service
ExternalIP
for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).
OSSM-882 This applies for Service Mesh 2.1 and earlier. Namespace is in the accessible_namespace list but does not appear in Kiali UI. By default, Kiali will not show any namespaces that start with "kube" because these namespaces are typically internal-use only and not part of a mesh.
For example, if you create a namespace called 'akube-a' and add it to the Service Mesh member roll, then the Kiali UI does not display the namespace. For defined exclusion patterns, the software excludes namespaces that start with or contain the pattern.
Workaround: Change the Kiali Custom Resource setting so it prefixes the setting with a carat (^). For example:
api: namespaces: exclude: - "^istio-operator" - "^kube-.*" - "^openshift.*" - "^ibm.*" - "^kiali-operator"
-
MAISTRA-2692 With Mixer removed, custom metrics that have been defined in Service Mesh 2.0.x cannot be used in 2.1. Custom metrics can be configured using
EnvoyFilter
. Red Hat is unable to supportEnvoyFilter
configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained. -
MAISTRA-2648
ServiceMeshExtensions
are currently not compatible with meshes deployed on IBM Z Systems. MAISTRA-1959 Migration to 2.0 Prometheus scraping (
spec.addons.prometheus.scrape
set totrue
) does not work when mTLS is enabled. Additionally, Kiali displays extraneous graph data when mTLS is disabled.This problem can be addressed by excluding port 15020 from proxy configuration, for example,
spec: proxy: networking: trafficControl: inbound: excludedPorts: - 15020
- MAISTRA-1314 Red Hat OpenShift Service Mesh does not yet support IPv6.
-
MAISTRA-453 If you create a new project and deploy pods immediately, sidecar injection does not occur. The operator fails to add the
maistra.io/member-of
before the pods are created, therefore the pods must be deleted and recreated for sidecar injection to occur. - MAISTRA-158 Applying multiple gateways referencing the same hostname will cause all gateways to stop functioning.
1.2.5.2. Kiali known issues
New issues for Kiali should be created in the OpenShift Service Mesh project with the Component
set to Kiali
.
These are the known issues in Kiali:
- KIALI-2206 When you are accessing the Kiali console for the first time, and there is no cached browser data for Kiali, the “View in Grafana” link on the Metrics tab of the Kiali Service Details page redirects to the wrong location. The only way you would encounter this issue is if you are accessing Kiali for the first time.
- KIALI-507 Kiali does not support Internet Explorer 11. This is because the underlying frameworks do not support Internet Explorer. To access the Kiali console, use one of the two most recent versions of the Chrome, Edge, Firefox or Safari browser.
1.2.5.3. Red Hat OpenShift distributed tracing known issues
These limitations exist in Red Hat OpenShift distributed tracing:
- Apache Spark is not supported.
- The streaming deployment via AMQ/Kafka is unsupported on IBM Z and IBM Power Systems.
These are the known issues for Red Hat OpenShift distributed tracing:
TRACING-2057 The Kafka API has been updated to
v1beta2
to support the Strimzi Kafka Operator 0.23.0. However, this API version is not supported by AMQ Streams 1.6.3. If you have the following environment, your Jaeger services will not be upgraded, and you cannot create new Jaeger services or modify existing Jaeger services:- Jaeger Operator channel: 1.17.x stable or 1.20.x stable
AMQ Streams Operator channel: amq-streams-1.6.x
To resolve this issue, switch the subscription channel for your AMQ Streams Operator to either amq-streams-1.7.x or stable.
1.2.6. Fixed issues
The following issues been resolved in the current release:
1.2.6.1. Service Mesh fixed issues
OSSM-2053 Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, during SMCP reconciliation, the SMMR controller removed the member namespaces from
SMMR.status.configuredMembers
. This caused the services in the member namespaces to become unavailable for a few moments.Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, the SMMR controller no longer removes the namespaces from
SMMR.status.configuredMembers
. Instead, the controller adds the namespaces toSMMR.status.pendingMembers
to indicate that they are not up-to-date. During reconciliation, as each namespace synchronizes with the SMCP, the namespace is automatically removed fromSMMR.status.pendingMembers
.-
OSSM-1668 A new field
spec.security.jwksResolverCA
was added to the Version 2.1SMCP
but was missing in the 2.2.0 and 2.2.1 releases. When upgrading from an Operator version where this field was present to an Operator version that was missing this field, the.spec.security.jwksResolverCA
field was not available in theSMCP
. -
OSSM-1325 istiod pod crashes and displays the following error message:
fatal error: concurrent map iteration and map write
. OSSM-1211 Configuring Federated service meshes for failover does not work as expected.
The Istiod pilot log displays the following error:
envoy connection [C289] TLS error: 337047686:SSL routines:tls_process_server_certificate:certificate verify failed
-
OSSM-1099 The Kiali console displayed the message
Sorry, there was a problem. Try a refresh or navigate to a different page.
- OSSM-1074 Pod annotations defined in SMCP are not injected in the pods.
- OSSM-999 Kiali retention did not work as expected. Calendar times were greyed out in the dashboard graph.
-
OSSM-797 Kiali Operator pod generates
CreateContainerConfigError
while installing or updating the operator. -
OSSM-722 Namespace starting with
kube
is hidden from Kiali. -
OSSM-569 There is no CPU memory limit for the Prometheus
istio-proxy
container. The Prometheusistio-proxy
sidecar now uses the resource limits defined inspec.proxy.runtime.container
. - OSSM-449 VirtualService and Service causes an error "Only unique values for domains are permitted. Duplicate entry of domain."
- OSSM-419 Namespaces with similar names will all show in Kiali namespace list, even though namespaces may not be defined in Service Mesh Member Role.
- OSSM-296 When adding health configuration to the Kiali custom resource (CR) is it not being replicated to the Kiali configmap.
- OSSM-291 In the Kiali console, on the Applications, Services, and Workloads pages, the "Remove Label from Filters" function is not working.
- OSSM-289 In the Kiali console, on the Service Details pages for the 'istio-ingressgateway' and 'jaeger-query' services there are no Traces being displayed. The traces exist in Jaeger.
- OSSM-287 In the Kiali console there are no traces being displayed on the Graph Service.
OSSM-285 When trying to access the Kiali console, receive the following error message "Error trying to get OAuth Metadata".
Workaround: Restart the Kiali pod.
MAISTRA-2735 The resources that the Service Mesh Operator deletes when reconciling the SMCP changed in Red Hat OpenShift Service Mesh version 2.1. Previously, the Operator deleted a resource with the following labels:
-
maistra.io/owner
-
app.kubernetes.io/version
Now, the Operator ignores resources that does not also include the
app.kubernetes.io/managed-by=maistra-istio-operator
label. If you create your own resources, you should not add theapp.kubernetes.io/managed-by=maistra-istio-operator
label to them.-
-
MAISTRA-2687 Red Hat OpenShift Service Mesh 2.1 federation gateway does not send the full certificate chain when using external certificates. The Service Mesh federation egress gateway only sends the client certificate. Because the federation ingress gateway only knows about the root certificate, it cannot verify the client certificate unless you add the root certificate to the federation import
ConfigMap
. -
MAISTRA-2635 Replace deprecated Kubernetes API. To remain compatible with OpenShift Container Platform 4.8, the
apiextensions.k8s.io/v1beta1
API was deprecated as of Red Hat OpenShift Service Mesh 2.0.8. -
MAISTRA-2631 The WASM feature is not working because podman is failing due to nsenter binary not being present. Red Hat OpenShift Service Mesh generates the following error message:
Error: error configuring CNI network plugin exec: "nsenter": executable file not found in $PATH
. The container image now contains nsenter and WASM works as expected. - MAISTRA-2534 When istiod attempted to fetch the JWKS for an issuer specified in a JWT rule, the issuer service responded with a 502. This prevented the proxy container from becoming ready and caused deployments to hang. The fix for the community bug has been included in the Service Mesh 2.0.7 release.
MAISTRA-2411 When the Operator creates a new ingress gateway using
spec.gateways.additionaIngress
in theServiceMeshControlPlane
, Operator is not creating aNetworkPolicy
for the additional ingress gateway like it does for the default istio-ingressgateway. This is causing a 503 response from the route of the new gateway.Workaround: Manually create the
NetworkPolicy
in the <istio-system> namespace.MAISTRA-2401 CVE-2021-3586 servicemesh-operator: NetworkPolicy resources incorrectly specified ports for ingress resources. The NetworkPolicy resources installed for Red Hat OpenShift Service Mesh did not properly specify which ports could be accessed. This allowed access to all ports on these resources from any pod. Network policies applied to the following resources are affected:
- Galley
- Grafana
- Istiod
- Jaeger
- Kiali
- Prometheus
- Sidecar injector
-
MAISTRA-2378 When the cluster is configured to use OpenShift SDN with
ovs-multitenant
and the mesh contains a large number of namespaces (200+), the OpenShift Container Platform networking plugin is unable to configure the namespaces quickly. Service Mesh times out causing namespaces to be continuously dropped from the service mesh and then reenlisted. - MAISTRA-2370 Handle tombstones in listerInformer. The updated cache codebase was not handling tombstones when translating the events from the namespace caches to the aggregated cache, leading to a panic in the go routine.
MAISTRA-2117 Add optional
ConfigMap
mount to operator. The CSV now contains an optionalConfigMap
volume mount, which mounts thesmcp-templates
ConfigMap
if it exists. If thesmcp-templates
ConfigMap
does not exist, the mounted directory is empty. When you create theConfigMap
, the directory is populated with the entries from theConfigMap
and can be referenced inSMCP.spec.profiles
. No restart of the Service Mesh operator is required.Customers using the 2.0 operator with a modified CSV to mount the smcp-templates ConfigMap can upgrade to Red Hat OpenShift Service Mesh 2.1. After upgrading, you can continue using an existing ConfigMap, and the profiles it contains, without editing the CSV. Customers that previously used ConfigMap with a different name will either have to rename the ConfigMap or update the CSV after upgrading.
-
MAISTRA-2010 AuthorizationPolicy does not support
request.regex.headers
field. Thevalidatingwebhook
rejects any AuthorizationPolicy with the field, and even if you disable that, Pilot tries to validate it using the same code, and it does not work. MAISTRA-1979 Migration to 2.0 The conversion webhook drops the following important fields when converting
SMCP.status
from v2 to v1:- conditions
- components
- observedGeneration
annotations
Upgrading the operator to 2.0 might break client tools that read the SMCP status using the maistra.io/v1 version of the resource.
This also causes the READY and STATUS columns to be empty when you run
oc get servicemeshcontrolplanes.v1.maistra.io
.
MAISTRA-1947 Technology Preview Updates to ServiceMeshExtensions are not applied.
Workaround: Remove and recreate the
ServiceMeshExtensions
.-
MAISTRA-1983 Migration to 2.0 Upgrading to 2.0.0 with an existing invalid
ServiceMeshControlPlane
cannot easily be repaired. The invalid items in theServiceMeshControlPlane
resource caused an unrecoverable error. The fix makes the errors recoverable. You can delete the invalid resource and replace it with a new one or edit the resource to fix the errors. For more information about editing your resource, see [Configuring the Red Hat OpenShift Service Mesh installation]. - MAISTRA-1502 As a result of CVEs fixes in version 1.0.10, the Istio dashboards are not available from the Home Dashboard menu in Grafana. To access the Istio dashboards, click the Dashboard menu in the navigation panel and select the Manage tab.
- MAISTRA-1399 Red Hat OpenShift Service Mesh no longer prevents you from installing unsupported CNI protocols. The supported network configurations has not changed.
- MAISTRA-1089 Migration to 2.0 Gateways created in a non-control plane namespace are automatically deleted. After removing the gateway definition from the SMCP spec, you need to manually delete these resources.
MAISTRA-858 The following Envoy log messages describing deprecated options and configurations associated with Istio 1.1.x are expected:
- [2019-06-03 07:03:28.943][19][warning][misc] [external/envoy/source/common/protobuf/utility.cc:129] Using deprecated option 'envoy.api.v2.listener.Filter.config'. This configuration will be removed from Envoy soon.
- [2019-08-12 22:12:59.001][13][warning][misc] [external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file lds.proto. This configuration will be removed from Envoy soon.
MAISTRA-806 Evicted Istio Operator Pod causes mesh and CNI not to deploy.
Workaround: If the
istio-operator
pod is evicted while deploying the control pane, delete the evictedistio-operator
pod.- MAISTRA-681 When the Service Mesh control plane has many namespaces, it can lead to performance issues.
- MAISTRA-193 Unexpected console info messages are visible when health checking is enabled for citadel.
- Bugzilla 1821432 The toggle controls in OpenShift Container Platform Custom Resource details page does not update the CR correctly. UI Toggle controls in the Service Mesh Control Plane (SMCP) Overview page in the OpenShift Container Platform web console sometimes updates the wrong field in the resource. To update a SMCP, edit the YAML content directly or update the resource from the command line instead of clicking the toggle controls.
1.2.6.2. Red Hat OpenShift distributed tracing fixed issues
TRACING-2337 Jaeger is logging a repetitive warning message in the Jaeger logs similar to the following:
{"level":"warn","ts":1642438880.918793,"caller":"channelz/logging.go:62","msg":"[core]grpc: Server.Serve failed to create ServerTransport: connection error: desc = \"transport: http2Server.HandleStreams received bogus greeting from client: \\\"\\\\x16\\\\x03\\\\x01\\\\x02\\\\x00\\\\x01\\\\x00\\\\x01\\\\xfc\\\\x03\\\\x03vw\\\\x1a\\\\xc9T\\\\xe7\\\\xdaCj\\\\xb7\\\\x8dK\\\\xa6\\\"\"","system":"grpc","grpc_log":true}
This issue was resolved by exposing only the HTTP(S) port of the query service, and not the gRPC port.
- TRACING-2009 The Jaeger Operator has been updated to include support for the Strimzi Kafka Operator 0.23.0.
-
TRACING-1907 The Jaeger agent sidecar injection was failing due to missing config maps in the application namespace. The config maps were getting automatically deleted due to an incorrect
OwnerReference
field setting and as a result, the application pods were not moving past the "ContainerCreating" stage. The incorrect settings have been removed. - TRACING-1725 Follow-up to TRACING-1631. Additional fix to ensure that Elasticsearch certificates are properly reconciled when there are multiple Jaeger production instances, using same name but within different namespaces. See also BZ-1918920.
- TRACING-1631 Multiple Jaeger production instances, using same name but within different namespaces, causing Elasticsearch certificate issue. When multiple service meshes were installed, all of the Jaeger Elasticsearch instances had the same Elasticsearch secret instead of individual secrets, which prevented the OpenShift Elasticsearch Operator from communicating with all of the Elasticsearch clusters.
- TRACING-1300 Failed connection between Agent and Collector when using Istio sidecar. An update of the Jaeger Operator enabled TLS communication by default between a Jaeger sidecar agent and the Jaeger Collector.
-
TRACING-1208 Authentication "500 Internal Error" when accessing Jaeger UI. When trying to authenticate to the UI using OAuth, I get a 500 error because oauth-proxy sidecar doesn’t trust the custom CA bundle defined at installation time with the
additionalTrustBundle
. -
TRACING-1166 It is not currently possible to use the Jaeger streaming strategy within a disconnected environment. When a Kafka cluster is being provisioned, it results in a error:
Failed to pull image registry.redhat.io/amq7/amq-streams-kafka-24-rhel7@sha256:f9ceca004f1b7dccb3b82d9a8027961f9fe4104e0ed69752c0bdd8078b4a1076
. - TRACING-809 Jaeger Ingester is incompatible with Kafka 2.3. When there are two or more instances of the Jaeger Ingester and enough traffic it will continuously generate rebalancing messages in the logs. This is due to a regression in Kafka 2.3 that was fixed in Kafka 2.3.1. For more information, see Jaegertracing-1819.
BZ-1918920/LOG-1619 The Elasticsearch pods does not get restarted automatically after an update.
Workaround: Restart the pods manually.
1.3. Understanding Service Mesh
Red Hat OpenShift Service Mesh provides a platform for behavioral insight and operational control over your networked microservices in a service mesh. With Red Hat OpenShift Service Mesh, you can connect, secure, and monitor microservices in your OpenShift Container Platform environment.
1.3.1. Understanding service mesh
A service mesh is the network of microservices that make up applications in a distributed microservice architecture and the interactions between those microservices. When a Service Mesh grows in size and complexity, it can become harder to understand and manage.
Based on the open source Istio project, Red Hat OpenShift Service Mesh adds a transparent layer on existing distributed applications without requiring any changes to the service code. You add Red Hat OpenShift Service Mesh support to services by deploying a special sidecar proxy to relevant services in the mesh that intercepts all network communication between microservices. You configure and manage the Service Mesh using the Service Mesh control plane features.
Red Hat OpenShift Service Mesh gives you an easy way to create a network of deployed services that provide:
- Discovery
- Load balancing
- Service-to-service authentication
- Failure recovery
- Metrics
- Monitoring
Red Hat OpenShift Service Mesh also provides more complex operational functions including:
- A/B testing
- Canary releases
- Access control
- End-to-end authentication
1.3.2. Service Mesh architecture
Service mesh technology operates at the network communication level. That is, service mesh components capture or intercept traffic to and from microservices, either modifying requests, redirecting them, or creating new requests to other services.
At a high level, Red Hat OpenShift Service Mesh consists of a data plane and a control plane
The data plane is a set of intelligent proxies, running alongside application containers in a pod, that intercept and control all inbound and outbound network communication between microservices in the service mesh. The data plane is implemented in such a way that it intercepts all inbound (ingress) and outbound (egress) network traffic. The Istio data plane is composed of Envoy containers running along side application containers in a pod. The Envoy container acts as a proxy, controlling all network communication into and out of the pod.
Envoy proxies are the only Istio components that interact with data plane traffic. All incoming (ingress) and outgoing (egress) network traffic between services flows through the proxies. The Envoy proxy also collects all metrics related to services traffic within the mesh. Envoy proxies are deployed as sidecars, running in the same pod as services. Envoy proxies are also used to implement mesh gateways.
- Sidecar proxies manage inbound and outbound communication to the workload instance it is attached to.
Gateways are proxies operating as load balancers receiving incoming or outgoing HTTP/TCP connections. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads. You use a Gateway to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh.
- Ingress-gateway - Also known as an ingress controller, the Ingress Gateway is a dedicated Envoy proxy that receives and controls traffic entering the service mesh. An Ingress Gateway allows features such as monitoring and route rules to be applied to traffic entering the cluster.
- Egress-gateway - Also known as an egress controller, the Egress Gateway is a dedicated Envoy proxy that manages traffic leaving the service mesh. An Egress Gateway allows features such as monitoring and route rules to be applied to traffic exiting the mesh.
The control plane manages and configures the proxies that make up the data plane. It is the authoritative source for configuration, manages access control and usage policies, and collects metrics from the proxies in the service mesh.
The Istio control plane is composed of Istiod which consolidates several previous control plane components (Citadel, Galley, Pilot) into a single binary. Istiod provides service discovery, configuration, and certificate management. It converts high-level routing rules to Envoy configurations and propagates them to the sidecars at runtime.
- Istiod can act as a Certificate Authority (CA), generating certificates supporting secure mTLS communication in the data plane. You can also use an external CA for this purpose.
- Istiod is responsible for injecting sidecar proxy containers into workloads deployed to an OpenShift cluster.
Red Hat OpenShift Service Mesh uses the istio-operator to manage the installation of the control plane. An Operator is a piece of software that enables you to implement and automate common activities in your OpenShift cluster. It acts as a controller, allowing you to set or change the desired state of objects in your cluster, in this case, a Red Hat OpenShift Service Mesh installation.
Red Hat OpenShift Service Mesh also bundles the following Istio add-ons as part of the product:
- Kiali - Kiali is the management console for Red Hat OpenShift Service Mesh. It provides dashboards, observability, and robust configuration and validation capabilities. It shows the structure of your service mesh by inferring traffic topology and displays the health of your mesh. Kiali provides detailed metrics, powerful validation, access to Grafana, and strong integration with the distributed tracing platform.
- Prometheus - Red Hat OpenShift Service Mesh uses Prometheus to store telemetry information from services. Kiali depends on Prometheus to obtain metrics, health status, and mesh topology.
- Jaeger - Red Hat OpenShift Service Mesh supports the distributed tracing platform. Jaeger is an open source traceability server that centralizes and displays traces associated with a single request between multiple services. Using the distributed tracing platform you can monitor and troubleshoot your microservices-based distributed systems.
- Elasticsearch - Elasticsearch is an open source, distributed, JSON-based search and analytics engine. The distributed tracing platform uses Elasticsearch for persistent storage.
- Grafana - Grafana provides mesh administrators with advanced query and metrics analysis and dashboards for Istio data. Optionally, Grafana can be used to analyze service mesh metrics.
The following Istio integrations are supported with Red Hat OpenShift Service Mesh:
- 3scale - Istio provides an optional integration with Red Hat 3scale API Management solutions. For versions prior to 2.1, this integration was achieved via the 3scale Istio adapter. For version 2.1 and later, the 3scale integration is achieved via a WebAssembly module.
For information about how to install the 3scale adapter, refer to the 3scale Istio adapter documentation
1.3.3. Understanding Kiali
Kiali provides visibility into your service mesh by showing you the microservices in your service mesh, and how they are connected.
1.3.3.1. Kiali overview
Kiali provides observability into the Service Mesh running on OpenShift Container Platform. Kiali helps you define, validate, and observe your Istio service mesh. It helps you to understand the structure of your service mesh by inferring the topology, and also provides information about the health of your service mesh.
Kiali provides an interactive graph view of your namespace in real time that provides visibility into features like circuit breakers, request rates, latency, and even graphs of traffic flows. Kiali offers insights about components at different levels, from Applications to Services and Workloads, and can display the interactions with contextual information and charts on the selected graph node or edge. Kiali also provides the ability to validate your Istio configurations, such as gateways, destination rules, virtual services, mesh policies, and more. Kiali provides detailed metrics, and a basic Grafana integration is available for advanced queries. Distributed tracing is provided by integrating Jaeger into the Kiali console.
Kiali is installed by default as part of the Red Hat OpenShift Service Mesh.
1.3.3.2. Kiali architecture
Kiali is based on the open source Kiali project. Kiali is composed of two components: the Kiali application and the Kiali console.
- Kiali application (back end) – This component runs in the container application platform and communicates with the service mesh components, retrieves and processes data, and exposes this data to the console. The Kiali application does not need storage. When deploying the application to a cluster, configurations are set in ConfigMaps and secrets.
- Kiali console (front end) – The Kiali console is a web application. The Kiali application serves the Kiali console, which then queries the back end for data in order to present it to the user.
In addition, Kiali depends on external services and components provided by the container application platform and Istio.
- Red Hat Service Mesh (Istio) - Istio is a Kiali requirement. Istio is the component that provides and controls the service mesh. Although Kiali and Istio can be installed separately, Kiali depends on Istio and will not work if it is not present. Kiali needs to retrieve Istio data and configurations, which are exposed through Prometheus and the cluster API.
- Prometheus - A dedicated Prometheus instance is included as part of the Red Hat OpenShift Service Mesh installation. When Istio telemetry is enabled, metrics data are stored in Prometheus. Kiali uses this Prometheus data to determine the mesh topology, display metrics, calculate health, show possible problems, and so on. Kiali communicates directly with Prometheus and assumes the data schema used by Istio Telemetry. Prometheus is an Istio dependency and a hard dependency for Kiali, and many of Kiali’s features will not work without Prometheus.
- Cluster API - Kiali uses the API of the OpenShift Container Platform (cluster API) in order to fetch and resolve service mesh configurations. Kiali queries the cluster API to retrieve, for example, definitions for namespaces, services, deployments, pods, and other entities. Kiali also makes queries to resolve relationships between the different cluster entities. The cluster API is also queried to retrieve Istio configurations like virtual services, destination rules, route rules, gateways, quotas, and so on.
- Jaeger - Jaeger is optional, but is installed by default as part of the Red Hat OpenShift Service Mesh installation. When you install the distributed tracing platform as part of the default Red Hat OpenShift Service Mesh installation, the Kiali console includes a tab to display distributed tracing data. Note that tracing data will not be available if you disable Istio’s distributed tracing feature. Also note that user must have access to the namespace where the Service Mesh control plane is installed to view tracing data.
- Grafana - Grafana is optional, but is installed by default as part of the Red Hat OpenShift Service Mesh installation. When available, the metrics pages of Kiali display links to direct the user to the same metric in Grafana. Note that user must have access to the namespace where the Service Mesh control plane is installed to view links to the Grafana dashboard and view Grafana data.
1.3.3.3. Kiali features
The Kiali console is integrated with Red Hat Service Mesh and provides the following capabilities:
- Health – Quickly identify issues with applications, services, or workloads.
- Topology – Visualize how your applications, services, or workloads communicate via the Kiali graph.
- Metrics – Predefined metrics dashboards let you chart service mesh and application performance for Go, Node.js. Quarkus, Spring Boot, Thorntail and Vert.x. You can also create your own custom dashboards.
- Tracing – Integration with Jaeger lets you follow the path of a request through various microservices that make up an application.
- Validations – Perform advanced validations on the most common Istio objects (Destination Rules, Service Entries, Virtual Services, and so on).
- Configuration – Optional ability to create, update and delete Istio routing configuration using wizards or directly in the YAML editor in the Kiali Console.
1.3.4. Understanding distributed tracing
Every time a user takes an action in an application, a request is executed by the architecture that may require dozens of different services to participate to produce a response. The path of this request is a distributed transaction. The distributed tracing platform lets you perform distributed tracing, which follows the path of a request through various microservices that make up an application.
Distributed tracing is a technique that is used to tie the information about different units of work together—usually executed in different processes or hosts—in order to understand a whole chain of events in a distributed transaction. Distributed tracing lets developers visualize call flows in large service oriented architectures. It can be invaluable in understanding serialization, parallelism, and sources of latency.
The distributed tracing platform records the execution of individual requests across the whole stack of microservices, and presents them as traces. A trace is a data/execution path through the system. An end-to-end trace comprises one or more spans.
A span represents a logical unit of work that has an operation name, the start time of the operation, and the duration. Spans may be nested and ordered to model causal relationships.
1.3.4.1. Distributed tracing overview
As a service owner, you can use distributed tracing to instrument your services to gather insights into your service architecture. You can use distributed tracing for monitoring, network profiling, and troubleshooting the interaction between components in modern, cloud-native, microservices-based applications.
With distributed tracing you can perform the following functions:
- Monitor distributed transactions
- Optimize performance and latency
- Perform root cause analysis
Red Hat OpenShift distributed tracing consists of two main components:
- Red Hat OpenShift distributed tracing platform - This component is based on the open source Jaeger project.
- Red Hat OpenShift distributed tracing data collection - This component is based on the open source OpenTelemetry project.
Both of these components are based on the vendor-neutral OpenTracing APIs and instrumentation.
1.3.4.2. Red Hat OpenShift distributed tracing architecture
Red Hat OpenShift distributed tracing is made up of several components that work together to collect, store, and display tracing data.
Red Hat OpenShift distributed tracing platform - This component is based on the open source Jaeger project.
- Client (Jaeger client, Tracer, Reporter, instrumented application, client libraries)- The distributed tracing platform clients are language-specific implementations of the OpenTracing API. They can be used to instrument applications for distributed tracing either manually or with a variety of existing open source frameworks, such as Camel (Fuse), Spring Boot (RHOAR), MicroProfile (RHOAR/Thorntail), Wildfly (EAP), and many more, that are already integrated with OpenTracing.
- Agent (Jaeger agent, Server Queue, Processor Workers) - The distributed tracing platform agent is a network daemon that listens for spans sent over User Datagram Protocol (UDP), which it batches and sends to the Collector. The agent is meant to be placed on the same host as the instrumented application. This is typically accomplished by having a sidecar in container environments such as Kubernetes.
- Jaeger Collector (Collector, Queue, Workers) - Similar to the Jaeger agent, the Jaeger Collector receives spans and places them in an internal queue for processing. This allows the Jaeger Collector to return immediately to the client/agent instead of waiting for the span to make its way to the storage.
- Storage (Data Store) - Collectors require a persistent storage backend. Red Hat OpenShift distributed tracing platform has a pluggable mechanism for span storage. Note that for this release, the only supported storage is Elasticsearch.
- Query (Query Service) - Query is a service that retrieves traces from storage.
- Ingester (Ingester Service) - Red Hat OpenShift distributed tracing can use Apache Kafka as a buffer between the Collector and the actual Elasticsearch backing storage. Ingester is a service that reads data from Kafka and writes to the Elasticsearch storage backend.
- Jaeger Console – With the Red Hat OpenShift distributed tracing platform user interface, you can visualize your distributed tracing data. On the Search page, you can find traces and explore details of the spans that make up an individual trace.
Red Hat OpenShift distributed tracing data collection - This component is based on the open source OpenTelemetry project.
- OpenTelemetry Collector - The OpenTelemetry Collector is a vendor-agnostic way to receive, process, and export telemetry data. The OpenTelemetry Collector supports open-source observability data formats, for example, Jaeger and Prometheus, sending to one or more open-source or commercial back-ends. The Collector is the default location instrumentation libraries export their telemetry data.
1.3.4.3. Red Hat OpenShift distributed tracing features
Red Hat OpenShift distributed tracing provides the following capabilities:
- Integration with Kiali – When properly configured, you can view distributed tracing data from the Kiali console.
- High scalability – The distributed tracing back end is designed to have no single points of failure and to scale with the business needs.
- Distributed Context Propagation – Enables you to connect data from different components together to create a complete end-to-end trace.
- Backwards compatibility with Zipkin – Red Hat OpenShift distributed tracing has APIs that enable it to be used as a drop-in replacement for Zipkin, but Red Hat is not supporting Zipkin compatibility in this release.
1.3.5. Next steps
- Prepare to install Red Hat OpenShift Service Mesh in your OpenShift Container Platform environment.
1.4. Service mesh deployment models
Red Hat OpenShift Service Mesh supports several different deployment models that can be combined in different ways to best suit your business requirements.
1.4.1. Single mesh deployment model
The simplest Istio deployment model is a single mesh.
Service names within a mesh must be unique because Kubernetes only allows one service to be named myservice
in the mynamespace
namespace. However, workload instances can share a common identity since service account names can be shared across workloads in the same namespace
1.4.2. Single tenancy deployment model
In Istio, a tenant is a group of users that share common access and privileges for a set of deployed workloads. You can use tenants to provide a level of isolation between different teams. You can segregate access to different tenants using NetworkPolicies
, AuthorizationPolicies
, and exportTo
annotations on istio.io or service resources.
Single tenant, cluster-wide Service Mesh control plane configurations are deprecated as of Red Hat OpenShift Service Mesh version 1.0. Red Hat OpenShift Service Mesh defaults to a multitenant model.
1.4.3. Multitenant deployment model
Red Hat OpenShift Service Mesh installs a ServiceMeshControlPlane
that is configured for multitenancy by default. Red Hat OpenShift Service Mesh uses a multitenant Operator to manage the Service Mesh control plane lifecycle. Within a mesh, namespaces are used for tenancy.
Red Hat OpenShift Service Mesh uses ServiceMeshControlPlane
resources to manage mesh installations, whose scope is limited by default to namespace that contains the resource. You use ServiceMeshMemberRoll
and ServiceMeshMember
resources to include additional namespaces into the mesh. A namespace can only be included in a single mesh, and multiple meshes can be installed in a single OpenShift cluster.
Typical service mesh deployments use a single Service Mesh control plane to configure communication between services in the mesh. Red Hat OpenShift Service Mesh supports “soft multitenancy”, where there is one control plane and one mesh per tenant, and there can be multiple independent control planes within the cluster. Multitenant deployments specify the projects that can access the Service Mesh and isolate the Service Mesh from other control plane instances.
The cluster administrator gets control and visibility across all the Istio control planes, while the tenant administrator only gets control over their specific Service Mesh, Kiali, and Jaeger instances.
You can grant a team permission to deploy its workloads only to a given namespace or set of namespaces. If granted the mesh-user
role by the service mesh administrator, users can create a ServiceMeshMember
resource to add namespaces to the ServiceMeshMemberRoll
.
1.4.4. Multimesh or federated deployment model
Federation is a deployment model that lets you share services and workloads between separate meshes managed in distinct administrative domains.
The Istio multi-cluster model requires a high level of trust between meshes and remote access to all Kubernetes API servers on which the individual meshes reside. Red Hat OpenShift Service Mesh federation takes an opinionated approach to a multi-cluster implementation of Service Mesh that assumes minimal trust between meshes.
A federated mesh is a group of meshes behaving as a single mesh. The services in each mesh can be unique services, for example a mesh adding services by importing them from another mesh, can provide additional workloads for the same services across the meshes, providing high availability, or a combination of both. All meshes that are joined into a federated mesh remain managed individually, and you must explicitly configure which services are exported to and imported from other meshes in the federation. Support functions such as certificate generation, metrics and trace collection remain local in their respective meshes.
1.5. Service Mesh and Istio differences
Red Hat OpenShift Service Mesh differs from an installation of Istio to provide additional features or to handle differences when deploying on OpenShift Container Platform.
1.5.1. Differences between Istio and Red Hat OpenShift Service Mesh
The following features are different in Service Mesh and Istio.
1.5.1.1. Command line tool
The command line tool for Red Hat OpenShift Service Mesh is oc
. Red Hat OpenShift Service Mesh does not support istioctl
.
1.5.1.2. Installation and upgrades
Red Hat OpenShift Service Mesh does not support Istio installation profiles.
Red Hat OpenShift Service Mesh does not support canary upgrades of the service mesh.
1.5.1.3. Automatic injection
The upstream Istio community installation automatically injects the sidecar into pods within the projects you have labeled.
Red Hat OpenShift Service Mesh does not automatically inject the sidecar to any pods, but requires you to opt in to injection using an annotation without labeling projects. This method requires fewer privileges and does not conflict with other OpenShift capabilities such as builder pods. To enable automatic injection you specify the sidecar.istio.io/inject
annotation as described in the Automatic sidecar injection section.
Upstream Istio | Red Hat OpenShift Service Mesh | |
---|---|---|
Namespace Label | supports "enabled" and "disabled" | supports "disabled" |
Pod Label | supports "true" and "false" | not supported |
Pod Annotation | supports "false" only | supports "true" and "false" |
1.5.1.4. Istio Role Based Access Control features
Istio Role Based Access Control (RBAC) provides a mechanism you can use to control access to a service. You can identify subjects by user name or by specifying a set of properties and apply access controls accordingly.
The upstream Istio community installation includes options to perform exact header matches, match wildcards in headers, or check for a header containing a specific prefix or suffix.
Red Hat OpenShift Service Mesh extends the ability to match request headers by using a regular expression. Specify a property key of request.regex.headers
with a regular expression.
Upstream Istio community matching request headers example
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: httpbin-usernamepolicy spec: action: ALLOW rules: - when: - key: 'request.regex.headers[username]' values: - "allowed.*" selector: matchLabels: app: httpbin
1.5.1.5. OpenSSL
Red Hat OpenShift Service Mesh replaces BoringSSL with OpenSSL. OpenSSL is a software library that contains an open source implementation of the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols. The Red Hat OpenShift Service Mesh Proxy binary dynamically links the OpenSSL libraries (libssl and libcrypto) from the underlying Red Hat Enterprise Linux operating system.
1.5.1.6. External workloads
Red Hat OpenShift Service Mesh does not support external workloads, such as virtual machines running outside OpenShift on bare metal servers.
1.5.1.7. Virtual Machine Support
You can deploy virtual machines to OpenShift using OpenShift Virtualization. Then, you can apply a mesh policy, such as mTLS or AuthorizationPolicy, to these virtual machines, just like any other pod that is part of a mesh.
1.5.1.8. Component modifications
- A maistra-version label has been added to all resources.
- All Ingress resources have been converted to OpenShift Route resources.
- Grafana, distributed tracing (Jaeger), and Kiali are enabled by default and exposed through OpenShift routes.
- Godebug has been removed from all templates
-
The
istio-multi
ServiceAccount and ClusterRoleBinding have been removed, as well as theistio-reader
ClusterRole.
1.5.1.9. Envoy filters
Red Hat OpenShift Service Mesh does not support EnvoyFilter
configuration except where explicitly documented. Due to tight coupling with the underlying Envoy APIs, backward compatibility cannot be maintained. EnvoyFilter
patches are very sensitive to the format of the Envoy configuration that is generated by Istio. If the configuration generated by Istio changes, it has the potential to break the application of the EnvoyFilter
.
1.5.1.10. Envoy services
Red Hat OpenShift Service Mesh does not support QUIC-based services.
1.5.1.11. Istio Container Network Interface (CNI) plug-in
Red Hat OpenShift Service Mesh includes CNI plug-in, which provides you with an alternate way to configure application pod networking. The CNI plug-in replaces the init-container
network configuration eliminating the need to grant service accounts and projects access to security context constraints (SCCs) with elevated privileges.
1.5.1.12. Global mTLS settings
Red Hat OpenShift Service Mesh creates a PeerAuthentication
resource that enables or disables Mutual TLS authentication (mTLS) within the mesh.
1.5.1.13. Gateways
Red Hat OpenShift Service Mesh installs ingress and egress gateways by default. You can disable gateway installation in the ServiceMeshControlPlane
(SMCP) resource by using the following settings:
-
spec.gateways.enabled=false
to disable both ingress and egress gateways. -
spec.gateways.ingress.enabled=false
to disable ingress gateways. -
spec.gateways.egress.enabled=false
to disable egress gateways.
The Operator annotates the default gateways to indicate that they are generated by and managed by the Red Hat OpenShift Service Mesh Operator.
1.5.1.14. Multicluster configurations
Red Hat OpenShift Service Mesh does not provide support for multicluster configurations.
1.5.1.15. Custom Certificate Signing Requests (CSR)
You cannot configure Red Hat OpenShift Service Mesh to process CSRs through the Kubernetes certificate authority (CA).
1.5.1.16. Routes for Istio Gateways
OpenShift routes for Istio Gateways are automatically managed in Red Hat OpenShift Service Mesh. Every time an Istio Gateway is created, updated or deleted inside the service mesh, an OpenShift route is created, updated or deleted.
A Red Hat OpenShift Service Mesh control plane component called Istio OpenShift Routing (IOR) synchronizes the gateway route. For more information, see Automatic route creation.
1.5.1.16.1. Catch-all domains
Catch-all domains ("*") are not supported. If one is found in the Gateway definition, Red Hat OpenShift Service Mesh will create the route, but will rely on OpenShift to create a default hostname. This means that the newly created route will not be a catch all ("*") route, instead it will have a hostname in the form <route-name>[-<project>].<suffix>
. See the OpenShift Container Platform documentation for more information about how default hostnames work and how a cluster-admin
can customize it. If you use Red Hat OpenShift Dedicated, refer to the Red Hat OpenShift Dedicated the dedicated-admin
role.
1.5.1.16.2. Subdomains
Subdomains (e.g.: "*.domain.com") are supported. However this ability doesn’t come enabled by default in OpenShift Container Platform. This means that Red Hat OpenShift Service Mesh will create the route with the subdomain, but it will only be in effect if OpenShift Container Platform is configured to enable it.
1.5.1.16.3. Transport layer security
Transport Layer Security (TLS) is supported. This means that, if the Gateway contains a tls
section, the OpenShift Route will be configured to support TLS.
Additional resources
1.5.2. Multitenant installations
Whereas upstream Istio takes a single tenant approach, Red Hat OpenShift Service Mesh supports multiple independent control planes within the cluster. Red Hat OpenShift Service Mesh uses a multitenant operator to manage the control plane lifecycle.
Red Hat OpenShift Service Mesh installs a multitenant control plane by default. You specify the projects that can access the Service Mesh, and isolate the Service Mesh from other control plane instances.
1.5.2.1. Multitenancy versus cluster-wide installations
The main difference between a multitenant installation and a cluster-wide installation is the scope of privileges used by istod. The components no longer use cluster-scoped Role Based Access Control (RBAC) resource ClusterRoleBinding
.
Every project in the ServiceMeshMemberRoll
members
list will have a RoleBinding
for each service account associated with the control plane deployment and each control plane deployment will only watch those member projects. Each member project has a maistra.io/member-of
label added to it, where the member-of
value is the project containing the control plane installation.
Red Hat OpenShift Service Mesh configures each member project to ensure network access between itself, the control plane, and other member projects. The exact configuration differs depending on how OpenShift Container Platform software-defined networking (SDN) is configured. See About OpenShift SDN for additional details.
If the OpenShift Container Platform cluster is configured to use the SDN plug-in:
NetworkPolicy
: Red Hat OpenShift Service Mesh creates aNetworkPolicy
resource in each member project allowing ingress to all pods from the other members and the control plane. If you remove a member from Service Mesh, thisNetworkPolicy
resource is deleted from the project.NoteThis also restricts ingress to only member projects. If you require ingress from non-member projects, you need to create a
NetworkPolicy
to allow that traffic through.-
Multitenant: Red Hat OpenShift Service Mesh joins the
NetNamespace
for each member project to theNetNamespace
of the control plane project (the equivalent of runningoc adm pod-network join-projects --to control-plane-project member-project
). If you remove a member from the Service Mesh, itsNetNamespace
is isolated from the control plane (the equivalent of runningoc adm pod-network isolate-projects member-project
). - Subnet: No additional configuration is performed.
1.5.2.2. Cluster scoped resources
Upstream Istio has two cluster scoped resources that it relies on. The MeshPolicy
and the ClusterRbacConfig
. These are not compatible with a multitenant cluster and have been replaced as described below.
- ServiceMeshPolicy replaces MeshPolicy for configuration of control-plane-wide authentication policies. This must be created in the same project as the control plane.
- ServicemeshRbacConfig replaces ClusterRbacConfig for configuration of control-plane-wide role based access control. This must be created in the same project as the control plane.
1.5.3. Kiali and service mesh
Installing Kiali via the Service Mesh on OpenShift Container Platform differs from community Kiali installations in multiple ways. These modifications are sometimes necessary to resolve issues, provide additional features, or to handle differences when deploying on OpenShift Container Platform.
- Kiali has been enabled by default.
- Ingress has been enabled by default.
- Updates have been made to the Kiali ConfigMap.
- Updates have been made to the ClusterRole settings for Kiali.
-
Do not edit the ConfigMap, because your changes might be overwritten by the Service Mesh or Kiali Operators. Files that the Kiali Operator manages have a
kiali.io/
label or annotation. Updating the Operator files should be restricted to those users withcluster-admin
privileges. If you use Red Hat OpenShift Dedicated, updating the Operator files should be restricted to those users withdedicated-admin
privileges.
1.5.4. Distributed tracing and service mesh
Installing the distributed tracing platform with the Service Mesh on OpenShift Container Platform differs from community Jaeger installations in multiple ways. These modifications are sometimes necessary to resolve issues, provide additional features, or to handle differences when deploying on OpenShift Container Platform.
- Distributed tracing has been enabled by default for Service Mesh.
- Ingress has been enabled by default for Service Mesh.
-
The name for the Zipkin port name has changed to
jaeger-collector-zipkin
(fromhttp
) -
Jaeger uses Elasticsearch for storage by default when you select either the
production
orstreaming
deployment option. - The community version of Istio provides a generic "tracing" route. Red Hat OpenShift Service Mesh uses a "jaeger" route that is installed by the Red Hat OpenShift distributed tracing platform Operator and is already protected by OAuth.
- Red Hat OpenShift Service Mesh uses a sidecar for the Envoy proxy, and Jaeger also uses a sidecar, for the Jaeger agent. These two sidecars are configured separately and should not be confused with each other. The proxy sidecar creates spans related to the pod’s ingress and egress traffic. The agent sidecar receives the spans emitted by the application and sends them to the Jaeger Collector.
1.6. Preparing to install Service Mesh
Before you can install Red Hat OpenShift Service Mesh, you must subscribe to OpenShift Container Platform and install OpenShift Container Platform in a supported configuration.
1.6.1. Prerequisites
- Maintain an active OpenShift Container Platform subscription on your Red Hat account. If you do not have a subscription, contact your sales representative for more information.
Review the OpenShift Container Platform 4.6 overview.
- Install OpenShift Container Platform 4.6 on AWS
- Install OpenShift Container Platform 4.6 on user-provisioned AWS
- Install OpenShift Container Platform 4.6 on bare metal
- Install OpenShift Container Platform 4.6 on vSphere
- Install OpenShift Container Platform 4.6 on IBM Z and LinuxONE
- Install OpenShift Container Platform 4.6 on IBM Power Systems
Install the version of the OpenShift Container Platform command line utility (the
oc
client tool) that matches your OpenShift Container Platform version and add it to your path.- If you are using OpenShift Container Platform 4.6, see About the OpenShift CLI.
For additional information about Red Hat OpenShift Service Mesh lifecycle and supported platforms, refer to the Support Policy.
1.6.2. Supported configurations
The following configurations are supported for the current release of Red Hat OpenShift Service Mesh.
1.6.2.1. Supported platforms
The Red Hat OpenShift Service Mesh Operator supports multiple versions of the ServiceMeshControlPlane
resource. Version 2.2 Service Mesh control planes are supported on the following platform versions:
- Red Hat OpenShift Container Platform version 4.9 or later.
- Red Hat OpenShift Dedicated version 4.
- Azure Red Hat OpenShift (ARO) version 4.
- Red Hat OpenShift Service on AWS (ROSA).
1.6.2.2. Unsupported configurations
Explicitly unsupported cases include:
- OpenShift Online is not supported for Red Hat OpenShift Service Mesh.
- Red Hat OpenShift Service Mesh does not support the management of microservices outside the cluster where Service Mesh is running.
1.6.2.3. Supported network configurations
Red Hat OpenShift Service Mesh supports the following network configurations.
- OpenShift-SDN
- OVN-Kubernetes is supported on OpenShift Container Platform 4.7.32+, OpenShift Container Platform 4.8.12+, and OpenShift Container Platform 4.9+.
- Third-Party Container Network Interface (CNI) plug-ins that have been certified on OpenShift Container Platform and passed Service Mesh conformance testing. See Certified OpenShift CNI Plug-ins for more information.
1.6.2.4. Supported configurations for Service Mesh
This release of Red Hat OpenShift Service Mesh is only available on OpenShift Container Platform x86_64, IBM Z, and IBM Power Systems.
- IBM Z is only supported on OpenShift Container Platform 4.6 and later.
- IBM Power Systems is only supported on OpenShift Container Platform 4.6 and later.
- Configurations where all Service Mesh components are contained within a single OpenShift Container Platform cluster.
- Configurations that do not integrate external services such as virtual machines.
-
Red Hat OpenShift Service Mesh does not support
EnvoyFilter
configuration except where explicitly documented.
1.6.2.5. Supported configurations for Kiali
- The Kiali console is only supported on the two most recent releases of the Chrome, Edge, Firefox, or Safari browsers.
1.6.2.6. Supported configurations for Distributed Tracing
- Jaeger agent as a sidecar is the only supported configuration for Jaeger. Jaeger as a daemonset is not supported for multitenant installations or OpenShift Dedicated.
1.6.2.7. Supported WebAssembly module
- 3scale WebAssembly is the only provided WebAssembly module. You can create custom WebAssembly modules.
1.6.3. Next steps
- Install Red Hat OpenShift Service Mesh in your OpenShift Container Platform environment.
1.7. Installing the Operators
To install Red Hat OpenShift Service Mesh, first install the required Operators on OpenShift Container Platform and then create a ServiceMeshControlPlane
resource to deploy the control plane.
This basic installation is configured based on the default OpenShift settings and is not designed for production use. Use this default installation to verify your installation, and then configure your service mesh for your specific environment.
Prerequisites
- Read the Preparing to install Red Hat OpenShift Service Mesh process.
-
An account with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role.
The following steps show how to install a basic instance of Red Hat OpenShift Service Mesh on OpenShift Container Platform.
1.7.1. Operator overview
Red Hat OpenShift Service Mesh requires the following four Operators:
- OpenShift Elasticsearch - (Optional) Provides database storage for tracing and logging with the distributed tracing platform. It is based on the open source Elasticsearch project.
- Red Hat OpenShift distributed tracing platform - Provides distributed tracing to monitor and troubleshoot transactions in complex distributed systems. It is based on the open source Jaeger project.
- Kiali - Provides observability for your service mesh. Allows you to view configurations, monitor traffic, and analyze traces in a single console. It is based on the open source Kiali project.
-
Red Hat OpenShift Service Mesh - Allows you to connect, secure, control, and observe the microservices that comprise your applications. The Service Mesh Operator defines and monitors the
ServiceMeshControlPlane
resources that manage the deployment, updating, and deletion of the Service Mesh components. It is based on the open source Istio project.
Do not install Community versions of the Operators. Community Operators are not supported.
1.7.2. Installing the Operators
To install Red Hat OpenShift Service Mesh, install following Operators in this order. Repeat the procedure for each Operator.
- OpenShift Elasticsearch
- Red Hat OpenShift distributed tracing platform
- Kiali
- Red Hat OpenShift Service Mesh
If you have already installed the OpenShift Elasticsearch Operator as part of OpenShift Logging, you do not need to install the OpenShift Elasticsearch Operator again. The Red Hat OpenShift distributed tracing platform Operator will create the Elasticsearch instance using the installed OpenShift Elasticsearch Operator.
Procedure
-
Log in to the OpenShift Container Platform web console as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role. - In the OpenShift Container Platform web console, click Operators → OperatorHub.
- Type the name of the Operator into the filter box and select the Red Hat version of the Operator. Community versions of the Operators are not supported.
- Click Install.
- On the Install Operator page for each Operator, accept the default settings.
Click Install. Wait until the Operator has installed before repeating the steps for the next Operator in the list.
-
The OpenShift Elasticsearch Operator is installed in the
openshift-operators-redhat
namespace and is available for all namespaces in the cluster. -
The Red Hat OpenShift distributed tracing platform is installed in the
openshift-distributed-tracing
namespace and is available for all namespaces in the cluster. -
The Kiali and Red Hat OpenShift Service Mesh Operators are installed in the
openshift-operators
namespace and are available for all namespaces in the cluster.
-
The OpenShift Elasticsearch Operator is installed in the
- After all you have installed all four Operators, click Operators → Installed Operators to verify that your Operators installed.
1.7.3. Next steps
The Red Hat OpenShift Service Mesh Operator does not create the various Service Mesh custom resource definitions (CRDs) until you deploy a Service Mesh control plane. You use the ServiceMeshControlPlane
resource to install and configure the Service Mesh components. For more information, see Creating the ServiceMeshControlPlane.
1.8. Creating the ServiceMeshControlPlane
You can deploy a basic installation of the ServiceMeshControlPlane
(SMCP) by using either the OpenShift Container Platform web console or from the command line using the oc
client tool.
This basic installation is configured based on the default OpenShift settings and is not designed for production use. Use this default installation to verify your installation, and then configure your ServiceMeshControlPlane
for your environment.
Red Hat OpenShift Service on AWS (ROSA) places additional restrictions on where you can create resources and as a result the default deployment does not work. See Installing Service Mesh on Red Hat OpenShift Service on AWS for additional requirements before deploying your SMCP in a ROSA environment.
The Service Mesh documentation uses istio-system
as the example project, but you can deploy the service mesh to any project.
1.8.1. Deploying the Service Mesh control plane from the web console
You can deploy a basic ServiceMeshControlPlane
by using the web console. In this example, istio-system
is the name of the Service Mesh control plane project.
Prerequisites
- The Red Hat OpenShift Service Mesh Operator must be installed.
-
An account with the
cluster-admin
role.
Procedure
-
Log in to the OpenShift Container Platform web console as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role. Create a project named
istio-system
.- Navigate to Home → Projects.
- Click Create Project.
In the Name field, enter
istio-system
. TheServiceMeshControlPlane
resource must be installed in a project that is separate from your microservices and Operators.These steps use
istio-system
as an example, but you can deploy your Service Mesh control plane in any project as long as it is separate from the project that contains your services.- Click Create.
- Navigate to Operators → Installed Operators.
- Click the Red Hat OpenShift Service Mesh Operator, then click Istio Service Mesh Control Plane.
- On the Istio Service Mesh Control Plane tab, click Create ServiceMeshControlPlane.
On the Create ServiceMeshControlPlane page, accept the default Service Mesh control plane version to take advantage of the features available in the most current version of the product. The version of the control plane determines the features available regardless of the version of the Operator.
You can configure
ServiceMeshControlPlane
settings later. For more information, see Configuring Red Hat OpenShift Service Mesh.- Click Create. The Operator creates pods, services, and Service Mesh control plane components based on your configuration parameters.
To verify the control plane installed correctly, click the Istio Service Mesh Control Plane tab.
- Click the name of the new control plane.
- Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources the Operator created and configured.
1.8.2. Deploying the Service Mesh control plane using the CLI
You can deploy a basic ServiceMeshControlPlane
from the command line.
Prerequisites
- The Red Hat OpenShift Service Mesh Operator must be installed.
-
Access to the OpenShift CLI (
oc
).
Procedure
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role.$ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443
Create a project named
istio-system
.$ oc new-project istio-system
Create a
ServiceMeshControlPlane
file namedistio-installation.yaml
using the following example. The version of the Service Mesh control plane determines the features available regardless of the version of the Operator.Example version 2.2 istio-installation.yaml
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic namespace: istio-system spec: version: v2.2 tracing: type: Jaeger sampling: 10000 addons: jaeger: name: jaeger install: storage: type: Memory kiali: enabled: true name: kiali grafana: enabled: true
Run the following command to deploy the Service Mesh control plane, where
<istio_installation.yaml>
includes the full path to your file.$ oc create -n istio-system -f <istio_installation.yaml>
To watch the progress of the pod deployment, run the following command:
$ oc get pods -n istio-system -w
You should see output similar to the following:
NAME READY STATUS RESTARTS AGE grafana-b4d59bd7-mrgbr 2/2 Running 0 65m istio-egressgateway-678dc97b4c-wrjkp 1/1 Running 0 108s istio-ingressgateway-b45c9d54d-4qg6n 1/1 Running 0 108s istiod-basic-55d78bbbcd-j5556 1/1 Running 0 108s jaeger-67c75bd6dc-jv6k6 2/2 Running 0 65m kiali-6476c7656c-x5msp 1/1 Running 0 43m prometheus-58954b8d6b-m5std 2/2 Running 0 66m wasm-cacher-basic-8c986c75-vj2cd 1/1 Running 0 65m
1.8.3. Validating your SMCP installation with the CLI
You can validate the creation of the ServiceMeshControlPlane
from the command line.
Procedure
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role.$ oc login https://<HOSTNAME>:6443
Run the following command to verify the Service Mesh control plane installation, where
istio-system
is the namespace where you installed the Service Mesh control plane.$ oc get smcp -n istio-system
The installation has finished successfully when the
STATUS
column isComponentsReady
.NAME READY STATUS PROFILES VERSION AGE basic 10/10 ComponentsReady ["default"] 2.1.1 66m
1.8.4. Validating your SMCP installation with Kiali
You can use the Kiali console to validate your Service Mesh installation. The Kiali console offers several ways to validate your Service Mesh components are deployed and configured properly.
Procedure
-
Log in to the OpenShift Container Platform web console as a user with cluster-admin rights. If you use Red Hat OpenShift Dedicated, you must have an account with the
dedicated-admin
role. - Navigate to Networking → Routes.
On the Routes page, select the Service Mesh control plane project, for example
istio-system
, from the Namespace menu.The Location column displays the linked address for each route.
- If necessary, use the filter to find the route for the Kiali console. Click the route Location to launch the console.
Click Log In With OpenShift.
When you first log in to the Kiali Console, you see the Overview page which displays all the namespaces in your service mesh that you have permission to view. When there are multiple namespaces shown on the Overview page, Kiali shows namespaces with health or validation problems first.
Figure 1.1. Kiali Overview page
The tile for each namespace displays the number of labels, the Istio Config health, the number of and Applications health, and Traffic for the namespace. If you are validating the console installation and namespaces have not yet been added to the mesh, there might not be any data to display other than
istio-system
.Kiali has four dashboards specifically for the namespace where the Service Mesh control plane is installed. To view these dashboards, click the Options menu on the tile for the control plane namespace, for example,
istio-system
, and select one of the following options:- Istio Mesh Dashboard
- Istio Control Plane Dashboard
- Istio Performance Dashboard
Istio Wasm Exetension Dashboard
Figure 1.2. Grafana Istio Control Plane Dashboard
Kiali also installs two additional Grafana dashboards, available from the Grafana Home page:
- Istio Workload Dashboard
- Istio Service Dashboard
To view the Service Mesh control plane nodes, click the Graph page, select the Namespace where you installed the
ServiceMeshControlPlane
from the menu, for exampleistio-system
.- If necessary, click Display idle nodes.
- To learn more about the Graph page, click the Graph tour link.
- To view the mesh topology, select one or more additional namespaces from the Service Mesh Member Roll from the Namespace menu.
To view the list of applications in the
istio-system
namespace, click the Applications page. Kiali displays the health of the applications.- Hover your mouse over the information icon to view any additional information noted in the Details column.
To view the list of workloads in the
istio-system
namespace, click the Workloads page. Kiali displays the health of the workloads.- Hover your mouse over the information icon to view any additional information noted in the Details column.
To view the list of services in the
istio-system
namespace, click the Services page. Kiali displays the health of the services and of the configurations.- Hover your mouse over the information icon to view any additional information noted in the Details column.
To view a list of the Istio Configuration objects in the
istio-system
namespace, click the Istio Config page. Kiali displays the health of the configuration.- If there are configuration errors, click the row and Kiali opens the configuration file with the error highlighted.
1.8.5. Installing on Red Hat OpenShift Service on AWS (ROSA)
Starting with version 2.2, Red Hat OpenShift Service Mesh supports installation on Red Hat OpenShift Service on AWS (ROSA). This section documents the additional requirements when installing Service Mesh on this platform.
1.8.5.1. Installation location
You must create a new namespace, for example istio-system
, when installing Red Hat OpenShift Service Mesh and creating the ServiceMeshControlPlane
.
1.8.5.2. Required Service Mesh control plane configuration
The default configuration in the ServiceMeshControlPlane
file does not work on a ROSA cluster. You must modify the default SMCP and set spec.security.identity.type=ThirdParty
when installing on Red Hat OpenShift Service on AWS.
Example ServiceMeshControlPlane
resource for ROSA
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic namespace: istio-system spec: version: v2.2 security: identity: type: ThirdParty #required setting for ROSA tracing: type: Jaeger sampling: 10000 policy: type: Istiod addons: grafana: enabled: true jaeger: install: storage: type: Memory kiali: enabled: true prometheus: enabled: true telemetry: type: Istiod
1.8.5.3. Restrictions on Kiali configuration
Red Hat OpenShift Service on AWS places additional restrictions on where you can create resources and does not let you create the Kiali resource in a Red Hat managed namespace.
This means that the following common settings for spec.deployment.accessible_namespaces
are not allowed in a ROSA cluster:
-
['**']
(all namespaces) -
default
-
codeready-*
-
openshift-*
-
redhat-*
The validation error message provides a complete list of all the restricted namespaces.
Example Kiali
resource for ROSA
apiVersion: kiali.io/v1alpha1 kind: Kiali metadata: name: kiali namespace: istio-system spec: auth: strategy: openshift deployment: accessible_namespaces: #restricted setting for ROSA - istio-system image_pull_policy: '' ingress_enabled: true namespace: istio-system
1.8.6. Additional resources
Red Hat OpenShift Service Mesh supports multiple independent control planes within the cluster. You can create reusable configurations with ServiceMeshControlPlane
profiles. For more information, see Creating control plane profiles.
1.8.7. Next steps
Create a ServiceMeshMemberRoll
resource to specify the namespaces associated with the Service Mesh. For more information, see Adding services to a service mesh.
1.9. Adding services to a service mesh
After installing the Operators and ServiceMeshControlPlane
resource, add applications, workloads, or services to your mesh by creating a ServiceMeshMemberRoll
resource and specifying the namespaces where your content is located. If you already have an application, workload, or service to add to a ServiceMeshMemberRoll
resource, use the following steps. Or, to install a sample application called Bookinfo and add it to a ServiceMeshMemberRoll
resource, skip to the tutorial for installing the Bookinfo example application to see how an application works in Red Hat OpenShift Service Mesh.
The items listed in the ServiceMeshMemberRoll
resource are the applications and workflows that are managed by the ServiceMeshControlPlane
resource. The control plane, which includes the Service Mesh Operators, Istiod, and ServiceMeshControlPlane
, and the data plane, which includes applications and Envoy proxy, must be in separate namespaces.
After you add the namespace to the ServiceMeshMemberRoll
, access to services or pods in that namespace will not be accessible to callers outside the service mesh.
1.9.1. Creating the Red Hat OpenShift Service Mesh member roll
The ServiceMeshMemberRoll
lists the projects that belong to the Service Mesh control plane. Only projects listed in the ServiceMeshMemberRoll
are affected by the control plane. A project does not belong to a service mesh until you add it to the member roll for a particular control plane deployment.
You must create a ServiceMeshMemberRoll
resource named default
in the same project as the ServiceMeshControlPlane
, for example istio-system
.
1.9.1.1. Creating the member roll from the web console
You can add one or more projects to the Service Mesh member roll from the web console. In this example, istio-system
is the name of the Service Mesh control plane project.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
- List of existing projects to add to the service mesh.
Procedure
- Log in to the OpenShift Container Platform web console.
If you do not already have services for your mesh, or you are starting from scratch, create a project for your applications. It must be different from the project where you installed the Service Mesh control plane.
- Navigate to Home → Projects.
- Enter a name in the Name field.
- Click Create.
- Navigate to Operators → Installed Operators.
-
Click the Project menu and choose the project where your
ServiceMeshControlPlane
resource is deployed from the list, for exampleistio-system
. - Click the Red Hat OpenShift Service Mesh Operator.
- Click the Istio Service Mesh Member Roll tab.
- Click Create ServiceMeshMemberRoll
-
Click Members, then enter the name of your project in the Value field. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. - Click Create.
1.9.1.2. Creating the member roll from the CLI
You can add a project to the ServiceMeshMemberRoll
from the command line.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
- List of projects to add to the service mesh.
-
Access to the OpenShift CLI (
oc
).
Procedure
Log in to the OpenShift Container Platform CLI.
$ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443
If you do not already have services for your mesh, or you are starting from scratch, create a project for your applications. It must be different from the project where you installed the Service Mesh control plane.
$ oc new-project <your-project>
To add your projects as members, modify the following example YAML. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. In this example,istio-system
is the name of the Service Mesh control plane project.Example servicemeshmemberroll-default.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshMemberRoll metadata: name: default namespace: istio-system spec: members: # a list of projects joined into the service mesh - your-project-name - another-project-name
Run the following command to upload and create the
ServiceMeshMemberRoll
resource in theistio-system
namespace.$ oc create -n istio-system -f servicemeshmemberroll-default.yaml
Run the following command to verify the
ServiceMeshMemberRoll
was created successfully.$ oc get smmr -n istio-system default
The installation has finished successfully when the
STATUS
column isConfigured
.
1.9.2. Adding or removing projects from the service mesh
You can add or remove projects from an existing Service Mesh ServiceMeshMemberRoll
resource using the web console.
-
You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. -
The
ServiceMeshMemberRoll
resource is deleted when its correspondingServiceMeshControlPlane
resource is deleted.
1.9.2.1. Adding or removing projects from the member roll using the web console
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
An existing
ServiceMeshMemberRoll
resource. -
Name of the project with the
ServiceMeshMemberRoll
resource. - Names of the projects you want to add or remove from the mesh.
Procedure
- Log in to the OpenShift Container Platform web console.
- Navigate to Operators → Installed Operators.
-
Click the Project menu and choose the project where your
ServiceMeshControlPlane
resource is deployed from the list, for exampleistio-system
. - Click the Red Hat OpenShift Service Mesh Operator.
- Click the Istio Service Mesh Member Roll tab.
-
Click the
default
link. - Click the YAML tab.
-
Modify the YAML to add or remove projects as members. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource. - Click Save.
- Click Reload.
1.9.2.2. Adding or removing projects from the member roll using the CLI
You can modify an existing Service Mesh member roll using the command line.
Prerequisites
- An installed, verified Red Hat OpenShift Service Mesh Operator.
-
An existing
ServiceMeshMemberRoll
resource. -
Name of the project with the
ServiceMeshMemberRoll
resource. - Names of the projects you want to add or remove from the mesh.
-
Access to the OpenShift CLI (
oc
).
Procedure
- Log in to the OpenShift Container Platform CLI.
Edit the
ServiceMeshMemberRoll
resource.$ oc edit smmr -n <controlplane-namespace>
Modify the YAML to add or remove projects as members. You can add any number of projects, but a project can only belong to one
ServiceMeshMemberRoll
resource.Example servicemeshmemberroll-default.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshMemberRoll metadata: name: default namespace: istio-system #control plane project spec: members: # a list of projects joined into the service mesh - your-project-name - another-project-name
1.9.3. Bookinfo example application
The Bookinfo example application allows you to test your Red Hat OpenShift Service Mesh 2.2.3 installation on OpenShift Container Platform.
The Bookinfo application displays information about a book, similar to a single catalog entry of an online book store. The application displays a page that describes the book, book details (ISBN, number of pages, and other information), and book reviews.
The Bookinfo application consists of these microservices:
-
The
productpage
microservice calls thedetails
andreviews
microservices to populate the page. -
The
details
microservice contains book information. -
The
reviews
microservice contains book reviews. It also calls theratings
microservice. -
The
ratings
microservice contains book ranking information that accompanies a book review.
There are three versions of the reviews microservice:
-
Version v1 does not call the
ratings
Service. -
Version v2 calls the
ratings
Service and displays each rating as one to five black stars. -
Version v3 calls the
ratings
Service and displays each rating as one to five red stars.
1.9.3.1. Installing the Bookinfo application
This tutorial walks you through how to create a sample application by creating a project, deploying the Bookinfo application to that project, and viewing the running application in Service Mesh.
Prerequisites:
- OpenShift Container Platform 4.1 or higher installed.
- Red Hat OpenShift Service Mesh 2.2.3 installed.
-
Access to the OpenShift CLI (
oc
). -
An account with the
cluster-admin
role.
The Bookinfo sample application cannot be installed on IBM Z and IBM Power Systems.
The commands in this section assume the Service Mesh control plane project is istio-system
. If you installed the control plane in another namespace, edit each command before you run it.
Procedure
-
Log in to the OpenShift Container Platform web console as a user with cluster-admin rights. If you use Red Hat OpenShift Dedicated, you must have an account with the
dedicated-admin
role. - Click Home → Projects.
- Click Create Project.
Enter
bookinfo
as the Project Name, enter a Display Name, and enter a Description, then click Create.Alternatively, you can run this command from the CLI to create the
bookinfo
project.$ oc new-project bookinfo
- Click Operators → Installed Operators.
-
Click the Project menu and use the Service Mesh control plane namespace. In this example, use
istio-system
. - Click the Red Hat OpenShift Service Mesh Operator.
Click the Istio Service Mesh Member Roll tab.
- If you have already created a Istio Service Mesh Member Roll, click the name, then click the YAML tab to open the YAML editor.
-
If you have not created a
ServiceMeshMemberRoll
, click Create ServiceMeshMemberRoll.
- Click Members, then enter the name of your project in the Value field.
Click Create to save the updated Service Mesh Member Roll.
Or, save the following example to a YAML file.
Bookinfo ServiceMeshMemberRoll example servicemeshmemberroll-default.yaml
apiVersion: maistra.io/v1 kind: ServiceMeshMemberRoll metadata: name: default spec: members: - bookinfo
Run the following command to upload that file and create the
ServiceMeshMemberRoll
resource in theistio-system
namespace. In this example,istio-system
is the name of the Service Mesh control plane project.$ oc create -n istio-system -f servicemeshmemberroll-default.yaml
Run the following command to verify the
ServiceMeshMemberRoll
was created successfully.$ oc get smmr -n istio-system -o wide
The installation has finished successfully when the
STATUS
column isConfigured
.NAME READY STATUS AGE MEMBERS default 1/1 Configured 70s ["bookinfo"]
From the CLI, deploy the Bookinfo application in the `bookinfo` project by applying the
bookinfo.yaml
file:$ oc apply -n bookinfo -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/platform/kube/bookinfo.yaml
You should see output similar to the following:
service/details created serviceaccount/bookinfo-details created deployment.apps/details-v1 created service/ratings created serviceaccount/bookinfo-ratings created deployment.apps/ratings-v1 created service/reviews created serviceaccount/bookinfo-reviews created deployment.apps/reviews-v1 created deployment.apps/reviews-v2 created deployment.apps/reviews-v3 created service/productpage created serviceaccount/bookinfo-productpage created deployment.apps/productpage-v1 created
Create the ingress gateway by applying the
bookinfo-gateway.yaml
file:$ oc apply -n bookinfo -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/networking/bookinfo-gateway.yaml
You should see output similar to the following:
gateway.networking.istio.io/bookinfo-gateway created virtualservice.networking.istio.io/bookinfo created
Set the value for the
GATEWAY_URL
parameter:$ export GATEWAY_URL=$(oc -n istio-system get route istio-ingressgateway -o jsonpath='{.spec.host}')
1.9.3.2. Adding default destination rules
Before you can use the Bookinfo application, you must first add default destination rules. There are two preconfigured YAML files, depending on whether or not you enabled mutual transport layer security (TLS) authentication.
Procedure
To add destination rules, run one of the following commands:
If you did not enable mutual TLS:
$ oc apply -n bookinfo -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/networking/destination-rule-all.yaml
If you enabled mutual TLS:
$ oc apply -n bookinfo -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/networking/destination-rule-all-mtls.yaml
You should see output similar to the following:
destinationrule.networking.istio.io/productpage created destinationrule.networking.istio.io/reviews created destinationrule.networking.istio.io/ratings created destinationrule.networking.istio.io/details created
1.9.3.3. Verifying the Bookinfo installation
To confirm that the sample Bookinfo application was successfully deployed, perform the following steps.
Prerequisites
- Red Hat OpenShift Service Mesh installed.
- Complete the steps for installing the Bookinfo sample app.
Procedure from CLI
- Log in to the OpenShift Container Platform CLI.
Verify that all pods are ready with this command:
$ oc get pods -n bookinfo
All pods should have a status of
Running
. You should see output similar to the following:NAME READY STATUS RESTARTS AGE details-v1-55b869668-jh7hb 2/2 Running 0 12m productpage-v1-6fc77ff794-nsl8r 2/2 Running 0 12m ratings-v1-7d7d8d8b56-55scn 2/2 Running 0 12m reviews-v1-868597db96-bdxgq 2/2 Running 0 12m reviews-v2-5b64f47978-cvssp 2/2 Running 0 12m reviews-v3-6dfd49b55b-vcwpf 2/2 Running 0 12m
Run the following command to retrieve the URL for the product page:
echo "http://$GATEWAY_URL/productpage"
- Copy and paste the output in a web browser to verify the Bookinfo product page is deployed.
Procedure from Kiali web console
Obtain the address for the Kiali web console.
-
Log in to the OpenShift Container Platform web console as a user with
cluster-admin
rights. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role. - Navigate to Networking → Routes.
On the Routes page, select the Service Mesh control plane project, for example
istio-system
, from the Namespace menu.The Location column displays the linked address for each route.
- Click the link in the Location column for Kiali.
- Click Log In With OpenShift. The Kiali Overview screen presents tiles for each project namespace.
-
Log in to the OpenShift Container Platform web console as a user with
- In Kiali, click Graph.
- Select bookinfo from the Namespace list, and App graph from the Graph Type list.
Click Display idle nodes from the Display menu.
This displays nodes that are defined but have not received or sent requests. It can confirm that an application is properly defined, but that no request traffic has been reported.
- Use the Duration menu to increase the time period to help ensure older traffic is captured.
- Use the Refresh Rate menu to refresh traffic more or less often, or not at all.
- Click Services, Workloads or Istio Config to see list views of bookinfo components, and confirm that they are healthy.
1.9.3.4. Removing the Bookinfo application
Follow these steps to remove the Bookinfo application.
Prerequisites
- OpenShift Container Platform 4.1 or higher installed.
- Red Hat OpenShift Service Mesh 2.2.3 installed.
-
Access to the OpenShift CLI (
oc
).
1.9.3.4.1. Delete the Bookinfo project
Procedure
- Log in to the OpenShift Container Platform web console.
- Click to Home → Projects.
-
Click the
bookinfo
menu , and then click Delete Project. Type
bookinfo
in the confirmation dialog box, and then click Delete.Alternatively, you can run this command using the CLI to create the
bookinfo
project.$ oc delete project bookinfo
1.9.3.4.2. Remove the Bookinfo project from the Service Mesh member roll
Procedure
- Log in to the OpenShift Container Platform web console.
- Click Operators → Installed Operators.
-
Click the Project menu and choose
istio-system
from the list. - Click the Istio Service Mesh Member Roll link under Provided APIS for the Red Hat OpenShift Service Mesh Operator.
-
Click the
ServiceMeshMemberRoll
menu and select Edit Service Mesh Member Roll. Edit the default Service Mesh Member Roll YAML and remove
bookinfo
from the members list.Alternatively, you can run this command using the CLI to remove the
bookinfo
project from theServiceMeshMemberRoll
. In this example,istio-system
is the name of the Service Mesh control plane project.$ oc -n istio-system patch --type='json' smmr default -p '[{"op": "remove", "path": "/spec/members", "value":["'"bookinfo"'"]}]'
- Click Save to update Service Mesh Member Roll.
1.9.4. Next steps
- To continue the installation process, you must enable sidecar injection.
1.10. Enabling sidecar injection
After adding the namespaces that contain your services to your mesh, the next step is to enable automatic sidecar injection in the Deployment resource for your application. You must enable automatic sidecar injection for each deployment.
If you have installed the Bookinfo sample application, the application was deployed and the sidecars were injected as part of the installation procedure. If you are using your own project and service, deploy your applications on OpenShift Container Platform. For more information, see the OpenShift Container Platform documentation, Understanding Deployment and DeploymentConfig objects.
1.10.1. Prerequisites
- Services deployed to the mesh, for example the Bookinfo sample application.
- A Deployment resource file.
1.10.2. Enabling automatic sidecar injection
When deploying an application, you must opt-in to injection by configuring the annotation sidecar.istio.io/inject
in spec.template.metadata.annotations
to true
in the deployment
object. Opting in ensures that the sidecar injection does not interfere with other OpenShift Container Platform features such as builder pods used by numerous frameworks within the OpenShift Container Platform ecosystem.
Prerequisites
- Identify the namespaces that are part of your service mesh and the deployments that need automatic sidecar injection.
Procedure
To find your deployments use the
oc get
command.$ oc get deployment -n <namespace>
For example, to view the deployment file for the 'ratings-v1' microservice in the
bookinfo
namespace, use the following command to see the resource in YAML format.oc get deployment -n bookinfo ratings-v1 -o yaml
- Open the application’s deployment configuration YAML file in an editor.
Add
spec.template.metadata.annotations.sidecar.istio/inject
to your Deployment YAML and setsidecar.istio.io/inject
totrue
as shown in the following example.Example snippet from bookinfo deployment-ratings-v1.yaml
apiVersion: apps/v1 kind: Deployment metadata: name: ratings-v1 namespace: bookinfo labels: app: ratings version: v1 spec: template: metadata: annotations: sidecar.istio.io/inject: 'true'
- Save the Deployment configuration file.
Add the file back to the project that contains your app.
$ oc apply -n <namespace> -f deployment.yaml
In this example,
bookinfo
is the name of the project that contains theratings-v1
app anddeployment-ratings-v1.yaml
is the file you edited.$ oc apply -n bookinfo -f deployment-ratings-v1.yaml
To verify that the resource uploaded successfully, run the following command.
$ oc get deployment -n <namespace> <deploymentName> -o yaml
For example,
$ oc get deployment -n bookinfo ratings-v1 -o yaml
1.10.3. Validating sidecar injection
The Kiali console offers several ways to validate whether or not your applications, services, and workloads have a sidecar proxy.
Figure 1.3. Missing sidecar badge
The Graph page displays a node badge indicating a Missing Sidecar on the following graphs:
- App graph
- Versioned app graph
- Workload graph
Figure 1.4. Missing sidecar icon
The Applications page displays a Missing Sidecar icon in the Details column for any applications in a namespace that do not have a sidecar.
The Workloads page displays a Missing Sidecar icon in the Details column for any applications in a namespace that do not have a sidecar.
The Services page displays a Missing Sidecar icon in the Details column for any applications in a namespace that do not have a sidecar. When there are multiple versions of a service, you use the Service Details page to view Missing Sidecar icons.
The Workload Details page has a special unified Logs tab that lets you view and correlate application and proxy logs. You can view the Envoy logs as another way to validate sidecar injection for your application workloads.
The Workload Details page also has an Envoy tab for any workload that is an Envoy proxy or has been injected with an Envoy proxy. This tab displays a built-in Envoy dashboard that includes subtabs for Clusters, Listeners, Routes, Bootstrap, Config, and Metrics.
For information about enabling Envoy access logs, see the Troubleshooting section.
For information about viewing Envoy logs, see Viewing logs in the Kiali console
1.10.4. Setting proxy environment variables through annotations
Configuration for the Envoy sidecar proxies is managed by the ServiceMeshControlPlane
.
You can set environment variables for the sidecar proxy for applications by adding pod annotations to the deployment in the injection-template.yaml
file. The environment variables are injected to the sidecar.
Example injection-template.yaml
apiVersion: apps/v1 kind: Deployment metadata: name: resource spec: replicas: 7 selector: matchLabels: app: resource template: metadata: annotations: sidecar.maistra.io/proxyEnv: "{ \"maistra_test_env\": \"env_value\", \"maistra_test_env_2\": \"env_value_2\" }"
You should never include maistra.io/
labels and annotations when creating your own custom resources. These labels and annotations indicate that the resources are generated and managed by the Operator. If you are copying content from an Operator-generated resource when creating your own resources, do not include labels or annotations that start with maistra.io/
. Resources that include these labels or annotations will be overwritten or deleted by the Operator during the next reconciliation.
1.10.5. Updating sidecar proxies
In order to update the configuration for sidecar proxies the application administrator must restart the application pods.
If your deployment uses automatic sidecar injection, you can update the pod template in the deployment by adding or modifying an annotation. Run the following command to redeploy the pods:
$ oc patch deployment/<deployment> -p '{"spec":{"template":{"metadata":{"annotations":{"kubectl.kubernetes.io/restartedAt": "'`date -Iseconds`'"}}}}}'
If your deployment does not use automatic sidecar injection, you must manually update the sidecars by modifying the sidecar container image specified in the deployment or pod, and then restart the pods.
1.10.6. Next steps
Configure Red Hat OpenShift Service Mesh features for your environment.
1.11. Upgrading Service Mesh
To access the most current features of Red Hat OpenShift Service Mesh, upgrade to the current version, 2.2.3.
1.11.1. Understanding versioning
Red Hat uses semantic versioning for product releases. Semantic Versioning is a 3-component number in the format of X.Y.Z, where:
- X stands for a Major version. Major releases usually denote some sort of breaking change: architectural changes, API changes, schema changes, and similar major updates.
- Y stands for a Minor version. Minor releases contain new features and functionality while maintaining backwards compatibility.
- Z stands for a Patch version (also known as a z-stream release). Patch releases are used to addresses Common Vulnerabilities and Exposures (CVEs) and release bug fixes. New features and functionality are generally not released as part of a Patch release.
1.11.1.1. How versioning affects Service Mesh upgrades
Depending on the version of the update you are making, the upgrade process is different.
- Patch updates - Patch upgrades are managed by the Operator Lifecycle Manager (OLM); they happen automatically when you update your Operators.
-
Minor upgrades - Minor upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the
spec.version
value in yourServiceMeshControlPlane
resources. -
Major upgrades - Major upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the
spec.version
value in yourServiceMeshControlPlane
resources. Because major upgrades can contain changes that are not backwards compatible, additional manual changes might be required.
1.11.1.2. Understanding Service Mesh versions
In order to understand what version of Red Hat OpenShift Service Mesh you have deployed on your system, you need to understand how each of the component versions is managed.
Operator version - The most current Operator version is 2.2.3. The Operator version number only indicates the version of the currently installed Operator. Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, the version of the Operator does not determine the version of your deployed
ServiceMeshControlPlane
resources.ImportantUpgrading to the latest Operator version automatically applies patch updates, but does not automatically upgrade your Service Mesh control plane to the latest minor version.
ServiceMeshControlPlane version - The
ServiceMeshControlPlane
version determines what version of Red Hat OpenShift Service Mesh you are using. The value of thespec.version
field in theServiceMeshControlPlane
resource controls the architecture and configuration settings that are used to install and deploy Red Hat OpenShift Service Mesh. When you create the Service Mesh control plane you can set the version in one of two ways:- To configure in the Form View, select the version from the Control Plane Version menu.
-
To configure in the YAML View, set the value for
spec.version
in the YAML file.
Operator Lifecycle Manager (OLM) does not manage Service Mesh control plane upgrades, so the version number for your Operator and ServiceMeshControlPlane
(SMCP) may not match, unless you have manually upgraded your SMCP.
1.11.2. Upgrade considerations
The maistra.io/
label or annotation should not be used on a user-created custom resource, because it indicates that the resource was generated by and should be managed by the Red Hat OpenShift Service Mesh Operator.
During the upgrade, the Operator makes changes, including deleting or replacing files, to resources that include the following labels or annotations that indicate that the resource is managed by the Operator.
Before upgrading check for user-created custom resources that include the following labels or annotations:
-
maistra.io/
AND theapp.kubernetes.io/managed-by
label set tomaistra-istio-operator
(Red Hat OpenShift Service Mesh) -
kiali.io/
(Kiali) -
jaegertracing.io/
(Red Hat OpenShift distributed tracing platform) -
logging.openshift.io/
(Red Hat Elasticsearch)
Before upgrading, check your user-created custom resources for labels or annotations that indicate they are Operator managed. Remove the label or annotation from custom resources that you do not want to be managed by the Operator.
When upgrading to version 2.0, the Operator only deletes resources with these labels in the same namespace as the SMCP.
When upgrading to version 2.1, the Operator deletes resources with these labels in all namespaces.
1.11.2.1. Known issues that may affect upgrade
Known issues that may affect your upgrade include:
-
Red Hat OpenShift Service Mesh does not support the use of
EnvoyFilter
configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained. If you are using Envoy Filters, and the configuration generated by Istio has changed due to the lastest version of Envoy introduced by upgrading yourServiceMeshControlPlane
, that has the potential to break anyEnvoyFilter
you may have implemented. -
OSSM-1505
ServiceMeshExtension
does not work with OpenShift Container Platform version 4.11. BecauseServiceMeshExtension
has been deprecated in Red Hat OpenShift Service Mesh 2.2, this known issue will not be fixed and you must migrate your extensions toWasmPluging
-
OSSM-1396 If a gateway resource contains the
spec.externalIPs
setting, rather than being recreated when theServiceMeshControlPlane
is updated, the gateway is removed and never recreated.
OSSM-1052 When configuring a Service
ExternalIP
for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).
1.11.3. Upgrading the Operators
In order to keep your Service Mesh patched with the latest security fixes, bug fixes, and software updates, you must keep your Operators updated. You initiate patch updates by upgrading your Operators.
The version of the Operator does not determine the version of your service mesh. The version of your deployed Service Mesh control plane determines your version of Service Mesh.
Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, updating the Red Hat OpenShift Service Mesh Operator does not update the spec.version
value of your deployed ServiceMeshControlPlane
. Also note that the spec.version
value is a two digit number, for example 2.2, and that patch updates, for example 2.2.1, are not reflected in the SMCP version value.
Operator Lifecycle Manager (OLM) controls the installation, upgrade, and role-based access control (RBAC) of Operators in a cluster. The OLM runs by default in OpenShift Container Platform. OLM queries for available Operators as well as upgrades for installed Operators.
Whether or not you have to take action to upgrade your Operators depends on the settings you selected when installing them. When you installed each of your Operators, you selected an Update Channel and an Approval Strategy. The combination of these two settings determine when and how your Operators are updated.
Versioned channel | "Stable" or "Preview" Channel | |
---|---|---|
Automatic | Automatically updates the Operator for minor and patch releases for that version only. Will not automatically update to the next major version (that is, from version 2.0 to 3.0). Manual change to Operator subscription required to update to the next major version. | Automatically updates Operator for all major, minor, and patch releases. |
Manual | Manual updates required for minor and patch releases for the specified version. Manual change to Operator subscription required to update to the next major version. | Manual updates required for all major, minor, and patch releases. |
When you update your Red Hat OpenShift Service Mesh Operator the Operator Lifecycle Manager (OLM) removes the old Operator pod and starts a new pod. Once the new Operator pod starts, the reconciliation process checks the ServiceMeshControlPlane
(SMCP), and if there are updated container images available for any of the Service Mesh control plane components, it replaces those Service Mesh control plane pods with ones that use the new container images.
When you upgrade the Kiali and Red Hat OpenShift distributed tracing platform Operators, the OLM reconciliation process scans the cluster and upgrades the managed instances to the version of the new Operator. For example, if you update the Red Hat OpenShift distributed tracing platform Operator from version 1.30.2 to version 1.34.1, the Operator scans for running instances of distributed tracing platform and upgrades them to 1.34.1 as well.
To stay on a particular patch version of Red Hat OpenShift Service Mesh, you would need to disable automatic updates and remain on that specific version of the Operator.
For more information about upgrading Operators, refer to the Operator Lifecycle Manager documentation.
1.11.4. Upgrading the control plane
You must manually update the control plane for minor and major releases. The community Istio project recommends canary upgrades, Red Hat OpenShift Service Mesh only supports in-place upgrades. Red Hat OpenShift Service Mesh requires that you upgrade from each minor release to the next minor release in sequence. For example, you must upgrade from version 2.0 to version 2.1, and then upgrade to version 2.2. You cannot update from Red Hat OpenShift Service Mesh 2.0 to 2.2 directly.
When you upgrade the service mesh control plane, all Operator managed resources, for example gateways, are also upgraded.
Although you can deploy multiple versions of the control plane in the same cluster, Red Hat OpenShift Service Mesh does not support canary upgrades of the service mesh. That is, you can have different SCMP resources with different values for spec.version
, but they cannot be managing the same mesh.
1.11.4.1. Upgrade changes from version 2.1 to version 2.2
Upgrading the Service Mesh control plane from version 2.1 to 2.2 introduces the following behavioral changes:
-
The
istio-node
DaemonSet is renamed toistio-cni-node
to match the name in upstream Istio. -
Istio 1.10 updated Envoy to send traffic to the application container using
eth0
rather thanlo
by default. -
This release adds support for the
WasmPlugin
API and deprecates theServiceMeshExtention
API.
For more information about migrating your extensions, refer to Migrating from ServiceMeshExtension to WasmPlugin resources.
1.11.4.2. Upgrade changes from version 2.0 to version 2.1
Upgrading the Service Mesh control plane from version 2.0 to 2.1 introduces the following architectural and behavioral changes.
Architecture changes
Mixer has been completely removed in Red Hat OpenShift Service Mesh 2.1. Upgrading from a Red Hat OpenShift Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled.
If you see the following message when upgrading from v2.0 to v2.1, update the existing Mixer
type to Istiod
type in the existing Control Plane spec before you update the .spec.version
field:
An error occurred admission webhook smcp.validation.maistra.io denied the request: [support for policy.type "Mixer" and policy.Mixer options have been removed in v2.1, please use another alternative, support for telemetry.type "Mixer" and telemetry.Mixer options have been removed in v2.1, please use another alternative]”
For example:
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: policy: type: Istiod telemetry: type: Istiod version: v2.2
Behavioral changes
AuthorizationPolicy
updates:-
With the PROXY protocol, if you’re using
ipBlocks
andnotIpBlocks
to specify remote IP addresses, update the configuration to useremoteIpBlocks
andnotRemoteIpBlocks
instead. - Added support for nested JSON Web Token (JWT) claims.
-
With the PROXY protocol, if you’re using
EnvoyFilter
breaking changes>-
Must use
typed_config
- xDS v2 is no longer supported
- Deprecated filter names
-
Must use
- Older versions of proxies may report 503 status codes when receiving 1xx or 204 status codes from newer proxies.
1.11.4.3. Upgrading the Service Mesh control plane
To upgrade Red Hat OpenShift Service Mesh, you must update the version field of the Red Hat OpenShift Service Mesh ServiceMeshControlPlane
v2 resource. Then, once it is configured and applied, restart the application pods to update each sidecar proxy and its configuration.
Prerequisites
- You are running OpenShift Container Platform 4.9 or later.
- You have the latest Red Hat OpenShift Service Mesh Operator.
Procedure
Switch to the project that contains your
ServiceMeshControlPlane
resource. In this example,istio-system
is the name of the Service Mesh control plane project.$ oc project istio-system
Check your v2
ServiceMeshControlPlane
resource configuration to verify it is valid.Run the following command to view your
ServiceMeshControlPlane
resource as a v2 resource.$ oc get smcp -o yaml
TipBack up your Service Mesh control plane configuration.
Update the
.spec.version
field and apply the configuration.For example:
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic spec: version: v2.2
Alternatively, instead of using the command line, you can use the web console to edit the Service Mesh control plane. In the OpenShift Container Platform web console, click Project and select the project name you just entered.
- Click Operators → Installed Operators.
-
Find your
ServiceMeshControlPlane
instance. - Select YAML view and update text of the YAML file, as shown in the previous example.
- Click Save.
1.11.4.4. Migrating Red Hat OpenShift Service Mesh from version 1.1 to version 2.0
Upgrading from version 1.1 to 2.0 requires manual steps that migrate your workloads and application to a new instance of Red Hat OpenShift Service Mesh running the new version.
Prerequisites
- You must upgrade to OpenShift Container Platform 4.7. before you upgrade to Red Hat OpenShift Service Mesh 2.0.
- You must have Red Hat OpenShift Service Mesh version 2.0 operator. If you selected the automatic upgrade path, the operator automatically downloads the latest information. However, there are steps you must take to use the features in Red Hat OpenShift Service Mesh version 2.0.
1.11.4.4.1. Upgrading Red Hat OpenShift Service Mesh
To upgrade Red Hat OpenShift Service Mesh, you must create an instance of Red Hat OpenShift Service Mesh ServiceMeshControlPlane
v2 resource in a new namespace. Then, once it’s configured, move your microservice applications and workloads from your old mesh to the new service mesh.
Procedure
Check your v1
ServiceMeshControlPlane
resource configuration to make sure it is valid.Run the following command to view your
ServiceMeshControlPlane
resource as a v2 resource.$ oc get smcp -o yaml
-
Check the
spec.techPreview.errored.message
field in the output for information about any invalid fields. - If there are invalid fields in your v1 resource, the resource is not reconciled and cannot be edited as a v2 resource. All updates to v2 fields will be overridden by the original v1 settings. To fix the invalid fields, you can replace, patch, or edit the v1 version of the resource. You can also delete the resource without fixing it. After the resource has been fixed, it can be reconciled, and you can to modify or view the v2 version of the resource.
To fix the resource by editing a file, use
oc get
to retrieve the resource, edit the text file locally, and replace the resource with the file you edited.$ oc get smcp.v1.maistra.io <smcp_name> > smcp-resource.yaml #Edit the smcp-resource.yaml file. $ oc replace -f smcp-resource.yaml
To fix the resource using patching, use
oc patch
.$ oc patch smcp.v1.maistra.io <smcp_name> --type json --patch '[{"op": "replace","path":"/spec/path/to/bad/setting","value":"corrected-value"}]'
To fix the resource by editing with command line tools, use
oc edit
.$ oc edit smcp.v1.maistra.io <smcp_name>
Back up your Service Mesh control plane configuration. Switch to the project that contains your
ServiceMeshControlPlane
resource. In this example,istio-system
is the name of the Service Mesh control plane project.$ oc project istio-system
Enter the following command to retrieve the current configuration. Your <smcp_name> is specified in the metadata of your
ServiceMeshControlPlane
resource, for examplebasic-install
orfull-install
.$ oc get servicemeshcontrolplanes.v1.maistra.io <smcp_name> -o yaml > <smcp_name>.v1.yaml
Convert your
ServiceMeshControlPlane
to a v2 control plane version that contains information about your configuration as a starting point.$ oc get smcp <smcp_name> -o yaml > <smcp_name>.v2.yaml
Create a project. In the OpenShift Container Platform console Project menu, click
New Project
and enter a name for your project,istio-system-upgrade
, for example. Or, you can run this command from the CLI.$ oc new-project istio-system-upgrade
-
Update the
metadata.namespace
field in your v2ServiceMeshControlPlane
with your new project name. In this example, useistio-system-upgrade
. -
Update the
version
field from 1.1 to 2.0 or remove it in your v2ServiceMeshControlPlane
. Create a
ServiceMeshControlPlane
in the new namespace. On the command line, run the following command to deploy the control plane with the v2 version of theServiceMeshControlPlane
that you retrieved. In this example, replace `<smcp_name.v2> `with the path to your file.$ oc create -n istio-system-upgrade -f <smcp_name>.v2.yaml
Alternatively, you can use the console to create the Service Mesh control plane. In the OpenShift Container Platform web console, click Project. Then, select the project name you just entered.
- Click Operators → Installed Operators.
- Click Create ServiceMeshControlPlane.
-
Select YAML view and paste text of the YAML file you retrieved into the field. Check that the
apiVersion
field is set tomaistra.io/v2
and modify themetadata.namespace
field to use the new namespace, for exampleistio-system-upgrade
. - Click Create.
1.11.4.4.2. Configuring the 2.0 ServiceMeshControlPlane
The ServiceMeshControlPlane
resource has been changed for Red Hat OpenShift Service Mesh version 2.0. After you created a v2 version of the ServiceMeshControlPlane
resource, modify it to take advantage of the new features and to fit your deployment. Consider the following changes to the specification and behavior of Red Hat OpenShift Service Mesh 2.0 as you’re modifying your ServiceMeshControlPlane
resource. You can also refer to the Red Hat OpenShift Service Mesh 2.0 product documentation for new information to features you use. The v2 resource must be used for Red Hat OpenShift Service Mesh 2.0 installations.
1.11.4.4.2.1. Architecture changes
The architectural units used by previous versions have been replaced by Istiod. In 2.0 the Service Mesh control plane components Mixer, Pilot, Citadel, Galley, and the sidecar injector functionality have been combined into a single component, Istiod.
Although Mixer is no longer supported as a control plane component, Mixer policy and telemetry plugins are now supported through WASM extensions in Istiod. Mixer can be enabled for policy and telemetry if you need to integrate legacy Mixer plugins.
Secret Discovery Service (SDS) is used to distribute certificates and keys to sidecars directly from Istiod. In Red Hat OpenShift Service Mesh version 1.1, secrets were generated by Citadel, which were used by the proxies to retrieve their client certificates and keys.
1.11.4.4.2.2. Annotation changes
The following annotations are no longer supported in v2.0. If you are using one of these annotations, you must update your workload before moving it to a v2.0 Service Mesh control plane.
-
sidecar.maistra.io/proxyCPULimit
has been replaced withsidecar.istio.io/proxyCPULimit
. If you were usingsidecar.maistra.io
annotations on your workloads, you must modify those workloads to usesidecar.istio.io
equivalents instead. -
sidecar.maistra.io/proxyMemoryLimit
has been replaced withsidecar.istio.io/proxyMemoryLimit
-
sidecar.istio.io/discoveryAddress
is no longer supported. Also, the default discovery address has moved frompilot.<control_plane_namespace>.svc:15010
(or port 15011, if mtls is enabled) toistiod-<smcp_name>.<control_plane_namespace>.svc:15012
. -
The health status port is no longer configurable and is hard-coded to 15021. * If you were defining a custom status port, for example,
status.sidecar.istio.io/port
, you must remove the override before moving the workload to a v2.0 Service Mesh control plane. Readiness checks can still be disabled by setting the status port to0
. - Kubernetes Secret resources are no longer used to distribute client certificates for sidecars. Certificates are now distributed through Istiod’s SDS service. If you were relying on mounted secrets, they are longer available for workloads in v2.0 Service Mesh control planes.
1.11.4.4.2.3. Behavioral changes
Some features in Red Hat OpenShift Service Mesh 2.0 work differently than they did in previous versions.
-
The readiness port on gateways has moved from
15020
to15021
. - The target host visibility includes VirtualService, as well as ServiceEntry resources. It includes any restrictions applied through Sidecar resources.
- Automatic mutual TLS is enabled by default. Proxy to proxy communication is automatically configured to use mTLS, regardless of global PeerAuthentication policies in place.
-
Secure connections are always used when proxies communicate with the Service Mesh control plane regardless of
spec.security.controlPlane.mtls
setting. Thespec.security.controlPlane.mtls
setting is only used when configuring connections for Mixer telemetry or policy.
1.11.4.4.2.4. Migration details for unsupported resources
Policy (authentication.istio.io/v1alpha1)
Policy resources must be migrated to new resource types for use with v2.0 Service Mesh control planes, PeerAuthentication and RequestAuthentication. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.
Mutual TLS
Mutual TLS enforcement is accomplished using the security.istio.io/v1beta1
PeerAuthentication resource. The legacy spec.peers.mtls.mode
field maps directly to the new resource’s spec.mtls.mode
field. Selection criteria has changed from specifying a service name in spec.targets[x].name
to a label selector in spec.selector.matchLabels
. In PeerAuthentication, the labels must match the selector on the services named in the targets list. Any port-specific settings will need to be mapped into spec.portLevelMtls
.
Authentication
Additional authentication methods specified in spec.origins
, must be mapped into a security.istio.io/v1beta1
RequestAuthentication resource. spec.selector.matchLabels
must be configured similarly to the same field on PeerAuthentication. Configuration specific to JWT principals from spec.origins.jwt
items map to similar fields in spec.rules
items.
-
spec.origins[x].jwt.triggerRules
specified in the Policy must be mapped into one or moresecurity.istio.io/v1beta1
AuthorizationPolicy resources. Anyspec.selector.labels
must be configured similarly to the same field on RequestAuthentication. -
spec.origins[x].jwt.triggerRules.excludedPaths
must be mapped into an AuthorizationPolicy whose spec.action is set to ALLOW, withspec.rules[x].to.operation.path
entries matching the excluded paths. -
spec.origins[x].jwt.triggerRules.includedPaths
must be mapped into a separate AuthorizationPolicy whosespec.action
is set toALLOW
, withspec.rules[x].to.operation.path
entries matching the included paths, andspec.rules.[x].from.source.requestPrincipals
entries that align with thespecified spec.origins[x].jwt.issuer
in the Policy resource.
ServiceMeshPolicy (maistra.io/v1)
ServiceMeshPolicy was configured automatically for the Service Mesh control plane through the spec.istio.global.mtls.enabled
in the v1 resource or spec.security.dataPlane.mtls
in the v2 resource setting. For v2 control planes, a functionally equivalent PeerAuthentication resource is created during installation. This feature is deprecated in Red Hat OpenShift Service Mesh version 2.0
RbacConfig, ServiceRole, ServiceRoleBinding (rbac.istio.io/v1alpha1)
These resources were replaced by the security.istio.io/v1beta1
AuthorizationPolicy resource.
Mimicking RbacConfig behavior requires writing a default AuthorizationPolicy whose settings depend on the spec.mode specified in the RbacConfig.
-
When
spec.mode
is set toOFF
, no resource is required as the default policy is ALLOW, unless an AuthorizationPolicy applies to the request. -
When
spec.mode
is set to ON, setspec: {}
. You must create AuthorizationPolicy policies for all services in the mesh. -
spec.mode
is set toON_WITH_INCLUSION
, must create an AuthorizationPolicy withspec: {}
in each included namespace. Inclusion of individual services is not supported by AuthorizationPolicy. However, as soon as any AuthorizationPolicy is created that applies to the workloads for the service, all other requests not explicitly allowed will be denied. -
When
spec.mode
is set toON_WITH_EXCLUSION
, it is not supported by AuthorizationPolicy. A global DENY policy can be created, but an AuthorizationPolicy must be created for every workload in the mesh because there is no allow-all policy that can be applied to either a namespace or a workload.
AuthorizationPolicy includes configuration for both the selector to which the configuration applies, which is similar to the function ServiceRoleBinding provides and the rules which should be applied, which is similar to the function ServiceRole provides.
ServiceMeshRbacConfig (maistra.io/v1)
This resource is replaced by using a security.istio.io/v1beta1
AuthorizationPolicy resource with an empty spec.selector in the Service Mesh control plane’s namespace. This policy will be the default authorization policy applied to all workloads in the mesh. For specific migration details, see RbacConfig above.
1.11.4.4.2.5. Mixer plugins
Mixer components are disabled by default in version 2.0. If you rely on Mixer plugins for your workload, you must configure your version 2.0 ServiceMeshControlPlane
to include the Mixer components.
To enable the Mixer policy components, add the following snippet to your ServiceMeshControlPlane
.
spec: policy: type: Mixer
To enable the Mixer telemetry components, add the following snippet to your ServiceMeshControlPlane
.
spec: telemetry: type: Mixer
Legacy mixer plugins can also be migrated to WASM and integrated using the new ServiceMeshExtension (maistra.io/v1alpha1) custom resource.
Built-in WASM filters included in the upstream Istio distribution are not available in Red Hat OpenShift Service Mesh 2.0.
1.11.4.4.2.6. Mutual TLS changes
When using mTLS with workload specific PeerAuthentication policies, a corresponding DestinationRule is required to allow traffic if the workload policy differs from the namespace/global policy.
Auto mTLS is enabled by default, but can be disabled by setting spec.security.dataPlane.automtls
to false in the ServiceMeshControlPlane
resource. When disabling auto mTLS, DestinationRules may be required for proper communication between services. For example, setting PeerAuthentication to STRICT
for one namespace may prevent services in other namespaces from accessing them, unless a DestinationRule configures TLS mode for the services in the namespace.
For information about mTLS, see Enabling mutual Transport Layer Security (mTLS)
1.11.4.4.2.6.1. Other mTLS Examples
To disable mTLS For productpage service in the bookinfo sample application, your Policy resource was configured the following way for Red Hat OpenShift Service Mesh v1.1.
Example Policy resource
apiVersion: authentication.istio.io/v1alpha1 kind: Policy metadata: name: productpage-mTLS-disable namespace: <namespace> spec: targets: - name: productpage
To disable mTLS For productpage service in the bookinfo sample application, use the following example to configure your PeerAuthentication resource for Red Hat OpenShift Service Mesh v2.0.
Example PeerAuthentication resource
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: productpage-mTLS-disable namespace: <namespace> spec: mtls: mode: DISABLE selector: matchLabels: # this should match the selector for the "productpage" service app: productpage
To enable mTLS With JWT authentication for the productpage
service in the bookinfo sample application, your Policy resource was configured the following way for Red Hat OpenShift Service Mesh v1.1.
Example Policy resource
apiVersion: authentication.istio.io/v1alpha1 kind: Policy metadata: name: productpage-mTLS-with-JWT namespace: <namespace> spec: targets: - name: productpage ports: - number: 9000 peers: - mtls: origins: - jwt: issuer: "https://securetoken.google.com" audiences: - "productpage" jwksUri: "https://www.googleapis.com/oauth2/v1/certs" jwtHeaders: - "x-goog-iap-jwt-assertion" triggerRules: - excludedPaths: - exact: /health_check principalBinding: USE_ORIGIN
To enable mTLS With JWT authentication for the productpage service in the bookinfo sample application, use the following example to configure your PeerAuthentication resource for Red Hat OpenShift Service Mesh v2.0.
Example PeerAuthentication resource
#require mtls for productpage:9000 apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: productpage-mTLS-with-JWT namespace: <namespace> spec: selector: matchLabels: # this should match the selector for the "productpage" service app: productpage portLevelMtls: 9000: mode: STRICT --- #JWT authentication for productpage apiVersion: security.istio.io/v1beta1 kind: RequestAuthentication metadata: name: productpage-mTLS-with-JWT namespace: <namespace> spec: selector: matchLabels: # this should match the selector for the "productpage" service app: productpage jwtRules: - issuer: "https://securetoken.google.com" audiences: - "productpage" jwksUri: "https://www.googleapis.com/oauth2/v1/certs" fromHeaders: - name: "x-goog-iap-jwt-assertion" --- #Require JWT token to access product page service from #any client to all paths except /health_check apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: productpage-mTLS-with-JWT namespace: <namespace> spec: action: ALLOW selector: matchLabels: # this should match the selector for the "productpage" service app: productpage rules: - to: # require JWT token to access all other paths - operation: notPaths: - /health_check from: - source: # if using principalBinding: USE_PEER in the Policy, # then use principals, e.g. # principals: # - “*” requestPrincipals: - “*” - to: # no JWT token required to access health_check - operation: paths: - /health_check
1.11.4.4.3. Configuration recipes
You can configure the following items with these configuration recipes.
1.11.4.4.3.1. Mutual TLS in a data plane
Mutual TLS for data plane communication is configured through spec.security.dataPlane.mtls
in the ServiceMeshControlPlane
resource, which is false
by default.
1.11.4.4.3.2. Custom signing key
Istiod manages client certificates and private keys used by service proxies. By default, Istiod uses a self-signed certificate for signing, but you can configure a custom certificate and private key. For more information about how to configure signing keys, see Adding an external certificate authority key and certificate
1.11.4.4.3.3. Tracing
Tracing is configured in spec.tracing
. Currently, the only type of tracer that is supported is Jaeger
. Sampling is a scaled integer representing 0.01% increments, for example, 1 is 0.01% and 10000 is 100%. The tracing implementation and sampling rate can be specified:
spec: tracing: sampling: 100 # 1% type: Jaeger
Jaeger is configured in the addons
section of the ServiceMeshControlPlane
resource.
spec: addons: jaeger: name: jaeger install: storage: type: Memory # or Elasticsearch for production mode memory: maxTraces: 100000 elasticsearch: # the following values only apply if storage:type:=Elasticsearch storage: # specific storageclass configuration for the Jaeger Elasticsearch (optional) size: "100G" storageClassName: "storageclass" nodeCount: 3 redundancyPolicy: SingleRedundancy runtime: components: tracing.jaeger: {} # general Jaeger specific runtime configuration (optional) tracing.jaeger.elasticsearch: #runtime configuration for Jaeger Elasticsearch deployment (optional) container: resources: requests: memory: "1Gi" cpu: "500m" limits: memory: "1Gi"
The Jaeger installation can be customized with the install
field. Container configuration, such as resource limits is configured in spec.runtime.components.jaeger
related fields. If a Jaeger resource matching the value of spec.addons.jaeger.name
exists, the Service Mesh control plane will be configured to use the existing installation. Use an existing Jaeger resource to fully customize your Jaeger installation.
1.11.4.4.3.4. Visualization
Kiali and Grafana are configured under the addons
section of the ServiceMeshControlPlane
resource.
spec: addons: grafana: enabled: true install: {} # customize install kiali: enabled: true name: kiali install: {} # customize install
The Grafana and Kiali installations can be customized through their respective install
fields. Container customization, such as resource limits, is configured in spec.runtime.components.kiali
and spec.runtime.components.grafana
. If an existing Kiali resource matching the value of name exists, the Service Mesh control plane configures the Kiali resource for use with the control plane. Some fields in the Kiali resource are overridden, such as the accessible_namespaces
list, as well as the endpoints for Grafana, Prometheus, and tracing. Use an existing resource to fully customize your Kiali installation.
1.11.4.4.3.5. Resource utilization and scheduling
Resources are configured under spec.runtime.<component>
. The following component names are supported.
Component | Description | Versions supported |
---|---|---|
security | Citadel container | v1.0/1.1 |
galley | Galley container | v1.0/1.1 |
pilot | Pilot/Istiod container | v1.0/1.1/2.0 |
mixer | istio-telemetry and istio-policy containers | v1.0/1.1 |
| istio-policy container | v2.0 |
| istio-telemetry container | v2.0 |
| oauth-proxy container used with various addons | v1.0/1.1/2.0 |
| sidecar injector webhook container | v1.0/1.1 |
| general Jaeger container - not all settings may be applied. Complete customization of Jaeger installation is supported by specifying an existing Jaeger resource in the Service Mesh control plane configuration. | v1.0/1.1/2.0 |
| settings specific to Jaeger agent | v1.0/1.1/2.0 |
| settings specific to Jaeger allInOne | v1.0/1.1/2.0 |
| settings specific to Jaeger collector | v1.0/1.1/2.0 |
| settings specific to Jaeger elasticsearch deployment | v1.0/1.1/2.0 |
| settings specific to Jaeger query | v1.0/1.1/2.0 |
prometheus | prometheus container | v1.0/1.1/2.0 |
kiali | Kiali container - complete customization of Kiali installation is supported by specifying an existing Kiali resource in the Service Mesh control plane configuration. | v1.0/1.1/2.0 |
grafana | Grafana container | v1.0/1.1/2.0 |
3scale | 3scale container | v1.0/1.1/2.0 |
| WASM extensions cacher container | v2.0 - tech preview |
Some components support resource limiting and scheduling. For more information, see Performance and scalability.
1.11.4.4.4. Next steps for migrating your applications and workloads
Move the application workload to the new mesh and remove the old instances to complete your upgrade.
1.11.5. Upgrading the data plane
Your data plane will still function after you have upgraded the control plane. But in order to apply updates to the Envoy proxy and any changes to the proxy configuration, you must restart your application pods and workloads.
1.11.5.1. Updating your applications and workloads
To complete the migration, restart all of the application pods in the mesh to upgrade the Envoy sidecar proxies and their configuration.
To perform a rolling update of a deployment use the following command:
$ oc rollout restart <deployment>
You must perform a rolling update for all applications that make up the mesh.
1.12. Managing users and profiles
1.12.1. Creating the Red Hat OpenShift Service Mesh members
ServiceMeshMember
resources provide a way for Red Hat OpenShift Service Mesh administrators to delegate permissions to add projects to a service mesh, even when the respective users don’t have direct access to the service mesh project or member roll. While project administrators are automatically given permission to create the ServiceMeshMember
resource in their project, they cannot point it to any ServiceMeshControlPlane
until the service mesh administrator explicitly grants access to the service mesh. Administrators can grant users permissions to access the mesh by granting them the mesh-user
user role. In this example, istio-system
is the name of the Service Mesh control plane project.
$ oc policy add-role-to-user -n istio-system --role-namespace istio-system mesh-user <user_name>
Administrators can modify the mesh-user
role binding in the Service Mesh control plane project to specify the users and groups that are granted access. The ServiceMeshMember
adds the project to the ServiceMeshMemberRoll
within the Service Mesh control plane project that it references.
apiVersion: maistra.io/v1 kind: ServiceMeshMember metadata: name: default spec: controlPlaneRef: namespace: istio-system name: basic
The mesh-users
role binding is created automatically after the administrator creates the ServiceMeshControlPlane
resource. An administrator can use the following command to add a role to a user.
$ oc policy add-role-to-user
The administrator can also create the mesh-user
role binding before the administrator creates the ServiceMeshControlPlane
resource. For example, the administrator can create it in the same oc apply
operation as the ServiceMeshControlPlane
resource.
This example adds a role binding for alice
:
apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: namespace: istio-system name: mesh-users roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: mesh-user subjects: - apiGroup: rbac.authorization.k8s.io kind: User name: alice
1.12.2. Creating Service Mesh control plane profiles
You can create reusable configurations with ServiceMeshControlPlane
profiles. Individual users can extend the profiles they create with their own configurations. Profiles can also inherit configuration information from other profiles. For example, you can create an accounting control plane for the accounting team and a marketing control plane for the marketing team. If you create a development template and a production template, members of the marketing team and the accounting team can extend the development and production profiles with team-specific customization.
When you configure Service Mesh control plane profiles, which follow the same syntax as the ServiceMeshControlPlane
, users inherit settings in a hierarchical fashion. The Operator is delivered with a default
profile with default settings for Red Hat OpenShift Service Mesh.
1.12.2.1. Creating the ConfigMap
To add custom profiles, you must create a ConfigMap
named smcp-templates
in the openshift-operators
project. The Operator container automatically mounts the ConfigMap
.
Prerequisites
- An installed, verified Service Mesh Operator.
-
An account with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role. - Location of the Operator deployment.
-
Access to the OpenShift Container Platform Command-line Interface (CLI) also known as
oc
.
Procedure
-
Log in to the OpenShift Container Platform CLI as a
cluster-admin
. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role. From the CLI, run this command to create the ConfigMap named
smcp-templates
in theopenshift-operators
project and replace<profiles-directory>
with the location of theServiceMeshControlPlane
files on your local disk:$ oc create configmap --from-file=<profiles-directory> smcp-templates -n openshift-operators
You can use the
profiles
parameter in theServiceMeshControlPlane
to specify one or more templates.apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic spec: profiles: - default
1.12.2.2. Setting the correct network policy
Service Mesh creates network policies in the Service Mesh control plane and member namespaces to allow traffic between them. Before you deploy, consider the following conditions to ensure the services in your service mesh that were previously exposed through an OpenShift Container Platform route.
- Traffic into the service mesh must always go through the ingress-gateway for Istio to work properly.
- Deploy services external to the service mesh in separate namespaces that are not in any service mesh.
-
Non-mesh services that need to be deployed within a service mesh enlisted namespace should label their deployments
maistra.io/expose-route: "true"
, which ensures OpenShift Container Platform routes to these services still work.
1.13. Security
If your service mesh application is constructed with a complex array of microservices, you can use Red Hat OpenShift Service Mesh to customize the security of the communication between those services. The infrastructure of OpenShift Container Platform along with the traffic management features of Service Mesh help you manage the complexity of your applications and secure microservices.
Before you begin
If you have a project, add your project to the ServiceMeshMemberRoll
resource.
If you don’t have a project, install the Bookinfo sample application and add it to the ServiceMeshMemberRoll
resource. The sample application helps illustrate security concepts.
1.13.1. About mutual Transport Layer Security (mTLS)
Mutual Transport Layer Security (mTLS) is a protocol that enables two parties to authenticate each other. It is the default mode of authentication in some protocols (IKE, SSH) and optional in others (TLS). You can use mTLS without changes to the application or service code. The TLS is handled entirely by the service mesh infrastructure and between the two sidecar proxies.
By default, mTLS in Red Hat OpenShift Service Mesh is enabled and set to permissive mode, where the sidecars in Service Mesh accept both plain-text traffic and connections that are encrypted using mTLS. If a service in your mesh is communicating with a service outside the mesh, strict mTLS could break communication between those services. Use permissive mode while you migrate your workloads to Service Mesh. Then, you can enable strict mTLS across your mesh, namespace, or application.
Enabling mTLS across your mesh at the Service Mesh control plane level secures all the traffic in your service mesh without rewriting your applications and workloads. You can secure namespaces in your mesh at the data plane level in the ServiceMeshControlPlane
resource. To customize traffic encryption connections, configure namespaces at the application level with PeerAuthentication
and DestinationRule
resources.
1.13.1.1. Enabling strict mTLS across the service mesh
If your workloads do not communicate with outside services, you can quickly enable mTLS across your mesh without communication interruptions. You can enable it by setting spec.security.dataPlane.mtls
to true
in the ServiceMeshControlPlane
resource. The Operator creates the required resources.
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: version: v2.2 security: dataPlane: mtls: true
You can also enable mTLS by using the OpenShift Container Platform web console.
Procedure
- Log in to the web console.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
- Click Operators → Installed Operators.
- Click Service Mesh Control Plane under Provided APIs.
-
Click the name of your
ServiceMeshControlPlane
resource, for example,basic
. - On the Details page, click the toggle in the Security section for Data Plane Security.
1.13.1.1.1. Configuring sidecars for incoming connections for specific services
You can also configure mTLS for individual services by creating a policy.
Procedure
Create a YAML file using the following example.
PeerAuthentication Policy example policy.yaml
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: <namespace> spec: mtls: mode: STRICT
-
Replace
<namespace>
with the namespace where the service is located.
-
Replace
Run the following command to create the resource in the namespace where the service is located. It must match the
namespace
field in the Policy resource you just created.$ oc create -n <namespace> -f <policy.yaml>
If you are not using automatic mTLS and you are setting PeerAuthentication
to STRICT, you must create a DestinationRule
resource for your service.
1.13.1.1.2. Configuring sidecars for outgoing connections
Create a destination rule to configure Service Mesh to use mTLS when sending requests to other services in the mesh.
Procedure
Create a YAML file using the following example.
DestinationRule example destination-rule.yaml
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: default namespace: <namespace> spec: host: "*.<namespace>.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL
-
Replace
<namespace>
with the namespace where the service is located.
-
Replace
Run the following command to create the resource in the namespace where the service is located. It must match the
namespace
field in theDestinationRule
resource you just created.$ oc create -n <namespace> -f <destination-rule.yaml>
1.13.1.1.3. Setting the minimum and maximum protocol versions
If your environment has specific requirements for encrypted traffic in your service mesh, you can control the cryptographic functions that are allowed by setting the spec.security.controlPlane.tls.minProtocolVersion
or spec.security.controlPlane.tls.maxProtocolVersion
in your ServiceMeshControlPlane
resource. Those values, configured in your Service Mesh control plane resource, define the minimum and maximum TLS version used by mesh components when communicating securely over TLS.
The default is TLS_AUTO
and does not specify a version of TLS.
Value | Description |
---|---|
| default |
| TLS version 1.0 |
| TLS version 1.1 |
| TLS version 1.2 |
| TLS version 1.3 |
Procedure
- Log in to the web console.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
- Click Operators → Installed Operators.
- Click Service Mesh Control Plane under Provided APIs.
-
Click the name of your
ServiceMeshControlPlane
resource, for example,basic
. - Click the YAML tab.
Insert the following code snippet in the YAML editor. Replace the value in the
minProtocolVersion
with the TLS version value. In this example, the minimum TLS version is set toTLSv1_2
.ServiceMeshControlPlane snippet
kind: ServiceMeshControlPlane spec: security: controlPlane: tls: minProtocolVersion: TLSv1_2
- Click Save.
- Click Refresh to verify that the changes updated correctly.
1.13.1.2. Validating encryption with Kiali
The Kiali console offers several ways to validate whether or not your applications, services, and workloads have mTLS encryption enabled.
Figure 1.5. Masthead icon mesh-wide mTLS enabled
At the right side of the masthead, Kiali shows a lock icon when the mesh has strictly enabled mTLS for the whole service mesh. It means that all communications in the mesh use mTLS.
Figure 1.6. Masthead icon mesh-wide mTLS partially enabled
Kiali displays a hollow lock icon when either the mesh is configured in PERMISSIVE
mode or there is a error in the mesh-wide mTLS configuration.
Figure 1.7. Security badge
The Graph page has the option to display a Security badge on the graph edges to indicate that mTLS is enabled. To enable security badges on the graph, from the Display menu, under Show Badges, select the Security checkbox. When an edge shows a lock icon, it means at least one request with mTLS enabled is present. In case there are both mTLS and non-mTLS requests, the side-panel will show the percentage of requests that use mTLS.
The Applications Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present.
The Workloads Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present.
The Services Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present. Also note that Kiali displays a lock icon in the Network section next to ports that are configured for mTLS.
1.13.2. Configuring Role Based Access Control (RBAC)
Role-based access control (RBAC) objects determine whether a user or service is allowed to perform a given action within a project. You can define mesh-, namespace-, and workload-wide access control for your workloads in the mesh.
To configure RBAC, create an AuthorizationPolicy
resource in the namespace for which you are configuring access. If you are configuring mesh-wide access, use the project where you installed the Service Mesh control plane, for example istio-system
.
For example, with RBAC, you can create policies that:
- Configure intra-project communication.
- Allow or deny full access to all workloads in the default namespace.
- Allow or deny ingress gateway access.
- Require a token for access.
An authorization policy includes a selector, an action, and a list of rules:
-
The
selector
field specifies the target of the policy. -
The
action
field specifies whether to allow or deny the request. The
rules
field specifies when to trigger the action.-
The
from
field specifies constraints on the request origin. -
The
to
field specifies constraints on request target and parameters. -
The
when
field specifies additional conditions that to apply the rule.
-
The
Procedure
Create your
AuthorizationPolicy
resource. The following example shows a resource that updates the ingress-policyAuthorizationPolicy
to deny an IP address from accessing the ingress gateway.apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: ingress-policy namespace: istio-system spec: selector: matchLabels: app: istio-ingressgateway action: DENY rules: - from: - source: ipBlocks: ["1.2.3.4"]
Run the following command after you write your resource to create your resource in your namespace. The namespace must match your
metadata.namespace
field in yourAuthorizationPolicy
resource.$ oc create -n istio-system -f <filename>
Next steps
Consider the following examples for other common configurations.
1.13.2.1. Configure intra-project communication
You can use AuthorizationPolicy
to configure your Service Mesh control plane to allow or deny the traffic communicating with your mesh or services in your mesh.
1.13.2.1.1. Restrict access to services outside a namespace
You can deny requests from any source that is not in the bookinfo
namespace with the following AuthorizationPolicy
resource example.
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: httpbin-deny namespace: bookinfo spec: selector: matchLabels: app: httpbin version: v1 action: DENY rules: - from: - source: notNamespaces: ["bookinfo"]
1.13.2.1.2. Creating allow-all and default deny-all authorization policies
The following example shows an allow-all authorization policy that allows full access to all workloads in the bookinfo
namespace.
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: allow-all namespace: bookinfo spec: action: ALLOW rules: - {}
The following example shows a policy that denies any access to all workloads in the bookinfo
namespace.
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: deny-all namespace: bookinfo spec: {}
1.13.2.2. Allow or deny access to the ingress gateway
You can set an authorization policy to add allow or deny lists based on IP addresses.
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: ingress-policy namespace: istio-system spec: selector: matchLabels: app: istio-ingressgateway action: ALLOW rules: - from: - source: ipBlocks: ["1.2.3.4", "5.6.7.0/24"]
1.13.2.3. Restrict access with JSON Web Token
You can restrict what can access your mesh with a JSON Web Token (JWT). After authentication, a user or service can access routes, services that are associated with that token.
Create a RequestAuthentication
resource, which defines the authentication methods that are supported by a workload. The following example accepts a JWT issued by http://localhost:8080/auth/realms/master
.
apiVersion: "security.istio.io/v1beta1" kind: "RequestAuthentication" metadata: name: "jwt-example" namespace: bookinfo spec: selector: matchLabels: app: httpbin jwtRules: - issuer: "http://localhost:8080/auth/realms/master" jwksUri: "http://keycloak.default.svc:8080/auth/realms/master/protocol/openid-connect/certs"
Then, create an AuthorizationPolicy
resource in the same namespace to work with RequestAuthentication
resource you created. The following example requires a JWT to be present in the Authorization
header when sending a request to httpbin
workloads.
apiVersion: "security.istio.io/v1beta1" kind: "AuthorizationPolicy" metadata: name: "frontend-ingress" namespace: bookinfo spec: selector: matchLabels: app: httpbin action: DENY rules: - from: - source: notRequestPrincipals: ["*"]
1.13.3. Configuring cipher suites and ECDH curves
Cipher suites and Elliptic-curve Diffie–Hellman (ECDH curves) can help you secure your service mesh. You can define a comma separated list of cipher suites using spec.security.controlplane.tls.cipherSuites
and ECDH curves using spec.security.controlplane.tls.ecdhCurves
in your ServiceMeshControlPlane
resource. If either of these attributes are empty, then the default values are used.
The cipherSuites
setting is effective if your service mesh uses TLS 1.2 or earlier. It has no effect when negotiating with TLS 1.3.
Set your cipher suites in the comma separated list in order of priority. For example, ecdhCurves: CurveP256, CurveP384
sets CurveP256
as a higher priority than CurveP384
.
You must include either TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
or TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
when you configure the cipher suite. HTTP/2 support requires at least one of these cipher suites.
The supported cipher suites are:
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
- TLS_RSA_WITH_AES_128_GCM_SHA256
- TLS_RSA_WITH_AES_256_GCM_SHA384
- TLS_RSA_WITH_AES_128_CBC_SHA256
- TLS_RSA_WITH_AES_128_CBC_SHA
- TLS_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_RSA_WITH_3DES_EDE_CBC_SHA
The supported ECDH Curves are:
- CurveP256
- CurveP384
- CurveP521
- X25519
1.13.4. Adding an external certificate authority key and certificate
By default, Red Hat OpenShift Service Mesh generates a self-signed root certificate and key and uses them to sign the workload certificates. You can also use the user-defined certificate and key to sign workload certificates with user-defined root certificate. This task demonstrates an example to plug certificates and key into Service Mesh.
Prerequisites
- Install Red Hat OpenShift Service Mesh with mutual TLS enabled to configure certificates.
- This example uses the certificates from the Maistra repository. For production, use your own certificates from your certificate authority.
- Deploy the Bookinfo sample application to verify the results with these instructions.
- OpenSSL is required to verify certificates.
1.13.4.1. Adding an existing certificate and key
To use an existing signing (CA) certificate and key, you must create a chain of trust file that includes the CA certificate, key, and root certificate. You must use the following exact file names for each of the corresponding certificates. The CA certificate is named ca-cert.pem
, the key is ca-key.pem
, and the root certificate, which signs ca-cert.pem
, is named root-cert.pem
. If your workload uses intermediate certificates, you must specify them in a cert-chain.pem
file.
-
Save the example certificates from the Maistra repository locally and replace
<path>
with the path to your certificates. Create a secret named
cacert
that includes the input filesca-cert.pem
,ca-key.pem
,root-cert.pem
andcert-chain.pem
.$ oc create secret generic cacerts -n istio-system --from-file=<path>/ca-cert.pem \ --from-file=<path>/ca-key.pem --from-file=<path>/root-cert.pem \ --from-file=<path>/cert-chain.pem
In the
ServiceMeshControlPlane
resource setspec.security.dataPlane.mtls true
totrue
and configure thecertificateAuthority
field as shown in the following example. The defaultrootCADir
is/etc/cacerts
. You do not need to set theprivateKey
if the key and certs are mounted in the default location. Service Mesh reads the certificates and key from the secret-mount files.apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: security: dataPlane: mtls: true certificateAuthority: type: Istiod istiod: type: PrivateKey privateKey: rootCADir: /etc/cacerts
After creating/changing/deleting the
cacert
secret, the Service Mesh control planeistiod
andgateway
pods must be restarted so the changes go into effect. Use the following command to restart the pods:$ oc -n istio-system delete pods -l 'app in (istiod,istio-ingressgateway, istio-egressgateway)'
The Operator will automatically recreate the pods after they have been deleted.
Restart the bookinfo application pods so that the sidecar proxies pick up the secret changes. Use the following command to restart the pods:
$ oc -n bookinfo delete pods --all
You should see output similar to the following:
pod "details-v1-6cd699df8c-j54nh" deleted pod "productpage-v1-5ddcb4b84f-mtmf2" deleted pod "ratings-v1-bdbcc68bc-kmng4" deleted pod "reviews-v1-754ddd7b6f-lqhsv" deleted pod "reviews-v2-675679877f-q67r2" deleted pod "reviews-v3-79d7549c7-c2gjs" deleted
Verify that the pods were created and are ready with the following command:
$ oc get pods -n bookinfo
1.13.4.2. Verifying your certificates
Use the Bookinfo sample application to verify that the workload certificates are signed by the certificates that were plugged into the CA. This requires you have openssl
installed on your machine
To extract certificates from bookinfo workloads use the following command:
$ sleep 60 $ oc -n bookinfo exec "$(oc -n bookinfo get pod -l app=productpage -o jsonpath={.items..metadata.name})" -c istio-proxy -- openssl s_client -showcerts -connect details:9080 > bookinfo-proxy-cert.txt $ sed -n '/-----BEGIN CERTIFICATE-----/{:start /-----END CERTIFICATE-----/!{N;b start};/.*/p}' bookinfo-proxy-cert.txt > certs.pem $ awk 'BEGIN {counter=0;} /BEGIN CERT/{counter++} { print > "proxy-cert-" counter ".pem"}' < certs.pem
After running the command, you should have three files in your working directory:
proxy-cert-1.pem
,proxy-cert-2.pem
andproxy-cert-3.pem
.Verify that the root certificate is the same as the one specified by the administrator. Replace
<path>
with the path to your certificates.$ openssl x509 -in <path>/root-cert.pem -text -noout > /tmp/root-cert.crt.txt
Run the following syntax at the terminal window.
$ openssl x509 -in ./proxy-cert-3.pem -text -noout > /tmp/pod-root-cert.crt.txt
Compare the certificates by running the following syntax at the terminal window.
$ diff -s /tmp/root-cert.crt.txt /tmp/pod-root-cert.crt.txt
You should see the following result:
Files /tmp/root-cert.crt.txt and /tmp/pod-root-cert.crt.txt are identical
Verify that the CA certificate is the same as the one specified by the administrator. Replace
<path>
with the path to your certificates.$ openssl x509 -in <path>/ca-cert.pem -text -noout > /tmp/ca-cert.crt.txt
Run the following syntax at the terminal window.
$ openssl x509 -in ./proxy-cert-2.pem -text -noout > /tmp/pod-cert-chain-ca.crt.txt
Compare the certificates by running the following syntax at the terminal window.
$ diff -s /tmp/ca-cert.crt.txt /tmp/pod-cert-chain-ca.crt.txt
You should see the following result:
Files /tmp/ca-cert.crt.txt and /tmp/pod-cert-chain-ca.crt.txt are identical.
Verify the certificate chain from the root certificate to the workload certificate. Replace
<path>
with the path to your certificates.$ openssl verify -CAfile <(cat <path>/ca-cert.pem <path>/root-cert.pem) ./proxy-cert-1.pem
You should see the following result:
./proxy-cert-1.pem: OK
1.13.4.3. Removing the certificates
To remove the certificates you added, follow these steps.
Remove the secret
cacerts
. In this example,istio-system
is the name of the Service Mesh control plane project.$ oc delete secret cacerts -n istio-system
Redeploy Service Mesh with a self-signed root certificate in the
ServiceMeshControlPlane
resource.apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: security: dataPlane: mtls: true
1.14. Managing traffic in your service mesh
Red Hat OpenShift Service Mesh lets you control the flow of traffic and API calls between services. Some services in your service mesh may need to communicate within the mesh and others may need to be hidden. You can manage the traffic to hide specific backend services, expose services, create testing or versioning deployments, or add a security layer on a set of services.
1.14.1. Using gateways
You can use a gateway to manage inbound and outbound traffic for your mesh to specify which traffic you want to enter or leave the mesh. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads.
Unlike other mechanisms for controlling traffic entering your systems, such as the Kubernetes Ingress APIs, Red Hat OpenShift Service Mesh gateways allow you to use the full power and flexibility of traffic routing. The Red Hat OpenShift Service Mesh gateway resource can layer 4-6 load balancing properties, such as ports, to expose and configure Red Hat OpenShift Service Mesh TLS settings. Instead of adding application-layer traffic routing (L7) to the same API resource, you can bind a regular Red Hat OpenShift Service Mesh virtual service to the gateway and manage gateway traffic like any other data plane traffic in a service mesh.
Gateways are primarily used to manage ingress traffic, but you can also configure egress gateways. An egress gateway lets you configure a dedicated exit node for the traffic leaving the mesh. This enables you to limit which services have access to external networks, which adds security control to your service mesh. You can also use a gateway to configure a purely internal proxy.
Gateway example
A gateway resource describes a load balancer operating at the edge of the mesh receiving incoming or outgoing HTTP/TCP connections. The specification describes a set of ports that should be exposed, the type of protocol to use, SNI configuration for the load balancer, and so on.
The following example shows a sample gateway configuration for external HTTPS ingress traffic:
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: ext-host-gwy spec: selector: istio: ingressgateway # use istio default controller servers: - port: number: 443 name: https protocol: HTTPS hosts: - ext-host.example.com tls: mode: SIMPLE serverCertificate: /tmp/tls.crt privateKey: /tmp/tls.key
This gateway configuration lets HTTPS traffic from ext-host.example.com
into the mesh on port 443, but doesn’t specify any routing for the traffic.
To specify routing and for the gateway to work as intended, you must also bind the gateway to a virtual service. You do this using the virtual service’s gateways field, as shown in the following example:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: virtual-svc spec: hosts: - ext-host.example.com gateways: - ext-host-gwy
You can then configure the virtual service with routing rules for the external traffic.
1.14.1.1. Managing ingress traffic
In Red Hat OpenShift Service Mesh, the Ingress Gateway enables features such as monitoring, security, and route rules to apply to traffic that enters the cluster. Use a Service Mesh gateway to expose a service outside of the service mesh.
1.14.1.1.1. Determining the ingress IP and ports
Ingress configuration differs depending on if your environment supports an external load balancer. An external load balancer is set in the ingress IP and ports for the cluster. To determine if your cluster’s IP and ports are configured for external load balancers, run the following command. In this example, istio-system
is the name of the Service Mesh control plane project.
$ oc get svc istio-ingressgateway -n istio-system
That command returns the NAME
, TYPE
, CLUSTER-IP
, EXTERNAL-IP
, PORT(S)
, and AGE
of each item in your namespace.
If the EXTERNAL-IP
value is set, your environment has an external load balancer that you can use for the ingress gateway.
If the EXTERNAL-IP
value is <none>
, or perpetually <pending>
, your environment does not provide an external load balancer for the ingress gateway. You can access the gateway using the service’s node port.
1.14.1.1.1.1. Determining ingress ports with a load balancer
Follow these instructions if your environment has an external load balancer.
Procedure
Run the following command to set the ingress IP and ports. This command sets a variable in your terminal.
$ export INGRESS_HOST=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Run the following command to set the ingress port.
$ export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
Run the following command to set the secure ingress port.
$ export SECURE_INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
Run the following command to set the TCP ingress port.
$ export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].port}')
In some environments, the load balancer may be exposed using a hostname instead of an IP address. For that case, the ingress gateway’s EXTERNAL-IP
value is not an IP address. Instead, it’s a hostname, and the previous command fails to set the INGRESS_HOST
environment variable.
In that case, use the following command to correct the INGRESS_HOST
value:
$ export INGRESS_HOST=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
1.14.1.1.1.2. Determining ingress ports without a load balancer
If your environment does not have an external load balancer, determine the ingress ports and use a node port instead.
Procedure
Set the ingress ports.
$ export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
Run the following command to set the secure ingress port.
$ export SECURE_INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')
Run the following command to set the TCP ingress port.
$ export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].nodePort}')
1.14.1.2. Configuring an ingress gateway
An ingress gateway is a load balancer operating at the edge of the mesh that receives incoming HTTP/TCP connections. It configures exposed ports and protocols but does not include any traffic routing configuration. Traffic routing for ingress traffic is instead configured with routing rules, the same way as for internal service requests.
The following steps show how to create a gateway and configure a VirtualService
to expose a service in the Bookinfo sample application to outside traffic for paths /productpage
and /login
.
Procedure
Create a gateway to accept traffic.
Create a YAML file, and copy the following YAML into it.
Gateway example gateway.yaml
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: bookinfo-gateway spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "*"
Apply the YAML file.
$ oc apply -f gateway.yaml
Create a
VirtualService
object to rewrite the host header.Create a YAML file, and copy the following YAML into it.
Virtual service example
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: bookinfo spec: hosts: - "*" gateways: - bookinfo-gateway http: - match: - uri: exact: /productpage - uri: prefix: /static - uri: exact: /login - uri: exact: /logout - uri: prefix: /api/v1/products route: - destination: host: productpage port: number: 9080
Apply the YAML file.
$ oc apply -f vs.yaml
Test that the gateway and VirtualService have been set correctly.
Set the Gateway URL.
export GATEWAY_URL=$(oc -n istio-system get route istio-ingressgateway -o jsonpath='{.spec.host}')
Set the port number. In this example,
istio-system
is the name of the Service Mesh control plane project.export TARGET_PORT=$(oc -n istio-system get route istio-ingressgateway -o jsonpath='{.spec.port.targetPort}')
Test a page that has been explicitly exposed.
curl -s -I "$GATEWAY_URL/productpage"
The expected result is
200
.
1.14.2. Understanding automatic routes
OpenShift routes for gateways are automatically managed in Service Mesh. Every time an Istio Gateway is created, updated or deleted inside the service mesh, an OpenShift route is created, updated or deleted.
1.14.2.1. Routes with subdomains
Red Hat OpenShift Service Mesh creates the route with the subdomain, but OpenShift Container Platform must be configured to enable it. Subdomains, for example *.domain.com
, are supported, but not by default. Configure an OpenShift Container Platform wildcard policy before configuring a wildcard host gateway.
For more information, see Using wildcard routes.
1.14.2.2. Creating subdomain routes
The following example creates a gateway in the Bookinfo sample application, which creates subdomain routes.
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: gateway1 spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - www.bookinfo.com - bookinfo.example.com
The Gateway
resource creates the following OpenShift routes. You can check that the routes are created by using the following command. In this example, istio-system
is the name of the Service Mesh control plane project.
$ oc -n istio-system get routes
Expected output
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD gateway1-lvlfn bookinfo.example.com istio-ingressgateway <all> None gateway1-scqhv www.bookinfo.com istio-ingressgateway <all> None
If you delete the gateway, Red Hat OpenShift Service Mesh deletes the routes. However, routes you have manually created are never modified by Red Hat OpenShift Service Mesh.
1.14.2.3. Route labels and annotations
Sometimes specific labels or annotations are needed in an OpenShift route. For example, some advanced features in OpenShift routes are managed using special annotations. See "Route-specific annotations" in the following "Additional resources" section.
For this and other use cases, Red Hat OpenShift Service Mesh will copy all labels and annotations present in the Istio gateway resource (with the exception of annotations starting with kubectl.kubernetes.io
) into the managed OpenShift route resource.
If you need specific labels or annotations in the OpenShift routes created by Service Mesh, create them in the Istio gateway resource and they will be copied into the OpenShift route resources managed by the Service Mesh.
1.14.2.4. Disabling automatic route creation
By default, the ServiceMeshControlPlane
resource automatically synchronizes the Istio gateway resources with OpenShift routes. Disabling the automatic route creation allows you more flexibility to control routes if you have a special case or prefer to control routes manually.
1.14.2.4.1. Disabling automatic route creation for specific cases
If you want to disable the automatic management of OpenShift routes for a specific Istio gateway, you must add the annotation maistra.io/manageRoute: false
to the gateway metadata definition. Red Hat OpenShift Service Mesh will ignore Istio gateways with this annotation, while keeping the automatic management of the other Istio gateways.
1.14.2.4.2. Disabling automatic route creation for all cases
You can disable the automatic management of OpenShift routes for all gateways in your mesh.
Disable integration between Istio gateways and OpenShift routes by setting the ServiceMeshControlPlane
field gateways.openshiftRoute.enabled
to false
. For example, see the following resource snippet.
apiVersion: maistra.io/v1alpha1 kind: metadata: namespace: istio-system spec: gateways: openshiftRoute: enabled: false
1.14.3. Understanding service entries
A service entry adds an entry to the service registry that Red Hat OpenShift Service Mesh maintains internally. After you add the service entry, the Envoy proxies send traffic to the service as if it is a service in your mesh. Service entries allow you to do the following:
- Manage traffic for services that run outside of the service mesh.
- Redirect and forward traffic for external destinations (such as, APIs consumed from the web) or traffic to services in legacy infrastructure.
- Define retry, timeout, and fault injection policies for external destinations.
- Run a mesh service in a Virtual Machine (VM) by adding VMs to your mesh.
Add services from a different cluster to the mesh to configure a multicluster Red Hat OpenShift Service Mesh mesh on Kubernetes.
Service entry examples
The following example is a mesh-external service entry that adds the ext-resource
external dependency to the Red Hat OpenShift Service Mesh service registry:
apiVersion: networking.istio.io/v1alpha3 kind: ServiceEntry metadata: name: svc-entry spec: hosts: - ext-svc.example.com ports: - number: 443 name: https protocol: HTTPS location: MESH_EXTERNAL resolution: DNS
Specify the external resource using the hosts
field. You can qualify it fully or use a wildcard prefixed domain name.
You can configure virtual services and destination rules to control traffic to a service entry in the same way you configure traffic for any other service in the mesh. For example, the following destination rule configures the traffic route to use mutual TLS to secure the connection to the ext-svc.example.com
external service that is configured using the service entry:
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: ext-res-dr spec: host: ext-svc.example.com trafficPolicy: tls: mode: MUTUAL clientCertificate: /etc/certs/myclientcert.pem privateKey: /etc/certs/client_private_key.pem caCertificates: /etc/certs/rootcacerts.pem
1.14.4. Using VirtualServices
You can route requests dynamically to multiple versions of a microservice through Red Hat OpenShift Service Mesh with a virtual service. With virtual services, you can:
- Address multiple application services through a single virtual service. If your mesh uses Kubernetes, for example, you can configure a virtual service to handle all services in a specific namespace. A virtual service enables you to turn a monolithic application into a service consisting of distinct microservices with a seamless consumer experience.
- Configure traffic rules in combination with gateways to control ingress and egress traffic.
1.14.4.1. Configuring VirtualServices
Requests are routed to services within a service mesh with virtual services. Each virtual service consists of a set of routing rules that are evaluated in order. Red Hat OpenShift Service Mesh matches each given request to the virtual service to a specific real destination within the mesh.
Without virtual services, Red Hat OpenShift Service Mesh distributes traffic using round-robin load balancing between all service instances. With a virtual service, you can specify traffic behavior for one or more hostnames. Routing rules in the virtual service tell Red Hat OpenShift Service Mesh how to send the traffic for the virtual service to appropriate destinations. Route destinations can be versions of the same service or entirely different services.
Procedure
Create a YAML file using the following example to route requests to different versions of the Bookinfo sample application service depending on which user connects to the application.
Example VirtualService.yaml
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2 - route: - destination: host: reviews subset: v3
Run the following command to apply
VirtualService.yaml
, whereVirtualService.yaml
is the path to the file.$ oc apply -f <VirtualService.yaml>
1.14.4.2. VirtualService configuration reference
Parameter | Description |
---|---|
spec: hosts: |
The |
spec: http: - match: |
The |
spec: http: - match: - destination: |
The |
1.14.5. Understanding destination rules
Destination rules are applied after virtual service routing rules are evaluated, so they apply to the traffic’s real destination. Virtual services route traffic to a destination. Destination rules configure what happens to traffic at that destination.
By default, Red Hat OpenShift Service Mesh uses a round-robin load balancing policy, where each service instance in the pool gets a request in turn. Red Hat OpenShift Service Mesh also supports the following models, which you can specify in destination rules for requests to a particular service or service subset.
- Random: Requests are forwarded at random to instances in the pool.
- Weighted: Requests are forwarded to instances in the pool according to a specific percentage.
- Least requests: Requests are forwarded to instances with the least number of requests.
Destination rule example
The following example destination rule configures three different subsets for the my-svc
destination service, with different load balancing policies:
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: my-destination-rule spec: host: my-svc trafficPolicy: loadBalancer: simple: RANDOM subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 trafficPolicy: loadBalancer: simple: ROUND_ROBIN - name: v3 labels: version: v3
1.14.6. Understanding network policies
Red Hat OpenShift Service Mesh automatically creates and manages a number of NetworkPolicies
resources in the Service Mesh control plane and application namespaces. This is to ensure that applications and the control plane can communicate with each other.
For example, if you have configured your OpenShift Container Platform cluster to use the SDN plug-in, Red Hat OpenShift Service Mesh creates a NetworkPolicy
resource in each member project. This enables ingress to all pods in the mesh from the other mesh members and the control plane. This also restricts ingress to only member projects. If you require ingress from non-member projects, you need to create a NetworkPolicy
to allow that traffic through. If you remove a namespace from Service Mesh, this NetworkPolicy
resource is deleted from the project.
1.14.6.1. Disabling automatic NetworkPolicy creation
If you want to disable the automatic creation and management of NetworkPolicy
resources, for example to enforce company security policies, or to allow direct access to pods in the mesh, you can do so. You can edit the ServiceMeshControlPlane
and set spec.security.manageNetworkPolicy
to false
.
When you disable spec.security.manageNetworkPolicy
Red Hat OpenShift Service Mesh will not create any NetworkPolicy
objects. The system administrator is responsible for managing the network and fixing any issues this might cause.
Prerequisites
- Red Hat OpenShift Service Mesh Operator version 2.1.1 or higher installed.
-
ServiceMeshControlPlane
resource updated to version 2.1 or higher.
Procedure
- In the OpenShift Container Platform web console, click Operators → Installed Operators.
-
Select the project where you installed the Service Mesh control plane, for example
istio-system
, from the Project menu. -
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your
ServiceMeshControlPlane
, for examplebasic-install
. -
On the Create ServiceMeshControlPlane Details page, click
YAML
to modify your configuration. Set the
ServiceMeshControlPlane
fieldspec.security.manageNetworkPolicy
tofalse
, as shown in this example.apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane spec: security: manageNetworkPolicy: false
- Click Save.
1.14.7. Configuring sidecars for traffic management
By default, Red Hat OpenShift Service Mesh configures every Envoy proxy to accept traffic on all the ports of its associated workload, and to reach every workload in the mesh when forwarding traffic. You can use a sidecar configuration to do the following:
- Fine-tune the set of ports and protocols that an Envoy proxy accepts.
- Limit the set of services that the Envoy proxy can reach.
To optimize performance of your service mesh, consider limiting Envoy proxy configurations.
In the Bookinfo sample application, configure a Sidecar so all services can reach other services running in the same namespace and control plane. This Sidecar configuration is required for using Red Hat OpenShift Service Mesh policy and telemetry features.
Procedure
Create a YAML file using the following example to specify that you want a sidecar configuration to apply to all workloads in a particular namespace. Otherwise, choose specific workloads using a
workloadSelector
.Example sidecar.yaml
apiVersion: networking.istio.io/v1alpha3 kind: Sidecar metadata: name: default namespace: bookinfo spec: egress: - hosts: - "./*" - "istio-system/*"
Run the following command to apply
sidecar.yaml
, wheresidecar.yaml
is the path to the file.$ oc apply -f sidecar.yaml
Run the following command to verify that the sidecar was created successfully.
$ oc get sidecar
1.14.8. Routing Tutorial
This guide references the Bookinfo sample application to provide examples of routing in an example application. Install the Bookinfo application to learn how these routing examples work.
1.14.8.1. Bookinfo routing tutorial
The Service Mesh Bookinfo sample application consists of four separate microservices, each with multiple versions. After installing the Bookinfo sample application, three different versions of the reviews
microservice run concurrently.
When you access the Bookinfo app /product
page in a browser and refresh several times, sometimes the book review output contains star ratings and other times it does not. Without an explicit default service version to route to, Service Mesh routes requests to all available versions one after the other.
This tutorial helps you apply rules that route all traffic to v1
(version 1) of the microservices. Later, you can apply a rule to route traffic based on the value of an HTTP request header.
Prerequisites:
- Deploy the Bookinfo sample application to work with the following examples.
1.14.8.2. Applying a virtual service
In the following procedure, the virtual service routes all traffic to v1
of each micro-service by applying virtual services that set the default version for the micro-services.
Procedure
Apply the virtual services.
$ oc apply -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/networking/virtual-service-all-v1.yaml
To verify that you applied the virtual services, display the defined routes with the following command:
$ oc get virtualservices -o yaml
That command returns a resource of
kind: VirtualService
in YAML format.
You have configured Service Mesh to route to the v1
version of the Bookinfo microservices including the reviews
service version 1.
1.14.8.3. Testing the new route configuration
Test the new configuration by refreshing the /productpage
of the Bookinfo application.
Procedure
Set the value for the
GATEWAY_URL
parameter. You can use this variable to find the URL for your Bookinfo product page later. In this example, istio-system is the name of the control plane project.export GATEWAY_URL=$(oc -n istio-system get route istio-ingressgateway -o jsonpath='{.spec.host}')
Run the following command to retrieve the URL for the product page.
echo "http://$GATEWAY_URL/productpage"
- Open the Bookinfo site in your browser.
The reviews part of the page displays with no rating stars, no matter how many times you refresh. This is because you configured Service Mesh to route all traffic for the reviews service to the version reviews:v1
and this version of the service does not access the star ratings service.
Your service mesh now routes traffic to one version of a service.
1.14.8.4. Route based on user identity
Change the route configuration so that all traffic from a specific user is routed to a specific service version. In this case, all traffic from a user named jason
will be routed to the service reviews:v2
.
Service Mesh does not have any special, built-in understanding of user identity. This example is enabled by the fact that the productpage
service adds a custom end-user
header to all outbound HTTP requests to the reviews service.
Procedure
Run the following command to enable user-based routing in the Bookinfo sample application.
$ oc apply -f https://raw.githubusercontent.com/Maistra/istio/maistra-2.2/samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
Run the following command to confirm the rule is created. This command returns all resources of
kind: VirtualService
in YAML format.$ oc get virtualservice reviews -o yaml
-
On the
/productpage
of the Bookinfo app, log in as userjason
with no password. - Refresh the browser. The star ratings appear next to each review.
-
Log in as another user (pick any name you want). Refresh the browser. Now the stars are gone. Traffic is now routed to
reviews:v1
for all users except Jason.
You have successfully configured the Bookinfo sample application to route traffic based on user identity.
1.15. Metrics, logs, and traces
Once you have added your application to the mesh, you can observe the data flow through your application. If you do not have your own application installed, you can see how observability works in Red Hat OpenShift Service Mesh by installing the Bookinfo sample application.
1.15.1. Discovering console addresses
Red Hat OpenShift Service Mesh provides the following consoles to view your service mesh data:
- Kiali console - Kiali is the management console for Red Hat OpenShift Service Mesh.
- Jaeger console - Jaeger is the management console for Red Hat OpenShift distributed tracing.
- Grafana console - Grafana provides mesh administrators with advanced query and metrics analysis and dashboards for Istio data. Optionally, Grafana can be used to analyze service mesh metrics.
- Prometheus console - Red Hat OpenShift Service Mesh uses Prometheus to store telemetry information from services.
When you install the Service Mesh control plane, it automatically generates routes for each of the installed components. Once you have the route address, you can access the Kiali, Jaeger, Prometheus, or Grafana console to view and manage your service mesh data.
Prerequisite
- The component must be enabled and installed. For example, if you did not install distributed tracing, you will not be able to access the Jaeger console.
Procedure from OpenShift console
-
Log in to the OpenShift Container Platform web console as a user with cluster-admin rights. If you use Red Hat OpenShift Dedicated, you must have an account with the
dedicated-admin
role. - Navigate to Networking → Routes.
On the Routes page, select the Service Mesh control plane project, for example
istio-system
, from the Namespace menu.The Location column displays the linked address for each route.
- If necessary, use the filter to find the component console whose route you want to access. Click the route Location to launch the console.
- Click Log In With OpenShift.
Procedure from the CLI
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role.$ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443
Switch to the Service Mesh control plane project. In this example,
istio-system
is the Service Mesh control plane project. Run the following command:$ oc project istio-system
To get the routes for the various Red Hat OpenShift Service Mesh consoles, run the folowing command:
$ oc get routes
This command returns the URLs for the Kiali, Jaeger, Prometheus, and Grafana web consoles, and any other routes in your service mesh. You should see output similar to the following:
NAME HOST/PORT SERVICES PORT TERMINATION bookinfo-gateway bookinfo-gateway-yourcompany.com istio-ingressgateway http2 grafana grafana-yourcompany.com grafana <all> reencrypt/Redirect istio-ingressgateway istio-ingress-yourcompany.com istio-ingressgateway 8080 jaeger jaeger-yourcompany.com jaeger-query <all> reencrypt kiali kiali-yourcompany.com kiali 20001 reencrypt/Redirect prometheus prometheus-yourcompany.com prometheus <all> reencrypt/Redirect
-
Copy the URL for the console you want to access from the
HOST/PORT
column into a browser to open the console. - Click Log In With OpenShift.
1.15.2. Accessing the Kiali console
You can view your application’s topology, health, and metrics in the Kiali console. If your service is experiencing problems, the Kiali console lets you view the data flow through your service. You can view insights about the mesh components at different levels, including abstract applications, services, and workloads. Kiali also provides an interactive graph view of your namespace in real time.
To access the Kiali console you must have Red Hat OpenShift Service Mesh installed, Kiali installed and configured.
The installation process creates a route to access the Kiali console.
If you know the URL for the Kiali console, you can access it directly. If you do not know the URL, use the following directions.
Procedure for administrators
- Log in to the OpenShift Container Platform web console with an administrator role.
- Click Home → Projects.
- On the Projects page, if necessary, use the filter to find the name of your project.
-
Click the name of your project, for example,
bookinfo
. - On the Project details page, in the Launcher section, click the Kiali link.
Log in to the Kiali console with the same user name and password that you use to access the OpenShift Container Platform console.
When you first log in to the Kiali Console, you see the Overview page which displays all the namespaces in your service mesh that you have permission to view.
If you are validating the console installation and namespaces have not yet been added to the mesh, there might not be any data to display other than
istio-system
.
Procedure for developers
- Log in to the OpenShift Container Platform web console with a developer role.
- Click Project.
- On the Project Details page, if necessary, use the filter to find the name of your project.
-
Click the name of your project, for example,
bookinfo
. - On the Project page, in the Launcher section, click the Kiali link.
- Click Log In With OpenShift.
1.15.3. Viewing service mesh data in the Kiali console
The Kiali Graph offers a powerful visualization of your mesh traffic. The topology combines real-time request traffic with your Istio configuration information to present immediate insight into the behavior of your service mesh, letting you quickly pinpoint issues. Multiple Graph Types let you visualize traffic as a high-level service topology, a low-level workload topology, or as an application-level topology.
There are several graphs to choose from:
- The App graph shows an aggregate workload for all applications that are labeled the same.
- The Service graph shows a node for each service in your mesh but excludes all applications and workloads from the graph. It provides a high level view and aggregates all traffic for defined services.
- The Versioned App graph shows a node for each version of an application. All versions of an application are grouped together.
- The Workload graph shows a node for each workload in your service mesh. This graph does not require you to use the application and version labels. If your application does not use version labels, use this the graph.
Graph nodes are decorated with a variety of information, pointing out various route routing options like virtual services and service entries, as well as special configuration like fault-injection and circuit breakers. It can identify mTLS issues, latency issues, error traffic and more. The Graph is highly configurable, can show traffic animation, and has powerful Find and Hide abilities.
Click the Legend button to view information about the shapes, colors, arrows, and badges displayed in the graph.
To view a summary of metrics, select any node or edge in the graph to display its metric details in the summary details panel.
1.15.3.1. Changing graph layouts in Kiali
The layout for the Kiali graph can render differently depending on your application architecture and the data to display. For example, the number of graph nodes and their interactions can determine how the Kiali graph is rendered. Because it is not possible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts.
Prerequisites
If you do not have your own application installed, install the Bookinfo sample application. Then generate traffic for the Bookinfo application by entering the following command several times.
$ curl "http://$GATEWAY_URL/productpage"
This command simulates a user visiting the
productpage
microservice of the application.
Procedure
- Launch the Kiali console.
- Click Log In With OpenShift.
- In Kiali console, click Graph to view a namespace graph.
-
From the Namespace menu, select your application namespace, for example,
bookinfo
. To choose a different graph layout, do either or both of the following:
Select different graph data groupings from the menu at the top of the graph.
- App graph
- Service graph
- Versioned App graph (default)
- Workload graph
Select a different graph layout from the Legend at the bottom of the graph.
- Layout default dagre
- Layout 1 cose-bilkent
- Layout 2 cola
1.15.3.2. Viewing logs in the Kiali console
You can view logs for your workloads in the Kiali console. The Workload Detail page includes a Logs tab which displays a unified logs view that displays both application and proxy logs. You can select how often you want the log display in Kiali to be refreshed.
To change the logging level on the logs displayed in Kiali, you change the logging configuration for the workload or the proxy.
Prerequisites
- Service Mesh installed and configured.
- Kiali installed and configured.
- The address for the Kiali console.
- Application or Bookinfo sample application added to the mesh.
Procedure
- Launch the Kiali console.
Click Log In With OpenShift.
The Kiali Overview page displays namespaces that have been added to the mesh that you have permissions to view.
- Click Workloads.
- On the Workloads page, select the project from the Namespace menu.
- If necessary, use the filter to find the workload whose logs you want to view. Click the workload Name. For example, click ratings-v1.
- On the Workload Details page, click the Logs tab to view the logs for the workload.
If you do not see any log entries, you may need to adjust either the Time Range or the Refresh interval.
1.15.3.3. Viewing metrics in the Kiali console
You can view inbound and outbound metrics for your applications, workloads, and services in the Kiali console. The Detail pages include the following tabs:
- inbound Application metrics
- outbound Application metrics
- inbound Workload metrics
- outbound Workload metrics
- inbound Service metrics
These tabs display predefined metrics dashboards, tailored to the relevant application, workload or service level. The application and workload detail views show request and response metrics such as volume, duration, size, or TCP traffic. The service detail view shows request and response metrics for inbound traffic only.
Kiali lets you customize the charts by choosing the charted dimensions. Kiali can also present metrics reported by either source or destination proxy metrics. And for troubleshooting, Kiali can overlay trace spans on the metrics.
Prerequisites
- Service Mesh installed and configured.
- Kiali installed and configured.
- The address for the Kiali console.
- (Optional) Distributed tracing installed and configured.
Procedure
- Launch the Kiali console.
Click Log In With OpenShift.
The Kiali Overview page displays namespaces that have been added to the mesh that you have permissions to view.
- Click either Applications, Workloads, or Services.
- On the Applications, Workloads, or Services page, select the project from the Namespace menu.
- If necessary, use the filter to find the application, workload, or service whose logs you want to view. Click the Name.
- On the Application Detail, Workload Details, or Service Details page, click either the Inbound Metrics or Outbound Metrics tab to view the metrics.
1.15.4. Distributed tracing
Distributed tracing is the process of tracking the performance of individual services in an application by tracing the path of the service calls in the application. Each time a user takes action in an application, a request is executed that might require many services to interact to produce a response. The path of this request is called a distributed transaction.
Red Hat OpenShift Service Mesh uses Red Hat OpenShift distributed tracing to allow developers to view call flows in a microservice application.
1.15.4.1. Connecting an existing distributed tracing instance
If you already have an existing Red Hat OpenShift distributed tracing platform instance in OpenShift Container Platform, you can configure your ServiceMeshControlPlane
resource to use that instance for distributed tracing.
Prerequisites
- Red Hat OpenShift distributed tracing instance installed and configured.
Procedure
- In the OpenShift Container Platform web console, click Operators → Installed Operators.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
-
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your
ServiceMeshControlPlane
resource, for examplebasic
. Add the name of your distributed tracing platform instance to the
ServiceMeshControlPlane
.- Click the YAML tab.
Add the name of your distributed tracing platform instance to
spec.addons.jaeger.name
in yourServiceMeshControlPlane
resource. In the following example,distr-tracing-production
is the name of the distributed tracing platform instance.Example distributed tracing configuration
spec: addons: jaeger: name: distr-tracing-production
- Click Save.
-
Click Reload to verify the
ServiceMeshControlPlane
resource was configured correctly.
1.15.4.2. Adjusting the sampling rate
A trace is an execution path between services in the service mesh. A trace is comprised of one or more spans. A span is a logical unit of work that has a name, start time, and duration. The sampling rate determines how often a trace is persisted.
The Envoy proxy sampling rate is set to sample 100% of traces in your service mesh by default. A high sampling rate consumes cluster resources and performance but is useful when debugging issues. Before you deploy Red Hat OpenShift Service Mesh in production, set the value to a smaller proportion of traces. For example, set spec.tracing.sampling
to 100
to sample 1% of traces.
Configure the Envoy proxy sampling rate as a scaled integer representing 0.01% increments.
In a basic installation, spec.tracing.sampling
is set to 10000
, which samples 100% of traces. For example:
- Setting the value to 10 samples 0.1% of traces.
- Setting the value to 500 samples 5% of traces.
The Envoy proxy sampling rate applies for applications that are available to a Service Mesh, and use the Envoy proxy. This sampling rate determines how much data the Envoy proxy collects and tracks.
The Jaeger remote sampling rate applies to applications that are external to the Service Mesh, and do not use the Envoy proxy, such as a database. This sampling rate determines how much data the distributed tracing system collects and stores. For more information, see Distributed tracing configuration options.
Procedure
- In the OpenShift Container Platform web console, click Operators → Installed Operators.
- Click the Project menu and select the project where you installed the control plane, for example istio-system.
-
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your
ServiceMeshControlPlane
resource, for examplebasic
. To adjust the sampling rate, set a different value for
spec.tracing.sampling
.- Click the YAML tab.
Set the value for
spec.tracing.sampling
in yourServiceMeshControlPlane
resource. In the following example, set it to100
.Jaeger sampling example
spec: tracing: sampling: 100
- Click Save.
-
Click Reload to verify the
ServiceMeshControlPlane
resource was configured correctly.
1.15.5. Accessing the Jaeger console
To access the Jaeger console you must have Red Hat OpenShift Service Mesh installed, Red Hat OpenShift distributed tracing platform installed and configured.
The installation process creates a route to access the Jaeger console.
If you know the URL for the Jaeger console, you can access it directly. If you do not know the URL, use the following directions.
Procedure from OpenShift console
-
Log in to the OpenShift Container Platform web console as a user with cluster-admin rights. If you use Red Hat OpenShift Dedicated, you must have an account with the
dedicated-admin
role. - Navigate to Networking → Routes.
On the Routes page, select the Service Mesh control plane project, for example
istio-system
, from the Namespace menu.The Location column displays the linked address for each route.
-
If necessary, use the filter to find the
jaeger
route. Click the route Location to launch the console. - Click Log In With OpenShift.
Procedure from Kiali console
- Launch the Kiali console.
- Click Distributed Tracing in the left navigation pane.
- Click Log In With OpenShift.
Procedure from the CLI
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. If you use Red Hat OpenShift Dedicated, you must have an account with thededicated-admin
role.$ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443
To query for details of the route using the command line, enter the following command. In this example,
istio-system
is the Service Mesh control plane namespace.$ export JAEGER_URL=$(oc get route -n istio-system jaeger -o jsonpath='{.spec.host}')
-
Launch a browser and navigate to
https://<JAEGER_URL>
, where<JAEGER_URL>
is the route that you discovered in the previous step. - Log in using the same user name and password that you use to access the OpenShift Container Platform console.
If you have added services to the service mesh and have generated traces, you can use the filters and Find Traces button to search your trace data.
If you are validating the console installation, there is no trace data to display.
For more information about configuring Jaeger, see the distributed tracing documentation.
1.15.6. Accessing the Grafana console
Grafana is an analytics tool you can use to view, query, and analyze your service mesh metrics. In this example, istio-system
is the Service Mesh control plane namespace. To access Grafana, do the following:
Procedure
- Log in to the OpenShift Container Platform web console.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
- Click Routes.
- Click the link in the Location column for the Grafana row.
- Log in to the Grafana console with your OpenShift Container Platform credentials.
1.15.7. Accessing the Prometheus console
Prometheus is a monitoring and alerting tool that you can use to collect multi-dimensional data about your microservices. In this example, istio-system
is the Service Mesh control plane namespace.
Procedure
- Log in to the OpenShift Container Platform web console.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
- Click Routes.
- Click the link in the Location column for the Prometheus row.
- Log in to the Prometheus console with your OpenShift Container Platform credentials.
1.16. Performance and scalability
The default ServiceMeshControlPlane
settings are not intended for production use; they are designed to install successfully on a default OpenShift Container Platform installation, which is a resource-limited environment. After you have verified a successful SMCP installation, you should modify the settings defined within the SMCP to suit your environment.
1.16.1. Setting limits on compute resources
By default, spec.proxy
has the settings cpu: 10m
and memory: 128M
. If you are using Pilot, spec.runtime.components.pilot
has the same default values.
The settings in the following example are based on 1,000 services and 1,000 requests per second. You can change the values for cpu
and memory
in the ServiceMeshControlPlane
.
Procedure
- In the OpenShift Container Platform web console, click Operators → Installed Operators.
- Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
-
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your
ServiceMeshControlPlane
, for examplebasic
. Add the name of your standalone Jaeger instance to the
ServiceMeshControlPlane
.- Click the YAML tab.
Set the values for
spec.proxy.runtime.container.resources.requests.cpu
andspec.proxy.runtime.container.resources.requests.memory
in yourServiceMeshControlPlane
resource.Example version 2.2 ServiceMeshControlPlane
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic namespace: istio-system spec: version: v2.2 proxy: runtime: container: resources: requests: cpu: 600m memory: 50Mi limits: {} runtime: components: pilot: container: resources: requests: cpu: 1000m memory: 1.6Gi limits: {}
- Click Save.
-
Click Reload to verify the
ServiceMeshControlPlane
resource was configured correctly.
1.16.2. Load test results
The upstream Istio community load tests mesh consists of 1000 services and 2000 sidecars with 70,000 mesh-wide requests per second. Running the tests using Istio 1.12.3, generated the following results:
- The Envoy proxy uses 0.35 vCPU and 40 MB memory per 1000 requests per second going through the proxy.
- Istiod uses 1 vCPU and 1.5 GB of memory.
- The Envoy proxy adds 2.65 ms to the 90th percentile latency.
-
The legacy
istio-telemetry
service (disabled by default in Service Mesh 2.0) uses 0.6 vCPU per 1000 mesh-wide requests per second for deployments that use Mixer. The data plane components, the Envoy proxies, handle data flowing through the system. The Service Mesh control plane component, Istiod, configures the data plane. The data plane and control plane have distinct performance concerns.
1.16.2.1. Service Mesh Control plane performance
Istiod configures sidecar proxies based on user authored configuration files and the current state of the system. In a Kubernetes environment, Custom Resource Definitions (CRDs) and deployments constitute the configuration and state of the system. The Istio configuration objects like gateways and virtual services, provide the user-authored configuration. To produce the configuration for the proxies, Istiod processes the combined configuration and system state from the Kubernetes environment and the user-authored configuration.
The Service Mesh control plane supports thousands of services, spread across thousands of pods with a similar number of user authored virtual services and other configuration objects. Istiod’s CPU and memory requirements scale with the number of configurations and possible system states. The CPU consumption scales with the following factors:
- The rate of deployment changes.
- The rate of configuration changes.
- The number of proxies connecting to Istiod.
However this part is inherently horizontally scalable.
1.16.2.2. Data plane performance
Data plane performance depends on many factors, for example:
- Number of client connections
- Target request rate
- Request size and response size
- Number of proxy worker threads
- Protocol
- CPU cores
- Number and types of proxy filters, specifically telemetry v2 related filters.
The latency, throughput, and the proxies' CPU and memory consumption are measured as a function of these factors.
1.16.2.2.1. CPU and memory consumption
Since the sidecar proxy performs additional work on the data path, it consumes CPU and memory. As of Istio 1.12.3, a proxy consumes about 0.5 vCPU per 1000 requests per second.
The memory consumption of the proxy depends on the total configuration state the proxy holds. A large number of listeners, clusters, and routes can increase memory usage.
Since the proxy normally doesn’t buffer the data passing through, request rate doesn’t affect the memory consumption.
1.16.2.2.2. Additional latency
Since Istio injects a sidecar proxy on the data path, latency is an important consideration. Istio adds an authentication filter, a telemetry filter, and a metadata exchange filter to the proxy. Every additional filter adds to the path length inside the proxy and affects latency.
The Envoy proxy collects raw telemetry data after a response is sent to the client. The time spent collecting raw telemetry for a request does not contribute to the total time taken to complete that request. However, since the worker is busy handling the request, the worker won’t start handling the next request immediately. This process adds to the queue wait time of the next request and affects average and tail latencies. The actual tail latency depends on the traffic pattern.
Inside the mesh, a request traverses the client-side proxy and then the server-side proxy. In the default configuration of Istio 1.12.3 (that is, Istio with telemetry v2), the two proxies add about 1.7 ms and 2.7 ms to the 90th and 99th percentile latency, respectively, over the baseline data plane latency.
1.17. Configuring Service Mesh for production
When you are ready to move from a basic installation to production, you must configure your control plane, tracing, and security certificates to meet production requirements.
Prerequisites
- Install and configure Red Hat OpenShift Service Mesh.
- Test your configuration in a staging environment.
1.17.1. Configuring your ServiceMeshControlPlane resource for production
If you have installed a basic ServiceMeshControlPlane
resource to test Service Mesh, you must configure it to production specification before you use Red Hat OpenShift Service Mesh in production.
You cannot change the metadata.name
field of an existing ServiceMeshControlPlane
resource. For production deployments, you must customize the default template.
Procedure
Configure the distributed tracing platform for production.
Edit the
ServiceMeshControlPlane
resource to use theproduction
deployment strategy, by settingspec.addons.jaeger.install.storage.type
toElasticsearch
and specify additional configuration options underinstall
. You can create and configure your Jaeger instance and setspec.addons.jaeger.name
to the name of the Jaeger instance.Default Jaeger parameters including Elasticsearch
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: basic spec: version: v2.2 tracing: sampling: 100 type: Jaeger addons: jaeger: name: MyJaeger install: storage: type: Elasticsearch ingress: enabled: true runtime: components: tracing.jaeger.elasticsearch: # only supports resources and image name container: resources: {}
- Configure the sampling rate for production. For more information, see the Performance and scalability section.
- Ensure your security certificates are production ready by installing security certificates from an external certificate authority. For more information, see the Security section.
Verify the results. Enter the following command to verify that the
ServiceMeshControlPlane
resource updated properly. In this example,basic
is the name of theServiceMeshControlPlane
resource.$ oc get smcp basic -o yaml
1.17.2. Additional resources
- For more information about tuning Service Mesh for performance, see Performance and scalability.
1.18. Connecting service meshes
Federation is a deployment model that lets you share services and workloads between separate meshes managed in distinct administrative domains.
1.18.1. Federation overview
Federation is a set of features that let you connect services between separate meshes, allowing the use of Service Mesh features such as authentication, authorization, and traffic management across multiple, distinct administrative domains.
Implementing a federated mesh lets you run, manage, and observe a single service mesh running across multiple OpenShift clusters. Red Hat OpenShift Service Mesh federation takes an opinionated approach to a multi-cluster implementation of Service Mesh that assumes minimal trust between meshes.
Service Mesh federation assumes that each mesh is managed individually and retains its own administrator. The default behavior is that no communication is permitted and no information is shared between meshes. The sharing of information between meshes is on an explicit opt-in basis. Nothing is shared in a federated mesh unless it has been configured for sharing. Support functions such as certificate generation, metrics and trace collection remain local in their respective meshes.
You configure the ServiceMeshControlPlane
on each service mesh to create ingress and egress gateways specifically for the federation, and to specify the trust domain for the mesh.
Federation also involves the creation of additional federation files. The following resources are used to configure the federation between two or more meshes.
- A ServiceMeshPeer resource declares the federation between a pair of service meshes.
- An ExportedServiceSet resource declares that one or more services from the mesh are available for use by a peer mesh.
- An ImportedServiceSet resource declares which services exported by a peer mesh will be imported into the mesh.
1.18.2. Federation features
Features of the Red Hat OpenShift Service Mesh federated approach to joining meshes include the following:
- Supports common root certificates for each mesh.
- Supports different root certificates for each mesh.
- Mesh administrators must manually configure certificate chains, service discovery endpoints, trust domains, etc for meshes outside of the Federated mesh.
Only export/import the services that you want to share between meshes.
- Defaults to not sharing information about deployed workloads with other meshes in the federation. A service can be exported to make it visible to other meshes and allow requests from workloads outside of its own mesh.
- A service that has been exported can be imported to another mesh, enabling workloads on that mesh to send requests to the imported service.
- Encrypts communication between meshes at all times.
- Supports configuring load balancing across workloads deployed locally and workloads that are deployed in another mesh in the federation.
When a mesh is joined to another mesh it can do the following:
- Provide trust details about itself to the federated mesh.
- Discover trust details about the federated mesh.
- Provide information to the federated mesh about its own exported services.
- Discover information about services exported by the federated mesh.
1.18.3. Federation security
Red Hat OpenShift Service Mesh federation takes an opinionated approach to a multi-cluster implementation of Service Mesh that assumes minimal trust between meshes. Data security is built in as part of the federation features.
- Each mesh is considered to be a unique tenant, with a unique administration.
- You create a unique trust domain for each mesh in the federation.
- Traffic between the federated meshes is automatically encrypted using mutual Transport Layer Security (mTLS).
- The Kiali graph only displays your mesh and services that you have imported. You cannot see the other mesh or services that have not been imported into your mesh.
1.18.4. Federation limitations
The Red Hat OpenShift Service Mesh federated approach to joining meshes has the following limitations:
- Federation of meshes is not supported on OpenShift Dedicated.
- Federation of meshes is not supported on Microsoft Azure Red Hat OpenShift (ARO).
1.18.5. Federation prerequisites
The Red Hat OpenShift Service Mesh federated approach to joining meshes has the following prerequisites:
- Two or more OpenShift Container Platform 4.6 or above clusters.
- Federation was introduced in Red Hat OpenShift Service Mesh 2.1. You must have the Red Hat OpenShift Service Mesh 2.1 Operator installed on each mesh that you want to federate.
-
You must have a version 2.1
ServiceMeshControlPlane
deployed on each mesh that you want to federate. - You must configure the load balancers supporting the services associated with the federation gateways to support raw TLS traffic. Federation traffic consists of HTTPS for discovery and raw encrypted TCP for service traffic.
-
Services that you want to expose to another mesh should be deployed before you can export and import them. However, this is not a strict requirement. You can specify service names that do not yet exist for export/import. When you deploy the services named in the
ExportedServiceSet
andImportedServiceSet
they will be automatically made available for export/import.
1.18.6. Planning your mesh federation
Before you start configuring your mesh federation, you should take some time to plan your implementation.
- How many meshes do you plan to join in a federation? You probably want to start with a limited number of meshes, perhaps two or three.
What naming convention do you plan to use for each mesh? Having a pre-defined naming convention will help with configuration and troubleshooting. The examples in this documentation use different colors for each mesh. You should decide on a naming convention that will help you determine who owns and manages each mesh, as well as the following federation resources:
- Cluster names
- Cluster network names
- Mesh names and namespaces
- Federation ingress gateways
- Federation egress gateways
Security trust domains
NoteEach mesh in the federation must have its own unique trust domain.
Which services from each mesh do you plan to export to the federated mesh? Each service can be exported individually, or you can specify labels or use wildcards.
- Do you want to use aliases for the service namespaces?
- Do you want to use aliases for the exported services?
Which exported services does each mesh plan to import? Each mesh only imports the services that it needs.
- Do you want to use aliases for the imported services?
1.18.7. Mesh federation across clusters
To connect one instance of the OpenShift Service Mesh with one running in a different cluster, the procedure is not much different as when connecting two meshes deployed in the same cluster. However, the ingress gateway of one mesh must be reachable from the other mesh. One way of ensuring this is to configure the gateway service as a LoadBalancer
service if the cluster supports this type of service.
The service must be exposed through a load balancer that operates at Layer4 of the OSI model.
1.18.7.1. Exposing the federation ingress on clusters running on bare metal
If the cluster runs on bare metal and fully supports LoadBalancer
services, the IP address found in the .status.loadBalancer.ingress.ip
field of the ingress gateway Service
object should be specified as one of the entries in the .spec.remote.addresses
field of the ServiceMeshPeer
object.
If the cluster does not support LoadBalancer
services, using a NodePort
service could be an option if the nodes are accessible from the cluster running the other mesh. In the ServiceMeshPeer
object, specify the IP addresses of the nodes in the .spec.remote.addresses
field and the service’s node ports in the .spec.remote.discoveryPort
and .spec.remote.servicePort
fields.
1.18.7.2. Exposing the federation ingress on clusters running on IBM Power and IBM Z
If the cluster runs on IBM Power or IBM Z infrastructure and fully supports LoadBalancer
services, the IP address found in the .status.loadBalancer.ingress.ip
field of the ingress gateway Service
object should be specified as one of the entries in the .spec.remote.addresses
field of the ServiceMeshPeer
object.
If the cluster does not support LoadBalancer
services, using a NodePort
service could be an option if the nodes are accessible from the cluster running the other mesh. In the ServiceMeshPeer
object, specify the IP addresses of the nodes in the .spec.remote.addresses
field and the service’s node ports in the .spec.remote.discoveryPort
and .spec.remote.servicePort
fields.
1.18.7.3. Exposing the federation ingress on Amazon Web Services (AWS)
By default, LoadBalancer services in clusters running on AWS do not support L4 load balancing. In order for Red Hat OpenShift Service Mesh federation to operate correctly, the following annotation must be added to the ingress gateway service:
service.beta.kubernetes.io/aws-load-balancer-type: nlb
The Fully Qualified Domain Name found in the .status.loadBalancer.ingress.hostname
field of the ingress gateway Service
object should be specified as one of the entries in the .spec.remote.addresses
field of the ServiceMeshPeer
object.
1.18.7.4. Exposing the federation ingress on Azure
On Microsoft Azure, merely setting the service type to LoadBalancer
suffices for mesh federation to operate correctly.
The IP address found in the .status.loadBalancer.ingress.ip
field of the ingress gateway Service
object should be specified as one of the entries in the .spec.remote.addresses
field of the ServiceMeshPeer
object.
1.18.7.5. Exposing the federation ingress on Google Cloud Platform (GCP)
On Google Cloud Platform, merely setting the service type to LoadBalancer
suffices for mesh federation to operate correctly.
The IP address found in the .status.loadBalancer.ingress.ip
field of the ingress gateway Service
object should be specified as one of the entries in the .spec.remote.addresses
field of the ServiceMeshPeer
object.
1.18.8. Federation implementation checklist
Federating services meshes involves the following activities:
❏ Configure networking between the clusters that you are going to federate.
- ❏ Configure the load balancers supporting the services associated with the federation gateways to support raw TLS traffic.
- ❏ Installing the Red Hat OpenShift Service Mesh version 2.1 or later Operator in each of your clusters.
-
❏ Deploying a version 2.1 or later
ServiceMeshControlPlane
to each of your clusters. ❏ Configuring the SMCP for federation for each mesh that you want to federate:
- ❏ Create a federation egress gateway for each mesh you are going to federate with.
- ❏ Create a federation ingress gateway for each mesh you are going to federate with.
- ❏ Configure a unique trust domain.
-
❏ Federate two or more meshes by creating a
ServiceMeshPeer
resource for each mesh pair. -
❏ Export services by creating an
ExportedServiceSet
resource to make services available from one mesh to a peer mesh. -
❏ Import services by creating an
ImportedServiceSet
resource to import services shared by a mesh peer.
1.18.9. Configuring a Service Mesh control plane for federation
Before a mesh can be federated, you must configure the ServiceMeshControlPlane
for mesh federation. Because all meshes that are members of the federation are equal, and each mesh is managed independently, you must configure the SMCP for each mesh that will participate in the federation.
In the following example, the administrator for the red-mesh
is configuring the SMCP for federation with both the green-mesh
and the blue-mesh
.
Sample SMCP for red-mesh
apiVersion: maistra.io/v2 kind: ServiceMeshControlPlane metadata: name: red-mesh namespace: red-mesh-system spec: version: v2.2 runtime: defaults: container: imagePullPolicy: Always gateways: additionalEgress: egress-green-mesh: enabled: true requestedNetworkView: - green-network routerMode: sni-dnat service: metadata: labels: federation.maistra.io/egress-for: egress-green-mesh ports: - port: 15443 name: tls - port: 8188 name: http-discovery #note HTTP here egress-blue-mesh: enabled: true requestedNetworkView: - blue-network routerMode: sni-dnat service: metadata: labels: federation.maistra.io/egress-for: egress-blue-mesh ports: - port: 15443 name: tls - port: 8188 name: http-discovery #note HTTP here additionalIngress: ingress-green-mesh: enabled: true routerMode: sni-dnat service: type: LoadBalancer metadata: labels: federation.maistra.io/ingress-for: ingress-green-mesh ports: - port: 15443 name: tls - port: 8188 name: https-discovery #note HTTPS here ingress-blue-mesh: enabled: true routerMode: sni-dnat service: type: LoadBalancer metadata: labels: federation.maistra.io/ingress-for: ingress-blue-mesh ports: - port: 15443 name: tls - port: 8188 name: https-discovery #note HTTPS here security: trust: domain: red-mesh.local
Parameter | Description | Values | Default value |
---|---|---|---|
spec: cluster: name: | Name of the cluster. You are not required to specify a cluster name, but it is helpful for troubleshooting. | String | N/A |
spec: cluster: network: | Name of the cluster network. You are not required to specify a name for the network, but it is helpful for configuration and troubleshooting. | String | N/A |
1.18.9.1. Understanding federation gateways
You use a gateway to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh.
You use ingress and egress gateways to manage traffic entering and leaving the service mesh (North-South traffic). When you create a federated mesh, you create additional ingress/egress gateways, to facilitate service discovery between federated meshes, communication between federated meshes, and to manage traffic flow between service meshes (East-West traffic).
To avoid naming conflicts between meshes, you must create separate egress and ingress gateways for each mesh. For example, red-mesh
would have separate egress gateways for traffic going to green-mesh
and blue-mesh
.
Parameter | Description | Values | Default value |
---|---|---|---|
spec: gateways: additionalEgress: <egressName>: | Define an additional egress gateway for each mesh peer in the federation. | ||
spec: gateways: additionalEgress: <egressName>: enabled: | This parameter enables or disables the federation egress. |
|
|
spec: gateways: additionalEgress: <egressName>: requestedNetworkView: | Networks associated with exported services. |
Set to the value of | |
spec: gateways: additionalEgress: <egressName>: routerMode: | The router mode to be used by the gateway. |
| |
spec: gateways: additionalEgress: <egressName>: service: metadata: labels: federation.maistra.io/egress-for: | Specify a unique label for the gateway to prevent federated traffic from flowing through the cluster’s default system gateways. | ||
spec: gateways: additionalEgress: <egressName>: service: ports: |
Used to specify the |
Port | |
spec: gateways: additionalIngress: | Define an additional ingress gateway gateway for each mesh peer in the federation. | ||
spec: gateways: additionalIgress: <ingressName>: enabled: | This parameter enables or disables the federation ingress. |
|
|
spec: gateways: additionalIngress: <ingressName>: routerMode: | The router mode to be used by the gateway. |
| |
spec: gateways: additionalIngress: <ingressName>: service: type: | The ingress gateway service must be exposed through a load balancer that operates at Layer 4 of the OSI model and is publicly available. |
| |
spec: gateways: additionalIngress: <ingressName>: service: type: |
If the cluster does not support |
| |
spec: gateways: additionalIngress: <ingressName>: service: metadata: labels: federation.maistra.io/ingress-for: | Specify a unique label for the gateway to prevent federated traffic from flowing through the cluster’s default system gateways. | ||
spec: gateways: additionalIngress: <ingressName>: service: ports: |
Used to specify the |
Port | |
spec: gateways: additionalIngress: <ingressName>: service: ports: nodePort: |
Used to specify the |
If specified, is required in addition to |
In the following example, the administrator is configuring the SMCP for federation with the green-mesh
using a NodePort
service.
Sample SMCP for NodePort
gateways: additionalIngress: ingress-green-mesh: enabled: true routerMode: sni-dnat service: type: NodePort metadata: labels: federation.maistra.io/ingress-for: ingress-green-mesh ports: - port: 15443 nodePort: 30510 name: tls - port: 8188 nodePort: 32359 name: https-discovery
1.18.9.2. Understanding federation trust domain parameters
Each mesh in the federation must have its own unique trust domain. This value is used when configuring mesh federation in the ServiceMeshPeer
resource.
kind: ServiceMeshControlPlane metadata: name: red-mesh namespace: red-mesh-system spec: security: trust: domain: red-mesh.local
Parameter | Description | Values | Default value |
---|---|---|---|
spec: security: trust: domain: | Used to specify a unique name for the trust domain for the mesh. Domains must be unique for every mesh in the federation. |
| N/A |
Procedure from the Console
Follow this procedure to edit the ServiceMeshControlPlane
with the OpenShift Container Platform web console. This example uses the red-mesh
as an example.
- Log in to the OpenShift Container Platform web console as a user with the cluster-admin role.
- Navigate to Operators → Installed Operators.
-
Click the Project menu and select the project where you installed the Service Mesh control plane. For example,
red-mesh-system
. - Click the Red Hat OpenShift Service Mesh Operator.
-
On the Istio Service Mesh Control Plane tab, click the name of your
ServiceMeshControlPlane
, for examplered-mesh
. -
On the Create ServiceMeshControlPlane Details page, click
YAML
to modify your configuration. -
Modify your
ServiceMeshControlPlane
to add federation ingress and egress gateways and to specify the trust domain. - Click Save.
Procedure from the CLI
Follow this procedure to create or edit the ServiceMeshControlPlane
with the command line. This example uses the red-mesh
as an example.
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. Enter the following command. Then, enter your username and password when prompted.$ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443
Change to the project where you installed the Service Mesh control plane, for example red-mesh-system.
$ oc project red-mesh-system
-
Edit the
ServiceMeshControlPlane
file to add federation ingress and egress gateways and to specify the trust domain. Run the following command to edit the Service Mesh control plane where
red-mesh-system
is the system namespace andred-mesh
is the name of theServiceMeshControlPlane
object:$ oc edit -n red-mesh-system smcp red-mesh
Enter the following command, where
red-mesh-system
is the system namespace, to see the status of the Service Mesh control plane installation.$ oc get smcp -n red-mesh-system
The installation has finished successfully when the READY column indicates that all components are ready.
NAME READY STATUS PROFILES VERSION AGE red-mesh 10/10 ComponentsReady ["default"] 2.1.0 4m25s
1.18.10. Joining a federated mesh
You declare the federation between two meshes by creating a ServiceMeshPeer
resource. The ServiceMeshPeer
resource defines the federation between two meshes, and you use it to configure discovery for the peer mesh, access to the peer mesh, and certificates used to validate the other mesh’s clients.
Meshes are federated on a one-to-one basis, so each pair of peers requires a pair of ServiceMeshPeer
resources specifying the federation connection to the other service mesh. For example, federating two meshes named red
and green
would require two ServiceMeshPeer
files.
-
On red-mesh-system, create a
ServiceMeshPeer
for the green mesh. -
On green-mesh-system, create a
ServiceMeshPeer
for the red mesh.
Federating three meshes named red
, blue
, and green
would require six ServiceMeshPeer
files.
-
On red-mesh-system, create a
ServiceMeshPeer
for the green mesh. -
On red-mesh-system, create a
ServiceMeshPeer
for the blue mesh. -
On green-mesh-system, create a
ServiceMeshPeer
for the red mesh. -
On green-mesh-system, create a
ServiceMeshPeer
for the blue mesh. -
On blue-mesh-system, create a
ServiceMeshPeer
for the red mesh. -
On blue-mesh-system, create a
ServiceMeshPeer
for the green mesh.
Configuration in the ServiceMeshPeer
resource includes the following:
- The address of the other mesh’s ingress gateway, which is used for discovery and service requests.
- The names of the local ingress and egress gateways that is used for interactions with the specified peer mesh.
- The client ID used by the other mesh when sending requests to this mesh.
- The trust domain used by the other mesh.
-
The name of a
ConfigMap
containing a root certificate that is used to validate client certificates in the trust domain used by the other mesh.
In the following example, the administrator for the red-mesh
is configuring federation with the green-mesh
.
Example ServiceMeshPeer resource for red-mesh
kind: ServiceMeshPeer apiVersion: federation.maistra.io/v1 metadata: name: green-mesh namespace: red-mesh-system spec: remote: addresses: - ingress-red-mesh.green-mesh-system.apps.domain.com gateways: ingress: name: ingress-green-mesh egress: name: egress-green-mesh security: trustDomain: green-mesh.local clientID: green-mesh.local/ns/green-mesh-system/sa/egress-red-mesh-service-account certificateChain: kind: ConfigMap name: green-mesh-ca-root-cert
Parameter | Description | Values |
---|---|---|
metadata: name: | Name of the peer mesh that this resource is configuring federation with. | String |
metadata: namespace: | System namespace for this mesh, that is, where the Service Mesh control plane is installed. | String |
spec: remote: addresses: | List of public addresses of the peer meshes' ingress gateways that are servicing requests from this mesh. | |
spec: remote: discoveryPort: | The port on which the addresses are handling discovery requests. | Defaults to 8188 |
spec: remote: servicePort: | The port on which the addresses are handling service requests. | Defaults to 15443 |
spec: gateways: ingress: name: |
Name of the ingress on this mesh that is servicing requests received from the peer mesh. For example, | |
spec: gateways: egress: name: |
Name of the egress on this mesh that is servicing requests sent to the peer mesh. For example, | |
spec: security: trustDomain: | The trust domain used by the peer mesh. | <peerMeshName>.local |
spec: security: clientID: | The client ID used by the peer mesh when calling into this mesh. | <peerMeshTrustDomain>/ns/<peerMeshSystem>/sa/<peerMeshEgressGatewayName>-service-account |
spec: security: certificateChain: kind: ConfigMap name: |
The kind (for example, ConfigMap) and name of a resource containing the root certificate used to validate the client and server certificate(s) presented to this mesh by the peer mesh. The key of the config map entry containing the certificate should be | kind: ConfigMap name: <peerMesh>-ca-root-cert |
1.18.10.1. Creating a ServiceMeshPeer resource
Prerequisites
- Two or more OpenShift Container Platform 4.6 or above clusters.
- The clusters must already be networked.
- The load balancers supporting the services associated with the federation gateways must be configured to support raw TLS traffic.
-
Each cluster must have a version 2.1
ServiceMeshControlPlane
configured to support federation deployed. -
An account with the
cluster-admin
role.
Procedure from the CLI
Follow this procedure to create a ServiceMeshPeer
resource from the command line. This example shows the red-mesh
creating a peer resource for the green-mesh
.
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. Enter the following command. Then, enter your username and password when prompted.$ oc login --username=<NAMEOFUSER> <API token> https://<HOSTNAME>:6443
Change to the project where you installed the control plane, for example,
red-mesh-system
.$ oc project red-mesh-system
Create a
ServiceMeshPeer
file based the following example for the two meshes that you want to federate.Example ServiceMeshPeer resource for red-mesh to green-mesh
kind: ServiceMeshPeer apiVersion: federation.maistra.io/v1 metadata: name: green-mesh namespace: red-mesh-system spec: remote: addresses: - ingress-red-mesh.green-mesh-system.apps.domain.com gateways: ingress: name: ingress-green-mesh egress: name: egress-green-mesh security: trustDomain: green-mesh.local clientID: green-mesh.local/ns/green-mesh-system/sa/egress-red-mesh-service-account certificateChain: kind: ConfigMap name: green-mesh-ca-root-cert
Run the following command to deploy the resource, where
red-mesh-system
is the system namespace andservicemeshpeer.yaml
includes a full path to the file you edited:$ oc create -n red-mesh-system -f servicemeshpeer.yaml
To confirm that connection between the red mesh and green mesh is established, inspect the status of the green-mesh
ServiceMeshPeer
in the red-mesh-system namespace:$ oc -n red-mesh-system get servicemeshpeer green-mesh -o yaml
Example ServiceMeshPeer connection between red-mesh and green-mesh
status: discoveryStatus: active: - pod: istiod-red-mesh-b65457658-9wq5j remotes: - connected: true lastConnected: "2021-10-05T13:02:25Z" lastFullSync: "2021-10-05T13:02:25Z" source: 10.128.2.149 watch: connected: true lastConnected: "2021-10-05T13:02:55Z" lastDisconnectStatus: 503 Service Unavailable lastFullSync: "2021-10-05T13:05:43Z"
The
status.discoveryStatus.active.remotes
field shows that istiod in the peer mesh (in this example, the green mesh) is connected to istiod in the current mesh (in this example, the red mesh).The
status.discoveryStatus.active.watch
field shows that istiod in the current mesh is connected to istiod in the peer mesh.If you check the
servicemeshpeer
namedred-mesh
ingreen-mesh-system
, you’ll find information about the same two connections from the perspective of the green mesh.When the connection between two meshes is not established, the
ServiceMeshPeer
status indicates this in thestatus.discoveryStatus.inactive
field.For more information on why a connection attempt failed, inspect the Istiod log, the access log of the egress gateway handling egress traffic for the peer, and the ingress gateway handling ingress traffic for the current mesh in the peer mesh.
For example, if the red mesh can’t connect to the green mesh, check the following logs:
- istiod-red-mesh in red-mesh-system
- egress-green-mesh in red-mesh-system
- ingress-red-mesh in green-mesh-system
1.18.11. Exporting a service from a federated mesh
Exporting services allows a mesh to share one or more of its services with another member of the federated mesh.
You use an ExportedServiceSet
resource to declare the services from one mesh that you are making available to another peer in the federated mesh. You must explicitly declare each service to be shared with a peer.
- You can select services by namespace or name.
- You can use wildcards to select services; for example, to export all the services in a namespace.
-
You can export services using an alias. For example, you can export the
foo/bar
service ascustom-ns/bar
. -
You can only export services that are visible to the mesh’s system namespace. For example, a service in another namespace with a
networking.istio.io/exportTo
label set to ‘.’ would not be a candidate for export. - For exported services, their target services will only see traffic from the ingress gateway, not the original requestor (that is, they won’t see the client ID of either the other mesh’s egress gateway or the workload originating the request)
The following example is for services that red-mesh
is exporting to green-mesh
.
Example ExportedServiceSet resource
kind: ExportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: green-mesh namespace: red-mesh-system spec: exportRules: # export ratings.mesh-x-bookinfo as ratings.bookinfo - type: NameSelector nameSelector: namespace: red-mesh-bookinfo name: red-ratings alias: namespace: bookinfo name: ratings # export any service in red-mesh-bookinfo namespace with label export-service=true - type: LabelSelector labelSelector: namespace: red-mesh-bookinfo selector: matchLabels: export-service: "true" aliases: # export all matching services as if they were in the bookinfo namespace - namespace: "*" name: "*" alias: namespace: bookinfo
Parameter | Description | Values |
---|---|---|
metadata: name: | Name of the ServiceMeshPeer you are exposing this service to. |
Must match the |
metadata: namespace: | Name of the project/namespace containing this resource (should be the system namespace for the mesh) . | |
spec: exportRules: - type: | Type of rule that will govern the export for this service. The first matching rule found for the service will be used for the export. |
|
spec: exportRules: - type: NameSelector nameSelector: namespace: name: |
To create a | |
spec: exportRules: - type: NameSelector nameSelector: alias: namespace: name: |
To create a | |
spec: exportRules: - type: LabelSelector labelSelector: namespace: <exportingMesh> selector: matchLabels: <labelKey>: <labelValue> |
To create a | |
spec: exportRules: - type: LabelSelector labelSelector: namespace: <exportingMesh> selector: matchLabels: <labelKey>: <labelValue> aliases: - namespace: name: alias: namespace: name: |
To create a |
Export services with the name "ratings" from all namespaces in the red-mesh to blue-mesh.
kind: ExportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: blue-mesh namespace: red-mesh-system spec: exportRules: - type: NameSelector nameSelector: namespace: "*" name: ratings
Export all services from the west-data-center namespace to green-mesh
kind: ExportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: green-mesh namespace: red-mesh-system spec: exportRules: - type: NameSelector nameSelector: namespace: west-data-center name: "*"
1.18.11.1. Creating an ExportedServiceSet
You create an ExportedServiceSet
resource to explicitly declare the services that you want to be available to a mesh peer.
Services are exported as <export-name>.<export-namespace>.svc.<ServiceMeshPeer.name>-exports.local
and will automatically route to the target service. This is the name by which the exported service is known in the exporting mesh. When the ingress gateway receives a request destined for this name, it will be routed to the actual service being exported. For example, if a service named ratings.red-mesh-bookinfo
is exported to green-mesh
as ratings.bookinfo
, the service will be exported under the name ratings.bookinfo.svc.green-mesh-exports.local
, and traffic received by the ingress gateway for that hostname will be routed to the ratings.red-mesh-bookinfo
service.
Prerequisites
-
The cluster and
ServiceMeshControlPlane
have been configured for mesh federation. -
An account with the
cluster-admin
role.
You can configure services for export even if they don’t exist yet. When a service that matches the value specified in the ExportedServiceSet is deployed, it will be automatically exported.
Procedure from the CLI
Follow this procedure to create an ExportedServiceSet
from the command line.
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. Enter the following command. Then, enter your username and password when prompted.$ oc login --username=<NAMEOFUSER> <API token> https://<HOSTNAME>:6443
Change to the project where you installed the Service Mesh control plane; for example,
red-mesh-system
.$ oc project red-mesh-system
Create an
ExportedServiceSet
file based on the following example wherered-mesh
is exporting services togreen-mesh
.Example ExportedServiceSet resource from red-mesh to green-mesh
apiVersion: federation.maistra.io/v1 kind: ExportedServiceSet metadata: name: green-mesh namespace: red-mesh-system spec: exportRules: - type: NameSelector nameSelector: namespace: red-mesh-bookinfo name: ratings alias: namespace: bookinfo name: red-ratings - type: NameSelector nameSelector: namespace: red-mesh-bookinfo name: reviews
Run the following command to upload and create the
ExportedServiceSet
resource in the red-mesh-system namespace.$ oc create -n <ControlPlaneNamespace> -f <ExportedServiceSet.yaml>
For example:
$ oc create -n red-mesh-system -f export-to-green-mesh.yaml
-
Create additional
ExportedServiceSets
as needed for each mesh peer in your federated mesh. To validate the services you’ve exported from
red-mesh
to share withgreen-mesh
, run the following command:$ oc get exportedserviceset <PeerMeshExportedTo> -o yaml
For example:
$ oc get exportedserviceset green-mesh -o yaml
Run the following command to validate the services the red-mesh exports to share with green-mesh:
$ oc get exportedserviceset <PeerMeshExportedTo> -o yaml
For example:
$ oc -n red-mesh-system get exportedserviceset green-mesh -o yaml
Example validating the services exported from the red mesh that are shared with the green mesh.
status: exportedServices: - exportedName: red-ratings.bookinfo.svc.green-mesh-exports.local localService: hostname: ratings.red-mesh-bookinfo.svc.cluster.local name: ratings namespace: red-mesh-bookinfo - exportedName: reviews.red-mesh-bookinfo.svc.green-mesh-exports.local localService: hostname: reviews.red-mesh-bookinfo.svc.cluster.local name: reviews namespace: red-mesh-bookinfo
The
status.exportedServices
array lists the services that are currently exported (these services matched the export rules in theExportedServiceSet object
). Each entry in the array indicates the name of the exported service and details about the local service that is exported.If a service that you expected to be exported is missing, confirm the Service object exists, its name or labels match the
exportRules
defined in theExportedServiceSet
object, and that the Service object’s namespace is configured as a member of the service mesh using theServiceMeshMemberRoll
orServiceMeshMember
object.
1.18.12. Importing a service into a federated mesh
Importing services lets you explicitly specify which services exported from another mesh should be accessible within your service mesh.
You use an ImportedServiceSet
resource to select services for import. Only services exported by a mesh peer and explicitly imported are available to the mesh. Services that you do not explicitly import are not made available within the mesh.
- You can select services by namespace or name.
- You can use wildcards to select services, for example, to import all the services that were exported to the namespace.
- You can select services for export using a label selector, which may be global to the mesh, or scoped to a specific member namespace.
-
You can import services using an alias. For example, you can import the
custom-ns/bar
service asother-mesh/bar
. -
You can specify a custom domain suffix, which will be appended to the
name.namespace
of an imported service for its fully qualified domain name; for example,bar.other-mesh.imported.local
.
The following example is for the green-mesh
importing a service that was exported by red-mesh
.
Example ImportedServiceSet
kind: ImportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: red-mesh #name of mesh that exported the service namespace: green-mesh-system #mesh namespace that service is being imported into spec: importRules: # first matching rule is used # import ratings.bookinfo as ratings.bookinfo - type: NameSelector importAsLocal: false nameSelector: namespace: bookinfo name: ratings alias: # service will be imported as ratings.bookinfo.svc.red-mesh-imports.local namespace: bookinfo name: ratings
Parameter | Description | Values |
---|---|---|
metadata: name: | Name of the ServiceMeshPeer that exported the service to the federated mesh. | |
metadata: namespace: | Name of the namespace containing the ServiceMeshPeer resource (the mesh system namespace). | |
spec: importRules: - type: | Type of rule that will govern the import for the service. The first matching rule found for the service will be used for the import. |
|
spec: importRules: - type: NameSelector nameSelector: namespace: name: |
To create a | |
spec: importRules: - type: NameSelector importAsLocal: |
Set to |
|
spec: importRules: - type: NameSelector nameSelector: namespace: name: alias: namespace: name: |
To create a |
Import the "bookinfo/ratings" service from the red-mesh into blue-mesh
kind: ImportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: red-mesh namespace: blue-mesh-system spec: importRules: - type: NameSelector importAsLocal: false nameSelector: namespace: bookinfo name: ratings
Import all services from the red-mesh’s west-data-center namespace into the green-mesh. These services will be accessible as <name>.west-data-center.svc.red-mesh-imports.local
kind: ImportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: red-mesh namespace: green-mesh-system spec: importRules: - type: NameSelector importAsLocal: false nameSelector: namespace: west-data-center name: "*"
1.18.12.1. Creating an ImportedServiceSet
You create an ImportedServiceSet
resource to explicitly declare the services that you want to import into your mesh.
Services are imported with the name <exported-name>.<exported-namespace>.svc.<ServiceMeshPeer.name>.remote
which is a "hidden" service, visible only within the egress gateway namespace and is associated with the exported service’s hostname. The service will be available locally as <export-name>.<export-namespace>.<domainSuffix>
, where domainSuffix
is svc.<ServiceMeshPeer.name>-imports.local
by default, unless importAsLocal
is set to true
, in which case domainSuffix
is svc.cluster.local
. If importAsLocal
is set to false
, the domain suffix in the import rule will be applied. You can treat the local import just like any other service in the mesh. It automatically routes through the egress gateway, where it is redirected to the exported service’s remote name.
Prerequisites
-
The cluster and
ServiceMeshControlPlane
have been configured for mesh federation. -
An account with the
cluster-admin
role.
You can configure services for import even if they haven’t been exported yet. When a service that matches the value specified in the ImportedServiceSet is deployed and exported, it will be automatically imported.
Procedure from the CLI
Follow this procedure to create an ImportedServiceSet
from the command line.
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. Enter the following command. Then, enter your username and password when prompted.$ oc login --username=<NAMEOFUSER> <API token> https://<HOSTNAME>:6443
Change to the project where you installed the Service Mesh control plane; for example,
green-mesh-system
.$ oc project green-mesh-system
Create an
ImportedServiceSet
file based on the following example wheregreen-mesh
is importing services previously exported byred-mesh
.Example ImportedServiceSet resource from red-mesh to green-mesh
kind: ImportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: red-mesh namespace: green-mesh-system spec: importRules: - type: NameSelector importAsLocal: false nameSelector: namespace: bookinfo name: red-ratings alias: namespace: bookinfo name: ratings
Run the following command to upload and create the
ImportedServiceSet
resource in the green-mesh-system namespace.$ oc create -n <ControlPlaneNamespace> -f <ImportedServiceSet.yaml>
For example:
$ oc create -n green-mesh-system -f import-from-red-mesh.yaml
-
Create additional
ImportedServiceSet
resources as needed for each mesh peer in your federated mesh. To validate the services you’ve imported into
green-mesh
, run the following command:$ oc get importedserviceset <PeerMeshImportedInto> -o yaml
For example:
$ oc get importedserviceset green-mesh -o yaml
Run the following command to validate the services imported into a mesh.
$ oc get importedserviceset <PeerMeshImportedInto> -o yaml
Example validating that the services exported from the red mesh have been imported into the green mesh using the status section of the
importedserviceset/red-mesh' object in the 'green-mesh-system
namespace:$ oc -n green-mesh-system get importedserviceset/red-mesh -o yaml
status: importedServices: - exportedName: red-ratings.bookinfo.svc.green-mesh-exports.local localService: hostname: ratings.bookinfo.svc.red-mesh-imports.local name: ratings namespace: bookinfo - exportedName: reviews.red-mesh-bookinfo.svc.green-mesh-exports.local localService: hostname: "" name: "" namespace: ""
In the preceding example only the ratings service is imported, as indicated by the populated fields under
localService
. The reviews service is available for import, but isn’t currently imported because it does not match anyimportRules
in theImportedServiceSet
object.
1.18.13. Configuring a federated mesh for failover
Failover is the ability to switch automatically and seamlessly to a reliable backup system, for example another server. In the case of a federated mesh, you can configure a service in one mesh to failover to a service in another mesh.
You configure Federation for failover by setting the importAsLocal
and locality
settings in an ImportedServiceSet
resource and then configuring a DestinationRule
that configures failover for the service to the locality specified in the ImportedServiceSet
.
Prerequisites
- Two or more OpenShift Container Platform 4.6 or above clusters already networked and federated.
-
ExportedServiceSet
resources already created for each mesh peer in the federated mesh. -
ImportedServiceSet
resources already created for each mesh peer in the federated mesh. -
An account with the
cluster-admin
role.
1.18.13.1. Configuring an ImportedServiceSet for failover
Locality-weighted load balancing allows administrators to control the distribution of traffic to endpoints based on the localities of where the traffic originates and where it will terminate. These localities are specified using arbitrary labels that designate a hierarchy of localities in {region}/{zone}/{sub-zone} form.
In the examples in this section, the green-mesh
is located in the us-east
region, and the red-mesh
is located in the us-west
region.
Example ImportedServiceSet
resource from red-mesh to green-mesh
kind: ImportedServiceSet apiVersion: federation.maistra.io/v1 metadata: name: red-mesh #name of mesh that exported the service namespace: green-mesh-system #mesh namespace that service is being imported into spec: importRules: # first matching rule is used # import ratings.bookinfo as ratings.bookinfo - type: NameSelector importAsLocal: true nameSelector: namespace: bookinfo name: ratings alias: # service will be imported as ratings.bookinfo.svc.red-mesh-imports.local namespace: bookinfo name: ratings #Locality within which imported services should be associated. locality: region: us-west
Name | Description | Type |
---|---|---|
region: | Region within which imported services are located. | string |
subzone: | Subzone within which imported services are located. I Subzone is specified, Zone must also be specified. | string |
zone: | Zone within which imported services are located. If Zone is specified, Region must also be specified. | string |
Procedure
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role, enter the following command:$ oc login --username=<NAMEOFUSER> <API token> https://<HOSTNAME>:6443
Change to the project where you installed the Service Mesh control plane, enter the following command:
$ oc project <smcp-system>
For example,
green-mesh-system
.$ oc project green-mesh-system
Edit the
ImportedServiceSet
file, where<ImportedServiceSet.yaml>
includes a full path to the file you want to edit, enter the following command:$ oc edit -n <smcp-system> -f <ImportedServiceSet.yaml>
For example, if you want to modify the file that imports from the red-mesh-system to the green-mesh-system as shown in the previous
ImportedServiceSet
example.$ oc edit -n green-mesh-system -f import-from-red-mesh.yaml
Modify the file:
-
Set
spec.importRules.importAsLocal
totrue
. -
Set
spec.locality
to aregion
,zone
, orsubzone
. - Save your changes.
-
Set
1.18.13.2. Configuring a DestinationRule for failover
Create a DestinationRule
resource that configures the following:
- Outlier detection for the service. This is required in order for failover to function properly. In particular, it configures the sidecar proxies to know when endpoints for a service are unhealthy, eventually triggering a failover to the next locality.
- Failover policy between regions. This ensures that failover beyond a region boundary will behave predictably.
Procedure
Log in to the OpenShift Container Platform CLI as a user with the
cluster-admin
role. Enter the following command. Then, enter your username and password when prompted.$ oc login --username=<NAMEOFUSER> <API token> https://<HOSTNAME>:6443
Change to the project where you installed the Service Mesh control plane.
$ oc project <smcp-system>
For example,
green-mesh-system
.$ oc project green-mesh-system
Create a
DestinationRule
file based on the following example where if green-mesh is unavailable, the traffic should be routed from the green-mesh in theus-east
region to the red-mesh inus-west
.Example
DestinationRule
apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: default-failover namespace: bookinfo spec: host: "ratings.bookinfo.svc.cluster.local" trafficPolicy: loadBalancer: localityLbSetting: enabled: true failover: - from: us-east to: us-west outlierDetection: consecutive5xxErrors: 3 interval: 10s baseEjectionTime: 1m
Deploy the
DestinationRule
, where<DestinationRule>
includes the full path to your file, enter the following command:$ oc create -n <application namespace> -f <DestinationRule.yaml>
For example:
$ oc create -n bookinfo -f green-mesh-us-west-DestinationRule.yaml
1.18.14. Removing a service from the federated mesh
If you need to remove a service from the federated mesh, for example if it has become obsolete or has been replaced by a different service, you can do so.
1.18.14.1. To remove a service from a single mesh
Remove the entry for the service from the ImportedServiceSet
resource for the mesh peer that no longer should access the service.
1.18.14.2. To remove a service from the entire federated mesh
Remove the entry for the service from the ExportedServiceSet
resource for the mesh that owns the service.
1.18.15. Removing a mesh from the federated mesh
If you need to remove a mesh from the federation, you can do so.
-
Edit the removed mesh’s
ServiceMeshControlPlane
resource to remove all federation ingress gateways for peer meshes. For each mesh peer that the removed mesh has been federated with:
-
Remove the
ServiceMeshPeer
resource that links the two meshes. -
Edit the peer mesh’s
ServiceMeshControlPlane
resource to remove the egress gateway that serves the removed mesh.
-
Remove the
1.19. Extensions
You can use WebAssembly extensions to add new features directly into the Red Hat OpenShift Service Mesh proxies. This lets you move even more common functionality out of your applications, and implement them in a single language that compiles to WebAssembly bytecode.
WebAssembly extensions are not supported on IBM Z and IBM Power Systems.
1.19.1. WebAssembly modules overview
WebAssembly modules can be run on many platforms, including proxies, and have broad language support, fast execution, and a sandboxed-by-default security model.
Red Hat OpenShift Service Mesh extensions are Envoy HTTP Filters, giving them a wide range of capabilities:
- Manipulating the body and headers of requests and responses.
- Out-of-band HTTP requests to services not in the request path, such as authentication or policy checking.
- Side-channel data storage and queues for filters to communicate with each other.
When creating new WebAssembly extensions, use the WasmPlugin API. The ServiceMeshExtension API is deprecated as of Red Hat OpenShift Service Mesh version 2.2 and will be removed in a future release.
There are two parts to writing a Red Hat OpenShift Service Mesh extension:
- You must write your extension using an SDK that exposes the proxy-wasm API and compile it to a WebAssembly module.
- You must then package the module into a container.
Supported languages
You can use any language that compiles to WebAssembly bytecode to write a Red Hat OpenShift Service Mesh extension, but the following languages have existing SDKs that expose the proxy-wasm API so that it can be consumed directly.