Chapter 5. Service Mesh user guide
5.1. Traffic management
You can control the flow of traffic and API calls between services in Red Hat OpenShift Service Mesh. For example, some services in your service mesh may need to communicate within the mesh and others may need to be hidden. Manage the traffic to hide specific backend services, expose services, create testing or versioning deployments, or add a security layer on a set of services.
This guide references the Bookinfo sample application to provide examples of routing in an example application. Install the Bookinfo application to learn how these routing examples work.
5.1.1. Routing and managing traffic
Configure your service mesh by adding your own traffic configuration to Red Hat OpenShift Service Mesh with a custom resource definitions in a YAML file.
5.1.1.1. Traffic management with virtual services
You can route requests dynamically to multiple versions of a microservice through Red Hat OpenShift Service Mesh with a virtual service. With virtual services, you can:
- Address multiple application services through a single virtual service. If your mesh uses Kubernetes, for example, you can configure a virtual service to handle all services in a specific namespace. Mapping a single virtual service to many services is particularly useful in facilitating turning a monolithic application into a composite service built out of distinct microservices without requiring the consumers of the service to adapt to the transition.
- Configure traffic rules in combination with gateways to control ingress and egress traffic.
5.1.1.1.1. Configuring virtual services
Requests are routed to a services within a service mesh with virtual services. Each virtual service consists of a set of routing rules that are evaluated in order. Red Hat OpenShift Service Mesh matches each given request to the virtual service to a specific real destination within the mesh.
Without virtual services, Red Hat OpenShift Service Mesh distributes traffic using round-robin load balancing between all service instances. With a virtual service, you can specify traffic behavior for one or more hostnames. Routing rules in the virtual service tell Red Hat OpenShift Service Mesh how to send the traffic for the virtual service to appropriate destinations. Route destinations can be versions of the same service or entirely different services.
The following example routes requests to different versions of a service depending on which user connects to the application. Use this command to apply this example YAML file, or one you create.
$ oc apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2 - route: - destination: host: reviews subset: v3 EOF
5.1.1.2. Configuring your virtual host
The following sections explain each field in the YAML file and explain how you can create a virtual host in a virtual service.
5.1.1.2.1. Hosts
The hosts
field lists the virtual service’s user-addressable destination that these routing rules apply to. This is the address or addresses the client uses when sending requests to the service.
The virtual service hostname can be an IP address, a DNS name, or, depending on the platform, a short name that resolves to a fully qualified domain name.
spec: hosts: - reviews
5.1.1.2.2. Routing rules
The http
section contains the virtual service’s routing rules, describing match conditions and actions for routing HTTP/1.1, HTTP2, and gRPC traffic sent to the destination specified in the hosts field. A routing rule consists of the destination where you want the traffic to go and zero or more match conditions, depending on your use case.
Match condition
The first routing rule in the example has a condition and begins with the match field. In this example, this routing applies to all requests from the user jason
. Add the headers
, end-user
, and exact
fields to select the appropriate requests.
spec: hosts: - reviews http: - match: - headers: end-user: exact: jason
Destination
The destination
field in the route section specifies the actual destination for traffic that matches this condition. Unlike the virtual service’s host, the destination’s host must be a real destination that exists in the Red Hat OpenShift Service Mesh service registry. This can be a mesh service with proxies or a non-mesh service added using a service entry. In this example, the host name is a Kubernetes service name:
spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2
5.1.1.2.3. Destination rules
Destination rules are applied after virtual service routing rules are evaluated, so they apply to the traffic’s real destination. Virtual services route traffic to a destination. Destination rules configure what happens to traffic at that destination.
5.1.1.2.3.1. Load balancing options
By default, Red Hat OpenShift Service Mesh uses a round-robin load balancing policy, where each service instance in the instance pool gets a request in turn. Red Hat OpenShift Service Mesh also supports the following models, which you can specify in destination rules for requests to a particular service or service subset.
- Random: Requests are forwarded at random to instances in the pool.
- Weighted: Requests are forwarded to instances in the pool according to a specific percentage.
- Least requests: Requests are forwarded to instances with the least number of requests.
Destination rule example
The following example destination rule configures three different subsets for the my-svc
destination service, with different load balancing policies:
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: my-destination-rule spec: host: my-svc trafficPolicy: loadBalancer: simple: RANDOM subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 trafficPolicy: loadBalancer: simple: ROUND_ROBIN - name: v3 labels: version: v3
5.1.1.2.4. Gateways
You can use a gateway to manage inbound and outbound traffic for your mesh to specify which traffic you want to enter or leave the mesh. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads.
Unlike other mechanisms for controlling traffic entering your systems, such as the Kubernetes Ingress APIs, Red Hat OpenShift Service Mesh gateways let you use the full power and flexibility of traffic routing. The Red Hat OpenShift Service Mesh gateway resource can layer 4-6 load balancing properties such as ports to expose, Red Hat OpenShift Service Mesh TLS settings. Instead of adding application-layer traffic routing (L7) to the same API resource, you can bind a regular Red Hat OpenShift Service Mesh virtual service to the gateway and manage gateway traffic like any other data plane traffic in a service mesh.
Gateways are primarily used to manage ingress traffic, but you can also configure egress gateways. An egress gateway lets you configure a dedicated exit node for the traffic leaving the mesh, letting you limit which services have access to external networks, or to enable secure control of egress traffic to add security to your mesh, for example. You can also use a gateway to configure a purely internal proxy.
Gateway example
The following example shows a possible gateway configuration for external HTTPS ingress traffic:
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: ext-host-gwy spec: selector: istio: ingressgateway # use istio default controller servers: - port: number: 443 name: https protocol: HTTPS hosts: - ext-host.example.com tls: mode: SIMPLE serverCertificate: /tmp/tls.crt privateKey: /tmp/tls.key
This gateway configuration lets HTTPS traffic from ext-host.example.com
into the mesh on port 443, but doesn’t specify any routing for the traffic.
To specify routing and for the gateway to work as intended, you must also bind the gateway to a virtual service. You do this using the virtual service’s gateways field, as shown in the following example:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: virtual-svc spec: hosts: - ext-host.example.com gateways: - ext-host-gwy
You can then configure the virtual service with routing rules for the external traffic.
5.1.1.2.5. Service entries
A service entry adds an entry to the service registry that Red Hat OpenShift Service Mesh maintains internally. After you add the service entry, the Envoy proxies can send traffic to the service as if it was a service in your mesh. Configuring service entries allows you to manage traffic for services running outside of the mesh, including the following tasks:
- Redirect and forward traffic for external destinations, such as APIs consumed from the web, or traffic to services in legacy infrastructure.
- Define retry, timeout, and fault injection policies for external destinations.
- Run a mesh service in a Virtual Machine (VM) by adding VMs to your mesh.
- Logically add services from a different cluster to the mesh to configure a multicluster Red Hat OpenShift Service Mesh mesh on Kubernetes.
- You don’t need to add a service entry for every external service that you want your mesh services to use. By default, Red Hat OpenShift Service Mesh configures the Envoy proxies to passthrough requests to unknown services. However, you can’t use Red Hat OpenShift Service Mesh features to control the traffic to destinations that aren’t registered in the mesh.
Service entry examples
The following example mesh-external service entry adds the ext-resource
external dependency to the Red Hat OpenShift Service Mesh service registry:
apiVersion: networking.istio.io/v1alpha3 kind: ServiceEntry metadata: name: svc-entry spec: hosts: - ext-svc.example.com ports: - number: 443 name: https protocol: HTTPS location: MESH_EXTERNAL resolution: DNS
Specify the external resource using the hosts field. You can qualify it fully or use a wildcard prefixed domain name.
You can configure virtual services and destination rules to control traffic to a service entry in the same way you configure traffic for any other service in the mesh. For example, the following destination rule configures the traffic route to use mutual TLS to secure the connection to the ext-svc.example.com
external service that is configured using the service entry:
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: ext-res-dr spec: host: ext-svc.example.com trafficPolicy: tls: mode: MUTUAL clientCertificate: /etc/certs/myclientcert.pem privateKey: /etc/certs/client_private_key.pem caCertificates: /etc/certs/rootcacerts.pem
5.1.1.2.6. Sidecar
By default, Red Hat OpenShift Service Mesh configures every Envoy proxy to accept traffic on all the ports of its associated workload, and to reach every workload in the mesh when forwarding traffic. You can use a sidecar configuration to do the following:
- Fine-tune the set of ports and protocols that an Envoy proxy accepts.
- Limit the set of services that the Envoy proxy can reach.
- You might want to limit sidecar reachability like this in larger applications, where having every proxy configured to reach every other service in the mesh can potentially affect mesh performance due to high memory usage.
Sidecar example
You can specify that you want a sidecar configuration to apply to all workloads in a particular namespace, or choose specific workloads using a workloadSelector
. For example, the following sidecar configuration configures all services in the bookinfo
namespace to only reach services running in the same namespace and the Red Hat OpenShift Service Mesh control plane (currently needed to use the Red Hat OpenShift Service Mesh policy and telemetry features):
apiVersion: networking.istio.io/v1alpha3 kind: Sidecar metadata: name: default namespace: bookinfo spec: egress: - hosts: - "./*" - "istio-system/*"
5.1.2. Managing ingress traffic
In Red Hat OpenShift Service Mesh, the Ingress Gateway enables Service Mesh features such as monitoring, security, and route rules to be applied to traffic entering the cluster. Configure Service Mesh to expose a service outside of the service mesh using an Service Mesh gateway.
5.1.2.1. Determining the ingress IP and ports
Run the following command to determine if your Kubernetes cluster is running in an environment that supports external load balancers:
$ oc get svc istio-ingressgateway -n istio-system
That command returns the NAME
, TYPE
, CLUSTER-IP
, EXTERNAL-IP
, PORT(S)
, and AGE
of each item in your namespace.
If the EXTERNAL-IP
value is set, your environment has an external load balancer that you can use for the ingress gateway.
If the EXTERNAL-IP
value is <none>
, or perpetually <pending>
, your environment does not provide an external load balancer for the ingress gateway. You can access the gateway using the service’s node port.
Choose the instructions for your environment:
Configuring routing with a load balancer
Follow these instructions if your environment has an external load balancer.
Set the ingress IP and ports:
$ export INGRESS_HOST=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
In some environments, the load balancer may be exposed using a host name instead of an IP address. For that case, the ingress gateway’s EXTERNAL-IP
value is not be an IP address. Instead, it’s a host name, and the previous command fails to set the INGRESS_HOST
environment variable.
Use the following command to correct the INGRESS_HOST
value:
$ export INGRESS_HOST=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
Configuring routing without a load balancer
Follow these instructions if your environment does not have an external load balancer. You must use a node port instead.
Set the ingress ports:
$ export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
$ export SECURE_INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')
5.1.3. Routing example using the bookinfo application
The Service Mesh Bookinfo sample application consists of four separate microservices, each with multiple versions. Three different versions, one of the microservices called reviews
, have been deployed and are running concurrently.
Prerequisites:
- Deploy the Bookinfo sample application to work with the following examples.
About this task
To illustrate the problem this causes, access the bookinfo app /product page
in a browser and refresh several times.
Sometimes the book review output contains star ratings and other times it does not. Without an explicit default service version to route to, Service Mesh routes requests to all available versions one after the other.
This tutorial helps you apply rules that route all traffic to v1
(version 1) of the microservices. Later, you can apply a rule to route traffic based on the value of an HTTP request header.
5.1.3.1. Applying a virtual service
To route to one version only, apply virtual services that set the default version for the micro-services. In the following example, the virtual service routes all traffic to v1
of each micro-service
Run the following command to apply the virtual services:
$ oc apply -f https://raw.githubusercontent.com/Maistra/istio/maistra-1.1/samples/bookinfo/networking/virtual-service-all-v1.yaml
To test the command was successful, display the defined routes with the following command:
$ oc get virtualservices -o yaml
That command returns the following YAML file.
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: details ... spec: hosts: - details http: - route: - destination: host: details subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productpage ... spec: gateways: - bookinfo-gateway - mesh hosts: - productpage http: - route: - destination: host: productpage subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: ratings ... spec: hosts: - ratings http: - route: - destination: host: ratings subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews ... spec: hosts: - reviews http: - route: - destination: host: reviews subset: v1
You have configured Service Mesh to route to the
v1
version of the Bookinfo microservices, most importantly thereviews
service version 1.
5.1.3.2. Test the new routing configuration
You can easily test the new configuration by once again refreshing the /productpage
of the Bookinfo app.
Open the Bookinfo site in your browser. The URL is
http://$GATEWAY_URL/productpage
, where$GATEWAY_URL
is the External IP address of the ingress.The reviews part of the page displays with no rating stars, no matter how many times you refresh. This is because you configured Service Mesh to route all traffic for the reviews service to the version
reviews:v1
and this version of the service does not access the star ratings service.Your service mesh now routes traffic to one version of a service.
5.1.3.3. Route based on user identity
Next, change the route configuration so that all traffic from a specific user is routed to a specific service version. In this case, all traffic from a user named jason
will be routed to the service reviews:v2
.
Note that Service Mesh doesn’t have any special, built-in understanding of user identity. This example is enabled by the fact that the productpage
service adds a custom end-user
header to all outbound HTTP requests to the reviews service.
Run the following command to enable user-based routing:
$ oc apply -f https://raw.githubusercontent.com/Maistra/istio/maistra-1.1/samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
Confirm the rule is created:
$ oc get virtualservice reviews -o yaml
That command returns the following YAML file.
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews ... spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2 - route: - destination: host: reviews subset: v1
-
On the
/productpage
of the Bookinfo app, log in as userjason
. Refresh the browser. What do you see? The star ratings appear next to each review. -
Log in as another user (pick any name you wish). Refresh the browser. Now the stars are gone. This is because traffic is routed to
reviews:v1
for all users except Jason.
You have successfully configured Service Mesh to route traffic based on user identity.
5.2. Data visualization and observability
You can view your application’s topology, health and metrics in the Kiali console. If your service is having issues, the Kiali console offers ways to visualize the data flow through your service. You can view insights about the mesh components at different levels, including abstract applications, services, and workloads. It also provides an interactive graph view of your namespace in real time.
You can observe the data flow through your application if you have one installed. If you don’t have your own application installed, you can see how observability works in Red Hat OpenShift Service Mesh by installing the Bookinfo sample application.
After installing the Bookinfo sample application, send traffic to the mesh. Enter the following command a few times:
$ curl http://$GATEWAY_URL/productpage
If your sample application is configured correctly, this command simulates a user visiting the productpage
microservice of the application.
5.2.1. Accessing the Kiali console
To access the console, in the menu bar, click the Application launcher > Kiali.
- In the OpenShift Container Platform menu bar, click the Application launcher > Kiali.
- Log in to the Kiali console with the same user name and password as you use to access the OpenShift Container Platform console.
-
Select the project for your service in the Namespace field. If you have installed the Bookinfo example, select
bookinfo
.
Procedure from the command line
Run this command from the CLI to obtain the route and Kiali URL:
$ oc get routes
In the output on the
kiali
line, use the URL in the HOST/PORT column to open the Kiali console. Log in to the Kiali console with the same user name and password as you use to access the OpenShift Container Platform console. Select the project for your service in the Namespace field.
When you first log in, you see the Overview page which displays all the namespaces in your mesh that you have permission to view.
5.2.2. Visualizing your service
The Kiali operator works with the telemetry data gathered in Red Hat OpenShift Service Mesh to provide graphs and real time network diagrams of the applications, services, and workloads in your namespace.
The Overview page displays all the namespaces that have services in your mesh. You can reveal deeper insights about the data traveling through your Service mesh or help identify problems with services or workloads in your service mesh with the following graphs and visualizations.
5.2.2.1. Namespace graphs
The namespace graph is a map of the services, deployments and workflows in your namespace and arrows that show how data flows through them. To view a namespace graph:
- Click Graph in the main navigation.
-
Select
bookinfo
from the Namespace menu.
If your application uses version tags, like the Bookinfo sample application, you can see a Version graph. Select a graph from the Graph Type drop down menu. There are several graphs to choose from:
- The App graph shows an aggregate workload for all applications that are labeled the same.
- The Versioned App graph shows a node for each version of an app. All versions of an app are grouped together.
- The Workload graph shows a node for each workload in your service mesh. This graph does not require you to use the app and version labels. If your app does not use version labels, use this the graph.
- The Service graph shows a node for each service in your mesh but excludes all apps and workloads from the graph. It provides a high level view and aggregates all traffic for defined services.
To view a summary of metrics, select any node or edge in the graph to display its metric details in the summary details panel.
5.3. Customizing security in a Service Mesh
If your service mesh application is constructed with a complex array of microservices, you can use Red Hat OpenShift Service Mesh to customize the security of the communication between those services. The infrastructure of OpenShift Container Platform along with the traffic management features of Service Mesh can help you manage the complexity of your applications and provide service and identity security for microservices.
5.3.1. Enabling mutual Transport Layer Security (mTLS)
Mutual Transport Layer Security (mTLS) is a protocol where two parties authenticate each other at the same time. It is the default mode of authentication in some protocols (IKE, SSH) and optional in others (TLS).
MTLS can be used without changes to the application or service code. The TLS is handled entirely by the service mesh infrastructure and between the two sidecar proxies.
By default, Red Hat OpenShift Service Mesh is set to permissive mode, where the sidecars in Service Mesh accept both plain-text traffic and connections that are encrypted using mTLS. If a service in your mesh is communicating with a service outside the mesh, strict mTLS could break communication between those services. Use permissive mode while you migrate your workloads to Service Mesh.
5.3.1.1. Enabling strict mTLS across the mesh
If your workloads do not communicate with services outside your mesh and communication will not be interrupted by only accepting encrypted connections, you can enable mTLS across your mesh quickly. Set spec.istio.global.mtls.enabled
to true
in your ServiceMeshControlPlane resource. The operator creates the required resources.
apiVersion: maistra.io/v1 kind: ServiceMeshControlPlane spec: istio: global: mtls: enabled: true
5.3.1.1.1. Configuring sidecars for incoming connections for specific services
You can also configure mTLS for individual services or namespaces by creating a policy.
apiVersion: "authentication.maistra.io/v1" kind: "Policy" metadata: name: "default" namespace: <NAMESPACE> spec: peers: - mtls: {}
5.3.1.2. Configuring sidecars for outgoing connections
Create a destination rule to configure Service Mesh to use mTLS when sending requests to other services in the mesh.
apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "default" namespace: <CONTROL_PLANE_NAMESPACE> spec: host: "*.local" trafficPolicy: tls: mode: ISTIO_MUTUAL