OpenShift Service Mesh 3.0 is a Technology Preview feature only
Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. This documentation is a work in progress and might not be complete or fully tested.Gateways
Gateways and OpenShift Service Mesh
Abstract
Chapter 1. About gateways
A gateway is a standalone Envoy proxy deployment and an associated Kubernetes service operating at the edge of a service mesh. You can configure a gateway to provide fine-grained control over the traffic that enters or leaves the mesh. In Red Hat OpenShift Service Mesh, you install gateways using gateway injection.
1.1. About gateway injection
Gateway injection relies upon the same mechanism as sidecar injection to inject the Envoy proxy into gateway pods. To install a gateway using gateway injection, you create a Kubernetes Deployment
object and an associated Kubernetes Service
object in a namespace that is visible to the Istio control plane. When creating the Deployment
object you label and annotate it so that the Istio control plane injects a proxy, and the proxy is configured as a gateway. After installing the gateway, you configure it to control ingress and egress traffic using the Istio Gateway
and VirtualService
resources.
1.1.1. Installing a gateway by using gateway injection
This procedure explains how to install a gateway by using gateway injection.
You can use this procedure to create ingress or egress gateways.
Prerequisites
- You have installed the OpenShift Service Mesh Operator version 3.0 or later.
- You have created an Istio control plane.
-
You have created an
IstioCNI
resource.
Procedure
Create a namespace that you will use to install the gateway.
$ oc create namespace <gateway_namespace>
NoteInstall the gateway and the Istio control plane in different namespaces.
You can install the gateway in a dedicated gateway namespace. This approach allows the gateway to be shared by many applications operating in different namespaces. Alternatively, you can install the gateway in an application namespace. In this approach, the gateway acts as a dedicated gateway for the application in that namespace.
Create a YAML file named
secret-reader.yml
that defines the service account, role, and role binding for the gateway deployment. These settings enable the gateway to read the secrets, which is required for obtaining TLS credentials.apiVersion: v1 kind: ServiceAccount metadata: name: secret-reader namespace: <gateway_namespace> --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: secret-reader namespace: <gateway_namespace> rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: secret-reader namespace: <gateway_namespace> roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: secret-reader subjects: - kind: ServiceAccount name: secret-reader
Apply the YAML file by running the following command:
$ oc apply -f secret-reader.yml
Create a YAML file named
gateway-deployment.yml
that defines the KubernetesDeployment
object for the gateway.apiVersion: apps/v1 kind: Deployment metadata: name: <gateway_name> namespace: <gateway_namespace> spec: selector: matchLabels: istio: <gateway_name> template: metadata: annotations: inject.istio.io/templates: gateway 1 labels: istio: <gateway_name> 2 sidecar.istio.io/inject: "true" 3 spec: containers: - name: istio-proxy image: auto 4 securityContext: capabilities: drop: - ALL allowPrivilegeEscalation: false privileged: false readOnlyRootFilesystem: true runAsNonRoot: true ports: - containerPort: 15090 protocol: TCP name: http-envoy-prom resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 100m memory: 128Mi securityContext: sysctls: - name: net.ipv4.ip_unprivileged_port_start value: "0" serviceAccountName: secret-reader 5
- 1
- Indicates that the Istio control plane uses the gateway injection template instead of the default sidecar template.
- 2
- Ensure that a unique label is set for the gateway deployment. A unique label is required so that Istio
Gateway
resources can select gateway workloads. - 3
- Enables gateway injection by setting the
sidecar.istio.io/inject
label totrue
. If the name of the Istio resource is notdefault
you must use theistio.io/rev: <istio_revision>
label instead, where the revision represents the active revision of the Istio resource. - 4
- Sets the image field to
auto
so that the image automatically updates each time the pod starts. - 5
- Sets the
serviceAccountName
to the name of theServiceAccount
created previously.
Apply the YAML file by running the following command:
$ oc apply -f gateway-deployment.yml
Verify that the gateway
Deployment
rollout was successful by running the following command:$ oc rollout status deployment/<gateway_name> -n <gateway_namespace>
You should see output similar to the following:
Example output
Waiting for deployment "<gateway_name>" rollout to finish: 0 of 1 updated replicas are available... deployment "<gateway_name>" successfully rolled out
Create a YAML file named
gateway-service.yml
that contains the KubernetesService
object for the gateway.apiVersion: v1 kind: Service metadata: name: <gateway_name> namespace: <gateway_namespace> spec: type: ClusterIP 1 selector: istio: <gateway_name> 2 ports: - name: status-port port: 15021 protocol: TCP targetPort: 15021 - name: http2 port: 80 protocol: TCP targetPort: 80 - name: https port: 443 protocol: TCP targetPort: 443
- 1
- When you set
spec.type
toClusterIP
the gatewayService
object can be accessed only from within the cluster. If the gateway has to handle ingress traffic from outside the cluster, setspec.type
toLoadBalancer
. Alternatively, you can use OpenShift Routes. - 2
- Set the
selector
to the unique label or set of labels specified in the pod template of the gateway deployment that you previously created.
Apply the YAML file by running the following command:
$ oc apply -f gateway-service.yml
Verify that the gateway service is targeting the endpoint of the gateway pods by running the following command:
$ oc get endpoints <gateway_name> -n <gateway_namespace>
You should see output similar to the following example:
Example output
NAME ENDPOINTS AGE <gateway_name> 10.131.0.181:8080,10.131.0.181:8443 1m
Optional: Create a YAML file named
gateway-hpa.yml
that defines a horizontal pod autoscaler for the gateway. The following example sets the minimum replicas to2
and the maximum replicas to5
and scales the replicas up when average CPU utilization exceeds 80% of the CPU resource limit. This limit is specified in the pod template of the deployment for the gateway.apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: <gateway_name> namespace: <gateway_namespace> spec: minReplicas: 2 maxReplicas: 5 metrics: - resource: name: cpu target: averageUtilization: 80 type: Utilization type: Resource scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: <gateway_name> 1
- 1
- Set
spec.scaleTargetRef.name
to the name of the gateway deployment previously created.
Optional: Apply the YAML file by running the following command:
$ oc apply -f gateway-hpa.yml
Optional: Create a YAML file named
gateway-pdb.yml
that defines a pod disruption budget for the gateway. The following example allows gateway pods to be evicted only when at least 1 healthy gateway pod will remain on the cluster after the eviction.apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: <gateway_name> namespace: <gateway_namespace> spec: minAvailable: 1 selector: matchLabels: istio: <gateway_name> 1
- 1
- Set the
spec.selector.matchLabels
to the unique label or set of labels specified in the pod template of the gateway deployment previously created.
Optional: Apply the YAML file by running the following command:
$ oc apply -f gateway-pdb.yml
Chapter 2. Getting traffic into a mesh
Using Istio APIs, you can configure gateway proxies that were installed using gateway injection to accept traffic originating from outside the mesh, and route that traffic to the services within the mesh.
You can expose gateway proxies to traffic outside a cluster by using either a LoadBalancer
type Service
or OpenShift Routes
.
2.1. About configuring a gateway installed using gateway injection to accept ingress traffic
When you install a gateway using gateway injection you can configure it to receive ingress traffic using the Istio Gateway
and VirtualService
resources in combination. The Istio Gateway
resource describes a load balancer operating at the edge of the mesh that receives incoming or outgoing HTTP/TCP connections. The Gateway
specification describes a set of ports that should be exposed, the type of protocol to use, and the Server Name Indication (SNI) configuration for the load balancer. VirtualServices
define routing rules to apply to an Istio Gateway
, similar to how you can use VirtualServices
to define routing rules for internal mesh traffic.
In the following example an Istio Gateway
resource configures a gateway proxy to act as an entry point for external traffic. This configuration exposes port 443 (HTTPS) for the host, bookinfo.com
. The example configuration applies to pods with the istio: ingressgateway
label. The tls
mode is configured as SIMPLE
, which terminates the incoming HTTPS traffic using the certificate and private key the example provides.
Sample configuration
apiVersion: networking.istio.io/v1 kind: Gateway metadata: name: bookinfo-gateway namespace: bookinfo spec: selector: istio: ingressgateway servers: - port: number: 443 name: https-443 protocol: HTTPS hosts: - bookinfo.com tls: mode: SIMPLE serverCertificate: /etc/certs/servercert.pem privateKey: /etc/certs/privatekey.pem
The following VirtualService
is bound to the Istio Gateway
resource shown in the previous example configuration. The specification defines rules to route traffic with the /reviews/
path prefix to the reviews service in the bookinfo
namespace. The VirtualService
explicitly references the Gateway
resource shown previously. This ensures that the routing rules are only applied to the traffic that enters through the specified gateway.
Sample configuration
kind: VirtualService metadata: name: bookinfo-rule namespace: bookinfo spec: hosts: - bookinfo.com gateways: - bookinfo/bookinfo-gateway http: - match: - uri: prefix: /reviews/ route: - destination: port: number: 9080 host: reviews.bookinfo.svc.cluster.local
Additional resources
2.1.1. Exposing a service by using the Istio Gateway and VirtualService resources
This procedure uses the Istio Gateway
and VirtualService
resources to configure a gateway that was deployed by using gateway injection. The resources configure the gateway to expose a service in the mesh to traffic outside the mesh. Then, you expose the gateway to traffic outside the cluster by setting the Service
for the gateway to type LoadBalancer
.
Prerequisites
- You have installed an Istio gateway using gateway injection.
Procedure
Create namespace called
httpbin
by running the following command:$ oc create namespace httpbin
Enable sidecar injection in the namespace. If you are using the
InPlace
upgrade strategy, run the following command:$ oc label namespace httpbin istio-injection=enabled
NoteIf you are using the
RevisionBased
upgrade strategy, run the following commands:To find your
<revision-name>
, run the following command:$ oc get istiorevisions.sailoperator.io
Sample output:
NAME TYPE READY STATUS IN USE VERSION AGE default-v1-23-0 Local True Healthy True v1.23.0 3m33s
Label the namespace with the revision name to enable sidecar injection:
$ oc label namespace httpbin istio.io/rev=default-v1-23-0
Deploy a sample service named
httpbin
by running the following command:$ oc apply -n httpbin -f https://raw.githubusercontent.com/openshift-service-mesh/istio/refs/heads/master/samples/httpbin/httpbin.yaml
Create a YAML file named
httpbin-gw.yaml
that defines an IstioGateway
resource. This resource configures gateway proxies to expose port 80 (HTTP) for the host,httpbin.example.com
.apiVersion: networking.istio.io/v1 kind: Gateway metadata: name: httpbin-gateway namespace: httpbin spec: selector: istio: <gateway_name> 1 servers: - port: number: 80 name: http protocol: HTTP hosts: - httpbin.example.com 2
- 1
- Set the
selector
to the unique label or set of labels specified in the pod template of the gateway proxyDeployment
. By default, the IstioGateway
resource configuration will apply to matching gateway pods in all namespaces. - 2
- Using the
hosts
field, specify a list of addresses that can be used by clients when attempting to access a mesh service at the associated port.
Apply the YAML file by running the following command:
$ oc apply -f httpbin-gw.yaml
Create a YAML file named
httpbin-vs.yaml
for aVirtualService
. TheVirtualService
defines the rules that route traffic from the gateway proxy to thehttpbin
service.apiVersion: networking.istio.io/v1 kind: VirtualService metadata: name: httpbin namespace: httpbin spec: hosts: - httpbin.example.com 1 gateways: - httpbin-gateway 2 http: - match: - uri: prefix: /status - uri: prefix: /headers route: - destination: 3 port: number: 8000 host: httpbin
- 1
- Specify the
hosts
that the routing rules of theVirtualService
will be applied to. Thehosts
specified must be exposed by the IstioGateway
resource the VirtualService is bound to. - 2
- Bind the
VirtualService
to the IstioGateway
resource created in the previous step by adding theGateway
name to the list of gateways. - 3
- Route matching traffic to the
httpbin
service deployed earlier by defining adestination
that includes thehost
andport
of thehttpbin
Service
.
Apply the YAML file by running the following command:
$ oc apply -f httpbin-vs.yaml
For verification purposes, create a namespace for a
curl
client by running the following command:$ oc create namespace curl
Deploy the
curl
client by running the following command:$ oc apply -n curl -f https://raw.githubusercontent.com/openshift-service-mesh/istio/refs/heads/master/samples/curl/curl.yaml
Set a
CURL_POD
variable with the name of thecurl
pod by running the following command:$ CURL_POD=$(oc get pods -n curl -l app=curl -o jsonpath='{.items[*].metadata.name}')
Using the
curl
client, send a request to the/headers
endpoint of thehttpbin
application through the ingress gatewayService
resource. Set theHost
header of the request tohttpbin.example.com
to match the host that the IstioGateway
andVirtualService
resources specify. Run the followingcurl
command to send the request:$ oc exec $CURL_POD -n curl -- \ curl -s -I \ -H Host:httpbin.example.com \ <gateway_name>.<gateway_namespace>.svc.cluster.local/headers
The response should have a
200 OK HTTP
status indicating that the request was successful.Example output
HTTP/1.1 200 OK server: istio-envoy ...
Send a curl request to an endpoint that does not have a corresponding URI prefix match defined in the
httpbin
VirtualService
by running the following command:$ oc exec $CURL_POD -n curl -- \ curl -s -I \ -H Host:httpbin.example.com \ <gateway_name>.<gateway_namespace>.svc.cluster.local/get
The response should return a
404 Not Found
status. This is expected because the/get
endpoint does not have a matching URI prefix in thehttpbin
VirtualService
resource.Example output
HTTP/1.1 404 Not Found server: istio-envoy ...
Expose the gateway proxy to traffic outside the cluster by setting the
Service
type toLoadBalancer
:$ oc patch service <gateway_name> -n <gateway_namespace> -p '{"spec": {"type": "LoadBalancer"}}'
NoteA gateway can also be exposed to traffic outside the cluster by using OpenShift Routes. For more information, see "Exposing a gateway to traffic outside the cluster using OpenShift Routes".
Verify that
httpbin
service can be accessed from outside the cluster when using the external hostname or IP address of the gatewayService
resource. Ensure that you set theINGRESS_HOST
variable appropriately for the environment that your cluster is running in.If the cluster runs on AWS, set the
INGRESS_HOST
variable by running the following command:$ INGRESS_HOST=$(oc get service <gateway_name> -n <gateway_namespace> -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
If the cluster runs on GCP or Azure, set the
INGRESS_HOST
variable by running the following command:$ INGRESS_HOST=$(oc get service <gateway_name> -n <gateway_namespace> -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Send a
curl
request to thehttpbin
service using the host of the gateway by running the following command:$ curl -s -I -H Host:httpbin.example.com http://$INGRESS_HOST/headers
-
Verify that the response has the
HTTP/1.1 200 OK
status, which indicates that the request was successful.
Additional resources
2.2. About exposing services to traffic outside a cluster
To enable traffic from outside an OpenShift cluster to access services in a mesh, you must expose a gateway proxy by either setting its Service
type to LoadBalancer
or by using the OpenShift Router.
Using Kubernetes load balancing to handle incoming traffic directly through the inbound gateway can reduce latency associated with data encryption. By managing encryption at the inbound gateway, you avoid the intermediate decryption and re-encryption steps within the mesh that often add latency. This approach allows mesh traffic to be encrypted and decrypted only once, which is generally more efficient.
The OpenShift Router provides a standard approach for managing ingress traffic, and you can use the router to manage certificates for all cluster ingress traffic using the same methods. However, the OpenShift Router introduces an additional hop between the inbound traffic and the mesh applications. Typically, you route the traffic by decrypting it at the router and then re-encrypting it at the service mesh ingress gateway, which introduces latency.
2.2.1. Exposing a gateway to traffic outside the cluster by using OpenShift Routes
You can expose a gateway to traffic outside the cluster by using OpenShift Routes. This approach provides an alternative to using Kubernetes load balancer service when you have to expose gateways to traffic outside the cluster.
Prerequisites
- You have completed the procedure, Exposing a Service by using the Istio Gateway and VirtualService resources.
Procedure
Ensure that the
Service
type is set toClusterIP
by running the following command:$ oc patch service -n -p '{"spec": {"type": "ClusterIP"}}'
Create a YAML file named
httpbin-route.yaml
that defines aRoute
for thehttpbin
service.apiVersion: route.openshift.io/v1 kind: Route metadata: name: httpbin namespace: <gateway_namespace> spec: host: httpbin.example.com port: targetPort: http2 to: kind: Service name: <gateway_name> weight: 100 wildcardPolicy: None
Apply the YAML file by running the following command:
$ oc apply -f httpbin-route.yaml
Verify that
httpbin
service can be accessed from outside the cluster through the ingress router. Ensure that you set theINGRESS_HOST
variable appropriately for the environment that your cluster is running in.If the cluster runs on AWS, set the
INGRESS_HOST
variable by running the following command:$ INGRESS_HOST=$(oc get service router-default -n openshift-ingress -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
If the cluster runs on GCP or Azure, set the
INGRESS_HOST
variable by running the following command:$ INGRESS_HOST=$(oc get service router-default -n openshift-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Send a
curl
request to thehttpbin
service using the host of the ingress router by running the following command:$ curl -s -I -H Host:httpbin.example.com http://$INGRESS_HOST/headers
-
Verify that the response has the
HTTP/1.1 200 OK
status, which indicates that the request was successful.
Additional resources
Chapter 3. Directing outbound traffic through a gateway
Using Istio APIs, you can configure gateway proxies that were installed using gateway injection to direct traffic that is bound for an external service.
3.1. About directing egress traffic through a gateway
You can configure a gateway installed using gateway injection as an exit point for the traffic leaving a service mesh. In this configuration, the gateway acts as a forward proxy for requests sent to the services that are external to the mesh.
Configuring a gateway for egress traffic can help fulfill security requirements. For example, an egress gateway can be used in environments where traffic restrictions require that all traffic exiting a mesh flows through a dedicated set of nodes. Similarly, a gateway can be used when network policies prevent application nodes from directly accessing external services. In such scenarios, gateway proxies are deployed on dedicated egress nodes capable of accessing external services. These nodes can then be subjected to strict network policy enforcement or additional monitoring to enhance security.
To configure a gateway installed using gateway injection to direct the egress traffic, use a combination of the Istio ServiceEntry
, Gateway
, VirtualService
, and DestinationRule
resources. Use the ServiceEntry
resource to define the properties of an external service. The external service is added to the Istio service registry for the mesh. This enables you to apply Istio features, such as monitoring and routing rules, to the traffic exiting the mesh that is destined for an external service. Use the Gateway
, VirtualService
, and DestinationRule
resources to set up rules that route traffic from the mesh to the external service using the gateway proxy.
3.2. Directing egress traffic through a gateway using Istio APIs
Use Istio APIs to direct outbound HTTP traffic through a gateway that was installed using gateway injection.
Prerequisites
- You have installed a gateway using gateway injection.
Procedure
Create a namespace called
curl
by running the following command:$ oc create namespace curl
Depending on the update strategy you are using, enable sidecar injection in the namespace by running the appropriate commands:
If you are using the
InPlace
update strategy, run the following command:$ oc label namespace curl istio-injection=enabled
If you are using the
RevisionBased
update strategy, run the following commands:Display the revision name by running the following command:
$ oc get istiorevisions.sailoperator.io
Example output
NAME TYPE READY STATUS IN USE VERSION AGE default-v1-23-0 Local True Healthy True v1.23.0 3m33s
Label the namespace with the revision name to enable sidecar injection by running the following command:
$ oc label namespace curl istio.io/rev=default-v1-23-0
Deploy a
curl
application by running the following command:$ oc apply -n curl -f https://raw.githubusercontent.com/openshift-service-mesh/istio/refs/heads/master/samples/curl/curl.yaml
Export a
CURL_POD
environment variable that has been initialized with the name of the curl pod:$ export CURL_POD=$(oc get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')
Create a YAML file named
http-se.yaml
that directs traffic from the mesh to an external service. The following example defines aServiceEntry
for a URL.Example configuration
apiVersion: networking.istio.io/v1 kind: ServiceEntry metadata: name: egress-se namespace: curl spec: hosts: - docs.redhat.com ports: - number: 80 name: http-port protocol: HTTP location: MESH_EXTERNAL resolution: DNS
Apply the YAML file by running the following command:
$ oc apply -f http-se.yaml
Ensure the
ServiceEntry
configuration was applied correctly. Send an HTTP request to the host that you specified in the previous step by running the following command:$ oc exec "$CURL_POD" -n curl -c curl -- curl -sSL -o /dev/null -D - http://docs.redhat.com
This command should return HTTP status codes, such as
301
(redirect) or200
(success), indicating that the connection works.Create a YAML file named
http-gtw.yaml
that creates an egressGateway
and routes traffic from the mesh to the host specified for the external service.Example configuration
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: egress-gw namespace: <gateway_namespace> # Namespace where the egress gateway is deployed spec: selector: istio: <gateway_name> # Selects the egress-gateway instance to handle this traffic servers: - port: number: 80 name: http protocol: HTTP hosts: - docs.redhat.com # External service host, not a full URL. --- apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: egress-dr namespace: <gateway_namespace> # Namespace where the egress gateway is deployed spec: host: <gateway_name>.<gateway_namespace>.svc.cluster.local subsets: - name: rh-docs
Apply the YAML file by running the following command:
$ oc apply -f http-gtw.yaml
Create a YAML file named
http-vs.yaml
that sets up aVirtualService
to manage the flow of traffic from the application sidecars through the egress gateway to the external host.Example configuration
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: egress-vs namespace: curl # Namespace where the curl pod is running spec: hosts: - docs.redhat.com # External service host, not a full URL. gateways: - mesh - <gateway_namespace>/egress-gw # Egress gateway name defined in the file that you used in the previous step. http: - match: - gateways: - mesh port: 80 route: - destination: host: <gateway_name>.<gateway_namespace>.svc.cluster.local subset: rh-docs port: number: 80 weight: 100 - match: - gateways: - <gateway_namespace>/egress-gw # Egress gateway name defined in the file that you used in the previous step. port: 80 route: - destination: host: docs.redhat.com port: number: 80 weight: 100
Apply the YAML file by running the following command:
$ oc apply -f http-vs.yaml
Resend the HTTP request to the URL:
$ oc exec "$CURL_POD" -n curl -c curl -- curl -sSL -o /dev/null -D - http://docs.redhat.com
The terminal should display information similar to the following output:
Example output
... HTTP/1.1 301 Moved Permanently ... location: <example_url> ... HTTP/2 200 Content-Type: text/html; charset=utf-8
Ensure that the request was routed through the gateway by running the following command:
$ oc logs deployment/<gateway_name> -n <gateway_namespace> | tail -1
NoteAccess logging must be enabled for this verification step to work. You can enable access logging to the standard output by setting the
spec.values.meshConfig.accessLogFile
field to/dev/stdout
in the Istio resource.The terminal should display information similar to the following output:
Example output
[2024-11-07T14:35:52.428Z] "GET / HTTP/2" 301 - via_upstream - "-" 0 0 24 24 "10.128.2.30" "curl/8.11.0" "79551af2-341b-456d-b414-9220b487a03b" "docs.redhat.com" "23.55.176.201:80" outbound|80||docs.redhat.com 10.128.2.29:49766 10.128.2.29:80 10.128.2.30:38296 -