Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. Deploying multiple service meshes on a single cluster

You can use the Red Hat OpenShift Service Mesh to operate multiple service meshes in a single cluster, with each mesh managed by a separate control plane. Using discovery selectors and revisions prevents conflicts between control planes.

6.1. Prerequisites
Copy link

  • You have installed the OpenShift Service Mesh operator.

  • You have created an Istio Container Network Interface (CNI) resource.

    Note

    You can run the following command to check for existing Istio instances: 

    $ oc get istios
    Copy to Clipboard Toggle word wrap
  • You have installed the istioctl binary on your localhost.

6.2. About deploying multiple control planes
Copy link

To configure a cluster to host two control planes, set up separate Istio resources with unique names in independent Istio system namespaces. Assign a unique revision name to each Istio resource to identify the control planes, workloads, or namespaces it manages. Apply these revision names using injection or istio.io/rev labels to specify which control plane injects the sidecar proxy into application pods.

Each Istio resource must also configure discovery selectors to specify which namespaces the Istio control plane observes. Only namespaces with labels that match the configured discovery selectors can join the mesh. Additionally, discovery selectors determine which control plane creates the istio-ca-root-cert config map in each namespace, which is used to encrypt traffic between services with mutual TLS within each mesh.

When adding an additional Istio control plane to a cluster with an existing control plane, ensure that the existing Istio instance has discovery selectors configured to avoid overlapping with the new control plane.

Note

Only one IstioCNI resource is shared by all control planes in a cluster, and you must update this resource independent of other cluster resources.

You can use discovery selectors to limit the visibility of an Istio control plane to specific namespaces in a cluster. By combining discovery selectors with control plane revisions, you can deploy multiple control planes in a single cluster, ensuring that each control plane manages only its assigned namespaces. This approach avoids conflicts between control planes and enables soft multi-tenancy for service meshes.

6.4. Deploying multiple control planes
Copy link

You can have extended support for more than two control planes. The maximum number of service meshes in a single cluster depends on the available cluster resources.

6.4.1. Deploying the first control plane
Copy link

You deploy the first control plane by creating its assigned namespace.

Procedure

  1. Create the namespace for the first Istio control plane called istio-system-1 by running the following command: 

    $ oc new-project istio-system-1
    Copy to Clipboard Toggle word wrap

  2. Add the following label to the first namespace, which is used with the Istio discoverySelectors field by running the following command: 

    $ oc label namespace istio-system-1 istio-discovery=mesh-1
    Copy to Clipboard Toggle word wrap

  3. Create a YAML file named istio-1.yaml with the name mesh-1 and the discoverySelector as mesh-1:

    Example configuration

    kind: Istio
apiVersion: sailoperator.io/v1
metadata:
  name: mesh-1
spec:
  namespace: istio-system-1
  values:
    meshConfig:
      discoverySelectors:
        - matchLabels:
            istio-discovery: mesh-1
# ...
    Copy to Clipboard Toggle word wrap

  4. Create the first Istio resource by running the following command: 

    $ oc apply -f istio-1.yaml
    Copy to Clipboard Toggle word wrap

  5. To restrict workloads in mesh-1 from communicating freely with decrypted traffic between meshes, deploy a PeerAuthentication resource to enforce mutual TLS (mTLS) traffic within the mesh-1 data plane. Apply the PeerAuthentication resource in the istio-system-1 namespace by using a configuration file, such as peer-auth-1.yaml: 

    $ oc apply -f peer-auth-1.yaml
    Copy to Clipboard Toggle word wrap

    Example configuration

    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: "mesh-1-peerauth"
  namespace: "istio-system-1"
spec:
  mtls:
    mode: STRICT
    Copy to Clipboard Toggle word wrap

6.4.2. Deploying the second control plane
Copy link

After deploying the first control plane, you can deploy the second control plane by creating its assigned namespace.

Procedure

  1. Create a namespace for the second Istio control plane called istio-system-2 by running the following command: 

    $ oc new-project istio-system-2
    Copy to Clipboard Toggle word wrap

  2. Add the following label to the second namespace, which is used with the Istio discoverySelectors field by running the following command: 

    $ oc label namespace istio-system-2 istio-discovery=mesh-2
    Copy to Clipboard Toggle word wrap

  3. Create a YAML file named istio-2.yaml:

    Example configuration

    kind: Istio
apiVersion: sailoperator.io/v1
metadata:
  name: mesh-2
spec:
  namespace: istio-system-2
  values:
    meshConfig:
      discoverySelectors:
        - matchLabels:
            istio-discovery: mesh-2
# ...
    Copy to Clipboard Toggle word wrap

  4. Create the second Istio resource by running the following command: 

    $ oc apply -f istio-2.yaml
    Copy to Clipboard Toggle word wrap

  5. Deploy a policy for workloads in the istio-system-2 namespace to only accept mutual TLS traffic peer-auth-2.yaml by running the following command: 

    $ oc apply -f peer-auth-2.yaml
    Copy to Clipboard Toggle word wrap

    Example configuration

    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: "mesh-2-peerauth"
  namespace: "istio-system-2"
spec:
  mtls:
    mode: STRICT
    Copy to Clipboard Toggle word wrap

6.4.3. Verifying multiple control planes
Copy link

Verify that both of the Istio control planes are deployed and running properly. You can validate that the istiod pod is successfully running in each Istio system namespace.

  1. Verify that the workloads are assigned to the control plane in istio-system-1 by running the following command: 

    $ oc get pods -n istio-system-1
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                            READY   STATUS    RESTARTS   AGE
istiod-mesh-1-b69646b6f-kxrwk   1/1     Running   0          4m14s
    Copy to Clipboard Toggle word wrap

  2. Verify that the workloads are assigned to the control plane in istio-system-2 by running the following command: 

    $ oc get pods -n istio-system-2
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                            READY   STATUS    RESTARTS   AGE
istiod-mesh-2-8666fdfc6-mqp45   1/1     Running   0          118s
    Copy to Clipboard Toggle word wrap

6.5. Deploy application workloads in each mesh
Copy link

To deploy application workloads, assign each workload to a separate namespace.

Procedure

  1. Create an application namespace called app-ns-1 by running the following command: 

    $ oc create namespace app-ns-1
    Copy to Clipboard Toggle word wrap

  2. To ensure that the namespace is discovered by the first control plane, add the istio-discovery=mesh-1 label by running the following command: 

    $ oc label namespace app-ns-1 istio-discovery=mesh-1
    Copy to Clipboard Toggle word wrap

  3. To enable sidecar injection into all the pods by default while ensuring that pods in this namespace are mapped to the first control plane, add the istio.io/rev=mesh-1 label to the namespace by running the following command: 

    $ oc label namespace app-ns-1 istio.io/rev=mesh-1
    Copy to Clipboard Toggle word wrap

  4. Optional: You can verify the mesh-1 revision name by running the following command: 

    $ oc get istiorevisions
    Copy to Clipboard Toggle word wrap

  5. Deploy the sleep and httpbin applications by running the following command: 

    $ oc apply -n app-ns-1 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml
    Copy to Clipboard Toggle word wrap

  6. Wait for the httpbin and sleep pods to run with sidecars injected by running the following command: 

    $ oc get pods -n app-ns-1
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                       READY   STATUS    RESTARTS   AGE
httpbin-7f56dc944b-kpw2x   2/2     Running   0          2m26s
sleep-5577c64d7c-b5wd2     2/2     Running   0          91m
    Copy to Clipboard Toggle word wrap

  7. Create a second application namespace called app-ns-2 by running the following command: 

    $ oc create namespace app-ns-2
    Copy to Clipboard Toggle word wrap

  8. Create a third application namespace called app-ns-3 by running the following command: 

    $ oc create namespace app-ns-3
    Copy to Clipboard Toggle word wrap

  9. Add the label istio-discovery=mesh-2 to both namespaces and the revision label mesh-2 to match the discovery selector of the second control plane by running the following command: 

    $ oc label namespace app-ns-2 app-ns-3 istio-discovery=mesh-2 istio.io/rev=mesh-2
    Copy to Clipboard Toggle word wrap

  10. Deploy the sleep and httpbin applications to the app-ns-2 namespace by running the following command: 

    $ oc apply -n app-ns-2 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml
    Copy to Clipboard Toggle word wrap

  11. Deploy the sleep and httpbin applications to the app-ns-3 namespace by running the following command: 

    $ oc apply -n app-ns-3 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml
    Copy to Clipboard Toggle word wrap

  12. Optional: Use the following command to wait for a deployment to be available: 

    $ oc wait deployments -n app-ns-2 --all --for condition=Available
    Copy to Clipboard Toggle word wrap

Verification

  1. Verify that each application workload is managed by its assigned control plane by using the istioctl ps command after deploying the applications:

    1. Verify that the workloads are assigned to the control plane in istio-system-1 by running the following command: 

      $ istioctl ps -i istio-system-1
      Copy to Clipboard Toggle word wrap

      Example output

      NAME                                  CLUSTER        CDS              LDS              EDS              RDS              ECDS        ISTIOD                            VERSION
httpbin-7f56dc944b-vwfm5.app-ns-1     Kubernetes     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     IGNORED     istiod-mesh-1-b69646b6f-kxrwk     1.23.0
sleep-5577c64d7c-d675f.app-ns-1       Kubernetes     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     IGNORED     istiod-mesh-1-b69646b6f-kxrwk     1.23.0
      Copy to Clipboard Toggle word wrap

    2. Verify that the workloads are assigned to the control plane in istio-system-2 by running the following command: 

      $ istioctl ps -i istio-system-2
      Copy to Clipboard Toggle word wrap

      Example output

      NAME                                  CLUSTER        CDS                LDS                EDS                RDS                ECDS        ISTIOD                            VERSION
httpbin-7f56dc944b-54gjs.app-ns-3     Kubernetes     SYNCED (3m59s)     SYNCED (3m59s)     SYNCED (3m59s)     SYNCED (3m59s)     IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
httpbin-7f56dc944b-gnh72.app-ns-2     Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
sleep-5577c64d7c-k9mxz.app-ns-2       Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
sleep-5577c64d7c-m9hvm.app-ns-3       Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
      Copy to Clipboard Toggle word wrap

  2. Verify that the application connectivity is restricted to workloads within their respective mesh:

    1. Send a request from the sleep pod in app-ns-1 to the httpbin service in app-ns-2 to check that the communication fails by running the following command: 

      $ oc -n app-ns-1 exec deploy/sleep -c sleep -- curl -sIL http://httpbin.app-ns-2.svc.cluster.local:8000
      Copy to Clipboard Toggle word wrap

      The PeerAuthentication resources created earlier enforce mutual TLS (mTLS) traffic in STRICT mode within each mesh. Each mesh uses its own root certificate, managed by the istio-ca-root-cert config map, which prevents communication between meshes. The output indicates a communication failure, similar to the following example:

      Example output

      HTTP/1.1 503 Service Unavailable
content-length: 95
content-type: text/plain
date: Wed, 16 Oct 2024 12:05:37 GMT
server: envoy
      Copy to Clipboard Toggle word wrap

    2. Confirm that the communication works by sending a request from the sleep pod to the httpbin service that are present in the app-ns-2 namespace which is managed by mesh-2. Run the following command: 

      $ oc -n app-ns-2 exec deploy/sleep -c sleep -- curl -sIL http://httpbin.app-ns-3.svc.cluster.local:8000
      Copy to Clipboard Toggle word wrap

      Example output

      HTTP/1.1 200 OK
access-control-allow-credentials: true
access-control-allow-origin: *
content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' camo.githubusercontent.com
content-type: text/html; charset=utf-8
date: Wed, 16 Oct 2024 12:06:30 GMT
x-envoy-upstream-service-time: 8
server: envoy
transfer-encoding: chunked
      Copy to Clipboard Toggle word wrap

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat