Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 7. Deploying multiple service meshes on a single cluster

You can use the Red Hat OpenShift Service Mesh to operate many service meshes in a single cluster, with each mesh managed by a separate control plane. Using discovery selectors and revisions prevents conflicts between control planes.

7.1. About deploying multiple control planes
Copy link

To configure a cluster to host two control planes, set up separate Istio resources with unique names in independent Istio system namespaces. Assign a unique revision name to each Istio resource to identify the control planes, workloads, or namespaces it manages. Apply these revision names using injection or 

istio.io/rev
labels to specify which control plane injects the sidecar proxy into application pods.

Each 

Istio
resource must also configure discovery selectors to specify which namespaces the Istio control plane observes. Only namespaces with labels that match the configured discovery selectors can join the mesh. Additionally, discovery selectors determine which control plane creates the 
istio-ca-root-cert
config map in each namespace, which is used to encrypt traffic between services with mutual TLS within each mesh.

When adding an additional Istio control plane to a cluster with an existing control plane, ensure that the existing 

Istio
instance has discovery selectors configured to avoid overlapping with the new control plane.

Note

Only one 

IstioCNI
resource is shared by all control planes in a cluster, and you must update this resource independent of other cluster resources.

You can use discovery selectors to limit the visibility of an Istio control plane to specific namespaces in a cluster. By combining discovery selectors with control plane revisions, you can deploy multiple control planes in a single cluster, ensuring that each control plane manages only its assigned namespaces. This approach avoids conflicts between control planes and enables soft multi-tenancy for service meshes.

7.2.1. Deploying the first control plane
Copy link

You deploy the first control plane by creating its assigned namespace.

Prerequisites

  • You have installed the OpenShift Service Mesh operator.

  • You have created an Istio Container Network Interface (CNI) resource.

    Note

    You can run the following command to check for existing 

    Istio
    instances: 

    $ oc get istios
  • You have installed the 
    istioctl
    binary on your localhost.
Note

You can have extended support for more than two control planes. The maximum number of service meshes in a single cluster depends on the available cluster resources.

Procedure

  1. Create the namespace for the first Istio control plane called 

    istio-system-1
    by running the following command: 

    $ oc new-project istio-system-1

  2. Add the following label to the first namespace, which is used with the Istio 

    discoverySelectors
    field by running the following command: 

    $ oc label namespace istio-system-1 istio-discovery=mesh-1

  3. Create a YAML file named 

    istio-1.yaml
    with the name 
    mesh-1
    and the 
    discoverySelector
    as 
    mesh-1
    :

    Example configuration

    kind: Istio
apiVersion: sailoperator.io/v1
metadata:
  name: mesh-1
spec:
  namespace: istio-system-1
  values:
    meshConfig:
      discoverySelectors:
        - matchLabels:
            istio-discovery: mesh-1
# ...

  4. Create the first 

    Istio
    resource by running the following command: 

    $ oc apply -f istio-1.yaml

  5. To restrict workloads in 

    mesh-1
    from communicating freely with decrypted traffic between meshes, deploy a 
    PeerAuthentication
    resource to enforce mutual TLS (mTLS) traffic within the 
    mesh-1
    data plane. Apply the 
    PeerAuthentication
    resource in the 
    istio-system-1
    namespace by using a configuration file, such as 
    peer-auth-1.yaml
    : 

    $ oc apply -f peer-auth-1.yaml

    Example configuration

    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: "mesh-1-peerauth"
  namespace: "istio-system-1"
spec:
  mtls:
    mode: STRICT

7.2.2. Deploying the second control plane
Copy link

After deploying the first control plane, you can deploy the second control plane by creating its assigned namespace.

Procedure

  1. Create a namespace for the second Istio control plane called 

    istio-system-2
    by running the following command: 

    $ oc new-project istio-system-2

  2. Add the following label to the second namespace, which is used with the Istio 

    discoverySelectors
    field by running the following command: 

    $ oc label namespace istio-system-2 istio-discovery=mesh-2

  3. Create a YAML file named 

    istio-2.yaml
    :

    Example configuration

    kind: Istio
apiVersion: sailoperator.io/v1
metadata:
  name: mesh-2
spec:
  namespace: istio-system-2
  values:
    meshConfig:
      discoverySelectors:
        - matchLabels:
            istio-discovery: mesh-2
# ...

  4. Create the second 

    Istio
    resource by running the following command: 

    $ oc apply -f istio-2.yaml

  5. Deploy a policy for workloads in the 

    istio-system-2
    namespace to only accept mutual TLS traffic 
    peer-auth-2.yaml
    by running the following command: 

    $ oc apply -f peer-auth-2.yaml

    Example configuration

    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: "mesh-2-peerauth"
  namespace: "istio-system-2"
spec:
  mtls:
    mode: STRICT

7.2.3. Verifying multiple control planes
Copy link

Verify that both of the Istio control planes are deployed and running properly. You can validate that the 

istiod
pod is successfully running in each Istio system namespace.

  1. Verify that the workloads are assigned to the control plane in 

    istio-system-1
    by running the following command: 

    $ oc get pods -n istio-system-1

    Example output

    NAME                            READY   STATUS    RESTARTS   AGE
istiod-mesh-1-b69646b6f-kxrwk   1/1     Running   0          4m14s

  2. Verify that the workloads are assigned to the control plane in 

    istio-system-2
    by running the following command: 

    $ oc get pods -n istio-system-2

    Example output

    NAME                            READY   STATUS    RESTARTS   AGE
istiod-mesh-2-8666fdfc6-mqp45   1/1     Running   0          118s

7.3. Deploy application workloads in each mesh
Copy link

To deploy application workloads, assign each workload to a separate namespace.

Procedure

  1. Create an application namespace called 

    app-ns-1
    by running the following command: 

    $ oc create namespace app-ns-1

  2. To ensure that the namespace is discovered by the first control plane, add the 

    istio-discovery=mesh-1
    label by running the following command: 

    $ oc label namespace app-ns-1 istio-discovery=mesh-1

  3. To enable sidecar injection into all the pods by default while ensuring that pods in this namespace are mapped to the first control plane, add the 

    istio.io/rev=mesh-1
    label to the namespace by running the following command: 

    $ oc label namespace app-ns-1 istio.io/rev=mesh-1

  4. Optional: You can verify the 

    mesh-1
    revision name by running the following command: 

    $ oc get istiorevisions

  5. Deploy the 

    sleep
    and 
    httpbin
    applications by running the following command: 

    $ oc apply -n app-ns-1 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml

  6. Wait for the 

    httpbin
    and 
    sleep
    pods to run with sidecars injected by running the following command: 

    $ oc get pods -n app-ns-1

    Example output

    NAME                       READY   STATUS    RESTARTS   AGE
httpbin-7f56dc944b-kpw2x   2/2     Running   0          2m26s
sleep-5577c64d7c-b5wd2     2/2     Running   0          91m

  7. Create a second application namespace called 

    app-ns-2
    by running the following command: 

    $ oc create namespace app-ns-2

  8. Create a third application namespace called 

    app-ns-3
    by running the following command: 

    $ oc create namespace app-ns-3

  9. Add the label 

    istio-discovery=mesh-2
    to both namespaces and the revision label 
    mesh-2
    to match the discovery selector of the second control plane by running the following command: 

    $ oc label namespace app-ns-2 app-ns-3 istio-discovery=mesh-2 istio.io/rev=mesh-2

  10. Deploy the 

    sleep
    and 
    httpbin
    applications to the 
    app-ns-2
    namespace by running the following command: 

    $ oc apply -n app-ns-2 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml

  11. Deploy the 

    sleep
    and 
    httpbin
    applications to the 
    app-ns-3
    namespace by running the following command: 

    $ oc apply -n app-ns-3 \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml \
   -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/httpbin/httpbin.yaml

  12. Optional: Use the following command to wait for a deployment to be available: 

    $ oc wait deployments -n app-ns-2 --all --for condition=Available

Verification

  1. Verify that each application workload is managed by its assigned control plane by using the 

    istioctl ps
    command after deploying the applications:

    1. Verify that the workloads are assigned to the control plane in 

      istio-system-1
      by running the following command: 

      $ istioctl ps -i istio-system-1

      Example output

      NAME                                  CLUSTER        CDS              LDS              EDS              RDS              ECDS        ISTIOD                            VERSION
httpbin-7f56dc944b-vwfm5.app-ns-1     Kubernetes     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     IGNORED     istiod-mesh-1-b69646b6f-kxrwk     1.23.0
sleep-5577c64d7c-d675f.app-ns-1       Kubernetes     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     SYNCED (11m)     IGNORED     istiod-mesh-1-b69646b6f-kxrwk     1.23.0

    2. Verify that the workloads are assigned to the control plane in 

      istio-system-2
      by running the following command: 

      $ istioctl ps -i istio-system-2

      Example output

      NAME                                  CLUSTER        CDS                LDS                EDS                RDS                ECDS        ISTIOD                            VERSION
httpbin-7f56dc944b-54gjs.app-ns-3     Kubernetes     SYNCED (3m59s)     SYNCED (3m59s)     SYNCED (3m59s)     SYNCED (3m59s)     IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
httpbin-7f56dc944b-gnh72.app-ns-2     Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
sleep-5577c64d7c-k9mxz.app-ns-2       Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0
sleep-5577c64d7c-m9hvm.app-ns-3       Kubernetes     SYNCED (4m1s)      SYNCED (4m1s)      SYNCED (3m59s)     SYNCED (4m1s)      IGNORED     istiod-mesh-2-8666fdfc6-mqp45     1.23.0

  2. Verify that the application connectivity is restricted to workloads within their respective mesh:

    1. Send a request from the 

      sleep
      pod in 
      app-ns-1
      to the 
      httpbin
      service in 
      app-ns-2
      to check that the communication fails by running the following command: 

      $ oc -n app-ns-1 exec deploy/sleep -c sleep -- curl -sIL http://httpbin.app-ns-2.svc.cluster.local:8000

      The 

      PeerAuthentication
      resources created earlier enforce mutual TLS (mTLS) traffic in 
      STRICT
      mode within each mesh. Each mesh uses its own root certificate, managed by the 
      istio-ca-root-cert
      config map, which prevents communication between meshes. The output indicates a communication failure, similar to the following example:

      Example output

      HTTP/1.1 503 Service Unavailable
content-length: 95
content-type: text/plain
date: Wed, 16 Oct 2024 12:05:37 GMT
server: envoy

    2. Confirm that the communication works by sending a request from the 

      sleep
      pod to the 
      httpbin
      service that are present in the 
      app-ns-2
      namespace which is managed by 
      mesh-2
      . Run the following command: 

      $ oc -n app-ns-2 exec deploy/sleep -c sleep -- curl -sIL http://httpbin.app-ns-3.svc.cluster.local:8000

      Example output

      HTTP/1.1 200 OK
access-control-allow-credentials: true
access-control-allow-origin: *
content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' camo.githubusercontent.com
content-type: text/html; charset=utf-8
date: Wed, 16 Oct 2024 12:06:30 GMT
x-envoy-upstream-service-time: 8
server: envoy
transfer-encoding: chunked

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top