Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. Multi-cluster topologies

Multi-cluster topologies are useful for organizations with distributed systems or environments seeking enhanced scalability, fault tolerance, and regional redundancy.

6.1. About multi-cluster mesh topologies
Copy link

In a multi-cluster mesh topology, you install and manage a single Istio mesh across multiple OpenShift Container Platform clusters, enabling communication and service discovery between the services. Two factors determine the multi-cluster mesh topology: control plane topology and network topology. There are two options for each topology. Therefore, there are four possible multi-cluster mesh topology configurations.

  • Multi-Primary Single Network: Combines the multi-primary control plane topology and the single network network topology models.
  • Multi-Primary Multi-Network: Combines the multi-primary control plane topology and the multi-network network topology models.
  • Primary-Remote Single Network: Combines the primary-remote control plane topology and the single network network topology models.
  • Primary-Remote Multi-Network: Combines the primary-remote control plane topology and the multi-network network topology models.

6.1.1. Control plane topology models
Copy link

A multi-cluster mesh must use one of the following control plane topologies:

  • Multi-Primary: In this configuration, a control plane resides on every cluster. Each control plane observes the API servers in all of the other clusters for services and endpoints.
  • Primary-Remote: In this configuration, the control plane resides only on one cluster, called the primary cluster. No control plane runs on any of the other clusters, called remote clusters. The control plane on the primary cluster discovers services and endpoints and configures the sidecar proxies for the workloads in all clusters.

6.1.2. Network topology models
Copy link

A multi-cluster mesh must use one of the following network topologies:

  • Single Network: All clusters reside on the same network and there is direct connectivity between the services in all the clusters. There is no need to use gateways for communication between the services across cluster boundaries.
  • Multi-Network: Clusters reside on different networks and there is no direct connectivity between services. Gateways must be used to enable communication across network boundaries.

6.2. Multi-Cluster configuration overview
Copy link

To configure a multi-cluster topology you must perform the following actions:

  • Install the OpenShift Service Mesh Operator for each cluster.
  • Create or have access to root and intermediate certificates for each cluster.
  • Apply the security certificates for each cluster.
  • Install Istio for each cluster.

Create the root and intermediate certificate authority (CA) certificates for two clusters.

Prerequisites

  • You have OpenSSL installed locally.

Procedure

  1. Create the root CA certificate:

    1. Create a key for the root certificate by running the following command: 

      $ openssl genrsa -out root-key.pem 4096

    2. Create an OpenSSL configuration certificate file named 

      root-ca.conf
      for the root CA certificates:

      Example root certificate configuration file

      encrypt_key = no
prompt = no
utf8 = yes
default_md = sha256
default_bits = 4096
req_extensions = req_ext
x509_extensions = req_ext
distinguished_name = req_dn
[ req_ext ]
subjectKeyIdentifier = hash
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign
[ req_dn ]
O = Istio
CN = Root CA

    3. Create the certificate signing request by running the following command: 

      $ openssl req -sha256 -new -key root-key.pem \
  -config root-ca.conf \
  -out root-cert.csr

    4. Create a shared root certificate by running the following command: 

      $ openssl x509 -req -sha256 -days 3650 \
  -signkey root-key.pem \
  -extensions req_ext -extfile root-ca.conf \
  -in root-cert.csr \
  -out root-cert.pem

  2. Create the intermediate CA certificate for the East cluster:

    1. Create a directory named 

      east
      by running the following command: 

      $ mkdir east

    2. Create a key for the intermediate certificate for the East cluster by running the following command: 

      $ openssl genrsa -out east/ca-key.pem 4096

    3. Create an OpenSSL configuration file named 

      intermediate.conf
      in the 
      east/
      directory for the intermediate certificate of the East cluster. Copy the following example file and save it locally:

      Example configuration file

      [ req ]
encrypt_key = no
prompt = no
utf8 = yes
default_md = sha256
default_bits = 4096
req_extensions = req_ext
x509_extensions = req_ext
distinguished_name = req_dn
[ req_ext ]
subjectKeyIdentifier = hash
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign
subjectAltName=@san
[ san ]
DNS.1 = istiod.istio-system.svc
[ req_dn ]
O = Istio
CN = Intermediate CA
L = east

    4. Create a certificate signing request by running the following command: 

      $ openssl req -new -config east/intermediate.conf \
   -key east/ca-key.pem \
   -out east/cluster-ca.csr

    5. Create the intermediate CA certificate for the East cluster by running the following command: 

      $ openssl x509 -req -sha256 -days 3650 \
   -CA root-cert.pem \
   -CAkey root-key.pem -CAcreateserial \
   -extensions req_ext -extfile east/intermediate.conf \
   -in east/cluster-ca.csr \
   -out east/ca-cert.pem

    6. Create a certificate chain from the intermediate and root CA certificate for the east cluster by running the following command: 

      $ cat east/ca-cert.pem root-cert.pem > east/cert-chain.pem && cp root-cert.pem east

  3. Create the intermediate CA certificate for the West cluster:

    1. Create a directory named 

      west
      by running the following command: 

      $ mkdir west

    2. Create a key for the intermediate certificate for the West cluster by running the following command: 

      $ openssl genrsa -out west/ca-key.pem 4096

    3. Create an OpenSSL configuration file named 

      intermediate.conf
      in the 
      west/
      directory for for the intermediate certificate of the West cluster. Copy the following example file and save it locally:

      Example configuration file

      [ req ]
encrypt_key = no
prompt = no
utf8 = yes
default_md = sha256
default_bits = 4096
req_extensions = req_ext
x509_extensions = req_ext
distinguished_name = req_dn
[ req_ext ]
subjectKeyIdentifier = hash
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign
subjectAltName=@san
[ san ]
DNS.1 = istiod.istio-system.svc
[ req_dn ]
O = Istio
CN = Intermediate CA
L = west

    4. Create a certificate signing request by running the following command: 

      $ openssl req -new -config west/intermediate.conf \
   -key west/ca-key.pem \
   -out west/cluster-ca.csr

    5. Create the certificate by running the following command: 

      $ openssl x509 -req -sha256 -days 3650 \
   -CA root-cert.pem \
   -CAkey root-key.pem -CAcreateserial \
   -extensions req_ext -extfile west/intermediate.conf \
   -in west/cluster-ca.csr \
   -out west/ca-cert.pem

    6. Create the certificate chain by running the following command: 

      $ cat west/ca-cert.pem root-cert.pem > west/cert-chain.pem && cp root-cert.pem west

Apply root and intermediate certificate authority (CA) certificates to the clusters in a multi-cluster topology.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster.

Prerequisites

  • You have access to two OpenShift Container Platform clusters with external load balancer support.
  • You have created the root CA certificate and intermediate CA certificates for each cluster or someone has made them available for you.

Procedure

  1. Apply the certificates to the East cluster of the multi-cluster topology:

    1. Log in to East cluster by running the following command: 

      $ oc login -u https://<east_cluster_api_server_url>

    2. Set up the environment variable that contains the 

      oc
      command context for the East cluster by running the following command: 

      $ export CTX_CLUSTER1=$(oc config current-context)

    3. Create a project called 

      istio-system
      by running the following command: 

      $ oc get project istio-system --context "${CTX_CLUSTER1}" || oc new-project istio-system --context "${CTX_CLUSTER1}"

    4. Configure Istio to use 

      network1
      as the default network for the pods on the East cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" label namespace istio-system topology.istio.io/network=network1

    5. Create the CA certificates, certificate chain, and the private key for Istio on the East cluster by running the following command: 

      $ oc get secret -n istio-system --context "${CTX_CLUSTER1}" cacerts || oc create secret generic cacerts -n istio-system --context "${CTX_CLUSTER1}" \
  --from-file=east/ca-cert.pem \
  --from-file=east/ca-key.pem \
  --from-file=east/root-cert.pem \
  --from-file=east/cert-chain.pem
      Note

      If you followed the instructions in "Creating certificates for a multi-cluster mesh", your certificates will reside in the 

      east/
      directory. If your certificates reside in a different directory, modify the syntax accordingly.

  2. Apply the certificates to the West cluster of the multi-cluster topology:

    1. Log in to the West cluster by running the following command: 

      $ oc login -u https://<west_cluster_api_server_url>

    2. Set up the environment variable that contains the 

      oc
      command context for the West cluster by running the following command: 

      $ export CTX_CLUSTER2=$(oc config current-context)

    3. Create a project called 

      istio-system
      by running the following command: 

      $ oc get project istio-system --context "${CTX_CLUSTER2}" || oc new-project istio-system --context "${CTX_CLUSTER2}"

    4. Configure Istio to use 

      network2
      as the default network for the pods on the West cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" label namespace istio-system topology.istio.io/network=network2

    5. Create the CA certificate secret for Istio on the West cluster by running the following command: 

      $ oc get secret -n istio-system --context "${CTX_CLUSTER2}" cacerts || oc create secret generic cacerts -n istio-system --context "${CTX_CLUSTER2}" \
  --from-file=west/ca-cert.pem \
  --from-file=west/ca-key.pem \
  --from-file=west/root-cert.pem \
  --from-file=west/cert-chain.pem
      Note

      If you followed the instructions in "Creating certificates for a multi-cluster mesh", your certificates will reside in the 

      west/
      directory. If the certificates reside in a different directory, modify the syntax accordingly.

Next steps

Install Istio on all the clusters comprising the mesh topology.

6.3. Installing a multi-primary multi-network mesh
Copy link

Install Istio in the multi-primary multi-network topology on two OpenShift Container Platform clusters.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster.

You can adapt these instructions for a mesh spanning more than two clusters.

Prerequisites

  • You have installed the OpenShift Service Mesh 3 Operator on all of the clusters that include the mesh.
  • You have created certificates for the multi-cluster mesh.
  • You have applied certificates to the multi-cluster topology.
  • You have created an Istio Container Network Interface (CNI) resource.
  • You have 
    istioctl
    installed.
Important

In on-premise environments, such as those running on bare metal, OpenShift Container Platform clusters often do not include a native load-balancer capability. A service of type 

LoadBalancer
, such as the 
istio-eastwestgateway
, will not automatically be assigned an external IP address. To ensure the required external IP assignment for cross-cluster communication, cluster administrators must install and configure the MetalLB Operator. MetalLB is valuable in bare metal or bare metal-like infrastructures when fault-tolerant access to an application via an external IP address is necessary. Once deployed, MetalLB provides a platform-native load balancer. In addition to bare metal, the MetalLB Operator can offer load balancing for installations on other infrastructures that might lack native load-balancer capability, including:

  • VMware vSphere
  • IBM Z® and IBM® LinuxONE
  • IBM Z® and IBM® LinuxONE for Red Hat Enterprise Linux (RHEL) KVM
  • IBM Power®

For more information, see MetalLB Operator.

Procedure

  1. Create an 

    ISTIO_VERSION
    environment variable that defines the Istio version to install by running the following command: 

    $ export ISTIO_VERSION=1.24.3

  2. Install Istio on the East cluster:

    1. Create an 

      Istio
      resource on the East cluster by running the following command: 

      $ cat <<EOF | oc --context "${CTX_CLUSTER1}" apply -f -
apiVersion: sailoperator.io/v1
kind: Istio
metadata:
  name: default
spec:
  version: v${ISTIO_VERSION}
  namespace: istio-system
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: cluster1
      network: network1
EOF

    2. Wait for the control plane to return the 

      Ready
      status condition by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" wait --for condition=Ready istio/default --timeout=3m

    3. Create an East-West gateway on the East cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" apply -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/east-west-gateway-net1.yaml

    4. Expose the services through the gateway by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" apply -n istio-system -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/expose-services.yaml

  3. Install Istio on the West cluster:

    1. Create an 

      Istio
      resource on the West cluster by running the following command: 

      $ cat <<EOF | oc --context "${CTX_CLUSTER2}" apply -f -
apiVersion: sailoperator.io/v1
kind: Istio
metadata:
  name: default
spec:
  version: v${ISTIO_VERSION}
  namespace: istio-system
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: cluster2
      network: network2
EOF

    2. Wait for the control plane to return the 

      Ready
      status condition by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" wait --for condition=Ready istio/default --timeout=3m

    3. Create an East-West gateway on the West cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" apply -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/east-west-gateway-net2.yaml

    4. Expose the services through the gateway by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" apply -n istio-system -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/expose-services.yaml

  4. Create the 

    istio-reader-service-account
    service account for the East cluster by running the following command: 

    $ oc --context="${CTX_CLUSTER1}" create serviceaccount istio-reader-service-account -n istio-system

  5. Create the 

    istio-reader-service-account
    service account for the West cluster by running the following command: 

    $ oc --context="${CTX_CLUSTER2}" create serviceaccount istio-reader-service-account -n istio-system

  6. Add the 

    cluster-reader
    role to the East cluster by running the following command: 

    $ oc --context="${CTX_CLUSTER1}" adm policy add-cluster-role-to-user cluster-reader -z istio-reader-service-account -n istio-system

  7. Add the 

    cluster-reader
    role to the West cluster by running the following command: 

    $ oc --context="${CTX_CLUSTER2}" adm policy add-cluster-role-to-user cluster-reader -z istio-reader-service-account -n istio-system

  8. Install a remote secret on the East cluster that provides access to the API server on the West cluster by running the following command: 

    $ istioctl create-remote-secret \
  --context="${CTX_CLUSTER2}" \
  --name=cluster2 \
  --create-service-account=false | \
  oc --context="${CTX_CLUSTER1}" apply -f -

  9. Install a remote secret on the West cluster that provides access to the API server on the East cluster by running the following command: 

    $ istioctl create-remote-secret \
  --context="${CTX_CLUSTER1}" \
  --name=cluster1 \
  --create-service-account=false | \
  oc --context="${CTX_CLUSTER2}" apply -f -

6.3.1. Verifying a multi-cluster topology
Copy link

Deploy sample applications and verify traffic on a multi-cluster topology on two OpenShift Container Platform clusters.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster.

Prerequisites

  • You have installed the OpenShift Service Mesh Operator on all of the clusters that comprise the mesh.
  • You have completed "Creating certificates for a multi-cluster mesh".
  • You have completed "Applying certificates to a multi-cluster topology".
  • You have created an Istio Container Network Interface (CNI) resource.
  • You have 
    istioctl
    installed on the laptop you will use to run these instructions.
  • You have installed a multi-cluster topology.

Procedure

  1. Deploy sample applications on the East cluster:

    1. Create a sample application namespace on the East cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" get project sample || oc --context="${CTX_CLUSTER1}" new-project sample

    2. Label the application namespace to support sidecar injection by running the following command: 

      $ oc --context="${CTX_CLUSTER1}" label namespace sample istio-injection=enabled

    3. Deploy the 

      helloworld
      application:

      1. Create the 

        helloworld
        service by running the following command: 

        $ oc --context="${CTX_CLUSTER1}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/helloworld/helloworld.yaml \
  -l service=helloworld -n sample

      2. Create the 

        helloworld-v1
        deployment by running the following command: 

        $ oc --context="${CTX_CLUSTER1}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/helloworld/helloworld.yaml \
  -l version=v1 -n sample

    4. Deploy the 

      sleep
      application by running the following command: 

      $ oc --context="${CTX_CLUSTER1}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml -n sample

    5. Wait for the 

      helloworld
      application on the East cluster to return the 
      Ready
      status condition by running the following command: 

      $ oc --context="${CTX_CLUSTER1}" wait --for condition=available -n sample deployment/helloworld-v1

    6. Wait for the 

      sleep
      application on the East cluster to return the 
      Ready
      status condition by running the following command: 

      $ oc --context="${CTX_CLUSTER1}" wait --for condition=available -n sample deployment/sleep

  2. Deploy the sample applications on the West cluster:

    1. Create a sample application namespace on the West cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" get project sample || oc --context="${CTX_CLUSTER2}" new-project sample

    2. Label the application namespace to support sidecar injection by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" label namespace sample istio-injection=enabled

    3. Deploy the 

      helloworld
      application:

      1. Create the 

        helloworld
        service by running the following command: 

        $ oc --context="${CTX_CLUSTER2}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/helloworld/helloworld.yaml \
  -l service=helloworld -n sample

      2. Create the 

        helloworld-v2
        deployment by running the following command: 

        $ oc --context="${CTX_CLUSTER2}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/helloworld/helloworld.yaml \
  -l version=v2 -n sample

    4. Deploy the 

      sleep
      application by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" apply \
  -f https://raw.githubusercontent.com/openshift-service-mesh/istio/release-1.24/samples/sleep/sleep.yaml -n sample

    5. Wait for the 

      helloworld
      application on the West cluster to return the 
      Ready
      status condition by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" wait --for condition=available -n sample deployment/helloworld-v2

    6. Wait for the 

      sleep
      application on the West cluster to return the 
      Ready
      status condition by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" wait --for condition=available -n sample deployment/sleep

Verifying traffic flows between clusters

  1. For the East cluster, send 10 requests to the 

    helloworld
    service by running the following command: 

    $ for i in {0..9}; do \
  oc --context="${CTX_CLUSTER1}" exec -n sample deploy/sleep -c sleep -- curl -sS helloworld.sample:5000/hello; \
done

    Verify that you see responses from both clusters. This means version 1 and version 2 of the service can be seen in the responses.

  2. For the West cluster, send 10 requests to the 

    helloworld
    service: 

    $ for i in {0..9}; do \
  oc --context="${CTX_CLUSTER2}" exec -n sample deploy/sleep -c sleep -- curl -sS helloworld.sample:5000/hello; \
done

    Verify that you see responses from both clusters. This means version 1 and version 2 of the service can be seen in the responses.

After experimenting with the multi-cluster functionality in a development environment, remove the multi-cluster topology from all the clusters.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster.

Prerequisites

  • You have installed a multi-cluster topology.

Procedure

  1. Remove Istio and the sample applications from the East cluster of the development environment by running the following command: 

    $ oc --context="${CTX_CLUSTER1}" delete istio/default ns/istio-system ns/sample ns/istio-cni

  2. Remove Istio and the sample applications from the West cluster of development environment by running the following command: 

    $ oc --context="${CTX_CLUSTER2}" delete istio/default ns/istio-system ns/sample ns/istio-cni

Install Istio in a primary-remote multi-network topology on two OpenShift Container Platform clusters.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster. The East cluster is the primary cluster and the West cluster is the remote cluster.

You can adapt these instructions for a mesh spanning more than two clusters.

Prerequisites

  • You have installed the OpenShift Service Mesh 3 Operator on all of the clusters that comprise the mesh.
  • You have completed "Creating certificates for a multi-cluster mesh".
  • You have completed "Applying certificates to a multi-cluster topology".
  • You have created an Istio Container Network Interface (CNI) resource.
  • You have 
    istioctl
    installed on the laptop you will use to run these instructions.

Procedure

  1. Create an 

    ISTIO_VERSION
    environment variable that defines the Istio version to install by running the following command: 

    $ export ISTIO_VERSION=1.24.3

  2. Install Istio on the East cluster:

    1. Set the default network for the East cluster by running the following command: 

      $ oc --context="${CTX_CLUSTER1}" label namespace istio-system topology.istio.io/network=network1

    2. Create an 

      Istio
      resource on the East cluster by running the following command: 

      $ cat <<EOF | oc --context "${CTX_CLUSTER1}" apply -f -
apiVersion: sailoperator.io/v1
kind: Istio
metadata:
  name: default
spec:
  version: v${ISTIO_VERSION}
  namespace: istio-system
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: cluster1
      network: network1
      externalIstiod: true
      1 
      
EOF
      1
      This enables the control plane installed on the East cluster to serve as an external control plane for other remote clusters.

    3. Wait for the control plane to return the "Ready" status condition by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" wait --for condition=Ready istio/default --timeout=3m

    4. Create an East-West gateway on the East cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" apply -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/east-west-gateway-net1.yaml

    5. Expose the control plane through the gateway so that services in the West cluster can access the control plane by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" apply -n istio-system -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/expose-istiod.yaml

    6. Expose the application services through the gateway by running the following command: 

      $ oc --context "${CTX_CLUSTER1}" apply -n istio-system -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/expose-services.yaml

  3. Install Istio on the West cluster:

    1. Save the IP address of the East-West gateway running in the East cluster by running the following command: 

      $ export DISCOVERY_ADDRESS=$(oc --context="${CTX_CLUSTER1}" \
    -n istio-system get svc istio-eastwestgateway \
    -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

    2. Create an 

      Istio
      resource on the West cluster by running the following command: 

      $ cat <<EOF | oc --context "${CTX_CLUSTER2}" apply -f -
apiVersion: sailoperator.io/v1
kind: Istio
metadata:
  name: default
spec:
  version: v${ISTIO_VERSION}
  namespace: istio-system
  profile: remote
  values:
    istiodRemote:
      injectionPath: /inject/cluster/cluster2/net/network2
    global:
      remotePilotAddress: ${DISCOVERY_ADDRESS}
EOF

    3. Annotate the 

      istio-system
      namespace in the West cluster so that it is managed by the control plane in the East cluster by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" annotate namespace istio-system topology.istio.io/controlPlaneClusters=cluster1

    4. Set the default network for the West cluster by running the following command: 

      $ oc --context="${CTX_CLUSTER2}" label namespace istio-system topology.istio.io/network=network2

    5. Install a remote secret on the East cluster that provides access to the API server on the West cluster by running the following command: 

      $ istioctl create-remote-secret \
  --context="${CTX_CLUSTER2}" \
  --name=cluster2 | \
  oc --context="${CTX_CLUSTER1}" apply -f -

    6. Wait for the 

      Istio
      resource to return the "Ready" status condition by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" wait --for condition=Ready istio/default --timeout=3m

    7. Create an East-West gateway on the West cluster by running the following command: 

      $ oc --context "${CTX_CLUSTER2}" apply -f https://raw.githubusercontent.com/istio-ecosystem/sail-operator/main/docs/deployment-models/resources/east-west-gateway-net2.yaml
      Note

      Since the West cluster is installed with a remote profile, exposing the application services on the East cluster exposes them on the East-West gateways of both clusters.

6.5. Installing Kiali in a multi-cluster mesh
Copy link

Install Kiali in a multi-cluster mesh configuration on two OpenShift Container Platform clusters.

Note

In this procedure, 

CLUSTER1
is the East cluster and 
CLUSTER2
is the West cluster.

You can adapt these instructions for a mesh spanning more than two clusters.

Prerequisites

  • You have installed the latest Kiali Operator on each cluster.
  • Istio installed in a multi-cluster configuration on each cluster.
  • You have 
    istioctl
    installed on the laptop you can use to run these instructions.
  • You are logged in to the OpenShift Container Platform web console as a user with the 
    cluster-admin
    role.
  • You have configured a metrics store so that Kiali can query metrics from all the clusters. Kiali queries metrics and traces from their respective endpoints.

Procedure

  1. Install Kiali on the East cluster:

    1. Create a YAML file named 

      kiali.yaml
      that creates a namespace for the Kiali deployment.

      Example configuration

      apiVersion: kiali.io/v1alpha1
kind: Kiali
metadata:
  name: kiali
  namespace: istio-system
spec:
  version: default
  external_services:
    prometheus:
      auth:
        type: bearer
        use_kiali_token: true
      thanos_proxy:
        enabled: true
      url: https://thanos-querier.openshift-monitoring.svc.cluster.local:9091

      Note

      The endpoint for this example uses OpenShift Monitoring to configure metrics. For more information, see "Configuring OpenShift Monitoring with Kiali".

    2. Apply the YAML file on the East cluster by running the following command: 

      $ oc --context cluster1 apply -f kiali.yaml

      Example output

      kiali-istio-system.apps.example.com

  2. Ensure that the Kiali custom resource (CR) is ready by running the following command: 

    $ oc wait --context cluster1 --for=condition=Successful kialis/kiali -n istio-system --timeout=3m

    Example output

    kiali.kiali.io/kiali condition met

  3. Display your Kiali Route hostname. 

    $ oc --context cluster1 get route kiali -n istio-system -o jsonpath='{.spec.host}'

  4. Create a Kiali CR on the West cluster.

    Example configuration

    apiVersion: kiali.io/v1alpha1
kind: Kiali
metadata:
  name: kiali
  namespace: istio-system
spec:
  version: default
  auth:
    openshift:
      redirect_uris:
        # Replace kiali-route-hostname with the hostname from the previous step.
        - "https://{kiali-route-hostname}/api/auth/callback/cluster2"
  deployment:
    remote_cluster_resources_only: true

    The Kiali Operator creates the resources necessary for the Kiali server on the East cluster to connect to the West cluster. The Kiali server is not installed on the West cluster.

  5. Apply the YAML file on the West cluster by running the following command: 

    $ oc --context cluster2 apply -f kiali-remote.yaml

  6. Ensure that the Kiali CR is ready by running the following command: 

    $ oc wait --context cluster2 --for=condition=Successful kialis/kiali -n istio-system --timeout=3m

  7. Create a remote cluster secret so that Kiali installation in the East cluster can access the West cluster.

    1. Create a long lived API token bound to the kiali-service-account in the West cluster. Kiali uses this token to authenticate to the West cluster.

      Example configuration

      apiVersion: v1
kind: Secret
metadata:
  name: "kiali-service-account"
  namespace: "istio-system"
  annotations:
    kubernetes.io/service-account.name: "kiali-service-account"
type: kubernetes.io/service-account-token

    2. Apply the YAML file on the West cluster by running the following command: 

      $ oc --context cluster2 apply -f kiali-svc-account-token.yaml

    3. Create a 

      kubeconfig
      file and save it as a secret in the namespace on the East cluster where the Kiali deployment resides.

      To simplify this process, use the 

      kiali-prepare-remote-cluster.sh
      script to generate the 
      kubeconfig
      file by running the following 
      curl
      command: 

      $ curl -L -o kiali-prepare-remote-cluster.sh https://raw.githubusercontent.com/kiali/kiali/master/hack/istio/multicluster/kiali-prepare-remote-cluster.sh

    4. Modify the script to make it executeable by running the following command: 

      chmod +x kiali-prepare-remote-cluster.sh

    5. Execute the script so that it passes the East and West cluster contexts to the 

      kubeconfig
      file by running the following command: 

      $ ./kiali-prepare-remote-cluster.sh --kiali-cluster-context cluster1 --remote-cluster-context cluster2 --view-only false --kiali-resource-name kiali-service-account --remote-cluster-namespace istio-system --process-kiali-secret true --process-remote-resources false --remote-cluster-name cluster2
      Note

      Use the 

      --help
      option to display additional details about how to use the script.

  8. Trigger the reconciliation loop so that the Kiali Operator registers the remote secret that the CR contains by running the following command: 

    $ oc --context cluster1 annotate kiali kiali -n istio-system --overwrite kiali.io/reconcile="$(date)"

  9. Wait for Kiali resource to become ready by running the following command: 

    oc --context cluster1 wait --for=condition=Successful --timeout=2m kialis/kiali -n istio-system

  10. Wait for Kiali server to become ready by running the following command: 

    oc --context cluster1 rollout status deployments/kiali -n istio-system

  11. Log in to Kiali.

    1. When you first access Kiali, log in to the cluster that contains the Kiali deployment. In this example, access the 
      East
      cluster.

    2. Display the hostname of the Kiali route by running the following command: 

      oc --context cluster1 get route kiali -n istio-system -o jsonpath='{.spec.host}'
    3. Navigate to the Kiali URL in your browser: https://<your-kiali-route-hostname>.

  12. Log in to the West cluster through Kiali.

    In order to see other clusters in the Kiali UI, you must first login as a user to those clusters through Kiali.

    1. Click on the user profile dropdown in the top right hand menu.
    2. Select Login to West. You are redirected to an OpenShift login page and prompted for credentials for the West cluster.

  13. Verify that Kiali shows information from both clusters.

    1. Click Overview and verify that you can see namespaces from both clusters.
    2. Click Navigate and verify that you see both clusters on the mesh graph.
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top