Chapter 3. Multitenant migration guide

Procedure

1. Create your Istio resource.

   Example Istio resource

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
     name: istio-tenant-a
   spec:
     namespace: istio-system-tenant-a 

   1

   
     version: v1.24.3
     values:
       meshConfig:
         discoverySelectors: 

   2

   
           - matchLabels:
               tenant: tenant-a
         extensionProviders:  

   3

   
           - name: prometheus
             prometheus: {}
           - name: otel
             opentelemetry:
               port: 4317
               service: otel-collector.opentelemetrycollector-3.svc.cluster.local

   Copy to Clipboard Toggle word wrap

   1

   The spec.namespace field in your Istio resource must be the same namespace as your ServiceMeshControlPlane resource. If you set the spec.namespace field in your Istio resource to a different namespace than your ServiceMeshControlPlane resource, the migration does not complete successfully.

   2

   By default, control planes watch the entire cluster. When managing multiple control planes on a single cluster, you must narrow the scope of each control plane by setting discoverySelectors fields. In this example, the label tenant-a is used, but you can use any label or combination of labels.

   3

   Optional: If you are migrating metrics and tracing, update the extensionProviders fields according to your tracing and metrics configurations.

2. Add your tenant label to each one of your dataplane namespaces by running the following command for each dataplane namespace:

   $ oc label ns bookinfo tenant=tenant-a

   Copy to Clipboard Toggle word wrap
   Note

   With OpenShift Service Mesh 2.6, namespaces were enrolled into the mesh by adding them to the ServiceMeshMemberRoll resource. In OpenShift Service Mesh 3, you must label each one of your dataplane namespaces to match your discoverySelectors fields.

Procedure

1. Find the current IstioRevision for your OpenShift Service Mesh 3.0 control plane by running the following command:

   $ oc get istios istio-tenant-a

   Copy to Clipboard Toggle word wrap

   Example output

   NAME             REVISIONS   READY   IN USE   ACTIVE REVISION   STATUS    VERSION   AGE
   istio-tenant-a   1           1       0        istio-tenant-a    Healthy   v1.24.3   30s

   Copy to Clipboard Toggle word wrap

   Note

   The naming format of your revisions depends on which upgrade strategy you choose for your Istio instance.

2. Copy the ACTIVE REVISION to use as your istio.io/rev label in the next step.

3. Update injection labels on the dataplane namespace by running the following command:

   $ oc label ns bookinfo istio.io/rev=istio-tenant-a maistra.io/ignore-namespace="true" --overwrite=true

   Copy to Clipboard Toggle word wrap

   This adds the following labels to the namespace:

   1. The istio.io/rev: istio-tenant-a label: Ensures that any new pods that get created in that namespace connect to the OpenShift Service Mesh 3.0 proxy.

   2. The maistra.io/ignore-namespace: "true" label: Disables sidecar injection for OpenShift Service Mesh 2.6 proxies in the namespace so OpenShift Service Mesh 2.6 stops injecting proxies in this namespace, and any new proxies are injected by OpenShift Service Mesh 3.0. Without this label, the OpenShift Service Mesh 2.6 injection webhook tries to inject the pod and the injected sidecar proxy will refuse to start since it will have both the OpenShift Service Mesh 2.6 and the OpenShift Service Mesh 3.0 Container Network Interface(CNI) annotations.

      Note

      Once you apply the maistra.io/ignore-namespace label, any new pod that gets created in the namespace will connect to the OpenShift Service Mesh 3.0 proxy. Workloads can still communicate with each other regardless of which controlplane they are connected to.

4. Restart the workloads by using one of the following options:

   1. To restart all the workloads at once so that the new pods are injected with the OpenShift Service Mesh 3.0 proxy, run the following command:

      Example command for bookinfo application

      $ oc rollout restart deployments -n bookinfo

      Copy to Clipboard Toggle word wrap

   2. To restart each workload individually, run the following command for each workload:

      Example command with bookinfo application

      $ oc rollout restart deployments productpage-v1 -n bookinfo

      Copy to Clipboard Toggle word wrap

5. Wait for the productpage application to restart by running the following command:

   $ oc rollout status deployment productpage-v1 -n bookinfo

   Copy to Clipboard Toggle word wrap

Verification

1. Check that your workload is connected to the new control plane.

   1. Fetch the list of proxies that are still connected to the OpenShift Service Mesh 2.6 control plane with the istioctl tool by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision basic

      Copy to Clipboard Toggle word wrap

      In this example, basic is the name of your ServiceMeshControlPlane:

      Example output

         NAME                                              CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                            VERSION
         details-v1-7b49464bc-zr7nr.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         ratings-v1-d6f449f59-9rds2.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v1-686cd989df-9x59z.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v2-785b8b48fc-l7xkj.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v3-67889ffd49-7bhxn.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8

      Copy to Clipboard Toggle word wrap

   2. View the list proxies that have been migrated to the new OpenShift Service Mesh 3.0 control plane by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision istio-tenant-a

      Copy to Clipboard Toggle word wrap

      Example output

      NAME                                         CLUSTER        CDS        LDS        EDS        RDS        ECDS     ISTIOD                      VERSION
      productpage-v1-7745c5cc94-wpvth.bookinfo     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED              istiod-5bbf98dccf-n8566     1.24.3

      Copy to Clipboard Toggle word wrap

2. Verify your application is still working correctly. For the bookinfo application, run the following command:

   $ oc exec -it -n bookinfo deployments/productpage-v1 -c istio-proxy -- curl localhost:9080/productpage

   Copy to Clipboard Toggle word wrap

Procedure

1. Check that your OpenShift Service Mesh 2 ServiceMeshControlPlane is configured with the cert-manager-tool:

   Example ServiceMeshControlPlane cert-manager configuration

   apiVersion: maistra.io/v2
   kind: ServiceMeshControlPlane
   metadata:
     name: basic
     namespace: istio-system
   spec:
     ...
     security:
       certificateAuthority:
         cert-manager:
           address: cert-manager-istio-csr.istio-system.svc:443
         type: cert-manager
       dataPlane:
         mtls: true
       identity:
         type: ThirdParty
       manageNetworkPolicy: false

   Copy to Clipboard Toggle word wrap

2. Update the istio-csr deployment to include your OpenShift Service Mesh 3 control plane by running the following command:

     helm upgrade cert-manager-istio-csr jetstack/cert-manager-istio-csr \
         --install \
         --reuse-values \
         --namespace istio-system \
         --wait \
         --set "app.istio.revisions={basic,istio-tenant-a}" 

   1

   Copy to Clipboard Toggle word wrap

   1

   The app.istio.revisions field needs to include your OpenShift Service Mesh 3.0 control plane revision before you create your Istio resource so that proxies can properly communicate with the OpenShift Service Mesh 3.0 control plane.

3. Create your Istio resource.

   Example Istio resource with cert-manager

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
     name: istio-tenant-a
   spec:
     namespace: istio-system-tenant-a 

   1

   
     version: v1.24.3
     values:
       meshConfig:
         discoverySelectors: 

   2

   
           - matchLabels:
               tenant: tenant-a
         extensionProviders:  

   3

   
           - name: prometheus
             prometheus: {}
           - name: otel
             opentelemetry:
              port: 4317
              service: otel-collector.opentelemetrycollector-3.svc.cluster.local
       global:
         caAddress: cert-manager-istio-csr.istio-system.svc:443
       pilot:
         env:
           ENABLE_CA_SERVER: "false"

   Copy to Clipboard Toggle word wrap

   1

   The spec.namespace field in your Istio resource must be the same namespace as your ServiceMeshControlPlane resource. If you set the spec.namespace field in your Istio resource to a different namespace than your ServiceMeshControlPlane resource, the migration will not work properly.

   2

   By default, control planes watch the entire cluster. When managing multiple control planes on a single cluster, you must narrow the scope of each control plane by setting discoverySelectors fields. In this example, the label tenant-a is used, but you can use any label or combination of labels.

   3

   Optional: If you are migrating metrics and tracing, update the extensionProviders fields according to your tracing and metrics configurations.

4. Add your tenant label to each one of your dataplane namespaces by running the following command for each dataplane namespace:

   $ oc label ns bookinfo tenant=tenant-a

   Copy to Clipboard Toggle word wrap
   Note

   With OpenShift Service Mesh 2.6, namespaces were enrolled into the mesh by adding them to the ServiceMeshMemberRoll resource. In OpenShift Service Mesh 3, you must label each one of your dataplane namespaces to match your discoverySelectors fields.

Procedure

1. Find the current IstioRevision for your OpenShift Service Mesh 3.0 control plane by running the following command:

   $ oc get istios istio-tenant-a

   Copy to Clipboard Toggle word wrap

   Example output

   NAME             REVISIONS   READY   IN USE   ACTIVE REVISION   STATUS    VERSION   AGE
   istio-tenant-a   1           1       0        istio-tenant-a    Healthy   v1.24.3   30s

   Copy to Clipboard Toggle word wrap

   Note

   The naming format of your revisions depends on which upgrade strategy you choose for your Istio instance.

2. Copy the ACTIVE REVISION to use as your istio.io/rev label in the next step.

3. Update injection labels on the dataplane namespace by running the following command:

   $ oc label ns bookinfo istio.io/rev=istio-tenant-a maistra.io/ignore-namespace="true" --overwrite=true

   Copy to Clipboard Toggle word wrap

   This adds the following labels to the namespace:

   1. The istio.io/rev: istio-tenant-a label: Ensures that any new pods that get created in that namespace connect to the OpenShift Service Mesh 3.0 proxy.

   2. The maistra.io/ignore-namespace: "true" label: Disables sidecar injection for OpenShift Service Mesh 2.6 proxies in the namespace so OpenShift Service Mesh 2.6 stops injecting proxies in this namespace, and any new proxies are injected by OpenShift Service Mesh 3.0. Without this label, the OpenShift Service Mesh 2.6 injection webhook tries to inject the pod and the injected sidecar proxy will refuse to start since it will have both the OpenShift Service Mesh 2.6 and the OpenShift Service Mesh 3.0 Container Network Interface(CNI) annotations.

      Note

      Once you apply the maistra.io/ignore-namespace label, any new pod that gets created in the namespace connects to the OpenShift Service Mesh 3.0 proxy. Workloads can still communicate with each other regardless of which controlplane they are connected to.

4. Restart the workloads by using one of the following options:

   1. To restart all the workloads at once so that the new pods are injected with the OpenShift Service Mesh 3.0 proxy, run the following command:

      Example command for bookinfo application

      $ oc rollout restart deployments -n bookinfo

      Copy to Clipboard Toggle word wrap

   2. To restart each workload individually, run the following command for each workload:

      Example command with bookinfo application

      $ oc rollout restart deployments productpage-v1 -n bookinfo

      Copy to Clipboard Toggle word wrap

5. Wait for the productpage application to restart by running the following command:

   $ oc rollout status deployment productpage-v1 -n bookinfo

   Copy to Clipboard Toggle word wrap

Verification

1. Check that your workload is connected to the new control plane.

   1. Fetch the list of proxies that are still connected to the OpenShift Service Mesh 2.6 control plane with the istioctl tool by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision basic

      Copy to Clipboard Toggle word wrap

      In this example, basic is the name of your ServiceMeshControlPlane:

      Example output

         NAME                                              CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                            VERSION
         details-v1-7b49464bc-zr7nr.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         ratings-v1-d6f449f59-9rds2.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v1-686cd989df-9x59z.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v2-785b8b48fc-l7xkj.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v3-67889ffd49-7bhxn.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8

      Copy to Clipboard Toggle word wrap

   2. View the list proxies that have been migrated to the new OpenShift Service Mesh 3.0 control plane by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision istio-tenant-a

      Copy to Clipboard Toggle word wrap

      Example output

      NAME                                         CLUSTER        CDS        LDS        EDS        RDS        ECDS     ISTIOD                      VERSION
      productpage-v1-7745c5cc94-wpvth.bookinfo     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED              istiod-5bbf98dccf-n8566     1.24.3

      Copy to Clipboard Toggle word wrap

2. Verify your application is still working correctly. For the bookinfo application, run the following command:

   $ oc exec -it -n bookinfo deployments/productpage-v1 -c istio-proxy -- curl localhost:9080/productpage

   Copy to Clipboard Toggle word wrap