Chapter 3. Multitenant migration guide

Procedure

1. Create your Istio resource based on the following example:

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
     name: istio-tenant-a
   spec:
     namespace: istio-system-tenant-a
     version: v1.24.3
     values:
       meshConfig:
         discoverySelectors:
           - matchLabels:
               tenant: tenant-a
         extensionProviders:
           - name: prometheus
             prometheus: {}
           - name: otel
             opentelemetry:
               port: 4317
               service: otel-collector.opentelemetrycollector-3.svc.cluster.local

   • spec.namespace specifies the field in your Istio resource. It must be the same namespace as your ServiceMeshControlPlane resource. If you set the spec.namespace field in your Istio resource to a different namespace, the migration does not complete successfully.

     Warning

     Adding new namespaces to the mesh during the migration is not recommended and requires extra caution. In multitenant mode, a control plane only interacts with namespaces that are part of its mesh. When you install the 3.0 control plane in the same namespace as the 2.6 control plane, leader election determines which one manages the istio-ca-root-cert ConfigMap. If the 2.6 control plane becomes the leader, it does not distribute this ConfigMap to new namespaces managed by the 3.0 control plane. As a result, sidecar injection fails because the required root certificate is missing.

   • spec.values.meshConfig.discoverySelectors specifies the labels that the control plane uses to identify the namespaces it should manage. By default, control planes watch the entire cluster. When managing multiple control planes on a single cluster, you must narrow the scope of each control plane by setting discoverySelectors fields.

     This example uses the label tenant-a, but you can use any label or combination of labels. The labels must match your discoverySelectors fields.

   • spec.values.meshConfig.extensionProviders specifies the metrics and tracing configurations for the control plane. Optional: If you are migrating metrics and tracing, update the extensionProviders fields according to your tracing and metrics configurations.

2. Add your tenant label to each one of your data plane namespaces by running the following command for each data plane namespace:

   $ oc label ns bookinfo tenant=tenant-a

Procedure

1. Find the current IstioRevision for your OpenShift Service Mesh 3.0 control plane by running the following command:

   $ oc get istios istio-tenant-a

   Example output:

   NAME             REVISIONS   READY   IN USE   ACTIVE REVISION   STATUS    VERSION   AGE
   istio-tenant-a   1           1       0        istio-tenant-a    Healthy   v1.24.3   30s

   Note

   The naming format of your revisions depends on which upgrade strategy you choose for your Istio instance.

2. Copy the ACTIVE REVISION to use as your istio.io/rev label in the next step.

3. Update injection labels on the dataplane namespace by running the following command:

   $ oc label ns bookinfo istio.io/rev=istio-tenant-a maistra.io/ignore-namespace="true" --overwrite=true

   This adds the following labels to the namespace:

   1. The istio.io/rev: istio-tenant-a label: Ensures that any new pods that get created in that namespace connect to the OpenShift Service Mesh 3.0 proxy.

   2. The maistra.io/ignore-namespace: "true" label: Disables sidecar injection for OpenShift Service Mesh 2.6 proxies in the namespace so OpenShift Service Mesh 2.6 stops injecting proxies in this namespace, and any new proxies are injected by OpenShift Service Mesh 3.0. Without this label, the OpenShift Service Mesh 2.6 injection webhook tries to inject the pod and the injected sidecar proxy will refuse to start since it will have both the OpenShift Service Mesh 2.6 and the OpenShift Service Mesh 3.0 Container Network Interface(CNI) annotations.

      Note

      Once you apply the maistra.io/ignore-namespace label, any new pod that gets created in the namespace will connect to the OpenShift Service Mesh 3.0 proxy. Workloads can still communicate with each other regardless of which controlplane they are connected to.

4. Restart the workloads by using one of the following options:

   1. To restart all the workloads at once so that the new pods are injected with the OpenShift Service Mesh 3.0 proxy, run the following command:

      $ oc rollout restart deployments -n bookinfo

   2. To restart each workload individually, run the following command for each workload:

      $ oc rollout restart deployments productpage-v1 -n bookinfo

5. Wait for the productpage application to restart by running the following command:

   $ oc rollout status deployment productpage-v1 -n bookinfo

Verification

1. Check that your workload is connected to the new control plane.

   1. Fetch the list of proxies that are still connected to the OpenShift Service Mesh 2.6 control plane with the istioctl tool by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision basic

      In the following example, basic is the name of your ServiceMeshControlPlane:

         NAME                                              CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                            VERSION
         details-v1-7b49464bc-zr7nr.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         ratings-v1-d6f449f59-9rds2.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v1-686cd989df-9x59z.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v2-785b8b48fc-l7xkj.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v3-67889ffd49-7bhxn.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8

   2. View the list proxies that have been migrated to the new OpenShift Service Mesh 3.0 control plane by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision istio-tenant-a

      Example output:

      NAME                                         CLUSTER        CDS        LDS        EDS        RDS        ECDS     ISTIOD                      VERSION
      productpage-v1-7745c5cc94-wpvth.bookinfo     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED              istiod-5bbf98dccf-n8566     1.24.3

2. Verify your application is still working correctly. For the bookinfo application, run the following command:

   $ oc exec -it -n bookinfo deployments/productpage-v1 -c istio-proxy -- curl localhost:9080/productpage

Procedure

1. Check that your OpenShift Service Mesh 2 ServiceMeshControlPlane is configured with the cert-manager-tool:

   You can see the following example configuration for reference:

   apiVersion: maistra.io/v2
   kind: ServiceMeshControlPlane
   metadata:
     name: basic
     namespace: istio-system
   spec:
     ...
     security:
       certificateAuthority:
         cert-manager:
           address: cert-manager-istio-csr.istio-system.svc:443
         type: cert-manager
       dataPlane:
         mtls: true
       identity:
         type: ThirdParty
       manageNetworkPolicy: false

2. Update the istio-csr deployment to include your OpenShift Service Mesh 3 control plane by running the following command:

     helm upgrade cert-manager-istio-csr jetstack/cert-manager-istio-csr \
         --install \
         --reuse-values \
         --namespace istio-system \
         --wait \
         --set "app.istio.revisions={basic,istio-tenant-a}"

   where:

   app.istio.revisions

   This field needs to include your OpenShift Service Mesh 3.0 control plane revision before you create your Istio resource so that proxies can properly communicate with the OpenShift Service Mesh 3.0 control plane.

3. Create your Istio resource.

   You can see the following example configuration for reference:

   apiVersion: sailoperator.io/v1
   kind: Istio
   metadata:
     name: istio-tenant-a
   spec:
     namespace: istio-system-tenant-a
     version: v1.24.3
     values:
       meshConfig:
         discoverySelectors:
           - matchLabels:
               tenant: tenant-a
         extensionProviders:
           - name: prometheus
             prometheus: {}
           - name: otel
             opentelemetry:
              port: 4317
              service: otel-collector.opentelemetrycollector-3.svc.cluster.local
       global:
         caAddress: cert-manager-istio-csr.istio-system.svc:443
       pilot:
         env:
           ENABLE_CA_SERVER: "false"

   • spec.namespace specifies the namespace for your Istio resource must be the same namespace as your ServiceMeshControlPlane resource. If you set the spec.namespace field in your Istio resource to a different namespace than your ServiceMeshControlPlane resource, the migration will not work properly.
   • spec.values.meshConfig.discoverySelectors specifies the label selector for your Istio resource. By default, control planes watch the entire cluster. When managing multiple control planes on a single cluster, you must narrow the scope of each control plane by setting discoverySelectors fields. In this example, the label tenant-a is used, but you can use any label or combination of labels.
   • spec.values.meshConfig.extensionProviders is an optional field. If you are migrating metrics and tracing, update the extensionProviders fields according to your tracing and metrics configurations.

4. Add your tenant label to each one of your dataplane namespaces by running the following command for each dataplane namespace:

   $ oc label ns bookinfo tenant=tenant-a

   Note

   With OpenShift Service Mesh 2.6, namespaces were enrolled into the mesh by adding them to the ServiceMeshMemberRoll resource. In OpenShift Service Mesh 3, you must label each one of your dataplane namespaces to match your discoverySelectors fields.

Procedure

1. Find the current IstioRevision for your OpenShift Service Mesh 3.0 control plane by running the following command:

   $ oc get istios istio-tenant-a

   Example output:

   NAME             REVISIONS   READY   IN USE   ACTIVE REVISION   STATUS    VERSION   AGE
   istio-tenant-a   1           1       0        istio-tenant-a    Healthy   v1.24.3   30s

   Note

   The naming format of your revisions depends on which upgrade strategy you choose for your Istio instance.

2. Copy the ACTIVE REVISION to use as your istio.io/rev label in the next step.

3. Update injection labels on the dataplane namespace by running the following command:

   $ oc label ns bookinfo istio.io/rev=istio-tenant-a maistra.io/ignore-namespace="true" --overwrite=true

   This adds the following labels to the namespace:

   1. The istio.io/rev: istio-tenant-a label: Ensures that any new pods that get created in that namespace connect to the OpenShift Service Mesh 3.0 proxy.

   2. The maistra.io/ignore-namespace: "true" label: Disables sidecar injection for OpenShift Service Mesh 2.6 proxies in the namespace so OpenShift Service Mesh 2.6 stops injecting proxies in this namespace, and any new proxies are injected by OpenShift Service Mesh 3.0. Without this label, the OpenShift Service Mesh 2.6 injection webhook tries to inject the pod and the injected sidecar proxy will refuse to start since it will have both the OpenShift Service Mesh 2.6 and the OpenShift Service Mesh 3.0 Container Network Interface(CNI) annotations.

      Note

      Once you apply the maistra.io/ignore-namespace label, any new pod that gets created in the namespace connects to the OpenShift Service Mesh 3.0 proxy. Workloads can still communicate with each other regardless of which controlplane they are connected to.

4. Restart the workloads by using one of the following options:

   1. To restart all the workloads at once so that the new pods are injected with the OpenShift Service Mesh 3.0 proxy, run the following command:

      $ oc rollout restart deployments -n bookinfo

   2. To restart each workload individually, run the following command for each workload:

      $ oc rollout restart deployments productpage-v1 -n bookinfo

5. Wait for the productpage application to restart by running the following command:

   $ oc rollout status deployment productpage-v1 -n bookinfo

Verification

1. Check that your workload is connected to the new control plane.

   1. Fetch the list of proxies that are still connected to the OpenShift Service Mesh 2.6 control plane with the istioctl tool by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision basic

      In the following example, basic is the name of your ServiceMeshControlPlane:

         NAME                                              CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                            VERSION
         details-v1-7b49464bc-zr7nr.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         ratings-v1-d6f449f59-9rds2.bookinfo               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v1-686cd989df-9x59z.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v2-785b8b48fc-l7xkj.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8
         reviews-v3-67889ffd49-7bhxn.bookinfo              Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-basic-6c9f8d9894-sh6lx     1.20.8

   2. View the list proxies that have been migrated to the new OpenShift Service Mesh 3.0 control plane by running the following command:

      $ istioctl ps --istioNamespace istio-system-tenant-a --revision istio-tenant-a

      Example output:

      NAME                                         CLUSTER        CDS        LDS        EDS        RDS        ECDS     ISTIOD                      VERSION
      productpage-v1-7745c5cc94-wpvth.bookinfo     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED              istiod-5bbf98dccf-n8566     1.24.3

2. Verify your application is still working correctly. For the bookinfo application, run the following command:

   $ oc exec -it -n bookinfo deployments/productpage-v1 -c istio-proxy -- curl localhost:9080/productpage