Installing and uninstalling OpenShift AI Self-Managed

Procedure

1. Open a new terminal window.

2. In the OpenShift command-line interface (CLI), log in to your OpenShift cluster as a cluster administrator, as shown in the following example:

   $ oc login <openshift_cluster_url> -u <admin_username> -p <password>

3. Create a new YAML file with the following contents:

   apiVersion: v1
   kind: Namespace 1
   metadata:
     name: redhat-ods-operator
   ---
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup 2
   metadata:
     name: rhods-operator
     namespace: redhat-ods-operator
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription 3
   metadata:
     name: rhods-operator
     namespace: redhat-ods-operator
   spec:
     name: rhods-operator
     channel: embedded 4
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     config:
        env:
           - name: "DISABLE_DSC_CONFIG" 5

   1

   Defines the required redhat-ods-operator namespace for installation of the Operator.

   2

   Defines an Operator group for installation of the Operator. You must specify the same namespace that you defined earlier in the file. Also, another Operator group must not exist in the namespace.

   3

   Defines a subscription for the Operator. You must specify the same namespace that you defined earlier in the file.

   4

   Sets the update channel. You must specify the embedded update channel.

   5

   Specifies that DSCI and DSC custom resources are not dynamically created. This setting is required to install the KServe component of OpenShift AI in raw mode, which means that KServe does not have any other components as dependencies.

4. Deploy the YAML file to create the namespace, Operator group, and subscription that you defined.

   $ oc create -f <file_name>.yaml

   You see output similar to the following:

   namespace/redhat-ods-operator created
   operatorgroup.operators.coreos.com/rhods-operator created
   subscription.operators.coreos.com/rhods-operator created

5. Create another new YAML file with the following contents:

   apiVersion: dscinitialization.opendatahub.io/v1
   kind: DSCInitialization 1
   metadata:
     name: default-dsci
   spec:
     applicationsNamespace: redhat-ods-applications
     monitoring:
       managementState: Managed
       namespace: redhat-ods-monitoring
     serviceMesh:
       managementState: Removed 2
     trustedCABundle:
       managementState: Managed
       customCABundle: ""

   1

   Defines a DSCInitialization object called default-dsci. The DSCInitialization object is used by the Operator to manage resources that are common to all product components.

   2

   Specifies that the serviceMesh component (which is a dependency for KServe in some setups) is not installed. This setting is required when preparing OpenShift AI for use in IBM products.

6. Create the DSCInitialization object on your OpenShift cluster.

   $ oc create -f <dsci_file_name>.yaml

Procedure

1. Open a new terminal window.

2. In the OpenShift command-line interface (CLI), log in to your OpenShift cluster as a cluster administrator, as shown in the following example:

   $ oc login <openshift_cluster_url> -u <admin_username> -p <password>

3. Create a new YAML file with the following contents:

   apiVersion: datasciencecluster.opendatahub.io/v1
   kind: DataScienceCluster 1
   metadata:
     name: default-dsc
   spec:
     components:
       codeflare:
         managementState: Removed
       dashboard:
         managementState: Removed
       datasciencepipelines:
         managementState: Removed
       kserve:
         managementState: Managed
         defaultDeploymentMode: RawDeployment 2
         serving:
           managementState: Removed 3
           name: knative-serving
       kueue:
         managementState: Removed
       modelmeshserving:
         managementState: Removed
       ray:
         managementState: Removed
       trainingoperator:
         managementState: Managed 4
       trustyai:
         managementState: Removed
       workbenches:
         managementState: Removed

   1

   Defines a new DataScienceCluster object called default-dsc. The DataScienceCluster object is used by the Operator to manage OpenShift AI components.

   2

   Specifies that the KServe component is installed and managed by the Operator in raw mode, which means that KServe does not have any other components as dependencies.

   3

   Specifies that the serving component (which is a dependency for KServe in some setups) is not installed. This setting is required when preparing OpenShift AI for use in Cloud Pak for Data.

   4

   Specifies that the Training Operator is installed and managed by the Operator. This component is required if you want to use the Kubeflow Training Operator to tune models.

   In addition, observe that the value of the managementState field for all other OpenShift AI components is set to Removed. This value means that the components are not installed.

   In general, the Managed and Removed values are described as follows:

   Managed

   The Operator actively manages the component, installs it, and tries to keep it active. The Operator will upgrade the component only if it is safe to do so.

   Removed

   The Operator actively manages the component but does not install it. If the component is already installed, the Operator will try to remove it.

4. Create the DataScienceCluster object on your OpenShift cluster.

   $ oc create -f <file_name>.yaml

   You see output similar to the following:

   datasciencecluster.datasciencecluster.opendatahub.io/default-dsc created

Procedure

1. Log in to the OpenShift web console as a cluster administrator.
2. In the web console, click Workloads → ConfigMaps.
3. In the Project list, click redhat-ods-applications.
4. On the list of ConfigMap resources, click the inferenceservice-config resource and then click the YAML tab.

5. In the metadata.annotations section of the file, add opendatahub.io/managed: 'false' as shown:

   metadata:
     annotations:
       internal.config.kubernetes.io/previousKinds: ConfigMap
       internal.config.kubernetes.io/previousNames: inferenceservice-config
       internal.config.kubernetes.io/previousNamespaces: opendatahub
       opendatahub.io/managed: 'false'

   The additional annotation makes the inferencing configuration file editable.

6. To prevent errors when using OpenShift AI in Cloud Pak for Data (including watsonx.ai), update the configuration as follows:

   1. In the YAML file, locate the following entry:

      "domainTemplate": "{{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }}

   2. Update the value of the domainTemplate field as shown:

      "domainTemplate": "example.com"

      This new, explicitly specified value ensures that OpenShift AI cannot generate a value for the domainTemplate field that exceeds the maximum length and causes an error. The domainTemplate field is not used by the raw deployment mode that you configured for the KServe component when preparing OpenShift AI for use in Cloud Pak for Data.

7. Click Save.

Procedure

1. Log in to the OpenShift.
2. Click Operators → Installed Operators and then click the Red Hat OpenShift AI Operator.
3. Click the DSC Initialization tab.
4. Click the default-dsci object.
5. Click the YAML tab.

6. In the spec section, add the custom certificate to the customCABundle field for trustedCABundle, as shown in the following example:

   spec:
     trustedCABundle:
       managementState: Managed
       customCABundle: |
         -----BEGIN CERTIFICATE-----
         examplebundle123
         -----END CERTIFICATE-----

7. Click Save.

Procedure

1. In the OpenShift web console, click Operators → Installed Operators and then click the Red Hat OpenShift AI Operator.
2. Click the DSC Initialization tab.
3. Click the default-dsci object.
4. Click the YAML tab.

5. In the spec section, change the value of the managementState field for trustedCABundle to Removed:

   spec:
     trustedCABundle:
       managementState: Removed

6. Click Save.

Procedure

1. In the OpenShift web console, click Operators → Installed Operators and then click the Red Hat OpenShift AI Operator.
2. Click the DSC Initialization tab.
3. Click the default-dsci object.
4. Click the YAML tab.

5. In the spec section, change the value of the managementState field for trustedCABundle to Unmanaged, as shown:

   spec:
     trustedCABundle:
       managementState: Unmanaged

6. Click Save.

   Note that changing the managementState from Managed to Unmanaged does not remove the odh-trusted-ca-bundle ConfigMap, but the ConfigMap is not updated if you make changes to the customCABundle field.

Verification

1. In the spec section, set or change the value of the customCABundle field for trustedCABundle, for example:

   spec:
     trustedCABundle:
       managementState: Unmanaged
       customCABundle: example123

2. Click Save.
3. Click Workloads → ConfigMaps.
4. Select a project from the project list.
5. Click the odh-trusted-ca-bundle ConfigMap.
6. Click the YAML tab and verify that the value of the customCABundle field did not update.

1. Log in to the OpenShift CLI.

2. Run the following command:

   oc get pods -l name=rhods-operator -o name -n redhat-ods-operator |  xargs -I {} oc logs -f {} -n redhat-ods-operator

   The operator pod log opens.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Deploying on $infrastructure, which is not supported. Failing Installation error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Attempt to create the ODH CR failed. error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Attempt to create the RHODS Notebooks CR failed. error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects.

4. Click Filter and select the checkbox for every status except Running and Completed.

   The page displays the pods that have an error.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Attempt to create the RBAC policy for dedicated admins group in $target_project failed. error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Pagerduty secret does not exist error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: SMTP secret does not exist error message.

Diagnosis

1. In the OpenShift web console, switch to the Administrator perspective.
2. Click Workloads → Pods.
3. Set the Project to All Projects or redhat-ods-operator.

4. Click the rhods-operator-<random string> pod.

   The Pod details page appears.

5. Click Logs.
6. Select rhods-operator from the drop-down list.
7. Check the log for the ERROR: Addon managed odh parameter secret does not exist. error message.

Procedure

1. Open a new terminal window.

2. In the OpenShift command-line interface (CLI), log in to your OpenShift cluster as a cluster administrator, as shown in the following example:

   $ oc login <openshift_cluster_url> -u system:admin

3. Create a ConfigMap object for deletion of the Red Hat OpenShift AI Operator.

   $ oc create configmap delete-self-managed-odh -n redhat-ods-operator

4. To delete the rhods-operator, set the addon-managed-odh-delete label to true.

   $ oc label configmap/delete-self-managed-odh api.openshift.com/addon-managed-odh-delete=true -n redhat-ods-operator

5. When all objects associated with the Operator are removed, delete the redhat-ods-operator project.

   1. Set an environment variable for the redhat-ods-applications project.

      $ PROJECT_NAME=redhat-ods-applications

   2. Wait until the redhat-ods-applications project has been deleted.

      $ while oc get project $PROJECT_NAME &> /dev/null; do
      echo "The $PROJECT_NAME project still exists"
      sleep 1
      done
      echo "The $PROJECT_NAME project no longer exists"

      When the redhat-ods-applications project has been deleted, you see the following output.

      The redhat-ods-applications project no longer exists

   3. When the redhat-ods-applications project has been deleted, delete the redhat-ods-operator project.

      $ oc delete namespace redhat-ods-operator