Chapter 2. Configuring TrustyAI

Procedure

1. In a terminal window, if you are not already logged in to your OpenShift cluster as a cluster administrator, log in to the OpenShift CLI as shown in the following example:

   $ oc login <openshift_cluster_url> -u <admin_username> -p <password>

2. Define a ConfigMap object in a YAML file called uwm-cm-conf.yaml with the following contents:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: user-workload-monitoring-config
     namespace: openshift-user-workload-monitoring
   data:
     config.yaml: |
       prometheus:
         logLevel: debug
         retention: 15d

   The user-workload-monitoring-config object configures the components that monitor user-defined projects. Observe that the retention time is set to the recommended value of 15 days.

3. Apply the configuration to create the user-workload-monitoring-config object.

   $ oc apply -f uwm-cm-conf.yaml

4. Define another ConfigMap object in a YAML file called uwm-cm-enable.yaml with the following contents:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cluster-monitoring-config
     namespace: openshift-monitoring
   data:
     config.yaml: |
       enableUserWorkload: true

   The cluster-monitoring-config object enables monitoring for user-defined projects.

5. Apply the configuration to create the cluster-monitoring-config object.

   $ oc apply -f uwm-cm-enable.yaml

Procedure

1. In the OpenShift console, click Operators Installed Operators.
2. Search for the Red Hat OpenShift AI Operator, and then click the Operator name to open the Operator details page.
3. Click the Data Science Cluster tab.
4. Click the default instance name (for example, default-dsc) to open the instance details page.
5. Click the YAML tab to show the instance specifications.

6. In the spec:components section, set the managementState field for the trustyai component to Managed:

    trustyai:
       managementState: Managed

7. Click Save.

1. In the OpenShift console, from the Project list, select redhat-ods-applications.
2. Click Workloads Deployments.

3. Search for the trustyai-service-operator-contoller-manager deployment. Check the status:

   1. Click the deployment name to open the deployment details page.
   2. Click the Pods tab.

   3. View the pod status.

      When the status of the trustyai-service-operator-controller-manager-<pod-id> pod is Running, the pod is ready to use.

Procedure

1. In a terminal window, if you are not already logged in to your OpenShift cluster as a cluster administrator, log in to the OpenShift CLI as shown in the following example:

   $ oc login <openshift_cluster_url> -u <admin_username> -p <password>

2. Optional: If you want to use a TLS connection between TrustyAI and the database, create a TrustyAI service database TLS secret that uses the same certificates that you want to use for the database.

   1. Create a YAML file to contain your TLS secret and add the following code:

      apiVersion: v1
      kind: Secret
      metadata:
        name: <service_name>-db-tls
      type: kubernetes.io/tls
      data:
        tls.crt: |
          <TLS CERTIFICATE>
      
        tls.key: |
          <TLS KEY>

   2. Save the file with the file name <service_name>-db-tls.yaml. For example, if your service name is trustyai-service, save the file as trustyai-service-db-tls.yaml.

   3. Apply the YAML file in the data science project that contains the models that the data scientist wants to monitor:

      $ oc apply -f <service_name>-db-tls.yaml -n <project_name>

3. Create a secret (or specify an existing one) that has your database credentials.

   1. Create a YAML file to contain your secret and add the following code:

      apiVersion: v1
      kind: Secret
      metadata:
        name: db-credentials
      type: Opaque
      stringData:
        databaseKind: <mariadb> 

      1

      
        databaseUsername: <TrustyAI_username> 

      2

      
        databasePassword: <TrustyAI_password> 

      3

      
        databaseService: mariadb-service 

      4

      
        databasePort: 3306 

      5

      
        databaseGeneration: update 

      6

      
        databaseName: trustyai_service 

      7

      1

      The only currently supported databaseKind value is mariadb.

      2

      The username you want TrustyAI to use when interfacing with the database.

      3

      The password that TrustyAI must use when connecting to the database.

      4

      The Kubernetes (K8s) service that TrustyAI must use when connecting to the database (the default mariadb) .

      5

      The port that TrustyAI must use when connecting to the database (default is 3306).

      6

      The database schema generation strategy to be used by TrustyAI. It is the setting for the quarkus.hibernate-orm.database.generation argument, which determines how TrustyAI interacts with the database on its initial connection. Set to none, create, drop-and-create, drop, update, or validate.

      7

      The name of the individual database within the database service that the username and password authenticate to, as well as the specific database name that TrustyAI should read and write to on the database server.

   2. Save the file with the file name db-credentials.yaml. You will need this name later when you install or change the TrustyAI service.

   3. Apply the YAML file in the data science project that contains the models that the data scientist wants to monitor:

      $ oc apply -f db-credentials.yaml -n <project_name>

4. If you are installing TrustyAI for the first time on a project, continue to Installing the TrustyAI service for a project.

   If you already installed TrustyAI on a project, you can migrate the existing TrustyAI service from using a PVC to using a database.

   1. Create a YAML file to update the TrustyAI service custom resource (CR) and add the following code:

      apiVersion: trustyai.opendatahub.io/v1alpha1
      kind: TrustyAIService
      metadata:
        annotations:
          trustyai.opendatahub.io/db-migration: "true" 

      1

      
        name: trustyai-service 

      2

      
      spec:
        storage:
          format: "DATABASE" 

      3

      
          folder: "/inputs" 

      4

      
            size: "1Gi" 

      5

      
          databaseConfigurations: <database_secret_credentials> 

      6

      
        data:
          filename: "data.csv" 

      7

      
        metrics:
          schedule: "5s" 

      8

      1

      Set to true to prompt the migration from PVC to database storage.

      2

      The name of the TrustyAI service instance.

      3

      The storage format for the data. Set this field to DATABASE.

      4

      The location within the PVC where you were storing the data. This must match the value specified in the existing CR.

      5

      The size of the data to request.

      6

      The name of the secret with your database credentials that you created in an earlier step. For example, db-credentials.

      7

      The suffix for the existing stored data files. This must match the value specified in the existing CR.

      8

      The interval at which to calculate the metrics. The default is 5s. The duration is specified with the ISO-8601 format. For example, 5s for 5 seconds, 5m for 5 minutes, and 5h for 5 hours.

   2. Save the file. For example, trustyai_crd.yaml.

   3. Apply the new TrustyAI service CR to the data science project that contains the models that the data scientist wants to monitor:

      $ oc apply -f trustyai_crd.yaml -n <project_name>