Chapter 1. Configuring and deploying OpenShift Lightspeed

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Select All Projects in the Project dropdown at the top of the screen.
3. Click OpenShift Lightspeed Operator.
4. Click OLSConfig, then click the cluster configuration instance in the list.
5. Click the YAML tab.

6. Update the OLSconfig CR to contain the file that has the TLS secret.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     ols:
       tlsConfig:
         keyCertSecretRef:
           name: <lightspeed_tls>
   ---
   apiVersion: v1
   data:
     tls.crt: LS0tLS1CRUd...
     tls.key: LS0tLS1CRUd...
   kind: Secret
   metadata:
     name: <lightspeed_tls>
     namespace: <openshift_lightspeed>

   Copy to Clipboard Toggle word wrap
   • spec.ols.tlsConfig.keyCertSecretRef.name specifies the secret that has the tls.crt and tls.key file.
   • apiVersion.data.tls specifies that the name of the certificate must be tls.crt and the name of the key must be tls.key.
7. Click Save.

Verification

1. Verify that a new pod exists in the lightspeed-app-server deployment by running the following command:

   $ oc get pod -n openshift-lightspeed

   Copy to Clipboard Toggle word wrap

Procedure

1. Copy the contents of the certificate file and paste it into a file called caCertFileName.

2. Create a ConfigMap object called trusted-certs by running the following command:

   $ oc create configmap trusted-certs --from-file=caCertFileName --namespace openshift-lightspeed

   Copy to Clipboard Toggle word wrap

   This command returns an output similar to the following example:

   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: trusted-certs
     namespace: openshift-lightspeed
   data:
     caCertFileName: |
       -----BEGIN CERTIFICATE-----
       .
       .
       .
       -----END CERTIFICATE-----

   Copy to Clipboard Toggle word wrap
   • data.caCertFileName specifies the CA certificates required to connect to your LLM provider. You can include one or more certificates within this block to ensure secure communication.

3. Update the OLSConfig custom resource (CR) file to include the name of the ConfigMap object you just created. The following example uses Red Hat Enterprise Linux AI as the LLM provider.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     ols:
       defaultProvider: rhelai
       defaultModel: models/<model_name>
       additionalCAConfigMapRef:
         name: trusted-certs

   Copy to Clipboard Toggle word wrap
   • spec.ols.additionalCAConfigMapRef.name specifies the name of ConfigMap object.

4. Create the custom CR by running the following command:

   $ oc apply -f <olfconfig_cr_filename>

   Copy to Clipboard Toggle word wrap

Procedure

1. Generate a YAML configuration file for the cluster role binding. Enter the actual user name in place of <user_name> when running the following command:

   $ oc adm policy add-cluster-role-to-user lightspeed-operator-query-access <user_name> -o yaml --dry-run

   Copy to Clipboard Toggle word wrap

2. Apply the generated configuration file to the cluster:

   $ oc apply -f ols-user-access.yaml

   Copy to Clipboard Toggle word wrap

Verification

1. Verify the creation of the ClusterRoleBinding by running the following command:

   $ oc get clusterrolebinding lightspeed-operator-query-access

   Copy to Clipboard Toggle word wrap

2. Inspect the YAML configuration and ensure that it lists the correct user in the subjects section by running the following command:

   $ oc get clusterrolebinding lightspeed-operator-query-access -o yaml

   Copy to Clipboard Toggle word wrap

   This command returns an output similar to the following example:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     creationTimestamp: null
     name: lightspeed-operator-query-access
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: lightspeed-operator-query-access
   subjects:
   - apiGroup: rbac.authorization.k8s.io
     kind: User
     name: <user_name>

   Copy to Clipboard Toggle word wrap

Procedure

1. Grant the lightspeed-operator-query-access role to your user group by running the following command. Replace <group_name> with the actual name of the user group in your cluster.

   $ oc adm policy add-cluster-role-to-group \
   lightspeed-operator-query-access <group_name>

   Copy to Clipboard Toggle word wrap

2. Optional: Verify that the role binding contains the user group by running the following command:

   $ oc get clusterrolebinding lightspeed-operator-query-access -o wide

   Copy to Clipboard Toggle word wrap

Procedure

1. Generate the YAML configuration by running the following command:

   $ oc adm policy add-cluster-role-to-group lightspeed-operator-query-access <group_name> -o yaml --dry-run > access-policy.yaml

   Copy to Clipboard Toggle word wrap

2. Open the access-policy.yaml file and verify the subjects section contains the correct group name:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     creationTimestamp: null
     name: lightspeed-operator-query-access
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: lightspeed-operator-query-access
   subjects:
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: <user_group>

   Copy to Clipboard Toggle word wrap
   • subjects.name specifies the name of the user group you are granting access to.

3. Apply the configuration file to the cluster by running the following command:

   $ oc create -f access-policy.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Specify the location of the directory with the Markdown files for the retrieval-augmented generation (RAG) database and the path for the image that the BYO Knowledge tool generates by running the following command:

   $ podman run -it --rm --device=/dev/fuse \
     -v $XDG_RUNTIME_DIR/containers/auth.json:/run/user/0/containers/auth.json:Z \
     -v <dir_tree_with_markdown_files>:/markdown:Z \
     -v <dir_for_image_tar>:/output:Z \
     registry.redhat.io/openshift-lightspeed-tech-preview/lightspeed-rag-tool-rhel9:latest

   Copy to Clipboard Toggle word wrap

2. Load the container image that the BYO Knowledge tool generated by running the following command:

   $ podman load < <directory_for_image_tar>/<my-byok-image.tar>

   Copy to Clipboard Toggle word wrap

3. Display the Podman images that are on your local computer by running the following command:

   $ podman images

   Copy to Clipboard Toggle word wrap

   This command returns an output similar to the following example:

   REPOSITORY                            TAG                IMAGE ID      CREATED       SIZE
   localhost/my-byok-image               latest             be7d1770bf10  1 minute  ago    2.37 GB
   ...

   Copy to Clipboard Toggle word wrap

4. Tag the local image with a name and destination so that you can push the image to the container image registry by running the following command:

   $ podman tag localhost/my-byok-image:latest quay.io/<username>/my-byok-image:latest

   Copy to Clipboard Toggle word wrap

5. Push the local container image to the container image registry by running the following command:

   $ podman push quay.io/<username>/my-byok-image:latest

   Copy to Clipboard Toggle word wrap

6. Update the OLSconfig CR to deploy the newly created RAG database alongside the existing one:

   1. In the OpenShift Container Platform web console, click Operators Installed Operators.
   2. Select All Projects in the Project dropdown at the top of the screen.
   3. Click OpenShift Lightspeed Operator.
   4. Click OLSConfig, then click the cluster configuration instance in the list.
   5. Click the YAML tab.

   6. Insert the spec.ols.rag YAML code:

      apiVersion: ols.openshift.io/v1alpha1
      kind: OLSConfig
      metadata:
        name: cluster
      spec:
        ols:
          rag:
            - image: quay.io/<username>/my-byok-image:latest

      Copy to Clipboard Toggle word wrap
      • spec.ols.rag.image specifies the tag for the image that you pushed to the image registry so that the OpenShift Lightspeed Operator can access the custom content. The OpenShift Lightspeed Operator can work with more than one RAG database that you create.

7. Optional: Specify pull secrets in the OLSSpec section of the OLSConfig CR file. These secrets provide authentication for remote registries. Use this optional field if your RAG BYO Knowledge images reside in a private registry that the standard cluster-wide pull secret cannot access.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
     namespace: openshift-lightspeed
   spec:
     llm:
       providers:
   ...
     ols:
       imagePullSecrets:
         - name: <my_pull_secret_1>
         - name: <my_pull_secret_2>

   Copy to Clipboard Toggle word wrap
   • spec.ols.imagePullSecrets defines the pull secrets that OpenShift Lightspeed uses only when you specify BYO Knowledge RAG images. Instead of linking a specific secret to a specific image, the system maintains a general list of pull secrets. For every BYO Knowledge image, the system tries each pull secret in sequential order until it achieves a successful authentication.
8. Click Save.

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Select All Projects in the Project list at the top of the screen.
3. Click OpenShift Lightspeed Operator.
4. Click OLSConfig, then click the cluster configuration instance in the list.
5. Click the YAML tab.

6. Insert the spec.ols.byokRAGOnly YAML code.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     ols:
       byokRAGOnly: true

   Copy to Clipboard Toggle word wrap
   • spec.ols.byokRAGOnly specifies if the Service limits responses by using only the information found in the local documentation that you provide. Specify true so that OpenShift Lightspeed only uses RAG databases that you create by using the BYO Knowledge feature. When true, OpenShift Lightspeed does not use the default RAG database that contains the OpenShift Container Platform documentation.
7. Click Save.

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Click OpenShift Lightspeed Operator.
3. Click OLSConfig, then click the cluster configuration instance in the list.
4. Click the YAML tab.

5. Set the spec.ols.introspectionEnabled parameter to true to enable cluster interaction:

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     ols:
       introspectionEnabled: true

   Copy to Clipboard Toggle word wrap
6. Click Save.

Procedure

1. Open the OpenShift Lightspeed OLSconfig custom resource (CR) file by running the following command:

   $ oc edit olsconfig cluster

   Copy to Clipboard Toggle word wrap

2. Add MCPServer to the spec.featureGates specification file and include the MCP server information.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     featureGates:
       - MCPServer
     mcpServers:
     - name: mcp-server-1
       url: https://mcp.example.com
       timeout: 30
       headers:
         - name: Authorization
           valueFrom:
            type: kubernetes
     - name: mcp-server-2
       url: https://mcp.example.com
       timeout: 30
       headers:
         - name: X-Special
           valueFrom:
             type: secret
             secretRef:
               name: <secret_name>

   Copy to Clipboard Toggle word wrap
   • spec.featureGates specifies the MCP server functionality on the OpenShift Lightspeed pod.
   • spec.mcpServers.name specifies the name of the MCP server.
   • spec.mcpServers.url specifies the URL path that the MCP server uses to communicate
   • spec.mcpServers.timeout specifies the time that the MCP server has to respond to a query. If the client does not receive a query within the time specified, the MCP server times out. In this example, the timeout is 30 seconds.
   • spec.mcpServers.headers specifies MCP headers as an array of structured objects containing the key-value pairs required for server authentication and context.
   • spec.mcpServers.headers.name specifies the name of the header that gets sent to the MCP server.
   • spec.mcpServers.headers.valueFrom.type specify the string kubernetes to provide the user’s bearer token
   • spec.mcpServers.headers.valueFrom.secretRef.name specifies the name of the secret that contains the header value. Ensure that the secret has the key name header.

3. Click Save.

   The save operation saves the file and applies the changes so that the MCP server is available to the OpenShift Lightspeed service.

Procedure

1. Open the OpenShift Lightspeed OLSconfig CR file by running the following command:

   $ oc edit olsconfig cluster

   Copy to Clipboard Toggle word wrap

2. Update the spec.ols.quotaHandlersConfig specification to include token quota limit information.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     ols:
       quotaHandlersConfig:
         limitersConfig:
         - name: user_limits
           type: user_limiter
           initialQuota: 100000
           quotaIncrease: 1000
           period: 30 days
         - name: cluster_limits
           type: cluster_limiter
           initialQuota: 1000000
           quotaIncrease: 100000
           period: 30 days

   Copy to Clipboard Toggle word wrap

   • User quota configuration fields

     • spec.ols.quotaHandlersConfig.limitersConfig.name specifies the token limit for the user account.
     • spec.ols.quotaHandlersConfig.limitersConfig.initialQuota specifies a token quota limit of 100,000 for each user over the time period specified in the period field.
     • spec.ols.quotaHandlersConfig.limitersConfig.quotaIncrease increases the token quota limit for the user by 1,000 at the end of the time period specified in the period field.

   • Cluster quota configuration fields

     • spec.ols.quotaHandlersConfig.limitersConfig.name specifies the token limit for a cluster.
     • spec.ols.quotaHandlersConfig.limitersConfig.initialQuota specifies a token quota limit of 1,000,000 for each cluster over the time period specified in the period field.
     • spec.ols.quotaHandlersConfig.limitersConfig.quotaIncrease increases the token quota limit for the cluster by 100,000 at the end of the time period specified in the period field.

   • Shared configuration field

     • spec.ols.quotaHandlersConfig.limitersConfig.period defines the amount of time that the scheduler waits before the period resets or the quota limit increases.

3. Click Save.

   The save operation saves the file and applies the changes to activate the token quota limits.

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Select All Projects in the Project list at the top of the screen.
3. Click OpenShift Lightspeed Operator.
4. Click OLSConfig, then click the cluster configuration instance in the list.
5. Click the YAML tab.

6. Insert the spec.ols.storage YAML code.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
     namespace: openshift-lightspeed
   spec:
     llm:
       providers:
   ...
     ols:
       storage: {}

   Copy to Clipboard Toggle word wrap
   • spec.ols.storage specifies how the assistant stores persistent data, specifically conversation history. The class depends on the existing instances of the storage class in the cluster. If you leave the storage class empty, the assistant uses default values. The persistent volume allocated for the PostgreSQL database is 1 GB in size and uses the storage class of the default cluster. Specify empty braces for the storage parameter to use the default values.
7. Click Save.

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Select All Projects in the Project dropdown at the top of the screen.
3. Click OpenShift Lightspeed Operator.
4. Click OLSConfig, then click the cluster configuration instance in the list.
5. Click the YAML tab.

6. Update the OLSconfig CR to override the default PVC specifications as shown in the following example.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
     namespace: openshift-lightspeed
   spec:
     ols:
       storage:
         size: 768Mi
         class: gp2-csi

   Copy to Clipboard Toggle word wrap
   • spec.ols.storage.size specifies the total storage capacity for the database. If you do not specify this parameter, the Operator uses the default size of 1 GiB.
   • spec.ols.storage.class specifies the Storage Class for the database volume. If you do not specify this parameter, the Operator uses the default storage class setting of the cluster.
7. Click Save.

Verification

1. Verify that the cluster has successfully provisioned the storage by checking the status of the Persistent Volume Claim.

   $ oc get pvc -n openshift-lightspeed

   Copy to Clipboard Toggle word wrap

Procedure

1. In the OpenShift Container Platform web console, click Operators Installed Operators.
2. Select All Projects in the Project dropdown at the top of the screen.
3. Click OpenShift Lightspeed Operator.
4. Click OLSConfig, then click the cluster configuration instance in the list.
5. Click the YAML tab.

6. Modify the ols.Config custom resource (CR) file to define a feature gate and the tools filtering configuration.

   apiVersion: ols.openshift.io/v1alpha1
   kind: OLSConfig
   metadata:
     name: cluster
   spec:
     featureGates:
       - ToolFiltering
     olsConfig:
       toolFilteringConfig:
         alpha: 0.8
         topK: 10
         threshold: 0.01

   Copy to Clipboard Toggle word wrap
   • spec.featureGates.ToolFilter specifies the feature gate.
   • spec.olsConfig.toolFilteringConfig.alpha specifies the weight balance between semantic (RAG-based) and keyword matching. Increasing the value provides more weight to the semantic search. The valid range of values is 0 to 1.
   • spec.olsConfig.toolFilteringConfig.topk specifies the maximum number of tools available for the LLM.

   • spec.olsConfig.toolFilteringConfig.threshold specifies the minimum score for the tool to be considered as a candidate. Tools with a value of the score lower then the threshold value are discarded. Increasing the value discards more tools. The valid range of values is 0.01 to 0.1.

     Note

     This example uses the default values for the alpha, tpok and threshold fields. If you use the default values in your configuration you do not have to specify them.

7. Click Save.

Verification

1. Navigate to the OpenShift Container Platform web console.
2. Select Workloads Pods and then click the pod that contains OpenShift Lightspeed.
3. Click Logs and confirm that the log displays RAG information.