Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 1. Configuring and deploying OpenShift Lightspeed

After the OpenShift Lightspeed Operator is installed, configuring and deploying Lightspeed consists of three tasks.

Note

The instructions assume that you are installing OpenShift Lightspeed using the kubeadmin user account. If you are using a regular user account with cluster-admin privileges, read the section of the documentation that discusses RBAC.

First, create a credential secret using the credentials for your Large Language Model (LLM) provider. Next, create the OLSConfig custom resource (CR) that the Operator uses to deploy the service. Finally, verify that the Lightspeed service is operating.

Important

Starting with OpenShift Container Platform 4.19, the perspectives in the web console are unified. The Developer perspective is no longer enabled by default.

All users can interact with all OpenShift Container Platform web console features. However, if you are not the cluster owner, you might need to request permission to certain features from the cluster owner.

You can still enable the Developer perspective. On the Getting Started pane in the web console, you can take a tour of the console, find information on setting up your cluster, view a quick start for enabling the Developer perspective, and follow links to explore new features and capabilities.

Create a file that is associated with the API token used to access the API of your large language model (LLM) provider. Typically, you use API tokens to authenticate your LLM provider. Alternatively, Microsoft Azure also supports authentication using Microsoft Entra ID.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create a secret to store the Provider tokens.
  • You have installed the OpenShift Lightspeed Operator.

Procedure

  1. Click the Quick create ( fa plus circle ) menu in the upper-right corner of the OpenShift web console and select Import YAML.

  2. Paste the YAML content for the LLM provider that you are using into the text area of the web console.

    Note

    The YAML parameter is always apitoken regardless of what the LLM provider calls the access details.

    Credential secret for LLM provider

    apiVersion: v1
kind: Secret
metadata:
  name: credentials
  namespace: openshift-lightspeed
type: Opaque
stringData:
  apitoken: <api_token>
    1
    Copy to Clipboard Toggle word wrap

    1
    The api_token is not base64 encoded.

    Credential secret for Red Hat Enterprise Linux AI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: rhelai-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for Red Hat OpenShift AI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: rhoai-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for IBM watsonx

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: watsonx-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for Microsoft Azure OpenAI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: azure-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Alternatively, for Microsoft Azure OpenAI you can use Microsoft Entra ID to authenticate your LLM provider. Microsoft Entra ID users must configure the required roles for their Microsoft Azure OpenAI resource. For more information, see the official Microsoft Cognitive Services OpenAI Contributor(Microsoft Azure OpenAI Service documentation).

    Credential secret for Microsoft Entra ID

    apiVersion: v1
data:
  client_id: <base64_encoded_client_id>
  client_secret: <base64_encoded_client_secret>
  tenant_id: <base64_encoded_tenant_id>
kind: Secret
metadata:
  name: azure-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

  3. Click Create.

The Custom Resource (CR) file contains information that the Operator uses to deploy OpenShift Lightspeed. The specific content of the CR file is unique for each large language model (LLM) provider. To create the CR file, choose the configuration file for the LLM provider that you are using.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file.
  • You have an LLM provider available for use with the OpenShift Lightspeed Service.
  • You have installed the OpenShift Lightspeed Operator.

Procedure

  1. Click the Quick create ( fa plus circle ) menu in the upper-right corner of the OpenShift web console and select Import YAML.

  2. Paste the YAML content for the LLM provider you use into the text area of the web console.

    OpenAI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: myOpenai
        type: openai
        credentialsSecretRef:
          name: credentials
        url: https://api.openai.com/v1
        models:
          - name: <model_name>
  ols:
    defaultModel: <model_name>
    defaultProvider: myOpenai
    Copy to Clipboard Toggle word wrap

    Red Hat Enterprise Linux AI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
    - credentialsSecretRef:
        name: <rhelai_api_keys>
    1 
    
      models:
      - name: models/<model_name>
      name: rhelai
      type: rhelai_vllm
      url: <url>
    2 
    
  ols:
    defaultProvider: rhelai
    defaultModel: models/<model_name>
    Copy to Clipboard Toggle word wrap

    1
    By default, the Red Hat Enterprise Linux AI API key requires a token as part of the request. If your Red Hat Enterprise Linux AI configuration does not require a token, you must set the token value to any valid string for the request to authenticate.
    2
    The URL endpoint must end with v1 to be valid. For example, https://http://3.23.103.8:8000/v1.

    Red Hat OpenShift AI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
    - credentialsSecretRef:
        name: <rhoai_api_keys>
    1 
    
      models:
      - name: <model_name>
      name: red_hat_openshift_ai
      type: rhoai_vllm
      url: <url>
    2 
    
  ols:
    defaultProvider: red_hat_openshift_ai
    defaultModel: <model_name>
    Copy to Clipboard Toggle word wrap

    1
    By default, the Red Hat OpenShift AI API key requires a token as part of the request. If your Red Hat OpenShift AI configuration does not require a token, you must set the token value to any valid string for the request to authenticate.
    2
    The URL endpoint must end with v1 to be valid. For example, https://<model_name>.<domain_name>.com:443/v1.

    Microsoft Azure OpenAI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - credentialsSecretRef:
          name: credentials
        apiVersion: <api_version_for_azure_model>
        deploymentName: <azure_ai_deployment_name>
        models:
          - name: <model_name>
        name: myAzure
        type: azure_openai
        url: <azure_ai_deployment_url>
  ols:
    defaultModel: <model_name>
    defaultProvider: myAzure
    Copy to Clipboard Toggle word wrap

    IBM watsonx CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: myWatsonx
        type: watsonx
        credentialsSecretRef:
          name: credentials
        url: <ibm_watsonx_deployment_name>
        projectID: <ibm_watsonx_project_id>
        models:
          - name: ibm/<model_name>
  ols:
    defaultModel: ibm/<model_name>
    defaultProvider: myWatsonx
    Copy to Clipboard Toggle word wrap

  3. Click Create.

1.2.1. Configuring custom TLS certificates
Copy link

Configure custom TLS certificates for secure OpenShift Lightspeed service communication.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create or edit the OLSConfig custom resource (CR).
  • You have a large language model (LLM) provider.
  • You have installed the OpenShift Lightspeed Operator.
  • You have created the credentials secret and the OLSconfig CR.

Procedure

  1. In the OpenShift Container Platform web console, click Operators Installed Operators.
  2. Select All Projects in the Project dropdown at the top of the screen.
  3. Click OpenShift Lightspeed Operator.
  4. Click OLSConfig, then click the cluster configuration instance in the list.
  5. Click the YAML tab.

  6. Modify the OLSconfig CR to contain the file that contains the TLS secret.

    Example credentials secret and the OLSconfig CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    tlsConfig:
      keyCertSecretRef:
        name: <lightspeed_tls>
    1 
    
---
apiVersion: v1
data:
  tls.crt: LS0tLS1CRUd...
    2 
    
  tls.key: LS0tLS1CRUd...
kind: Secret
metadata:
  name: <lightspeed_tls>
  namespace: <openshift_lightspeed>
    Copy to Clipboard Toggle word wrap

    1
    Refers to the secret that contains the tls.crt and tls.key file.
    2
    The name of the certificate must be tls.crt and the name of the key must be tls.key.
  7. Click Save.

Verification

  1. Verify that a new pod was created in the lightspeed-app-server deployment by running the following command: 

    $ oc get pod -n openshift-lightspeed
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                                                     READY   STATUS    RESTARTS   AGE
lightspeed-app-server-5b45d6dd99-5599w                   2/2     Running   2          8h
lightspeed-console-plugin-88d878686-tjt5p                1/1     Running   1          8d
lightspeed-operator-controller-manager-7d7cc4588-p7442   1/1     Running   9          8d
lightspeed-postgres-server-5484fcfdfc-kcpjh              1/1     Running   2          8d
    Copy to Clipboard Toggle word wrap

Create a file that is associated with the API token used to access the API of your large language model (LLM) provider. Typically, you use API tokens to authenticate your LLM provider. Alternatively, Microsoft Azure also supports authentication using Microsoft Entra ID.

Prerequisites

  • You have access to the OpenShift CLI (oc) as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create a secret to store the Provider tokens.
  • You have installed the OpenShift Lightspeed Operator.

Procedure

  1. Create a YAML file that contains the content for the LLM provider that you are using.

    Note

    The YAML parameter is always apitoken regardless of what the LLM provider calls the access details.

    Credential secret for LLM provider

    apiVersion: v1
kind: Secret
metadata:
  name: credentials
  namespace: openshift-lightspeed
type: Opaque
stringData:
  apitoken: <api_token>
    1
    Copy to Clipboard Toggle word wrap

    1
    The api_token is not base64 encoded.

    Credential secret for Red Hat Enterprise Linux AI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: rhelai-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for Red Hat OpenShift AI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: rhoai-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for IBM watsonx

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: watsonx-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Credential secret for Microsoft Azure OpenAI

    apiVersion: v1
data:
  apitoken: <api_token>
    1 
    
kind: Secret
metadata:
  name: azure-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

    1
    The api_token must be base64 encoded when stored in a secret.

    Alternatively, for Microsoft Azure OpenAI you can use Microsoft Entra ID to authenticate your LLM provider. Microsoft Entra ID users must configure the required roles for their Microsoft Azure OpenAI resource. For more information, see the official Microsoft Cognitive Services OpenAI Contributor(Microsoft Azure OpenAI Service documentation).

    Credential secret for Microsoft Entra ID

    apiVersion: v1
data:
  client_id: <base64_encoded_client_id>
  client_secret: <base64_encoded_client_secret>
  tenant_id: <base64_encoded_tenant_id>
kind: Secret
metadata:
  name: azure-api-keys
  namespace: openshift-lightspeed
type: Opaque
    Copy to Clipboard Toggle word wrap

  2. Run the following command to create the secret: 

    $ oc create -f /path/to/secret.yaml
    Copy to Clipboard Toggle word wrap

The Custom Resource (CR) file contains information that the Operator uses to deploy OpenShift Lightspeed. The specific content of the CR file is unique for each large language model (LLM) provider. To create the CR file, choose the configuration file for the LLM provider that you are using.

Prerequisites

  • You have access to the OpenShift CLI (oc) and are logged in as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file.
  • You have an LLM provider available for use with the OpenShift Lightspeed Service.
  • You have installed the OpenShift Lightspeed Operator.

Procedure

  1. Create an OLSConfig file that contains the YAML content for the LLM provider you use.

    OpenAI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: myOpenai
        type: openai
        credentialsSecretRef:
          name: credentials
        url: https://api.openai.com/v1
        models:
          - name: <model_name>
  ols:
    defaultModel: <model_name>
    defaultProvider: myOpenai
    Copy to Clipboard Toggle word wrap

    Red Hat Enterprise Linux AI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
    - credentialsSecretRef:
        name: <rhelai_api_keys>
    1 
    
      models:
      - name: models/<model_name>
      name: rhelai
      type: rhelai_vllm
      url: <url>
    2 
    
  ols:
    defaultProvider: rhelai
    defaultModel: models/<model_name>
    additionalCAConfigMapRef:
      name: openshift-service-ca.crt
    Copy to Clipboard Toggle word wrap

    1
    By default, the Red Hat Enterprise Linux AI API key requires a token as part of the request. If your Red Hat Enterprise Linux AI configuration does not require a token, you must set the token value to any valid string for the request to authenticate.
    2
    The URL endpoint must end with v1 to be valid. For example, https://http://3.23.103.8:8000/v1.

    Red Hat OpenShift AI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
    - credentialsSecretRef:
        name: <rhoai_api_keys>
    1 
    
      models:
      - name: <model_name>
      name: red_hat_openshift_ai
      type: rhoai_vllm
      url: <url>
    2 
    
  ols:
    defaultProvider: red_hat_openshift_ai
    defaultModel: <model_name>
    Copy to Clipboard Toggle word wrap

    1
    By default, the Red Hat OpenShift AI API key requires a token as part of the request. If your Red Hat OpenShift AI configuration does not require a token, you must set the token value to any valid string for the request to authenticate.
    2
    The URL endpoint must end with v1 to be valid. For example, https://<model_name>.<domain_name>.com:443/v1.

    Microsoft Azure OpenAI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - credentialsSecretRef:
          name: credentials
        apiVersion: <api_version_for_azure_model>
        deploymentName: <azure_ai_deployment_name>
        models:
          - name: <model_name>
        name: myAzure
        type: azure_openai
        url: <azure_ai_deployment_url>
  ols:
    defaultModel: <model_name>
    defaultProvider: myAzure
    Copy to Clipboard Toggle word wrap

    IBM watsonx CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: myWatsonx
        type: watsonx
        credentialsSecretRef:
          name: credentials
        url: <ibm_watsonx_deployment_name>
        projectID: <ibm_watsonx_project_id>
        models:
          - name: ibm/<model_name>
  ols:
    defaultModel: ibm/<model_name>
    defaultProvider: myWatsonx
    Copy to Clipboard Toggle word wrap

  2. Run the following command: 

    $ oc create -f /path/to/config-cr.yaml
    Copy to Clipboard Toggle word wrap

    The Operator deploys OpenShift Lightspeed using the information in YAML configuration file.

Configure OpenShift Lightspeed with a trust provider certificate for the large language model (LLM) provider.

Note

If the LLM provider you are using requires a trust certificate to authenticate the OpenShift Lightspeed service you must perform this procedure. If the LLM provider does not require a trust certificate to authenticate the service, you should skip this procedure.

Procedure

  1. Copy the contents of the certificate file and paste it into a file called caCertFileName.

  2. Create a ConfigMap object called trusted-certs by running the following command: 

    $ oc create configmap trusted-certs --from-file=caCertFileName --namespace openshift-lightspeed
    Copy to Clipboard Toggle word wrap

    Example output

    kind: ConfigMap
apiVersion: v1
metadata:
  name: trusted-certs
  namespace: openshift-lightspeed
data:
  caCertFileName: |
    1 
    
    -----BEGIN CERTIFICATE-----
    .
    .
    .
    -----END CERTIFICATE-----
    Copy to Clipboard Toggle word wrap

    1
    Specify the CA certificates required to connect to your LLM provider. You can include one or more certificates.

  3. Update the OLSConfig custom resource file to include the name of the ConfigMap object you just created.

    Example Red Hat Enterprise Linux AI CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    defaultProvider: rhelai
    defaultModel: models/<model_name>
    additionalCAConfigMapRef:
      name: trusted-certs
    1
    Copy to Clipboard Toggle word wrap

    1
    Specifies the name of ConfigMap object.

  4. Create the custom CR. 

    $ oc apply -f <olfconfig_cr_filename>
    Copy to Clipboard Toggle word wrap

1.5. Verifying the OpenShift Lightspeed deployment
Copy link

After the OpenShift Lightspeed service is deployed, verify that it is operating.

Important

Starting with OpenShift Container Platform 4.19, the perspectives in the web console are unified. The Developer perspective is no longer enabled by default.

All users can interact with all OpenShift Container Platform web console features. However, if you are not the cluster owner, you might need to request permission to certain features from the cluster owner.

You can still enable the Developer perspective. On the Getting Started pane in the web console, you can take a tour of the console, find information on setting up your cluster, view a quick start for enabling the Developer perspective, and follow links to explore new features and capabilities.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role.
  • You have access to the OpenShift CLI (oc).
  • You have installed the OpenShift Lightspeed Operator.
  • You have created the credentials secret and the OLSConfig Custom Resource configuration file.

Procedure

  1. In the OpenShift Container Platform web console, click the Project drop-down list.

    Note

    For OpenShift Container Platform 4.18 and earlier, select the Developer perspective from the drop-down list at the top of the pane to access the Project drop-down list.

  2. Enable the toggle switch to show default projects.
  3. Select openshift-lightspeed from the list.

  4. Verify that the OpenShift Lightspeed is ready by running the following command: 

    $ oc logs deployment/lightspeed-app-server -c lightspeed-service-api -n openshift-lightspeed | grep Uvicorn
    Copy to Clipboard Toggle word wrap

    Example output

    INFO: 	Uvicorn running on https://0.0.0.0:8443 (Press CTRL+C to quit)
    Copy to Clipboard Toggle word wrap

Role-Based Access Control (RBAC) is a system security approach to restricting system access to authorized users who have defined roles and permissions.

OpenShift Lightspeed RBAC is binary. By default, not all cluster users have access to the OpenShift Lightspeed interface. Access must be granted by a user who can grant permissions. All users of an OpenShift cluster with OpenShift Lightspeed installed can see the Lightspeed button; however, only users with permissions can submit questions to Lightspeed.

If you want to evaluate the RBAC features of OpenShift Lightspeed, your cluster will need users other than the kubeadmin account. The kubeadmin account always has access to OpenShift Lightspeed.

1.6.1. Granting access to an individual user
Copy link

This procedure explains how to grant access to an individual user.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in as a user with the ability to grant permissions.
  • You have deployed the OpenShift Lightspeed service.
  • You have access to the OpenShift CLI (oc).

Procedure

  • Run the following command at the command line: 

    $ oc adm policy add-cluster-role-to-user \
lightspeed-operator-query-access <user_name>
    Copy to Clipboard Toggle word wrap

    Alternatively, you can use a YAML file when granting access to an individual user by using the following command: 

    $ oc adm policy add-cluster-role-to-user lightspeed-operator-query-access <user_name> -o yaml --dry-run
    Copy to Clipboard Toggle word wrap

    The terminal returns the following output: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: null
  name: lightspeed-operator-query-access
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: lightspeed-operator-query-access
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: <user_name>
    1
    Copy to Clipboard Toggle word wrap
    1
    Enter the actual user name in place of <user_name> before creating the object.

    Save the output as a YAML file, and run the following command to grant user access: 

    $ oc create -f <filename>
    Copy to Clipboard Toggle word wrap

1.6.2. Granting access to a user group
Copy link

This procedure explains how to grant access to a user group. If your cluster has more advanced identity management configured, including user groups, you can grant all users of a specific group access to the OpenShift Lightspeed service.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in as a user with the ability to grant permissions.
  • You have deployed the OpenShift Lightspeed service.
  • You have access to the OpenShift CLI (oc).

Procedure

  • Run the following command at the command line: 

    $ oc adm policy add-cluster-role-to-group \
lightspeed-operator-query-access <group_name>
    Copy to Clipboard Toggle word wrap

    Alternatively, you can use a YAML file when granting access to a user group by using the following command: 

    $ oc adm policy add-cluster-role-to-group lightspeed-operator-query-access <group_name> -o yaml --dry-run
    Copy to Clipboard Toggle word wrap

    The terminal returns the following output: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: null
  name: lightspeed-operator-query-access
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: lightspeed-operator-query-access
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: <user_group>
    1
    Copy to Clipboard Toggle word wrap
    1
    Enter the actual user group in place of <user_group> before creating the object.

    Save the output as a YAML file, and run the following command to grant access to the user group: 

    $ oc create -f <filename>
    Copy to Clipboard Toggle word wrap

1.7. Filtering and redacting information
Copy link

You can configure OpenShift Lightspeed to filter or redact information from being sent to the LLM provider. The following example shows how to modify the OLSConfig file to redact IP addresses.

Note

You should test your regular expressions against sample data to confirm that they are catching the information you want to filter or redact, and that they are not accidentally catching information you do not want to filter or redact. There are several third-party websites that you can use to test your regular expressions. When using third-party sites, you should practice caution with regards to sharing your private data. Alternatively, you can test the regular expressions locally using Python. In Python, it is possible to design very computationally-expensive regular expressions. Using several complex expressions as query filters can adversely impact the performance of OpenShift Lightspeed.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role.
  • You have access to the OpenShift CLI (oc).
  • You have installed the OpenShift Lightspeed Operator and deployed the OpenShift Lightspeed service.

Procedure

  1. Modify the OLSConfig file and create an entry for each regular expression to filter. The following example redacts IP addresses:

    Example custom resource file

    spec:
  ols:
    queryFilters:
      - name: ip-address
        pattern: '((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.?\b){4}'
        replaceWith: <IP_ADDRESS>
    Copy to Clipboard Toggle word wrap

  2. Run the following command to apply the modified OpenShift Lightspeed custom configuration: 

    $ oc apply -f OLSConfig.yaml
    Copy to Clipboard Toggle word wrap

1.8. About the BYO Knowledge tool
Copy link

The OpenShift Lightspeed service uses a large language model (LLM) to generate responses to questions. You can enhance the knowledge that is available to the LLM by using the BYO Knowledge tool to create a retrieval-augmented generation (RAG) database.

When you create a RAG database, you customize the OpenShift Lightspeed service for your environment. For example, a network administrator can develop a standard operating procedure (SOP) that is used to provision an OpenShift Container Platform cluster. Then, the network administrator can use the BYO Knowledge tool to enhance the knowledge available to the LLM by including information from the SOP.

To bring your own knowledge to an LLM, you complete the following steps:

  • Create the custom content in Markdown format.
  • Use the BYO Knowledge tool to package the content as a container image.
  • Push the container image to an image registry, such as quay.io.
  • Update the OLSConfig custom resource file to list the image that you pushed to the image registry.

  • Access the OpenShift Lightspeed virtual assistant and submit a question that is associated with the custom knowledge that you made available to the LLM.

    Note

    When you use the BYO Knowledge tool, the documents that you make available to the LLM are sent to the LLM provider.

1.8.1. Providing custom knowledge to the LLM
Copy link

Customize the information available to the large language model (LLM) by providing access to a container image that resides in a remote image registry. The examples in this procedure use quay.io as the remote container image registry, and the path for the custom image is quay.io/<username>/my-byok-image:latest.

Important

The BYO Knowledge tool is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user account that has permission to create a cluster-scoped custom resource (CR) file, such as a user with the cluster-admin role.
  • You have an LLM provider available for use with the OpenShift Lightspeed Service.
  • You have installed the OpenShift Lightspeed Operator.
  • The custom information you want to add resides as a collection of Markdown files with .md extensions. No other file format is supported.
  • You have logged in to registry.redhat.io by using Podman.
  • You have an account for a container image registry, such as quay.io.

Procedure

  1. Specify the location of the directory with the Markdown files for the retrieval-augmented generation (RAG) database and the path for the image that the BYO Knowledge tool generates by running the following command: 

    $ podman run -it --rm --device=/dev/fuse \
  -v $XDG_RUNTIME_DIR/containers/auth.json:/run/user/0/containers/auth.json:Z \
  -v <dir_tree_with_markdown_files>:/markdown:Z \
  -v <dir_for_image_tar>:/output:Z \
  registry.redhat.io/openshift-lightspeed-tech-preview/lightspeed-rag-tool-rhel9:latest
    Copy to Clipboard Toggle word wrap
    Note

    The container image that the tool generates contains a custom RAG database. The RAG database provides additional information to the LLM.

  2. Load the container image that the BYO Knowledge tool generated by running the following command: 

    $ podman load < <directory_for_image_tar>/<my-byok-image.tar>
    Copy to Clipboard Toggle word wrap

  3. Display the podman images that are on your local computer by running the following command: 

    $ podman images
    Copy to Clipboard Toggle word wrap

    Example output

    REPOSITORY                            TAG                IMAGE ID      CREATED       SIZE
localhost/my-byok-image               latest             be7d1770bf10  1 minute  ago    2.37 GB
...
    Copy to Clipboard Toggle word wrap

  4. Tag the local image with a name and destination so that the image can be pushed to the container image registry by running the following command: 

    $ podman tag localhost/my-byok-image:latest quay.io/<username>/my-byok-image:latest
    Copy to Clipboard Toggle word wrap

  5. Push the local container image to the container image registry by running the following command: 

    $ podman push quay.io/<username>/my-byok-image:latest
    Copy to Clipboard Toggle word wrap

  6. Modify the OLSconfig CR to deploy the newly created RAG database alongside the existing one:

    1. In the OpenShift Container Platform web console, click Operators Installed Operators.
    2. Select All Projects in the Project dropdown at the top of the screen.
    3. Click OpenShift Lightspeed Operator.
    4. Click OLSConfig, then click the cluster configuration instance in the list.
    5. Click the YAML tab.

    6. Insert the spec.ols.rag yaml code:

      Example OLSconfig CR file

      apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    rag:
      - image: quay.io/<username>/my-byok-image:latest
      1
      Copy to Clipboard Toggle word wrap

      1
      Where image specifies the tag for the image that was pushed to the image registry so that the OpenShift Lightspeed Operator can access the custom content. The OpenShift Lightspeed Operator can work with more than one RAG database that you create.
  7. Click Save.

Verification

  • Access the OpenShift Lightspeed virtual assistant and submit a question associated with the custom content that you provided to the LLM.

    The OpenShift Lightspeed virtual assistant generates a response based on the custom content.

1.9. About cluster interaction
Copy link

The OpenShift Lightspeed Service uses a large language model (LLM) to generate responses to questions. You can enable the cluster interaction feature to enhance the knowledge available to the LLM with information about your OpenShift Container Platform cluster. Providing information about the Kubernetes objects that the cluster contains enables the LLM to generate highly specific responses for your environment.

The Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to an LLM. Using the protocol, an MCP server offers a standardized way for an LLM to increase context by requesting and receiving real-time information from external resources.

When you enable cluster interaction, the OpenShift Lightspeed Operator installs an MCP server. The MCP server provides the OpenShift Lightspeed Service with access to the OpenShift API. Through this access, the Service performs read operations to gather more context for the LLM, enabling the service to answer questions about the Kubernetes objects that reside in your OpenShift cluster.

Note

The ability of OpenShift Lightspeed to choose and use a tool effectively is very sensitive to the large language (LLM) model. In general, a larger model with more parameters performs better, and the best performance comes from an extremely large frontier model that represents the latest AI capabilities. When using a small model, you might notice poor performance in tool selection or other aspects of cluster interaction.

To activate the cluster interaction feature in the OpenShift Lightspeed Service, tool calling must be enabled in the LLM provider.

Note

Enabling tool calling can dramatically increase token usage. When you use public model providers, increased token usage can increase billing costs.

1.9.1. Enabling cluster interaction
Copy link

Modify the OLSConfig custom resource to enable the cluster interaction feature.

Important

The cluster interaction feature is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user with the cluster-admin role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR.
  • You have configured the large language model (LLM) provider.
  • You have installed the OpenShift Lightspeed Operator.

Procedure

  1. In the OpenShift Container Platform web console, click Operators Installed Operators.
  2. Click OpenShift Lightspeed Operator.
  3. Click OLSConfig, then click the cluster configuration instance in the list.
  4. Click the YAML tab.

  5. Set the spec.ols.introspectionEnabled parameter to true to enable cluster interaction: 

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    introspectionEnabled: true
    Copy to Clipboard Toggle word wrap
  6. Click Save.

Verification

  • Access the OpenShift Lightspeed virtual assistant and submit a question associated with your cluster.

    The OpenShift Lightspeed virtual assistant generates a highly refined response specific to your environment.

1.9.2. Enabling a custom MCP server
Copy link

Add an additional MCP server that interfaces with a tool in your environment so that the large language model (LLM) uses the tool to generate answers to your questions.

Important

The cluster interaction feature is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Prerequisites

  • You have installed the OpenShift Lightspeed Operator.
  • You have configured a large language model provider.
  • You have configured the OLSConfig CR file, which automatically deploys the OpenShift Lightspeed Service.

Procedure

  1. Open the OpenShift Lightspeed OLSconfig custom resource (CR) file by running the following command: 

    $ oc edit olsconfig cluster
    Copy to Clipboard Toggle word wrap

  2. Add MCPServer to the spec.featureGates specification file and include the MCP server information. 

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  featureGates:
    - MCPServer
    1 
    
  mcpServers:
    - name: mcp-server-1
    2 
    
      streamableHTTP:
        url: http://localhost:8080/mcp
    3 
    
        timeout: 30
        sseReadTimeout: 10
        headers:
          - Authorization: Bearer <token>
          - Content-Type: application/json
          - Accept: application/json
        enableSSE: true
    - name: mcp-server-2
      streamableHTTP:
        url: http://localhost:8080/mcp
        timeout: 30
    4 
    
        sseReadTimeout: 10
    5 
    
        headers:
          - <key1>: <value1>
    6 
    
          - <key2>: <value2>
        enableSSE: true
    7
    Copy to Clipboard Toggle word wrap
    1
    Specifies the MCP server functionality.
    2
    Specifies the name of the MCP server.
    3
    Specifies the URL path that the MCP server uses to communicate.
    4
    Specifies the time that the MCP server has to respond to a query. If the client does not receive a query within the time specified, the MCP server times out. In this example, the timeout is 30 seconds.
    5
    Specifies the amount of time a client waits for new data from a Server-Sent Events (SSE) connection. If the client does not receive data within that time, the client closes the connection.
    6
    Specifies the additional header that the HTTP request sends to the MCP server.
    7
    When you set enableSSE to true, the MCP server establishes a one-way channel that the MCP server uses to push updates to the client whenever the server has new information. The default setting is false.

  3. Click Save.

    The save operation saves the file and applies the changes so that the MCP server is available to the OpenShift Lightspeed service.

1.10. Tokens and token quota limits
Copy link

Tokens are small chunks of text, which can be as small as one character or as large as one word. Tokens are the units of measurement used to quantify the amount of text that the OpenShift Lightspeed service sends to, or receives from, a large language model (LLM). Every interaction with the service and the LLM is counted in tokens.

Token quota limits define the number of tokens that can be used in a certain timeframe. Implementing token quota limits helps control costs, encourage more efficient use of queries, and regulate system demands. In a multi-user configuration, token quota limits help provide equal access to all users ensuring everyone has an opportunity to submit queries.

You can define token quota limits for OpenShift clusters or OpenShift user accounts.

1.10.1. Activating token quota limits
Copy link

Activate token quota limits for the OpenShift Lightspeed service by defining key-value pairs in the ConfigMap resource. The OpenShift Lightspeed pod mounts the ConfigMap resource as a volume, enabling access to the file stored within it. The OLSConfig custom resource (CR) references the ConfigMap resource to obtain the quota limit information.

Prerequisites

  • You have installed the the OpenShift Lightspeed Operator.
  • You have configured a large language model provider.
  • A PostgreSQL database is configured and the OpenShift Lightspeed service can access the database.

Procedure

  1. Open the OpenShift Lightspeed OLSconfig CR file by running the following command: 

    $ oc edit olsconfig cluster
    Copy to Clipboard Toggle word wrap

  2. Modify the spec.ols.quotaHandlersConfig specification to include token quota limit information.

    Example OpenShift Lightspeed OLSConfig CR

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    quotaHandlersConfig:
      limitersConfig:
      - name: user_limits
    1 
    
        type: user_limiter
        initialQuota: 100000
    2 
    
        quotaIncrease: 1000
    3 
    
        period: 30 days
      - name: cluster_limits
    4 
    
        type: cluster_limiter
        initialQuota: 1000000
    5 
    
        quotaIncrease: 100000
    6 
    
        period: 30 days
    7
    Copy to Clipboard Toggle word wrap

    1
    Specifies the token limit for user account.
    2
    Specifies a token quota limit of 100,000 for each user over the time period specified in the period field.
    3
    Increases the token quota limit for the user by 1,000 at the end of the time period specified in the period field.
    4
    Specifies the token limit for a cluster.
    5
    Specifies a token quota limit of 1,000,000 for each cluster over the time period specified in the period field.
    6
    Increases the token quota limit for the cluster by 100,000 at the end of the time period specified in the period field.
    7
    Defines the amount of time that the scheduler waits before the period resets or the quota limit increases.

  3. Click Save.

    The save operation saves the file and applies the changes to activate the token quota limits.

1.11. About Lightspeed and PostgreSQL persistence
Copy link

Using PostgreSQL persistence, you can save data, such as OpenShift Lightspeed conversation history or quota usage, to a PostgreSQL database so that the data is retained in persistent volume. With PostgreSQL persistence, the data persists if the PostgreSQL pod restarts or is rescheduled in your OpenShift Container Platform installation.

Important

PostgreSQL persistence is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

PostgreSQL persistence is disabled by default. To enable the functionality, add the spec.ols.storage specification to the OLSConfig custom resource (CR).

1.11.1. Enabling PostgreSQL persistence
Copy link

Modify the OLSConfig custom resource (CR) file to enable PostgreSQL persistence for the OpenShift Lightspeed service.

Prerequisites

  • You are logged in to the OpenShift Container Platform web console as a user account with permission to create a cluster-scoped CR file, such as a user with the cluster-admin role.
  • You have installed the OpenShift Lightspeed Operator.
  • You have configured the large language model provider.

Procedure

  1. In the OpenShift Container Platform web console, click Operators Installed Operators.
  2. Select All Projects in the Project list at the top of the screen.
  3. Click OpenShift Lightspeed Operator.
  4. Click OLSConfig, then click the cluster configuration instance in the list.
  5. Click the YAML tab.

  6. Insert the spec.ols.storage YAML code.

    Example OLSconfig CR file

    apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
  namespace: openshift-lightspeed
spec:
  llm:
    providers:
...
  ols:
    storage: {}
    1
    Copy to Clipboard Toggle word wrap

    1
    The class depends on the existing instances of the storage class in the cluster. If you leave the storage class empty, Lightspeed uses default values. The persistent volume allocated for the PostgreSQL database is 1 GB in size and uses the storage class of the default cluster. Specify empty braces for the storage parameter to use the default values.

    If you want to change the size and class parameters, you can independently specify an explicit value for the parameter. 

    ols:
  storage:
    size: 768Mi
    1 
    
    class: gp2-csi
    2
    Copy to Clipboard Toggle word wrap
    1
    Specifies the size of the Requested Volume. If the size is not specified, the default value is 1 GB.
    2
    Specifies the Storage Class of the Requested Volume. If no class is specified, the storage class uses the default storage class setting for the cluster.
  7. Click Save.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat