1.10. Tokens and token quota limits


Tokens are small chunks of text, which can be as small as one character or as large as one word. Tokens are the units of measurement used to quantify the amount of text that the OpenShift Lightspeed service sends to, or receives from, a large language model (LLM). Every interaction with the service and the LLM is counted in tokens.

Token quota limits define the number of tokens that can be used in a certain timeframe. Implementing token quota limits helps control costs, encourage more efficient use of queries, and regulate system demands. In a multi-user configuration, token quota limits help provide equal access to all users ensuring everyone has an opportunity to submit queries.

You can define token quota limits for OpenShift clusters or OpenShift user accounts.

1.10.1. Activating token quota limits

Activate token quota limits for the OpenShift Lightspeed service by defining key-value pairs in the ConfigMap resource. The OpenShift Lightspeed pod mounts the ConfigMap resource as a volume, enabling access to the file stored within it. The OLSConfig custom resource (CR) references the ConfigMap resource to obtain the quota limit information.

Prerequisites

  • You have installed the the OpenShift Lightspeed Operator.
  • You have configured a large language model provider.
  • A PostgreSQL database is configured and the OpenShift Lightspeed service can access the database.

Procedure

  1. Open the OpenShift Lightspeed OLSconfig CR file by running the following command:

    $ oc edit olsconfig cluster
  2. Modify the spec.ols.quotaHandlersConfig specification to include token quota limit information.

    Example OpenShift Lightspeed OLSConfig CR

    apiVersion: ols.openshift.io/v1alpha1
    kind: OLSConfig
    metadata:
      name: cluster
    spec:
      ols:
        quotaHandlersConfig:
          limitersConfig:
          - name: user_limits 
    1
    
            type: user_limiter
            initialQuota: 100000 
    2
    
            quotaIncrease: 1000 
    3
    
            period: 30 days
          - name: cluster_limits 
    4
    
            type: cluster_limiter
            initialQuota: 1000000 
    5
    
            quotaIncrease: 100000 
    6
    
            period: 30 days 
    7

    1
    Specifies the token limit for user account.
    2
    Specifies a token quota limit of 100,000 for each user over the time period specified in the period field.
    3
    Increases the token quota limit for the user by 1,000 at the end of the time period specified in the period field.
    4
    Specifies the token limit for a cluster.
    5
    Specifies a token quota limit of 1,000,000 for each cluster over the time period specified in the period field.
    6
    Increases the token quota limit for the cluster by 100,000 at the end of the time period specified in the period field.
    7
    Defines the amount of time that the scheduler waits before the period resets or the quota limit increases.
  3. Click Save.

    The save operation saves the file and applies the changes to activate the token quota limits.

Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

關於紅帽

我们提供强化的解决方案,使企业能够更轻松地跨平台和环境(从核心数据中心到网络边缘)工作。

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情,请参阅红帽博客.

关于红帽文档

Legal Notice

Theme

© 2026 Red Hat
返回顶部