Chapter 1. Deploy and manage Models-as-a-Service

Procedure

1. In the OpenShift AI dashboard, click Settings Subscriptions.

2. Review the information in the Subscriptions table:

   Name

   The unique identifier for the subscription.

   Phase

   The current status of the subscription. Possible values: Active, Failed.

   Groups

   The number of OCP or custom user groups assigned to the subscription.

   Models

   The count of Models-as-a-Service (MaaS) model references included in the subscription.

   Priority

   The priority level assigned to the subscription. Higher numbers indicate higher priority.

3. Optional: To filter and organize the view:

   • Use the Keyword dropdown to select a filter criterion.
   • Enter text in the Filter by name or description field to search for specific subscriptions.
   • Click the column headers to sort by Name, Phase, Groups, Models, or Priority.
4. Click a subscription name to open the details pane.
5. In the details view, review the subscription configuration including groups, models, token limits, and synchronization status.

Procedure

1. In the OpenShift AI dashboard, click Settings Subscriptions.
2. Click Create subscription.

3. In the Create subscription form, configure the subscription:

   1. Name: Enter a descriptive name for the subscription.
   2. Optional: Click Edit resource name to set a custom internal identifier for the subscription. If not specified, the resource name is generated automatically from the display name.
   3. Optional: Description: Provide a brief description of the purpose of the subscription.
   4. Priority: Set the subscription priority level using a numeric value. Higher numbers indicate higher priority. When a user belongs to multiple groups with different subscriptions, the subscription with the highest priority level is used.
   5. Groups: Select OpenShift groups or enter custom group names from your external OIDC provider. Users who are members of these groups can access the models included in this subscription.

4. Configure models and token limits:

   1. Click Add models.
   2. In the Add models dialog, review the available models and their associated projects, model IDs, existing subscriptions, and policies.
   3. Click Add model for each model you want to include in this subscription.
   4. Click Add models to confirm your selection.
   5. For each model in the subscription, click Add token limit to configure token consumption limits.
   6. In the Edit subscription token limits dialog, enter the number of tokens allowed.
   7. Enter the time period value.
   8. Select the time unit: hour, minute, or second.
   9. Optional: Click Add token rate limit to configure additional token limits with different time windows.

   10. Click Save.

       Note

       At least one token limit is required for each model in the subscription.

5. Optional: Select Create a matching authorization policy to automatically create an authorization policy for this subscription.

   Note

   To consume model endpoints through the API gateway, users must have both a subscription and an authorization policy. A subscription defines quota for models. An authorization policy is a separate resource that authorizes specific groups to access model endpoints through the API gateway. While you can automatically create a matching authorization policy during subscription creation, creating authorization policies separately provides more flexibility and control.

   Important

   The matching authorization policy feature creates an initial authorization policy with the same groups and models as the subscription. However, changes made to the subscription later such as adding or removing groups or models require manual updates to the authorization policy. When you modify a subscription, you must manually update the corresponding authorization policy to keep them synchronized.

6. Click Create subscription.

Procedure

1. In the OpenShift AI dashboard, click Settings Subscriptions.
2. Locate the subscription you want to modify and click its name.
3. Click the action menu (⋮), and then select Edit.

4. Modify the subscription properties:

   1. Update the name or description if needed.
   2. Adjust the priority level.
   3. Add or remove groups.
   4. Add or remove models by clicking Add models or using the action menu (⋮) next to each model.

   5. Modify token limits for existing models by clicking Add token limit or editing existing limits.

      Important

      The metadata.name of the MaaSSubscription custom resource cannot be changed after creation. If you need a different resource name, delete the subscription and create a new one.

5. Click Save.

   Important

   If you created a matching authorization policy when you created this subscription, changes to the subscription such as adding or removing groups or models require manual updates to the authorization policy. After modifying a subscription, manually update the corresponding authorization policy to keep them synchronized.

Procedure

1. In the OpenShift AI dashboard, click Settings Subscriptions.
2. Locate the subscription you want to delete and click its name.
3. Click the action menu (⋮), and then select Delete.

4. In the Delete subscription dialog, review the impact of deletion:

   • Users in groups assigned to this subscription lose access to the included models.
   • All API keys bound to this subscription are invalidated.
   • Authorization policies are not deleted automatically. After deletion, review and update any authorization policies that referenced this subscription’s groups or models.
5. To confirm, type the subscription name and click Delete.

Procedure

1. In the OpenShift AI dashboard, click Settings Authorization policies.

   The Authorization policies page displays a table with the following columns:

   • Name: The name of the authorization policy. Policies created automatically from subscriptions include the subscription name in the policy name.
   • Phase: The current status of the policy. Possible values: Active, Failed, Unknown.
   • Groups: The number of OpenShift groups authorized by this policy.
   • Models: The number of model endpoints included in this policy.

2. Optional: Filter the list of authorization policies:

   • To filter by keyword, click the Keyword dropdown and select a filter option.
   • To search by name or description, enter text in the search field.
3. Optional: Sort the table by clicking any column header.

4. To view details of a specific authorization policy:

   1. Click the action menu (⋮) for the policy you want to view.

   2. Select View details.

      The policy details page displays:

      • Policy name, description, phase, and resource name
      • Creation date
      • Groups authorized by this policy
      • Models included in this policy, along with the namespace each model is deployed in

Procedure

1. In the OpenShift AI dashboard, click Settings Authorization policies.
2. Click Create authorization policy.

3. In the Create authorization policy dialog, configure the following settings:

   1. In the Name field, enter a unique name for the authorization policy.

      Note

      By default, the resource name matches the policy name. To customize the resource name, click Edit resource name and enter a different value. The resource name identifies the underlying MaaSAuthPolicy custom resource in OpenShift.

   2. Optional: In the Description field, enter a description explaining the purpose of the policy.

   3. From the Groups dropdown, select the groups to authorize for API gateway access.

      You can select multiple groups or type to add a new group name. Groups can come from OpenShift Group objects, API key group snapshots, or OIDC token claims depending on how users authenticate.

   4. In the Models section, click Add models.
   5. In the Add models to authorization policy dialog, browse the list of available models or use the search field to filter by name or description.
   6. Review the Subscriptions and Policies columns to see which subscriptions and policies already include each model.
   7. For each model you want to add, click Add model.

   8. Click Add models to add the selected models to the policy.

      The Models section displays a table with the selected models and their project namespaces.

4. Click Create authorization policy.

Procedure

1. In the OpenShift AI dashboard, click Settings Authorization policies.
2. In the row for the authorization policy you want to edit, click the action menu (⋮), and then select Edit.

3. In the Edit authorization policy dialog, modify the following settings as needed:

   1. Update the Name field to change the policy display name.
   2. Update the Description field to change the policy description.

   3. From the Groups dropdown, add or remove groups:

      • To add groups, select additional groups from the dropdown or type to add a new group name.

      • To remove a group, click the remove icon (×) next to the group name.

        Note

        The available groups depend on your authentication configuration: OpenShift groups when using OpenShift authentication, or OIDC group claims when using external OIDC.

   4. In the Models section, add or remove models:

      • To add models, click Add models to open the Add models to authorization policy dialog. Select the models to add, and then click Add models in the dialog to confirm.
      • To remove a model, click the action menu (⋮) for the model, and then select Remove.
4. Click Save to apply your changes.

Procedure

1. In the OpenShift AI dashboard, click Settings Authorization policies.
2. In the row for the authorization policy you want to delete, click the action menu (⋮), and then select Delete.
3. In the Delete policy dialog, enter the authorization policy name to confirm deletion.
4. Click Delete.

Procedure

1. In the OpenShift AI dashboard, click Gen AI studio API keys.

   The API keys page displays a table with the following columns:

   • Name: The name assigned to the API key
   • Status: The current state of the key. Possible values: Active, Expired, Revoked.
   • Subscription: The subscription that scopes the key’s access to models
   • Owner: The username of the key owner
   • Created: The date when the key was created
   • Last used: The date when the key was last used to access a model
   • Expires: The expiration date for the key

2. Optional: Filter the list of API keys:

   • To filter by status, click the Status dropdown and select Active, Expired, or Revoked.
   • To filter by username, enter a username in the search field.
3. Optional: Sort the table by clicking any column header.
4. Optional: If the list contains more keys than fit on one page, use the pagination controls at the bottom of the table to navigate between pages.

Procedure

1. In the OpenShift AI dashboard, click Gen AI studio API keys.
2. Click Create API key.

3. In the Create API key dialog, configure the following settings:

   1. In the Name field, enter a descriptive name for the API key.
   2. Optional: In the Description field, enter additional details about what the key is for.

   3. From the Subscription dropdown, select the subscription that determines which models the key can access and the applicable token limits.

      The Models section displays the models included in the selected subscription and their configured token limits.

   4. From the Expiration dropdown, select the number of days until the key expires.

      Note

      You can select an expiration period from 1 to 365 days. The default expiration is 30 days. As an administrator, you can set a maximum expiration limit in the Tenant custom resource. If not set, the default maximum is 90 days.

4. Click Create.

   The API key created dialog displays the generated key with a prefix of sk-oai-.

   Important

   The plaintext key is displayed only during creation and cannot be retrieved later. Save the key in a secrets manager before closing the API key created dialog. If you lose the key, you must revoke it and create a new one.

5. Click the copy icon next to the API key field to copy the key, and then provide it to the user through a secure channel.
6. Click Close.

1. In the OpenShift AI dashboard, click Gen AI studio API keys.
2. In the row for the API key you want to revoke, click the action menu (⋮) and select Revoke.
3. In the Revoke API key? dialog, enter the API key name to confirm revocation.
4. Click Revoke.

1. In the OpenShift AI dashboard, click Gen AI studio API keys.
2. Click the action menu (⋮) in the table header and select Revoke user API keys.
3. In the Revoke user API keys dialog, enter the username in the Username field.
4. Click the search icon to display the keys for that user.
5. Click Revoke all keys to revoke all API keys for the user.

Procedure

1. Create a YAML file named, for example, maasmodelref.yaml, defining the MaaSModelRef custom resource:

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSModelRef
   metadata:
     name: <model-ref-name>
     namespace: <model-namespace>
   spec:
     modelRef:
       kind: <backend-kind>
       name: <backend-resource-name>

   where:

   metadata.name

   Specifies a unique resource name for the model reference.

   metadata.namespace

   Specifies the namespace where the model reference is created. This must be the same namespace as the backend resource.

   spec.modelRef.kind

   Specifies the backend resource type. Allowed values: LLMInferenceService, ExternalModel.

   spec.modelRef.name

   Specifies the name of the backend resource (for example, an LLMInferenceService or ExternalModel resource).

   Note

   For models served using the vLLM runtime or LLM-D distributed inference, use kind: LLMInferenceService. For models hosted by external providers such as OpenAI or Anthropic, use kind: ExternalModel.

   Example configuration for vLLM model

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSModelRef
   metadata:
     name: llama-3-8b-instruct
     namespace: model-serving
   spec:
     modelRef:
       kind: LLMInferenceService
       name: llama-3-8b-instruct

   Example configuration for external model

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSModelRef
   metadata:
     name: gpt-4-turbo
     namespace: external-models
   spec:
     modelRef:
       kind: ExternalModel
       name: gpt-4-turbo

2. Optional: If you need to override the auto-discovered endpoint URL, add the endpointOverride field:

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSModelRef
   metadata:
     name: <model-ref-name>
     namespace: <model-namespace>
   spec:
     modelRef:
       kind: <backend-kind>
       name: <backend-resource-name>
     endpointOverride: <custom-endpoint-url>

3. Apply the YAML file:

   $ oc apply -f maasmodelref.yaml

Procedure

1. Create a YAML file defining the MaaSSubscription custom resource:

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSSubscription
   metadata:
     name: <subscription-name>
     namespace: models-as-a-service
   spec:
     owner:
       groups:
         - kind: Group
           name: <group-1>
         - kind: Group
           name: <group-2>
       users:
         - <user-1>
         - <user-2>
     modelRefs:
       - name: <model-name>
         namespace: <model-namespace>
         tokenRateLimits:
           - limit: <token-limit>
             window: "<time-window>"
           - limit: <token-limit>
             window: "<time-window>"
     priority: <priority-level>

   where:

   metadata.name

   Specifies the Kubernetes resource name for the subscription.

   metadata.namespace

   Specifies the namespace where the subscription is created. This must be models-as-a-service.

   spec.owner.groups

   Lists the groups whose members can use this subscription. Each group entry requires kind: Group and a name field that references an OpenShift Group object or external OIDC group.

   spec.owner.users

   Lists individual users who can use this subscription. Users must be valid Kubernetes user identities.

   spec.modelRefs

   Defines the models available in this subscription with their token rate limits.

   spec.modelRefs.name

   Specifies the MaaSModelRef resource name.

   spec.modelRefs.namespace

   Specifies the namespace containing the MaaSModelRef resource.

   spec.modelRefs.tokenRateLimits

   Configures token consumption limits with time windows. At least one rate limit is required per model.

   spec.modelRefs.tokenRateLimits.limit

   Specifies the maximum number of tokens allowed within the time window.

   spec.modelRefs.tokenRateLimits.window

   Specifies the time window for the rate limit. Supported units: s (seconds), m (minutes), h (hours). Pattern: 1-9999 followed by a unit (for example, 30s, 5m, 1h, 24h).

   spec.priority

   Sets the subscription priority level when a user has multiple subscriptions. Higher numbers indicate higher priority. Default: 0.

   Note

   At least one of groups or users must be specified under owner. Multiple token rate limits can be configured for each model to enforce different time windows (for example, per-minute and per-day limits).

   Important

   The window field no longer accepts d (days) as a unit. Use hours instead. For example, use 24h instead of 1d, or 168h for a week.

   Example configuration

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSSubscription
   metadata:
     name: data-science-team
     namespace: models-as-a-service
   spec:
     owner:
       groups:
         - kind: Group
           name: data-scientists
         - kind: Group
           name: ml-engineers
       users:
         - alice@example.com
     modelRefs:
       - name: llama-3-8b-instruct
         namespace: model-serving
         tokenRateLimits:
           - limit: 1000000
             window: "1h"
           - limit: 5000000
             window: "24h"
       - name: granite-7b-lab
         namespace: model-serving
         tokenRateLimits:
           - limit: 500000
             window: "1h"
     priority: 10

2. Apply the YAML file:

   $ oc apply -f <subscription-file>.yaml

3. Create a matching authorization policy:

   $ oc apply -f - <<EOF
   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSAuthPolicy
   metadata:
     name: <policy-name>
     namespace: models-as-a-service
   spec:
     subjects:
       groups:
         - name: <group-1>
         - name: <group-2>
       users:
         - <user-1>
         - <user-2>
     modelRefs:
       - name: <model-name>
         namespace: <model-namespace>
   EOF

   Authorization policies are required in addition to subscriptions to grant API gateway access.

Procedure

1. Create a YAML file defining the MaaSAuthPolicy custom resource:

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSAuthPolicy
   metadata:
     name: <policy-name>
     namespace: models-as-a-service
   spec:
     subjects:
       groups:
         - name: <group-1>
         - name: <group-2>
       users:
         - <user-1>
         - <user-2>
     modelRefs:
       - name: <model-name-1>
         namespace: <model-namespace>
       - name: <model-name-2>
         namespace: <model-namespace>

   where:

   metadata.name

   Specifies the Kubernetes resource name for the authorization policy.

   metadata.namespace

   Specifies the namespace where the authorization policy is created. This must be models-as-a-service.

   spec.subjects.groups

   Lists the groups authorized to access model endpoints. Each group entry requires a name field that references a Kubernetes Group object or external OIDC group.

   spec.subjects.users

   Lists the individual users authorized to access model endpoints. Users must be valid Kubernetes user identities.

   spec.modelRefs

   Defines the model endpoints that authorized subjects can access.

   spec.modelRefs.name

   Specifies the MaaSModelRef resource name.

   spec.modelRefs.namespace

   Specifies the namespace containing the MaaSModelRef resource.

   Note

   The authorization policy uses OR logic: any matching group or user grants access to all specified models. At least one of groups or users must be specified under subjects.

   Example configuration

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSAuthPolicy
   metadata:
     name: ml-team-auth-policy
     namespace: models-as-a-service
   spec:
     subjects:
       groups:
         - name: ml-engineers
         - name: data-scientists
       users:
         - alice@example.com
     modelRefs:
       - name: llama-3-8b-instruct
         namespace: model-serving
       - name: granite-7b-lab
         namespace: model-serving

2. Optional: Add metering metadata for billing attribution and cost tracking:

   apiVersion: models.opendatahub.io/v1alpha1
   kind: MaaSAuthPolicy
   metadata:
     name: <policy-name>
     namespace: models-as-a-service
   spec:
     subjects:
       groups:
         - name: <group-name>
     modelRefs:
       - name: <model-name>
         namespace: <model-namespace>
     meteringMetadata:
       organizationId: <organization-id>
       costCenter: <cost-center>
       labels:
         <key>: <value>

   where:

   meteringMetadata.organizationId

   Specifies the organization identifier for billing purposes.

   meteringMetadata.costCenter

   Specifies the cost center for billing attribution.

   meteringMetadata.labels

   Provides additional key-value pairs for tracking and reporting.

3. Apply the YAML file:

   $ oc apply -f <auth-policy-file>.yaml

Procedure

1. Obtain the MaaS gateway URL:

   $ oc get gateway maas-default-gateway -n openshift-ingress -o jsonpath='{.status.addresses[0].value}'

2. Obtain an authentication token:

   For OpenShift users

   $ oc whoami -t

   For external OIDC users

   $ curl -X POST "<oidc_token_endpoint>" \
     -d "client_id=<client_id>" \
     -d "client_secret=<client_secret>" \
     -d "grant_type=client_credentials"

3. List the available subscriptions:

   $ curl -X GET https://<maas_gateway_url>/maas-api/v1/subscriptions \
     -H "Authorization: Bearer <auth_token>"

   Note the subscription name you want to associate with the API key. Administrators can view all subscriptions, while regular users can view only subscriptions associated with their user or group.

4. To create an API key, send a POST request to the API keys endpoint:

   $ curl -X POST https://<maas_gateway_url>/maas-api/v1/api-keys \
     -H "Authorization: Bearer <auth_token>" \
     -H "Content-Type: application/json" \
     -d { "name": "<key_name>", "subscription": "<subscription_name>", "expiresInDays": <days> }

   where:

   <maas_gateway_url>

   Specifies your MaaS gateway URL.

   <auth_token>

   Specifies the authentication token obtained in step 2.

   <key_name>

   Specifies a descriptive name for the API key.

   <subscription_name>

   Specifies the subscription name that scopes the key’s access.

   <days>

   Specifies the number of days until the key expires. The maximum expiration period is configured in the Tenant custom resource. If not set, the default maximum is 90 days.

   Important

   The plain text key is returned only in this response and cannot be retrieved later. Save the key in a secrets manager or other secure storage before sending further requests.

5. To list API keys, send a GET request to the API keys endpoint:

   $ curl -X GET https://<maas_gateway_url>/maas-api/v1/api-keys \
     -H "Authorization: Bearer <auth_token>"

   The API returns a list of API keys with their names, subscription associations, creation dates, and expiration dates.

6. To revoke an API key, send a DELETE request to the API key endpoint:

   $ curl -X DELETE https://<maas_gateway_url>/maas-api/v1/api-keys/<key_id> \
     -H "Authorization: Bearer <auth_token>"

   where:

   <key_id>

   Specifies the ID of the API key to revoke. You can obtain the key ID from the list API keys response.

Procedure

1. Enable Kuadrant observability using the OpenShift console:

   1. In the OpenShift console, navigate to Administration CustomResourceDefinitions.
   2. Search for Kuadrant and click the resource name.
   3. Click the Instances tab.
   4. Click kuadrant.
   5. Click the YAML tab.

   6. In the YAML editor, locate the spec section and add or update the observability configuration:

      spec:
        observability:
          enable: true

      The patch uses the following fields:

      enable: true

      Enables the Kuadrant observability stack, which creates a PodMonitor resource that configures Prometheus to scrape rate-limiting metrics from Limitador. These metrics are used by the MaaS observability dashboard to track token consumption, request counts, and rate-limit violations.

   7. Click Save.

      Alternatively, you can configure the resource using the command line:

      $ oc patch kuadrant kuadrant -n kuadrant-system \
        --type merge \
        -p '{"spec":{"observability":{"enable":true}}}'

Procedure

1. Enable telemetry using the OpenShift console:

   1. In the OpenShift console, navigate to Administration CustomResourceDefinitions.
   2. Search for Tenant and click the resource name.
   3. Click the Instances tab.
   4. Click default-tenant.
   5. Click the YAML tab.

   6. In the YAML editor, locate the spec section and add or update the telemetry configuration:

      spec:
        telemetry:
          enabled: true
          metrics:
            captureOrganization: true
            captureUser: false
            captureGroup: false
            captureModelUsage: true

      The patch uses the following fields:

      enabled: true

      Activates TelemetryPolicy and Istio Telemetry to collect MaaS usage metrics.

      captureOrganization

      Includes organization identifiers in metrics. Default is true.

      captureUser

      Includes user labels in metrics. Default is false due to privacy and cardinality considerations. Enabling this option with a large number of users can significantly increase Prometheus database size.

      captureGroup

      Includes group labels in metrics. Default is false to reduce metric cardinality.

      captureModelUsage

      Tracks model-specific usage patterns. Default is true.

   7. Click Save.

      Alternatively, you can configure the resource using the command line:

      $ oc patch tenants.maas.opendatahub.io default-tenant -n models-as-a-service \
        --type merge \
        -p '{
          "spec": {
            "telemetry": {
              "enabled": true,
              "metrics": {
                "captureOrganization": true,
                "captureUser": false,
                "captureGroup": false,
                "captureModelUsage": true
              }
            }
          }
        }'

Verification

1. In the OpenShift console, navigate to Observe Metrics.

2. In the query field, enter the following metric name:

   authorized_calls

3. Click Run queries.

Procedure

1. In the OpenShift AI dashboard, click Observe & monitor Dashboard in the left navigation menu.

   The Observability dashboard page displays three tabs: Cluster, Models, and Usage.

2. Click the Usage tab to view Models-as-a-Service usage metrics.
3. Optional: To change the time range, select a value from the Time period dropdown. Options range from the last 5 minutes to the last 14 days. You can also specify a custom date range.
4. Optional: Filter the metrics by user, subscription, or model by selecting values from the User, Subscription, or Model dropdowns. Select All in any dropdown to view metrics for all items in that category.
5. Review the Overview section, which displays summary metrics including Total Tokens, Total Requests, Total Errors, Success Rate, and Active Users.
6. Review the Token Consumption by User table, which shows detailed per-user usage data.
7. Optional: Click column headers in the table to sort by that column.
8. Optional: Use the pagination controls at the bottom of the table to navigate through multiple pages of results or adjust the number of rows displayed per page.

Procedure

1. In the OpenShift AI dashboard, click Observe & monitor Dashboard.
2. Click the Usage tab.

3. Optional: Configure filters to export specific usage data:

   1. Select a time period from the Time period dropdown to export metrics for a specific timeframe.
   2. Select filters for User, Subscription, or Model to export metrics for specific resources.
4. Hover over the Token Consumption by User table.

5. Click Export as CSV to download the usage data.

   The system generates a CSV file containing the filtered usage data.

6. Save the CSV file to your local system.

Procedure

1. Edit the Tenant custom resource to add the externalOIDC configuration, using one of the following methods:

   Using the OpenShift console

   1. In the OpenShift console, navigate to Administration CustomResourceDefinitions.
   2. Search for Tenant and click the resource name.
   3. Click the Instances tab.
   4. Click default-tenant.
   5. Click the YAML tab.

   6. In the YAML editor, locate the spec section and add or update the externalOIDC configuration:

      spec:
        externalOIDC:
          issuerUrl: <oidc-provider-issuer-url>
          clientId: <oidc-client-id>

      The configuration uses the following fields:

   <oidc-provider-issuer-url>

   Specifies the endpoint URL for your external identity provider.

   <oidc-client-id>

   Specifies the client ID for your MaaS application registered with the OIDC provider.

   1. Click Save.

   Using the OpenShift CLI

   Run the following command:

   $ oc patch tenants.maas.opendatahub.io default-tenant -n models-as-a-service \
     --type merge \
     -p { "spec": { "externalOIDC": { "issuerUrl": "<oidc-provider-issuer-url>", "clientId": "<oidc-client-id>" } } }

2. Create MaaS subscriptions that include the groups from your OIDC provider.

   Note

   Group names in subscriptions must match the group names in the OIDC token claims exactly. MaaS validates group memberships directly from the OIDC token.

   When configuring subscriptions, enter group names exactly as they appear in your OIDC provider’s group claims. For example, if your OIDC token includes groups: ["data-scientists"], enter data-scientists in the subscription.

   For information about creating subscriptions, see Managing subscriptions for Models-as-a-Service.

1. Obtain an OIDC token from your identity provider using the OAuth 2.0 client credentials grant:

   $ curl -X POST "<oidc-token-endpoint>" \
     -d "client_id=<client-id>" \
     -d "client_secret=<client-secret>" \
     -d "grant_type=client_credentials"

   The configuration uses the following fields:

   <oidc-token-endpoint>

   Specifies the token endpoint URL from your OIDC provider.

   <client-id>

   Specifies your OIDC client ID.

   <client-secret>

   Specifies your OIDC client secret.

2. Use the OIDC token to list available models:

   $ curl -H "Authorization: Bearer <oidc-token>" \
     https://<maas-gateway-url>/maas-api/v1/models

   The configuration uses the following fields:

   <oidc-token>

   Specifies the JWT obtained from your OIDC provider.

   <maas-gateway-url>

   Specifies your MaaS gateway URL.

   If authentication is successful, the API returns a list of models available to the groups in your token.

Procedure

1. Create a Kubernetes secret with the external provider API key:

   $ oc create secret generic <provider-api-key-secret> \
     --from-literal=api-key=<provider-api-key> \
     -n <model-namespace>

   The command uses the following placeholders:

   <provider-api-key-secret>

   Specifies the name of the secret containing the provider API key.

   <provider-api-key>

   Specifies the API key for the external provider.

   <model-namespace>

   Specifies the name of the model namespace you created, such as llm.

   This secret stores the provider API key that MaaS uses to authenticate requests to the external model.

2. Create an ExternalModel custom resource, using one of the following methods:

   Using the OpenShift console

   1. In the OpenShift console, navigate to Administration CustomResourceDefinitions.
   2. Search for ExternalModel and click the resource name.
   3. Click the Instances tab.
   4. From the Project dropdown, select your model namespace, such as llm.
   5. Click Create ExternalModel.

   6. In the YAML editor, replace the default content with the following configuration:

      apiVersion: maas.opendatahub.io/v1alpha1
      kind: ExternalModel
      metadata:
        name: <external-model-name>
        namespace: <model-namespace>
      spec:
        provider: <provider-type>
        endpoint: <external-provider-hostname>
        targetModel: <target-model-id>
        credentialRef:
          name: <provider-api-key-secret>

      The YAML file uses the following placeholders:

   <external-model-name>

   Specifies the name for the external model resource.

   <provider-type>

   Specifies the provider type. Allowed values: openai, anthropic, azure-openai, vertex, bedrock-openai.

   <external-provider-hostname>

   Specifies the fully qualified domain name (FQDN) of the external provider without scheme or path, such as api.openai.com.

   <target-model-id>

   Specifies the upstream model identifier at the provider, such as gpt-4o.

   <provider-api-key-secret>

   Specifies the name of the secret created in the previous step.

   <model-namespace>

   Specifies the name of the model namespace you created, such as llm.

   1. Click Create.

   Using the OpenShift CLI

   Run the following command:

   $ cat <<EOF | oc apply -f -
   apiVersion: maas.opendatahub.io/v1alpha1
   kind: ExternalModel
   metadata:
     name: <external-model-name>
     namespace: <model-namespace>
   spec:
     provider: <provider-type>
     endpoint: <external-provider-hostname>
     targetModel: <target-model-id>
     credentialRef:
       name: <provider-api-key-secret>
   EOF

   Note

   When the ExternalModel resource is created, the MaaS controller automatically creates the required networking resources: Service, HTTPRoute, ServiceEntry, and DestinationRule. These resources enable routing from the MaaS gateway to the external provider. If external model routing fails, verify that these resources were created successfully in your model namespace.

3. Add the external model to a MaaS subscription.

   When creating or updating subscriptions, the external model appears in the model list alongside locally deployed models. Select the external model and configure token limits the same way as for internal models.

   For information about creating subscriptions, see Managing subscriptions for Models-as-a-Service.

Verification

1. Verify that the external model was added to the subscription:

   1. In the OpenShift AI dashboard, navigate to Settings Subscriptions.
   2. Click the subscription name.
   3. In the Models section, verify that the external model appears in the list.
2. Log in to the OpenShift AI dashboard as a user who belongs to a group included in the subscription.
3. Click Gen AI studio AI asset endpoints.
4. Verify that the external model appears in the model list.

5. Create a MaaS API key and make a test inference request to the external model:

   $ curl -X POST https://<maas-gateway-url>/maas-api/v1/chat/completions \
     -H "Authorization: Bearer <maas-api-key>" \
     -H "Content-Type: application/json" \
     -d { "model": "<external-model-name>", "messages": [{"role": "user", "content": "Hello"}] }

   The command uses the following placeholders:

   <maas-api-key>

   Specifies your MaaS API key created in the dashboard.

   <maas-gateway-url>

   Specifies your MaaS gateway URL.

   <external-model-name>

   Specifies the name of the external model resource you created.

   A successful response confirms that MaaS routed the request to the external provider and returns the model completion.