Chapter 8. Deploying OGX for multi-tenancy

Procedure

1. Deploy an OGXServer CR with your custom configuration. For more information, see "Deploying an OGX Server".

2. Configure claims mapping in the config.yaml file

   server:
     auth:
       provider_config:
         claims_mapping:
           realm_access.roles: roles
           groups: teams
           tenant_id: namespaces

3. You now need to enable authentication on your server. You can enable Kubernetes OIDC, which is recommended for in-cluster workloads, or Keycloak for basic users and external clients.

   • Enabling authorization with Kubernetes OIDC

     1. Get the clusters OIDC endpoints with the following command:

        $ AUTH_ISSUER=$(oc get --raw /.well-known/openid-configuration | jq .issuer -r)
        $ AUTH_JWKS_URI=$(oc get --raw /.well-known/openid-configuration | jq .jwks_uri -r)

     2. Set the following additional environment variables:

        AUTH_AUDIENCE=ogx
        AUTH_VERIFY_TLS=true

     3. The default distribution configuration activates when you set the necessary environment variables.

        Default auth configuration YAML

        auth:
          provider_config:
            type: ${env.AUTH_ISSUER:+oauth2_token}   # activates only when AUTH_ISSUER is set
            audience: ${env.AUTH_AUDIENCE:=ogx}
            issuer: ${env.AUTH_ISSUER:=}
            jwks:
              uri: ${env.AUTH_JWKS_URI:=}
            verify_tls: ${env.AUTH_VERIFY_TLS:=true}

   • Enabling authorization with Keycloak

     1. Set your Keycloak details in the OGXServer custom resource

        apiVersion: ogx.io/v1beta1
        kind: OGXServer
        metadata:
          name: ogx-shared-server
          namespace: ogx-system
        spec:
          env:
            - name: AUTH_ISSUER
              value: "https://keycloak.example.com/realms/my-org"
            - name: AUTH_JWKS_URI
              value: "https://keycloak.example.com/realms/my-org/protocol/openid-connect/certs"
            - name: AUTH_AUDIENCE
              value: "ogx-api"

4. You then need to configure the access policy of your server. OpenShift AI ships a default access policy that provides owner-based isolation.

   Default access policy YAML

   auth:
     access_policy:
       # System resources are readable by all
       - permit:
           actions: [read]
         when: resource is unowned
         description: "All users can read system resources"
       # Any authenticated user can create resources
       - permit:
           actions: [create]
         description: "Authenticated users can create resources"
       # Only the owner can read, update, or delete resources
       - permit:
           actions: [read, update, delete]
         when: user is owner
         description: "Owners can manage their own resources"

   1. For team-based sharing where users can see the resources of users on the same team, use the following example access_policy config:

      auth:
        access_policy:
          - permit:
              actions: [read]
            when: resource is unowned
            description: "All users can read system resources"
          - permit:
              actions: [create]
            description: "Authenticated users can create resources"
          - permit:
              actions: [read, update, delete]
            when: user is owner
            description: "Owners can manage their own resources"
          - permit:
              actions: [read]
            when: user in owners teams
            description: "Team members can read each others resources"

   2. For permissions where an admin can access everything, use the following example access_policy config:

      auth:
        access_policy:
          # Admin bypass: full access to all resources
          - permit:
              actions: [create, read, update, delete]
            when: user with admin in roles
            description: "Admins have full access"
          # Standard user policies
          - permit:
              actions: [read]
            when: resource is unowned
          - permit:
              actions: [create]
          - permit:
              actions: [read, update, delete]
            when: user is owner

5. Create the project namespaces, for example:

   $ oc new-project <project-a>
   $ oc new-project <project-b>

6. Set the service accounts for each role, for example

   $ oc create serviceaccount ogx-developer -n <project-a>
   $ oc create serviceaccount ogx-agent -n  <project-b>

7. The tenant admin is responsible for provisioning auth tokens for the tenant users. You can generate the tokens using Kubernetes OIDC, which is recommended for cluster workloads, or Keycloak for basic users or external clients.

   • Accessing a Kubernetes OIDC token

     1. You can create a token based on team roles and project names:

        $ TOKEN=$(oc create token ogx-developer -n <project-a> --audience ogx --duration=3600s)

   • Accessing a Keycloak token

     1. Configure your config.yaml file to trust KeyCloak

        auth.provider_config.issuer: https://keycloak.example.com/realms/ai-platform
        auth.provider_config.jwks.uri: https://keycloak.example.com/realms/ai-platform/protocol/openid-connect/certs

     2. Access and set the token environment variable:

        TOKEN=$(curl -s -X POST \
          "https://keycloak.example.com/realms/ai-platform/protocol/openid-connect/token" \
          -d "grant_type=password&client_id=ogx&username=alice&password=***" \
          | jq -r .access_token)

8. Your tenant users can now access the resources on the namespace. For more information, see "Using APIs as a tenant user".

Procedure

1. Create a ClusterRole custom resource for configuring the platform admin permissions.

   Example platform admin ClusterRole CR

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: ogx-platform-admin
   rules:
   - apiGroups: ["ogx.io"]
     resources: ["ogxservers"]
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
   - apiGroups: [""]
     resources: ["namespaces"]
     verbs: ["get", "list", "watch", "create"]
   - apiGroups: ["rbac.authorization.k8s.io"]
     resources: ["roles", "rolebindings"]
     verbs: ["get", "list", "watch", "create", "update", "patch"]

2. Grant the tenant admin permissions to manage CR in their namespace with the following example Role configurations:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: ogx-tenant-admin
     namespace: tenant-a
   rules:
   - apiGroups: ["ogx.io"]
     resources: ["ogxservers"]
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
   - apiGroups: [""]
     resources: ["secrets", "configmaps"]
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: ogx-tenant-admin
     namespace: tenant-a
   subjects:
   - kind: Group
     name: tenant-a-admins
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: Role
     name: ogx-tenant-admin
     apiGroup: rbac.authorization.k8s.io

3. Enable ResourceQuota CR using the following configurations as an example:

   apiVersion: v1
   kind: ResourceQuota
   metadata:
     name: ogx-tenant-quota
     namespace: tenant-a
   spec:
     hard:
       requests.cpu: "8"
       requests.memory: 32Gi
       limits.cpu: "16"
       limits.memory: 64Gi
       pods: "10"

4. Enable preferred NetworkPolicy CR configurations:

   • Allow specific namespace

     spec:
       network:
         policy:
           ingress:
           - from:
             - namespaceSelector:
                 matchLabels:
                   kubernetes.io/metadata.name: frontend-ns
             ports:
             - port: 8321
               protocol: TCP

   • Allow namespace by label

     spec:
       network:
         policy:
           ingress:
           - from:
             - namespaceSelector:
                 matchLabels:
                   tenant: customer-a
             ports:
             - port: 8321
               protocol: TCP

   • Allow specific pods in a namespace

     spec:
       network:
         policy:
           ingress:
           - from:
             - namespaceSelector:
                 matchLabels:
                   kubernetes.io/metadata.name: frontend-ns
               podSelector:
                 matchLabels:
                   app: api-gateway
             ports:
             - port: 8321
               protocol: TCP

   • Allow external CIDR

     spec:
       network:
         policy:
           ingress:
           - from:
             - ipBlock:
                 cidr: 10.0.0.0/8
                 except:
                 - 10.0.1.0/24
             ports:
             - port: 8321
               protocol: TCP

Procedure

1. Obtain your credentials from the tenant admin. Your tenant admin will provide you with:

   • An OGX endpoint URL, for example:

     https://ogx.apps.cluster.example.com/v1

   • An API Key or token. You can obtain the keys in various ways:

     1. An environment variable: The OGX_API_KEY is already set by the platform admin when you open the notebook.
     2. Keycloak login: You login to keycloak and the OGX SDK is responsible for refreshing.
     3. API key: Tenant admin generates and provides you with the key that you use in your configuration.

2. You can now access the APIs in the namespaces

   Example using the APIs

   OGX_URL="https://ogx.apps.cluster.example.com"
   TOKEN="<your-api-key-from-admin>"
   
   # List models
   curl -s -H "Authorization: Bearer $TOKEN" "$OGX_URL/v1/models" | python3 -m json.tool
   
   # Create a response
   curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
     "$OGX_URL/v1/responses" \
     -d '{"model":"vllm-inference/llama-3-2-3b","input":"Summarize quantum computing","store":true}' \
     | python3 -m json.tool
   
   # List your responses
   curl -s -H "Authorization: Bearer $TOKEN" "$OGX_URL/v1/responses" | python3 -m json.tool
   
   # Upload a file (owned by you)
   curl -s -H "Authorization: Bearer $TOKEN" \
     -F "file=@dataset.jsonl" -F "purpose=assistants" \
     "$OGX_URL/v1/files" | python3 -m json.tool
   
   # Create a vector store (owned by you)
   curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
     "$OGX_URL/v1/vector_stores" \
     -d '{"name":"my-knowledge-base"}' | python3 -m json.tool

   Example using the Python SDK

   import os
   from openai import OpenAI
   
   # Token and URL are provided by your platform admin
   # (typically pre-set as environment variables in your notebook/workspace)
   client = OpenAI(
       base_url=os.environ.get("OGX_URL", "https://ogx.apps.cluster.example.com/v1"),
       api_key=os.environ["OGX_API_KEY"],
   )
   
   # Create a stored response — automatically owned by your identity
   response = client.responses.create(
       model="vllm-inference/llama-3-2-3b",
       input="Explain transformers",
       store=True,
   )
   
   # List responses — only returns yours, other teams' responses are invisible
   my_responses = client.responses.list()