Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Integrating Google Vertex AI with OpenShift Lightspeed

As an administrator, you can integrate Google Vertex AI as a large language model (LLM) provider for OpenShift Lightspeed.

4.1. Google Vertex AI provider types
Copy link

OpenShift Lightspeed supports Google Vertex AI as an LLM provider. You can deploy Google-native models or Anthropic models hosted on the Google Cloud Platform (GCP) infrastructure.

Both provider types authenticate using a GCP service account JSON key stored within a Kubernetes Secret.

Expand
Table 4.1. Supported provider types
Provider typeUse caseRequired configuration field

google_vertex

Google-native models like Gemini.

googleVertexConfig

google_vertex_anthropic

Anthropic models like Claude hosted on Vertex AI.

googleVertexAnthropicConfig

4.2. Configuring Google Vertex AI
Copy link

To use Google Vertex AI, create a credentials secret and apply an OLSConfig custom resource (CR).

Prerequisites

  • The OpenShift Lightspeed Operator must be installed.
  • You must possess a valid GCP service account JSON key file.
  • The Vertex AI API must be enabled in your Google Cloud project.
  • Your GCP service account must have appropriate Vertex AI permissions.

Procedure

  1. Create the credentials Secret in the operator namespace by running the following command: 

    oc create secret generic llmcreds \
  --from-file=gcp-service-account.json=/path/to/service-account-key.json \
  -n openshift-lightspeed
    Note

    The Operator looks for a key named apitoken by default if you omit the credentialKey field later.

  2. Create an OLSConfig CR file named olsconfig.yaml using one of the following examples:

    • Example configuration for Gemini (google_vertex): 

      apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: google
        type: google_vertex
        credentialsSecretRef:
          name: llmcreds
        credentialKey: gcp-service-account.json
        googleVertexConfig:
          projectID: my-gcp-project-123
          location: us-central1
        models:
          - name: gemini-2.5-flash-lite
  ols:
    defaultModel: gemini-2.5-flash-lite
    defaultProvider: google

    • Example configuration for Claude (google_vertex_anthropic): 

      apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  llm:
    providers:
      - name: google-anthropic
        type: google_vertex_anthropic
        credentialsSecretRef:
          name: llmcreds
        credentialKey: gcp-service-account.json
        googleVertexAnthropicConfig:
          projectID: my-gcp-project-123
          location: us-east4
        models:
          - name: claude-3-sonnet
  ols:
    defaultModel: claude-3-sonnet
    defaultProvider: google-anthropic

  3. Apply the configuration file to your cluster: 

    oc apply -f olsconfig.yaml

Verification

  1. Verify that the Operator has completed reconciliation: 

    oc get olsconfig cluster -o jsonpath='{.status.overallStatus}'

    Expected output: Ready

The following reference tables describe the configuration schema for Google Vertex AI providers.

Expand
Table 4.2. Provider fields (spec.llm.providers[])
FieldTypeRequiredDescription

name

string

Yes

Logical name for the provider. Referenced by spec.ols.defaultProvider.

type

string

Yes

Must be set to google_vertex or google_vertex_anthropic.

credentialsSecretRef.name

string

Yes

Name of the Secret in the operator namespace that contains provider credentials.

credentialKey

string

No

Key name inside the Secret to read. Defaults to apitoken.

url

string

No

The provider API endpoint URL. This field is typically not required for Vertex AI.

models

array

Yes

List of models available from the provider.

Expand
Table 4.3. Google Vertex configuration (spec.llm.providers[].googleVertexConfig)
FieldTypeRequiredDescription

projectID

string

Yes

The Google Cloud project ID (for example, my-gcp-project-123).

location

string

Yes

The target GCP region for Vertex AI (for example, us-central1).

Expand
Table 4.4. Google Vertex Anthropic configuration (spec.llm.providers[].googleVertexAnthropicConfig)
FieldTypeRequiredDescription

projectID

string

Yes

The Google Cloud project ID.

location

string

Yes

The target GCP region for Vertex AI (for example, us-east4).

Expand
Table 4.5. Model fields (spec.llm.providers[].models[])
FieldTypeRequiredDescription

name

string

Yes

Model name (such as gemini-2.5-flash-lite). Referenced by spec.ols.defaultModel.

url

string

No

The model-specific API endpoint URL.

contextWindowSize

integer

No

Context window size in tokens. Minimum value: 1024.

parameters.maxTokensForResponse

integer

No

Maximum tokens allowed for responses. Default value: 2048.

parameters.toolBudgetRatio

float

No

Ratio of the context window allocated for the tool token budget. Range: 0.1 to 0.5. Default value: 0.5.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top