SupportConsoleDevelopersStart a trialContact

Chapter 7. Select and deploy a vector database

When your application requires retrieval-augmented generation (RAG), choose and configure a vector store so that you can store and query document embeddings for retrieval during inference. OGX in OpenShift AI supports remote Milvus, PostgreSQL with the pgvector extension, and Qdrant as vector store providers. After configuring a vector store, you can build a RAG application, as described in Deploying a RAG stack in a project.

7.1. Overview of vector databases
Copy link

Vector databases are a core component of retrieval-augmented generation (RAG) in OpenShift AI. They store and index vector embeddings that represent the semantic meaning of text or other data. When integrated with OGX, vector databases enable applications to retrieve relevant context and combine it with large language model (LLM) inference.

Vector databases provide the following capabilities:

  • Store vector embeddings generated by embedding models.
  • Support efficient similarity search to retrieve semantically related content.
  • Enable RAG workflows by supplying the LLM with contextually relevant data.

In OpenShift AI, vector databases are configured and managed through the OGX Operator as part of a OGXServer. PostgreSQL is the default and recommended metadata store for OGX, supporting production-ready persistence, concurrency, and scalability.

The following vector database options are supported in OpenShift AI:

  • Remote Milvus Remote Milvus runs as a standalone vector database service, either within the cluster or as an external managed deployment. This option is suitable for large-scale or production-grade RAG workloads that require high availability, horizontal scalability, and isolation from the OGX server. In OpenShift environments, Milvus typically requires an accompanying etcd service for coordination. For more information, see Providing redundancy with etcd.
  • Remote PostgreSQL with pgvector PostgreSQL with the pgvector extension provides a production-ready vector database option that integrates vector similarity search directly into PostgreSQL. This option is well suited for environments that already operate PostgreSQL and require durable storage, transactional consistency, and centralized management. pgvector enables OGX to store embeddings and perform similarity search without deploying a separate vector database service.

Consider the following guidance when choosing a vector database for your RAG workloads:

  • Use Remote Milvus when you require large-scale vector indexing and high-throughput similarity search.
  • Use PostgreSQL with pgvector when you want production-ready persistence and integration with existing PostgreSQL-based data platforms.

SQLite-based storage is no longer recommended for production deployments. PostgreSQL-based backends provide improved reliability, concurrency, and scalability as OGX moves toward general availability.

7.1.1. Overview of Milvus vector databases
Copy link

Milvus is an open source vector database designed for high-performance similarity search across large volumes of embedding data. In OpenShift AI, Milvus is supported as a vector store provider for OGX and enables retrieval-augmented generation (RAG) workloads that require efficient vector indexing, scalable search, and durable storage.

Production-grade OGX deployments default to PostgreSQL for metadata persistence. When Milvus is used as the vector store, PostgreSQL is typically used for OGX metadata, while Milvus manages vector indexes and similarity search.

Milvus vector databases provide the following capabilities in OpenShift AI:

  • High-performance similarity search using Approximate Nearest Neighbor (ANN) algorithms
  • Efficient indexing and query optimization for dense embeddings
  • Persistent storage of vector data
  • Integration with OGX through an OpenAI-compatible Vector Stores API

In a typical RAG workflow in OpenShift AI, the following responsibilities are separated:

  • Embedding generation Embeddings are generated by the configured embedding provider. Remote embedding models are the recommended and default option for production deployments.
  • Vector storage and retrieval Milvus stores embedding vectors and performs similarity search operations.
  • Metadata persistence OGX stores vector store metadata, file references, and configuration state using PostgreSQL in production deployments.
  • OGX server Coordinates ingestion, retrieval, and model inference through a unified API surface.

In OpenShift AI, Milvus can be used in the following operational modes:

  • Remote Milvus Runs as a standalone service within your OpenShift project or as an external managed Milvus deployment. Remote Milvus is recommended for production-grade RAG workloads.

A remote Milvus deployment typically includes the following components:

  • A Milvus service that exposes a gRPC endpoint (port 19530) for client traffic
  • An etcd service that Milvus uses for metadata coordination, collection state, and index management
  • Persistent storage for durable vector data

Milvus requires a dedicated etcd instance for metadata coordination, even when running in standalone mode. Do not use the OpenShift control plane etcd for this purpose. For more information about etcd, see Providing redundancy with etcd.

Important

You must deploy a dedicated etcd service for Milvus or connect Milvus to an external etcd instance. Do not share the OpenShift control plane etcd with application workloads.

Use Remote Milvus when you require scalable vector search, high-performance retrieval, and integration with production-grade OGX deployments in OpenShift AI.

For instructions on deploying Milvus as a remote vector database, see Deploying a remote Milvus vector database.

7.1.2. Overview of pgvector vector databases
Copy link

pgvector is an open source PostgreSQL extension that enables vector similarity search on embedding data stored in relational tables. In OpenShift AI, PostgreSQL with the pgvector extension is supported as a remote vector database provider for the OGX Operator. pgvector supports retrieval augmented generation workflows that require persistent vector storage while integrating with existing PostgreSQL environments.

pgvector vector databases provide the following capabilities in OpenShift AI:

  • Storage of vector embeddings in PostgreSQL tables.
  • Similarity search across embeddings by using pgvector distance metrics.
  • Persistent storage of vectors alongside structured relational data.
  • Integration with existing PostgreSQL security and operational tooling.

In a typical retrieval augmented generation workflow in OpenShift AI, your application uses the following components:

  • Inference provider Generates embeddings and model responses.
  • Vector store provider Stores embeddings and performs similarity search. When you use pgvector, PostgreSQL provides this capability as a remote vector store.
  • File storage provider Stores the source files that are ingested into vector stores.
  • OGX server Provides a unified API surface, including an OpenAI compatible Vector Stores API.

When you ingest content, OGX splits source material into chunks, generates embeddings, and stores them in PostgreSQL through the pgvector extension. When you query a vector store, OGX performs similarity search and returns the most relevant chunks for use in prompts.

In OpenShift AI, pgvector is used in the following operational mode:

  • Remote PostgreSQL with pgvector, which runs as a standalone PostgreSQL database service accessed by the OGX server. This mode is suitable for development and production workloads that require persistent storage and integration with existing PostgreSQL infrastructure.

When you deploy PostgreSQL with the pgvector extension, you typically manage the following components:

  • Secrets for PostgreSQL connection credentials.
  • Persistent storage for durable database data.
  • A PostgreSQL service that exposes a network endpoint.

PostgreSQL with pgvector does not require an external coordination service. Vector data, indexes, and metadata are stored directly in PostgreSQL tables and managed through standard database mechanisms.

Use PostgreSQL with pgvector when you require persistent vector storage and want to integrate vector search into existing PostgreSQL based data platforms within OpenShift AI. Deploying a PostgreSQL instance with pgvector.

7.2. Deploying a remote Milvus vector database
Copy link

To use Milvus as a remote vector database provider for OGX in OpenShift AI, you must deploy Milvus and its required etcd service in your OpenShift project. This procedure shows how to deploy Milvus in standalone mode without the Milvus Operator.

Note

The following example configuration is intended for testing or evaluation environments. For production-grade deployments, see https://milvus.io/docs in the Milvus documentation.

Prerequisites

  • You have installed OpenShift 4.19 or newer.
  • You have enabled GPU support in OpenShift AI. This includes installing the Node Feature Discovery operator and NVIDIA GPU Operators. For more information, see Installing the Node Feature Discovery operator and Enabling NVIDIA GPUs.
  • You have cluster administrator privileges for your OpenShift cluster.
  • You are logged in to Red Hat OpenShift AI.
  • You have a StorageClass available that can provision persistent volumes.
  • You created a root password to secure your Milvus service.
  • You have deployed an inference model with vLLM, for example, the llama-3.2-3b-instruct model, and you have selected Make deployed models available through an external route and Require token authentication during model deployment.
  • You have the correct inference model identifier, for example, llama-3-2-3b.
  • You have the model endpoint URL, ending with /v1, such as https://llama-32-3b-instruct-predictor:8443/v1.
  • You have the API token required to access the model endpoint.
  • You have installed the OpenShift command line interface (oc) as described in Installing the OpenShift CLI.

Procedure

  1. In the OpenShift console, click the Quick Create ( quick create icon ) icon and then click the Import YAML option.
  2. Verify that your project is the selected project.

  3. In the Import YAML editor, paste the following manifest and click Create: 

    apiVersion: v1
kind: Secret
metadata:
  name: milvus-secret
type: Opaque
stringData:
  root-password: "MyStr0ngP@ssw0rd"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: milvus-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  volumeMode: Filesystem
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: etcd-deployment
  labels:
    app: etcd
spec:
  replicas: 1
  selector:
    matchLabels:
      app: etcd
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: etcd
    spec:
      containers:
        - name: etcd
          image: quay.io/coreos/etcd:v3.5.5
          command:
            - etcd
            - --advertise-client-urls=http://127.0.0.1:2379
            - --listen-client-urls=http://0.0.0.0:2379
            - --data-dir=/etcd
          ports:
            - containerPort: 2379
          volumeMounts:
            - name: etcd-data
              mountPath: /etcd
          env:
            - name: ETCD_AUTO_COMPACTION_MODE
              value: revision
            - name: ETCD_AUTO_COMPACTION_RETENTION
              value: "1000"
            - name: ETCD_QUOTA_BACKEND_BYTES
              value: "4294967296"
            - name: ETCD_SNAPSHOT_COUNT
              value: "50000"
      volumes:
        - name: etcd-data
          emptyDir: {}
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: etcd-service
spec:
  ports:
    - port: 2379
      targetPort: 2379
  selector:
    app: etcd
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: milvus-standalone
  name: milvus-standalone
spec:
  replicas: 1
  selector:
    matchLabels:
      app: milvus-standalone
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: milvus-standalone
    spec:
      containers:
        - name: milvus-standalone
          image: milvusdb/milvus:v2.6.0
          args: ["milvus", "run", "standalone"]
          env:
            - name: DEPLOY_MODE
              value: standalone
            - name: ETCD_ENDPOINTS
              value: etcd-service:2379
            - name: COMMON_STORAGETYPE
              value: local
            - name: MILVUS_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: milvus-secret
                  key: root-password
          livenessProbe:
            exec:
              command: ["curl", "-f", "http://localhost:9091/healthz"]
            initialDelaySeconds: 90
            periodSeconds: 30
            timeoutSeconds: 20
            failureThreshold: 5
          ports:
            - containerPort: 19530
              protocol: TCP
            - containerPort: 9091
              protocol: TCP
          volumeMounts:
            - name: milvus-data
              mountPath: /var/lib/milvus
      restartPolicy: Always
      volumes:
        - name: milvus-data
          persistentVolumeClaim:
            claimName: milvus-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: milvus-service
spec:
  selector:
    app: milvus-standalone
  ports:
    - name: grpc
      port: 19530
      targetPort: 19530
    - name: http
      port: 9091
      targetPort: 9091
    Note
    • Use the gRPC port (19530) for the MILVUS_ENDPOINT setting in OGX.
    • The HTTP port (9091) is reserved for health checks.
    • If you deploy Milvus in a different namespace, use the fully qualified service name in your OGX configuration. For example: http://milvus-service.<namespace>.svc.cluster.local:19530

Verification

  1. In the OpenShift web console, click Workloads Deployments.
  2. Verify that both etcd-deployment and milvus-standalone show a status of 1 of 1 pods available.
  3. Click Pods in the navigation panel and confirm that pods for both deployments are Running.
  4. Click the milvus-standalone pod name, then select the Logs tab.

  5. Verify that Milvus reports a healthy startup with output similar to: 

    Milvus Standalone is ready to serve ...
Listening on 0.0.0.0:19530 (gRPC)
  6. Click Networking Services and confirm that the milvus-service and etcd-service resources exist and are exposed on ports 19530 and 2379, respectively.

  7. (Optional) Click Pods milvus-standalone Terminal and run the following health check: 

    curl http://localhost:9091/healthz

    A response of {"status": "healthy"} confirms that Milvus is running correctly.

7.3. Using PostgreSQL in OGX
Copy link

PostgreSQL is a dependency for OGX deployments in OpenShift AI, where it serves as the mandatory metadata storage backend for supported vector storage configurations. Additionally, you can configure PostgreSQL as a remote vector database provider by enabling the pgvector extension.

In OpenShift AI, PostgreSQL serves the following roles in OGX deployments:

  • Required metadata storage for OGX APIs and orchestration services.
  • An optional remote vector database when the pgvector provider is enabled.

Depending on your deployment requirements, these roles can be fulfilled by the same PostgreSQL instance or separate instances. For example, you might use a single instance for development and testing environments, and separate instances for production deployments that require independent scaling or isolation.

Important

The procedures provide basic configuration suitable for development and testing. Production deployments require additional planning, including the following considerations:

  • High availability and replication
  • Backup and disaster recovery
  • Security hardening and encryption
  • Performance tuning and monitoring

7.3.1. Understanding PostgreSQL in OGX
Copy link

7.3.1.1. Understanding OGX metadata storage
Copy link

In OpenShift AI, OGX requires PostgreSQL as a metadata storage backend to persist state and configuration data across multiple components. Metadata storage provides durable persistence for vector stores, file management, agent state, conversation history, and other OGX services.

PostgreSQL is required as a metadata storage backend for all OpenShift AI deployments.

7.3.1.1.1. Role of metadata storage in OGX
Copy link

OGX components require persistent storage beyond in-memory data structures. Without metadata storage, component state would be lost on pod restarts or application failures.

OGX uses metadata storage to persist:

  • Vector store metadata, such as collection identifiers and document mappings.
  • File metadata, including file locations, identifiers, and attributes.
  • Agent state and conversation history.
  • Dataset configurations and batch processing state.
  • Model registry information and prompt templates.

This persistent storage allows OGX to maintain operational state across pod restarts, rescheduling, and application updates.

7.3.1.1.2. PostgreSQL metadata storage backends
Copy link

OGX uses PostgreSQL to store multiple categories of metadata, including vector store metadata, file records, agent state, conversation history, and configuration data. These data types have different storage characteristics but are managed automatically within a single PostgreSQL instance.

Important

PostgreSQL version 14 or later is required for all OGX deployments, including development, testing, and production environments.

If validation errors occur, confirm that the deployed OGX image version matches the configuration schema referenced by your run.yaml.

OGX does not provision or manage the PostgreSQL instance used for metadata storage. You must deploy and manage the PostgreSQL database and supply its connection details when deploying OGX.

7.3.2. Deploying and Configuring PostgreSQL
Copy link

You can connect OGX in OpenShift AI to an existing PostgreSQL instance that has the pgvector extension enabled. For development or evaluation, you can also deploy a PostgreSQL instance with the pgvector extension directly in your OpenShift project by creating Kubernetes resources through the OpenShift web console. This procedure focuses on deploying PostgreSQL with the pgvector extension for use as a remote vector store. It does not cover preparing a PostgreSQL database for use as OGX metadata storage.

Prerequisites

  • You have installed OpenShift 4.19 or newer.
  • You have permissions to create resources in a project in your OpenShift cluster.
  • You have PostgreSQL connection details available, including the database name, user name, and password.
  • If you plan to deploy PostgreSQL in-cluster, you have a StorageClass that can provision persistent volumes.
  • If you are using an existing PostgreSQL instance, the pgvector extension is installed and enabled on the target database.

Procedure

  1. Log in to the OpenShift web console.
  2. Select the project where you want to deploy the PostgreSQL instance.
  3. Click the Quick Create ( quick create icon ) icon, and then click Import YAML.
  4. Verify that the correct project is selected.

  5. Copy the following YAML, replace the placeholder values, paste it into the YAML editor, and then click Create.

    Important

    This example deploys a standalone PostgreSQL service with the pgvector extension enabled.

    OGX does not automatically use this database. To use this PostgreSQL instance as a vector store, you must explicitly configure the pgvector provider in a OGXServer.

    This example is intended for development or evaluation purposes. For production deployments, review and adapt the configuration to meet your organization’s security, availability, backup, and lifecycle requirements.

    Example PostgreSQL deployment with pgvector (development or evaluation)

    apiVersion: v1
kind: Secret
metadata:
  name: <pgvector-postgresql-credentials-secret>
type: Opaque
stringData:
  POSTGRES_DB: "<database-name>"
  POSTGRES_USER: "<database-username>"
  POSTGRES_PASSWORD: "<database-password>"

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: <pgvector-postgresql-pvc>
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: <storage-size>

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: <pgvector-postgresql-deployment>
spec:
  replicas: 1
  selector:
    matchLabels:
      app: <pgvector-postgresql-app-label>
  template:
    metadata:
      labels:
        app: <pgvector-postgresql-app-label>
    spec:
      containers:
      - name: postgres
        image: pgvector/pgvector:pg16
        ports:
        - name: postgres
          containerPort: 5432
        env:
        - name: POSTGRES_DB
          valueFrom:
            secretKeyRef:
              name: <pgvector-postgresql-credentials-secret>
              key: POSTGRES_DB
        - name: POSTGRES_USER
          valueFrom:
            secretKeyRef:
              name: <pgvector-postgresql-credentials-secret>
              key: POSTGRES_USER
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: <pgvector-postgresql-credentials-secret>
              key: POSTGRES_PASSWORD
        volumeMounts:
        - name: pgdata
          mountPath: /var/lib/postgresql/data

        # Replace TCP socket probes with exec probes that validate SQL readiness.
        readinessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - pg_isready -h 127.0.0.1 -U "$POSTGRES_USER" -d "$POSTGRES_DB"
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 6
        livenessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - pg_isready -h 127.0.0.1 -U "$POSTGRES_USER" -d "$POSTGRES_DB"
          initialDelaySeconds: 30
          periodSeconds: 20
          timeoutSeconds: 5
          failureThreshold: 6

        # Create the pgvector extension after PostgreSQL is actually accepting SQL.
        lifecycle:
          postStart:
            exec:
              command:
              - /bin/sh
              - -c
              - |
                set -e
                echo "Waiting for PostgreSQL to be ready before enabling pgvector..."
                until PGPASSWORD="$POSTGRES_PASSWORD" psql -h 127.0.0.1 -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 1" >/dev/null 2>&1; do
                  sleep 2
                done
                PGPASSWORD="$POSTGRES_PASSWORD" psql -h 127.0.0.1 -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "CREATE EXTENSION IF NOT EXISTS vector;"

      volumes:
      - name: pgdata
        persistentVolumeClaim:
          claimName: <pgvector-postgresql-pvc>

---
apiVersion: v1
kind: Service
metadata:
  name: <pgvector-postgresql-service>
spec:
  selector:
    app: <pgvector-postgresql-app-label>
  ports:
  - name: postgres
    port: 5432
    targetPort: 5432
  type: ClusterIP

  6. Click Create.

Verification

  1. Navigate to Networking Services.
  2. Confirm that the PostgreSQL Service is listed and exposes port 5432.
  3. Navigate to Workloads Pods.
  4. Confirm that the PostgreSQL pod is running.
Note

This procedure verifies only that PostgreSQL with pgvector is deployed and reachable within the project. It does not verify integration with OGX.

To use PostgreSQL with the pgvector extension as a remote vector store, configure pgvector in your existing OGXServer and provide PostgreSQL connection details as environment variables. Ensure that your OGXServer already includes the PostgreSQL metadata storage configuration. This setup enables retrieval augmented generation (RAG) workflows in OpenShift AI by using PostgreSQL-based vector storage.

Prerequisites

  • You have installed and enabled the OGX Operator in OpenShift AI.
  • You have a PostgreSQL database with the pgvector extension enabled. OGX uses PostgreSQL for two purposes: metadata storage and the optional pgvector remote vector store. You can use a single PostgreSQL instance for both roles or deploy separate instances.
  • You have the PostgreSQL connection details, including the host name, port number, database name, user name, and password.
  • You have permissions to create Secrets and edit custom resources in your project.

Procedure

  1. In the OpenShift web console, switch to the Administrator perspective.

  2. Create a Secret that stores the PostgreSQL connection details.

    1. Ensure that the correct project is selected.
    2. Click Workloads Secrets.
    3. Click Create From YAML.

    4. Paste the following YAML, update the placeholder values, and then click Create.

      Example Secret for pgvector connection details

      apiVersion: v1
kind: Secret
metadata:
  name: pgvector-connection
type: Opaque
stringData:
  PGVECTOR_HOST: "<pgvector-hostname>"
  PGVECTOR_PORT: "<pgvector-port>"
  PGVECTOR_DB: "<database-name>"
  PGVECTOR_USER: "<database-username>"
  PGVECTOR_PASSWORD: "<database-password>"

      Important

      The pgvector provider is not enabled automatically.

      You must explicitly enable pgvector and supply its connection details through environment variables in your OGXServer.

      In OpenShift AI, the pgvector provider is enabled when the ENABLE_PGVECTOR environment variable is set.

  3. Update your OGXServer custom resource to enable pgvector and reference the Secret.

    1. Select the OGX Operator.
    2. Click the OGXServer tab.
    3. Select your OGXServer resource.
    4. Click YAML.

    5. Update the resource to include the following fields, and then click Save.

      Before you enable pgvector, deploy a OGX server and configure the PostgreSQL metadata store.

For more information, see Deploying a OGX server.

Then update your existing OGXServer to add the pgvector configuration shown in the following example. The example shows only the additional environment variables required to enable the pgvector provider.

Example OGXServer configuration for pgvector

apiVersion: ogx.io/v1beta1
kind: OGXServer
metadata:
  name: ogx
spec:
  distribution:
    name: rh-dev
  workload:
    overrides:
      env:
        - name: ENABLE_PGVECTOR
          value: "true"
        - name: PGVECTOR_HOST
          valueFrom:
            secretKeyRef:
              name: pgvector-connection
              key: PGVECTOR_HOST
        - name: PGVECTOR_PORT
          valueFrom:
            secretKeyRef:
              name: pgvector-connection
              key: PGVECTOR_PORT
        - name: PGVECTOR_DB
          valueFrom:
            secretKeyRef:
              name: pgvector-connection
              key: PGVECTOR_DB
        - name: PGVECTOR_USER
          valueFrom:
            secretKeyRef:
              name: pgvector-connection
              key: PGVECTOR_USER
        - name: PGVECTOR_PASSWORD
          valueFrom:
            secretKeyRef:
              name: pgvector-connection
              key: PGVECTOR_PASSWORD

Verification

  1. Click Workloads Pods.
  2. Confirm that the OGX pod restarts and reaches the Running state.
  3. Open the pod logs and confirm that the server starts successfully and initializes the pgvector provider without errors.

7.4. Using Qdrant in OGX
Copy link

Qdrant is a supported remote vector store provider for OGX in OpenShift AI. You can deploy Qdrant in your OpenShift project or connect to an existing Qdrant instance, and configure OGX to use Qdrant for retrieval-augmented generation (RAG) workloads.

To use Qdrant with OGX, complete the following tasks:

  • Review how Qdrant integrates with OGX.
  • Deploy a Qdrant instance or connect to an existing deployment.
  • Configure your OGXServer to use Qdrant as the vector store provider.
  • Perform vector operations through the OpenAI-compatible Vector Stores API.

7.4.1. Overview of Qdrant vector databases
Copy link

Qdrant is an open source vector database optimized for high-performance similarity search and advanced filtering. In OpenShift AI, Qdrant is supported as a remote vector store provider for OGX and can be used in retrieval-augmented generation (RAG) workloads that require efficient vector indexing and durable storage.

When used with OGX in OpenShift AI, Qdrant provides:

  • High-performance similarity search using Hierarchical Navigable Small World (HNSW) indexing
  • Filtering based on stored metadata during vector search
  • Persistent storage of vector data
  • Integration through the OpenAI-compatible Vector Stores API

In a RAG workflow:

  • Embeddings are generated by the configured embedding provider.
  • Qdrant stores embedding vectors and performs similarity search.
  • OGX manages ingestion, retrieval, and model inference through a unified API.

In OpenShift AI, you must deploy Qdrant as a remote service, either within your OpenShift project or as an externally managed deployment.

Note

Inline Qdrant is not supported. To use Qdrant with OGX in OpenShift AI, deploy Qdrant as a remote service.

A typical remote deployment includes:

  • A Qdrant service exposing HTTP (port 6333) and gRPC (port 6334) endpoints
  • Persistent storage for vector data
  • Optional API key authentication

For deployment and configuration instructions, see Using Qdrant in OGX.

7.4.2. Deploying a Qdrant vector database
Copy link

You can connect OGX in OpenShift AI to an existing Qdrant instance or deploy a Qdrant vector database in your OpenShift project. For development or evaluation purposes, you can deploy Qdrant by creating Kubernetes resources in the OpenShift web console.

Prerequisites

  • You have installed OpenShift 4.19 or later.
  • You have permission to create resources in a project.

  • A StorageClass is available that can provision a PersistentVolume for the PersistentVolumeClaim used by this deployment.

    Note

    This example uses a single PersistentVolumeClaim. If your cluster uses dynamic provisioning, the StorageClass provisions the required PersistentVolume automatically.

  • Optional: You have an API key for Qdrant authentication. If your Qdrant instance does not require authentication, remove the Secret and the QDRANT__SERVICE__API_KEY environment variable from the deployment example.

Procedure

  1. Log in to the OpenShift web console.
  2. From the Project list, select the project where you want to deploy Qdrant.
  3. Click Import YAML.

  4. Paste the following YAML:

    Important

    This example deploys a standalone Qdrant service for development or evaluation. For production deployments, review and adapt the configuration to meet your organization’s security, availability, backup, and lifecycle requirements. 

    apiVersion: v1
kind: Secret
metadata:
  name: <qdrant_credentials_secret>
type: Opaque
stringData:
  QDRANT_API_KEY: "<api_key>"

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: <qdrant_pvc>
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: <storage_size>

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: <qdrant_deployment>
spec:
  replicas: 1
  selector:
    matchLabels:
      app: <qdrant_app_label>
  template:
    metadata:
      labels:
        app: <qdrant_app_label>
    spec:
      containers:
      - name: qdrant
        image: qdrant/qdrant:v1.12.0
        ports:
        - name: http
          containerPort: 6333
        - name: grpc
          containerPort: 6334
        env:
        - name: QDRANT__SERVICE__API_KEY
          valueFrom:
            secretKeyRef:
              name: <qdrant_credentials_secret>
              key: QDRANT_API_KEY
        volumeMounts:
        - name: qdrant-storage
          mountPath: /qdrant/storage
        - name: qdrant-storage
          mountPath: /qdrant/snapshots
          subPath: snapshots
        readinessProbe:
          httpGet:
            path: /readyz
            port: 6333
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /healthz
            port: 6333
          initialDelaySeconds: 10
          periodSeconds: 20
      volumes:
      - name: qdrant-storage
        persistentVolumeClaim:
          claimName: <qdrant_pvc>

---
apiVersion: v1
kind: Service
metadata:
  name: <qdrant_service>
spec:
  selector:
    app: <qdrant_app_label>
  ports:
  - name: http
    port: 6333
    targetPort: 6333
  - name: grpc
    port: 6334
    targetPort: 6334
  type: ClusterIP
    Note

    If your Qdrant instance does not require authentication, remove the Secret and the QDRANT__SERVICE__API_KEY environment variable from the Deployment configuration.

  5. Replace the placeholder values as follows:

    • <qdrant_credentials_secret>: A name for the Secret that stores the Qdrant API key, for example qdrant-credentials.
    • <api_key>: An API key for authenticating with Qdrant. If authentication is not required, remove the Secret and the QDRANT__SERVICE__API_KEY environment variable from the Deployment.
    • <qdrant_pvc>: A name for the PersistentVolumeClaim, for example qdrant-pvc.
    • <storage_size>: The storage capacity to request, for example 10Gi.
    • <qdrant_deployment>: A name for the Deployment, for example qdrant.
    • <qdrant_app_label>: A label for the application, for example qdrant.
    • <qdrant_service>: A name for the Service, for example qdrant-service.
  6. Click Create.

Verification

  • The Qdrant Service is present in the project and exposes ports 6333 (HTTP) and 6334 (gRPC). You can confirm this on the Networking Services page in the OpenShift web console.
  • The Qdrant pod reaches the Running state. You can confirm this on the Workloads Pods page in the OpenShift web console.
Note

This verification confirms only that Qdrant is deployed and reachable within the project. To use this Qdrant instance with OGX, configure the Qdrant provider in a OGXServer.

To use Qdrant as a remote vector store, configure your OGXServer resource with the connection details for your Qdrant service. This configuration enables OGX to store and retrieve embedding vectors using Qdrant in OpenShift AI.

Prerequisites

  • You have installed and enabled the OGX Operator in OpenShift AI.
  • You have a running Qdrant instance that is accessible from your OpenShift cluster.
  • You have the Qdrant connection details, including the service URL and, if required, an API key.
  • You have permission to create Secrets and modify custom resources in your project.

Procedure

  1. In the OpenShift web console, switch to the Administrator perspective.

  2. Create a Secret that stores the Qdrant connection details used by OGX. This Secret must contain the URL of the Qdrant service and, if required, the API key.

    Note

    If you deployed Qdrant by using the procedure in Deploying a Qdrant vector database, create this Secret separately for the OGX configuration. The Secret created during the Qdrant deployment does not contain the QDRANT_URL value required by the OGX provider.

    1. From the Project list, select the project where the OGXServer resource is deployed.
    2. Click Workloads Secrets.
    3. Click Create From YAML.

    4. Paste the following YAML: 

      apiVersion: v1
kind: Secret
metadata:
  name: qdrant-connection
type: Opaque
stringData:
  QDRANT_URL: "<qdrant_url>"
  QDRANT_API_KEY: "<api_key>"

    5. Replace the placeholder values as follows:

      • <qdrant_url>: The full URL to the Qdrant service, for example http://qdrant-service:6333. For in-cluster deployments, use the Service name and port. For external deployments, use the external URL.
      • <api_key>: The API key for authenticating with Qdrant. If authentication is not enabled for your Qdrant instance, remove the QDRANT_API_KEY entry from both the Secret and the env section in the OGXServer configuration.
    6. Click Create.
  3. Update your OGXServer custom resource to reference the Secret and supply the required environment variables.

  4. Go to the Installed Operators page. The navigation path depends on your OpenShift version:

    • On OpenShift 4.20 and later, click Ecosystem Installed Operators.
    • On OpenShift 4.19, click Operators Installed Operators.
    1. Select the OGX Operator.
    2. Click the OGXServer tab.
    3. Select your OGXServer resource.
    4. Click YAML.

    5. Update the resource to include the following fields.

      Note

      The environment variable names and configuration fields used by the Qdrant provider can vary depending on the OGX version included with OpenShift AI. Before applying this configuration, verify that the variables and fields match the supported versions listed in Supported Configurations for 3.x. 

      apiVersion: ogx.io/v1beta1
kind: OGXServer
metadata:
  name: ogx
spec:
  server:
    containerSpec:
      env:
        - name: ENABLE_QDRANT
          value: "true"
        - name: QDRANT_URL
          valueFrom:
            secretKeyRef:
              name: qdrant-connection
              key: QDRANT_URL
        - name: QDRANT_API_KEY
          valueFrom:
            secretKeyRef:
              name: qdrant-connection
              key: QDRANT_API_KEY
    6. Click Save.

Verification

  • The OGX pod reaches the Running state. You can confirm this on the Workloads Pods page in the OpenShift web console.
  • The pod logs show that the Qdrant provider initializes successfully and does not report connection errors.

  • Vector operations executed through the OGX API complete successfully, confirming that OGX can communicate with Qdrant.

    For information about performing vector operations, see:

    Performing vector operations with Qdrant.

7.4.4. Performing vector operations with Qdrant
Copy link

After configuring Qdrant as the vector store provider in OGX, you can perform vector operations by using the OpenAI-compatible Vector Stores API exposed by OGX. These operations include creating vector stores, adding documents, performing similarity search, and deleting vector stores. You interact with the OGX API rather than connecting directly to Qdrant. OGX manages collection creation, embedding generation, and query execution on your behalf.

Prerequisites

  • You have installed and enabled the OGX Operator in OpenShift AI.
  • You have configured Qdrant as the vector store provider in your OGXServer.
  • You have an embedding model available through a configured inference provider.
  • You have network access to the OGX API endpoint.

  • You have installed the jq command-line utility.

    For installation instructions, see jq.

  • You have the curl command-line tool installed.

Procedure

  1. Determine how you will access the OGX API.

    You can access the API from within the cluster or from outside the cluster.

    • In-cluster access: Run the curl commands from a pod in the same project, or from a workstation that has network access to the OGX Service.

    • External access: Expose the OGX Service by creating a Route, and then use the Route URL from your local workstation.

      For this procedure, set OGX_URL to the service or route root URL without the /v1 suffix. The example commands append /v1 as part of the endpoint path.

      For more information about API compatibility and base URL requirements, see

      OpenAI compatibility for RAG APIs in OGX.

      Example base URL for in-cluster access

      OGX_URL="http://ogx-service:8321"

      Example base URL for external access through a Route

      OGX_URL="https://ogx-route.example.com"

  2. Create a vector store and capture its ID. 

    CREATE_RESPONSE=$(curl -s -X POST "${OGX_URL}/v1/vector_stores" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-rag-store",
    "embedding_model": "vllm/ibm-granite/granite-embedding-125m-english",
    "embedding_dimension": 768,
    "provider_id": "qdrant-remote"
  }')

VECTOR_STORE_ID=$(echo "$CREATE_RESPONSE" | jq -r '.id')
echo "Vector store ID: ${VECTOR_STORE_ID}"

    Ensure that the VECTOR_STORE_ID variable contains a valid value before continuing.

7.4.4.1. Add files to a vector store
Copy link

Upload files to the vector store for ingestion. OGX automatically splits the content into chunks, generates embeddings, and stores them in Qdrant.

Example using curl

FILE_RESPONSE=$(curl -s -X POST "${OGX_URL}/v1/vector_stores/${VECTOR_STORE_ID}/files" \
  -F "file=@/path/to/document.pdf" \
  -F "purpose=assistants")

FILE_ID=$(echo "$FILE_RESPONSE" | jq -r '.id')
echo "File ID: ${FILE_ID}"

7.4.4.2. Query a vector store
Copy link

Perform similarity search to retrieve relevant content from the vector store. The search query is converted into an embedding and compared with stored vectors in Qdrant.

Example using curl

curl -X POST "${OGX_URL}/v1/vector_stores/${VECTOR_STORE_ID}/search" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is retrieval-augmented generation?",
    "max_results": 5
  }'

7.4.4.3. Delete a vector store
Copy link

Delete a vector store when it is no longer required. This removes the vector store and its associated data from Qdrant.

Example using curl

curl -X DELETE "${OGX_URL}/v1/vector_stores/${VECTOR_STORE_ID}"

Verification

  • Creating a vector store returns a valid vector store ID.
  • File uploads complete successfully and are accepted by the API.
  • Search queries return results from the ingested content.
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top