Chapter 7. AutoRAG configuration parameters

Optimization metric

The metric that AutoRAG uses to rank RAG patterns.

Answer faithfulness (default), Answer correctness, Context correctness

Maximum RAG patterns

The number of RAG configurations that AutoRAG evaluates.

A value between 4 and 20. Default: 8.

Foundation models

The large language models used for answer generation. Discovered from your OGX instance.

All available models are selected by default. Clear the checkbox for models to exclude them.

Embedding models

The models used to create vector embeddings and to encode queries during retrieval. Discovered from your OGX instance.

All available models are selected by default. Clear the checkbox for models to exclude them.

Vector database

The vector database where AutoRAG stores document embeddings.

A remote Milvus or pgvector (PostgreSQL) instance registered as a vector I/O provider with your OGX instance.

Input documents

Documents that AutoRAG processes and indexes for retrieval.

PDF, DOCX, PPTX, Markdown, HTML, TXT. Maximum 32 MiB per file when uploading. Documents in S3 can be selected via the file browser without upload size restrictions.

Evaluation dataset

A JSON file with test questions and expected answers for evaluating RAG quality.

JSON format. See Prepare test data for AutoRAG.

Chunking method

Recursive (recursive character text splitting)

The method used to split documents into chunks.

Chunk size

1024, 2048

The target size of each document chunk in characters.

Chunk overlap

128, 256

The number of overlapping characters between consecutive chunks.

Retrieval method

Simple (direct chunk retrieval)

The method used to retrieve relevant chunks from the vector database.

Number of chunks

3, 5, 10

The number of document chunks retrieved per query.

Search mode

Vector, Hybrid

The search strategy. Hybrid search combines vector and keyword search and is available only with Milvus.

Lightweight

Qwen2.5-7B-Instruct

Highest retrieval precision among small models. Suitable for low-latency, high-concurrency workloads on limited GPU resources.

Lightweight

Llama-3.1-8B-Instruct

Text extraction and summarization on edge clusters or single-GPU environments.

Medium

Mistral-3-14B-Instruct

Strict instruction following, structured JSON output, and tool-call formatting. Suitable for pipelines that require precise output control.

Enterprise

Llama-3.3-70B-Instruct

Cross-document synthesis, multi-step reasoning, and complex tool execution across large datasets.