Chapter 1. AutoRAG overview
AutoRAG is an automated optimization system in Red Hat OpenShift AI that finds the best retrieval-augmented generation (RAG) configuration for your documents and use case. You provide documents and test data. AutoRAG tests different RAG configurations, ranks results by evaluation metrics, and generates notebooks to run RAG patterns.
1.1. AutoRAG workflow
When you create an AutoRAG optimization run, AutoRAG tests combinations of chunking, embedding, retrieval, and generation settings against your test data. Each combination produces a RAG pattern with evaluation scores. AutoRAG ranks patterns on a leaderboard and generates Jupyter notebooks that you can use to run the best pattern.
AutoRAG automatically samples up to 1 GiB of relevant documents based on your test data, so you do not need to filter your document set in advance. Documents referenced in your test data are prioritized during sampling.
1.2. AutoRAG terminology
- RAG pattern
- An optimized RAG configuration that includes performance metrics, a leaderboard position, and indexing and inference notebooks.
- Search space
- The set of parameter combinations that AutoRAG tests during optimization. The search space includes chunking, embedding, retrieval, and generation settings.
1.3. Technology Preview limitations
AutoRAG is a Technology Preview feature. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. For more information, see Technology Preview Features Support Scope.
The following limitations apply during Technology Preview:
- Only English language documents are supported.
- Remote vector databases only: Milvus and pgvector (PostgreSQL) are supported. Inline vector databases are not supported.
- A maximum of three foundation models and two embedding models per optimization run. Specifying more models can cause the run to fail.
- Images embedded in documents are not processed.
- Optical character recognition (OCR) is not available for PDF documents.
- Table structure detection is not available for PDF documents.
1.4. Viewing externally created runs
The AutoRAG pipeline is automatically registered with your pipeline server when your pipeline server starts. If you create runs from the pipeline outside of the AutoRAG interface, the runs appear on the AutoRAG page in the dashboard.
To find the best RAG configuration for your documents, prepare test data, create an AutoRAG optimization run, evaluate the results, and run the best-performing pattern.
Before you begin, ensure that the following prerequisites are met:
-
A cluster administrator has set the values of the
spec.dashboardConfig.genAiStudioandspec.dashboardConfig.autoragdashboard configuration options totrue. For more information, see Dashboard configuration options. -
You have a pipeline server configured in your project. When configuring the pipeline server, select the Enable AutoML and AutoRAG pipelines checkbox in Advanced settings. If you create the
DataSciencePipelinesApplicationinstance with YAML, setspec.apiServer.managedPipelines: {}. For more information, see Configuring a pipeline server. - An OGX instance is available and configured with foundation and embedding models. For more information, see Working with OGX.
- A remote vector database is registered as a vector I/O provider with your OGX instance. Supported vector databases are Milvus and pgvector (PostgreSQL).
- Your documents are available in an S3-compatible storage bucket or locally for upload.
Upload the updated AutoRAG pipeline definition before you create your first run. For instructions and download links, see RHOAIENG-64768 - AutoML and AutoRAG pipeline runs fail with image pull errors in the release notes.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}