Chapter 1. Overview of evaluating AI systems

Evaluate your AI systems to generate an analysis of your model’s ability by using the following TrustyAI tools:

LM-Eval: You can use TrustyAI to monitor your LLM against a range of different evaluation tasks and to ensure the accuracy and quality of its output. Features such as summarization, language toxicity, and question-answering accuracy are assessed to inform and improve your model parameters.
RAGAS: Use Retrieval-Augmented Generation Assessment (RAGAS) with TrustyAI to measure and improve the quality of your RAG systems in OpenShift AI. RAGAS provides objective metrics that assess retrieval quality, answer relevance, and factual consistency.
Llama Stack: Use Llama Stack components and providers with TrustyAI to evaluate and work with LLMs.