Chapter 1. About Red Hat AI Model Optimization Toolkit

Red Hat AI Model Optimization Toolkit is an open source library that incorporates the latest research in model compression, allowing you to generate compressed models with minimal effort. Red Hat AI Model Optimization Toolkit is based on the upstream LLM Compressor project.

The Red Hat AI Model Optimization Toolkit framework leverages the latest quantization, sparsity, and general compression techniques to improve generative AI model efficiency, scalability, and performance while maintaining accuracy. With native Hugging Face and vLLM support, you can seamlessly integrate optimized models with deployment pipelines for faster, cost-saving inference at scale, powered by the compressed-tensors model format.