Red Hat SummitValidated models
Red Hat AI validated models
Abstract
Chapter 1. About Red Hat AI validated models
Red Hat AI validated models have been tested and verified to work correctly across supported hardware and product configurations. These models are available as Hugging Face downloads, as OCI artifact images, and as modelcar container images. Platform-specific validated models are also available for IBM Spyre on IBM Power and IBM Z systems.
If you are using AI Inference Server as part of a RHEL AI deployment, use OCI artifact images.
If you are using AI Inference Server as part of a OpenShift AI deployment, use ModelCar images.
Red Hat uses GuideLLM for performance benchmarking and Language Model Evaluation Harness for accuracy evaluations.
Explore the Red Hat AI validated models collections on Hugging Face.
AMD GPUs support FP8 (W8A8) and GGUF quantization variant models only. For more information, see Supported hardware.
Chapter 2. Red Hat AI validated models - February 2026
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
| Model | Quantized variants | Hugging Face model cards | Validated on |
|---|---|---|---|
| granite-4.0-h-small | FP8 |
| |
| granite-4.0-h-tiny | FP8 |
| |
| Ministral-3-14B-Instruct-2512 | None |
| |
| Phi-4-reasoning | FP8 |
| |
| Qwen3-Next-80B-A3B-Instruct | INT4 |
| |
| Qwen3-VL-235B-A22B-Instruct-NVFP4 | None |
|
Chapter 3. Red Hat AI validated models - January 2026
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
| Model | Quantized variants | Hugging Face model cards | Validated on |
|---|---|---|---|
| Apertus-8B-Instruct-2509 | FP8 |
| |
| Mistral-Large-3-675B-Instruct-2512 | None |
| |
| Mistral-Large-3-675B-Instruct-2512-NVFP4 | None |
| |
| NVIDIA-Nemotron-3-Nano-30B-A3B | FP8 |
|
Chapter 4. NVFP4 Models
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
Chapter 5. Red Hat AI validated models - October 2025 collection
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
| Model | Quantized variants | Hugging Face model card | Validated on |
|---|---|---|---|
| gpt-oss-120b | None |
| |
| gpt-oss-20b | None |
| |
| NVIDIA-Nemotron-Nano-9B-v2 | INT4, FP8 |
| |
| Qwen3-Coder-480B-A35B-Instruct | FP8 |
| |
| Voxtral-Mini-3B-2507 | FP8 |
| |
| whisper-large-v3-turbo | INT4 |
|
Chapter 6. Validated models on Hugging Face - September 2025 collection
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
| Model | Quantized variants | Hugging Face model card | Validated on |
|---|---|---|---|
| DeepSeek-R1-0528 | INT4 |
| |
| gemma-3n-E4B-it | FP8 |
| |
| Kimi-K2-Instruct | INT4 |
| |
| Qwen3-8B | FP8 |
|
Chapter 7. Validated models on Hugging Face - May 2025 collection
The following models, available from RedHat AI on Hugging Face, are validated for use with Red Hat AI Inference Server.
| Model | Quantized variants | Hugging Face model card | Validated on |
|---|---|---|---|
| gemma-2-9b-it | FP8 |
| |
| granite-3.1-8b-base | INT4 |
| |
| granite-3.1-8b-instruct | INT4, INT8, FP8 |
| |
| Llama-3.1-8B-Instruct | None |
| |
| Llama-3.1-Nemotron-70B-Instruct-HF | FP8 |
| |
| Llama-3.3-70B-Instruct | INT4, INT8, FP8 |
| |
| Llama-4-Maverick-17B-128E-Instruct | FP8 |
| |
| Llama-4-Scout-17B-16E-Instruct | INT4, FP8 |
| |
| Meta-Llama-3.1-8B-Instruct | INT4, INT8, FP8 |
| |
| Mistral-Small-24B-Instruct-2501 | INT4, INT8, FP8 |
| |
| Mistral-Small-3.1-24B-Instruct-2503 | INT4, INT8, FP8 |
| |
| Mixtral-8x7B-Instruct-v0.1 | None |
| |
| phi-4 | INT4, INT8, FP8 |
| |
| Qwen2.5-7B-Instruct | INT4, INT8, FP8 |
|
Chapter 8. Validated OCI artifact model container images
The following table lists validated OCI artifact model container images available from the Red Hat container registry, including baseline and quantized variants for each supported model.
| Model | Quantized variants | ModelCar images |
|---|---|---|
| llama-4-scout-17b-16e-instruct | INT4, FP8 |
|
| llama-4-maverick-17b-128e-instruct | FP8 |
|
| mistral-small-3-1-24b-instruct-2503 | INT4, INT8, FP8 |
|
| llama-3-3-70b-instruct | INT4, INT8, FP8 |
|
| llama-3-1-8b-instruct | INT4, INT8, FP8 |
|
| granite-3-1-8b-instruct | INT4, INT8, FP8 |
|
| phi-4 | INT4, INT8, FP8 |
|
| qwen2-5-7b-instruct | INT4, INT8, FP8 |
|
| mistral-small-24b-instruct-2501 | INT4, INT8, FP8 |
|
| mixtral-8x7b-instruct-v0-1 | None |
|
| granite-3-1-8b-base | INT4 (baseline currently unavailable) |
|
| granite-3.1-8b-starter-v2 | None |
|
| llama-3-1-nemotron-70b-instruct-hf | FP8 |
|
| gemma-2-9b-it | FP8 |
|
| deepseek-r1-0528 | INT4 (baseline currently unavailable) |
|
| qwen3-8b | FP8 (baseline currently unavailable) |
|
| kimi-k2-instruct | INT4 (baseline currently unavailable) |
|
| gemma-3n-e4b-it | FP8 (baseline currently unavailable) |
|
| gpt-oss-120b | None |
|
| gpt-oss-20b | None |
|
| qwen3-coder-480b-a35b-instruct | FP8 (baseline currently unavailable) |
|
| whisper-large-v3-turbo | INT4 (baseline currently unavailable) |
|
| voxtral-mini-3b-2507 | FP8 (baseline currently unavailable) |
|
| nvidia-nemotron-nano-9b-v2 | FP8 (baseline currently unavailable) |
|
Chapter 9. Validated Red Hat AI ModelCar container images
| Model | Quantized variants | ModelCar images |
|---|---|---|
| llama-4-scout-17b-16e-instruct | INT4, FP8 |
|
| llama-4-maverick-17b-128e-instruct | FP8 |
|
| mistral-small-3-1-24b-instruct-2503 | INT4, INT8, FP8 |
|
| llama-3-3-70b-instruct | INT4, INT8, FP8 |
|
| llama-3-1-8b-instruct | INT4, INT8, FP8 |
|
| granite-3-1-8b-instruct | INT4, INT8, FP8 |
|
| phi-4 | INT4, INT8, FP8 |
|
| qwen2-5-7b-instruct | INT4, INT8, FP8 |
|
| mistral-small-24b-instruct-2501 | INT4, INT8, FP8 |
|
| mixtral-8x7b-instruct-v0-1 | None |
|
| granite-3-1-8b-base | INT4 (baseline currently unavailable) |
|
| granite-3-1-8b-starter-v2 | None |
|
| llama-3-1-nemotron-70b-instruct-hf | FP8 |
|
| gemma-2-9b-it | FP8 |
|
| deepseek-r1-0528 | INT4 (baseline currently unavailable) |
|
| qwen3-8b | FP8 (baseline currently unavailable) |
|
| kimi-k2-instruct | INT4 (baseline currently unavailable) |
|
| gemma-3n-e4b-it | FP8 |
|
| gpt-oss-120b | None |
|
| gpt-oss-20b | None |
|
| qwen3-coder-480b-a35b-instruct | FP8 (baseline currently unavailable) |
|
| whisper-large-v3-turbo | INT4 (baseline currently unavailable) |
|
| voxtral-mini-3b-2507 | FP8 (baseline currently unavailable) |
|
| nvidia-nemotron-nano-9b-v2 | FP8 (baseline currently unavailable) |
|
| phi-4-reasoning | FP8 (baseline currently unavailable) |
|
| qwen3-vl-235b-a22b-instruct-nvfp4 | None |
|
| qwen3-next-80b-a3b-instruct | INT4 (baseline currently unavailable) |
|
| granite-4-0-h-tiny | FP8 |
|
| granite-4-0-h-small | FP8 |
|
| mistral-large-3-675b-instruct-2512 | None |
|
| mistral-large-3-675b-instruct-2512-nvfp4 | None |
|
| apertus-8b-instruct-2509 | FP8 (baseline currently unavailable) |
|
| nvidia-nemotron-3-nano-30b-a3b | FP8 (baseline currently unavailable) |
|
| ministral-3-14b-instruct-2512 | None |
|
Chapter 10. Validated models for x86_64 CPU inference serving
The following large language models have been validated for use with Red Hat AI Inference Server on x86_64 CPUs with AVX2 instruction set support. CPU inference is optimized for smaller models that can run efficiently without GPU acceleration.
x86_64 CPU inference is best suited for smaller models, typically under 3 billion parameters. Performance depends on your CPU specifications, available system RAM, and model size. For larger models or production workloads requiring high throughput, consider using GPU acceleration.
{feature-name} is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
| Model | Hugging Face model card | Number of parameters |
|---|---|---|
| TinyLlama-1.1B-Chat-v1.0 | 1.1B | |
| Llama-3.2-1B-Instruct | 1B | |
| granite-3.2-2b-instruct | 2B | |
| TinyLlama-1.1B-Chat-v1.0-pruned2.4 | 1.1B (pruned) | |
| TinyLlama-1.1B-Chat-v0.4-pruned50-quant-ds | 1.1B (pruned + quantized) | |
| opt-125m | 125M | |
| Qwen2-0.5B-Instruct-AWQ | 0.5B |
Quantization formats that require GPU-specific kernels, such as Marlin format, are not supported for CPU inference. Use AWQ or GPTQ quantization formats that are compatible with CPU execution.
The following table provides general guidance for approximate system RAM requirements based on model size:
| Model size | Minimum RAM | Recommended RAM |
|---|---|---|
| 125M - 500M | 8GB | 16GB |
| 500M - 1B | 16GB | 32GB |
| 1B - 3B | 32GB | 64GB |
Actual memory usage depends on the model architecture, context length, and batch size. Increase the VLLM_CPU_KVCACHE_SPACE environment variable to allocate more memory for the key-value cache when using longer context lengths.
Chapter 11. Validated models for use with IBM Power and IBM Spyre AI accelerators
The following large language models are supported for IBM Power systems with IBM Spyre AI accelerators.
IBM Spyre AI accelerator cards support FP16 format model weights only. For compatible models, the Red Hat AI Inference Server inference engine automatically converts weights to FP16 at startup. No additional configuration is needed.
| Model | Hugging Face model card |
|---|---|
| granite-3.3-8b-instruct | |
| granite-embedding-30m-english | |
| granite-embedding-107m-multilingual | |
| granite-embedding-125m-english | |
| granite-embedding-278m-multilingual |
| Model | Hugging Face model card |
|---|---|
| bge-reranker-v2-m3 |
Pre-built IBM Granite models run with the specific Python packages that are included in the Red Hat AI Inference Server Spyre container image. The models are tied to fixed configurations for Spyre card count, batch size, and input/output context sizes.
Updating or replacing Python packages in the Red Hat AI Inference Server Spyre container image is not supported.
Chapter 12. Validated models for use with IBM Z and IBM Spyre AI accelerators
The following large language models are supported for IBM Z systems with IBM Spyre AI accelerators.
IBM Spyre AI accelerator cards support FP16 format model weights only. For compatible models, the Red Hat AI Inference Server inference engine automatically converts weights to FP16 at startup. No additional configuration is needed.
| Model | Hugging Face model card |
|---|---|
| granite-3.3-8b-instruct |
Pre-built IBM Granite models run with the specific Python packages that are included in the Red Hat AI Inference Server Spyre container image. The models are tied to fixed configurations for Spyre card count, batch size, and input/output context sizes.
Updating or replacing Python packages in the Red Hat AI Inference Server Spyre container image is not supported.
Chapter 13. Validated models for geospatial inference with TerraTorch
The following IBM and NASA Prithvi geospatial foundation models are validated for use with AI Inference Server and TerraTorch.
Prithvi-EO-2.0 models use the Vision Transformer (ViT) architecture and require TerraTorch as the model implementation backend. These models accept GeoTIFF imagery as input and return segmentation predictions.
| Model | Use case | Hugging Face model card | Validated on |
|---|---|---|---|
| Prithvi-EO-2.0-300M-TL-Sen1Floods11 | Flood detection and mapping | RHAIIS 3.3 | |
| Prithvi-EO-2.0-300M-BurnScars | Burn scar detection | RHAIIS 3.3 |
Explore the IBM and NASA geospatial models collection on Hugging Face.
Prithvi geospatial models are validated for use with NVIDIA CUDA AI accelerators only.
These models require specific vLLM server arguments to function correctly. You must include --skip-tokenizer-init, --enforce-eager, and --enable-mm-embeds when starting the inference server.
For the complete list of required server arguments, see TerraTorch configuration options for geospatial model serving and Serving TerraTorch Models with vLLM.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}