Red Hat Summit Red Hat SummitApoyoConsolaDesarrolladoresIniciar una pruebaContacto

Este contenido no está disponible en el idioma seleccionado.

Chapter 1. Red Hat AI validated models

The following table lists the Red Hat AI validated models for use with Red Hat AI Inference Server 3.1.

  • If you are using AI Inference Server as standalone product, use the Hugging Face images.
  • If you are using AI Inference Server as part of a RHEL AI deployment, use the model OCI artifact image.
  • If you are using AI Inference Server as part of a OpenShift AI deployment, use the model ModelCar image.
Important

AMD GPUs support FP8 (W8A8) and GGUF quantization variant models only. For more information, see Supported hardware.

Expand
Table 1.1. Red Hat AI validated models
ModelQuantized variantsHugging Face model cards [1]OCI artifact images [2]ModelCar images [3]

Llama-4-Scout-17B-16E-Instruct

INT4, FP8

  • Baseline:

    registry.redhat.io/rhelai1/llama-4-scout-17b-16e-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/llama-4-scout-17b-16e-instruct-quantized-w4a16:1.5

  • FP8:

    registry.redhat.io/rhelai1/llama-4-scout-17b-16e-instruct-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-llama-4-scout-17b-16e-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-llama-4-scout-17b-16e-instruct-quantized-w4a16:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-llama-4-scout-17b-16e-instruct-fp8-dynamic:1.5

Llama-4-Maverick-17B-128E-Instruct

FP8

  • Baseline:

    registry.redhat.io/rhelai1/llama-4-maverick-17b-128e-instruct:1.5

  • FP8:

    registry.redhat.io/rhelai1/llama-4-maverick-17b-128e-instruct-fp8:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-llama-4-maverick-17b-128e-instruct:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-llama-4-maverick-17b-128e-instruct-fp8:1.5

Mistral-Small-3.1-24B-Instruct-2503

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/mistral-small-3-1-24b-instruct-2503:1.5

  • INT4:

    registry.redhat.io/rhelai1/mistral-small-3-1-24b-instruct-2503-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/mistral-small-3-1-24b-instruct-2503-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/mistral-small-3-1-24b-instruct-2503-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-mistral-small-3-1-24b-instruct-2503:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-mistral-small-3-1-24b-instruct-2503-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-mistral-small-3-1-24b-instruct-2503-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-mistral-small-3-1-24b-instruct-2503-fp8-dynamic:1.5

Llama-3.3-70B-Instruct

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/llama-3-3-70b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/llama-3-3-70b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/llama-3-3-70b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/llama-3-3-70b-instruct-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-llama-3-3-70b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-llama-3-3-70b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-llama-3-3-70b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-llama-3-3-70b-instruct-fp8-dynamic:1.5

Llama-3.1-8B-Instruct

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/llama-3-1-8b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/llama-3-1-8b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/llama-3-1-8b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/llama-3-1-8b-instruct-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-8b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-8b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-8b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-8b-instruct-fp8-dynamic:1.5

granite-3.1-8b-instruct

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/granite-3-1-8b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/granite-3-1-8b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/granite-3-1-8b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/granite-3-1-8b-instruct-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-instruct-fp8-dynamic:1.5

phi-4

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/phi-4:1.5

  • INT4:

    registry.redhat.io/rhelai1/phi-4-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/phi-4-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/phi-4-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-phi-4:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-phi-4-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-phi-4-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-phi-4-fp8-dynamic:1.5

Qwen2.5-7B-Instruct

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/qwen2-5-7b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/qwen2-5-7b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/qwen2-5-7b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/qwen2-5-7b-instruct-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-qwen2-5-7b-instruct:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-qwen2-5-7b-instruct-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-qwen2-5-7b-instruct-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-qwen2-5-7b-instruct-fp8-dynamic:1.5

Mistral-Small-24B-Instruct-2501

INT4, INT8, FP8

  • Baseline:

    registry.redhat.io/rhelai1/mistral-small-24b-instruct-2501:1.5

  • INT4:

    registry.redhat.io/rhelai1/mistral-small-24b-instruct-2501-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/mistral-small-24b-instruct-2501-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/mistral-small-24b-instruct-2501-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-mistral-small-24b-instruct-2501:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-mistral-small-24b-instruct-2501-quantized-w4a16:1.5

  • INT8:

    registry.redhat.io/rhelai1/modelcar-mistral-small-24b-instruct-2501-quantized-w8a8:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-mistral-small-24b-instruct-2501-fp8-dynamic:1.5

Mixtral-8x7B-Instruct-v0.1

None

  • Baseline:

    registry.redhat.io/rhelai1/mixtral-8x7b-instruct-v0-1:1.4

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-mixtral-8x7b-instruct-v0-1:1.4

granite-3.1-8b-base

INT4 (baseline currently unavailable)

  • INT4:

    registry.redhat.io/rhelai1/granite-3-1-8b-base-quantized-w4a16:1.5

  • INT4:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-base-quantized-w4a16:1.5

granite-3.1-8b-starter-v2

None

  • Unavailable on Hugging Face

  • Baseline:

    registry.redhat.io/rhelai1/granite-3.1-8b-starter-v2:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-granite-3-1-8b-starter-v2:1.5

Llama-3.1-Nemotron-70B-Instruct-HF

FP8

  • Baseline:

    registry.redhat.io/rhelai1/llama-3-1-nemotron-70b-instruct-hf:1.5

  • FP8:

    registry.redhat.io/rhelai1/llama-3-1-nemotron-70b-instruct-hf-fp8-dynamic:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-nemotron-70b-instruct-hf:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-llama-3-1-nemotron-70b-instruct-hf-fp8-dynamic:1.5

gemma-2-9b-it

FP8

  • Baseline:

    registry.redhat.io/rhelai1/gemma-2-9b-it:1.5

  • FP8:

    registry.redhat.io/rhelai1/gemma-2-9b-it-fp8:1.5

  • Baseline:

    registry.redhat.io/rhelai1/modelcar-gemma-2-9b-it:1.5

  • FP8:

    registry.redhat.io/rhelai1/modelcar-gemma-2-9b-it-fp8:1.5

  1. For use with standalone Red Hat AI Inference Server
  2. For use with RHEL AI
  3. For use with Red Hat OpenShift AI
Red Hat logoGithubredditYoutubeTwitter

Aprender

Pruebe, compre y venda

Comunidades

Acerca de la documentación de Red Hat

Ayudamos a los usuarios de Red Hat a innovar y alcanzar sus objetivos con nuestros productos y servicios con contenido en el que pueden confiar. Explore nuestras recientes actualizaciones.

Hacer que el código abierto sea más inclusivo

Red Hat se compromete a reemplazar el lenguaje problemático en nuestro código, documentación y propiedades web. Para más detalles, consulte el Blog de Red Hat.

Acerca de Red Hat

Ofrecemos soluciones reforzadas que facilitan a las empresas trabajar en plataformas y entornos, desde el centro de datos central hasta el perímetro de la red.

Theme

© 2026 Red HatVolver arriba