1.3. Hugging Face 中的验证模型 - 2025 年 5 月收集

红帽 AI 在 Hugging Face 上提供以下模型验证是否与红帽 AI Inference Server 搭配使用。

Expand

表 1.3. Red Hat AI 验证模型 - 2025 年 5 月收集
model	Quantized 变体	Hugging Face model 卡	验证于
gemma-2-9b-it	FP8	Baseline FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
granite-3.1-8b-base	INT4	INT4	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
granite-3.1-8b-instruct	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Llama-3.1-8B-Instruct	None	Baseline	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Llama-3.1-Nemotron-70B-Instruct-HF	FP8	Baseline FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Llama-3.3-70B-Instruct	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Llama-4-Maverick-17B-128E-Instruct	FP8	Baseline FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Llama-4-Scout-17B-16E-Instruct	INT4, FP8	Baseline INT4 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Meta-Llama-3.1-8B-Instruct	INT4, INT8, FP8	INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Mistral-Small-24B-Instruct-2501	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Mistral-Small-3.1-24B-Instruct-2503	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Mixtral-8x7B-Instruct-v0.1	None	Baseline	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
phi-4	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20
Qwen2.5-7B-Instruct	INT4, INT8, FP8	Baseline INT4 INT8 FP8	RHAIIS 3.0 RHELAI 1.5 RHOAI 2.20