Red Hat Summit Red Hat Summit지원콘솔개발자평가판 시작연락처

이 콘텐츠는 선택한 언어로 제공되지 않습니다.

Chapter 3. Version 3.2.0 release notes

Red Hat AI Inference Server 3.2.0 release provides container images that optimizes inferencing with large language models (LLMs) for NVIDIA CUDA and AMD ROCm AI accelerators. The container images are available from registry.redhat.io:

  • registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.0
  • registry.redhat.io/rhaiis/vllm-rocm-rhel9:3.2.0

With Red Hat AI Inference Server, you can serve and inference models with higher performance, lower cost, and enterprise-grade stability and security. Red Hat AI Inference Server is built on the upstream, open source vLLM software project.

New versions of vLLM and LLM Compressor are included in this release:

The Red Hat AI Inference Server supported product and hardware configurations have been expanded. For more information, see Supported product and hardware configurations.

Expand
Table 3.1. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell support

Runs on NVIDIA B200 compute capability 10.0 GPUs with FP8 kernels and full CUDA Graph acceleration

NVIDIA Blackwell

FP8 KV-cache on ROCm

Roughly twice as large context windows with no accuracy loss

All AMD GPUs

Skinny GEMMs

Roughly 10% lower inference latency

AMD MI300X

Full CUDA Graph mode

6–8% improved average Time Per Output Token (TPOT) for small models.

NVIDIA A100 and H100

Auto FP16 fallback

Stable runs on pre-Ampere cards without manual flags, for example, NVIDIA T4 GPUs

Older NVIDIA GPUs

3.1. New models enabled
링크 복사

Expand
Table 3.2. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell compute capability 12.0

Runs on NVIDIA RTX PRO 6000 Blackwell Server Edition supporting W8A8/FP8 kernels and related tuning

NVIDIA RTX PRO 6000 Blackwell Server Edition

ROCm improvements

Full‑graph capture for TritonAttention, quick All‑Reduce, and chunked pre‑fill

AMD ROCm

3.2. New models enabled
링크 복사

Red Hat AI Inference Server 3.2.0 expands capabilities by enabling the following models added in vLLM v0.9.1:

  • LoRA support for InternVL
  • Magistral
  • Minicpm eagle support
  • NemotronH

The following models were added in vLLM v0.9.0:

  • dots1
  • Ernie 4.5
  • FalconH1
  • Gemma‑3
  • GLM‑4.1 V
  • GPT‑2 for Sequence Classification
  • Granite 4
  • Keye‑VL‑8B‑Preview
  • LlamaGuard4
  • MiMo-7B
  • MiniMax-M1
  • MiniMax-VL-01
  • Ovis 1.6, Ovis 2
  • Phi‑tiny‑MoE‑instruct
  • Qwen 3 Embedding & Reranker
  • Slim-MoE
  • Tarsier 2
  • Tencent HunYuan‑MoE‑V1

3.3. New developer features
링크 복사

Improved scheduler performance
The vLLM scheduler API CachedRequestData class has been updated, resulting in improved performance for object and cached sampler‑ID stores.
CUDA graph execution
  • CUDA graph execution is now available for all FlashAttention-3 (FA3) and FlashMLA paths, including prefix‑caching.
  • New live CUDA graph capture progress bar makes debugging easier.
Scheduling
Priority scheduling is now implemented in the vLLM V1 engine.
맨 위로 이동
Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2025 Red Hat