Red Hat SummitChapter 1. Version 3.4 release notes
Red Hat Enterprise Linux AI is a generative AI inference platform for Linux environments that uses Red Hat AI Inference Server for running and optimizing models, and includes Red Hat AI Model Optimization Toolkit for model quantization, sparsity, and general compression for supported AI accelerators. Red Hat AI Model Optimization Toolkit has native Hugging Face and vLLM support. You can seamlessly integrate optimized models with deployment pipelines for faster, cost-saving inference at scale, powered by the compressed-tensors model format.
Red Hat Enterprise Linux AI 3.4.0-ea.1 is an Early Access release. Early Access releases are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Early Access releases for production or business-critical workloads. Use Early Access releases to test upcoming product features in advance of their possible inclusion in a Red Hat product offering, and to test functionality and provide feedback during the development process. These features might not have any documentation, are subject to change or removal at any time, and testing is limited. Red Hat might provide ways to submit feedback on Early Access features without an associated SLA.
Red Hat Enterprise Linux AI is packaged as a bootc container image for easy deployment on a Linux server appliance with NVIDIA CUDA or AMD ROCm AI accelerators installed. The following container images are available as early access releases from registry.redhat.io:
-
registry.redhat.io/rhelai-early-access/bootc-cuda-rhel9:3.4.0-ea.1 -
registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.4.0-ea.1
There is no direct upgrade path from Red Hat Enterprise Linux AI 1.5 to Red Hat Enterprise Linux AI 3.0. You can upgrade from Red Hat Enterprise Linux AI 3.0 to 3.4 and all versions in-between.
The registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.4.0-ea.1 image does not include Red Hat AI Model Optimization Toolkit, which is not supported for AMD ROCm AI accelerators.
1.1. New features
Red Hat Enterprise Linux AI 3.4 packages Red Hat AI Inference Server 3.4, which includes the following highlights:
- Upgraded vLLM to v0.14.1
- Red Hat AI Inference Server 3.4 packages the upstream vLLM v0.14.1 release with asynchronous scheduling enabled by default, a new gRPC server entrypoint, auto-context length fitting, and security fixes including token leak prevention in crash logs.
- New model support
- Red Hat AI Inference Server 3.4 adds support for Grok-2, Mistral 3, MiMo-V2-Flash, Nemotron Parse 1.1, and various other model architectures. LoRA multimodal support has been expanded for LLaVA, BLIP2, PaliGemma, Pixtral, and GLM4-V models. Tool calling enhancements include FunctionGemma and GLM-4.7 parsers.
- Performance improvements
- Asynchronous scheduling now overlaps engine core scheduling with GPU execution, improving throughput without manual configuration. CUTLASS MoE optimizations deliver up to 5.3% throughput gain and up to 10.8% time to first token improvement. Fused RoPE and MLA KV-cache write optimization improves DeepSeek-style model performance.
- New AI accelerator support
- Red Hat AI Inference Server 3.4 adds RTX PRO 4500 Blackwell Server Edition GPU support for NVIDIA, AITER RMSNorm fusion for AMD, and chunked prefill and prefix caching for IBM Spyre accelerators. CPU backend adds support for head sizes 80 and 112.
- Quantization advances
- Marlin support extends to Turing (sm75) architecture. New Quark int4-fp8 w4a8 MoE support, MXFP4 W4A16 support for dense models, and ModelOpt FP8 variants are now available.
- Large-scale serving updates
- Extended Dual-Batch Overlap (XBO) implementation, NVIDIA Inference Xfer Library (NIXL) asymmetric tensor parallelism, and LMCache KV cache registration improve large-scale serving capabilities.
1.2. Known issues
There are no known issues for Red Hat Enterprise Linux AI 3.4.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}