Red Hat SummitChapter 1. Version 3.4.0-ea.2 release notes
Red Hat Enterprise Linux AI 3.4.0-ea.2 is the second Early Access release for the Red Hat Enterprise Linux AI 3.4 release. This release adds Intel Gaudi 3 AI accelerator support to the RHEL AI bootc container image.
Red Hat Enterprise Linux AI 3.4.0-ea.2 is an Early Access release. Early Access releases are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Early Access releases for production or business-critical workloads. Use Early Access releases to test upcoming product features in advance of their possible inclusion in a Red Hat product offering, and to test functionality and provide feedback during the development process. These features might not have any documentation, are subject to change or removal at any time, and testing is limited. Red Hat might provide ways to submit feedback on Early Access features without an associated SLA.
The following container images are available as early access releases from registry.redhat.io:
-
registry.redhat.io/rhelai-early-access/bootc-cuda-rhel9:3.4.0-ea.2 -
registry.redhat.io/rhelai-early-access/bootc-rocm-rhel9:3.4.0-ea.2 -
registry.redhat.io/rhelai-early-access/bootc-gaudi-rhel9:3.4.0-ea.2
1.1. New features
Red Hat Enterprise Linux AI 3.4.0-ea.2 packages Red Hat AI Inference Server 3.4.0-ea.2, which includes the following highlights:
- Upgraded vLLM to v0.16.0
- Red Hat AI Inference Server 3.4.0-ea.2 packages the upstream vLLM v0.16.0. You can review the complete list of updates in the upstream vLLM v0.16.0 release notes.
- Introducing Speculators library (Technology Preview)
- Speculators is an end-to-end training framework for creating EAGLE3 draft models that accelerate inference through speculative decoding, reducing model latency by 1.5-3x while maintaining output quality. for more information, see Red Hat AI Inference Server version 3.4.0-ea.2 release notes
- Intel Gaudi 3 AI accelerator support (Technology Preview)
Red Hat Enterprise Linux AI 3.4.0-ea.2 adds Intel Gaudi 3 AI accelerator support as a Technology Preview feature through the
vllm-gaudihardware plugin version 0.16.0. Thevllm-gaudi-rhel9container image is available fromregistry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2. Intel Gaudi 3 support includes paged KV cache optimized for Gaudi HPU devices, tensor parallel inference across multiple HPU devices, and FP8 quantization with Intel Neural Compressor (INC).ImportantIntel Gaudi 3 AI accelerator support is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
1.2. Known issues
There are no known issues for Red Hat Enterprise Linux AI 3.4.0-ea.2.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}