Red Hat SummitChapter 2. Red Hat Enterprise Linux AI product architecture
Red Hat Enterprise Linux AI contains various distinct features and consists of the following components.
2.1. Bootable Red Hat Enterprise Linux with InstructLab
You can install RHEL AI and deploy the InstructLab tooling using a bootable RHEL container image provided by Red Hat. The current supported installation methods for this image are on Amazon Web Services (AWS), IBM Cloud, and bare-metal machines with NVIDIA GPUs.
This RHEL AI image includes InstructLab, RHEL 9.4, and various inference and training software, including vLLM and DeepSpeed. After you boot this image, you can download various Red Hat and IBM developed Granite models to serve or train. The image and all the tools are compiled to specific Independent Software Vendor (ISV) hardware. For more information about the architecture of the image, see Installation overview
RHEL AI currently only includes bootable images for NVIDIA accelerators.
2.1.1. InstructLab model alignment
The Red Hat Enterprise Linux AI bootable image contains InstructLab and its tooling. InstructLab uses a novel approach to LLM fine-tuning called LAB (Large-Scale Alignment for ChatBots). The LAB method uses a taxonomy-based system that implements high-quality synthetic data generation (SDG) and multi-phase training.
Using the RHEL AI command line interface (CLI), which is built from the InstructLab CLI, you can create your own custom LLM by tuning a Granite base model on synthetic data generated from your own domain-specific knowledge.
For general availability, the RHEL AI LLMs customization workflow consists of the following steps:
- Installing and initializing RHEL AI on your preferred platform.
- Using a CLI and Git workflow for adding skills and knowledge to your taxonomy tree.
-
Running synthetic data generation (SDG) using the
mixtral-8x7B-Instructteacher model. SDG can generate hundreds or thousands of synthetic question-and-answer pairs for model tuning based on user-provided specific samples. -
Using the InstructLab to train the base model with the new synthetically generated data. The
prometheus-8x7B-V2.0judge model evaluates the performance of the newly trained model. - Using InstructLab with vLLM to serve the new custom model for inferencing.
2.1.2. Open source licensed Granite models
With RHEL AI, you can download the open source licensed IBM Granite family of LLMs.
Using the granite-7b-starter model as a base, you can create your model using knowledge data. You can keep these custom LLMs private or you can share them with the AI community.
Red Hat Enterprise Linux AI also allows you to serve and chat with Granite models created and fine-tuned by Red Hat and IBM.
