Chapter 1. Red Hat Enterprise Linux AI 1.1 release notes

You can install Red Hat Enterprise Linux AI on various NVIDIA bare-metal hardware in RHEL AI version 1.1. For the documentation on installing Red Hat Enterprise Linux AI on bare metal, see Installing RHEL AI on bare metal.

RHEL AI currently supports 2x-8x NVIDIA A100, 2x-8x NVIDIA H100, and 4x-8x NVIDIA L40S GPU accelerators for the full end-to-end workflow. You can also inference serve LLMs provided by Red Hat on NVIDIA bare-metal hardware. For more details on the RHEL AI hardware requirements for NVIDIA bare-metal machines, see Red Hat Enterprise Linux AI hardware requirements.

You can install and deploy Red Hat Enterprise Linux AI on Amazon Web Services (AWS) in RHEL AI version 1.1. You can deploy an Red Hat Enterprise Linux AI AWS instance by converting a RHEL AI raw disk image into an Amazon Machine Image (AMI) and launching an instance with the AMI. For the documentation on converting the RHEL AI image to an AMI and deploying the instance, see Installing RHEL AI on AWS.

RHEL AI currently supports 8xA100 and 8xH100 accelerators on AWS instances for the full end-to-end workflow. You can also inference serve LLMs provided by Red Hat on AWS instances. For more details on the RHEL AI hardware requirements for AWS, see Red Hat Enterprise Linux AI hardware requirements.

You can install and deploy Red Hat Enterprise Linux AI on IBM Cloud in RHEL AI version 1.1. You can deploy an Red Hat Enterprise Linux AI IBM Cloud instance by converting the RHEL AI image into an QCOW image, then into a IBM Cloud image and launching an instance. For the documentation on converting the RHEL AI image to an IBM Cloud and deploying the instance, see Installing RHEL AI on IBM Cloud.

RHEL AI currently only supports inference serving Large Language Models (LLMs) on version 1.1. For more details on the RHEL AI hardware requirements for IBM Cloud, see Red Hat Enterprise Linux AI hardware requirements

You can initialize and set up your RHEL AI environment by running the ilab config init command. This command creates the necessary configurations for interacting with RHEL AI and fine-tuning models. It also creates proper directories for your data files. Red Hat Enterprise Linux AI version 1.1 includes the ability to select a training profile that matches your hardware configurations through the InstructLab CLI. The training profiles in the CLI allows you to easily configure your machine for training when setting up your RHEL AI environment. For more information about initializing InstructLab, see the Initialize InstructLab documentation.

You can download various Large Language Models (LLMs) provided by Red Hat to your RHEL AI machine or instance. You can download these models from a Red Hat registry after creating and logging in to your Red Hat registry account. RHEL AI allows you to fine-tune the granite-7b-starter base model with your own knowledge data. There are various models necessary for the end-to-end workflow as well that are downloadable on RHEL AI, including the mixtral-8x7B-instruct-v0-1 teacher model for SDG and prometheus-8x7b-v2.0 judge model for training and evaluation. For more information about downloading models, see the Downloading models documentation.

The granite-8b-code-instruct and granite-8b-code-base code models are currently in Technology Preview on RHEL AI version 1.1. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Red Hat Enterprise Linux AI version 1.1 allows you to run an vLLM inference server on various LLMs. The vLLM tool is a memory-efficient inference and serving engine library for LLMs that is included in the RHEL AI image. After you serve a specified model, you can chat with the model via the InstructLab CLI. For more information about serving and chatting with models, see Serving and chatting with the models documentation.

On Red Hat Enterprise Linux AI, you can customize your taxonomy tree so a model can learn domain-specific information. You host your knowledge data in a Git repository and fine-tune a model with that data. Red Hat Enterprise Linux AI version 1.1 currently only supports knowledge data in markdown format. In the RHEL AI workflow, you create a qna.yaml file that includes questions and answers for the model to learn. This file gets run through the synthetic data generation (SDG) process, training, and evaluation, to then create a new LLM that contains the data from the Git repository and qna.yaml file. For detailed documentation on how to create a knowledge markdown and YAML file, see Adding knowledge to your taxonomy tree.

Red Hat Enterprise Linux AI includes the LAB enhanced method of synthetic data generation (SDG). You can use the qna.yaml files with your own knowledge data to create hundreds of artifical datasets in the SDG process. Red Hat Enterprise Linux AI includes the mixtral-8x7B-instruct-v0-1 LLM, available for downloading, as the teacher and critic model for SDG. For more information on running the SDG process, see Generating a new dataset with Synthetic data generation (SDG).

Red Hat Enterprise Linux AI includes the LAB enhanced method of multi-phase training: A fine-tuning strategy where datasets are trained and evaluated in multiple phases to create the best possible model. You can use the synthetic data set generated during SDG to train, evaluate and create a new model with your data. Red Hat Enterprise Linux AI includes the prometheus-8x7b-v2.0 LLM, available for downloading, as the judge model for multi-phase training and evaluation. For more details on multi-phase training, see Training your data on the model.

Red Hat Enterprise Linux AI includes the ability to run benchmark evaluations on the newly trained models. On your trained model, you can evaluate how well the model knows the model you added with the MMLU_BRANCH benchmark. Red Hat Enterprise Linux AI includes the prometheus-8x7b-v2.0 LLM, available for downloading, as the judge model for benchmark evaluation. For more details on benchmark evaluation, see Evaluating your new model.