Chapter 1. Red Hat Enterprise Linux AI 1.2 release notes

On RHEL AI version 1.2, you can now install and deploy Red Hat Enterprise Linux AI on machines with AMD accelerators as a technology preview. RHEL AI currently only supports AMD hardware on bare metal and Azure. For more information about RHEL AI hardware requirements for AMD, see Red Hat Enterprise Linux AI hardware requirements.

RHEL AI verion 1.2 currently does not provide a training profile for AMD hardware. You need to manually configure your config.yaml file and add the proper training configurations for training on AMD. For the documentation on how to manually configure your AMD training profile, see Configuring the training profile for AMD accelerators (Technology preview).

On RHEL AI version 1.2, you can now install and deploy Red Hat Enterprise Linux AI on Google Cloud Platform (GCP) instances as a technology preview. For the documentation on installing Red Hat Enterprise Linux AI on GCP, see Installing on Google Cloud Platform (GCP)

RHEL AI currently supports 8xA100 and 8xH100 accelerators on GCP instances for the full end-to-end workflow. You can also serve LLMs provided by Red Hat for inferencing on GCP instances. For more details on the RHEL AI hardware requirements for GCP, see Red Hat Enterprise Linux AI hardware requirements. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

You can now install and deploy Red Hat Enterprise Linux AI version 1.2 on Microsoft Azure as a technology preview. For the documentation on installing Red Hat Enterprise Linux AI on Azure, see Installing on Azure

RHEL AI currently supports 8xA100 and 8xH100 accelerators on Azure instances for the full end-to-end workflow. You can also serve LLMs provided by Red Hat for inferencing on Azure instances. For more details on the RHEL AI hardware requirements for AWS, see Red Hat Enterprise Linux AI hardware requirements. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

You can initialize and set up your RHEL AI environment by running the ilab config init command. This command creates the necessary configurations for interacting with RHEL AI and fine-tuning models. It also creates proper directories for your data files.

You can download various Large Language Models (LLMs) provided by Red Hat to your RHEL AI machine or instance. You can download these models from a Red Hat registry after creating and logging in to your Red Hat registry account. For more information about the supported RHEL AI LLMs, see the Downloading models documentation and the "Large Language Models (LLMs) technology preview status".

The granite-8b-code-instruct and granite-8b-code-base code models are continuing in Technology Preview on RHEL AI version 1.2. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Red Hat Enterprise Linux AI version 1.2 allows you to run a vLLM inference server on various LLMs. The vLLM tool is a memory-efficient inference and serving engine library for LLMs that is included in the RHEL AI image. For more information about serving and chatting with models, see Serving and chatting with the models documentation.

There are various networking endpoint customizations you can create on Red Hat Enterprise Linux AI. Including creating a API enpoint for serving a model, setting up your machine as a inference server, and various other options. For more information about these customizations, see Serving and chatting with the models documentation.

On Red Hat Enterprise Linux AI version 1.2, you can now use IBM cloud to customize the Granite model and run the end-to-end InstructLab workflow. For more details on the RHEL AI end-to-end hardware requirements for IBM Cloud, see Red Hat Enterprise Linux AI hardware requirements.

On Red Hat Enterprise Linux AI, you can customize your taxonomy tree so a model can learn domain-specific information. You host your knowledge data in a Git repository and fine-tune a model with that data. In the RHEL AI workflow, you create a qna.yaml file that includes questions and answers for the model to learn. This file gets run through the synthetic data generation (SDG) process, training, and evaluation, to then create a new LLM that contains the data from the Git repository and qna.yaml file. For detailed documentation on how to create a knowledge markdown and YAML file, see Adding knowledge to your taxonomy tree.

Red Hat Enterprise Linux AI includes the LAB enhanced method of synthetic data generation (SDG). You can use the qna.yaml files with your own knowledge data to create hundreds of artifical datasets in the SDG process. For more information about running the SDG process, see Generating a new dataset with Synthetic data generation (SDG).

Red Hat Enterprise Linux AI includes the LAB enhanced method of multi-phase training: A fine-tuning strategy where datasets are trained and evaluated in multiple phases to create the best possible model. For more details on multi-phase training, see Training your data on the model.

Red Hat Enterprise Linux AI includes the ability to run benchmark evaluations on the newly trained models. On your trained model, you can evaluate how well the model knows the model you added with the MMLU_BRANCH benchmark. For more details on benchmark evaluation, see Evaluating your new model.