Installing
Installation documentation on various platforms
Abstract
Chapter 1. Installation overview Copy linkLink copied to clipboard!
Red Hat Enterprise Linux AI is distributed and installable as a bootable image. This bootable image includes a container that hold various software and tools for RHEL AI. Each image is compiled to support specific hardware vendors. Each RHEL AI image includes:
- Red Hat Enterprise Linux 9.4: A RHEL version 9.4 operating system (OS) for your machine.
The InstructLab container: Contains InstructLab and various other tools required for RHEL AI. This includes:
- Python version 3.11: A Python 3.11 installation used internally by InstructLab.
The InstructLab tools:
- The InstructLab command line interface (CLI).
- The LAB enhanced method of synthetic data generation (SDG).
- The LAB enhanced method of multi-phase training.
- InstructLab with vLLM: A high-input inference and serving engine for Large Language models (LLMs).
- InstructLab with DeepSpeed: A hardware optimization software that speeds up the training process. Similar functionalities of FSDP.
- InstructLab with FSDP: A training framework that makes training faster and more efficient. Similar functionalities of DeepSpeed
Red Hat Enterprise Linux AI version 1.5 also includes a sample taxonomy tree with example skills and knowledge that you can download and use for training a model.
For general availability, there are three options for installing and deploying Red Hat Enterprise Linux AI.
After installation with Red Hat Enterprise Linux AI general availability, you can manually download open source Granite LLMs that you can chat and interact with. For more information about downloading these models, see Downloading additional models.
Chapter 2. Installing RHEL AI on bare metal Copy linkLink copied to clipboard!
For installing Red Hat Enterprise Linux AI on bare metal, you can use various methods provided in the following procedure to boot and deploy your machine and start interacting with Red Hat Enterprise Linux AI.
2.1. Deploying RHEL AI on bare metal Copy linkLink copied to clipboard!
You can deploy Red Hat Enterprise Linux AI with the RHEL AI ISO image in the following ways:
- Kickstart
- RHEL Graphical User Interface (GUI)
This image is bootable on various hardware accelerators. For more information about supported hardware, see "Red Hat Enterprise Linux AI hardware requirements" in "Getting Started".
Prerequisites
- You have downloaded and unziped the Red Hat Enterprise Linux AI ISO image from the RHEL AI download page.
Red Hat Enterprise Linux AI requires additional storage for the RHEL AI data as well as the update of image-mode Red Hat Enterprise Linux. The default location for the InstructLab data is in the home/<user>
directory. The minimum recommendation for data storage in the /home
directory is 1 TB. During updates, the bootc
command needs extra space to store temporary data. The minimum storage recommendation for the /
path is 120 GB. You need to consider your machine’s storage when partitioning the schemes of your disks.
Procedure
Interactive GUI
You can use the interactive Red Hat Enterprise Linux graphical installer and the RHEL AI ISO image to deploy RHEL AI on your machine. For more information about booting RHEL using an ISO file using the GUI, see the Interactively installing RHEL from installation media.
Kickstart with embedded container image
You can customize the RHEL AI installation by using your own Kickstart file.
Create your own Kickstart file with your preferred parameters. For more information about creating Kickstart files, see the Creating Kickstart files in the RHEL documentation.
Sample Kickstart file for RHEL AI called
rhelai-bootc.ks
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The sample Kickstart uses the embedded container image in the ISO file, signaled by the
ostreecontainer
command with the--url=/run/install/repo/container
parameter. Thebootc switch
parameter points to the Red Hat registry for future updates and then you can add your own customizations.You need to embed the Kickstart into the RHEL AI ISO so your machine can restart and deploy RHEL AI. In the following example,
rhelai-bootc.ks
is the name of the Kickstart file you’re embedding into the boot ISO. Themkksiso
utility is found in thelorax
rpm package.mkksiso rhelai-bootc.ks <downloaded-iso-image> rhelai-bootc-ks.iso
$ mkksiso rhelai-bootc.ks <downloaded-iso-image> rhelai-bootc-ks.iso
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where
- <downloaded-iso-image>
- Specify the ISO image you downloaded from the RHEL AI download page.
You can then boot your machine using this boot ISO and the installation starts automatically. After the installation is complete, the host reboots and you can login to the new system using the credentials used in the Kickstart file.
ImportantBe aware that having a custom Kickstart in your ISO will automatically start the installation, and disk partitioning, without prompting the user. Based on configuration, the local storage may be completely wiped or overwritten.
Kickstart with custom container image
You can customize a Kickstart file with your preferred parameters to boot Red Hat Enterprise Linux AI on your machine
Create your own Kickstart file with your preferred parameters. For more information on creating Kickstart files, see the Creating Kickstart files in the RHEL documentation.
Sample Kickstart file for RHEL AI called
rhelai-bootc.ks
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to embed the Kickstart into the RHEL AI ISO so your machine can restart and deploy RHEL AI. In the following example,
rhelai-bootc.ks
is the name of the Kickstart file you’re embedding into the boot ISO. Themkksiso
utility is found in thelorax
rpm package.mkksiso rhelai-bootc.ks <downloaded-iso-image> rhelai-bootc-ks.iso
$ mkksiso rhelai-bootc.ks <downloaded-iso-image> rhelai-bootc-ks.iso
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where
- <downloaded-iso-image>
- Specify the ISO image you downloaded from the RHEL AI download page.
You can then boot your machine using this boot ISO and the installation starts automatically. After the installation is complete, the host reboots and you can login to the new system using the credentials used in the Kickstart file.
ImportantBe aware that having a custom Kickstart in your ISO will automatically start the installation, and disk partitioning, without prompting the user. Based on configuration, the local storage may be completely wiped or overwritten.
Verification
To verify that your Red Hat Enterprise Linux AI tools installed correctly, you need to run the
ilab
command:ilab
$ ilab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Updating Red Hat Enterprise Linux AI
To update to the latest z-stream version of RHEL AI, follow the procedures in the Updating Red Hat Enterprise Linux AI documentation.
Chapter 3. Installing RHEL AI on AWS Copy linkLink copied to clipboard!
There are multiple ways you can install and deploy Red Hat Enterprise Linux AI on AWS.
- You can purchase RHEL AI from the AWS marketplace.
- You can download the RHEL AI RAW file on the RHEL AI download page and convert it to an AWS image.
For installing and deploying RHEL AI using the RAW file, you must first convert the RHEL AI image into an Amazon Machine Image (AMI).
3.1. Converting the RHEL AI image to an AWS AMI Copy linkLink copied to clipboard!
Before deploying RHEL AI on an AWS machine, you must set up a S3 bucket and convert the RHEL AI image to a AWS AMI.
In the following process, you create the following resources:
- An S3 bucket with the RHEL AI image
- AWS EC2 snapshots
- An AWS AMI
- An AWS instance
Prerequisites
- You have an Access Key ID configured in the AWS IAM account manager.
Procedure
- Install the AWS command-line tool by following the AWS documentation
You need to create a S3 bucket and set the permissions to allow image file conversion to AWS snapshots.
Create the necessary environment variables by running the following commands:
export BUCKET=<custom_bucket_name> export RAW_AMI=nvidia-bootc.ami export AMI_NAME="rhel-ai" export DEFAULT_VOLUME_SIZE=1000
$ export BUCKET=<custom_bucket_name> $ export RAW_AMI=nvidia-bootc.ami $ export AMI_NAME="rhel-ai" $ export DEFAULT_VOLUME_SIZE=1000
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteOn AWS, the
DEFAULT_VOLUME_SIZE
is measured GBs.You can create an S3 bucket by running the following command:
aws s3 mb s3://$BUCKET
$ aws s3 mb s3://$BUCKET
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You must create a
trust-policy.json
file with the necessary configurations for generating a S3 role for your bucket:printf '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vmie.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals":{ "sts:Externalid": "vmimport" } } } ] }' > trust-policy.json
$ printf '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vmie.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals":{ "sts:Externalid": "vmimport" } } } ] }' > trust-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an S3 role for your bucket that you can name. In the following example command,
vmiport
is the name of the role.aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json
$ aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You must create a
role-policy.json
file with the necessary configurations for generating a policy for your bucket:printf '{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Resource":[ "arn:aws:s3:::%s", "arn:aws:s3:::%s/*" ] }, { "Effect":"Allow", "Action":[ "ec2:ModifySnapshotAttribute", "ec2:CopySnapshot", "ec2:RegisterImage", "ec2:Describe*" ], "Resource":"*" } ] }' $BUCKET $BUCKET > role-policy.json
$ printf '{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Resource":[ "arn:aws:s3:::%s", "arn:aws:s3:::%s/*" ] }, { "Effect":"Allow", "Action":[ "ec2:ModifySnapshotAttribute", "ec2:CopySnapshot", "ec2:RegisterImage", "ec2:Describe*" ], "Resource":"*" } ] }' $BUCKET $BUCKET > role-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a policy for your bucket by running the following command:
aws iam put-role-policy --role-name vmimport --policy-name vmimport-$BUCKET --policy-document file://role-policy.json
$ aws iam put-role-policy --role-name vmimport --policy-name vmimport-$BUCKET --policy-document file://role-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Now that your S3 bucket is set up, you need to download the RAW image from Red Hat Enterprise Linux AI download page
Copy the RAW image link and add it to the following command:
curl -Lo disk.raw.gz <link-to-raw-file>
$ curl -Lo disk.raw.gz <link-to-raw-file>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Unzip the
raw.gz
file with the following command:gunzip disk.raw.gz
$ gunzip disk.raw.gz
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the image to the S3 bucket with the following command:
aws s3 cp disk.raw s3://$BUCKET/$RAW_AMI
$ aws s3 cp disk.raw s3://$BUCKET/$RAW_AMI
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Convert the image to a snapshot and store it in the
task_id
variable name by running the following commands:printf '{ "Description": "my-image", "Format": "raw", "UserBucket": { "S3Bucket": "%s", "S3Key": "%s" } }' $BUCKET $RAW_AMI > containers.json
$ printf '{ "Description": "my-image", "Format": "raw", "UserBucket": { "S3Bucket": "%s", "S3Key": "%s" } }' $BUCKET $RAW_AMI > containers.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow task_id=$(aws ec2 import-snapshot --disk-container file://containers.json | jq -r .ImportTaskId)
$ task_id=$(aws ec2 import-snapshot --disk-container file://containers.json | jq -r .ImportTaskId)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can check the progress of the disk image to snapshot conversion job with the following command:
aws ec2 describe-import-snapshot-tasks --filters Name=task-state,Values=active
$ aws ec2 describe-import-snapshot-tasks --filters Name=task-state,Values=active
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Once the conversion job is complete, you can get the snapshot ID and store it in a variable called
snapshot_id
by running the following command:snapshot_id=$(aws ec2 describe-import-snapshot-tasks | jq -r '.ImportSnapshotTasks[] | select(.ImportTaskId=="'${task_id}'") | .SnapshotTaskDetail.SnapshotId')
$ snapshot_id=$(aws ec2 describe-import-snapshot-tasks | jq -r '.ImportSnapshotTasks[] | select(.ImportTaskId=="'${task_id}'") | .SnapshotTaskDetail.SnapshotId')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add a tag name to the snapshot, so it’s easier to identify, by running the following command:
aws ec2 create-tags --resources $snapshot_id --tags Key=Name,Value="$AMI_NAME"
$ aws ec2 create-tags --resources $snapshot_id --tags Key=Name,Value="$AMI_NAME"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Register an AMI from the snapshot with the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can add another tag name to identify the AMI by running the following command:
aws ec2 create-tags --resources $ami_id --tags Key=Name,Value="$AMI_NAME"
$ aws ec2 create-tags --resources $ami_id --tags Key=Name,Value="$AMI_NAME"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2. Deploying your instance on AWS using the CLI Copy linkLink copied to clipboard!
You can launch the AWS instance with your new RHEL AI AMI from the AWS web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch your AWS instance with the custom AMI.
If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites".
Prerequisites
- You created your RHEL AI AMI. For more information, see "Converting the RHEL AI image to an AWS AMI".
- You have the AWS command-line tool installed and is properly configured with your aws_access_key_id and aws_secret_access_key.
- You configured your Virtual Private Cloud (VPC).
- You created a subnet for your instance.
- You created a SSH key-pair.
- You created a security group on AWS.
Procedure
For various parameters, you need to gather the ID of the variable.
To access the image ID, run the following command:
aws ec2 describe-images --owners self
$ aws ec2 describe-images --owners self
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To access the security group ID, run the following command:
aws ec2 describe-security-groups
$ aws ec2 describe-security-groups
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To access the subnet ID, run the following command:
aws ec2 describe-subnets
$ aws ec2 describe-subnets
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Populate environment variables for when you create the instance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create your instance using the variables by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
User account
The default user account in the RHEL AI AMI is cloud-user
. It has all permissions via sudo
without password.
Verification
To verify that your Red Hat Enterprise Linux AI tools are installed correctly, you need to run the
ilab
command:ilab
$ ilab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Updating Red Hat Enterprise Linux AI
To update to the latest z-stream version of RHEL AI, follow the procedures in the Updating Red Hat Enterprise Linux AI documentation.
Chapter 4. Installing RHEL AI on IBM cloud Copy linkLink copied to clipboard!
For installing and deploying Red Hat Enterprise Linux AI on IBM Cloud, you must first convert the RHEL AI image into an IBM Cloud image. You can then launch an instance using the IBM Cloud image and deploy RHEL AI on an IBM Cloud machine.
4.1. Converting the RHEL AI image into a IBM Cloud image. Copy linkLink copied to clipboard!
To create a bootable image in IBM Cloud you must configure your IBM Cloud accounts, set up a Cloud Object Storage (COS) bucket, and create a IBM Cloud image using the RHEL AI image.
Prerequisites
- You installed the IBM CLI on your specific machine. For more information about installing IBM Cloud CLI, see Installing the stand-alone IBM Cloud CLI.
Procedure
Log in to IBM Cloud with the following command:
ibmcloud login
$ ibmcloud login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow When prompted, select your desired account to log in to.
Example output of the login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to set up various IBM Cloud configurations and create your COS bucket before generating a QCOW2 image.
You can install the necessary IBM Cloud plugins by running the following command:
ibmcloud plugin install cloud-object-storage infrastructure-service
$ ibmcloud plugin install cloud-object-storage infrastructure-service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set your preferred resource group, the following example command sets the resource group named
Default
.ibmcloud target -g Default
$ ibmcloud target -g Default
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set your preferred region, the following example command sets the
us-east
region.ibmcloud target -r us-east
$ ibmcloud target -r us-east
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to select a deployment plan for your service instance. Ensure you check the properties and pricing on the IBM cloud website.
You can list the available deployment plans by running the following command:
ibmcloud catalog service cloud-object-storage --output json | jq -r '.[].children[] | select(.children != null) | .children[].name'
$ ibmcloud catalog service cloud-object-storage --output json | jq -r '.[].children[] | select(.children != null) | .children[].name'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The following example command uses the
premium-global-deployment
plan and puts it in the environment variablecos_deploy_plan
:cos_deploy_plan=premium-global-deployment
$ cos_deploy_plan=premium-global-deployment
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a Cloud Object Storage (COS) service instance and save the name in an environment variable named
cos_si_name
and create thecloud-object-storage
and by running the following commands:cos_si_name=THE_NAME_OF_YOUR_SERVICE_INSTANCE
$ cos_si_name=THE_NAME_OF_YOUR_SERVICE_INSTANCE
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ibmcloud resource service-instance-create ${cos_si_name} cloud-object-storage standard global -d ${cos_deploy_plan}
$ ibmcloud resource service-instance-create ${cos_si_name} cloud-object-storage standard global -d ${cos_deploy_plan}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Get the Cloud Resource Name (CRN) for your Cloud Object Storage (COS) bucket in a variable named
cos_crn
by running the following commands:cos_crn=$(ibmcloud resource service-instance ${cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .crn')
$ cos_crn=$(ibmcloud resource service-instance ${cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .crn')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ibmcloud cos config crn --crn ${cos_crn} --force
$ ibmcloud cos config crn --crn ${cos_crn} --force
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create your Cloud Object Storage (COS) bucket named as the environment variable
bucket_name
with the following commands:bucket_name=NAME_OF_MY_BUCKET
$ bucket_name=NAME_OF_MY_BUCKET
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ibmcloud cos bucket-create --bucket ${bucket_name}
$ ibmcloud cos bucket-create --bucket ${bucket_name}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Allow the infrastructure service to read the buckets that are in the service instance
${cos_si_guid}
variable by running the following commands:cos_si_guid=$(ibmcloud resource service-instance ${cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .guid')
$ cos_si_guid=$(ibmcloud resource service-instance ${cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .guid')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ibmcloud iam authorization-policy-create is cloud-object-storage Reader --source-resource-type image --target-service-instance-id ${cos_si_guid}
$ ibmcloud iam authorization-policy-create is cloud-object-storage Reader --source-resource-type image --target-service-instance-id ${cos_si_guid}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Now that your IBM Cloud Object Storage (CoS) service instance bucket is set up, you need to download the QCOW2 image from Red Hat Enterprise Linux AI download page
Copy the QCOW2 image link and add it to the following command:
curl -Lo disk.qcow2 "PASTE_HERE_THE_LINK_OF_THE_QCOW2_FILE"
$ curl -Lo disk.qcow2 "PASTE_HERE_THE_LINK_OF_THE_QCOW2_FILE"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the name you want to use as the RHEL AI IBM Cloud image
image_name=rhel-ai-20240703v0
$ image_name=rhel-ai-20240703v0
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the QCOW2 image to the Cloud Object Storage (COS) bucket with your selected region by running following command:
ibmcloud cos upload --bucket ${bucket_name} --key ${image_name}.qcow2 --file disk.qcow2 --region <region>
$ ibmcloud cos upload --bucket ${bucket_name} --key ${image_name}.qcow2 --file disk.qcow2 --region <region>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Convert the QCOW2 you just uploaded to an IBM Cloud image with the following commands:
ibmcloud is image-create ${image_name} --file cos://<region>/${bucket_name}/${image_name}.qcow2 --os-name red-ai-9-amd64-nvidia-byol
$ ibmcloud is image-create ${image_name} --file cos://<region>/${bucket_name}/${image_name}.qcow2 --os-name red-ai-9-amd64-nvidia-byol
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Once the job launches, set the IBM Cloud image configurations into a variable called
image_id
by running the following command:image_id=$(ibmcloud is images --visibility private --output json | jq -r '.[] | select(.name=="'$image_name'") | .id')
$ image_id=$(ibmcloud is images --visibility private --output json | jq -r '.[] | select(.name=="'$image_name'") | .id')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can view the progress of the job with the following command:
while ibmcloud is image --output json ${image_id} | jq -r .status | grep -xq pending; do sleep 1; done
$ while ibmcloud is image --output json ${image_id} | jq -r .status | grep -xq pending; do sleep 1; done
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can view the information of the newly created image with the following command:
ibmcloud is image ${image_id}
$ ibmcloud is image ${image_id}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.2. Deploying your instance on IBM Cloud using the CLI Copy linkLink copied to clipboard!
You can launch an instance with your new RHEL AI IBM Cloud image from the IBM Cloud web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch an IBM Cloud instance with the custom IBM Cloud image
If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites".
Prerequisites
- You created your RHEL AI IBM Cloud image. For more information, see "Converting the RHEL AI image to an IBM Cloud image".
- You installed the IBM CLI on your specific machine, see Installing the stand-alone IBM Cloud CLI.
- You configured your Virtual private cloud (VPC).
- You created a subnet for your instance.
Procedure
Log in to your IBM Cloud account and select the Account, Region and Resource Group by running the following command:
ibmcloud login -c <ACCOUNT_ID> -r <REGION> -g <RESOURCE_GROUP>
$ ibmcloud login -c <ACCOUNT_ID> -r <REGION> -g <RESOURCE_GROUP>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Before launching your IBM Cloud instance on the CLI, you need to create several configuration variables for your instance.
Install the
infrastructure-service
plugin for IBM Cloud by running the following commandibmcloud plugin install infrastructure-service
$ ibmcloud plugin install infrastructure-service
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to create an SSH public key for your IBM Cloud account. IBM Cloud supports RSA and ed25519 keys. The following example command uses the ed25519 key types and names it
ibmcloud
.ssh-keygen -f ibmcloud -t ed25519
$ ssh-keygen -f ibmcloud -t ed25519
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can now upload the public key to your IBM Cloud account by following the example command.
ibmcloud is key-create my-ssh-key @ibmcloud.pub --key-type ed25519
$ ibmcloud is key-create my-ssh-key @ibmcloud.pub --key-type ed25519
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to create a Floating IP for your IBM Cloud instance by following the example command. Ensure you change the region to your preferred zone.
ibmcloud is floating-ip-reserve my-public-ip --zone <region>
$ ibmcloud is floating-ip-reserve my-public-ip --zone <region>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
You need to select the instance profile that you want to use for the deployment. List all the profiles by running the following command:
ibmcloud is instance-profiles
$ ibmcloud is instance-profiles
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Make a note of your preferred instance profile, you will need it for your instance deployment.
You can now start creating your IBM Cloud instance. Populate environment variables for when you create the instance.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can now launch your instance, by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Link the Floating IP to the instance by running the following command:
ibmcloud is floating-ip-update $floating_ip --nic primary --in $name
$ ibmcloud is floating-ip-update $floating_ip --nic primary --in $name
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
User account
The default user account in the RHEL AI AMI is cloud-user
. It has all permissions via sudo
without password.
Verification
To verify that your Red Hat Enterprise Linux AI tools are installed correctly, run the
ilab
command:ilab
$ ilab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.3. Adding more storage to your IBM Cloud instance Copy linkLink copied to clipboard!
In [ibm-c], there is a size restriction of 250 GB of storage in the main IBM Cloud disk. RHEL AI might require more storage for models and generation data.
You can add more storage by attaching an extra disk to your instance and using it to hold data for RHEL AI.
Prerequisites
- You have a IBM Cloud RHEL AI instance.
Procedure
Create an environment variable called
name
that has the name of your instance by running the following command:name=my-rhelai-instance
$ name=my-rhelai-instance
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the size of the new volume by running the following command:
data_volume_size=1000
$ data_volume_size=1000
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create and attach the instance volume by running the following command:
ibmcloud is instance-volume-attachment-add data ${name} \ --new-volume-name ${name}-data \ --profile general-purpose \ --capacity ${data_volume_size}
$ ibmcloud is instance-volume-attachment-add data ${name} \ --new-volume-name ${name}-data \ --profile general-purpose \ --capacity ${data_volume_size}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can list all the disks with the following command:
lsblk
$ lsblk
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
disk
variable with the content of the disk path your using. The following example command uses the/dev/vdb
path.disk=/dev/vdb
$ disk=/dev/vdb
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a partition on your disk by running the following command:
sgdisk -n 1:0:0 $disk
$ sgdisk -n 1:0:0 $disk
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Format and label the partition by running the following command:
mkfs.xfs -L ilab-data ${disk}1
$ mkfs.xfs -L ilab-data ${disk}1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can configure your system to auto mount to your preferred directory. The following example command uses the
/mnt
directory.echo LABEL=ilab-data /mnt xfs defaults 0 0 >> /etc/fstab
$ echo LABEL=ilab-data /mnt xfs defaults 0 0 >> /etc/fstab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Reload the
systemd
service to acknowledge the new configuration on mounts by running the following command:systemctl daemon-reload
$ systemctl daemon-reload
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Mount the disk with the following command:
mount -a
$ mount -a
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Grant write permissions to all users in the new file system by running the following command:
chmod 1777 /mnt/
$ chmod 1777 /mnt/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Updating Red Hat Enterprise Linux AI
To update to the latest z-stream version of RHEL AI, follow the procedures in the Updating Red Hat Enterprise Linux AI documentation.
Chapter 5. Installing RHEL AI on Google Cloud Platform (GCP) Copy linkLink copied to clipboard!
For installing and deploying Red Hat Enterprise Linux AI on Google Cloud Platform, you must first convert the RHEL AI image into an GCP image. You can then launch an instance using the GCP image and deploy RHEL AI on a Google Cloud Platform machine.
5.1. Converting the RHEL AI image into a Google Cloud Platform image. Copy linkLink copied to clipboard!
To create a bootable image in Google Cloud Platform you must configure your Google Cloud Platform account, create an Google Cloud Storage bucket, and create an Google Cloud Platform image using the RHEL AI raw image.
Prerequisites
- You installed the Google Cloud Platform CLI on your specific machine. For more information about installing the GCP CLI, see Install the Google Cloud Platform CLI on Linux.
- You must be on a Red Hat Enterprise Linux version 9.2 - 9.4 system
- Your machine must have an additional 100 GB of disk space.
Procedure
Log in to Google Cloud Platform with the following command:
gcloud auth login
$ gcloud auth login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output of the login.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to set up some Google Cloud Platform configurations and create your GCP Storage Container before creating the GCP image.
Configure Google Cloud Platform CLI to use your project.
gcloud_project=your-gcloud-project gcloud config set project $gcloud_project
$ gcloud_project=your-gcloud-project $ gcloud config set project $gcloud_project
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an environment variable defining the region where you want to operate.
gcloud_region=us-central1
$ gcloud_region=us-central1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a Google Cloud Platform Storage Container.
gcloud_bucket=name-for-your-bucket gsutil mb -l $gcloud_region gs://$gcloud_bucket
$ gcloud_bucket=name-for-your-bucket $ gsutil mb -l $gcloud_region gs://$gcloud_bucket
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
-
Now that your GCP Storage Container is set up, you need to download the GCP
tar.gz
image from the Red Hat Enterprise Linux AI download page. Unzip the
tar.gz
file with the following command:gunzip <downloaded-gcp-gz-image>
$ gunzip <downloaded-gcp-gz-image>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the name you want to use as the RHEL AI Google Cloud Platform image.
image_name=rhel-ai-1-2
$ image_name=rhel-ai-1-2
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the
tar.gz
file to the Google Cloud Platform Storage Container by running the following command:gsutil cp rhelai_gcp.tar.gz "gs://${gcloud_bucket}/$image_name.tar.gz"
$ gsutil cp rhelai_gcp.tar.gz "gs://${gcloud_bucket}/$image_name.tar.gz"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an Google Cloud Platform image from the
tar.gz
file you just uploaded with the following command:gcloud compute images create \ "$image_name" \ --source-uri="gs://${gcloud_bucket}/$image_name.tar.gz" \ --family "rhel-ai" \ --guest-os-features=GVNIC
$ gcloud compute images create \ "$image_name" \ --source-uri="gs://${gcloud_bucket}/$image_name.tar.gz" \ --family "rhel-ai" \ --guest-os-features=GVNIC
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
5.2. Deploying your instance on Google Cloud Platform using the CLI Copy linkLink copied to clipboard!
You can launch an instance with your new RHEL AI Google Cloud Platform image from the Google Cloud Platform web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch an Google Cloud Platform instance with the custom Google Cloud Platform image
If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites".
Prerequisites
- You created your RHEL AI Google Cloud Platform image. For more information, see "Converting the RHEL AI image to a Google Cloud Platform image".
- You installed the Google Cloud Platform CLI on your specific machine, see Install the Google Cloud Platform CLI on Linux.
Procedure
Log in to your Google Cloud Platform account by running the following command:
gcloud auth login
$ gcloud auth login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Before launching your Google Cloud Platform instance on the CLI, you need to create several configuration variables for your instance.
You need to select the instance profile that you want to use for the deployment. List all the profiles in the desired region by running the following command:
gcloud compute machine-types list --zones=<zone>
$ gcloud compute machine-types list --zones=<zone>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Make a note of your preferred machine type, you will need it for your instance deployment.
You can now start creating your Google Cloud Platform instance. Populate environment variables for when you create the instance.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure the zone to be used.
gcloud config set compute/zone $zone
$ gcloud config set compute/zone $zone
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can now launch your instance, by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
To verify that your Red Hat Enterprise Linux AI tools are installed correctly, run the
ilab
command:ilab
$ ilab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Updating Red Hat Enterprise Linux AI
To update to the latest z-stream version of RHEL AI, follow the procedures in the Updating Red Hat Enterprise Linux AI documentation.
Chapter 6. Installing RHEL AI on Azure Copy linkLink copied to clipboard!
There are multiple ways you can install, and deploy Red Hat Enterprise Linux AI on Azure.
- You can purchase RHEL AI from the Azure marketplace.
- You can download the RHEL AI VHD on the RHEL AI download page and convert it to an Azure image.
For installing and deploying Red Hat Enterprise Linux AI on Azure using the VHD, you must first convert the RHEL AI image into an Azure image. You can then launch an instance using the Azure image and deploy RHEL AI on an Azure machine.
6.1. Converting the RHEL AI image into a Azure image Copy linkLink copied to clipboard!
To create a bootable image on Azure you must configure your Azure account, create an Azure Storage Container, and create an Azure image using the RHEL AI VHD image.
Prerequisites
- You installed the Azure CLI on your specific machine. For more information on installing the Azure CLI, see Install the Azure CLI on Linux.
- You installed the AzCopy on your specific machine. For more information on installing AzCopy, see Install AzCopy on Linux.
Procedure
Log in to Azure by running the following command:
az login
$ az login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output of the login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Log in with the
azcopy
tool using the following commands:keyctl new_session azcopy login
$ keyctl new_session $ azcopy login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to set up various Azure configurations and create your Azure Storage Container before creating the Azure image.
Create an environment variable defining the location of your instance with the following command:
az_location=eastus
$ az_location=eastus
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a resource group and save the name in an environment variable named
az_resource_group
. The following example creates a resource group namedDefault
in the locationeastus
. (You can omit this step if you want to use an already existing resource group).az_resource_group=Default az group create --name ${az_resource_group} --location ${az_location}
$ az_resource_group=Default $ az group create --name ${az_resource_group} --location ${az_location}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an Azure storage account and save the name in an environment variable named
az_storage_account
by running the following commands:az_storage_account=THE_NAME_OF_YOUR_STORAGE_ACCOUNT
$ az_storage_account=THE_NAME_OF_YOUR_STORAGE_ACCOUNT
Copy to Clipboard Copied! Toggle word wrap Toggle overflow az storage account create \ --name ${az_storage_account} \ --resource-group ${az_resource_group} \ --location ${az_location} \ --sku Standard_LRS
$ az storage account create \ --name ${az_storage_account} \ --resource-group ${az_resource_group} \ --location ${az_location} \ --sku Standard_LRS
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create your Azure Storage Container named as the environment variable
az_storage_container
with the following commands:az_storage_container=NAME_OF_MY_BUCKET az storage container create \ --name ${az_storage_container} \ --account-name ${az_storage_account} \ --public-access off
$ az_storage_container=NAME_OF_MY_BUCKET $ az storage container create \ --name ${az_storage_container} \ --account-name ${az_storage_account} \ --public-access off
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can get your Subscription ID from the Azure account list by running the following command:
az account list --output table
$ az account list --output table
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a variable named ` az_subscription_id` with your Subscription ID .
az_subscription_id=46c08fb3-83c5-4b59-8372-bf9caf15a681
$ az_subscription_id=46c08fb3-83c5-4b59-8372-bf9caf15a681
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Grant
azcopy
write permission to user into the storage container. This example grants permission to the useruser@example.com
.az role assignment create \ --assignee user@example.com \ --role "Storage Blob Data Contributor" \ --scope /subscriptions/${az_subscription_id}/resourceGroups/${az_resource_group}/providers/Microsoft.Storage/storageAccounts/${az_storage_account}/blobServices/default/containers/${az_storage_container}
$ az role assignment create \ --assignee user@example.com \ --role "Storage Blob Data Contributor" \ --scope /subscriptions/${az_subscription_id}/resourceGroups/${az_resource_group}/providers/Microsoft.Storage/storageAccounts/${az_storage_account}/blobServices/default/containers/${az_storage_container}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Now that your Azure storage container is set up, you need to download the Azure VHD image from Red Hat Enterprise Linux AI download page.
Unzip the
vhd.gz
file with the following command:gunzip <downloaded-azure-gz-image>
$ gunzip <downloaded-azure-gz-image>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the name you want to use as the RHEL AI Azure image.
image_name=rhel-ai-1.5
$ image_name=rhel-ai-1.5
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the VHD file to the Azure Storage Container by running the following command:
az_vhd_url="https://${az_storage_account}.blob.core.windows.net/${az_storage_container}/$(basename ${vhd_file})" azcopy copy "$vhd_file" "$az_vhd_url"
$ az_vhd_url="https://${az_storage_account}.blob.core.windows.net/${az_storage_container}/$(basename ${vhd_file})" $ azcopy copy "$vhd_file" "$az_vhd_url"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an Azure image from the VHD file you just uploaded with the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
6.2. Deploying your instance on Azure using the CLI Copy linkLink copied to clipboard!
You can launch an instance with your new RHEL AI Azure image from the Azure web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch an Azure instance with the custom Azure image
If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites".
Prerequisites
- You created your RHEL AI Azure image. For more information, see "Converting the RHEL AI image to an Azure image".
- You installed the Azure CLI on your specific machine, see Install the Azure CLI on Linux.
Procedure
Log in to your Azure account by running the following command:
az login
$ az login
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You need to select the instance profile that you want to use for the deployment. List all the profiles in the desired region by running the following command:
az vm list-sizes --location <region> --output table
$ az vm list-sizes --location <region> --output table
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Make a note of your preferred instance profile, you will need it for your instance deployment.
You can now start creating your Azure instance. Populate environment variables for when you create the instance.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can launch your instance, by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
To verify that your Red Hat Enterprise Linux AI tools are installed correctly, run the
ilab
command:ilab
$ ilab
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Updating Red Hat Enterprise Linux AI
To update to the latest z-stream version of RHEL AI, follow the procedures in the Updating Red Hat Enterprise Linux AI documentation.
Chapter 7. Adding additional storage to your instance Copy linkLink copied to clipboard!
Some cloud instances do not have enough storage for the RHEL AI end-to-end workflow in the default disk. You can add a directory that holds additional data.
7.1. Adding a data storage directory to your instance Copy linkLink copied to clipboard!
By default RHEL AI holds configuration data in the $HOME
directory. You can change this default to a different directory for holding InstructLab data.
Prerequisites
- You have a Red Hat Enterprise Linux AI instance
- You added an extra storage disk to your instance
Procedure
You can configure the
ILAB_HOME
environment variable by writing it to the$HOME/.bash_profile
file by running the following commands:echo 'export ILAB_HOME=/mnt' >> $HOME/.bash_profile
$ echo 'export ILAB_HOME=/mnt' >> $HOME/.bash_profile
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can make that change effective by reloading the
$HOME/.bash_profile
file with the following command:source $HOME/.bash_profile
$ source $HOME/.bash_profile
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You then need to create a
containers
directory with the following command:mkdir /mnt/.config/containers
$ mkdir /mnt/.config/containers
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy over the
storage.conf
file to thecontainers
directory with the following command:cp /etc/skel/.config/containers/storage.conf /mnt/.config/containers/
$ cp /etc/skel/.config/containers/storage.conf /mnt/.config/containers/
Copy to Clipboard Copied! Toggle word wrap Toggle overflow