Deploying and managing RHEL on Microsoft Azure

Procedure

1. Import the Microsoft repository key:

   $ sudo rpm --import https://packages.microsoft.com/keys/microsoft-2025.asc

2. Create a packages-microsoft-com-prod repository:

   [azure-cli]
   name=Azure CLI
   baseurl=https://packages.microsoft.com/yumrepos/packages.microsoft.com/rhel/10/prod/
   enabled=1
   gpgcheck=1
   gpgkey=https://packages.microsoft.com/keys/microsoft.asc

3. Install Microsoft Azure CLI. The downloaded version of the Microsoft Azure CLI package can vary depending on the currently available version.

   $ sudo dnf install azure-cli

4. Run Microsoft Azure CLI:

   $ az login

   The terminal shows the following message: Note, we have launched a browser for you to login. For old experience with device code, use az login --use-device-code. Then, the terminal opens the Login from where you can log in.

   Note

   If you are running a remote (SSH) session, the login page link does not open in the browser. In this case, you can copy the link to a browser and log in to authenticate your remote session. To sign in, use a web browser to open the Login page and enter the device code to authenticate.

5. List the keys for the storage account in Microsoft Azure and make note of the value key1 from the output of the previous command.

   $ az storage account keys list --resource-group <resource_group_name> --account-name <account_name>

   Replace resource-group-name with the name of your Microsoft Azure resource group and storage-account-name with the name of your Microsoft Azure storage account.

   1. To list the available resources using the following command:

      $ az resource list

6. Create a storage container:

   $ az storage container create --account-name <storage_account_name> \
   --account-key <key1_value> --name <storage_account_name>

   Replace storage-account-name with the name of the storage account.

Procedure

1. Create a azure.toml blueprint, and add the following information to it:

   provider = "azure"
   
   [settings]
   storageAccount = "<your-storage-account-name>"
   storageAccessKey = "<storage-access-key-you-copied-in-the-Azure-portal>"
   container = "<your-storage-container-name>"

2. Build the image, passing the following arguments:

   $ image-builder build <your-blueprint> vhd <your-image-key> azure.toml

   Replace <your-image-key> with the name of the image that you want.

3. Push the image to Microsoft Azure and create an instance from it:

   $ az storage blob upload --account-name <account-name> --container-name <container-name> --file <image-disk>.vhd --name image-disk.vhd --type page

4. After the upload to the Microsoft Azure Blob storage completes, create a Microsoft Azure image from it. Because the images that you create with RHEL image builder generate hybrid images that support both the V1 = BIOS and V2 = UEFI instance types, you can specify the --hyper-v-generation argument. The default instance type is V1.

   $ az image create --resource-group resource_group_name --name image-disk.vhd --os-type linux --location location \
   --source https://$account_name.blob.core.windows.net/container_name/image-disk.vhd
   - Running

Verification

1. Create an instance either with the Microsoft Azure portal, or a command similar to the following:

   $ az vm create --resource-group resource_group_name --location location --name vm_name --image image-disk.vhd(--admin-username azure-user --generate-ssh-keys*
   - Running

2. Use your private key by using SSH to access the resulting instance. Log in as azure-user. This username was set on the previous step.

Procedure

1. In the RHEL image builder dashboard, select the blueprint you want to use.
2. Click the Images tab.

3. Click Create Image to create your customized .vhd image.

   The Create image wizard opens.

   1. Select Microsoft Azure (.vhd) from the Type drop-down menu list.
   2. Check the Upload to Azure checkbox to upload your image to the Microsoft Azure Cloud.
   3. Enter the Image Size and click Next.

4. On the Upload to Azure page, enter the following information:

   1. On the Authentication page, enter:

      1. Your Storage account name. You can find it on the Storage account page in the Microsoft Azure portal.
      2. Your Storage access key: You can find it on the Access Key Storage page.
      3. Click Next.

   2. On the Authentication page, enter:

      1. The image name.
      2. The Storage container. It is the blob container to which you will upload the image. Find it under the Blob service section, in the Microsoft Azure portal.
      3. Click Next.

5. On the Review page, click Create. The RHEL image builder and upload processes start.

   Access the image you pushed into Microsoft Azure Cloud.

6. Access the Microsoft Azure portal.
7. In the search bar, type "storage account" and click Storage accounts from the list.
8. On the search bar, type "Images" and select the first entry under Services. You are redirected to the Image dashboard.
9. On the navigation panel, click Containers.
10. Find the container you created. Inside the container is the .vhd file you created and pushed by using RHEL image builder.

Verification

1. Verify that you can create a VM image and launch it.

   1. In the search bar, type images account and click Images from the list.
   2. Click +Create.
   3. From the dropdown list, choose the resource group you used earlier.
   4. Enter a name for the image.
   5. For the OS type, select Linux.
   6. For the VM generation, select Gen 2.
   7. Under Storage Blob, click Browse and click through the storage accounts and containers until you reach your VHD file.
   8. Click Select at the end of the page.
   9. Choose an Account Type, for example, Standard SSD.
   10. Click Review + Create and then Create. Wait a few moments for the image creation.

2. To launch the VM, follow the steps:

   1. Click Go to resource.
   2. Click + Create VM from the menu bar on the header.
   3. Enter a name for your virtual machine.
   4. Complete the Size and Administrator account sections.

   5. Click Review + Create and then Create. You can see the deployment progress.

      After the deployment finishes, click the virtual machine name to retrieve the public IP address of the instance to connect by using SSH.

   6. Open a terminal to create an SSH connection to connect to the VM.

Procedure

1. Configure the Red Hat Enterprise Linux (RHEL) VM:

   1. To install from the command line (CLI), ensure that you set the default memory, network interfaces, and CPUs according to your requirement for the VM. For details, see Creating virtual machines by using the command line
   2. To install from the web console, see Creating virtual machines by using the web console

2. When the installation starts:

   1. Create a root password.
   2. Create an administrative user account.
3. After the installation completes, reboot the VM and log in to the root account.
4. After logging in as root, you can configure the image.

5. Register the VM and enable the RHEL repository:

   # subscription-manager register

Procedure

1. Import the Microsoft repository key:

   $ sudo dnf --import https://packages.microsoft.com/keys/microsoft.asc

2. Create a local Azure CLI repository entry:

   $ sudo sh -c 'echo -e "[azure-cli]\nname=Azure CLI\nbaseurl=https://packages.microsoft.com/yumrepos/azure-cli\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/azure-cli.repo'

3. Update the dnf package index:

   $ sudo dnf update

4. Install the Azure CLI:

   $ sudo dnf install -y azure-cli

5. Run the Azure CLI:

   $ az login

Procedure

1. Verify if the system has Hyper-V device drivers:

   # lsinitrd | grep hv
   
   drwxr-xr-x   2 root     root            0 Aug 12 14:21 usr/lib/modules/3.10.0-932.el10.x86_64/kernel/drivers/hv
   -rw-r--r--   1 root     root        31272 Aug 11 08:45 usr/lib/modules/3.10.0-932.el10.x86_64/kernel/drivers/hv/hv_vmbus.ko.xz
   -rw-r--r--   1 root     root        25132 Aug 11 08:46 usr/lib/modules/3.10.0-932.el10.x86_64/kernel/drivers/net/hyperv/hv_netvsc.ko.xz
   -rw-r--r--   1 root     root         9796 Aug 11 08:45 usr/lib/modules/3.10.0-932.el10.x86_64/kernel/drivers/scsi/hv_storvsc.ko.xz

   In case, all the drivers are not installed or even the hv_vmbus driver is listed, complete the remaining steps.

2. Create the hv.conf file in the /etc/dracut.conf.d directory and add the following driver parameters:

   # vi hv.conf
   
   add_drivers+=" hv_vmbus "
   add_drivers+=" hv_netvsc "
   add_drivers+=" hv_storvsc "
   add_drivers+=" nvme "

   Ensure that you have added spaces after and before the double quotes add_drivers+=" hv_vmbus " to load unique drivers, in case other Hyper-V drivers already exist in the environment.

3. Regenerate the initramfs image:

   # dracut -f -v --regenerate-all

Verification

1. Reboot the machine.

2. Verify installation of drivers:

   # lsinitrd | grep hv

Procedure

1. Log in to the VM.

2. Create and edit the /etc/cloud/cloud.cfg.d/00-azure-swap.cfg configuration file and add the following cloud-init configuration to the file:

   # vi /etc/cloud/cloud.cfg.d/00-azure-swap.cfg

   #cloud-config
   disk_setup:
     ephemeral0:
       table_type: gpt
       layout: [66, [33,82]]
       overwrite: true
   fs_setup:
     - device: ephemeral0.1
       filesystem: ext4
     - device: ephemeral0.2
       filesystem: swap
   mounts:
     - ["ephemeral0.1", "/mnt"]
     - ["ephemeral0.2", "none", "swap", "sw,nofail,x-systemd.requires=cloud-init.service", "0", "0"]

   This configuration:

   • Partitions the ephemeral disk (ephemeral0) with a GPT partition table.
   • Creates two partitions: 66% for a file system (mounted at /mnt) and 33% for swap space.
   • Formats the first partition as ext4 and the second partition as swap.

   • Configures automatic mounting of both partitions at boot time.

     Note

     The partition layout [66, [33,82]] allocates 66% of the disk to the first partition and 33% to the second partition. The 82 in the second partition specification indicates a Linux swap partition type. You can adjust these percentages based on your requirements.

3. Verify the configuration file for any errors:

   # cloud-init devel schema --config-file /etc/cloud/cloud.cfg.d/00-azure-swap.cfg

   If the configuration is valid, the command returns no errors.

Procedure

1. Log in and register the VM to enable the Red Hat Enterprise Linux (RHEL) repository:

   # subscription-manager register
   
   Installed Product Current Status:
   Product Name: Red Hat Enterprise Linux for x86_64
   Status: Subscribed

2. Install the cloud-init and hyperv-daemons packages:

   # dnf install cloud-init hyperv-daemons -y

3. Create the cloud-init configuration files and edit them to offer integration with Azure services:

   1. To enable logging to the Hyper-V Data Exchange Service (KVP), edit the /etc/cloud/cloud.cfg.d/10-azure-kvp.cfg file and append the following lines:

      reporting:
          logging:
              type: log
          telemetry:
              type: hyperv

   2. To add the Azure datasource, edit the /etc/cloud/cloud.cfg.d/91-azure_datasource.cfg file and append the following lines:

      datasource_list: [ Azure ]
      datasource:
          Azure:
              apply_network_config: False

   3. To configure swap space on the ephemeral disk, create the /etc/cloud/cloud.cfg.d/00-azure-swap.cfg configuration file and add the following lines to that file.

      Important

      The ephemeral disk is temporary storage. Therefore, data stored on it, including swap space, is lost when the VM is deallocated or moved. Use the ephemeral disk only for temporary data such as swap space.

      #cloud-config
      disk_setup:
        ephemeral0:
          table_type: gpt
          layout: [66, [33,82]]
          overwrite: true
      fs_setup:
        - device: ephemeral0.1
          filesystem: ext4
        - device: ephemeral0.2
          filesystem: swap
      mounts:
        - ["ephemeral0.1", "/mnt"]
        - ["ephemeral0.2", "none", "swap", "sw,nofail,x-systemd.requires=cloud-init.service", "0", "0"]

4. To block automatic loading of specific kernel modules, edit the /etc/modprobe.d/blocklist.conf file and append the following lines:

   blacklist nouveau
   blacklist lbm-nouveau
   blacklist floppy
   blacklist amdgpu
   blacklist skx_edac
   blacklist intel_cstate

5. Modify udev network device rules:

   1. If present, remove the following persistent network device rules:

      # rm -f /etc/udev/rules.d/70-persistent-net.rules
      # rm -f /etc/udev/rules.d/75-persistent-net-generator.rules
      # rm -f /etc/udev/rules.d/80-net-name-slot-rules

   2. To ensure working of accelerated networking on Azure, edit the /etc/udev/rules.d/68-azure-sriov-nm-unmanaged.rules new network device rule and append the following line:

      SUBSYSTEM=="net", DRIVERS=="hv_pci", ACTION=="add", ENV{NM_UNMANAGED}="1"

6. Set the sshd service to start automatically:

   # systemctl enable sshd
   # systemctl is-enabled sshd

7. Modify kernel boot parameters:

   1. Update the GRUB_TIMEOUT parameter value in the /etc/default/grub file:

      GRUB_TIMEOUT=10

   2. Remove the following option from the end of the GRUB_CMDLINE_LINUX line, if present:

      rhgb quiet

   3. Update the /etc/default/grub file with the following configuration details:

      GRUB_CMDLINE_LINUX="loglevel=3 crashkernel=auto console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
      GRUB_TIMEOUT_STYLE=countdown
      GRUB_TERMINAL="serial console"
      GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

      Note

      By adding the elevator=none option to the end of the GRUB_CMDLINE_LINUX line disables the I/O scheduler entirely. This option processes I/O requests according to the order of execution, without optimizing disk performance. With elevator=none on:

      • Hard disk drive (HDD): Performance and throughput decreases, therefore not suitable for running workloads.
      • Solid state drive (SSD): High performance and low latency, therefore suitable for running workloads.

   4. Regenerate the grub.cfg file:

      • On a BIOS-based machine:

        # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline

      • On a UEFI-based machine:

        # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline

        Warning

        The path to rebuild grub.cfg is same for both BIOS and UEFI based machines. Original grub.cfg is present at BIOS path only. The UEFI path has a stub file that must not be modified or recreated using grub2-mkconfig command.

        If your system uses a non-default location for grub.cfg, adjust the command accordingly.

8. Configure the Windows Azure Linux Agent (WALinuxAgent):

   1. Install and enable the WALinuxAgent package:

      # dnf install WALinuxAgent -y
      # systemctl enable waagent

   2. To disable swap configuration in WALinuxAgent (required when using cloud-init to manage swap), edit the following lines in the /etc/waagent.conf file:

      Provisioning.DeleteRootPassword=y
      ResourceDisk.Format=n
      ResourceDisk.EnableSwap=n
      ResourceDisk.SwapSizeMB=0

      Note

      By disabling swap in WALinuxAgent, you enable cloud-init to manage the swap configuration on the ephemeral disk.

9. Prepare the VM for Azure provisioning:

   1. Unregister the VM from Red Hat Subscription Manager:

      # subscription-manager unregister

   2. Clean up the existing provisioning details:

      # waagent -force -deprovision

      Note

      This command generates warnings as Azure automatically handles the VM provisioning.

   3. Clear the shell history and shut down the VM:

      # export HISTSIZE=0
      # poweroff

Procedure

1. Convert the image from qcow2 to raw format.

   $ qemu-img convert -f qcow2 -O raw <image_example_name>.qcow2 <image_name>.raw

2. Edit the align.sh shell script:

   $ vi align.sh
   
   #!/bin/bash
   MB=$((1024 * 1024))
   size=$(qemu-img info -f raw --output json "$1" | gawk 'match($0, /"virtual-size": ([0-9]+),/, val) {print val[1]}')
   rounded_size=$((($size/$MB + 1) * $MB))
   if [ $(($size % $MB)) -eq  0 ]
   then
    echo "Your image is already aligned. You do not need to resize."
    exit 1
   fi
   echo "rounded size = $rounded_size"
   export rounded_size

3. Run the script:

   $ sh align.sh <image_example_name>.raw

4. If the Your image is already aligned. You do not need to resize. message displays:

   1. Convert the file to a fixed VHD format:

      $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image_example_name>.raw <image_example_name>.vhd

      Once converted, the VHD file is ready to upload to Azure.

5. If a value displays mean the raw image is not aligned:

   1. Resize the raw file by using the rounded value as displayed above:

      $ qemu-img resize -f raw <image_example_name>.raw +1G

   2. Convert the raw image file to a VHD format.

      $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image_example_name>.raw <image_example_name>.vhd

      Once converted, the VHD file is ready to upload to Azure.

Procedure

1. Authenticate your host with Azure credentials and log in:

   $ az login

   To login from a browser, open the Azure sign-in page in the browser from the CLI. For details, see Sign in with a browser.

2. Create a resource group in an Azure region:

   $ az group create --name <resource-group> --location <azure-region>

   Example:

   [clouduser@localhost]$ az group create --name azrhelclirsgrp --location southcentralus
   {
     "id": "/subscriptions//resourceGroups/azrhelclirsgrp",
     "location": "southcentralus",
     "managedBy": null,
     "name": "azrhelclirsgrp",
     "properties": {
       "provisioningState": "Succeeded"
     },
     "tags": null
   }

3. Create a storage account with a valid stock keeping units (SKU) types. For details, see SKU Types:

   $ az storage account create -l <azure-region> -n <storage-account-name> -g <resource-group> --sku <sku_type>

   Example:

   $ az storage account create -l southcentralus -n azrhelclistact -g azrhelclirsgrp --sku Standard_LRS
   {
     "accessTier": null,
     "creationTime": "2017-04-05T19:10:29.855470+00:00",
     "customDomain": null,
     "encryption": null,
     "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Storage/storageAccounts/azrhelclistact",
     "kind": "StorageV2",
     "lastGeoFailoverTime": null,
     "location": "southcentralus",
     "name": "azrhelclistact",
     "primaryEndpoints": {
       "blob": "https://azrhelclistact.blob.core.windows.net/",
       "file": "https://azrhelclistact.file.core.windows.net/",
       "queue": "https://azrhelclistact.queue.core.windows.net/",
       "table": "https://azrhelclistact.table.core.windows.net/"
   },
   "primaryLocation": "southcentralus",
   "provisioningState": "Succeeded",
   "resourceGroup": "azrhelclirsgrp",
   "secondaryEndpoints": null,
   "secondaryLocation": null,
   "sku": {
     "name": "Standard_LRS",
     "tier": "Standard"
   },
   "statusOfPrimary": "available",
   "statusOfSecondary": null,
   "tags": {},
     "type": "Microsoft.Storage/storageAccounts"
   }

4. Display the storage account details:

   $ az storage account show-connection-string -n <storage_account_name> -g <resource_group>

   Example:

   $ az storage account show-connection-string -n azrhelclistact -g azrhelclirsgrp
   {
     "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
   }

5. Set the environment variable by exporting the existing connection string to connect system to the storage account:

   $ export AZURE_STORAGE_CONNECTION_STRING="<storage_connection_string>"

   Example:

   $ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="

6. Create a storage container:

   $ az storage container create -n <container_name>

   Example:

   $ az storage container create -n azrhelclistcont
   {
     "created": true
   }

7. Create a virtual network:

   $ az network vnet create -g <resource group> --name <vnet_name> --subnet-name <subnet_name>

   Example:

   $ az network vnet create --resource-group azrhelclirsgrp --name azrhelclivnet1 --subnet-name azrhelclisubnet1
   {
     "newVNet": {
       "addressSpace": {
         "addressPrefixes": [
         "10.0.0.0/16"
         ]
     },
     "dhcpOptions": {
       "dnsServers": []
     },
     "etag": "W/\"\"",
     "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1",
     "location": "southcentralus",
     "name": "azrhelclivnet1",
     "provisioningState": "Succeeded",
     "resourceGroup": "azrhelclirsgrp",
     "resourceGuid": "0f25efee-e2a6-4abe-a4e9-817061ee1e79",
     "subnets": [
       {
         "addressPrefix": "10.0.0.0/24",
         "etag": "W/\"\"",
         "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1/subnets/azrhelclisubnet1",
         "ipConfigurations": null,
         "name": "azrhelclisubnet1",
         "networkSecurityGroup": null,
         "provisioningState": "Succeeded",
         "resourceGroup": "azrhelclirsgrp",
         "resourceNavigationLinks": null,
         "routeTable": null
       }
     ],
     "tags": {},
     "type": "Microsoft.Network/virtualNetworks",
     "virtualNetworkPeerings": null
     }
   }

Procedure

1. Upload the VHD file to the storage container:

   $ az storage blob upload \
       --account-name _<storage_account_name> --container-name _<container_name> \
       --type page --file _<path_to_vhd> --name _<image_name>.vhd

   Example:

   $ az storage blob upload \
   --account-name azrhelclistact --container-name azrhelclistcont \
   --type page --file ~/Downloads/rhel-image-10.vhd --name rhel-image-10.vhd
   
   Percent complete: 100.0%

2. List the storage containers:

   1. To display in the tabular format, enter:

      $ az storage container list --output table

   2. To display in the YAML format, enter:

      $ az storage container list --output yaml

3. Use the URL for the uploaded VHD file from the 1st step:

   $ az storage blob url -c <container_name> -n _<image_name>.vhd _<url_of_vhd_file>_

   Example:

   $ az storage blob url -c azrhelclistcont -n rhel-image-10.vhd "https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-10.vhd"

4. Create the Azure custom image:

   $ az image create -n _<image_name> -g _<resource_group> -l _<azure_region> --source _<URL> --os-type linux

   Note

   The default hypervisor generation of the VM is V1. You can optionally specify a V2 hypervisor generation by including the option --hyper-v-generation V2. Generation 2 VMs use an UEFI-based boot architecture. For details, see Support for generation 2 VMs on Azure. If the command returns the error Only blobs formatted as VHDs can be imported, it means that the image was not aligned to the nearest 1 MB boundary before it was converted to VHD. See Converting a RHEL image to Azure disk iamge.

   Example:

   $ az image create -n rhel10 -g azrhelclirsgrp2 -l southcentralus --source https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-10.vhd --os-type linux

Procedure

1. Create the VM:

   $ az vm create \
       -g <resource_group> -l <azure_region> -n <vm_name> \
       --vnet-name <vnet_name> --subnet <subnet_name> --size Standard_A2 \
       --os-disk-name <simple_name> --admin-username <administrator_name> \
       --generate-ssh-keys --image <path_to_image>

   Example:

   $ az vm create \
   -g azrhelclirsgrp2 -l southcentralus -n rhel-azure-vm-1 \ 

   1

   
   --vnet-name azrhelclivnet1 --subnet azrhelclisubnet1 --size Standard_A2 \
   --os-disk-name vm-1-osdisk --admin-username clouduser \ 

   2

   
   --generate-ssh-keys --image rhel10
   
   {
     "fqdns": "",
     "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Compute/virtualMachines/rhel-azure-vm-1",
     "location": "southcentralus",
     "macAddress": "",
     "powerState": "VM running",
     "privateIpAddress": "10.0.0.4",
     "publicIpAddress": "<public_ip_address>",
     "resourceGroup": "azrhelclirsgrp2"
   }

   • --generate-ssh-keys option creates a private and public key pair files in the ~/.ssh directory.

   • The public key is added to the authorized_keys file on the VM for the user specified by the --admin-username option.

     For details, see Types of SSH authentication methods. Note the publicIpAddress, which you will need to log in to the VM in the following step.

2. Start an SSH session and log in to the Azure VM:

   [clouduser@localhost]$ ssh -i /home/clouduser/.ssh/id_rsa clouduser@<public_ip_address>
   
   The authenticity of host '<public_ip_address>' can't be established.
   Are you sure you want to continue connecting (yes/no)? yes
   Warning: Permanently added '<public_ip_address>' (ECDSA) to the list of known hosts.

Procedure

1. Register your system:

   # subscription-manager register

2. Attach your subscriptions:

   • You can use an activation key to attach subscriptions. See Creating Red Hat Customer Portal Activation Keys for more information.
   • Otherwise, you can manually attach a subscription by using the ID of the subscription pool (Pool ID). See Attaching a host-based subscription to hypervisors.

3. Optional: To collect various system metrics about the instance in the Red Hat Hybrid Cloud Console, you can register the instance with Red Hat Lightspeed.

   # insights-client register --display-name <display_name_value>

   For information about further configuration of Red Hat Lightspeed, see Client Configuration Guide for Red Hat Lightspeed.

Procedure

1. Install kdump and other necessary packages:

   # dnf install kexec-tools kdump-utils makedumpfile

2. Verify that the default location for crash dump files is set in the kdump configuration file and that the /var/crash file is available:

   # grep -v "#" /etc/kdump.conf
   
   path /var/crash
   core_collector makedumpfile -l --message-level 7 -d 31

3. Based on the RHEL VM size and version, check if you need a vmcore target with more free space, such as /mnt/crash:

   Expand

   Table 3.3. Virtual machine sizes that have been tested with GEN2 VM on Azure

   RHEL Version	Standard DS1 v2 (1 vCPU, 3.5GiB)	Standard NV16as v4 (16 vCPUs, 56 GiB)	Standard M416-208s v2 (208 vCPUs, 5700 GiB)	Standard M416ms v2 (416 vCPUs, 11400 GiB)
   RHEL 9.4 - RHEL 10	Default	Default	Target	Target

   Show more
   • Default indicates that kdump works as expected with the default memory and the default kdump target. The default kdump target is the /var/crash file.
   • Target indicates that kdump works as expected with the default memory. However, you might need to assign a target with more free space.

4. To assign a target with free space, such as /mnt/crash, edit the /etc/kdump.conf file and replace the default path:

   $ sed s/"path /var/crash"/"path /mnt/crash"

   The option path /mnt/crash represents the path to the file system where kdump saves the crash dump file.

   For details, such as writing the crash dump file to a different partition, directly to a device or storing it to a remote machine, see Configuring the kdump target.

5. Increase the crash kernel size to the sufficient size for kdump to capture the vmcore by adding the relative boot parameter if the instance required:

   For example, for a Standard M416-208s v2 VM, the sufficient size is 512 MB, so the boot parameter would be crashkernel=512M.

   1. Open the GRUB configuration file and add crashkernel=512M to the boot parameter line:

      # vi /etc/default/grub
      
      GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=512M"

   2. Update the GRUB configuration file:

      # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline

6. Reboot the VM to allocate separate kernel crash memory to the VM.

Procedure

1. To get the public IP address for an Azure VM, open the VM properties in the Azure Portal or enter:

   $ az vm list -g <resource_group> -d --output table

   Example:

   $ az vm list -g azrhelclirsgrp -d --output table
   
   Name    ResourceGroup  PowerState      PublicIps        Location
   ------  ----------------------  --------------  -------------    --------------
   node01 azrhelclirsgrp VM running 192.98.152.251 southcentralus

2. Register the VM with Red Hat:

   $ sudo -i
   # subscription-manager register

3. Disable all repositories:

   # subscription-manager repos --disable= *

4. Enable the RHEL Server HA repositories:

   # subscription-manager repos --enable=rhel-10-for-x86_64-highavailability-rpms

5. Update all packages:

   # dnf update -y

6. Install the Red Hat HA Add-On packages, along with the Azure fencing agent:

   # dnf install pcs pacemaker fence-agents-azure-arm

7. The pcs and pacemaker installation created the hacluster user in the last step. Now, create a password for the hacluster and use it for all cluster nodes:

   # passwd hacluster

8. Add the high availability service to the RHEL Firewall if your system has firewalld.service:

   # firewall-cmd --permanent --add-service=high-availability
   # firewall-cmd --reload

9. Start the pcs service and enable it to start on boot.

   # systemctl start pcsd.service
   # systemctl enable pcsd.service
   
   Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.

Procedure

1. From any node in the HA cluster, log in to your Azure account:

   $ az login

2. Create a json configuration file for a custom role for the Azure fence agent. Use the following configuration, but replace <subscription_id> with your subscription IDs:

   {
         "Name": "Linux Fence Agent Role",
         "description": "Allows to power-off and start virtual machines",
         "assignableScopes": [
                 "/subscriptions/<subscription_id>"
         ],
         "actions": [
                 "Microsoft.Compute/*/read",
                 "Microsoft.Compute/virtualMachines/powerOff/action",
                 "Microsoft.Compute/virtualMachines/start/action"
         ],
         "notActions": [],
         "dataActions": [],
         "notDataActions": []
   }

3. Define the custom role for the Azure fence agent. Use the json file created in the earlier step:

   $ az role definition create --role-definition <azure_fence_role.json>
   
   {
     "assignableScopes": [
       "/subscriptions/<my_subscription_id>"
     ],
     "description": "Allows to power-off and start virtual machines",
     "id": "/subscriptions/<my_subscription_id>/providers/Microsoft.Authorization/roleDefinitions/<role_id>",
     "name": "<role_id>",
     "permissions": [
       {
         "actions": [
           "Microsoft.Compute/*/read",
           "Microsoft.Compute/virtualMachines/powerOff/action",
           "Microsoft.Compute/virtualMachines/start/action"
         ],
         "dataActions": [],
         "notActions": [],
         "notDataActions": []
       }
     ],
     "roleName": "Linux Fence Agent Role",
     "roleType": "CustomRole",
     "type": "Microsoft.Authorization/roleDefinitions"
   }

4. In the Azure web console interface, select Virtual Machine → click Identity in the left-side menu.
5. Select On → click Save → click Yes to confirm.
6. Click Azure role assignments → Add role assignment.
7. Select the Scope required for the role, for example Resource Group.
8. Select the required Resource Group.
9. Optional: Change the Subscription if necessary.
10. Select the Linux Fence Agent Role role.
11. Click Save.

Procedure

1. Authenticate the pcs user hacluster for each node in the cluster on the node from which you will be running pcs.

   The following command authenticates user hacluster on z1.example.com for both of the nodes in a two-node cluster that will consist of z1.example.com and z2.example.com.

   [root@z1 ~]# pcs host auth z1.example.com z2.example.com
   Username: hacluster
   Password:
   z1.example.com: Authorized
   z2.example.com: Authorized

2. Execute the following command from z1.example.com to create the two-node cluster my_cluster that consists of nodes z1.example.com and z2.example.com. This will propagate the cluster configuration files to both nodes in the cluster. This command includes the --start option, which will start the cluster services on both nodes in the cluster.

   [root@z1 ~]# pcs cluster setup my_cluster --start z1.example.com z2.example.com

3. Enable the cluster services to run on each node in the cluster when the node is booted.

   Note

   For your particular environment, you can skip this step by keeping the cluster services disabled. If enabled and a node goes down, any issues with your cluster or your resources are resolved before the node rejoins the cluster. If you keep the cluster services disabled, you need to manually start the services when you reboot a node by executing the pcs cluster start command on that node.

   [root@z1 ~]# pcs cluster enable --all

4. Display the status of the cluster you created with the pcs cluster status command. Because there could be a slight delay before the cluster is up and running when you start the cluster services with the --start option of the pcs cluster setup command, you should ensure that the cluster is up and running before performing any subsequent actions on the cluster and its configuration.

   [root@z1 ~]# pcs cluster status
   Cluster Status:
    Stack: corosync
    Current DC: z2.example.com (version 2.0.0-10.el8-b67d8d0de9) - partition with quorum
    Last updated: Thu Oct 11 16:11:18 2018
    Last change: Thu Oct 11 16:11:00 2018 by hacluster via crmd on z2.example.com
    2 Nodes configured
    0 Resources configured
   
   ...

Procedure

1. Create a Basic load balancer. Select Internal load balancer, the Basic SKU, and Dynamic for the type of IP address assignment.
2. Create a back-end address pool. Associate the backend pool to the availability set created while creating Azure resources in HA. Do not set any target network IP configurations.
3. Create a health probe. For the health probe, select TCP and enter port 61000. You can use TCP port number that does not interfere with another service. For certain HA product applications (for example, SAP HANA and SQL Server), you need to work with Microsoft to identify the correct port to use.
4. Create a load balancer rule. To create the load balancing rule, the default values are provided. Ensure to set Floating IP (direct server return) to Enabled.

Procedure

1. Install the nmap-ncat resource agents on all nodes:

   # dnf install nmap-ncat resource-agents-cloud

2. Perform the following steps on a single node:

   1. Create the pcs resources and group by using the FrontendIP load balancer for the IPaddr2 address:

      # pcs resource create resource-name IPaddr2 ip="10.0.0.7" --group <cluster_resources_group>

   2. Configure the load balancer resource agent:

      # pcs resource create resource-loadbalancer-name azure-lb port=port-number --group <cluster_resources_group>

Procedure

1. Create a shared block volume:

   $ az disk create -g <resource_group> -n <shared_block_volume_name> --size-gb <disk_size> --max-shares <number_vms> -l <location>

   For example, the following command creates a shared block volume named shared-block-volume.vhd in the resource group sharedblock within the Azure Availability Zone westcentralus.

   $ az disk create -g sharedblock-rg -n shared-block-volume.vhd --size-gb 1024 --max-shares 3 -l westcentralus
   
   {
     "creationData": {
       "createOption": "Empty",
       "galleryImageReference": null,
       "imageReference": null,
       "sourceResourceId": null,
       "sourceUniqueId": null,
       "sourceUri": null,
       "storageAccountId": null,
       "uploadSizeBytes": null
     },
     "diskAccessId": null,
     "diskIopsReadOnly": null,
     "diskIopsReadWrite": 5000,
     "diskMbpsReadOnly": null,
     "diskMbpsReadWrite": 200,
     "diskSizeBytes": 1099511627776,
     "diskSizeGb": 1024,
     "diskState": "Unattached",
     "encryption": {
       "diskEncryptionSetId": null,
       "type": "EncryptionAtRestWithPlatformKey"
     },
     "encryptionSettingsCollection": null,
     "hyperVgeneration": "V1",
     "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/disks/shared-block-volume.vhd",
     "location": "westcentralus",
     "managedBy": null,
     "managedByExtended": null,
     "maxShares": 3,
     "name": "shared-block-volume.vhd",
     "networkAccessPolicy": "AllowAll",
     "osType": null,
     "provisioningState": "Succeeded",
     "resourceGroup": "sharedblock-rg",
     "shareInfo": null,
     "sku": {
       "name": "Premium_LRS",
       "tier": "Premium"
     },
     "tags": {},
     "timeCreated": "2020-08-27T15:36:56.263382+00:00",
     "type": "Microsoft.Compute/disks",
     "uniqueId": "cd8b0a25-6fbe-4779-9312-8d9cbb89b6f2",
     "zones": null
   }

2. Verify the shared block volume:

   $ az disk show -g <resource_group> -n <shared_block_volume_name>

   For example, the following command shows details for the shared block volume shared-block-volume.vhd within the resource group sharedblock-rg.

   $ az disk show -g sharedblock-rg -n shared-block-volume.vhd
   
   {
     "creationData": {
       "createOption": "Empty",
       "galleryImageReference": null,
       "imageReference": null,
       "sourceResourceId": null,
       "sourceUniqueId": null,
       "sourceUri": null,
       "storageAccountId": null,
       "uploadSizeBytes": null
     },
     "diskAccessId": null,
     "diskIopsReadOnly": null,
     "diskIopsReadWrite": 5000,
     "diskMbpsReadOnly": null,
     "diskMbpsReadWrite": 200,
     "diskSizeBytes": 1099511627776,
     "diskSizeGb": 1024,
     "diskState": "Unattached",
     "encryption": {
       "diskEncryptionSetId": null,
       "type": "EncryptionAtRestWithPlatformKey"
     },
     "encryptionSettingsCollection": null,
     "hyperVgeneration": "V1",
     "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/disks/shared-block-volume.vhd",
     "location": "westcentralus",
     "managedBy": null,
     "managedByExtended": null,
     "maxShares": 3,
     "name": "shared-block-volume.vhd",
     "networkAccessPolicy": "AllowAll",
     "osType": null,
     "provisioningState": "Succeeded",
     "resourceGroup": "sharedblock-rg",
     "shareInfo": null,
     "sku": {
       "name": "Premium_LRS",
       "tier": "Premium"
     },
     "tags": {},
     "timeCreated": "2020-08-27T15:36:56.263382+00:00",
     "type": "Microsoft.Compute/disks",
     "uniqueId": "cd8b0a25-6fbe-4779-9312-8d9cbb89b6f2",
     "zones": null
   }

3. Create three network interfaces. Run the following command three times by using a different <nic_name> for each node:

   $ az network nic create \
       -g <resource_group> -n <nic_name> --subnet <subnet_name> \
       --vnet-name <virtual_network> --location <location> \
       --network-security-group <network_security_group> --private-ip-address-version IPv4

   For example, the following command creates a network interface with the name shareblock-nodea-vm-nic-protected.

   $ az network nic create \
       -g sharedblock-rg -n sharedblock-nodea-vm-nic-protected --subnet sharedblock-subnet-protected \
       --vnet-name sharedblock-vn --location westcentralus \
       --network-security-group sharedblock-nsg --private-ip-address-version IPv4

4. Create three VMs and attach the shared block volume. Option values are the same for each VM except that each VM has its own <vm_name>, <new_vm_disk_name>, and <nic_name>:

   $ az vm create \
       -n <vm_name> -g <resource_group> --attach-data-disks <shared_block_volume_name> \
       --data-disk-caching None --os-disk-caching ReadWrite --os-disk-name <new_vm_disk_name> \
       --os-disk-size-gb <disk_size> --location <location> --size <virtual_machine_size> \
       --image <image_name> --admin-username <vm_username> --authentication-type ssh \
       --ssh-key-values <ssh_key> --nics <nic_name> --availability-set <availability_set> --ppg <proximity_placement_group>

   For example, the following command creates a VM named sharedblock-nodea-vm:

   $ az vm create \
   -n sharedblock-nodea-vm -g sharedblock-rg --attach-data-disks shared-block-volume.vhd \
   --data-disk-caching None --os-disk-caching ReadWrite --os-disk-name sharedblock-nodea-vm.vhd \
   --os-disk-size-gb 64 --location westcentralus --size Standard_D2s_v3 \
   --image /subscriptions/12345678910-12345678910/resourceGroups/sample-azureimagesgroupwestcentralus/providers/Microsoft.Compute/images/sample-azure-rhel-10.3.0-20200713.n.0.x86_64 --admin-username sharedblock-user --authentication-type ssh \
   --ssh-key-values @sharedblock-key.pub --nics sharedblock-nodea-vm-nic-protected --availability-set sharedblock-as --ppg sharedblock-ppg
   
   {
     "fqdns": "",
     "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/virtualMachines/sharedblock-nodea-vm",
     "location": "westcentralus",
     "macAddress": "00-22-48-5D-EE-FB",
     "powerState": "VM running",
     "privateIpAddress": "198.51.100.3",
     "publicIpAddress": "",
     "resourceGroup": "sharedblock-rg",
     "zones": ""
   }

Verification

1. For each VM in your cluster, verify that the block device is available. To do so, connect to the vm by using the ssh command with your VM’s IP address:

   # ssh <ip_address> "hostname ; lsblk -d | grep ' 1T '"

   For example, the following command lists details including the hostname and block device for the VM IP 198.51.100.3:

   # ssh 198.51.100.3 "hostname ; lsblk -d | grep ' 1T '"
   
   nodea
   sdb    8:16   0    1T  0 disk

2. Use the ssh utility to verify that each VM in your cluster uses the same shared disk:

   # ssh <ip_address> "hostname ; lsblk -d | grep ' 1T ' | awk '{print \$1}' | xargs -i udevadm info --query=all --name=/dev/{} | grep '^E: ID_SERIAL='"

   For example, the following command lists details including the hostname and shared disk volume ID for the instance IP address 198.51.100.3:

   # ssh 198.51.100.3 "hostname ; lsblk -d | grep ' 1T ' | awk '{print \$1}' | xargs -i udevadm info --query=all --name=/dev/{} | grep '^E: ID_SERIAL='"
   
   nodea
   E: ID_SERIAL=3600224808dd8eb102f6ffc5822c41d89

   After you verify that each VM connects to the shared disk, you can configure resilient storage for the cluster.

Procedure

1. Generate a custom certificate custom_db.cer by using the openssl utility:

   $ openssl req -quiet \
   -newkey rsa:4096 \
   -nodes -keyout custom_db.key \
   -new -x509 \
   -sha256 -days 3650 \
   -subj "/CN=Signature Database key/" \
   --outform DER \
   -out custom_db.cer

2. Convert the certificate into base64-encoded format:

   $ echo base64 -w0 custom_db.cer
   
   MIIFIjCCAwqgAwIBAgITNf23J4k0d8c0NR ....

3. Create and edit an azure-example-template.json Azure Resource Manager (ARM) file for registering a new Azure Compute Gallery image version:

   $ vi azure-example-template.json
   
   {
       "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
       "contentVersion": "1.0.0.0",
       "resources": [
           {
               "type": "Microsoft.Compute/galleries/images/versions",
               "apiVersion": "2023-07-03",
               "name": "<your compute gallery/your image definition/version>",
               "location": "<location of the VHD>",
               "properties": {
                   "storageProfile": {
                       "osDiskImage": {
                           "source": {
                               "id": "<your-storage-account-id>",
                               "uri": "<url-with-the-vhd>"
                           },
                           "hostCaching": "ReadOnly"
                       }
                   },
                   "securityProfile": {
                       "uefiSettings": {
                           "signatureTemplateNames": [
                               "MicrosoftUefiCertificateAuthorityTemplate"
                           ],
                           "additionalSignatures": {
                               "db": [
                                   {
                                       "type": "x509",
                                       "value": [
                                           "<base64 of custom_db.cer>"
                                       ]
                                   }
                               ]
                           }
                       }
                   }
               }
           }
       ]
   }

4. Use the azure-cli utility to register the image version:

   $ az deployment group create --name <example_deployment> \
   --resource-group <example_resource_group> \
   --template-file <example_template.json>

5. Reboot the instance from the Azure Portal.

Verification

1. Check if the newly created RHEL instance has Secure Boot enabled:

   $ mokutil --sb-state
   SecureBoot enabled

2. Use the keyctl utility to verify the kernel keyring for the custom certificate:

   $ sudo keyctl list %:.platform
   
   keys in keyring:
   ...
   586621657: ---lswrv 0 0
   asymmetric: Signature Database key: f064979641c24e1b935e402bdbc3d5c4672a1acc
   ...

Procedure

1. Initialization and measurement: A TDX-enabled hypervisor sets the initial state of a VM. This hypervisor loads the firmware binary file into the VM memory and sets the initial register state. The Intel processor measures the initial state of the VM and provides details to verify the initial state of the VM.
2. Firmware: The VM initiates the UEFI firmware. The firmware might include stateful or stateless Virtual Trusted Platform Module (vTPM) implementation. Stateful vTPM maintains persistent cryptographic state across VM reboots and migrations, whereas stateless vTPM generates fresh cryptographic state for each VM session without persistence. Virtual Machine Privilege Levels (VMPL) technology isolates vTPM from the guest. VMPL offers hardware-enforced privilege isolation between different VM components and the hypervisor.
3. vTPM: Depending on your cloud service provider, for stateful vTPM implementation, the UEFI firmware might perform a remote attestation to decrypt the persistent state of vTPM. The vTPM also gathers data about the boot process, such as Secure Boot state, certificates used for signing boot artifacts, or UEFI binary hashes.

4. Shim : When the UEFI firmware finishes the initialization process, it searches for the extended firmware interface (EFI) system partition. Then, the UEFI firmware verifies and executes the first stage boot loader from there. For RHEL, this is shim. The shim program allows non-Microsoft operating systems to load the second stage boot loader from the EFI system partition.

   1. shim uses a Red Hat certificate to verify the second stage boot loader (grub) or Red Hat Unified Kernel Image (UKI).
   2. grub or UKI unpacks, verifies, and executes Linux kernel and initramfs, and the kernel command line. This process ensures that the Linux kernel is loaded in a trusted and secured environment.

5. Initramfs: In initramfs, vTPM information automatically unlocks the encrypted root partition in case of full disk encryption technology.

   1. When the root volume becomes available, initramfs transfers the execution flow there.
6. Attestation: The VM tenant gets access to the system and can perform a remote attestation to ensure that the accessed VM is an untampered Confidential Virtual Machine (CVM). Attestation is performed based on information from the Intel processor and vTPM. This process confirms the authenticity and reliability of the initial CPU and memory state of the RHEL instance and Intel processor.
7. TEE: This process creates a Trusted Execution Environment (TEE) to ensure that booting of the VM is in a trusted and secured environment.

Procedure

1. UEFI Firmware: The boot process starts from the Unified Extensible Firmware Interface (UEFI) firmware. For boot, Red Hat Enterprise Linux (RHEL) UKI requires UEFI firmware, because legacy basic input/output system (BIOS) firmware is not supported.
2. Shim boot loader: Use the shim boot loader for booting rather than directly booting the UKI (a PE binary) from the UEFI firmware. shim includes additional security mechanisms such as Machine Owner Key (MOK) and Secure Boot Advanced Targeting (SBAT).
3. Signature verification (Secure Boot UEFI mechanism): During boot, shim reads the UKI binary and the secure boot UEFI mechanism verifies the signature of UKI against trusted keys stored in the Secure Boot Allowed Signature Database (db) of the system, the MOK database, and the built-in database of the shim binary. If the signature key is valid, the verification passes.

4. SBAT verification: Immediately after signature verification, the shim boot loader verifies the SBAT rules at startup.

   During SBAT verification, the system compares generation numbers, for components such as systemd.rhel or linux.rhel, embedded in the UKI by using the .sbat section against values in the shim boot loader. If the generation number for a component in the shim is higher than the generation number in the UKI, the binary gets automatically discarded, even if it was signed by a trusted key.

   Note that the generation number is a version identifier for UEFI applications, such as shim and grub.

5. Unpacking and Execution: If verification passes, control passes from shim to the systemd-stub code in the UKI to continue the boot process.

6. systemd-stub add-ons: During execution, systemd-stub unpacks and extracts the contents of the .cmdline section, the plain text kernel command line, and the .initrd section, the temporary root file system, for the boot process.

   Note that systemd-stub reads UKI add-ons, which are also PE binaries, and verifies their signatures to safely extend the kernel command line of UKI by appending the .cmdline content from add-ons. systemd-stub reads add-ons from two locations:

   • Global (UKI-independent) add-ons from the /loader/addons/ directory on the Extensible Firmware Interface (EFI) System Partition (ESP).
   • Per-UKI add-ons from the /EFI/Linux/<UKI-name>.extra.d/ directory on the ESP.

7. Control passes from systemd-stub to the Linux kernel and the operating system boot process continues.

   From this point, secure boot with UKI mechanisms follows the standard kernel boot process.