Deploying RHEL 9 on Microsoft Azure


Red Hat Enterprise Linux 9

Obtaining RHEL system images and creating RHEL instances on Azure

Red Hat Customer Content Services

Abstract

To use Red Hat Enterprise Linux (RHEL) in a public cloud environment, you can create and deploy RHEL system images on various cloud platforms, including Microsoft Azure. You can also create and configure a Red Hat High Availability (HA) cluster on Azure.
The following chapters provide instructions for creating cloud RHEL instances and HA clusters on Azure. These processes include installing the required packages and agents, configuring fencing, and installing network resource agents.

Providing feedback on Red Hat documentation

We appreciate your feedback on our documentation. Let us know how we can improve it.

Submitting feedback through Jira (account required)

  1. Log in to the Jira website.
  2. Click Create in the top navigation bar
  3. Enter a descriptive title in the Summary field.
  4. Enter your suggestion for improvement in the Description field. Include links to the relevant parts of the documentation.
  5. Click Create at the bottom of the dialogue.

Public cloud platforms offer computing resources as a service. Instead of using on-premise hardware, you can run your IT workloads, including Red Hat Enterprise Linux (RHEL) systems, as public cloud instances.

1.1. Benefits of using RHEL in a public cloud

Red Hat Enterprise Linux (RHEL) cloud instances on public cloud platforms have these benefits over on-premise RHEL systems or virtual machines (VMs):

Flexible and fine-grained allocation of resources

A RHEL cloud instance runs as a VM on a cloud platform. The platform is a cluster of remote servers that the cloud service provider maintains. You can select hardware resources at the software level. For example, you can select a CPU type or storage setup.

Unlike a local RHEL system, you are not limited by what your physical host can do. Instead, you can select from many features that the cloud provider offers.

Space and cost efficiency

You do not need to own on-premise servers to host cloud workloads. This removes the space, power, and maintenance needs for physical hardware.

On public cloud platforms, you pay the cloud provider for cloud instance usage. Costs depend on the hardware you use and how long you use it. You can control costs to meet your needs.

Software-controlled configurations

You can save a cloud instance configuration as data on the cloud platform and control it with software. With this configuration, you can create, remove, clone, or migrate instances easily. You can also manage a cloud instance remotely through a cloud provider console. The instance connects to remote storage by default.

You can back up a cloud instance as a snapshot at any time. You can then load the snapshot to restore the instance to the saved state.

Separation from the host and software compatibility

Unlike a local VM, a RHEL cloud instance uses Kernel-based Virtual Machine (KVM) virtualization. The guest kernel is separate from the host operating system. It is also separate from the client system you use to connect to the instance.

You can install any operating system on the cloud instance. On a RHEL public cloud instance, you can run RHEL apps you cannot use on your local operating system.

If the instance operating system becomes unstable or compromised, it does not affect your client system.

1.2. Public cloud use cases for RHEL

Deploying applications on a public cloud offers many benefits, but might not be the most efficient solution for every scenario. If you are evaluating the migration of your Red Hat Enterprise Linux (RHEL) deployments to the public cloud, consider whether your use case will benefit from the advantages of the public cloud.

Beneficial use cases

  • Deploying public cloud instances is effective for increasing and decreasing the active computing power of your deployments, also known as scaling up and scaling down. Therefore, consider using RHEL on public cloud for the following scenarios:

    • Clusters with high peak workloads and low general performance requirements. Scaling up and down based on your demands can be efficient in terms of resource costs.
    • Setting up or expanding your clusters to a public cloud to avoid high upfront costs of setting up local servers.
  • Cloud instances are agnostic of the local environment. Therefore, you can use them for backup and disaster recovery.

Potentially problematic use cases

  • You are running an existing environment that is not flexible to migrate to a public cloud. Therefore, customizing a cloud instance to fit the specific needs of an existing deployment might not be suitable for your use case and compared to your current host platform.
  • You are operating on a tight resource budget. Maintaining your deployment in a local data center typically provides less flexibility but more control over the maximum resource costs than the public cloud.

Next steps

Moving your RHEL workloads from a local environment to a public cloud platform might raise concerns about the changes involved. The following are the most commonly asked questions.

Will my RHEL work differently as a cloud instance than as a local virtual machine?

In most respects, RHEL instances on a public cloud platform work the same as RHEL virtual machines on a local host, such as an on-premises server. Notable exceptions include:

  • Instead of private orchestration interfaces, public cloud instances use provider-specific console interfaces for managing your cloud resources.
  • Certain features, such as nested virtualization, may not work correctly. If a specific feature is critical for your deployment, check the feature’s compatibility in advance with your chosen public cloud provider.

Will my data stay safe in a public cloud as opposed to a local server?

The data in your RHEL cloud instances is in your ownership, and your public cloud provider does not have any access to it. In addition, major cloud providers support data encryption in transit, which improves the security of data when migrating your virtual machines to the public cloud.

The general security of your RHEL public cloud instances is managed as follows:

  • Your public cloud provider is responsible for the security of the cloud hypervisor
  • Red Hat provides the security features of the RHEL guest operating systems in your instances
  • You manage the specific security settings and practices in your cloud infrastructure

What effect does my geographic region have on the functionality of RHEL public cloud instances?

You can use RHEL instances on a public cloud platform regardless of your geographical location. Therefore, you can run your instances in the same region as your on-premises server.

However, hosting your instances in a physically distant region might cause high latency when operating them. In addition, depending on the public cloud provider, certain regions may provide additional features or be more cost-efficient. Before creating your RHEL instances, review the properties of the hosting regions available for your chosen cloud provider.

1.4. Obtaining RHEL for public cloud deployments

To deploy a Red Hat Enterprise Linux (RHEL) system in a public cloud environment, you need to:

  1. Select the optimal cloud provider for your use case, based on your requirements and the current offer on the market. The cloud providers currently certified for running RHEL instances are:

  2. Create a RHEL cloud instance on your chosen cloud platform. For more information, see Methods for creating RHEL cloud instances.
  3. To keep your RHEL deployment up-to-date, use Red Hat Update Infrastructure (RHUI).

1.5. Methods for creating RHEL cloud instances

To deploy a RHEL instance on a public cloud platform, you can use one of the following methods:

Expand

Create a system image of RHEL and import it to the cloud platform.

  • To create the system image, you can use the: RHEL image builder or you can build the image manually.
  • This method uses your existing RHEL subscription, and is also referred to as bring your own subscription (BYOS).
  • You pre-pay a yearly subscription, and you can use your Red Hat customer discount.
  • Your customer service is provided by Red Hat.
  • For creating multiple images effectively, you can use the cloud-init tool.

Purchase a RHEL instance directly from the cloud provider marketplace.

  • You post-pay an hourly rate for using the service. Therefore, this method is also referred to as pay as you go (PAYG).
  • Your customer service is provided by the cloud platform provider.

You can create .vhd images by using RHEL image builder that will be automatically uploaded to a Blob Storage of the Microsoft Azure Cloud service provider.

Prerequisites

Procedure

  1. In the RHEL image builder dashboard, select the blueprint you want to use.
  2. Click the Images tab.
  3. Click Create Image to create your customized .vhd image.

    The Create image wizard opens.

    1. Select Microsoft Azure (.vhd) from the Type drop-down menu list.
    2. Check the Upload to Azure checkbox to upload your image to the Microsoft Azure Cloud.
    3. Enter the Image Size and click Next.
  4. On the Upload to Azure page, enter the following information:

    1. On the Authentication page, enter:

      1. Your Storage account name. You can find it on the Storage account page, in the Microsoft Azure portal.
      2. Your Storage access key: You can find it on the Access Key Storage page.
      3. Click Next.
    2. On the Authentication page, enter:

      1. The image name.
      2. The Storage container. It is the blob container to which you will upload the image. Find it under the Blob service section, in the Microsoft Azure portal.
      3. Click Next.
  5. On the Review page, click Create. The RHEL image builder and upload processes start.

    Access the image you pushed into Microsoft Azure Cloud.

  6. Access the Microsoft Azure portal.
  7. In the search bar, type "storage account" and click Storage accounts from the list.
  8. On the search bar, type "Images" and select the first entry under Services. You are redirected to the Image dashboard.
  9. On the navigation panel, click Containers.
  10. Find the container you created. Inside the container is the .vhd file you created and pushed by using RHEL image builder.

Verification

  1. Verify that you can create a VM image and launch it.

    1. In the search bar, type images account and click Images from the list.
    2. Click +Create.
    3. From the dropdown list, choose the resource group you used earlier.
    4. Enter a name for the image.
    5. For the OS type, select Linux.
    6. For the VM generation, select Gen 2.
    7. Under Storage Blob, click Browse and click through the storage accounts and container until you reach your VHD file.
    8. Click Select at the end of the page.
    9. Choose an Account Type, for example, Standard SSD.
    10. Click Review + Create and then Create. Wait a few moments for the image creation.
  2. To launch the VM, follow the steps:

    1. Click Go to resource.
    2. Click + Create VM from the menu bar on the header.
    3. Enter a name for your virtual machine.
    4. Complete the Size and Administrator account sections.
    5. Click Review + Create and then Create. You can see the deployment progress.

      After the deployment finishes, click the virtual machine name to retrieve the public IP address of the instance to connect by using SSH.

    6. Open a terminal to create an SSH connection to connect to the VM.

To deploy a Red Hat Enterprise Linux 9 (RHEL 9) image on Microsoft Azure, follow the information below. This chapter:

  • Discusses your options for choosing an image
  • Lists or refers to system requirements for your host system and virtual machine (VM)
  • Provides procedures for creating a custom VM from an ISO image, uploading it to Azure, and launching an Azure VM instance
Important

You can create a custom VM from an ISO image, but Red Hat recommends that you use the Red Hat Image Builder product to create customized images for use on specific cloud providers. With Image Builder, you can create and upload an Azure Disk Image (VHD format). See Composing a Customized RHEL System Image for more information.

For a list of Red Hat products that you can use securely on Azure, refer to Red Hat on Microsoft Azure.

Prerequisites

The following table lists image choices for Red Hat Enterprise Linux (RHEL) on Microsoft Azure, and notes the differences in the image options.

Expand
Table 3.1. Image options
Image optionSubscriptionsSample scenarioConsiderations

Deploy a Red Hat Gold Image.

Use your existing Red Hat subscriptions.

Select a Red Hat Gold Image on Azure. For details on Gold Images and how to access them on Azure, see the Red Hat Cloud Access Reference Guide.

The subscription includes the Red Hat product cost; you pay Microsoft for all other instance costs.

Deploy a custom image that you move to Azure.

Use your existing Red Hat subscriptions.

Upload your custom image and attach your subscriptions.

The subscription includes the Red Hat product cost; you pay Microsoft for all other instance costs.

Deploy an existing Azure image that includes RHEL.

The Azure images include a Red Hat product.

Choose a RHEL image when you create a VM by using the Azure console, or choose a VM from the Azure Marketplace.

You pay Microsoft hourly on a pay-as-you-go model. These images are on-demand. Azure provides support for on-demand images through a support agreement.

Red Hat provides updates to the images. Azure makes the updates available through the Red Hat Update Infrastructure (RHUI).

3.2. Understanding base images

This section includes information about using preconfigured base images and their configuration settings.

3.2.1. Using a custom base image

To manually configure a virtual machine (VM), first create a base (starter) VM image. Then, you can modify configuration settings and add the packages the VM requires to operate on the cloud. You can make additional configuration changes for your specific application after you upload the image.

To prepare a cloud image of RHEL, follow the instructions in the sections below. To prepare a Hyper-V cloud image of RHEL, see the Prepare a Red Hat-based virtual machine from Hyper-V Manager.

3.2.2. Required system packages

To create and configure a base image of RHEL, your host system must have the following packages installed.

Expand
Table 3.2. System packages
PackageRepositoryDescription

libvirt

rhel-9-for-x86_64-appstream-rpms

Open source API, daemon, and management tool for managing platform virtualization

virt-install

rhel-9-for-x86_64-appstream-rpms

A command-line utility for building VMs

libguestfs

rhel-9-for-x86_64-appstream-rpms

A library for accessing and modifying VM file systems

guestfs-tools

rhel-9-for-x86_64-appstream-rpms

System administration tools for VMs; includes the virt-customize utility

3.2.3. Azure VM configuration settings

Azure virtual machines (VMs) must have the following configuration settings. Some of these settings are enabled during the initial VM creation. Other settings are set when provisioning the VM image for Azure. Keep these settings in mind as you move through the procedures. Refer to them as necessary.

Expand
Table 3.3. VM configuration settings
SettingRecommendation

SSH

SSH must be enabled to provide remote access to your Azure VMs.

dhcp

The primary virtual adapter should be configured for dhcp (IPv4 only).

swap space

Do not create a dedicated swap file or swap partition on the operating system (OS) disk or storage disk during installation. Configure the cloud-init utility to automatically create a swap partition on an ephemeral disk of the VM. Ephemeral disk is a local storage of the VM, while resource disk is mounted storage on VM itself. Both storage types store data temporarily.

NIC

Choose virtio for the primary virtual network adapter.

encryption

For custom images, use Network Bound Disk Encryption (NBDE) for full disk encryption on Azure.

To use swap space for a Red Hat Enterprise Linux (RHEL) virtual machine (VM) on Microsoft Azure, you need to create a swap partition on the ephemeral disk. Only use the ephemeral disk for creating a swap partition, not the operating system (OS) disk or data (storage) disk. Because the ephemeral disk is deleted when the virtual machine is deleted, the swap partition is also removed.

You can use the cloud-init utility to configure a swap partition on the ephemeral disk on-demand. Ephemeral disk is a local storage of the VM, while a resource disk is mounted storage on VM itself. Both storage types store data temporarily. Deleting, moving, stopping, or failure of the VM will result in the loss of the data stored on the ephemeral or resource disk.

Important

Do not use the ephemeral disk for persistent data. All contents, including the swap partition, are deleted when the VM is stopped or moved.

Prerequisites

  • You have installed the cloud-init utility on the VM.
  • You have disabled the swap configuration in the Windows Azure Linux Agent (WALA) by setting the parameters in the /etc/waagent.conf file:

    ResourceDisk.Format=n
    ResourceDisk.EnableSwap=n
    ResourceDisk.SwapSizeMB=0
    Copy to Clipboard Toggle word wrap
  • You have an ephemeral disk available on the VM.

Procedure

  1. Log in to the VM.
  2. Create and edit the /etc/cloud/cloud.cfg.d/00-azure-swap.cfg configuration file and add the following cloud-init configuration to the file:

    # vi /etc/cloud/cloud.cfg.d/00-azure-swap.cfg
    Copy to Clipboard Toggle word wrap
    #cloud-config
    disk_setup:
      ephemeral0:
        table_type: gpt
        layout: [66, [33,82]]
        overwrite: true
    fs_setup:
      - device: ephemeral0.1
        filesystem: ext4
      - device: ephemeral0.2
        filesystem: swap
    mounts:
      - ["ephemeral0.1", "/mnt"]
      - ["ephemeral0.2", "none", "swap", "sw,nofail,x-systemd.requires=cloud-init.service", "0", "0"]
    Copy to Clipboard Toggle word wrap

    This configuration:

    • Partitions the ephemeral disk (ephemeral0) with a GPT partition table.
    • Creates two partitions: 66% for a file system (mounted at /mnt) and 33% for swap space.
    • Formats the first partition as ext4 and the second partition as swap.
    • Configures automatic mounting of both partitions at boot time.

      Note

      The partition layout [66, [33,82]] allocates 66% of the disk to the first partition and 33% to the second partition. The 82 in the second partition specification indicates a Linux swap partition type. You can adjust these percentages based on your requirements.

  3. Verify the configuration file for any errors:

    # cloud-init devel schema --config-file /etc/cloud/cloud.cfg.d/00-azure-swap.cfg
    Copy to Clipboard Toggle word wrap

    If the configuration is valid, the command returns no errors.

Verification

  • After you reboot the VM, check that the swap partition is configured and active by verifying the active swap space, swap usage, and the swap partition entry in the /etc/fstab file.

    • Check active swap space:

      $ swapon -s
      Copy to Clipboard Toggle word wrap

      The output should show the swap partition from ephemeral0.2:

      Filename                 Type        Size      Used    Priority
      /dev/ephemeral0.2     partition     8388604     0      -2
      Copy to Clipboard Toggle word wrap
    • Check swap usage:

      $ free -h
      Copy to Clipboard Toggle word wrap

      The output should show swap space in the Swap row:

               total        used        free      shared      buffered/cache   available
      Mem:     7.8Gi        1.2Gi       5.8Gi        16MiB       800MiB       6.3Gi
      Swap:    8.0Gi        0B          8.0Gi
      Copy to Clipboard Toggle word wrap
    • Verify the swap partition is present in the /etc/fstab file:

      $ grep swap /etc/fstab
      Copy to Clipboard Toggle word wrap

      The output should include an entry for the swap partition, for example:

      /dev/ephemeral0.2   none     swap  sw,nofail,x-systemd.requires=cloud-init.service   0       0
      Copy to Clipboard Toggle word wrap

3.2.5. Creating a base image from an ISO image

The following procedure lists the steps and initial configuration requirements for creating a custom ISO image. Once you have configured the image, you can use the image as a template for creating additional VM instances.

Prerequisites

Procedure

  1. Create and start a basic Red Hat Enterprise Linux (RHEL) VM. For instructions, see Creating virtual machines.

    1. Set the default memory and CPUs to the capacity you need for the VM and the virtual network interface to virtio.

      For example, the following command creates a kvmtest VM by using the rhel-9.0-aarch64-kvm.qcow2 image:

      # virt-install \
          --name kvmtest --memory 2048 --vcpus 2 \
          --disk rhel-9.0-aarch64-kvm.qcow2,bus=virtio \
          --import --os-variant=rhel9.0
      Copy to Clipboard Toggle word wrap
    2. If you use the web console to create your VM, follow the procedure in Creating virtual machines using the web console, with these caveats:

      • Do not check Immediately Start VM.
      • Change your Memory size to your preferred settings.
      • Before you start the installation, ensure that you have changed Model under Virtual Network Interface Settings to virtio and change your vCPUs to the capacity settings you want for the VM.
  2. Review the following additional installation selection and modifications.

    • Select Minimal Install with the standard RHEL option.
    • For Installation Destination, select Custom Storage Configuration. Use the following configuration information to make your selections.

      • Ensure allocation of at least 500 MB and maximum 1 GB or more for /boot.
      • In the filesystem section, use the extended File System (XFS), ext4, or ext3 for both boot and root partitions.
      • During installation, remove swap space from the operating system disk. Use cloud-init on the ephemeral disk after deployment to configure swap space.
    • On the Installation Summary screen, select Network and hostname. Switch Ethernet to ON.
  3. When the installation starts:

    • Create a root password.
    • Create an administrative user account.
  4. After installation is complete, reboot the VM.
  5. Log in to the root account to configure the VM.

To deploy a RHEL 9 virtual machine (VM) with specific settings in Azure, you can create a custom base image for the VM. The following sections describe additional configuration changes that Azure requires.

3.3.1. Installing Hyper-V device drivers

Microsoft Azure provides network and storage device drivers as part of their Linux Integration Services (LIS) for Hyper-V package. You need to install Hyper-V device drivers on the VM image prior to provisioning it as an Azure virtual machine (VM). Use the lsinitrd | grep hv command to verify that the drivers are installed.

Prerequisites

Procedure

  1. Enter the following grep command to determine if the required Hyper-V device drivers are installed.

    # lsinitrd | grep hv
    Copy to Clipboard Toggle word wrap

    In the example below, all required drivers are installed.

    # lsinitrd | grep hv
    drwxr-xr-x   2 root     root            0 Aug 12 14:21 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/hv
    -rw-r--r--   1 root     root        31272 Aug 11 08:45 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/hv/hv_vmbus.ko.xz
    -rw-r--r--   1 root     root        25132 Aug 11 08:46 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/net/hyperv/hv_netvsc.ko.xz
    -rw-r--r--   1 root     root         9796 Aug 11 08:45 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/scsi/hv_storvsc.ko.xz
    Copy to Clipboard Toggle word wrap

    If all the drivers are not installed, complete the remaining steps.

    Note

    An hv_vmbus driver may exist in the environment. Even if this driver is present, complete the following steps.

  2. Create a file named hv.conf in /etc/dracut.conf.d.
  3. Add the following driver parameters to the hv.conf file.

    add_drivers+=" hv_vmbus "
    add_drivers+=" hv_netvsc "
    add_drivers+=" hv_storvsc "
    add_drivers+=" nvme "
    Copy to Clipboard Toggle word wrap
    Note

    Note the spaces before and after the quotes, for example, add_drivers+=" hv_vmbus ". This ensures that unique drivers are loaded in the event that other Hyper-V drivers already exist in the environment.

  4. Regenerate the initramfs image.

    # dracut -f -v --regenerate-all
    Copy to Clipboard Toggle word wrap

Verification

  1. Reboot the machine.
  2. Run the lsinitrd | grep hv command to verify that the drivers are installed.

Before you deploy a custom base image to Azure, perform additional configuration changes to ensure that the virtual machine (VM) can properly operate in Azure.

Prerequisites

Procedure

  1. Log in to the VM.
  2. Register the VM, and enable the Red Hat Enterprise Linux 9 repository.

    # subscription-manager register
    Installed Product Current Status:
    Product Name: Red Hat Enterprise Linux for x86_64
    Status: Subscribed
    Copy to Clipboard Toggle word wrap
  3. Ensure that the cloud-init and hyperv-daemons packages are installed.

    # dnf install cloud-init hyperv-daemons -y
    Copy to Clipboard Toggle word wrap
  4. Create cloud-init configuration files that are needed for integration with Azure services:

    1. To enable logging to the Hyper-V Data Exchange Service (KVP), create the /etc/cloud/cloud.cfg.d/10-azure-kvp.cfg configuration file and add the following lines to that file.

      reporting:
          logging:
              type: log
          telemetry:
              type: hyperv
      Copy to Clipboard Toggle word wrap
    2. To add Azure as a datasource, create the /etc/cloud/cloud.cfg.d/91-azure_datasource.cfg configuration file, and add the following lines to that file.

      datasource_list: [ Azure ]
      datasource:
          Azure:
              apply_network_config: False
      Copy to Clipboard Toggle word wrap
    3. To configure swap space on the ephemeral disk, create the /etc/cloud/cloud.cfg.d/00-azure-swap.cfg configuration file and add the following lines.

      Important

      The ephemeral disk is temporary storage. Therefore, data stored on it, including swap space, is lost when the VM is deallocated or moved. Use the ephemeral disk only for temporary data such as swap space.

      #cloud-config
      disk_setup:
        ephemeral0:
          table_type: gpt
          layout: [66, [33,82]]
          overwrite: true
      fs_setup:
        - device: ephemeral0.1
          filesystem: ext4
        - device: ephemeral0.2
          filesystem: swap
      mounts:
        - ["ephemeral0.1", "/mnt"]
        - ["ephemeral0.2", "none", "swap", "sw,nofail,x-systemd.requires=cloud-init.service", "0", "0"]
      Copy to Clipboard Toggle word wrap
  5. To ensure that specific kernel modules are blocked from loading automatically, edit or create the /etc/modprobe.d/blocklist.conf file and add the following lines to that file.

    blacklist nouveau
    blacklist lbm-nouveau
    blacklist floppy
    blacklist amdgpu
    blacklist skx_edac
    blacklist intel_cstate
    Copy to Clipboard Toggle word wrap
  6. Modify udev network device rules:

    1. Remove the following persistent network device rules if present.

      # rm -f /etc/udev/rules.d/70-persistent-net.rules
      # rm -f /etc/udev/rules.d/75-persistent-net-generator.rules
      # rm -f /etc/udev/rules.d/80-net-name-slot-rules
      Copy to Clipboard Toggle word wrap
    2. To ensure that Accelerated Networking on Azure works as intended, create a new network device rule /etc/udev/rules.d/68-azure-sriov-nm-unmanaged.rules and add the following line to it.

      SUBSYSTEM=="net", DRIVERS=="hv_pci", ACTION=="add", ENV{NM_UNMANAGED}="1"
      Copy to Clipboard Toggle word wrap
  7. Set the sshd service to start automatically.

    # systemctl enable sshd
    # systemctl is-enabled sshd
    Copy to Clipboard Toggle word wrap
  8. Modify kernel boot parameters:

    1. Open the /etc/default/grub file, and ensure the GRUB_TIMEOUT line has the following value.

      GRUB_TIMEOUT=10
      Copy to Clipboard Toggle word wrap
    2. Remove the following options from the end of the GRUB_CMDLINE_LINUX line if present.

      rhgb quiet
      Copy to Clipboard Toggle word wrap
    3. Ensure the /etc/default/grub file contains the following lines with all the specified options.

      GRUB_CMDLINE_LINUX="loglevel=3 crashkernel=auto console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
      GRUB_TIMEOUT_STYLE=countdown
      GRUB_TERMINAL="serial console"
      GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"
      Copy to Clipboard Toggle word wrap
      Note

      If you are not running workloads on HDDs, add elevator=none to the end of the GRUB_CMDLINE_LINUX line. This sets the I/O scheduler to none, which improves I/O performance on SSD-based systems.

    4. Regenerate the grub.cfg file.

      • On a BIOS-based machine:

        • In RHEL 9.2 and earlier:

          # grub2-mkconfig -o /boot/grub2/grub.cfg
          Copy to Clipboard Toggle word wrap
        • In RHEL 9.3 and later:

          # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline
          Copy to Clipboard Toggle word wrap
      • On a UEFI-based machine:

        • In RHEL 9.2 and earlier:

          # grub2-mkconfig -o /boot/grub2/grub.cfg
          Copy to Clipboard Toggle word wrap
        • In RHEL 9.3 and later:

          # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline
          Copy to Clipboard Toggle word wrap
          Warning

          The path to rebuild grub.cfg is same for both BIOS and UEFI based machines. Actual grub.cfg is present at BIOS path only. The UEFI path has a stub file that must not be modified or recreated using grub2-mkconfig command.

          If your system uses a non-default location for grub.cfg, adjust the command accordingly.

  9. Configure the Windows Azure Linux Agent (WALinuxAgent):

    1. Install and enable the WALinuxAgent package.

      # dnf install WALinuxAgent -y
      # systemctl enable waagent
      Copy to Clipboard Toggle word wrap
    2. To disable swap configuration in WALinuxAgent (required when using cloud-init to manage swap), edit the following lines in the /etc/waagent.conf file.

      Provisioning.DeleteRootPassword=y
      ResourceDisk.Format=n
      ResourceDisk.EnableSwap=n
      ResourceDisk.SwapSizeMB=0
      Copy to Clipboard Toggle word wrap
      Note

      By disabling swap in WALinuxAgent, you enable cloud-init to manage the swap configuration on the ephemeral disk.

  10. Prepare the VM for Azure provisioning:

    1. Unregister the VM from Red Hat Subscription Manager.

      # subscription-manager unregister
      Copy to Clipboard Toggle word wrap
    2. Clean up the existing provisioning details.

      # waagent -force -deprovision
      Copy to Clipboard Toggle word wrap
      Note

      This command generates warnings, which are expected because Azure handles the provisioning of VMs automatically.

    3. Clean the shell history and shut down the VM.

      # export HISTSIZE=0
      # poweroff
      Copy to Clipboard Toggle word wrap

3.4. Converting the image to a fixed VHD format

All Microsoft Azure VM images must be in a fixed VHD format. The image must be aligned on a 1 MB boundary before it is converted to VHD. To convert the image from qcow2 to a fixed VHD format and align the image, see the following procedure. Once you have converted the image, you can upload it to Azure.

Prerequisites

Procedure

  1. Convert the image from qcow2 to raw format.

    $ qemu-img convert -f qcow2 -O raw <image-name>.qcow2 <image-name>.raw
    Copy to Clipboard Toggle word wrap
  2. Create a shell script with the following content.

    #!/bin/bash
    MB=$((1024 * 1024))
    size=$(qemu-img info -f raw --output json "$1" | gawk 'match($0, /"virtual-size": ([0-9]+),/, val) {print val[1]}')
    rounded_size=$((($size/$MB + 1) * $MB))
    if [ $(($size % $MB)) -eq  0 ]
    then
     echo "Your image is already aligned. You do not need to resize."
     exit 1
    fi
    echo "rounded size = $rounded_size"
    export rounded_size
    Copy to Clipboard Toggle word wrap
  3. Run the script. This example uses the name align.sh.

    $ sh align.sh <image-xxx>.raw
    Copy to Clipboard Toggle word wrap
    • If the message "Your image is already aligned. You do not need to resize." displays, proceed to the following step.
    • If a value displays, your image is not aligned.
  4. Use the following command to convert the file to a fixed VHD format.

    The sample uses qemu-img version 2.12.0.

    $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image-xxx>.raw <image.xxx>.vhd
    Copy to Clipboard Toggle word wrap

    Once converted, the VHD file is ready to upload to Azure.

  5. If the raw image is not aligned, complete the following steps to align it.

    1. Resize the raw file by using the rounded value displayed when you ran the verification script.

      $ qemu-img resize -f raw <image-xxx>.raw <rounded-value>
      Copy to Clipboard Toggle word wrap
    2. Convert the raw image file to a VHD format.

      The sample uses qemu-img version 2.12.0.

      $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image-xxx>.raw <image.xxx>.vhd
      Copy to Clipboard Toggle word wrap

      Once converted, the VHD file is ready to upload to Azure.

3.5. Installing the Azure CLI

Complete the following steps to install the Azure command-line interface (Azure CLI 2.1). Azure CLI 2.1 is a Python-based utility that creates and manages VMs in Azure.

Prerequisites

  • You need to have an account with Microsoft Azure before you can use the Azure CLI.
  • The Azure CLI installation requires Python 3.x.

Procedure

  1. Import the Microsoft repository key.

    $ sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
    Copy to Clipboard Toggle word wrap
  2. Create a local Azure CLI repository entry.

    $ sudo sh -c 'echo -e "[azure-cli]\nname=Azure CLI\nbaseurl=https://packages.microsoft.com/yumrepos/azure-cli\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/azure-cli.repo'
    Copy to Clipboard Toggle word wrap
  3. Update the dnf package index.

    $ dnf check-update
    Copy to Clipboard Toggle word wrap
  4. Check your Python version (python --version) and install Python 3.x, if necessary.

    $ sudo dnf install python3
    Copy to Clipboard Toggle word wrap
  5. Install the Azure CLI.

    $ sudo dnf install -y azure-cli
    Copy to Clipboard Toggle word wrap
  6. Run the Azure CLI.

    $ az
    Copy to Clipboard Toggle word wrap

3.6. Creating resources in Azure

Complete the following procedure to create the Azure resources that you need before you can upload the VHD file and create the Azure image.

Procedure

  1. Authenticate your system with Azure and log in.

    $ az login
    Copy to Clipboard Toggle word wrap
    Note

    If a browser is available in your environment, the CLI opens your browser to the Azure sign-in page. See Sign in with Azure CLI for more information and options.

  2. Create a resource group in an Azure region.

    $ az group create --name <resource-group> --location <azure-region>
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az group create --name azrhelclirsgrp --location southcentralus
    {
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp",
      "location": "southcentralus",
      "managedBy": null,
      "name": "azrhelclirsgrp",
      "properties": {
        "provisioningState": "Succeeded"
      },
      "tags": null
    }
    Copy to Clipboard Toggle word wrap
  3. Create a storage account. See SKU Types for more information about valid SKU values.

    $ az storage account create -l <azure-region> -n <storage-account-name> -g <resource-group> --sku <sku_type>
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage account create -l southcentralus -n azrhelclistact -g azrhelclirsgrp --sku Standard_LRS
    {
      "accessTier": null,
      "creationTime": "2017-04-05T19:10:29.855470+00:00",
      "customDomain": null,
      "encryption": null,
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Storage/storageAccounts/azrhelclistact",
      "kind": "StorageV2",
      "lastGeoFailoverTime": null,
      "location": "southcentralus",
      "name": "azrhelclistact",
      "primaryEndpoints": {
        "blob": "https://azrhelclistact.blob.core.windows.net/",
        "file": "https://azrhelclistact.file.core.windows.net/",
        "queue": "https://azrhelclistact.queue.core.windows.net/",
        "table": "https://azrhelclistact.table.core.windows.net/"
    },
    "primaryLocation": "southcentralus",
    "provisioningState": "Succeeded",
    "resourceGroup": "azrhelclirsgrp",
    "secondaryEndpoints": null,
    "secondaryLocation": null,
    "sku": {
      "name": "Standard_LRS",
      "tier": "Standard"
    },
    "statusOfPrimary": "available",
    "statusOfSecondary": null,
    "tags": {},
      "type": "Microsoft.Storage/storageAccounts"
    }
    Copy to Clipboard Toggle word wrap
  4. Get the storage account connection string.

    $ az storage account show-connection-string -n <storage-account-name> -g <resource-group>
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage account show-connection-string -n azrhelclistact -g azrhelclirsgrp
    {
      "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
    }
    Copy to Clipboard Toggle word wrap
  5. Export the connection string by copying the connection string and pasting it into the following command. This string connects your system to the storage account.

    $ export AZURE_STORAGE_CONNECTION_STRING="<storage-connection-string>"
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
    Copy to Clipboard Toggle word wrap
  6. Create the storage container.

    $ az storage container create -n <container-name>
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage container create -n azrhelclistcont
    {
      "created": true
    }
    Copy to Clipboard Toggle word wrap
  7. Create a virtual network.

    $ az network vnet create -g <resource group> --name <vnet-name> --subnet-name <subnet-name>
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az network vnet create --resource-group azrhelclirsgrp --name azrhelclivnet1 --subnet-name azrhelclisubnet1
    {
      "newVNet": {
        "addressSpace": {
          "addressPrefixes": [
          "10.0.0.0/16"
          ]
      },
      "dhcpOptions": {
        "dnsServers": []
      },
      "etag": "W/\"\"",
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1",
      "location": "southcentralus",
      "name": "azrhelclivnet1",
      "provisioningState": "Succeeded",
      "resourceGroup": "azrhelclirsgrp",
      "resourceGuid": "0f25efee-e2a6-4abe-a4e9-817061ee1e79",
      "subnets": [
        {
          "addressPrefix": "10.0.0.0/24",
          "etag": "W/\"\"",
          "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1/subnets/azrhelclisubnet1",
          "ipConfigurations": null,
          "name": "azrhelclisubnet1",
          "networkSecurityGroup": null,
          "provisioningState": "Succeeded",
          "resourceGroup": "azrhelclirsgrp",
          "resourceNavigationLinks": null,
          "routeTable": null
        }
      ],
      "tags": {},
      "type": "Microsoft.Network/virtualNetworks",
      "virtualNetworkPeerings": null
      }
    }
    Copy to Clipboard Toggle word wrap

3.7. Uploading and creating an Azure image

To deploy RHEL virtual machine (VM) in Microsoft Azure with your custom configuration, you need to upload a RHEL virtual hard drive (VHD) file to an Azure storage container and create a custom Azure image from that VHD file.

Note

The exported storage connection string does not persist after a system reboot. If any of the commands in the following steps fail, export the connection string again.

Procedure

  1. Upload the VHD file to the storage container. To get a list of storage containers, enter the az storage container list command.

    $ az storage blob upload \
        --account-name <storage_account_name> --container-name <container_name> \
        --type page --file <path_to_vhd> --name <image_name>.vhd
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage blob upload \
    --account-name azrhelclistact --container-name azrhelclistcont \
    --type page --file rhel-image-<ProductNumber>.vhd --name rhel-image-<ProductNumber>.vhd
    
    Percent complete: %100.0
    Copy to Clipboard Toggle word wrap
  2. Get the URL for the uploaded VHD file to use in the following step.

    $ az storage blob url -c <container_name> -n <image_name>.vhd
    Copy to Clipboard Toggle word wrap

    Example:

    $ az storage blob url -c azrhelclistcont -n rhel-image-<ProductNumber>.vhd "https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-<ProductNumber>.vhd"
    Copy to Clipboard Toggle word wrap
  3. Create the Azure custom image.

    $ az image create -n <image_name> -g <resource_group> -l <azure_region> --source <URL> --os-type linux
    Copy to Clipboard Toggle word wrap
    Note

    The default hypervisor generation of the VM is V1. You can optionally specify a V2 hypervisor generation by including the option --hyper-v-generation V2. Generation 2 VMs use a UEFI-based boot architecture. See Support for generation 2 VMs on Azure for information about generation 2 VMs.

    The command might return the error "Only blobs formatted as VHDs can be imported." This error might mean that the image was not aligned to the nearest 1 MB boundary before converted to VHD format.

    Example:

    $ az image create -n rhel<ProductNumber> -g azrhelclirsgrp2 -l southcentralus --source https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-<ProductNumber>.vhd --os-type linux
    Copy to Clipboard Toggle word wrap

3.8. Creating and starting the VM in Azure

To manage your Red Hat Enterprise Linux (RHEL) virtual machine (VM) and resources in Microsoft Azure, you must create a VM from the custom image you created.

For details, see az vm create.

Procedure

  1. Enter the following command to create the VM.

    $ az vm create \
        -g <resource_group> -l <azure_region> -n <vm_name> \
        --vnet-name <vnet_name> --subnet <subnet_name> --size Standard_A2 \
        --os-disk-name <simple_name> --admin-username <administrator_name> \
        --generate-ssh-keys --image <path_to_image>
    Copy to Clipboard Toggle word wrap
    Note

    The option --generate-ssh-keys creates a private/public key pair. Private and public key files are created in ~/.ssh on your system. The public key is added to the authorized_keys file on the VM for the user specified by the --admin-username option. See Other authentication methods for additional information.

    Example:

    [clouduser@localhost]$ az vm create \
    -g azrhelclirsgrp2 -l southcentralus -n rhel-azure-vm-1 \
    --vnet-name azrhelclivnet1 --subnet azrhelclisubnet1 --size Standard_A2 \
    --os-disk-name vm-1-osdisk --admin-username clouduser \
    --generate-ssh-keys --image rhel-image-<example_productnumber>
    
    {
      "fqdns": "",
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Compute/virtualMachines/rhel-azure-vm-1",
      "location": "southcentralus",
      "macAddress": "",
      "powerState": "VM running",
      "privateIpAddress": "10.0.0.4",
      "publicIpAddress": "<public-IP-address>",
      "resourceGroup": "azrhelclirsgrp2"
    Copy to Clipboard Toggle word wrap

    Note the publicIpAddress. You need this address to log in to the VM in the following step.

  2. Start an SSH session and log in to the VM.

    [clouduser@localhost]$ ssh -i /home/clouduser/.ssh/id_rsa clouduser@<public-IP-address>.
    The authenticity of host ',<public-IP-address>' can't be established.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '<public-IP-address>' (ECDSA) to the list of known hosts.
    Copy to Clipboard Toggle word wrap

    If you see a user prompt, you have successfully deployed your Azure virtual machine (VM).

Verification

  • You can now go to the Microsoft Azure portal and check the audit logs and properties of your resources.
  • You can manage your VMs directly in this portal. If you are managing multiple VMs, you should use the Azure CLI. Enter az --help in the CLI or see the Azure CLI command reference to learn more about the commands you use to manage your VMs in Microsoft Azure.

3.9. Other authentication methods

While recommended for increased security, using the Azure-generated key pair is not required. The following examples show two methods for SSH authentication.

Example 1: These command options provision a new VM without generating a public key file. They allow SSH authentication by using a password.

$ az vm create \
    -g <resource-group> -l <azure-region> -n <vm-name> \
    --vnet-name <vnet-name> --subnet <subnet-name> --size Standard_A2 \
    --os-disk-name <simple-name> --authentication-type password \
    --admin-username <administrator-name> --admin-password <ssh-password> --image <path-to-image>
Copy to Clipboard Toggle word wrap
$ ssh <admin-username>@<public-ip-address>
Copy to Clipboard Toggle word wrap

Example 2: These command options provision a new Azure VM and allow SSH authentication by using an existing public key file.

$ az vm create \
    -g <resource-group> -l <azure-region> -n <vm-name> \
    --vnet-name <vnet-name> --subnet <subnet-name> --size Standard_A2 \
    --os-disk-name <simple-name> --admin-username <administrator-name> \
    --ssh-key-value <path-to-existing-ssh-key> --image <path-to-image>
Copy to Clipboard Toggle word wrap
$ ssh -i <path-to-existing-ssh-key> <admin-username>@<public-ip-address>
Copy to Clipboard Toggle word wrap

3.10. Attaching Red Hat subscriptions

Using the subscription-manager command, you can register and attach your Red Hat subscription to a RHEL instance.

Prerequisites

  • You must have enabled your subscriptions.

Procedure

  1. Register your system.

    # subscription-manager register
    Copy to Clipboard Toggle word wrap
  2. Attach your subscriptions.

  3. Optional: To collect various system metrics about the instance in the Red Hat Hybrid Cloud Console, you can register the instance with Red Hat Lightspeed.

    # insights-client register --display-name <display_name_value>
    Copy to Clipboard Toggle word wrap

    For information about further configuration of Red Hat Lightspeed, see Client Configuration Guide for Red Hat Lightspeed.

To deploy Red Hat Enterprise Linux (RHEL) virtual machines (VMs) on Microsoft Azure, you can set up RHEL Gold Images to automatically register with the Red Hat Subscription Manager (RHSM).

Prerequisites

  • You have downloaded the latest RHEL Gold Image for Azure. For instructions, see Using Gold Images on Azure.

    Note

    At a time, you can only attach an Azure account to a single Red Hat account. Therefore, ensure no other users require access to the Azure account before attaching it to your Red Hat account.

Procedure

  1. Upload the Gold Image to Azure. For instructions, see Creating and starting the VM in Azure.
  2. Start the created VM.
  3. For the RHEL VM, enable automatic registration:

    # subscription-manager config --rhsmcertd.auto_registration=1
    Copy to Clipboard Toggle word wrap
  4. Enable the rhsmcertd service:

    # systemctl enable rhsmcertd.service
    Copy to Clipboard Toggle word wrap
  5. Disable the redhat.repo repository:

    # subscription-manager config --rhsm.manage_repos=0
    Copy to Clipboard Toggle word wrap
  6. Power off the VM, and save it as a managed image on Azure. For instructions, see How to create a managed image of a virtual machine or VHD.
  7. Create VMs by using the managed image. They will be automatically subscribed with RHSM.

Verification

  • In a RHEL VM created by using the managed image, verify the system is registered with RHSM by executing the subscription-manager identity command. On a successfully registered system, this displays the UUID of the system. For example:

    # subscription-manager identity
    system identity: fdc46662-c536-43fb-a18a-bbcb283102b7
    name: 192.168.122.222
    org name: 6340056
    org ID: 6340056
    Copy to Clipboard Toggle word wrap

You can configure the kdump service on Microsoft Azure virtual machine (VM) to generate dump files when a kernel crash occurs. These files are known as a crash dump or a vmcore file. If kdump is configured correctly and a kernel instance terminates unexpectedly, you can analyze these files to diagnose the cause of the crash.

For kdump to work on Microsoft Azure VMs, you might need to adjust the kdump reserved memory and the vmcore target to fit VM sizes and Red Hat Enterprise Linux (RHEL) versions. .Prerequisites

  • You are using a Microsoft Azure environment that supports kdump:

    • Standard_DS2_v2 VM
    • Standard NV16as v4
    • Standard M416-208s v2
    • Standard M416ms v2
  • You have root permissions on the system.
  • Your system meets the requirements for kdump configurations and targets. For details, see Supported kdump configurations and targets.

Procedure

  1. Ensure the installation of kdump and other necessary packages:

    # dnf install kexec-tools
    Copy to Clipboard Toggle word wrap
  2. Verify that the default location for crash dump files sets in the kdump configuration file and the /var/crash file is available.

    # grep -v "#" /etc/kdump.conf
    
    path /var/crash
    core_collector makedumpfile -l --message-level 7 -d 31
    Copy to Clipboard Toggle word wrap
  3. Based on the size and version of your RHEL virtual machine (VM) instance, decide whether you need a vmcore target with more free space, such as /mnt/crash. To do so, use the following table.

    Expand
    Table 3.4. Virtual machine sizes that have been tested with GEN2 VM on Azure
    RHEL VersionStandard DS1 v2 (1 vCPU, 3.5GiB)Standard NV16as v4 (16 vCPUs, 56 GiB)Standard M416-208s v2 (208 vCPUs, 5700 GiB)Standard M416ms v2 (416 vCPUs, 11400 GiB)

    RHEL 9.0 - RHEL 9.3

    Default

    Default

    Target

    Target

    • Default indicates that kdump works as expected with the default memory and the default kdump target. The default kdump target is /var/crash.
    • Target indicates that kdump works as expected with the default memory. However, you might need to assign a target with more free space.
  4. If your instance requires it, assign a target with more free space, such as /mnt/crash. To do so, edit the /etc/kdump.conf file and replace the default path.

    $ sed s/"path /var/crash"/"path /mnt/crash"
    Copy to Clipboard Toggle word wrap

    The option path /mnt/crash represents the path to the file system in which kdump saves the crash dump file.

    For more options, such as writing the crash dump file to a different partition, directly to a device or storing it to a remote machine, see Configuring the kdump target.

  5. If your instance requires it, increase the crash kernel size to the required size for kdump to capture the vmcore by adding the required boot parameter.

    For example, for a Standard M416-208s v2 VM, the required size is 512 MB, so the boot parameter would be crashkernel=512M.

    1. Open the GRUB configuration file and add crashkernel=512M to the boot parameter line.

      # vi /etc/default/grub
      
      GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=512M"
      Copy to Clipboard Toggle word wrap
    2. Update the GRUB configuration file.

      • In RHEL 9.2 and earlier:

        # grub2-mkconfig -o /boot/grub2/grub.cfg
        Copy to Clipboard Toggle word wrap
      • In RHEL 9.3 and later:

        # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline
        Copy to Clipboard Toggle word wrap
  6. Reboot the VM to assign separate kernel crash memory to the VM.

Verification

  • Ensure that kdump is active and running.

    # systemctl status kdump
    ● kdump.service - Crash recovery kernel arming
       Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor prese>
       Active: active (exited) since Fri 2024-02-09 10:50:18 CET; 1h 20min ago
      Process: 1252 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCES>
     Main PID: 1252 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 16975)
       Memory: 512B
       CGroup: /system.slice/kdump.service
    Copy to Clipboard Toggle word wrap

To create a cluster where Red Hat Enterprise Linux (RHEL) nodes automatically redistribute their workloads if a node failure occurs, use the Red Hat High Availability Add-On. You can choose public cloud platforms, such as Microsoft Azure, to host high availability (HA) clusters.

Configure a Red Hat HA cluster on Azure by using Azure virtual machine (VM) as cluster nodes. Creating RHEL HA clusters on Azure is similar to creating HA clusters in non-cloud environments with certain specifications.

A high-availability (HA) cluster is a set of computers, also known as nodes, linked together to run a specific workload. The purpose of HA clusters is to offer redundancy in case of a hardware or software failure. If a node in the HA cluster fails, the Pacemaker cluster resource manager distributes the workload to other nodes. No noticeable downtime occurs in the services that are running on the cluster.

You can also run HA clusters on public cloud platforms. In this case, you would use virtual machine (VM) instances in the cloud as the individual cluster nodes. Using HA clusters on a public cloud platform has the following benefits:

  • Improved availability: In case of a VM failure, the workload is quickly redistributed to other nodes, so running services are not disrupted.
  • Scalability: You can start additional nodes when demand is high and stop them when demand is low.
  • Cost-effectiveness: With the pay-as-you-go pricing, you pay only for nodes that are running.
  • Simplified management: Some public cloud platforms offer management interfaces to make configuring HA clusters easier.

To enable HA on your Red Hat Enterprise Linux (RHEL) systems, Red Hat offers a High Availability Add-On. The High Availability Add-On provides all necessary components for creating HA clusters on RHEL systems. The components include high availability service management and cluster administration tools.

4.2. Creating resources in Azure

Complete the following procedure to create a region, resource group, storage account, virtual network, and availability set. You need these resources to set up a cluster on Microsoft Azure.

Prerequisites

Procedure

  1. Authenticate your system with Azure and log in.

    $ az login
    Copy to Clipboard Toggle word wrap
    Note

    If a browser is available in your environment, the CLI opens your browser to the Azure sign-in page.

    Example:

    [clouduser@localhost]$ az login
    
    To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code FDMSCMETZ to authenticate.
      [
        {
          "cloudName": "AzureCloud",
          "id": "Subscription ID",
          "isDefault": true,
          "name": "MySubscriptionName",
          "state": "Enabled",
          "tenantId": "Tenant ID",
          "user": {
            "name": "clouduser@company.com",
            "type": "user"
          }
        }
      ]
    Copy to Clipboard Toggle word wrap
  2. Create a resource group in an Azure region.

    $ az group create --name resource-group --location azure-region
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az group create --name azrhelclirsgrp --location southcentralus
    
    {
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp",
      "location": "southcentralus",
      "managedBy": null,
      "name": "azrhelclirsgrp",
      "properties": {
        "provisioningState": "Succeeded"
      },
      "tags": null
    }
    Copy to Clipboard Toggle word wrap
  3. Create a storage account.

    $ az storage account create -l azure-region -n storage-account-name -g resource-group --sku sku_type --kind StorageV2
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage account create -l southcentralus -n azrhelclistact -g azrhelclirsgrp --sku Standard_LRS --kind StorageV2
    
    {
      "accessTier": null,
      "creationTime": "2017-04-05T19:10:29.855470+00:00",
      "customDomain": null,
      "encryption": null,
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Storage/storageAccounts/azrhelclistact",
      "kind": "StorageV2",
      "lastGeoFailoverTime": null,
      "location": "southcentralus",
      "name": "azrhelclistact",
      "primaryEndpoints": {
        "blob": "https://azrhelclistact.blob.core.windows.net/",
        "file": "https://azrhelclistact.file.core.windows.net/",
        "queue": "https://azrhelclistact.queue.core.windows.net/",
        "table": "https://azrhelclistact.table.core.windows.net/"
    },
    "primaryLocation": "southcentralus",
    "provisioningState": "Succeeded",
    "resourceGroup": "azrhelclirsgrp",
    "secondaryEndpoints": null,
    "secondaryLocation": null,
    "sku": {
      "name": "Standard_LRS",
      "tier": "Standard"
    },
    "statusOfPrimary": "available",
    "statusOfSecondary": null,
    "tags": {},
      "type": "Microsoft.Storage/storageAccounts"
    }
    Copy to Clipboard Toggle word wrap
  4. Get the storage account connection string.

    $ az storage account show-connection-string -n storage-account-name -g resource-group
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage account show-connection-string -n azrhelclistact -g azrhelclirsgrp
    {
      "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
    }
    Copy to Clipboard Toggle word wrap
  5. Export the connection string by copying the connection string and pasting it into the following command. This string connects your system to the storage account.

    $ export AZURE_STORAGE_CONNECTION_STRING="storage-connection-string"
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
    Copy to Clipboard Toggle word wrap
  6. Create the storage container.

    $ az storage container create -n container-name
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage container create -n azrhelclistcont
    
    {
      "created": true
    }
    Copy to Clipboard Toggle word wrap
  7. Create a virtual network. All cluster nodes must be in the same virtual network.

    $ az network vnet create -g resource group --name vnet-name --subnet-name subnet-name
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az network vnet create --resource-group azrhelclirsgrp --name azrhelclivnet1 --subnet-name azrhelclisubnet1
    {
      "newVNet": {
        "addressSpace": {
          "addressPrefixes": [
          "10.0.0.0/16"
          ]
      },
      "dhcpOptions": {
        "dnsServers": []
      },
      "etag": "W/\"\"",
      "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1",
      "location": "southcentralus",
      "name": "azrhelclivnet1",
      "provisioningState": "Succeeded",
      "resourceGroup": "azrhelclirsgrp",
      "resourceGuid": "0f25efee-e2a6-4abe-a4e9-817061ee1e79",
      "subnets": [
        {
          "addressPrefix": "10.0.0.0/24",
          "etag": "W/\"\"",
          "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1/subnets/azrhelclisubnet1",
          "ipConfigurations": null,
          "name": "azrhelclisubnet1",
          "networkSecurityGroup": null,
          "provisioningState": "Succeeded",
          "resourceGroup": "azrhelclirsgrp",
          "resourceNavigationLinks": null,
          "routeTable": null
        }
      ],
      "tags": {},
      "type": "Microsoft.Network/virtualNetworks",
      "virtualNetworkPeerings": null
      }
    }
    Copy to Clipboard Toggle word wrap
  8. Create an availability set. All cluster nodes must be in the same availability set.

    $ az vm availability-set create --name MyAvailabilitySet --resource-group MyResourceGroup
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az vm availability-set create --name rhelha-avset1 --resource-group azrhelclirsgrp
    {
      "additionalProperties": {},
        "id": "/subscriptions/.../resourceGroups/azrhelclirsgrp/providers/Microsoft.Compute/availabilitySets/rhelha-avset1",
        "location": "southcentralus",
        "name": “rhelha-avset1",
        "platformFaultDomainCount": 2,
        "platformUpdateDomainCount": 5,
    
    [omitted]
    Copy to Clipboard Toggle word wrap

The procedure assumes you are creating a VM image for Azure HA that uses Red Hat Enterprise Linux. To successfully complete the procedure, the following packages must be installed.

Expand
Table 4.1. System packages
PackageRepositoryDescription

libvirt

rhel-9-for-x86_64-appstream-rpms

Open source API, daemon, and management tool for managing platform virtualization

virt-install

rhel-9-for-x86_64-appstream-rpms

A command-line utility for building VMs

libguestfs

rhel-9-for-x86_64-appstream-rpms

A library for accessing and modifying VM file systems

guestfs-tools

rhel-9-for-x86_64-appstream-rpms

System administration tools for VMs; includes the virt-customize utility

4.4. Azure VM configuration settings

Azure virtual machines (VMs) must have the following configuration settings. Some of these settings are enabled during the initial VM creation. Other settings are set when provisioning the VM image for Azure. Keep these settings in mind as you move through the procedures. Refer to them as necessary.

Expand
Table 4.2. VM configuration settings
SettingRecommendation

SSH

SSH must be enabled to provide remote access to your Azure VMs.

dhcp

The primary virtual adapter should be configured for dhcp (IPv4 only).

swap space

Do not create a dedicated swap file or swap partition on the operating system (OS) disk or storage disk during installation. Configure the cloud-init utility to automatically create a swap partition on an ephemeral disk of the VM. Ephemeral disk is a local storage of the VM, while resource disk is mounted storage on VM itself. Both storage types store data temporarily.

NIC

Choose virtio for the primary virtual network adapter.

encryption

For custom images, use Network Bound Disk Encryption (NBDE) for full disk encryption on Azure.

4.5. Installing Hyper-V device drivers

Microsoft Azure provides network and storage device drivers as part of their Linux Integration Services (LIS) for Hyper-V package. You need to install Hyper-V device drivers on the VM image prior to provisioning it as an Azure virtual machine (VM). Use the lsinitrd | grep hv command to verify that the drivers are installed.

Prerequisites

Procedure

  1. Enter the following grep command to determine if the required Hyper-V device drivers are installed.

    # lsinitrd | grep hv
    Copy to Clipboard Toggle word wrap

    In the example below, all required drivers are installed.

    # lsinitrd | grep hv
    drwxr-xr-x   2 root     root            0 Aug 12 14:21 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/hv
    -rw-r--r--   1 root     root        31272 Aug 11 08:45 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/hv/hv_vmbus.ko.xz
    -rw-r--r--   1 root     root        25132 Aug 11 08:46 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/net/hyperv/hv_netvsc.ko.xz
    -rw-r--r--   1 root     root         9796 Aug 11 08:45 usr/lib/modules/3.10.0-932.el9.x86_64/kernel/drivers/scsi/hv_storvsc.ko.xz
    Copy to Clipboard Toggle word wrap

    If all the drivers are not installed, complete the remaining steps.

    Note

    An hv_vmbus driver may exist in the environment. Even if this driver is present, complete the following steps.

  2. Create a file named hv.conf in /etc/dracut.conf.d.
  3. Add the following driver parameters to the hv.conf file.

    add_drivers+=" hv_vmbus "
    add_drivers+=" hv_netvsc "
    add_drivers+=" hv_storvsc "
    add_drivers+=" nvme "
    Copy to Clipboard Toggle word wrap
    Note

    Note the spaces before and after the quotes, for example, add_drivers+=" hv_vmbus ". This ensures that unique drivers are loaded in the event that other Hyper-V drivers already exist in the environment.

  4. Regenerate the initramfs image.

    # dracut -f -v --regenerate-all
    Copy to Clipboard Toggle word wrap

Verification

  1. Reboot the machine.
  2. Run the lsinitrd | grep hv command to verify that the drivers are installed.

Before you deploy a custom base image to Azure, perform additional configuration changes to ensure that the virtual machine (VM) can properly operate in Azure.

Prerequisites

Procedure

  1. Log in to the VM.
  2. Register the VM, and enable the Red Hat Enterprise Linux 9 repository.

    # subscription-manager register
    Installed Product Current Status:
    Product Name: Red Hat Enterprise Linux for x86_64
    Status: Subscribed
    Copy to Clipboard Toggle word wrap
  3. Ensure that the cloud-init and hyperv-daemons packages are installed.

    # dnf install cloud-init hyperv-daemons -y
    Copy to Clipboard Toggle word wrap
  4. Create cloud-init configuration files that are needed for integration with Azure services:

    1. To enable logging to the Hyper-V Data Exchange Service (KVP), create the /etc/cloud/cloud.cfg.d/10-azure-kvp.cfg configuration file and add the following lines to that file.

      reporting:
          logging:
              type: log
          telemetry:
              type: hyperv
      Copy to Clipboard Toggle word wrap
    2. To add Azure as a datasource, create the /etc/cloud/cloud.cfg.d/91-azure_datasource.cfg configuration file, and add the following lines to that file.

      datasource_list: [ Azure ]
      datasource:
          Azure:
              apply_network_config: False
      Copy to Clipboard Toggle word wrap
    3. To configure swap space on the ephemeral disk, create the /etc/cloud/cloud.cfg.d/00-azure-swap.cfg configuration file and add the following lines.

      Important

      The ephemeral disk is temporary storage. Therefore, data stored on it, including swap space, is lost when the VM is deallocated or moved. Use the ephemeral disk only for temporary data such as swap space.

      #cloud-config
      disk_setup:
        ephemeral0:
          table_type: gpt
          layout: [66, [33,82]]
          overwrite: true
      fs_setup:
        - device: ephemeral0.1
          filesystem: ext4
        - device: ephemeral0.2
          filesystem: swap
      mounts:
        - ["ephemeral0.1", "/mnt"]
        - ["ephemeral0.2", "none", "swap", "sw,nofail,x-systemd.requires=cloud-init.service", "0", "0"]
      Copy to Clipboard Toggle word wrap
  5. To ensure that specific kernel modules are blocked from loading automatically, edit or create the /etc/modprobe.d/blocklist.conf file and add the following lines to that file.

    blacklist nouveau
    blacklist lbm-nouveau
    blacklist floppy
    blacklist amdgpu
    blacklist skx_edac
    blacklist intel_cstate
    Copy to Clipboard Toggle word wrap
  6. Modify udev network device rules:

    1. Remove the following persistent network device rules if present.

      # rm -f /etc/udev/rules.d/70-persistent-net.rules
      # rm -f /etc/udev/rules.d/75-persistent-net-generator.rules
      # rm -f /etc/udev/rules.d/80-net-name-slot-rules
      Copy to Clipboard Toggle word wrap
    2. To ensure that Accelerated Networking on Azure works as intended, create a new network device rule /etc/udev/rules.d/68-azure-sriov-nm-unmanaged.rules and add the following line to it.

      SUBSYSTEM=="net", DRIVERS=="hv_pci", ACTION=="add", ENV{NM_UNMANAGED}="1"
      Copy to Clipboard Toggle word wrap
  7. Set the sshd service to start automatically.

    # systemctl enable sshd
    # systemctl is-enabled sshd
    Copy to Clipboard Toggle word wrap
  8. Modify kernel boot parameters:

    1. Open the /etc/default/grub file, and ensure the GRUB_TIMEOUT line has the following value.

      GRUB_TIMEOUT=10
      Copy to Clipboard Toggle word wrap
    2. Remove the following options from the end of the GRUB_CMDLINE_LINUX line if present.

      rhgb quiet
      Copy to Clipboard Toggle word wrap
    3. Ensure the /etc/default/grub file contains the following lines with all the specified options.

      GRUB_CMDLINE_LINUX="loglevel=3 crashkernel=auto console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
      GRUB_TIMEOUT_STYLE=countdown
      GRUB_TERMINAL="serial console"
      GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"
      Copy to Clipboard Toggle word wrap
      Note

      If you are not running workloads on HDDs, add elevator=none to the end of the GRUB_CMDLINE_LINUX line. This sets the I/O scheduler to none, which improves I/O performance on SSD-based systems.

    4. Regenerate the grub.cfg file.

      • On a BIOS-based machine:

        • In RHEL 9.2 and earlier:

          # grub2-mkconfig -o /boot/grub2/grub.cfg
          Copy to Clipboard Toggle word wrap
        • In RHEL 9.3 and later:

          # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline
          Copy to Clipboard Toggle word wrap
      • On a UEFI-based machine:

        • In RHEL 9.2 and earlier:

          # grub2-mkconfig -o /boot/grub2/grub.cfg
          Copy to Clipboard Toggle word wrap
        • In RHEL 9.3 and later:

          # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline
          Copy to Clipboard Toggle word wrap
          Warning

          The path to rebuild grub.cfg is same for both BIOS and UEFI based machines. Actual grub.cfg is present at BIOS path only. The UEFI path has a stub file that must not be modified or recreated using grub2-mkconfig command.

          If your system uses a non-default location for grub.cfg, adjust the command accordingly.

  9. Configure the Windows Azure Linux Agent (WALinuxAgent):

    1. Install and enable the WALinuxAgent package.

      # dnf install WALinuxAgent -y
      # systemctl enable waagent
      Copy to Clipboard Toggle word wrap
    2. To disable swap configuration in WALinuxAgent (required when using cloud-init to manage swap), edit the following lines in the /etc/waagent.conf file.

      Provisioning.DeleteRootPassword=y
      ResourceDisk.Format=n
      ResourceDisk.EnableSwap=n
      ResourceDisk.SwapSizeMB=0
      Copy to Clipboard Toggle word wrap
      Note

      By disabling swap in WALinuxAgent, you enable cloud-init to manage the swap configuration on the ephemeral disk.

  10. Prepare the VM for Azure provisioning:

    1. Unregister the VM from Red Hat Subscription Manager.

      # subscription-manager unregister
      Copy to Clipboard Toggle word wrap
    2. Clean up the existing provisioning details.

      # waagent -force -deprovision
      Copy to Clipboard Toggle word wrap
      Note

      This command generates warnings, which are expected because Azure handles the provisioning of VMs automatically.

    3. Clean the shell history and shut down the VM.

      # export HISTSIZE=0
      # poweroff
      Copy to Clipboard Toggle word wrap

Complete the following procedure to create an Azure Active Directory (AD) application. The Azure AD application authorizes and automates access for HA operations for all nodes in the cluster.

Prerequisites

Procedure

  1. On any node in the HA cluster, log in to your Azure account.

    $ az login
    Copy to Clipboard Toggle word wrap
  2. Create a json configuration file for a custom role for the Azure fence agent. Use the following configuration, but replace <subscription_id> with your subscription IDs.

    {
          "Name": "Linux Fence Agent Role",
          "description": "Allows to power-off and start virtual machines",
          "assignableScopes": [
            "/subscriptions/<subscription_id>"
          ],
          "actions": [
            "Microsoft.Compute/*/read",
            "Microsoft.Compute/virtualMachines/powerOff/action",
            "Microsoft.Compute/virtualMachines/start/action"
          ],
          "notActions": [],
          "dataActions": [],
          "notDataActions": []
    }
    Copy to Clipboard Toggle word wrap
  3. Define the custom role for the Azure fence agent. Use the json file created in the earlier step to do this.

    $ az role definition create --role-definition azure-fence-role.json
    
    {
      "assignableScopes": [
        "/subscriptions/<my_subscription_id>"
      ],
      "description": "Allows to power-off and start virtual machines",
      "id": "/subscriptions/<my_subscription_id>/providers/Microsoft.Authorization/roleDefinitions/<role_id>",
      "name": "<role_id>",
      "permissions": [
        {
          "actions": [
            "Microsoft.Compute/*/read",
            "Microsoft.Compute/virtualMachines/powerOff/action",
            "Microsoft.Compute/virtualMachines/start/action"
          ],
          "dataActions": [],
          "notActions": [],
          "notDataActions": []
        }
      ],
      "roleName": "Linux Fence Agent Role",
      "roleType": "CustomRole",
      "type": "Microsoft.Authorization/roleDefinitions"
    }
    Copy to Clipboard Toggle word wrap
  4. In the Azure web console interface, select Virtual Machine → Click Identity in the left-side menu.
  5. Select On → Click Save → click Yes to confirm.
  6. Click Azure role assignmentsAdd role assignment.
  7. Select the Scope required for the role, for example Resource Group.
  8. Select the required Resource Group.
  9. Optional: Change the Subscription if necessary.
  10. Select the Linux Fence Agent Role role.
  11. Click Save.

Verification

  • Display nodes visible to Azure AD.

    # fence_azure_arm --msi -o list
    node1,
    node2,
    [...]
    Copy to Clipboard Toggle word wrap

    If this command outputs all nodes in your cluster, you have configured the AD application successfully.

4.8. Converting the image to a fixed VHD format

All Microsoft Azure VM images must be in a fixed VHD format. The image must be aligned on a 1 MB boundary before it is converted to VHD. To convert the image from qcow2 to a fixed VHD format and align the image, see the following procedure. Once you have converted the image, you can upload it to Azure.

Prerequisites

Procedure

  1. Convert the image from qcow2 to raw format.

    $ qemu-img convert -f qcow2 -O raw <image-name>.qcow2 <image-name>.raw
    Copy to Clipboard Toggle word wrap
  2. Create a shell script with the following content.

    #!/bin/bash
    MB=$((1024 * 1024))
    size=$(qemu-img info -f raw --output json "$1" | gawk 'match($0, /"virtual-size": ([0-9]+),/, val) {print val[1]}')
    rounded_size=$((($size/$MB + 1) * $MB))
    if [ $(($size % $MB)) -eq  0 ]
    then
     echo "Your image is already aligned. You do not need to resize."
     exit 1
    fi
    echo "rounded size = $rounded_size"
    export rounded_size
    Copy to Clipboard Toggle word wrap
  3. Run the script. This example uses the name align.sh.

    $ sh align.sh <image-xxx>.raw
    Copy to Clipboard Toggle word wrap
    • If the message "Your image is already aligned. You do not need to resize." displays, proceed to the following step.
    • If a value displays, your image is not aligned.
  4. Use the following command to convert the file to a fixed VHD format.

    The sample uses qemu-img version 2.12.0.

    $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image-xxx>.raw <image.xxx>.vhd
    Copy to Clipboard Toggle word wrap

    Once converted, the VHD file is ready to upload to Azure.

  5. If the raw image is not aligned, complete the following steps to align it.

    1. Resize the raw file by using the rounded value displayed when you ran the verification script.

      $ qemu-img resize -f raw <image-xxx>.raw <rounded-value>
      Copy to Clipboard Toggle word wrap
    2. Convert the raw image file to a VHD format.

      The sample uses qemu-img version 2.12.0.

      $ qemu-img convert -f raw -o subformat=fixed,force_size -O vpc <image-xxx>.raw <image.xxx>.vhd
      Copy to Clipboard Toggle word wrap

      Once converted, the VHD file is ready to upload to Azure.

4.9. Uploading and creating an Azure image

To deploy RHEL virtual machine (VM) in Microsoft Azure with your custom configuration, you need to upload a RHEL virtual hard drive (VHD) file to an Azure storage container and create a custom Azure image from that VHD file.

Note

The exported storage connection string does not persist after a system reboot. If any of the commands in the following steps fail, export the connection string again.

Procedure

  1. Upload the VHD file to the storage container. To get a list of storage containers, enter the az storage container list command.

    $ az storage blob upload \
        --account-name <storage_account_name> --container-name <container_name> \
        --type page --file <path_to_vhd> --name <image_name>.vhd
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost]$ az storage blob upload \
    --account-name azrhelclistact --container-name azrhelclistcont \
    --type page --file rhel-image-<ProductNumber>.vhd --name rhel-image-<ProductNumber>.vhd
    
    Percent complete: %100.0
    Copy to Clipboard Toggle word wrap
  2. Get the URL for the uploaded VHD file to use in the following step.

    $ az storage blob url -c <container_name> -n <image_name>.vhd
    Copy to Clipboard Toggle word wrap

    Example:

    $ az storage blob url -c azrhelclistcont -n rhel-image-<ProductNumber>.vhd "https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-<ProductNumber>.vhd"
    Copy to Clipboard Toggle word wrap
  3. Create the Azure custom image.

    $ az image create -n <image_name> -g <resource_group> -l <azure_region> --source <URL> --os-type linux
    Copy to Clipboard Toggle word wrap
    Note

    The default hypervisor generation of the VM is V1. You can optionally specify a V2 hypervisor generation by including the option --hyper-v-generation V2. Generation 2 VMs use a UEFI-based boot architecture. See Support for generation 2 VMs on Azure for information about generation 2 VMs.

    The command might return the error "Only blobs formatted as VHDs can be imported." This error might mean that the image was not aligned to the nearest 1 MB boundary before converted to VHD format.

    Example:

    $ az image create -n rhel<ProductNumber> -g azrhelclirsgrp2 -l southcentralus --source https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-<ProductNumber>.vhd --os-type linux
    Copy to Clipboard Toggle word wrap

4.10. Installing Red Hat HA packages and agents

Complete the following steps on all nodes.

Prerequisites

Procedure

  1. Launch an SSH terminal session and connect to the VM by using the administrator name and public IP address.

    $ ssh administrator@PublicIP
    Copy to Clipboard Toggle word wrap

    To get the public IP address for an Azure VM, open the VM properties in the Azure Portal or enter the following Azure CLI command.

    $ az vm list -g <resource_group> -d --output table
    Copy to Clipboard Toggle word wrap

    Example:

    [clouduser@localhost ~] $ az vm list -g azrhelclirsgrp -d --output table
    Name    ResourceGroup           PowerState      PublicIps        Location
    ------  ----------------------  --------------  -------------    --------------
    node01  azrhelclirsgrp          VM running      192.98.152.251    southcentralus
    Copy to Clipboard Toggle word wrap
  2. Register the VM with Red Hat.

    $ sudo -i
    # subscription-manager register
    Copy to Clipboard Toggle word wrap
  3. Disable all repositories.

    # subscription-manager repos --disable=*
    Copy to Clipboard Toggle word wrap
  4. Enable the RHEL 9 Server HA repositories.

    # subscription-manager repos --enable=rhel-9-for-x86_64-highavailability-rpms
    Copy to Clipboard Toggle word wrap
  5. Update all packages.

    # dnf update -y
    Copy to Clipboard Toggle word wrap
  6. Install the Red Hat High Availability Add-On software packages, along with the Azure fencing agent from the High Availability channel.

    # dnf install pcs pacemaker fence-agents-azure-arm
    Copy to Clipboard Toggle word wrap
  7. The user hacluster was created during the pcs and pacemaker installation in the last step. Create a password for hacluster on all cluster nodes. Use the same password for all nodes.

    # passwd hacluster
    Copy to Clipboard Toggle word wrap
  8. Add the high availability service to the RHEL Firewall if firewalld.service is installed.

    # firewall-cmd --permanent --add-service=high-availability
    # firewall-cmd --reload
    Copy to Clipboard Toggle word wrap
  9. Start the pcs service and enable it to start on boot.

    # systemctl start pcsd.service
    # systemctl enable pcsd.service
    
    Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.
    Copy to Clipboard Toggle word wrap

Verification

  • Ensure the pcs service is running.

    # systemctl status pcsd.service
    pcsd.service - PCS GUI and remote configuration interface
    Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)
    Active: active (running) since Fri 2018-02-23 11:00:58 EST; 1min 23s ago
    Docs: man:pcsd(8)
              man:pcs(8)
    Main PID: 46235 (pcsd)
      CGroup: /system.slice/pcsd.service
              └─46235 /usr/bin/ruby /usr/lib/pcsd/pcsd > /dev/null &
    Copy to Clipboard Toggle word wrap

4.11. Creating a cluster

Create a Red Hat High Availability cluster on a public cloud platform by configuring and initializing the cluster nodes.

Procedure

  1. On one of the nodes, enter the following command to authenticate the pcs user hacluster. In the command, specify the name of each node in the cluster.

    # pcs host auth <hostname1> <hostname2> <hostname3>
    Copy to Clipboard Toggle word wrap

    Example:

    [root@node01 clouduser]# pcs host auth node01 node02 node03
    Username: hacluster
    Password:
    node01: Authorized
    node02: Authorized
    node03: Authorized
    Copy to Clipboard Toggle word wrap
  2. Create the cluster.

    # pcs cluster setup <cluster_name> <hostname1> <hostname2> <hostname3>
    Copy to Clipboard Toggle word wrap

    Example:

    [root@node01 clouduser]# pcs cluster setup new_cluster node01 node02 node03
    
    [...]
    
    Synchronizing pcsd certificates on nodes node01, node02, node03...
    node02: Success
    node03: Success
    node01: Success
    Restarting pcsd on the nodes in order to reload the certificates...
    node02: Success
    node03: Success
    node01: Success
    Copy to Clipboard Toggle word wrap

Verification

  1. Enable the cluster.

    [root@node01 clouduser]# pcs cluster enable --all
    node02: Cluster Enabled
    node03: Cluster Enabled
    node01: Cluster Enabled
    Copy to Clipboard Toggle word wrap
  2. Start the cluster.

    [root@node01 clouduser]# pcs cluster start --all
    node02: Starting Cluster...
    node03: Starting Cluster...
    node01: Starting Cluster...
    Copy to Clipboard Toggle word wrap

When a node in the cluster fails to connect to the rest of the cluster, the other nodes must restrict or release access of the failed node to shared resources. This is to ensure that resources should not remain allocated to the failed node.

Though you cannot establish communication with the failed node, as it is unresponsive, you need to fence the failed node so that the data on the failed node remains safe. Use Shoot The Other Note in The Head (STONITH), a fencing mechanism to protect the data on the failed node from getting corrupted by rogue nodes or concurrent access. STONITH ensures that rogue or unresponsive nodes are offline before another node takes over the resources of the failed node.

4.13. Creating a fencing device

Complete the following steps to configure fencing. Complete these commands from any node in the cluster

Prerequisites

Procedure

  1. Identify the Azure node name for each RHEL VM. You use the Azure node names to configure the fence device.

    # fence_azure_arm \
        -l <AD-Application-ID> -p <AD-Password> \
        --resourceGroup <MyResourceGroup> --tenantId <Tenant-ID> \
        --subscriptionId <Subscription-ID> -o list
    Copy to Clipboard Toggle word wrap

    Example:

    [root@node01 clouduser]# fence_azure_arm \
    -l e04a6a49-9f00-xxxx-xxxx-a8bdda4af447 -p z/a05AwCN0IzAjVwXXXXXXXEWIoeVp0xg7QT//JE=
    --resourceGroup azrhelclirsgrp --tenantId 77ecefb6-cff0-XXXX-XXXX-757XXXX9485
    --subscriptionId XXXXXXXX-38b4-4527-XXXX-012d49dfc02c -o list
    
    node01,
    node02,
    node03,
    Copy to Clipboard Toggle word wrap
  2. View the options for the Azure ARM STONITH agent.

    # pcs stonith describe fence_azure_arm
    Copy to Clipboard Toggle word wrap

    Example:

    # pcs stonith describe fence_apc
    Stonith options:
    password: Authentication key
    password_script: Script to run to retrieve password
    Copy to Clipboard Toggle word wrap
    Warning

    For fence agents that offer a method option, do not specify a value of cycle as it is not supported and can cause data corruption.

    Some fence devices can fence only a single node, while other devices can fence many nodes. The parameters you specify when you create a fencing device depend on what your fencing device supports and requires.

    You can use the pcmk_host_list parameter when creating a fencing device to specify all machines that the fencing device controls.

    You can use pcmk_host_map parameter when creating a fencing device to map host names to the specifications that comprehends the fence device.

  3. Create a fence device.

    # pcs stonith create clusterfence fence_azure_arm
    Copy to Clipboard Toggle word wrap
  4. To ensure immediate and complete fencing, disable ACPI Soft-Off on all cluster nodes. For information about disabling ACPI Soft-Off, see Disabling ACPI for use with integrated fence device.

Verification

  1. Test the fencing agent for one of the other nodes:

    # pcs stonith fence azurenodename
    Copy to Clipboard Toggle word wrap

    Example:

    [root@node01 clouduser]# pcs status
    Cluster name: newcluster
    Stack: corosync
    Current DC: node01 (version 1.1.18-11.el7-2b07d5c5a9) - partition with quorum
    Last updated: Fri Feb 23 11:44:35 2018
    Last change: Fri Feb 23 11:21:01 2018 by root via cibadmin on node01
    
    3 nodes configured
    1 resource configured
    
    Online: [ node01 node03 ]
    OFFLINE: [ node02 ]
    
    Full list of resources:
    
      clusterfence  (stonith:fence_azure_arm):  Started node01
    
    Daemon Status:
      corosync: active/disabled
      pacemaker: active/disabled
      pcsd: active/enabled
    Copy to Clipboard Toggle word wrap
  2. Start the node already fenced in the earlier step:

    # pcs cluster start <hostname>
    Copy to Clipboard Toggle word wrap
  3. Check the status to verify the node started:

    # pcs status
    Copy to Clipboard Toggle word wrap

    Example:

    [root@node01 clouduser]# pcs status
    Cluster name: newcluster
    Stack: corosync
    Current DC: node01 (version 1.1.18-11.el7-2b07d5c5a9) - partition with quorum
    Last updated: Fri Feb 23 11:34:59 2018
    Last change: Fri Feb 23 11:21:01 2018 by root via cibadmin on node01
    
    3 nodes configured
    1 resource configured
    
    Online: [ node01 node02 node03 ]
    
    Full list of resources:
    
    clusterfence    (stonith:fence_azure_arm):  Started node01
    
    Daemon Status:
      corosync: active/disabled
      pacemaker: active/disabled
      pcsd: active/enabled
    Copy to Clipboard Toggle word wrap

4.14. Creating an Azure internal load balancer

To remove cluster nodes that do not respond to health probe requests, create an Azure internal load balancer.

Prerequisites

Procedure

  1. Create a Basic load balancer. Select Internal load balancer, the Basic SKU, and Dynamic for the type of IP address assignment.
  2. Create a back-end address pool. Associate the backend pool to the availability set created while creating Azure resources in HA. Do not set any target network IP configurations.
  3. Create a health probe. For the health probe, select TCP and enter port 61000. You can use TCP port number that does not interfere with another service. For certain HA product applications (for example, SAP HANA and SQL Server), you may need to work with Microsoft to identify the correct port to use.
  4. Create a load balancer rule. To create the load balancing rule, the default values are prepopulated. Ensure to set Floating IP (direct server return) to Enabled.

4.15. Configuring the load balancer resource agent

To ensure that the resource agent based service answers health probe requests from the Azure load balancer and removes cluster nodes that do not answer requests, configure the load balancer resource agent after creating a health probe.

Prerequisites

Procedure

  1. Install the nmap-ncat resource agents on all nodes.

    # dnf install nmap-ncat resource-agents-cloud
    Copy to Clipboard Toggle word wrap

    Perform the following steps on a single node.

  2. Create the pcs resources and group. Use your load balancer FrontendIP for the IPaddr2 address.

    # pcs resource create resource-name IPaddr2 ip="10.0.0.7" --group cluster-resources-group
    Copy to Clipboard Toggle word wrap
  3. Configure the load balancer resource agent.

    # pcs resource create resource-loadbalancer-name azure-lb port=port-number --group cluster-resources-group
    Copy to Clipboard Toggle word wrap

Verification

  • Run pcs status to see the results.

    [root@node01 clouduser]# pcs status
    Copy to Clipboard Toggle word wrap

    Example output:

    Cluster name: clusterfence01
    Stack: corosync
    Current DC: node02 (version 1.1.16-12.el7_4.7-94ff4df) - partition with quorum
    Last updated: Tue Jan 30 12:42:35 2018
    Last change: Tue Jan 30 12:26:42 2018 by root via cibadmin on node01
    
    3 nodes configured
    3 resources configured
    
    Online: [ node01 node02 node03 ]
    
    Full list of resources:
    
    clusterfence (stonith:fence_azure_arm):      Started node01
    Resource Group: g_azure
        vip_azure  (ocf::heartbeat:IPaddr2):       Started node02
        lb_azure   (ocf::heartbeat:azure-lb):      Started node02
    
    Daemon Status:
      corosync: active/disabled
      pacemaker: active/disabled
      pcsd: active/enabled
    Copy to Clipboard Toggle word wrap

4.16. Configuring shared block storage

To ensure data consistency and failover capabilities in a Red Hat High Availability cluster deployed in Microsoft Azure, you can use Azure shared block storage.

Use Azure shared block storage to enable multiple virtual machines in a cluster to simultaneously access the same storage volume, providing data consistency and failover capabilities for high-availability deployments.

Prerequisites

Procedure

  1. Create a shared block volume by using the Azure command az disk create.

    $ az disk create -g <resource_group> -n <shared_block_volume_name> --size-gb <disk_size> --max-shares <number_vms> -l <location>
    Copy to Clipboard Toggle word wrap

    For example, the following command creates a shared block volume named shared-block-volume.vhd in the resource group sharedblock within the Azure Availability Zone westcentralus.

    $ az disk create -g sharedblock-rg -n shared-block-volume.vhd --size-gb 1024 --max-shares 3 -l westcentralus
    
    {
      "creationData": {
        "createOption": "Empty",
        "galleryImageReference": null,
        "imageReference": null,
        "sourceResourceId": null,
        "sourceUniqueId": null,
        "sourceUri": null,
        "storageAccountId": null,
        "uploadSizeBytes": null
      },
      "diskAccessId": null,
      "diskIopsReadOnly": null,
      "diskIopsReadWrite": 5000,
      "diskMbpsReadOnly": null,
      "diskMbpsReadWrite": 200,
      "diskSizeBytes": 1099511627776,
      "diskSizeGb": 1024,
      "diskState": "Unattached",
      "encryption": {
        "diskEncryptionSetId": null,
        "type": "EncryptionAtRestWithPlatformKey"
      },
      "encryptionSettingsCollection": null,
      "hyperVgeneration": "V1",
      "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/disks/shared-block-volume.vhd",
      "location": "westcentralus",
      "managedBy": null,
      "managedByExtended": null,
      "maxShares": 3,
      "name": "shared-block-volume.vhd",
      "networkAccessPolicy": "AllowAll",
      "osType": null,
      "provisioningState": "Succeeded",
      "resourceGroup": "sharedblock-rg",
      "shareInfo": null,
      "sku": {
        "name": "Premium_LRS",
        "tier": "Premium"
      },
      "tags": {},
      "timeCreated": "2020-08-27T15:36:56.263382+00:00",
      "type": "Microsoft.Compute/disks",
      "uniqueId": "cd8b0a25-6fbe-4779-9312-8d9cbb89b6f2",
      "zones": null
    }
    Copy to Clipboard Toggle word wrap
  2. Verify that you have created the shared block volume by using the Azure command az disk show.

    $ az disk show -g <resource_group> -n <shared_block_volume_name>
    Copy to Clipboard Toggle word wrap

    For example, the following command shows details for the shared block volume shared-block-volume.vhd within the resource group sharedblock-rg.

    $ az disk show -g sharedblock-rg -n shared-block-volume.vhd
    
    {
      "creationData": {
        "createOption": "Empty",
        "galleryImageReference": null,
        "imageReference": null,
        "sourceResourceId": null,
        "sourceUniqueId": null,
        "sourceUri": null,
        "storageAccountId": null,
        "uploadSizeBytes": null
      },
      "diskAccessId": null,
      "diskIopsReadOnly": null,
      "diskIopsReadWrite": 5000,
      "diskMbpsReadOnly": null,
      "diskMbpsReadWrite": 200,
      "diskSizeBytes": 1099511627776,
      "diskSizeGb": 1024,
      "diskState": "Unattached",
      "encryption": {
        "diskEncryptionSetId": null,
        "type": "EncryptionAtRestWithPlatformKey"
      },
      "encryptionSettingsCollection": null,
      "hyperVgeneration": "V1",
      "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/disks/shared-block-volume.vhd",
      "location": "westcentralus",
      "managedBy": null,
      "managedByExtended": null,
      "maxShares": 3,
      "name": "shared-block-volume.vhd",
      "networkAccessPolicy": "AllowAll",
      "osType": null,
      "provisioningState": "Succeeded",
      "resourceGroup": "sharedblock-rg",
      "shareInfo": null,
      "sku": {
        "name": "Premium_LRS",
        "tier": "Premium"
      },
      "tags": {},
      "timeCreated": "2020-08-27T15:36:56.263382+00:00",
      "type": "Microsoft.Compute/disks",
      "uniqueId": "cd8b0a25-6fbe-4779-9312-8d9cbb89b6f2",
      "zones": null
    }
    Copy to Clipboard Toggle word wrap
  3. Create three network interfaces by using the Azure command az network nic create. Run the following command three times by using a different <nic_name> for each.

    $ az network nic create \
    -g <resource_group> -n <nic_name> --subnet <subnet_name> \
    --vnet-name <virtual_network> --location <location> \
    --network-security-group <network_security_group> --private-ip-address-version IPv4
    Copy to Clipboard Toggle word wrap

    For example, the following command creates a network interface with the name sharedblock-nodea-vm-nic-protected.

    $ az network nic create \
    -g sharedblock-rg -n sharedblock-nodea-vm-nic-protected --subnet sharedblock-subnet-protected \
    --vnet-name sharedblock-vn --location westcentralus \
    --network-security-group sharedblock-nsg --private-ip-address-version IPv4
    Copy to Clipboard Toggle word wrap
  4. Create three VMs and attach the shared block volume by using the Azure command az vm create. Option values are the same for each VM except that each VM has its own <vm_name>, <new_vm_disk_name>, and <nic_name>.

    $ az vm create \
    -n <vm_name> -g <resource_group> --attach-data-disks <shared_block_volume_name> \
    --data-disk-caching None --os-disk-caching ReadWrite --os-disk-name <example_vm_disk_name> \
    --os-disk-size-gb <disk_size> --location <location> --size <virtual_machine_size> \
    --image <image_name> --admin-username <vm_username> --authentication-type ssh \
    --ssh-key-values <ssh_key> --nics <nic_name> --availability-set <availability_set> --ppg <proximity_placement_group>
    Copy to Clipboard Toggle word wrap

    For example, the following command creates a VM named sharedblock-nodea-vm.

    $ az vm create \
    -n sharedblock-nodea-vm -g sharedblock-rg --attach-data-disks shared-block-volume.vhd \
    --data-disk-caching None --os-disk-caching ReadWrite --os-disk-name sharedblock-nodea-vm.vhd \
    --os-disk-size-gb 64 --location westcentralus --size Standard_D2s_v3 \
    --image /subscriptions/12345678910-12345678910/resourceGroups/sample-azureimagesgroupwestcentralus/providers/Microsoft.Compute/images/sample-azure-rhel-9.3.0-20200713.n.0.x86_64 --admin-username sharedblock-user --authentication-type ssh \
    --ssh-key-values @sharedblock-key.pub --nics sharedblock-nodea-vm-nic-protected --availability-set sharedblock-as --ppg sharedblock-ppg
    
    {
      "fqdns": "",
      "id": "/subscriptions/12345678910-12345678910/resourceGroups/sharedblock-rg/providers/Microsoft.Compute/virtualMachines/sharedblock-nodea-vm",
      "location": "westcentralus",
      "macAddress": "00-22-48-5D-EE-FB",
      "powerState": "VM running",
      "privateIpAddress": "198.51.100.3",
      "publicIpAddress": "",
      "resourceGroup": "sharedblock-rg",
      "zones": ""
    }
    Copy to Clipboard Toggle word wrap

Verification

  1. For each VM in your cluster, verify that the block device is available by using the ssh command with your IP address of VM.

    # ssh <ip_address> "hostname ; lsblk -d | grep ' 1T '"
    Copy to Clipboard Toggle word wrap

    For example, the following command lists details including the hostname and block device for the VM IP 198.51.100.3.

    # ssh 198.51.100.3 "hostname ; lsblk -d | grep ' 1T '"
    
    nodea
    sdb    8:16   0    1T  0 disk
    Copy to Clipboard Toggle word wrap
  2. Use the ssh command to verify that each VM in your cluster uses the same shared disk.

    # ssh <ip_address> "hostname ; lsblk -d | grep ' 1T ' | awk '{print \$1}' | xargs -i udevadm info --query=all --name=/dev/{} | grep '^E: ID_SERIAL='"
    Copy to Clipboard Toggle word wrap

    For example, the following command lists details including the hostname and shared disk volume ID for the instance IP address 198.51.100.3.

    # ssh 198.51.100.3 "hostname ; lsblk -d | grep ' 1T ' | awk '{print \$1}' | xargs -i udevadm info --query=all --name=/dev/{} | grep '^E: ID_SERIAL='"
    
    nodea
    E: ID_SERIAL=3600224808dd8eb102f6ffc5822c41d89
    Copy to Clipboard Toggle word wrap

After verifying that the shared disk is attached to each VM, you can configure resilient storage for the cluster.

Secure Boot is a mechanism in the Unified Extensible Firmware Interface (UEFI) specification to control the execution of programs at boot time. Secure Boot verifies digital signatures of the boot loader and its components at boot time, to ensure only trusted and authorized programs are executed, and also prevent unauthorized programs from loading.

Secure Boot is enabled for publicly available RHEL images on Azure platform. By default, it has the Allowed Signature database (db) with Microsoft certificates. Microsoft Azure allows adding custom certificates to UEFI Secure Boot variables when a new image version is registered in Azure Compute Gallery.

5.1. Understanding secure boot for RHEL on cloud

Secure Boot is a feature of Unified Extensible Firmware Interface (UEFI). It ensures that only trusted and digitally signed programs and components, such as the boot loader and kernel, run during boot time. Secure Boot checks digital signatures against trusted keys stored in hardware. If it detects any tampered components or components signed by untrusted entities, it aborts the boot process. This action prevents malicious software from compromising the operating system.

Secure Boot plays a critical role in configuring a Confidential Virtual Machine (CVM) by ensuring that only trusted entities participate in the boot chain. It authenticates access to specific device paths through defined interfaces, enforces the use of the latest configuration, and permanently overwrites earlier configurations. When the Red Hat Enterprise Linux (RHEL) kernel boots with Secure Boot enabled, it enters the lockdown mode, allowing only kernel modules signed by a trusted vendor to load. As a result, Secure Boot strengthens the security of the operating system boot sequence.

5.1.1. Components of secure boot

The Secure Boot mechanism consists of firmware, signature databases, cryptographic keys, boot loader, hardware modules, and the operating system. The following are the components of the UEFI trusted variables:

  • Key Exchange Key database (KEK): An exchange of public keys to establish trust between the RHEL operating system and the VM firmware. You can also update Allowed Signature database (db) and Forbidden Signature database (dbx) by using these keys.
  • Platform Key database (PK): A self-signed single-key database to establish trust between the VM firmware and the cloud platform. The PK also updates the KEK database.
  • Allowed Signature database (db): A database that maintains a list of certificates or binary hashes to check whether the binary file can boot on the system. Additionally, all certificates from db are imported to the .platform keyring of the RHEL kernel. With this feature, you can add and load signed third party kernel modules in the lockdown mode.
  • Forbidden Signature database (dbx): A database that maintains a list of certificates or binary hashes that are not allowed to boot on the system.
Note

Binary files check against the dbx database and the Secure Boot Advanced Targeting (SBAT) mechanism. With SBAT, you can revoke older versions of specific binaries by keeping the certificate that has signed binaries as valid.

5.1.2. Stages of secure boot for RHEL on cloud

When a RHEL instance boots in the Unified Kernel Image (UKI) mode and with Secure Boot enabled, the RHEL instance interacts with the cloud service infrastructure in the following sequence:

  1. Initialization: When a RHEL instance boots, the cloud-hosted firmware initially boots and implements the Secure Boot mechanism.
  2. Variable store initialization: The firmware initializes UEFI variables from a variable store, a dedicated storage area for information that firmware needs to manage for the boot process and runtime operations. When the RHEL instance boots for the first time, the store initializes from default values associated with the VM image.
  3. Boot loader: When booted, the firmware loads the first stage boot loader. For the RHEL instance in a x86 UEFI environment, the first stage boot loader is shim. The shim boot loader authenticates and loads the next stage of the boot process and acts as a bridge between UEFI and GRUB.

    1. The shim x86 binary in RHEL is currently signed by the Microsoft Corporation UEFI CA 2011 Microsoft certificate so that the RHEL instance can boot in the Secure Boot enabled mode on various hardware and virtualized platforms where the Allowed Signature database (db) has the default Microsoft certificates.
    2. The shim binary extends the list of trusted certificates with Red Hat Secure Boot CA and optionally, with Machine Owner Key (MOK).
  4. UKI: The shim binary loads the RHEL UKI (the kernel-uki-virt package). The corresponding certificate, Red Hat Secure Boot Signing 504 on the x86_64 architecture, signs the UKI. You can find this certificate in the redhat-sb-certs package. Red Hat Secure Boot CA signs this certificate, so the check succeeds.
  5. UKI add-ons: When you use the UKI cmdline extensions, the RHEL kernel actively checks their signatures against db, MOK, and certificates shipped with shim. This process ensures that either the operating system vendor RHEL or a user has signed the extensions.

When the RHEL kernel boots in the Secure Boot mode, it enters lockdown mode. After entering lockdown, the RHEL kernel adds the db keys to the .platform keyring and the MOK keys to the .machine keyring. During the kernel build process, the build system works with an ephemeral key, which consists of private and public keys. The build system signs standard RHEL kernel modules, such as kernel-modules-core, kernel-modules, and kernel-modules-extra. After the completion of each kernel build, the private key becomes obsolete to sign third-party modules. You can use certificates from db and MOK for this purpose.

To ensure that your Red Hat Enterprise Linux instance on the Azure cloud platform has a secured operating system booting process, use Secure Boot. When a custom RHEL Azure image is registered, the image consists of pre-stored Unified Extensible Firmware Interface (UEFI) variables for Secure Boot. This enables all the instances launched from the RHEL images to use the Secure Boot mechanism with the required variables on the first boot.

Microsoft Azure supports Secure Boot with Trusted Launch VMs. These VMs provide security mechanisms to protect against rootkits and bootkits, while providing additional features such as Virtual Trusted Platform Manager (vTPM). When creating an instance by using the GUI, you can find the Enable secure boot option under the Configure security features setting.

Prerequisites

  • You have installed the packages:

    • python3
    • openssl
    • efivar
    • keyutils
    • python3-virt-firmware
  • You have installed the azure-cli utility. For details, see Installing the Azure CLI on Linux.

Procedure

  1. Generate a custom certificate custom_db.cer by using the openssl utility:

    $ openssl req -quiet \
    -newkey rsa:4096 \
    -nodes -keyout custom_db.key \
    -new -x509 \
    -sha256 -days 3650 \
    -subj "/CN=Signature Database key/" \
    --outform DER \
    -out custom_db.cer
    Copy to Clipboard Toggle word wrap
  2. Convert the certificate into base64-encoded format:

    $ echo base64 -w0 custom_db.cer
    
    MIIFIjCCAwqgAwIBAgITNf23J4k0d8c0NR ....
    Copy to Clipboard Toggle word wrap
  3. Create and edit an azure-example-template.json Azure Resource Manager (ARM) file for registering a new Azure Compute Gallery image version:

    $ vi azure-example-template.json
    
    {
        "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
        "contentVersion": "1.0.0.0",
        "resources": [
            {
                "type": "Microsoft.Compute/galleries/images/versions",
                "apiVersion": "2023-07-03",
                "name": "<your compute gallery/your image definition/version>",
                "location": "<location of the VHD>",
                "properties": {
                    "storageProfile": {
                        "osDiskImage": {
                            "source": {
                                "id": "<your-storage-account-id>",
                                "uri": "<url-with-the-vhd>"
                            },
                            "hostCaching": "ReadOnly"
                        }
                    },
                    "securityProfile": {
                        "uefiSettings": {
                            "signatureTemplateNames": [
                                "MicrosoftUefiCertificateAuthorityTemplate"
                            ],
                            "additionalSignatures": {
                                "db": [
                                    {
                                        "type": "x509",
                                        "value": [
                                            "<base64 of custom_db.cer>"
                                        ]
                                    }
                                ]
                            }
                        }
                    }
                }
            }
        ]
    }
    Copy to Clipboard Toggle word wrap
  4. Use the azure-cli utility to register the image version:

    $ az deployment group create --name <example-deployment> \
    --resource-group <example-resource-group> \
    --template-file <example-template.json>
    Copy to Clipboard Toggle word wrap
  5. Reboot the instance from the Azure Portal.

Verification

  1. Check if the newly created RHEL instance has Secure Boot enabled:

    $ mokutil --sb-state
    SecureBoot enabled
    Copy to Clipboard Toggle word wrap
  2. Use the keyctl utility to verify the kernel keyring for the custom certificate:

    $ sudo keyctl list %:.platform
    
    keys in keyring:
    ...
    586621657: ---lswrv   0   0
    asymmetric: Signature Database key: f064979641c24e1b935e402bdbc3d5c4672a1acc
    ...
    Copy to Clipboard Toggle word wrap

Intel Trust Domain Extensions (TDX) is a security type of Confidential Virtual Machine (CVM), which provides a secure and isolated environment for VM. This approach is an advancement to the former technology, Intel Software Guard Extensions (SGX).

SGX provides VM isolation from the hypervisor and cloud service providers by creating secure memory regions known as enclaves. Application code stored in enclaves has access to memory and data stored inside the enclave, making it inaccessible to outside entities.

TDX creates hardware-isolated VMs called Trusted Domains (TDs). It ensures that only a VM accesses its memory and TD VMs are isolated from Virtual Machine Manager (VMM), hypervisors, other VMs, and the host. This ensures that while using resources from hypervisor, CPU, TD VMs remain secure by maintaining data confidentiality and integrity.

The main difference between SGX and TDX is that SGX works at application level while TDX works at virtualization level by limiting hypervisor access.

Note

Before deploying Red Hat Enterprise Linux (RHEL) on a public cloud platform, always check with the corresponding cloud service provider for the support status and certification of the particular RHEL instance type.

6.1. Understanding Intel TDX secure boot process

  1. Initialization and measurement: A TDX-enabled hypervisor sets the initial state of a VM. This hypervisor loads the firmware binary file into the VM memory and sets the initial register state. The Intel processor measures the initial state of the VM and provides details to verify the initial state of the VM.
  2. Firmware: The VM initiates the UEFI firmware. The firmware might include stateful or stateless Virtual Trusted Platform Module (vTPM) implementation. Stateful vTPM maintains persistent cryptographic state across VM reboots and migrations, whereas stateless vTPM generates fresh cryptographic state for each VM session without persistence. Virtual Machine Privilege Levels (VMPL) technology isolates vTPM from the guest. VMPL offers hardware-enforced privilege isolation between different VM components and the hypervisor.
  3. vTPM: Depending on your cloud service provider, for stateful vTPM implementation, the UEFI firmware might perform a remote attestation to decrypt the persistent state of vTPM. The vTPM also gathers data about the boot process, such as Secure Boot state, certificates used for signing boot artifacts, or UEFI binary hashes.
  4. Shim : When the UEFI firmware finishes the initialization process, it searches for the extended firmware interface (EFI) system partition. Then, the UEFI firmware verifies and executes the first stage boot loader from there. For RHEL, this is shim. The shim program allows non-Microsoft operating systems to load the second stage boot loader from the EFI system partition.

    1. shim uses a Red Hat certificate to verify the second stage boot loader (grub) or Red Hat Unified Kernel Image (UKI).
    2. grub or UKI unpacks, verifies, and executes Linux kernel and initramfs, and the kernel command line. This process ensures that the Linux kernel is loaded in a trusted and secured environment.
  5. Initramfs: In initramfs, vTPM information automatically unlocks the encrypted root partition in case of full disk encryption technology.

    1. When the root volume becomes available, initramfs transfers the execution flow there.
  6. Attestation: The VM tenant gets access to the system and can perform a remote attestation to ensure that the accessed VM is an untampered Confidential Virtual Machine (CVM). Attestation is performed based on information from the Intel processor and vTPM. This process confirms the authenticity and reliability of the initial CPU and memory state of the RHEL instance and Intel processor.
  7. TEE: This process creates a Trusted Execution Environment (TEE) to ensure that booting of the VM is in a trusted and secured environment.

6.2. Configuring a RHEL VM on Azure with Intel TDX

By using Intel TDX, you can create hardware-assisted isolated VMs known as trusted domains (TDs). It ensures that only the VM has access to its resources, while remaining inaccessible to hypervisors and hosts.

Prerequisites

Procedure

  1. Log in to Azure by using the azure cli utility:

    $ az login
    Copy to Clipboard Toggle word wrap
  2. Create an Azure resource group for the selected availability zone:

    $ az group create --name <example_resource_group> --location westeurope
    Copy to Clipboard Toggle word wrap
  3. Deploy a RHEL instance with TDX enabled, for example, the Standard_DC2eds_v5 instance type:

    $ az vm create --resource-group <example_resource_group> \
    --name <example_rhel_instance> \
    --image <"RedHat:rhel-cvm:9_5_cvm:latest"> \
    --size <Standard_DC2eds_v5> \
    --admin-username <example_azure_user> \
    --generate-ssh-keys \
    --security-type ConfidentialVM \
    --os-disk-security-encryption-type DiskWithVMGuestState
    Copy to Clipboard Toggle word wrap
  4. Connect to the RHEL instance:

    $ ssh <example_azure_user>@<example_ip_address_of_the_instance>
    Copy to Clipboard Toggle word wrap

Verification

  • Check kernel logs to verify status of TDX:

    $ dmesg | grep -i tdx
    Copy to Clipboard Toggle word wrap
    ...
    [    0.733613] Memory Encryption Features active: Intel TDX
    [    4.320222] systemd[1]: Detected confidential virtualization tdx.
    [    5.977432] systemd[1]: Detected confidential virtualization tdx.
    ...
    Copy to Clipboard Toggle word wrap
  • Check metadata of the RHEL instance configuration:

    $ az vm show --resource-group <example_resource_group> \
    --name <example_rhel_instance> \
    --query "securityProfile.enableTrustedDomainExtensions" \
    --output json
    Copy to Clipboard Toggle word wrap

AMD Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP) aims to prevent VM integrity-based attacks and reduce the dangers of memory integrity violations. For the secure boot process, AMD processors offer three hardware-based security mechanisms; Secure Encrypted Virtualization (SEV), SEV Encrypted State (SEV-ES), and SEV Secure Nested Paging (SEV-SNP).

  • SEV: The SEV mechanism encrypts virtual machine (VM) memory to prevent the hypervisor from accessing VM data.
  • SEV-ES: SEV with Encrypted State (SEV-ES) extends SEV by encrypting CPU register states. This mechanism prevents the hypervisor from accessing or modifying VM CPU registers. Despite providing isolation between hypervisor and VM, it is still vulnerable to memory integrity attacks.
  • SEV-SNP: SEV-SNP is an enhancement to SEV-ES that adds memory integrity protection along with VM encryption. This mechanism prevents the hypervisor from modifying page tables to redirect VM memory access, protecting against replay attacks and memory tampering.

    Note

    Before deploying Red Hat Enterprise Linux (RHEL) on a public cloud platform, always check with the corresponding cloud service provider for the support status and certification of the particular RHEL instance type.

7.1. Properties of SEV-SNP

  • Secure Processor: The AMD EPYC processor integrates a Secure Processor (SP) subsystem. AMD SP is a dedicated hardware component to manage keys and encryption operations.
  • Memory Integrity: For managing virtualization and isolation, memory management unit (MMU) utilizes page tables to translate virtual addresses to guest-physical addresses. SEV-SNP uses nested page tables for translating guest-physical addresses to host-physical addresses. Once nested page tables are defined, the hypervisor or host cannot alter page tables to modify the VM into accessing different pages, resulting in protection of memory integrity. SEV-SNP uses this method to offer protection against replay attacks and malicious modifications to VM memory.
  • Memory Encryption: The AMD EPYC processor hides the memory encryption key, which remains hidden from both host and VM.
  • Attestation report for verification: A CPU-generated report about RHEL instance information in an authorized cryptographic format. This process confirms the authenticity and reliability of the initial CPU and memory state of the RHEL instance and AMD processor.

    Note

    Even if a hypervisor creates the primary memory and CPU register state of the VM, they remain hidden and inaccessible to the hypervisor after initialization of that VM.

7.2. Understanding AMD SEV SNP secure boot process

  1. Initialization and measurement: A SEV-SNP enabled hypervisor sets the initial state of a VM. This hypervisor loads firmware binary into the VM memory and sets the initial register state. AMD Secure Processor (SP) measures the initial state of the VM and provides details to verify the initial state of the VM.
  2. Firmware: The VM initiates the UEFI firmware. The firmware might include either stateful or stateless Virtual Trusted Platform Module (vTPM) implementation. Stateful vTPM maintains persistent cryptographic state across VM reboots and migrations, whereas stateless vTPM generates fresh cryptographic state for each VM session without persistence. Virtual Machine Privilege Levels (VMPL) technology isolates vTPM from the guest. VMPL offers hardware-enforced privilege isolation between different VM components and the hypervisor.
  3. vTPM: Depending on your cloud service provider, for stateful vTPM implementation, the UEFI firmware might perform a remote attestation to decrypt the persistent state of vTPM.

    1. The vTPM also measures facts about the boot process such as Secure Boot state, certificates used for signing boot artifacts, UEFI binary hashes, and so on.
  4. Shim: When the UEFI firmware finishes the initialization process, it searches for the extended firmware interface (EFI) system partition. Then, the UEFI firmware verifies and executes the first stage boot loader from there. For RHEL, this is shim. The shim program allows non-Microsoft operating systems to load the second stage boot loader from the EFI system partition.

    1. shim uses a Red Hat certificate to verify the second stage boot loader (grub) or Red Hat Unified Kernel Image (UKI).
    2. grub or UKI unpacks, verifies, and executes Linux kernel and initial RAM filesystem (initramfs), and the kernel command line. This process ensures that the Linux kernel is loaded in a trusted and secured environment.
  5. Initramfs: In initramfs, vTPM information automatically unlocks the encrypted root partition in case of full disk encryption technology.

    1. When the root volume becomes available, initramfs transfers the execution flow to the root volume.
  6. Attestation: The VM tenant gets access to the system and can perform a remote attestation to ensure that the accessed VM is an untampered Confidential Virtual Machine (CVM). Attestation is performed based on information from AMD SP and vTPM. This process confirms the authenticity and reliability of the initial CPU and memory state of the RHEL instance and AMD processor.
  7. TEE: This process creates a Trusted Execution Environment (TEE) to ensure that booting of the VM is in a trusted and secured environment.

AMD Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP) is a security type of the Confidential Virtual Machine (CVM) technology for Red Hat Enterprise Linux (RHEL) on Azure Virtual Machines (VMs) and available only for AMD EPYC processor family. SEV-SNP provides a trusted boot environment so that the entire process becomes secured and protected such that hypervisor and cloud service provider cannot access the data.

Prerequisites

  • You have installed the openssh and openssh-clients packages.
  • You have installed the Azure CLI utility. For details, see Installing the Azure CLI.
  • You have launched the instance only from the mentioned Azure instance types. For details, see Supported VM sizes for CVM.

Procedure

  1. Log in to Azure by using the Azure CLI utility:

    $ az login
    Copy to Clipboard Toggle word wrap
  2. Create an Azure resource group for selected availability zone:

    $ az group create --name <example_resource_group> --location eastus
    Copy to Clipboard Toggle word wrap
  3. Deploy a RHEL instance with SEV-SNP, for example, the Standard_DC4as_V5 instance type:

    $ az vm create --resource-group <example_resource_group> \
    --name <example-rhel-9-instance> \
    --image <"RedHat:rhel-cvm:9_5_cvm:latest"> \
    --size <Standard_DC4as_V5> \
    --admin-username <example_azure_user> \
    --generate-ssh-keys \
    --security-type ConfidentialVM \
    --os-disk-security-encryption-type DiskWithVMGuestState
    Copy to Clipboard Toggle word wrap
  4. Connect to the RHEL instance:

    $ ssh <example_azure_user>@<example_ip_address_of_VM>
    Copy to Clipboard Toggle word wrap

Verification

  • Check kernel logs to verify status of SEV-SNP:

    $ sudo dmesg | grep -i sev
    Copy to Clipboard Toggle word wrap
    ...
    [    0.547223] Memory Encryption Features active: AMD SEV
    [    4.843171] kvm-guest: setup_efi_kvm_sev_migration : EFI live migration variable not found
    ...
    Copy to Clipboard Toggle word wrap

High-performance computing (HPC) workloads on Microsoft Azure require specialized configurations for optimal performance and scalability. The HPC RHEL system role automates the configuration of RHEL images with HPC-specific optimizations, including InfiniBand support, performance tuning, and required libraries.

After configuring an HPC-enabled image, you can generalize the virtual machine and create reusable image versions in Azure Compute Gallery. These images serve as the foundation for deploying HPC clusters on Microsoft Azure by using Azure CycleCloud, a cluster orchestration tool that integrates with the Slurm workload manager to schedule and manage computational jobs. Environment Modules provides a flexible framework for managing multiple software versions and their dependencies across HPC cluster nodes.

To configure a high-performance computing (HPC) RHEL system role on a customized Red Hat Enterprise Linux (RHEL) image, you can use the cloud-init utility. With cloud-init, you can automate configuration of Ansible collection and running Ansible playbooks on Microsoft Azure.

Use one of the following methods to configure an HPC RHEL system role on a customized RHEL image.

By using Ansible with the ansible-core utility, you can automate the configuration of custom RHEL images for Azure by applying RHEL system roles during the image building process. Utilities like cloud-init embed high-performance computing (HPC) RHEL system role configurations to create and configure an HPC RHEL image before deployment in Azure.

Prerequisites

  • You have an active Azure cloud subscription.

Procedure

  1. Go to the Azure Console.
  2. Click Virtual MachinesCreate Virtual Machine.
  3. Select the following configurations for a virtual machine from the Basics tab:

    1. In the Virtual machine name field, enter your VM name.
    2. Security type: Standard
    3. ImageSee All Images → Search for Red Hat Enterprise Linux (RHEL) for High Performance Computing (HPC) on Azure → Select Red Hat Enterprise Linux for HPC 9.6 VM - x64 Gen2
    4. VM architecture: x64
    5. Size: Standard_NC4as_T4_v3 - 4 vcpus, 28 GiB memory

      Note

      To get optimal performance, use only GPU-optimized VMs, such as NC and ND series. For details, see Virtual machine sizes in Azure.

  4. Go to the Advanced tab and enter the following details to the Custom data field:

    #cloud-config
    #Please check RHEL HPC Ansible system role documentation for all available options
    
    write_files:
      - path: /root/hpc_full_install.yaml
        permissions: 644
        content: |
          ---
          - name: Install and configure HPC
            hosts: localhost
            become: true
            vars:
              hpc_reboot_ok: false
              hpc_update_all_packages: true
              hpc_manage_firewall: true
            roles:
              - redhat.rhel_system_roles.hpc
      - path: /etc/dnf/azure-rhel9-eus.config
        permissions: 644
        content: |
          [rhui-microsoft-azure-rhel9]
          name=Microsoft Azure RPMs for Red Hat Enterprise Linux 9 (rhel9-eus)
          baseurl=https://rhui4-1.microsoft.com/pulp/repos/unprotected/microsoft-azure-rhel9-eus
          enabled=1
          gpgcheck=1
          sslverify=1
          gpgkey=/etc/pki/rpm-gpg/RPM-GPG-KEY-microsoft-azure-release
      - path: /etc/dnf/vars/releasever
        permissions: 644
        content: |
          9.6
    
    # Run custom commands
    runcmd:
      # lock VM to RHEL9.6 and enable EUS channels
      # https://learn.microsoft.com/en-us/azure/virtual-machines/workloads/redhat/redhat-rhui
      - dnf --assumeyes --disablerepo='*' remove "rhui-azure-rhel9"
      - dnf --assumeyes --config /etc/dnf/azure-rhel9-eus.config install rhui-azure-rhel9-eus
      - dnf --assumeyes clean all
      - dnf --assumeyes install rhel-system-roles
    Copy to Clipboard Toggle word wrap
  5. Click the Review + create button to create a VM with the given configuration.
  6. In the Azure Console, check that the VM has been deployed successfully and is ready for use.
  7. Click the Go to resource button.
  8. Copy the Public IP address.
  9. In the Azure Console, check that the VM is running.
  10. Connect to the VM:

    $ ssh -i ~/.ssh/azure_hpc <example_azureuser>@<192.0.2.101>
    Copy to Clipboard Toggle word wrap
  11. Check the VM status:

    $ sudo cloud-init status --wait
    Copy to Clipboard Toggle word wrap
  12. Once ready, run the HPC RHEL system role:

    $ sudo ANSIBLE_LOG_PATH=/var/log/ansible_hpc_full_install.log ansible-playbook /root/hpc_full_install.yaml --verbose
    Copy to Clipboard Toggle word wrap
  13. Reboot the VM.

    Important

    Wait for the completion of the initial reboot as the HPC RHEL system role configurations finalize during this phase.

Verification

  1. Connect to the VM through SSH:

    $ ssh -i <example_private_key.pem> <example_azureuser>@<192.0.2.101>
    Copy to Clipboard Toggle word wrap
  2. Verify the list of installed packages:

    $ sudo dnf list installed| grep -i -E 'nvidia-driver|cuda-toolkit|nccl|fabric-manager|rdma|openmpi'
    Copy to Clipboard Toggle word wrap
    cuda-toolkit-12-9.x86_64                12.9.1-1            @nvidia-cuda
    cuda-toolkit-12-9-config-common.noarch  12.9.79.1           @nvidia-cuda
    cuda-toolkit-12-config-common.noarch    12.9.79.1           @nvidia-cuda
    cuda-toolkit-config-common.noarch       12.9.79.1           @nvidia-cuda
    libnccl.x86_64                          2.27.5-1+cuda12.9   @nvidia-cuda
    libnccl-devel.x86_64                    2.27.5-1+cuda12.9   @nvidia-cuda
    librdma.x86_64                          54.0.1-e19          @rhel-9-for-x86_64-baseos-rhui-rpms
    nvidia-driver.x86_64                    3:575.57.08-1.e19   @nvidia-cuda
    nvidia-driver-cuda.x86_64               3:575.57.08-1.e19   @nvidia-cuda
    nvidia-driver-cuda-libs.x86_64          3:575.57.08-1.e19   @nvidia-cuda
    nvidia-driver-libs.x86_64               3:575.57.08-1.e19   @nvidia-cuda
    nvidia-fabric-manager.x86_64            575.57.08-1         @nvidia-cuda
    openmpi.x86_64                          2:4.1.7-7.e19       @rhel-9-for-x86_64-appstream-rhui-rpms
    openmpi-devel.x86_64                    2:4.1.7-7.e19       @rhel-9-for-x86_64-appstream-rhui-rpms
    rdma-core.x86_64                        54.0.1-e19          @rhel-9-for-x86_64-baseos-rhui-rpms
    Copy to Clipboard Toggle word wrap
  3. Verify installed Lmod environment modules:

    $ ml available
    Copy to Clipboard Toggle word wrap
    -----------------------/usr/share/modulefiles------------------------
       mpi/hpcx-2.24.1-pmix-4.2.9    mpi/openmpi-5.0.8-cuda12-gpu (L,D)
       mpi/hpcx-2.24.1               pmix/pmix-4.2.9              (L)
       mpi/openmpi-x86_64
    
    ----------------/usr/share/lmod/lmod/modulefiles/Core----------------
       lmod    settarg
    Copy to Clipboard Toggle word wrap

    where:

    • L: Module is loaded
    • D: Default Module

By using Ansible with the ansible-core utility, you can automate the configuration of custom Red Hat Enterprise Linux (RHEL) images for Azure by applying RHEL system roles during the image building process. Utilities like cloud-init and Azure CLI manage high-performance computing (HPC) RHEL system roles to create and configure an HPC RHEL image before deployment in Azure.

Prerequisites

  • You have an active Azure cloud subscription.

Procedure

  1. Connect to the Azure portal:

    $ az login
    Copy to Clipboard Toggle word wrap
  2. Create a key pair:

    $ ssh-keygen -t ed25519 -b 3072 -C "<azureuser@hpc>" -f ~/.ssh/azure_hpc
    Copy to Clipboard Toggle word wrap
  3. Edit the user-data.yml file with the following details:

    $ vi user-data.yml
    Copy to Clipboard Toggle word wrap
    #cloud-config
    # Please check RedHat HPC Ansible system role documentation for all available options
    
    write_files:
      - path: /root/hpc_full_install.yaml
        permissions: 644
        content: |
          ---
          - name: Install and configure HPC
            hosts: localhost
            become: true
            vars:
              hpc_reboot_ok: false
              hpc_update_all_packages: true
              hpc_manage_firewall: true
            roles:
              - redhat.rhel_system_roles.hpc
      - path: /etc/dnf/azure-rhel9-eus.config
        permissions: 644
        content: |
          [rhui-microsoft-azure-rhel9]
          name=Microsoft Azure RPMs for Red Hat Enterprise Linux 9 (rhel9-eus)
          baseurl=https://rhui4-1.microsoft.com/pulp/repos/unprotected/microsoft-azure-rhel9-eus
          enabled=1
          gpgcheck=1
          sslverify=1
          gpgkey=/etc/pki/rpm-gpg/RPM-GPG-KEY-microsoft-azure-release
      - path: /etc/dnf/vars/releasever
        permissions: 644
        content: |
          9.6
    
    # Run custom commands
    runcmd:
      # lock VM to RHEL9.6 and enable EUS channels
      # https://learn.microsoft.com/en-us/azure/virtual-machines/workloads/redhat/redhat-rhui
      - dnf --assumeyes --disablerepo='*' remove "rhui-azure-rhel9"
      - dnf --assumeyes --config /etc/dnf/azure-rhel9-eus.config install rhui-azure-rhel9-eus
      - dnf --assumeyes clean all
      - dnf --assumeyes install rhel-system-roles
    Copy to Clipboard Toggle word wrap
  4. Create a resource group:

    $ az group create --name <example_vm_resource_group>
    Copy to Clipboard Toggle word wrap
  5. Select and accept terms and conditions of the relevant image for your account:

    • For North America (NA) or Global accounts, use:

      $ az vm image terms accept --urn "redhat:rh-rhel-hpc:rh-rhel-hpc96:latest"
      Copy to Clipboard Toggle word wrap
    • For Europe, the Middle East and Africa (EMEA) accounts, use:

      $ az vm image terms accept --urn "redhat-limited:rh-rhel-hpc:rh-rhel-hpc96:latest"
      Copy to Clipboard Toggle word wrap
  6. Create an image based on the specified configuration in the last step:

    $ az vm create \
    --resource-group <example_vm_resource_group> \
    --name <example_vm_name> \
    --image <example_rhel_hpc_image_urn> \
    --size <Standard_NC4as_T4_v3> \
    --admin-username <example_azureuser> \
    --ssh-key-values ~/.ssh/azure_hpc.pub \
    --custom-data user-data.yaml \
    --security-type Standard \
    --public-ip-address-dns-name <example_vm_name>-$(openssl rand -hex 4) \
    --tags owner=$USER project=hpc
    Copy to Clipboard Toggle word wrap
  7. Check the Azure Console if the VM is running.
  8. Connect to the VM through SSH:

    $ ssh -i ~/.ssh/azure_hpc <example_azureuser>@<192.0.2.101>
    Copy to Clipboard Toggle word wrap
  9. Check the VM status:

    $ sudo cloud-init status --wait
    Copy to Clipboard Toggle word wrap
  10. Once ready, run the HPC RHEL system role:

    $ sudo ANSIBLE_LOG_PATH=/var/log/ansible_hpc_full_install.log ansible-playbook /root/hpc_full_install.yaml --verbose
    Copy to Clipboard Toggle word wrap
  11. Reboot the VM.

    Important

    Wait for the completion of the initial reboot as the HPC RHEL system role configurations finalize during this phase.

Verification

  1. Connect to the VM through SSH:

    $ ssh -i <example_private_key.pem> <example_azureuser>@<192.0.2.101>
    Copy to Clipboard Toggle word wrap
  2. Verify the list of installed packages:

    $ sudo dnf list installed| grep -i -E 'nvidia-driver|cuda-toolkit|nccl|fabric-manager|rdma|openmpi'
    Copy to Clipboard Toggle word wrap
    cuda-toolkit-12-9.x86_64                 12.9.1-1           @nvidia-cuda
    cuda-toolkit-12-9-config-common.noarch   12.9.79.1          @nvidia-cuda
    cuda-toolkit-12-config-common.noarch     12.9.79.1          @nvidia-cuda
    cuda-toolkit-config-common.noarch        12.9.79.1          @nvidia-cuda
    libnccl.x86_64                           2.27.5-1+cuda12.9  @nvidia-cuda
    libnccl-devel.x86_64                     2.27.5-1+cuda12.9  @nvidia-cuda
    librdma.x86_64                           54.0.1-e19         @rhel-9-for-x86_64-baseos-rhui-rpms
    nvidia-driver.x86_64                     3:575.57.08-1.e19  @nvidia-cuda
    nvidia-driver-cuda.x86_64                3:575.57.08-1.e19  @nvidia-cuda
    nvidia-driver-cuda-libs.x86_64           3:575.57.08-1.e19  @nvidia-cuda
    nvidia-driver-libs.x86_64                3:575.57.08-1.e19  @nvidia-cuda
    nvidia-fabric-manager.x86_64             575.57.08-1        @nvidia-cuda
    openmpi.x86_64                           2:4.1.7-7.e19      @rhel-9-for-x86_64-appstream-rhui-rpms
    openmpi-devel.x86_64                     2:4.1.7-7.e19      @rhel-9-for-x86_64-appstream-rhui-rpms
    rdma-core.x86_64                         54.0.1-e19         @rhel-9-for-x86_64-baseos-rhui-rpms
    Copy to Clipboard Toggle word wrap
  3. Verify installed Lmod environment modules:

    $ ml available
    Copy to Clipboard Toggle word wrap
    -----------------------/usr/share/modulefiles------------------------
       mpi/hpcx-2.24.1-pmix-4.2.9    mpi/openmpi-5.0.8-cuda12-gpu (L,D)
       mpi/hpcx-2.24.1               pmix/pmix-4.2.9              (L)
       mpi/openmpi-x86_64
    
    ----------------/usr/share/lmod/lmod/modulefiles/Core----------------
       lmod    settarg
    Copy to Clipboard Toggle word wrap

    where:

    • L: Module is loaded
    • D: Default Module

By generalizing a virtual machine (VM), you prepare the VM to use as a template or base image for image versioning. In this process, you need to remove specific data, stop the VM, deallocate resources, and mark the VM as generalized. By using generalized images, you can create multiple image versions from the same image.

Prerequisites

Procedure

  1. Connect to the VM through SSH:

    $ ssh -i <example_private_key.pem> <example_azureuser>@<192.0.2.101>
    Copy to Clipboard Toggle word wrap
  2. Remove any temporary users, network and host information:

    $ sudo waagent -deprovision+user -force
    Copy to Clipboard Toggle word wrap
  3. Log out or press Ctrl + D to close the SSH session.
  4. Stop the VM:

    $ az vm stop --name <example_vm_name> --resource-group <example_vm_resource_group>
    Copy to Clipboard Toggle word wrap
  5. Deallocate the resources to stop incurring Azure charges:

    $ az vm deallocate --name <example_vm_name> --resource-group <example_vm_resource_group>
    Copy to Clipboard Toggle word wrap
  6. Generalize the VM to make sure that this image is generic and ready to clone:

    $ az vm generalize --name <example_vm_name> --resource-group <example_vm_resource_group>
    Copy to Clipboard Toggle word wrap

To create a reusable Azure image version from a generalized virtual machine, you must first create a resource group to organize related resources such as compute, network, and storage. Within this resource group, set up an Azure Compute Gallery to manage and share images across your organization. Define image definitions in the gallery to logically group images and specify their properties and requirements. Based on these image definitions, you can create multiple image versions for consistency and scalability.

With an image version, you can create replicas and multiple versions of the same image. With Azure Compute Gallery, you can create a marketplace compatible custom images to share across your organization. You can use Azure CLI or Azure Cloud Shell. For details on Azure Cloud Shell, see Get started with Azure Cloud Shell.

Prerequisites

Procedure

  1. Create a resource group for hosting the gallery:

    $ az group create --name <example_image_resource_group>
    Copy to Clipboard Toggle word wrap
  2. Create a gallery in the above resource group:

    $ az sig create --resource-group <example_image_resource_group> \
    --gallery-name <example_image_gallery_name>
    Copy to Clipboard Toggle word wrap
  3. Set security type to Standard for subscription:

    $ az feature register --name UseStandardSecurityType \
    --namespace Microsoft.Compute
    Copy to Clipboard Toggle word wrap
  4. Register the provider:

    $ az provider register --namespace Microsoft.Compute
    Copy to Clipboard Toggle word wrap
  5. Create an image definition to manage image versions:

    $ az sig image-definition create --resource-group <example_image_resource_group> \
    --gallery-name <example_image_gallery_name> \
    --gallery-image-definition <example_image_definition> \
    --publisher <example_publisher> \
    --offer <example_offer> \
    --sku <example_sky> \
    --os-type Linux \
    --os-state Generalized \
    --hyper-v-generation V2 \
    --features SecurityType=Standard
    Copy to Clipboard Toggle word wrap
    <Publisher>
    The entity or organization that provides the image.
    <Offer>
    A collection of related images from a publisher.
    <Stock Keeping Unit (SKU)>
    An edition within an offer, indicating a major release.
    <VERSION>
    A version number of a given SKU.
  6. Get information about the image:

    $ az vm list --output table
    Copy to Clipboard Toggle word wrap
  7. Use generalized image name from the output:

    $ az vm get-instance-view --resource-group <example_vm_resource_group> \
    --name <example_vm_name> \
    --query id
    Copy to Clipboard Toggle word wrap
  8. Copy the ID of the image definition from the output.
  9. Create the image version by using the image ID obtained in the previous step:

    $ az sig image-version create \
    --resource-group <example_image_resource_group> \
    --gallery-name <example_image_gallery_name> \
    --gallery-image-definition <example_image_definition> \
    --gallery-image-version <example_version> \
    --virtual-machine <example_id>
    Copy to Clipboard Toggle word wrap
  10. Optional: Delete the VM and associated resources:

    $ az vm delete --resource-group <example_image_resource_group> \
    --name <example_vm_name>
    Copy to Clipboard Toggle word wrap

You can configure a Red Hat Enterprise Linux (RHEL) high-performance computing (HPC) cluster on Azure Cloud. HPC clusters are useful for solving complex problems that require intense processing and computation, by distributing tasks across multiple machines, also known as nodes.

Azure CycleCloud, a cloud-native orchestrator, manages HPC clusters for Azure Cloud. With Azure CycleCloud, you can administer HPC clusters for automatic deployment and scaling suitable workloads. Azure CycleCloud manages parallel computing jobs, resources, and sets the Slurm workload manager. Slurm, however, manages resource allocation for scheduling and running tasks in a cluster. The following steps use Slurm and Azure CycleCloud 8.x for deployment and managing a RHEL HPC cluster.

Warning

To configure a RHEL HPC cluster in the Azure environment, you can use Microsoft Azure services such as Azure CycleCloud. Use these tools at your own risk.

Prerequisites

Procedure

  1. Install and deploy CycleCloud on Azure:

  2. Display ID of the custom RHEL HPC image:

    $ az sig image-version show --resource-group="<example_resource_group>" \
    --gallery-name="<example_gallery>" \
    --gallery-image-definition="<example_image>" \
    --gallery-image-version="<example_version>" \
    --query="id" \
    --output="tsv"
    Copy to Clipboard Toggle word wrap
  3. Provision a Slurm workload manager with Azure CycleCloud by following the steps in the Run Slurm with CycleCloud:

    Warning

    Due to a known limitation of IPv4 addresses, selecting the Public Head Node option causes provisioning failure with the Slurm head node. As a workaround, ensure the Public Head Node option is unchecked and determine the most suitable way of accessing the Slurm head node in your environment. For details, see the related Slurm issue on GitHub.

    Note

    Use the custom RHEL image ID obtained in the earlier step for all cluster nodes. For details, see how to specify a custom OS image.

  4. On the CycleCloud homepage, select the existing Slurm cluster.
  5. To launch the Slurm cluster, click Start.
  6. Log in to the Slurm head node by selecting the cluster view and clicking Connect. Use the standard Slurm command-line tools to schedule HPC jobs. For details, see How do I submit jobs? (Slurm) section.

8.5. Managing Environment Modules for HPC clusters

The environment module subsystem uses Lua-based environment modules (Lmod) to list installed modules. With Lmod, you can dynamically modify the environment by loading and unloading various software packages and their dependencies. It manages multiple versions of compilers, libraries, and applications in the high-performance computing (HPC) environment, so you can select specific software configurations for each version. To do this, you can use the module utility, which is shortened to ml.

8.5.1. Commands for environment modules management

You can use the following commands to manage environment modules:

  • Display all commands related to the module utility:

    $ module help
    Copy to Clipboard Toggle word wrap
  • Display the individual command options and syntax:

    $ module <command> help
    Copy to Clipboard Toggle word wrap
  • Display details for a specific module:

    $ ml whatis pmix/pmix-4.2.9
    Copy to Clipboard Toggle word wrap
    pmix/pmix-4.2.9   : Description: PMIx 4.2.9 installed in /opt/pmix/4.2.9
    pmix/pmix-4.2.9   : Version: 4.2.9-1
    Copy to Clipboard Toggle word wrap
  • List available modules in the HPC environment:

    $ ml available
    Copy to Clipboard Toggle word wrap
    -----------------------/usr/share/modulefiles------------------------
      mpi/hpcx-2.24.1-pmix-4.2.9    mpi/openmpi-5.0.8-cuda12-gpu (L,D)
      mpi/hpcx-2.24.1               pmix/pmix-4.2.9              (L)
      mpi/openmpi-x86_64
    
    ----------------/usr/share/lmod/lmod/modulefiles/Core----------------
      lmod    settarg
    Copy to Clipboard Toggle word wrap

    where:

    • L: Module is loaded
    • D: Default Module
    • The (L) annotation indicates that the mpi/openmpi-5.0.8-cuda12-gpu module is loaded.
    • This also loads the pmix/pmix-4.2.9 module as a dependency.
    • The module system automatically loads and unloads dependent modules as needed.
  • List the loaded modules:

    $ ml list
    Copy to Clipboard Toggle word wrap
    Currently Loaded Modules:
    1) pmix/pmix-4.2.9
    2) mpi/openmpi-5.0.8-cuda12-gpu
    Copy to Clipboard Toggle word wrap
  • Unload a module and its dependencies:

    $ ml unload mpi/openmpi-5.0.8-cuda12-gpu
    $ ml list
    Copy to Clipboard Toggle word wrap
    No modules loaded
    Copy to Clipboard Toggle word wrap
  • Unload an existing module and load a new module:

    $ ml load mpi/openmpi-5.0.8-cuda12-gpu
    $ ml list
    Copy to Clipboard Toggle word wrap
    Currently Loaded Modules:
    
    1) pmix/pmix-4.2.9
    2) mpi/openmpi-5.0.8-cuda12-gpu
    Copy to Clipboard Toggle word wrap
  • Switch the loaded module:

    $ ml swap mpi/openmpi-x86_64
    Copy to Clipboard Toggle word wrap
    The following have been reloaded with a version change:
    
    1) mpi/openmpi-5.0.8-cuda12-gpu => mpi/openmpi-x86_64
    Copy to Clipboard Toggle word wrap
  • List the available modules:

    $ ml list
    Copy to Clipboard Toggle word wrap
    Currently Loaded Modules:
    
    1) mpi/openmpi-x86_64
    Copy to Clipboard Toggle word wrap

8.5.2. Modulefiles layout and rules

A modulefile is a script that defines and manages environment variables for loading, unloading, and switching software environments on HPC systems. The recommended directory structure and naming conventions enable efficient organization of environment modulefiles, including module definitions, wrapper modules, and consistent rules for managing multiple software environments in RHEL-based HPC deployments.

Manual method for module definitions
The module utility automatically discovers the module definitions placed in the /usr/share/modulefiles directory. If you have other directories with module definitions, you need to add them to the MODULEPATH environment variable.
Wrapper modules
To avoid modifying environment variables and allow Lmod to find package-specific modules, place the wrapper modules in the /usr/share/modulefiles directory so that these modules modify MODULEPATH and load the relevant module from the external location. An example of this style of wrapper is the mpi/hpcx-2.24.1 environment module.
Module file format

Lmod supports environment modules written in Lua as well as the Tool Command Language (Tcl) format. Lua scripting is the preferred method for defining environment modules. Use the .lua extension with the 755 permission set. The module name for the ml command omits the .lua suffix. Examples given in this document use the Lua script interfaces. It has the following requirements:

  • All packages that provide the same functionality, for example MPI libraries, are in a common module subdirectory. In this case, ../mpi/ for all the MPI library variants.
  • Add a conflict() definition to the module for the package subdirectory (e.g. conflict("mpi")) which allows only one module of that package type to load at a time.
  • Keep package naming consistent.
  • Use the feature name as the subdirectory name, while the individual modules for each package instance should be named according to the following pattern:

    <package provider>-<version>-<build>-<options>
    Copy to Clipboard Toggle word wrap

    For multiple MPI library variants, some are from different providers, while others are multiple builds of a given package with different compilers and build options. In such cases, the naming scheme provides consistency.

8.5.3. Available MPI environments

In an HPC Slurm cluster, the message passing interface (MPI) environment provides runtime support for applications to communicate and synchronize across nodes. Slurm integrates with MPI implementations such as OpenMPI to manage distributed jobs efficiently and optimize use of cluster resources. Slurm uses mpirun to orchestrate execution across allocated nodes for scalable, high-performance jobs. With the provided RHEL HPC cluster, the following MPI environments are available:

  • Openmpi.x86_64: Standard open MPI build that does not provide CUDA or GPU acceleration support.
  • Openmpi-5.0.8-cuda-gpu: Open MPI module compiled with CUDA support for GPU-aware MPI communication
  • Hpcx-2.24.1: NVIDIA comprehensive HPC software stack based on Open MPI.
  • Hpcx-2.24.1-pmix: HPC-X build configured with Process Management Interface (PMIx) for Slurm integration and job scheduling.

For consistency in naming of MPI libraries, specify Unified Communication X (UCX) as the point to point messaging layer (PML) implementation for the MPI infrastructure:

$ mpirun -mca pml ucx .
Copy to Clipboard Toggle word wrap
openmpi.x86_64
This module provides a standard build of OpenMPI 4.1.1. Although it includes PMIx 3.x support, it does not provide CUDA or GPU acceleration support. The RHEL InfiniBand drivers and infrastructure provide InfiniBand (IB)/Remote Direct Memory Access (RDMA) functionality. The OpenMPI libraries are compiled with gcc-11.4. Use this module if your application does not use CUDA language extension or require GPU offload support.
openmpi-5.0.8-cuda-gpu

This module provides OpenMPI 5.0.8 with PMIx 4.x support and CUDA and NVIDIA GPU acceleration support by using the NVIDIA HPC-X package libraries. The RHEL InfiniBand drivers and infrastructure provide InfiniBand (IB)/Remote Direct Memory Access (RDMA) functionality. The OpenMPI libraries are compiled with gcc-11.5. Only select this module if you are using CUDA-enabled applications that require GPU acceleration.

Note

In the absence of InfiniBand NICs, a warning is displayed for failure of InfiniBand NIC autodiscovery:

$ mpirun -n 2 /lib64/openmpi/bin/mpitests-osu_allreduce
Copy to Clipboard Toggle word wrap
...
[test-vm] Error: coll_hcoll_module.c:312 - mca_coll_hcoll_comm_query() Hcol library init failed
...
Copy to Clipboard Toggle word wrap

You can remove this warning by adding the -mca coll ^hcoll parameter to the mpirun command:

$ mpirun -mca coll ^hcoll -n 2 /lib64/openmpi/bin/mpitests-osu_allreduce
Copy to Clipboard Toggle word wrap
# Size   Avg Latency(us)
  1      8.36
  2      6.85
Copy to Clipboard Toggle word wrap
hpcx-2.24.1
This module provides an NVIDIA built OpenMPI 4.1.5 environment. It has no PMIx support, but provides CUDA and NVIDIA GPU acceleration support. The RHEL InfiniBand drivers and infrastructure provide InfiniBand (IB)/Remote Direct Memory Access (RDMA) functionality.
hpcx-2.24.1-pmix
This module provides the same environment as the mpi/hpcx-2.24.1 module and also has PMIx 4.x support enabled.

To ensure that a Red Hat Enterprise Linux (RHEL) instance has a secured boot process from an untrusted storage such as confidential virtual machine (CVM) on a public cloud platform, use Unified Kernel Image (UKI).

9.1. Introduction to Unified Kernel Image

To extend the secure boot protection throughout the entire boot chain, use Unified Kernel Image (UKI).

Components of UKI

Unified Kernel Image (UKI) is a Unified Extensible Firmware Interface (UEFI) Portable Executable (PE) binary for the UEFI firmware environment, which bundles the essential components of an operating system. UKI binary components extend the Secure Boot mechanism with initramfs and the kernel command line. Initramfs is a part of the Linux startup process, while the kernel command line gives you limited access to define parameters. Components are as follows:

  • The .linux section stores the Linux kernel image.
  • The .initrd section stores the initial RAM filesystem initramfs.
  • The .cmdline section stores the kernel command line.
  • Additional sections, such as .sbat.
  • The Red Hat signature.
Features of RHEL UKI with pre-built initramfs
  • Prohibits any malicious agent or component to alter any objects in the boot chain.
  • Due to pre-built initramfs, the user does not need to build its custom initramfs, which results in a faster kernel installation.
  • Provides support for the pre-built initramfs systems as it is similar in all installations such as virtual machine (VMs), containers, or cloud instances.
  • Provides support for the x86_64 architecture.
  • Includes the kernel-uki-virt package.
  • Built for virtual machines and cloud instances.
Limitation of UKI because of the reduced flexibility of the boot process
  • When building the UKI, the operating system vendor creates initramfs. As a consequence, the listed and included kernel modules are static. You can use the systemd system and configuration extensions to address this limitation.
  • The kernel command line parameters are static, which limits the use of parameters for different instance sizes or debugging options.

You can use the UKI command line extensions to overcome this limitation.

9.2. Understanding the UKI secure boot process

To protect your system against unauthorized boot-time modifications, use the secure boot mechanism with Unified Kernel Image (UKI).

When using UKI with secure boot, the system verifies each component in the boot chain to ensure system integrity and prevent malicious code execution.

Procedure

  1. UEFI Firmware: The boot process starts from the Unified Extensible Firmware Interface (UEFI) firmware. For boot, Red Hat Enterprise Linux (RHEL) UKI requires UEFI firmware, because legacy basic input/output system (BIOS) firmware is not supported.
  2. Shim boot loader: Use the shim boot loader for booting rather than directly booting the UKI from the UEFI firmware. shim includes additional security mechanisms such as Machine Owner Key (MOK) and Secure Boot Advanced Targeting (SBAT).
  3. Signature verification (Secure Boot UEFI mechanism): During boot, shim reads the UKI binary and the secure boot UEFI mechanism verifies the signature of UKI against trusted keys stored in the Secure Boot Allowed Signature Database (db) of the system, the MOK database, and the built-in database of the shim binary. If the signature key is valid, the verification passes.
  4. SBAT verification: Immediately after signature verification, the shim boot loader verifies the SBAT rules at startup.

    During SBAT verification, the system compares generation numbers, for components such as systemd.rhel or linux.rhel, embedded in the UKI by using the .sbat section against values in the shim boot loader. If the generation number for a component in the shim is higher than the generation number in the UKI, the binary gets automatically discarded, even if it was signed by a trusted key.

    Note that the generation number is a version identifier for UEFI applications, such as shim and grub.

  5. Unpacking and Execution: If verification passes, control passes from shim to the systemd-stub code in the UKI to continue the boot process.
  6. systemd-stub add-ons: During execution, systemd-stub unpacks and extracts the contents of the .cmdline section, the plain text kernel command line, and the .initrd section, the temporary root file system, for the boot process.

    Note that systemd-stub reads UKI add-ons and verifies their signatures to safely extend the kernel command line of UKI by appending the .cmdline content from add-ons. systemd-stub reads add-ons from two locations:

    • Global (UKI-independent) add-ons from the /loader/addons/ directory on the Extensible Firmware Interface (EFI) System Partition (ESP).
    • Per-UKI add-ons from the /EFI/Linux/<UKI-name>.extra.d/ directory on the ESP.
  7. Control passes from systemd-stub to the Linux kernel and the operating system boot process continues.

    From this point, secure boot with UKI mechanisms follows the standard kernel boot process.

Legal Notice

Copyright © Red Hat.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red Hat
Back to top