Chapter 5. Managing RHEL bootable images

download PDF

After installing and deploying RHEL bootable images, you can perform management operations on your container images, such as changing or updating the systems. The system supports in-place transactional updates with rollback after deployment.

This kind of management, also known as Day 2 management baseline, consists of transactionally fetching new operating system updates from a container registry and booting the system into them, while supporting manual, or automated rollbacks in case of failures.

You can also rely on automatic updates, that are turned on by default. The systemd service unit and the systemd timer unit files check the container registry for updates and apply them to the system. You can trigger an update process with different events, such as updating an application. There are automation tools watching these updates and then triggering the CI/CD pipelines. A reboot is required, because the updates are transactional. For environments that require more sophisticated or scheduled rollouts, you must disable auto updates and use the bootc utility to update your operating system.

See Day 2 operations support for more details.

Figure 5.1. Manually updating an installed operating system, changing the container image reference or rolling back changes if needed

639 RHEL Bootable Container Bifrost 0524 5

5.1. Switching the container image reference

You can change the container image reference used for upgrades by using the bootc switch command. For example, you can switch from the stage to the production tag. The bootc switch command performs the same operations as the bootc upgrade command and additionally changes the container image reference.

To manually switch an existing ostree-based container image reference, use the bootc switch command.


The use of rpm-ostree to make changes, or install content, is not supported.


  • A booted system using bootc.


  • Run the following command:

    $ bootc switch [--apply]<namespace>/<image>:<tag>

    Optionally, you can use the --apply option when you want to automatically take actions, such as rebooting if the system has changed.


The bootc switch command has the same effect as bootc upgrade. The only difference is the container image reference is changed. This allows preserving the existing states in /etc and /var, for example, host SSH keys and home directories.

Additional resources

5.2. Adding modules to the bootc image initramfs

The rhel9/rhel-bootc image uses the dracut infrastructure to build an initial RAM disk, the initrd during the image build time. The initrd is built and included in the /usr/lib/modules/$kver/initramfs.img location inside the container.

You can use a drop-in configuration file to override the dracut configuration, and place it in /usr/lib/dracut/dracut.conf.d/<50-custom-added-modules.conf> And thus re-create initrd with the modules you want to add.


  • A booted system using bootc.


  • Re-create the initrd as part of a container build:

    FROM <baseimage>
    COPY <50-custom-added-modules>.conf /usr/lib/dracut/dracut.conf.d
    RUN set -x; kver=$(cd /usr/lib/modules && echo *); dracut -vf /usr/lib/modules/$kver/initramfs.img $kver

    By default the command attempts to pull the running kernel version, which causes an error. Explicitly pass to dracut the kernel version of the target to avoid errors.

5.3. Modifying and regenerating initrd

The default container image includes a pre-generated initial RAM disk (initrd) in /usr/lib/modules/$kver/initramfs.img. To regenerate the initrd, for example, to add a dracut module, follow the steps:


  1. Write your drop-in configuration file. For example:

    dracutmodules = "module"
  2. Place your drop-in configuration file in the location that dracut normally uses: /usr. For example:

  3. Regenerate the initrd as part of the container build. You must explicitly pass the kernel version to target to dracut, because it tries to pull the running kernel version, which can cause an error. The following is an example:

    FROM <baseimage>
    COPY 50-custom-added-modules.conf /usr/lib/dracut/dracut.conf.d
    RUN set -x; kver=$(cd /usr/lib/modules && echo *); dracut -vf /usr/lib/modules/$kver/initramfs.img $kver

5.4. Performing manual updates from an installed operating system

Installing image mode for RHEL is a one time task. You can perform any other management task, such as changing or updating the system, by pushing the changes to the container registry.

When using image mode for RHEL, you can choose to perform manual updates for your systems. Manual updates are also useful if you have an automated way to perform updates, for example, by using Ansible. Because the automatic updates are enabled by default, to perform manual updates you must turn the automatic updates off. You can do this by choosing one of the following options:

  • Running the bootc upgrade command
  • Modifying the systemd timer file

5.5. Turning off automatic updates

To perform manual updates you must turn off automatic updates. You can do this by choosing one of the following options in the procedure below.


  • Disable the timer of a container build.

    • By running the bootc upgrade command:

      $ systemctl mask bootc-fetch-apply-updates.timer
    • By modifying the systemd timer file. Use systemd "drop-ins" to override the timer. In the following example, updates are scheduled for once a week.

      1. Create an updates.conf file with the following content:

        # Clear previous timers
        OnBootSec= OnBootSec=1w OnUnitInactiveSec=1w
      2. Add your container to the file that you created:

        $ mkdir -p /usr/lib/systemd/system/bootc-fetch-apply-updates.timer.d
        $ cp updates.conf /usr/lib/systemd/system/bootc-fetch-apply-updates.timer.d

5.6. Manually updating an installed operating system

To manually fetch updates from a registry and boot the system into the new updates, use bootc upgrade. This command fetches the transactional in-place updates from the installed operating system to the container image registry. The command queries the registry and queues an updated container image for the next boot. It stages the changes to the base image, while not changing the running system by default.


  • Run the following command:

    $ bootc upgrade [--apply]

    The apply argument is optional and you can use it when you want to automatically take actions, such as rebooting if the system has changed.


The bootc upgrade and bootc update commands are aliases.

Additional resources

5.7. Performing rollbacks from a updated operating system

You can roll back to a previous boot entry to revert changes by using the bootc rollback command. This command changes the boot loader entry ordering by making the deployment under rollback queued for the next boot. The current deployment then becomes the rollback. Any staged changes, such as a queued upgrade that was not applied, are discarded.

After a rollback completes, the system reboots and the update timer run within 1 to 3 hours which automatically update and reboot your system to the image you just rolled back from.


If you perform a rollback, the system will automatically update again unless you turn off auto-updates. See Turning off automatic updates.


  • You performed an update to the system.


  • Run the following command:

    $ bootc rollback [-h|--help] [-V|--version]

The bootc rollback command has the same effect as bootc upgrade. The only difference is the container image being tracked. This enables preserving the existing states in /etc and /var, for example, host SSH keys and home directories.


  • Use systemd journal to check the logged message for the detected rollback invocation.

    $ journalctl -b

    You can see a log similar to:


Additional resources

5.8. Deploying updates to system groups

You can change the configuration of your operating system by modifying the Containerfile. Then you can build and push your container image to the registry. When you next boot your operating system, an update will be applied.

You can also change the container image source by using the bootc switch command. The container registry is the source of truth. See Switching the container image reference.

Usually, when deploying updates to system groups, you can use a central management service to provide a client to be installed on each system which connects to the central service. Often, the management service requires the client to perform a one time registration. The following is an example on how to deploy updates to system groups. You can modify it to create a persistent systemd service, if required.


For clarity reasons, the Containerfile in the example is not optimized. For example, a better optimization to avoid creating multiple layers in the image is by invoking RUN a single time.

You can install a client into a image mode for RHEL image and run it at startup to register the system.


  • The management-client handles future connections to the server, by using a cron job or a separate systemd service.


  • Create a management service with the following characteristics. It determines when to upgrade the system.

    1. Disable bootc-fetch-apply-updates.timer if it is included in the base image.
    2. Install the client by using dnf, or some other method that applies for your client.
    3. Inject the credentials for the management service into the image.

5.9. Checking inventory health

Health checks are one of the Day 2 Operations. You can manually check the system health of the container images and events that are running inside the container.

You can set health checks by creating the container on the command line. You can display the health check status of a container by using the podman inspect or podman ps commands.

You can monitor and print events that occur in Podman by using the podman events command. Each event includes a timestamp, a type, a status, a name, if applicable, and an image, if applicable.

For more information about health checks and events, see chapter Monitoring containers.

5.10. Automation and GitOps

You can automate the building process by using CI/CD pipelines so that an update process can be triggered by events, such as updating an application. You can use automation tools that track these updates and trigger the CI/CD pipelines. The pipeline keeps the systems up to date by using the transactional background operating system updates.

Red Hat logoGithubRedditYoutubeTwitter


Try, buy, & sell


About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.