Greenboot health checks are helpful on edge devices where direct serviceability is either limited or non-existent. You can use greenboot health checks to assess the health of your workloads and applications. These additional health checks are useful for software problem detection and automatic system rollbacks.
Workload or application health checks can use the MicroShift basic health check functions already implemented for the MicroShift core services. Creating your own comprehensive scripts for your applications is recommended. For example, you can write one that verifies that a service has started.
You can also use the microshift healthcheck command, which can run checks that the basic functions of the workload are operating as expected.
Important
The following functions related to checking workload health in /usr/share/microshift/functions/greenboot.sh are deprecated and planned for removal in a future release:
The microshift healthcheck command checks whether a workload of the provided type exists and verifies its status for the specified timeout duration. The number of ready replicas, that is, pods, must match the expected amount.
To run the microshift healthcheck command successfully, use the following prerequisites:
Execute commands from a root user account.
Enable the MicroShift service.
You can add the following actions to the microshift healthcheck command:
-v=2 to increase verbosity of the output
--timeout="${WAIT_TIMEOUT_SECS}s" to override default 600s timeout value
--namespace `<namespace> to specify the namespace of the workloads
--deployments `<application-deployment> to check the readiness of a specific deployment
You can create workload or application health check scripts in the text editor of your choice. Save the scripts in the /etc/greenboot/check/required.d directory. When a script in the /etc/greenboot/check/required.d directory exits with an error, greenboot triggers a reboot in an attempt to heal the system.
Note
Any script in the /etc/greenboot/check/required.d directory triggers a reboot if it exits with an error.
If your health check logic requires any post-check steps, you can also create additional scripts and save them in the relevant greenboot directories. For example:
You can also place shell scripts you want to run after a boot has been declared successful in /etc/greenboot/green.d.
You can place shell scripts you want to run after a boot has been declared failed in /etc/greenboot/red.d. For example, if you have steps to heal the system before restarting, you can create scripts for your use case and place them in the /etc/greenboot/red.d directory.
You can start with the following example health check script. Add to it for your use case. In your custom workload health check script, you must define the relevant namespace, deployment, daemonset, and statefulset.
Important
Choose a name prefix for your application that ensures it runs after the 40_microshift_running_check.sh script, which implements the MicroShift health check procedure for its core services.
Example greenboot health check script
Load the workload health check functions library
Stop the script if the user running it is not 'root'
Set the wait timeout for the current check based on the boot counter
#!/bin/bashset-eSCRIPT_NAME=$(basename $0)# Load the workload health check functions librarysource /usr/share/microshift/functions/greenboot.sh
# Stop the script if the user running it is not 'root'if[$(id-u)-ne0];thenecho"The '${SCRIPT_NAME}' script must be run with the 'root' user privileges"exit1fiecho"STARTED"# Set the wait timeout for the current check based on the boot counterWAIT_TIMEOUT_SECS=$(get_wait_timeout)
/usr/bin/microshift healthcheck -v=2--timeout="${WAIT_TIMEOUT_SECS}s"--namespace busybox --deployments busybox-deployment
Copy to ClipboardCopied!Toggle word wrapToggle overflow
Important
Functions related to checking workload health previously included in the /usr/share/microshift/functions/greenboot.sh script file are deprecated. You can write a custom script, or use the microshift healthcheck command with various options instead. See "How workload health check scripts work" for more information.
The output of the greenboot workload health check script varies with the host system type. Example outputs for Red Hat Enterprise Linux (RHEL) system types are included for reference only.
Prerequisites
You have root access.
You installed a workload.
You created a health check script for the workload.
The MicroShift service is enabled.
Procedure
To test that greenboot is running a health check script file, reboot the host by running the following command:
sudo reboot
$sudoreboot
Copy to ClipboardCopied!Toggle word wrapToggle overflow
Examine the output of greenboot health checks by running the following command:
Copy to ClipboardCopied!Toggle word wrapToggle overflow
Example partial output for an RPM system
#...
GRUB boot variables:
boot_success=1
boot_indeterminate=0
Greenboot variables:
GREENBOOT_WATCHDOG_CHECK_ENABLED=true
System installation type:
RPM
System installation status:
Not an ostree / bootc system
#...
#...
GRUB boot variables:
boot_success=1
boot_indeterminate=0
Greenboot variables:
GREENBOOT_WATCHDOG_CHECK_ENABLED=true
System installation type:
RPM
System installation status:
Not an ostree / bootc system
#...
Copy to ClipboardCopied!Toggle word wrapToggle overflow
We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.
Making open source more inclusive
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.
About Red Hat
We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Red Hat Summit
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}