Chapter 5. Troubleshoot updates
To troubleshoot MicroShift updates, use the following guide.
5.1. Troubleshooting MicroShift updates
In some cases, MicroShift might fail to update. In these events, it is helpful to understand failure types and how to troubleshoot them.
5.1.1. Update path is blocked by version incompatibility
RPM dependency errors result if a MicroShift update is incompatible with the version of Red Hat Enterprise Linux for Edge (RHEL for Edge) or Red Hat Enterprise Linux (RHEL).
5.1.1.1. Compatibility table
Check the following compatibility table:
Red Hat Device Edge release compatibility matrix
Red Hat Enterprise Linux (RHEL) and MicroShift work together as a single solution for device-edge computing. You can update each component separately, but the product versions must be compatible. Supported configurations of Red Hat Device Edge use verified releases for each together as listed in the following table:
| RHEL Version(s) | MicroShift Version | Supported MicroShift Version → Version Updates |
|---|---|---|
| 9.4 | 4.17 | 4.17.1 → 4.17.z |
| 9.4 | 4.16 | 4.16.0 → 4.16.z, 4.16 → 4.17 |
| 9.2, 9.3 | 4.15 | 4.15.0 → 4.15.z, 4.15 → 4.16 on RHEL 9.4 |
| 9.2, 9.3 | 4.14 | 4.14.0 → 4.14.z, 4.14 → 4.15 or 4.14 → 4.16 on RHEL 9.4 |
5.1.1.2. Version compatibility
Check the following update paths:
Red Hat build of MicroShift update paths
- Generally Available Version 4.17.1 to 4.17.z on RHEL for Edge 9.4
- Generally Available Version 4.15.0 from RHEL 9.2 to 4.16.0 on RHEL 9.4
- Generally Available Version 4.14.0 from RHEL 9.2 to 4.15.0 on RHEL 9.4
5.1.2. OSTree update failed
If you updated on an OSTree system, the Greenboot health check automatically logs and acts on system health. A failure can be indicated by a system rollback by Greenboot. In cases where the update failed, but Greenboot did not complete a system rollback, you can troubleshoot using the RHEL for Edge documentation linked in the "Additional resources" section that follows this content.
- Checking the Greenboot logs manually
Manually check the Greenboot logs to verify system health by running the following command:
$ sudo systemctl restart --no-block greenboot-healthcheck && sudo journalctl -fu greenboot-healthcheck
5.1.3. Manual RPM update failed
If you updated by using RPMs on a non-OSTree system, an update failure can be indicated by Greenboot, but the health checks are only informative. Checking the system logs is the next step in troubleshooting a manual RPM update failure. You can use Greenboot and sos report to check both the MicroShift update and the host system.
5.2. Checking journal logs after updates
In some cases, MicroShift might fail to update. In these events, it is helpful to understand failure types and how to troubleshoot them. The journal logs can assist in diagnosing update failures.
The default configuration of the systemd journal service stores data in a volatile directory. To persist system logs across system starts and restarts, enable log persistence and set limits on the maximum journal data size.
Procedure
Get comprehensive MicroShift journal logs by running the following command:
$ sudo journalctl -u microshift
Check the Greenboot journal logs by running the following command:
$ sudo journalctl -u greenboot-healthcheck
Examining the comprehensive logs of a specific boot uses three steps. First list the boots, then select the one you want from the list you obtained:
List the boots present in the journal logs by running the following command:
$ sudo journalctl --list-boots
Example output
IDX BOOT ID FIRST ENTRY LAST ENTRY 0 681ece6f5c3047e183e9d43268c5527f <Day> <Date> 12:27:58 UTC <Day> <Date>> 13:39:41 UTC #....
Check the journal logs for the specific boot you want by running the following command:
$ sudo journalctl --boot <idx_or_boot_id> 1
- 1
- Replace
<idx_or_boot_id>with theIDXor theBOOT IDnumber assigned to the specific boot that you want to check.
Check the journal logs for the boot of a specific service by running the following command:
$ sudo journalctl --boot <idx_or_boot_id> -u <service_name> 1 2
5.3. Checking the status of greenboot health checks
Check the status of greenboot health checks before making changes to the system or during troubleshooting. You can use any of the following commands to help you ensure that greenboot scripts have finished running.
Procedure
To see a report of health check status, use the following command:
$ systemctl show --property=SubState --value greenboot-healthcheck.service
-
An output of
startmeans that greenboot checks are still running. -
An output of
exitedmeans that checks have passed and greenboot has exited. Greenboot runs the scripts in thegreen.ddirectory when the system is a healthy state. -
An output of
failedmeans that checks have not passed. Greenboot runs the scripts inred.ddirectory when the system is in this state and might restart the system.
-
An output of
To see a report showing the numerical exit code of the service where
0means success and non-zero values mean a failure occurred, use the following command:$ systemctl show --property=ExecMainStatus --value greenboot-healthcheck.service
To see a report showing a message about boot status, such as
Boot Status is GREEN - Health Check SUCCESS, use the following command:$ cat /run/motd.d/boot-status
