5.3. Troubleshooting CRI-O container runtime issues
5.3.1. About CRI-O container runtime engine
CRI-O is a Kubernetes-native container runtime implementation that integrates closely with the operating system to deliver an efficient and optimized Kubernetes experience. CRI-O provides facilities for running, stopping, and restarting containers.
The CRI-O container runtime engine is managed using a systemd service on each OpenShift Container Platform cluster node. When container runtime issues occur, verify the status of the crio systemd service on each node. Gather CRI-O journald unit logs from nodes that manifest container runtime issues.
5.3.2. Verifying CRI-O runtime engine status
You can verify CRI-O container runtime engine status on each cluster node.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI (
oc).
Procedure
Review CRI-O status by querying the
criosystemd service on a node, within a debug pod.Start a debug pod for a node:
$ oc debug node/my-node
Set
/hostas the root directory within the debug shell. The debug pod mounts the host’s root file system in/hostwithin the pod. By changing the root directory to/host, you can run binaries contained in the host’s executable paths:# chroot /host
注意OpenShift Container Platform 4.5 cluster nodes running Red Hat Enterprise Linux CoreOS (RHCOS) are immutable and rely on Operators to apply cluster changes. Accessing cluster nodes using SSH is not recommended and nodes will be tainted as accessed. However, if the OpenShift Container Platform API is not available, or the kubelet is not properly functioning on the target node,
ocoperations will be impacted. In such situations, it is possible to access nodes usingssh core@<node>.<cluster_name>.<base_domain>instead.Check whether the
criosystemd service is active on the node:# systemctl is-active crio
Output a more detailed
kubelet.servicestatus summary:# systemctl status crio
5.3.3. Gathering CRI-O journald unit logs
If you experience CRI-O issues, you can obtain CRI-O journald unit logs from a node.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. - Your API service is still functional.
-
You have installed the OpenShift CLI (
oc). - You have the fully qualified domain names of the control plane, or master machines.
Procedure
Gather CRI-O journald unit logs. The following example collects logs from all master nodes within the cluster:
$ oc adm node-logs --role=master -u crio
Gather CRI-O journald unit logs from a specific node:
$ oc adm node-logs <node_name> -u crio
If the API is not functional, review the logs using SSH instead. Replace
<node>.<cluster_name>.<base_domain>with appropriate values:$ ssh core@<node>.<cluster_name>.<base_domain> journalctl -b -f -u crio.service
注意OpenShift Container Platform 4.5 cluster nodes running Red Hat Enterprise Linux CoreOS (RHCOS) are immutable and rely on Operators to apply cluster changes. Accessing cluster nodes using SSH is not recommended and nodes will be tainted as accessed. Before attempting to collect diagnostic data over SSH, review whether the data collected by running
oc adm must gatherand otheroccommands is sufficient instead. However, if the OpenShift Container Platform API is not available, or the kubelet is not properly functioning on the target node,ocoperations will be impacted. In such situations, it is possible to access nodes usingssh core@<node>.<cluster_name>.<base_domain>.
