Chapter 1. Gathering installation logs
To assist in troubleshooting a failed OpenShift Container Platform installation, you can gather logs from the bootstrap and control plane, or master, machines.
Prerequisites
- You attempted to install an OpenShift Container Platform cluster, and installation failed.
-
You provided an SSH key to the installation program, and that key is in your running
ssh-agentprocess.
1.1. Gathering logs from a failed installation Copy linkLink copied to clipboard!
If you gave an SSH key to your installation program, you can gather data about your failed installation.
You use a different command to gather logs about an unsuccessful installation than to gather logs from a running cluster. If you must gather logs from a running cluster, use the oc adm must-gather command.
Prerequisites
- Your OpenShift Container Platform installation failed before the bootstrap process finished. The bootstrap node must be running and accessible through SSH.
-
The
ssh-agentprocess is active on your computer, and you provided both thessh-agentprocess and the installation program the same SSH key. - If you tried to install a cluster on infrastructure that you provisioned, you must have the fully-qualified domain names of the control plane, or master, machines.
Procedure
Generate the commands that are required to obtain the installation logs from the bootstrap and control plane machines:
If you used installer-provisioned infrastructure, run the following command:
$ ./openshift-install gather bootstrap --dir=<installation_directory>1 - 1
installation_directoryis the directory you specified when you ran./openshift-install create cluster. This directory contains the OpenShift Container Platform definition files that the installation program creates.
For installer-provisioned infrastructure, the installation program stores information about the cluster, so you do not specify the host names or IP addresses.
If you used infrastructure that you provisioned yourself, run the following command:
$ ./openshift-install gather bootstrap --dir=<installation_directory> \1 --bootstrap <bootstrap_address> \2 --master <master_1_address> \3 --master <master_2_address> \4 --master <master_3_address>"5 - 1
- For
installation_directory, specify the same directory you specified when you ran./openshift-install create cluster. This directory contains the OpenShift Container Platform definition files that the installation program creates. - 2
<bootstrap_address>is the fully-qualified domain name or IP address of the cluster’s bootstrap machine.- 3 4 5
- For each control plane, or master, machine in your cluster, replace
<master_*_address>with its fully-qualified domain name or IP address.
NoteA default cluster contains three control plane machines. List all of your control plane machines as shown, no matter how many your cluster uses.
The command output resembles the following example:
INFO Pulling debug logs from the bootstrap machine INFO Bootstrap gather logs captured here "<installation_directory>/log-bundle-<timestamp>.tar.gz"If you open a Red Hat support case about your installation failure, include the compressed logs in the case.
1.2. Manually gathering logs with SSH access to your host(s) Copy linkLink copied to clipboard!
Manually gather logs in situations where must-gather or automated collection methods do not work.
Prerequisites
- You must have SSH access to your host(s).
Procedure
Collect the
bootkube.serviceservice logs from the bootstrap host using thejournalctlcommand by running:$ journalctl -b -f -u bootkube.serviceCollect the bootstrap host’s container logs using the podman logs. This is shown as a loop to get all of the container logs from the host:
$ for pod in $(sudo podman ps -a -q); do sudo podman logs $pod; doneAlternatively, collect the host’s container logs using the
tailcommand by running:# tail -f /var/lib/containers/storage/overlay-containers/*/userdata/ctr.logCollect the
kubelet.serviceandcrio.serviceservice logs from the master and worker hosts using thejournalctlcommand by running:$ journalctl -b -f -u kubelet.service -u crio.serviceCollect the master and worker host container logs using the
tailcommand by running:$ sudo tail -f /var/log/containers/*
1.3. Manually gathering logs without SSH access to your host(s) Copy linkLink copied to clipboard!
Manually gather logs in situations where must-gather or automated collection methods do not work.
If you do not have SSH access to your node, you can access the systems journal to investigate what is happening on your host.
Prerequisites
- Your OpenShift Container Platform installation must be complete.
- Your API service is still functional.
- You have system administrator privileges.
Procedure
Access
journaldunit logs under/var/logby running:$ oc adm node-logs --role=master -u kubeletAccess host file paths under
/var/logby running:$ oc adm node-logs --role=master --path=openshift-apiserver