Chapter 3. Troubleshooting Red Hat CodeReady Containers
The goal of Red Hat CodeReady Containers is to deliver an OpenShift environment for development and testing purposes. Issues occurring during installation or usage of specific OpenShift applications are outside of the scope of CodeReady Containers and should be reported to the relevant project. For example, OpenShift issues are tracked on GitHub.
3.1. Basic troubleshooting
The majority of issues can be resolved by stopping a running CodeReady Containers virtual machine, deleting the virtual machine, and starting a new instance of the virtual machine.
Prerequisites
-
The host machine has been set up using the
crc setup
command. For more information, see Section 1.6, “Setting up CodeReady Containers”. -
The virtual machine has been started using the
crc start
command. For more information, see Section 1.7, “Starting the virtual machine”. - You are using the latest CodeReady Containers release. The generated certificates for the embedded system image bundle expire after 30 days. Using an older system image bundle may result in errors related to expired x509 certificates. For more information, see Section 3.4, “Troubleshooting expired certificates”.
Procedure
To troubleshoot CodeReady Containers, perform the following steps:
Stop the CodeReady Containers virtual machine:
$ crc stop
Delete the CodeReady Containers virtual machine:
$ crc delete
Start the CodeReady Containers virtual machine:
$ crc start
NoteThe cluster takes a minimum of four minutes to start the necessary containers and operators before serving a request.
If your issue has not been resolved by this procedure, perform the following steps:
- Search open issues for the issue that you are encountering.
-
If no existing issue addresses the encountered issue, create an issue and attach the
~/.crc/crc.log
file to the created issue. The~/.crc/crc.log
file contains detailed debugging and troubleshooting information which can help diagnose the problem that you are experiencing.
3.2. Getting shell access to the OpenShift cluster
Direct access to the OpenShift cluster is not needed for regular cluster use, and is strongly discouraged. If such access is needed for troubleshooting or debugging purposes, follow these steps:
-
Enable
oc
access to the cluster, logging in askubeadmin
. For detailed steps, see Section 1.8, “Accessing the OpenShift cluster”. Run
oc get nodes
. The output will be similar to this:$ oc get nodes NAME STATUS ROLES AGE VERSION crc-shdl4-master-0 Ready master,worker 7d7h v1.14.6+7e13ab9a7
-
Run
oc debug nodes/crc-shdl4-master-0
wherecrc-shdl4-master-0
is the name of the node which was printed at the previous step.
3.3. DNS configuration details
This section describes how CodeReady Containers configures networking. This can be useful for troubleshooting, or in case you are using a non-standard DNS setup, and want to adjust it for CodeReady Containers needs. The crc ip
command can be used to obtain the VM IP address as needed. This address can change each time the VM is restarted.
3.3.1. Linux
- CodeReady Containers expects NetworkManager to be used to manage networking.
-
NetworkManager is set up to use dnsmasq through the
/etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf
configuration file. -
The configuration file for this dnsmasq instance is
/etc/NetworkManager/dnsmasq.d/crc.conf
:
server=/crc.testing/192.168.130.11 server=/apps-crc.testing/192.168.130.11
-
NetworkManager dnsmasq instance forwards requests for the
crc.testing
andapps-crc.testing
domains to the 192.168.130.11 DNS server which is a dnsmasq instance running inside the virtual machine.
3.3.2. macOS
-
CodeReady Containers creates a
/etc/resolver/testing
file which instructs macOS to forward all DNS requests for thetesting
domain to the CodeReady Containers virtual machine. -
CodeReady Containers also adds an
api.crc.testing
entry to/etc/hosts
pointing at the VM IP address. This is needed by theoc
binary, see OpenShift issue #23266.
3.4. Troubleshooting expired certificates
Prior to Red Hat CodeReady Containers 1.2.0, the system bundle included in each released crc
binary expired 30 days after the release due to certificates embedded in the OpenShift cluster. Using an older crc
binary or system bundle will result in an expired certificates error.
Starting from CodeReady Containers 1.2.0, the embedded certificates can be automatically renewed by crc. The certificate renewal process is triggered when needed when crc start
is run. When certificates need to be renewed, this can add up to five minutes to the start time of the cluster.
Procedure
With CodeReady Containers releases older than 1.2.0, to resolve expired certificate errors:
-
Download the latest CodeReady Containers release and place the
crc
binary in your$PATH
. Remove the cluster with certificate errors using the
crc delete
command:$ crc delete
NoteThe
crc delete
command will result in the loss of data stored in the CodeReady Containers virtual machine.Set up the new release:
$ crc setup
Start the new cluster:
$ crc start