Search

Chapter 4. Troubleshooting CDK

download PDF

4.1. Overview

This section contains solutions to common problems that you might encounter while setting up and using CDK.

4.2. Troubleshooting Getting Started

4.2.1. Overview

This section contains solutions to common problems that you might encounter while installing and configuring CDK.

4.2.2. CDK startup check failed

While CDK starts, it runs several startup checks to make sure that the CDK VM and the OpenShift Cluster are able to start without any issues. If any configuration is incorrect or missing, the startup checks fail and CDK does not start.

The following sections describe the different startup checks.

4.2.2.1. Driver plug-in configuration

One of the startup checks verifies that the relevant driver plug-in is configured correctly. If this startup check fails, review the set up your virtualization environment topic and configure the driver.

If you want to force CDK to start despite a failing driver plugin-in check, you can instruct CDK to treat these errors as warnings:

  • For KVM/libvirt on Linux, run the following command:

    $ minishift config set warn-check-kvm-driver true
  • For xhyve on macOS, run the following command:

    $ minishift config set warn-check-xhyve-driver true
  • For Hyper-V on Windows, run the following command:

    $ minishift config set warn-check-hyperv-driver true

4.2.2.2. Persistent storage volume configuration and usage

CDK checks whether the persistent storage volume is mounted and that enough disk space is available. If the persistent storage volume, for example, uses more than 95% of the available disk space, CDK will not start.

If you want to recover the data, you can skip this test and start CDK to access the persistent volume:

$ minishift config set skip-check-storage-usage true

4.2.2.3. External network connectivity

After the CDK VM starts, it runs several network checks to verify whether external connectivity is possible from within the CDK VM.

By default, network checks are configured to treat any errors as warnings, because of the diversity of the development environments. You can configure the network checks to optimize them for your environment.

For example, one of the network checks pings an external host. You can change the host by running the following command:

$ minishift config set check-network-ping-host <host-IP-address>

Replace <host-IP-address> with the address of your internal DNS server, proxy host, or an external host that you can reach from your machine.

Because proxy connectivity might be problematic, you can run a check that tries to retrieve an external URL. You can configure the URL by running:

$ minishift config set check-network-http-host <URL>

4.3. Troubleshooting Driver Plug-ins

4.3.1. Overview

This section contains solutions to common problems that you might encounter while configuring the driver plug-ins for CDK.

4.3.2. KVM/libvirt

4.3.2.1. Undefining virsh snapshots fail

If you use virsh on KVM/libvirt to create snapshots in your development workflow, and then use minishift delete to delete the snapshots along with the VM, you might encounter the following error:

$ minishift delete
Deleting the {project} VM...
Error deleting the VM:  [Code-55] [Domain-10] Requested operation is not valid: cannot delete inactive domain with 4 snapshots

Cause: The snapshots are stored in ~/.minishift/machines, but the definitions are stored in /var/lib/libvirt/qemu/snapshot/minishift.

Workaround: To delete the snapshots, you need to perform the following steps as root.

  1. Delete the definitions.

    # virsh snapshot-delete --metadata minishift <snapshot-name>
  2. Undefine the CDK domain.

    # virsh undefine minishift

    You can now run minishift delete to delete the VM and restart CDK.

Note

If these steps do not resolve the issue, you can also use the following command to delete the snapshots:

$ rm -rf ~/.minishift/machines

It is recommended to avoid using metadata when you create snapshots. To ensure this, you can specify the --no-metadata flag. For example:

# virsh snapshot-create-as --domain vm1 overlay1 --diskspec vda,file=/export/overlay1.qcow2 --disk-only --atomic --no-metadata

4.3.2.2. Error creating new host: dial tcp: missing address

The problem is likely that the libvirtd service is not running. You can check this with the following command:

$ systemctl status libvirtd

If libvirtd is not running, start it and enable it to start on boot:

$ systemctl start libvirtd
$ systemctl enable libvirtd

4.3.2.3. Failed to connect socket to '/var/run/libvirt/virtlogd-sock'

The problem is likely that the virtlogd service is not running. You can check this with the following command:

$ systemctl status virtlogd

If virtlogd is not running, start it and enable it to start on boot:

$ systemctl start virtlogd
$ systemctl enable virtlogd

4.3.2.4. Domain 'minishift' already exists…​

If you try minishift start and this error appears, ensure that you use minishift delete to delete the VMs that you created earlier. However, if this fails and you want to completely clean up CDK and start fresh, do the following:

  1. As root, check if any existing CDK VMs are running:

    # virsh list --all
  2. If any CDK VM is running, stop it:

    # virsh destroy minishift
  3. Delete the VM:

    # virsh undefine minishift
  4. As your regular user, delete the ~/.minishift/machines directory:

    $ rm -rf ~/.minishift/machines

In case all of this fails, you might want to uninstall CDK and do a fresh install of CDK.

4.3.3. xhyve

4.3.3.1. Could not create vmnet interface

The problem is likely that the xhyve driver is not able to clean up vmnet when a VM is removed. vmnet.framework determines the IP address based on the following files:

  • /var/db/dhcpd_leases
  • /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist

Reset the CDK-specific IP database, ensure that you remove the minishift entry section from the dhcpd_leases file, and reboot your system.

{
  ip_address=192.168.64.2
  hw_address=1,2:51:8:22:87:a6
  identifier=1,2:51:8:22:87:a6
  lease=0x585e6e70
  name=minishift
}
Note

You can completely reset the IP database by removing the files manually, but this is very risky.

4.3.4. VirtualBox

4.3.4.1. Error machine does not exist

If you use Windows, ensure that you set the --vm-driver virtualbox flag in the minishift start command. Alternatively, the problem might be an outdated version of VirtualBox.

To avoid this issue, it is recommended to use VirtualBox 5.1.12 or later.

4.3.5. Hyper-V

4.3.5.1. Hyper-V commands must be run as an Administrator

If you run CDK with Hyper-V on Windows as a normal user or as a user with Administrator privileges, you might encounter the following error:

Error starting the VM: Error creating the VM. Error with pre-create check: "Hyper-V commands must be run as an Administrator".

Workaround: You can either add yourself to the Hyper-V Administrators group, which is recommended, or run the shell in an elevated mode.

If you are using PowerShell, you can add yourself to the Hyper-V Administrators group as follows:

  1. As an administrator, run the following command:

    ([adsi]”WinNT://./Hyper-V Administrators,group”).Add(“WinNT://$env:UserDomain/$env:Username,user”)
  2. Log out and log back in for the change to take effect.

You can also use the GUI to add yourself to the Hyper-V Administrators group as follows:

  1. Click the Start button and choose Computer Management.
  2. In the Computer Management window, select Local Users And Groups and then double click on Groups.
  3. Double click on the Hyper-V Administrators group, the Hyper-V Administrators Properties dialog box is displayed.
  4. Add your account to the Hyper-V Administrators group and log off and log in for the change to take effect.

Now you can run the Hyper-V commands as a normal user.

For more options for Hyper-V see creating Hyper-V administrators local group.

4.3.5.2. CDK running with Hyper-V fails when connected to OpenVPN

If you try to use CDK with Hyper-V using an external virtual switch while you are connected to a VPN such as OpenVPN, CDK might fail to provision the VM.

Cause: Hyper-V networking might not route the network traffic in both directions properly when connected to a VPN.

Workaround: Disconnect from the VPN and try again after stopping the VM from the Hyper-V manager.

4.4. Troubleshooting Miscellaneous

4.4.1. Overview

This section contains solutions to common problems that you might encounter while using various components of CDK.

4.4.2. The root filesystem of the CDK VM exceeds overlay size

Installing additional packages or copying large files to the root filesystem of the CDK VM might exceed the allocated overlay size and lock the CDK VM.

Cause: The CDK VM root filesystem contains core packages that are configured to optimize running the CDK VM and containers. The available storage on the root filesystem is determined by the overlay size, which is smaller than the total available storage.

Workaround: Avoid installing packages or storing large files in the root filesystem of the CDK VM. Instead, you can create a sub-directory in the mnt/sda1/ persistent storage volume or define and mount host folders that can share storage space between the host and the CDK VM.

If you want to perform development tasks inside the CDK VM, it is recommended that you use containers, which are stored in persistent storage volumes, and reuse the CDK Docker daemon.

4.4.3. Special characters cause passwords to fail

Depending on your operating system and shell environment, certain special characters can trigger variable interpolation and therefore cause passwords to fail.

Workaround: When creating and entering passwords, wrap the string with single quotes in the following format: '<password>'

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.