Chapter 3. Using CDK

When you use CDK, you interact with the following components:

The CDK architecture diagram outlines these components. The minishift binary, placed on the PATH for easy execution, is used to start, stop, and delete the CDK VM. The VM itself is bootstrapped off of a Red Hat Enterprise Linux Live ISO.

Some CDK commands, for example docker-env, interact with the Docker daemon, whilst others communicate with the OpenShift cluster, for example the openshift command.

Once the OpenShift cluster is up and running, you interact with it using the oc binary. CDK caches this binary under MINISHIFT_HOME (per default ~/.minishift). minishift oc-env is an easy way to add the oc binary to your PATH.

For more details about using CDK to manage your local OpenShift cluster, see the Interacting with OpenShift section.

Figure 3.1. : Minishift architecture

The runtime behavior of CDK can be controlled through flags, environment variables, and persistent configuration options.

The following precedence order is applied to control the behavior of CDK. Each action in the following list takes precedence over the action below it:

$ minishift setup-cdk --minishift-home ~/.mynewdir
$ export MINISHIFT_HOME=~/.mynewdir
$ echo 'export MINISHIFT_HOME=~/.mynewdir' >> ~/.bashrc

# Set default memory 4096 MB
$ minishift config set memory 4096

$ minishift start --insecure-registry hub.foo.com --insecure-registry hub.bar.com

$ minishift config set insecure-registry hub.foo.com,hub.bar.com

$ minishift config view
- memory: 4096

$ minishift config get memory
4096

$ minishift config unset memory

$ export XHYVE_EXPERIMENTAL_NFS_SHARE=true
$ minishift start --vm-driver xhyve

As part of the OpenShift cluster provisioning, 100 persistent volumes are created for your OpenShift cluster. This allows applications to make persistent volumes claims. The location of the persistent data is determined in the host-pv-dir flag of the minishift start command and defaults to /var/lib/minishift/openshift.local.pv on the CDK VM.

If you are behind an HTTP/HTTPS proxy, you need to supply proxy options to allow Docker and OpenShift to work properly. To do this, pass the required flags during minishift start.

For example:

$ minishift start --http-proxy http://YOURPROXY:PORT --https-proxy https://YOURPROXY:PORT

In an authenticated proxy environment, the proxy_user and proxy_password must be a part of proxy URI.

 $ minishift start --http-proxy http://<proxy_username>:<proxy_password>@YOURPROXY:PORT \
                   --https-proxy https://<proxy_username>:<proxy_password>@YOURPROXY:PORT

You can also use the --no-proxy flag to specify a comma-separated list of hosts that should not be proxied.

Using the proxy options will transparently configure the Docker daemon as well as OpenShift to use the specified proxies.

The CDK VM is exposed to the host system with a host-only IP address that can be obtained with the minishift ip command.

You can use the minishift ssh command to interact with the CDK VM.

You can run minishift ssh without a sub-command to open an interactive shell and run commands on the CDK VM in the same way that you run commands interactively on any remote machine using SSH.

You can also run minishift ssh with a sub-command to send the sub-command directly to the CDK VM and return the result to your local shell. For example:

$ minishift ssh -- docker ps
CONTAINER    IMAGE                   COMMAND                CREATED        STATUS        NAMES
71fe8ff16548 openshift/origin:v1.5.1 "/usr/bin/openshift s" 4 minutes ago  Up 4 minutes  origin

A profile is an instance of the Minishift VM along with all of its configuration and state. The profile feature allows you to create and manage these isolated instances of Minishift.

Each CDK profile is created with its own configuration (memory, CPU, disk size, add-ons, and so on) and completely independent of other profiles. Refer to the use of environment variables if you want to make sure that certain configuration settings, for example cpus or memory, get applied to all profiles.

The active profile is the profile against which all commands are executed, unless the global --profile flag is used. You can determine the active profile by using the minishift profile list command. You can execute single commands against a non-active profile by using the --profile flag, for example minishift --profile profile-demo console to open the OpenShift console for the specified profile-demo profile.

On top of the --profile flag, there are commands for listing, deleting and setting the active profile. These commands are described in the following sections.

There are two ways to create a new profile.

$ minishift --profile profile-demo start
-- Checking if requested hypervisor 'xhyve' is supported on this platform ... OK
-- Checking if xhyve driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-xhyve
   Checking for setuid bit ... OK
-- Checking the ISO URL ... OK
-- Starting local OpenShift cluster using 'xhyve' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    2 GB
   vCPUs :    2
   Disk size: 20 GB
...

$ minishift profile set demo
Profile 'demo' set as active profile

You can list all existing profiles with the minishift profile list command. You can also see the active profile highlighted in the output.

$ minishift profile list
- minishift     Running     	(Active)
- profile-demo  Does Not Exist

To switch between profiles use the minishift profile set command:

$ minishift profile set profile-demo
Profile 'profile-demo' set as active profile

To delete a profile, run:

$ minishift profile delete profile-demo
You are deleting the active profile. It will remove the VM and all related artifacts. Do you want to continue [y/N]?: y
Deleted:  /Users/john/.minishift/profiles/profile-demo
Profile 'profile-demo' deleted successfully
Switching to default profile 'minishift' as the active profile.

You have two options to create a new profile and configure its persistent configuration. The first option is to implicitly create the new profile by making it the active profile using the profile set command. Once the profile is active you can run any minishift config command. Lastly, start the instance:

$ minishift profile set profile-demo
$ minishift config set memory 8GB
$ minishift config set cpus 4
$ minishift addon enable anyuid
$ minishift start

The alternative is to execute a series of commands each specifying the targeted profile explicitly using the --profile flag:

$ minishift --profile profile-demo config set memory 8GB
$ minishift --profile profile-demo config set cpus 4
$ minishift --profile profile-demo addon enable anyuid
$ minishift --profile profile-demo minishift start

To speed up provisioning of the OpenShift cluster and to minimize network traffic, the core OpenShift images can be cached on the host. This feature is considered experimental and needs to be explicitly enabled.

To enable image caching you use the minishift config set command:

$ minishift config set image-caching true

Once enabled, caching occurs transparently, in a background process, the first time you use the minishift start command. Once the images are cached under $MINISHIFT_HOME/cache/images, successive CDK VM creations will use these cached images.

Each time an image exporting background process runs, a log file is generated under $MINISHIFT_HOME/logs which can be used to verify the progress of the export.

You can disable the caching of the OpenShift images by setting image-caching to false or removing the setting altogether using minishift config unset:

$ minishift config unset image-caching

CDK allows you to extend the vanilla OpenShift setup provided by cluster up with an add-on mechanism.

Add-ons are directories that contain a text file with the .addon extension. The directory can also contain other resource files such as JSON template files. However, only one .addon file is allowed per add-on.

The following example shows the contents of an add-on file, including the name and description of the add-on, additional metadata, and the actual add-on commands to apply.

# Name: anyuid                                                                          1
# Description: Allows authenticated users to run images under a non pre-allocated UID   2
# Required-Vars: ACME_TOKEN                                                             3
# OpenShift-Version: >3.6.0                                                             4

oc adm policy add-scc-to-group anyuid system:authenticated                              5

Enabled add-ons are applied during minishift start, immediately after the initial cluster provisioning is successfully completed.

As part of the add-on metadata you can specify the OpenShift version which needs to be running in order to apply the add-on. To do so you can specify the optional OpenShift-Version metadata field. The semantics are as follow:

This section describes the commands that an add-on file can contain. They are forming a mini-DSL for CDK add-ons:

CDK allows the use of variables within the add-on commands. Variables have the format #{<variable-name>}. The following example shows how the OpenShift routing suffix can be interpolated into an openshift command to create a new certificate as part of securing the OpenShift registry. The used variable #{routing-suffix} is part of the built-in add-on variables.

$ openshift admin ca create-server-cert \
  --signer-cert=/var/lib/origin/openshift.local.config/master/ca.crt \
  --signer-key=/var/lib/origin/openshift.local.config/master/ca.key \
  --signer-serial=/var/lib/origin/openshift.local.config/master/ca.serial.txt \
  --hostnames='docker-registry-default.#{routing-suffix},docker-registry.default.svc.cluster.local,172.30.1.1' \
  --cert=/etc/secrets/registry.crt \
  --key=/etc/secrets/registry.key

$ minishift addons apply --addon-env PROJECT_USER=john acme

$ minishift config set addon-env PROJECT_USER=john
$ minishift addons apply acme

$ minishift config set addon-env PROJECT_USER=env.USER        1
$ minishift addons apply acme                                 2

# Name: acme
# Description: ACME add-on
# Required-Vars: PROJECT_USER

CDK provides a set of built-in add-ons that offer some common OpenShift customization to assist with development. During minishift setup-cdk, minishift automatically installs and enables the xpaas, anyuid, and admin-user add-ons. To install the default add-ons, run:

$ minishift addons install --defaults

This command extracts the default add-ons to the add-on installation directory $MINISHIFT_HOME/addons. To view the list of installed add-ons, you can then run:

$ minishift addons list --verbose=true

This command prints a list of installed add-ons. You should at least see the anyuid add-on listed. This is an important add-on that allows you to run images that do not use a pre-allocated UID. By default, this is not allowed in OpenShift.

Add-ons are enabled with the minishift addons enable command and disabled with the minishift addons disable command. Enabled add-ons automatically get executed during minishift start.

The following examples show how to enable and disable the anyuid add-on.

$ minishift addons enable anyuid

$ minishift addons disable anyuid

$ minishift addons enable registry --priority=5

$ minishift addons list
- anyuid         : enabled    P(0)
- registry       : enabled    P(5)
- eap            : enabled    P(10)

Add-ons can be explicitly executed with the minishift addons apply command. You can use the apply command for both enabled and disabled add-ons. To apply multiple add-ons with a single command, specify add-on names separated by space.

The following example shows how to explicitly apply the anyuid and the admin-user add-ons.

$ minishift addons apply anyuid admin-user

Add-ons can be removed with the minishift addons remove command. It is the mirror command to minishift addons apply and similarly can be used regardless of whether the add-on is enabled or not. Provided the specified add-on is installed and has a <addon_name>.addon.remove file, minishift addons remove will execute the commands specified in this file.

To remove multiple add-ons with a single command, specify the add-on names separated by a space. The following example shows how to explicitly remove the admin-user add-on.

$ minishift addons remove admin-user
-- Removing addon 'admin-user':.
admin user deleted

Add-ons can be uninstalled with the minishift addons uninstall command. It is the mirror command to minishift addons install and can be used regardless of whether the add-on is enabled or not. Provided the specified add-on is installed, minishift addons uninstall will delete the corresponding add-on directory from $MINISHIFT_HOME/addons.

The following example shows how to explicitly uninstall the admin-user add-on:

$ minishift addons uninstall admin-user
Add-on 'admin-user' uninstalled

To write a custom add-on, you should create a directory and in it create one (and only one) text file with the extension .addon, for example admin-role.addon.

This file needs to contain the Name and Description metadata fields, as well as the commands that you want to execute as a part of the add-on.

The following example shows the definition of an add-on that gives the developer user cluster-admin privileges.

# Name: admin-role
# Description: Gives the developer user cluster-admin privileges

oc adm policy add-role-to-user cluster-admin developer

After you define the add-on, you can install it by running:

$ minishift addons install <ADDON_DIR_PATH>

Host folders are directories on the host which are shared between the host and the CDK VM. This allows for two way file synchronization between the host and the VM. The following sections discuss usage of the minishift hostfolder command.

CDK provides the minishift hostfolder command to list, add, mount, unmount and remove host folders. You can use the hostfolder command to mount multiple shared folders onto custom specified mount points.

$ minishift hostfolder list
Name        Mountpoint            Remote path              Mounted
myshare     /mnt/sda1/myshare     //192.168.1.82/MYSHARE   N

$ minishift hostfolder add myshare 1
UNC path: //192.168.99.1/MYSHARE   2
Mountpoint [/mnt/sda1/myshare]:    3
Username: john                     4
Password: [HIDDEN]                 5
Domain:                            6
Added: myshare

$ minishift hostfolder mount myshare
Mounting 'myshare': '//192.168.99.1/MYSHARE' as '/mnt/sda1/myshare' ... OK

$ minishift hostfolder list
Name       Mountpoint          Remote path              Mounted
myshare    /mnt/sda1/myshare   //192.168.99.1/MYSHARE   Y

$ minishift ssh "ls -al /mnt/sda1/myshare"

$ minishift config set hostfolders-automount true

$ minishift hostfolder umount myshare
Unmounting 'myshare' ... OK

$ minishift hostfolder list
Name       Mountpoint          Remote path              Mounted
myshare    /mnt/sda1/myshare   //192.168.99.1/MYSHARE   N

$ minishift hostfolder list
Name        Mountpoint            Remote path              Mounted
myshare     /mnt/sda1/myshare     //192.168.1.82/MYSHARE   N

$ minishift hostfolder remove myshare
Removed: myshare

$ minishift hostfolder list
No host folders defined

When running OpenShift in a single VM, you can reuse the docker daemon managed by CDK for other docker use-cases as well. By using the same docker daemon as the one used in CDK, you can speed up your local development.

In order to configure your console to reuse CDK docker daemon, follow these steps:

If you want to get early access to some upcoming features and experiment, you can set the environment variable MINISHIFT_ENABLE_EXPERIMENTAL, which makes additional features flags available:

$ export MINISHIFT_ENABLE_EXPERIMENTAL=y

By default, CDK does not expose all oc cluster up flags in the CDK CLI.

You can set the MINISHIFT_ENABLE_EXPERIMENTAL environment variable to enable the following options for the minishift start command:

Since the Internal Virtual Switch for Hyper-V does not provide a DHCP offer option, an IP address needs to be provided in a different way. For Hyper-V a functionality is provided to assign an IP address on startup using the Data Exchange Service.

To make this work you need to create a Virtual Switch using NAT

The following command will attempt to assign an IP address for use on the Internal Virtual Switch 'MyInternal':

PS> $env:MINISHIFT_ENABLE_EXPERIMENTAL="y"
PS> $env:HYPERV_VIRTUAL_SWITCH="MyInternal"
PS> minishift.exe start `
  --network-ipaddress 192.168.1.10 `
  --network-gateway 192.168.1.1 `
  --network-nameserver 8.8.8.8

If you want to use the 'DockerNAT' network, the following commands are needed to setup the correct NAT networking and assigning an IP in the range expected:

PS> New-NetNat -Name SharedNAT -InternalIPInterfaceAddressPrefix 10.0.75.1/24
PS> $env:MINISHIFT_ENABLE_EXPERIMENTAL="y"
PS> $env:HYPERV_VIRTUAL_SWITCH="DockerNAT"
PS> minishift.exe start `
  --network-ipaddress 10.0.75.128 `
  --network-gateway 10.0.75.1 `
  --network-nameserver 8.8.8.8