Chapter 4. Deploying workloads on IBM

Procedure

1. Log in to the KVM host.

2. Set the name of the libvirt pool by running the following command:

   $ export LIBVIRT_POOL=<libvirt_pool>

   You need the LIBVIRT_POOL value to create the secret for the libvirt provider.

3. Set the name of the libvirt pool by running the following command:

   $ export LIBVIRT_VOL_NAME=<libvirt_volume>

   You need the LIBVIRT_VOL_NAME value to create the secret for the libvirt provider.

4. Set the path of the default storage pool location, by running the following command:

   $ export LIBVIRT_POOL_DIRECTORY=<target_directory> 1

   1

   To ensure libvirt has read and write access permissions, use a subdirectory of the libvirt storage directory. The default is /var/lib/libvirt/images/.

5. Create a libvirt pool by running the following command:

   $ virsh pool-define-as $LIBVIRT_POOL --type dir --target "$LIBVIRT_POOL_DIRECTORY"

6. Start the libvirt pool by running the following command:

   $ virsh pool-start $LIBVIRT_POOL

7. Create a libvirt volume for the pool by running the following command:

   $ virsh -c qemu:///system \
     vol-create-as --pool $LIBVIRT_POOL \
     --name $LIBVIRT_VOL_NAME \
     --capacity 20G \
     --allocation 2G \
     --prealloc-metadata \
     --format qcow2

Procedure

1. Log in to your OpenShift Container Platform cluster.

2. If you have a RHEL subscription, set the subscription environment variables for Red Hat Subscription Management:

   • Set the organization ID by running the following command:

     $ export ORG_ID=$(cat ~/.rh_subscription/orgid)

   • Set the activation key by running the following command:

     $ export ACTIVATION_KEY=$(cat ~/.rh_subscription/activation_key)

3. If you do not have a RHEL subscription, set the subscription values for RHEL:

   • Set the organization ID by running the following command:

     $ export ORG_ID=<RHEL_ORGID_VALUE> 1

     1

     Specify your RHEL organization ID.

   • Set the activation key by running the following command:

     $ export ACTIVATION_KEY=<RHEL_ACTIVATION_KEY> 1

     1

     Specify your RHEL activation key.

4. Log in to your IBM Z® system.

5. Download the s390x RHEL KVM guest image from the Red Hat Customer Portal to your libvirt storage directory to grant libvirt correct access.

   The default directory is /var/lib/libvirt/images. This image is used to generate the peer pod VM image, which includes the relevant binaries.

6. Set the IMAGE_URL for the downloaded image by running the following command:

   $ export IMAGE_URL=<path/to/image> 1

   1

   Specify the path of the KVM guest image.

7. Register the guest KVM image by running the following command:

   $ export REGISTER_CMD="subscription-manager register --org=${ORG_ID} \
     --activationkey=${ACTIVATION_KEY}"

8. Customize the guest KVM image by running the following command:

   $ virt-customize -v -x -a ${IMAGE_URL} --run-command "${REGISTER_CMD}"

9. Set the checksum of the image by running the following command:

   $ export IMAGE_CHECKSUM=$(sha256sum ${IMAGE_URL} | awk '{ print $1 }')

Procedure

1. Log in to your OpenShift Container Platform cluster.

2. Clone the cloud-api-adaptor repository by running the following command:

   $ git clone --single-branch https://github.com/confidential-containers/cloud-api-adaptor.git

3. Change into the podvm directory by running the following command:

   $ cd cloud-api-adaptor && git checkout 8577093

4. Create a builder image from which the final QCOW2 image is generated.

   • If you have a subscribed RHEL system, run the following command:

     $ podman build -t podvm_builder_rhel_s390x \
       --build-arg ARCH="s390x" \
       --build-arg GO_VERSION="1.21.3" \
       --build-arg PROTOC_VERSION="25.1" \
       --build-arg PACKER_VERSION="v1.9.4" \
       --build-arg RUST_VERSION="1.72.0" \
       --build-arg YQ_VERSION="v4.35.1" \
       --build-arg YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" \
       -f podvm/Dockerfile.podvm_builder.rhel .

   • If you have an unsubscribed RHEL system, run the following command:

     $ podman build -t podvm_builder_rhel_s390x \
       --build-arg ORG_ID=$ORG_ID \
       --build-arg ACTIVATION_KEY=$ACTIVATION_KEY \
       --build-arg ARCH="s390x" \
       --build-arg GO_VERSION="1.21.3" \
       --build-arg PROTOC_VERSION="25.1" \
       --build-arg PACKER_VERSION="v1.9.4" \
       --build-arg RUST_VERSION="1.72.0" \
       --build-arg YQ_VERSION="v4.35.1" \
       --build-arg YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" \
       -f podvm/Dockerfile.podvm_builder.rhel .

5. Generate an intermediate image package with the required binaries for running peer pods by running the following command:

   $ podman build -t podvm_binaries_rhel_s390x \
     --build-arg BUILDER_IMG="podvm_builder_rhel_s390x:latest" \
     --build-arg ARCH=s390x \
     -f podvm/Dockerfile.podvm_binaries.rhel .

   This process takes a significant length of time.

6. Extract the binaries and build the peer pod QCOW2 image by running the following command:

   $ podman build -t podvm_rhel_s390x \
     --build-arg ARCH=s390x \
     --build-arg CLOUD_PROVIDER=libvirt \
     --build-arg BUILDER_IMG="localhost/podvm_builder_rhel_s390x:latest" \
     --build-arg BINARIES_IMG="localhost/podvm_binaries_rhel_s390x:latest" \
     -v ${IMAGE_URL}:/tmp/rhel.qcow2:Z \
     --build-arg IMAGE_URL="/tmp/rhel.qcow2" \
     --build-arg IMAGE_CHECKSUM=${IMAGE_CHECKSUM} \
     -f podvm/Dockerfile.podvm.rhel .

7. Create an image directory environment variable by running the following command:

   $ export IMAGE_OUTPUT_DIR=<image_output_directory> 1

   1

   Specify a directory for the image.

8. Create the image directory by running the following command:

   $ mkdir -p $IMAGE_OUTPUT_DIR

9. Save the extracted peer pod QCOW2 image by running the following command:

   $ podman save podvm_rhel_s390x | tar -xO --no-wildcards-match-slash '*.tar' | tar -x -C ${IMAGE_OUTPUT_DIR}

10. Upload the peer pod QCOW2 image to your libvirt volume:

    $ virsh -c qemu:///system vol-upload \
      --vol $LIBVIRT_VOL_NAME \
      $IMAGE_OUTPUT_DIR/podvm-*.qcow2 \
      --pool $LIBVIRT_POOL --sparse

Procedure

1. Create a peer-pods-secret.yaml manifest file according to the following example:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     CLOUD_PROVIDER: "libvirt"
     LIBVIRT_URI: "<libvirt_gateway_uri>" 1
     LIBVIRT_POOL: "<libvirt_pool>" 2
     LIBVIRT_VOL_NAME: "<libvirt_volume>" 3

   1

   Specify the libvirt URI.

   2

   Specify the libvirt pool.

   3

   Specify the libvirt volume name.

2. Create the secret object by applying the manifest:

   $ oc apply -f peer-pods-secret.yaml

Procedure

1. Create a peer-pods-cm.yaml manifest according to the following example:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: peer-pods-cm
     namespace: openshift-sandboxed-containers-operator
   data:
     CLOUD_PROVIDER: "libvirt"
     PROXY_TIMEOUT: "15m"

2. Apply the manifest to create a config map:

   $ oc apply -f peer-pods-cm.yaml

   A config map is created for your libvirt provider.

Procedure

1. Log in to your OpenShift Container Platform cluster.

2. Generate an SSH key pair by running the following command:

   $ ssh-keygen -f ./id_rsa -N ""

3. Copy the public SSH key to your KVM host:

   $ ssh-copy-id -i ./id_rsa.pub <KVM_HOST_IP>

4. Create the Secret object by running the following command:

   $ oc create secret generic ssh-key-secret \
       -n openshift-sandboxed-containers-operator \
       --from-file=id_rsa.pub=./id_rsa.pub \
       --from-file=id_rsa=./id_rsa

   The SSH key secret is created.

5. Delete the SSH keys you created:

   $ shred -remove id_rsa.pub id_rsa

Procedure

1. Create a cluster-kataconfig.yaml manifest file according to the following example:

   apiVersion: kataconfiguration.openshift.io/v1
   kind: KataConfig
   metadata:
     name: cluster-kataconfig
   spec:
     enablePeerPods: true
     logLevel: info

2. Optional: To install kata-remote on selected nodes, specify the node labels according to the following example:

   apiVersion: kataconfiguration.openshift.io/v1
   kind: KataConfig
   metadata:
     name: cluster-kataconfig
   spec:
     kataConfigPoolSelector:
       matchLabels:
         <label_key>: '<label_value>' 1
   # ...

   1

   Specify the labels of the selected nodes.

3. Create the KataConfig CR:

   $ oc create -f cluster-kataconfig.yaml

   The new KataConfig CR is created and installs kata-remote as a runtime class on the worker nodes.

   Wait for the kata-remote installation to complete and the worker nodes to reboot before verifying the installation.

Procedure

1. Check the current limit by running the following command:

   $ oc get peerpodconfig peerpodconfig-openshift -n openshift-sandboxed-containers-operator \
   -o jsonpath='{.spec.limit}{"\n"}'

2. Modify the limit attribute of the peerpodConfig CR by running the following command:

   $ oc patch peerpodconfig peerpodconfig-openshift -n openshift-sandboxed-containers-operator \
   --type merge --patch '{"spec":{"limit":"<value>"}}' 1

   1

   Replace <value> with the limit you want to define.

Procedure

1. Add spec.runtimeClassName: kata-remote to the manifest of each pod-templated workload object as in the following example:

   apiVersion: v1
   kind: <object>
   # ...
   spec:
     runtimeClassName: kata-remote
   # ...

   OpenShift Container Platform creates the workload object and begins scheduling it.