OpenShift sandboxed containers user guide

Procedure

1. Check the current limit by running the following command:

   $ oc get peerpodconfig peerpodconfig-openshift -n openshift-sandboxed-containers-operator \
   -o jsonpath='{.spec.limit}{"\n"}'

   Copy to Clipboard Toggle word wrap

2. Modify the limit attribute of the peerpodConfig CR by running the following command:

   $ oc patch peerpodconfig peerpodconfig-openshift -n openshift-sandboxed-containers-operator \
   --type merge --patch '{"spec":{"limit":"<value>"}}' 

   1

   Copy to Clipboard Toggle word wrap

   1

   Replace <value> with the limit you want to define.

Procedure

1. Retrieve the Instance ID:

   $ INSTANCE_ID=$(oc get nodes -l 'node-role.kubernetes.io/worker' -o jsonpath='{.items[0].spec.providerID}' | sed 's#[^ ]*/##g')

   Copy to Clipboard Toggle word wrap

2. Retrieve the AWS region:

   $ AWS_REGION=$(oc get infrastructure/cluster -o jsonpath='{.status.platformStatus.aws.region}')

   Copy to Clipboard Toggle word wrap

3. Retrieve the security groups:

   $ SG=$(aws ec2 describe-instances --instance-ids ${INSTANCE_ID} --query 'Reservations[*].Instances[*].SecurityGroups[*].GroupId' --output text --region ${AWS_REGION})

   Copy to Clipboard Toggle word wrap

4. Authorize the peer-pods shim and to access kata-agent communication. Run the following command:

   $ aws ec2 authorize-security-group-ingress --group-id ${SG} --protocol tcp --port 15150 --source-group ${SG} --region ${AWS_REGION}

   Copy to Clipboard Toggle word wrap

5. Set up the peer-pods tunnel. Run the following command:

   $ aws ec2 authorize-security-group-ingress --group-id ${SG} --protocol tcp --port 9000 --source-group ${SG} --region ${AWS_REGION}

   Copy to Clipboard Toggle word wrap

Procedure

1. From the Administrator perspective in the web console, navigate to Operators → Installed Operators.
2. Select the OpenShift sandboxed containers Operator from the list of operators.
3. Click the Import icon (+) in the top right corner.

4. In the Import YAML window, paste the following YAML manifest:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     AWS_ACCESS_KEY_ID: "<enter value>" 

   1

   
     AWS_SECRET_ACCESS_KEY: "<enter value>" 

   2

   
     AWS_REGION: "<enter value>" 

   3

   
     AWS_SUBNET_ID: "<enter value>" 

   4

   
     AWS_VPC_ID: "<enter value>" 

   5

   
     AWS_SG_IDS: "<enter value>" 

   6

   Copy to Clipboard Toggle word wrap

   1

   Enter the AWS_ACCESS_KEY_ID value you prepared before you began.

   2

   Enter the AWS_SECRET_ACCESS_KEY value you prepared before you began.

   3

   Enter the AWS_REGION value you retrieved.

   4

   Enter the AWS_SUBNET_ID value you retrieved.

   5

   Enter the AWS_VPC_ID value you retrieved.

   6

   Enter the AWS_SG_IDS value you retrieved.

5. Click Create.

Procedure

1. From the Administrator perspective in the web console, navigate to Operators → Installed Operators.
2. Select the OpenShift sandboxed containers Operator from the list of operators.
3. Click the Import icon (+) in the top right corner.

4. In the Import YAML window, paste the following YAML manifest:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: peer-pods-cm
     namespace: openshift-sandboxed-containers-operator
   data:
     CLOUD_PROVIDER: "aws"
     VXLAN_PORT: "9000"
     PODVM_INSTANCE_TYPE: "t3.medium" 

   1

   
     PODVM_INSTANCE_TYPES: "t2.small,t2.medium,t3.large" 

   2

   
     PROXY_TIMEOUT: "5m"

   Copy to Clipboard Toggle word wrap

   1

   Defines the default instance type that is used when a type is not defined in the workload.

   2

   Lists all of the instance types you can specify when creating the pod. This allows you to define smaller instances for workloads that need less memory and CPU, or larger instances for larger workloads.

5. Click Create.

Procedure

1. From the Administrator perspective in the web console, navigate to Operators → Installed Operators.
2. Select the OpenShift sandboxed containers Operator from the list of operators.
3. Click the Import icon (+) in the top right corner.

4. In the Import YAML window, paste the following YAML manifest:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     AZURE_CLIENT_ID: "<enter value>" 

   1

   
     AZURE_CLIENT_SECRET: "<enter value>" 

   2

   
     AZURE_TENANT_ID: "<enter value>" 

   3

   
     AZURE_SUBSCRIPTION_ID: "<enter value>" 

   4

   
     AZURE_REGION: "<enter value>" 

   5

   
     AZURE_RESOURCE_GROUP: "<enter value>" 

   6

   Copy to Clipboard Toggle word wrap

   1

   Enter the AZURE_CLIENT_ID value you generated in the RBAC file.

   2

   Enter the AZURE_CLIENT_SECRET value you generated in the RBAC file.

   3

   Enter the AZURE_TENANT_ID value you generated in the RBAC file.

   4

   Enter the AZURE_SUBSCRIPTION_ID value you retrieved.

   5

   Enter the AZURE_REGION value you retrieved.

   6

   Enter the AZURE_RESOURCE_GROUP value you retrieved.

5. Click Create.

Procedure

1. From the Administrator perspective in the web console, navigate to Operators → Installed Operators.
2. Select the OpenShift sandboxed containers Operator from the list of operators.
3. Click the Import icon (+) in the top right corner.

4. In the Import YAML window, paste the following YAML manifest:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: peer-pods-cm
     namespace: openshift-sandboxed-containers-operator
   data:
     CLOUD_PROVIDER: "azure"
     VXLAN_PORT: "9000"
     AZURE_INSTANCE_SIZE: "Standard_B2als_v2" 

   1

   
     AZURE_INSTANCE_SIZES: "Standard_B2als_v2,Standard_D2as_v5,Standard_D4as_v5,Standard_D2ads_v5" 

   2

   
     AZURE_SUBNET_ID: "<enter value>" 

   3

   
     AZURE_NSG_ID: "<enter value>" 

   4

   
     PROXY_TIMEOUT: "5m"
     DISABLECVM: "true"

   Copy to Clipboard Toggle word wrap

   1

   Defines the default instance size that is used when an instance is not defined in the workload.

   2

   List all of the instance sizes you can specify when creating the pod. This allows you to define smaller instances for workloads that need less memory and CPU, or larger instances for larger workloads.

   3

   Enter the AZURE_SUBNET_ID value that you retrieved.

   4

   Enter the AZURE_NSG_ID value that you retrieved.

5. Click Create.

Procedure

1. From the Administrator perspective in the web console, navigate to Workloads → Secrets.
2. In the Secrets window, in the top left corner, verify that you are in the openshift-sandboxed-containers-operator project.
3. Click Create and select Key/value secret from the list.
4. In the Secret name field, enter ssh-key-secret.
5. In the Key field, enter id_rsa.pub.
6. In the Value field, paste your public SSH key.
7. Click Create.

Procedure

1. Collect the necessary parameter values for the secret object. Make sure to write down each value.

   1. Retrieve the instance ID:

      $ INSTANCE_ID=$(oc get nodes -l 'node-role.kubernetes.io/worker' -o jsonpath='{.items[0].spec.providerID}' | sed 's#[^ ]*/##g')

      Copy to Clipboard Toggle word wrap

      This value is not needed for the secret object itself, but is used to retrieve other values for the secret object.

   2. Retrieve the AWS region:

      $ AWS_REGION=$(oc get infrastructure/cluster -o jsonpath='{.status.platformStatus.aws.region}') && echo "AWS_REGION: \"$AWS_REGION\""

      Copy to Clipboard Toggle word wrap

   3. Retrieve the AWS subnet ID:

      $ AWS_SUBNET_ID=$(aws ec2 describe-instances --instance-ids ${INSTANCE_ID} --query 'Reservations[*].Instances[*].SubnetId' --region ${AWS_REGION} --output text) && echo "AWS_SUBNET_ID: \"$AWS_SUBNET_ID\""

      Copy to Clipboard Toggle word wrap

   4. Retrieve the AWS VPC ID:

      $ AWS_VPC_ID=$(aws ec2 describe-instances --instance-ids ${INSTANCE_ID} --query 'Reservations[*].Instances[*].VpcId' --region ${AWS_REGION} --output text) && echo "AWS_VPC_ID: \"$AWS_VPC_ID\""

      Copy to Clipboard Toggle word wrap

   5. Retrieve the AWS security group IDs:

      $ AWS_SG_IDS=$(aws ec2 describe-instances --instance-ids ${INSTANCE_ID} --query 'Reservations[*].Instances[*].SecurityGroups[*].GroupId' --region $AWS_REGION --output json | jq -r '.[][][]' | paste -sd ",") && echo "AWS_SG_IDS: \"$AWS_SG_IDS\""

      Copy to Clipboard Toggle word wrap

2. Create a YAML file with the following manifest:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     AWS_ACCESS_KEY_ID: "<enter value>" 

   1

   
     AWS_SECRET_ACCESS_KEY: "<enter value>" 

   2

   
     AWS_REGION: "<enter value>" 

   3

   
     AWS_SUBNET_ID: "<enter value>" 

   4

   
     AWS_VPC_ID: "<enter value>" 

   5

   
     AWS_SG_IDS: "<enter value>" 

   6

   Copy to Clipboard Toggle word wrap

   1

   Enter the AWS_ACCESS_KEY_ID value you prepared before you began.

   2

   Enter the AWS_SECRET_ACCESS_KEY value you prepared before you began.

   3

   Enter the AWS_REGION value you retrieved.

   4

   Enter the AWS_SUBNET_ID value you retrieved.

   5

   Enter the AWS_VPC_ID value you retrieved.

   6

   Enter the AWS_SG_IDS value you retrieved.

3. Apply the secret object:

   $ oc apply -f peer-pods-secret.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a YAML file with the following manifest:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: peer-pods-cm
     namespace: openshift-sandboxed-containers-operator
   data:
     CLOUD_PROVIDER: "aws"
     VXLAN_PORT: "9000"
     PODVM_INSTANCE_TYPE: "t3.medium" 

   1

   
     PODVM_INSTANCE_TYPES: "t2.small,t2.medium,t3.large" 

   2

   
     PROXY_TIMEOUT: "5m"

   Copy to Clipboard Toggle word wrap

   1

   Defines the default instance type that is used when a type is not defined in the workload.

   2

   Lists all of the instance types you can specify when creating the pod. This allows you to define smaller instances for workloads that need less memory and CPU, or larger instances for larger workloads.

2. Apply the ConfigMap:

   $ oc apply -f peer-pods-cm.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Retrieve the Azure subscription ID:

   $ AZURE_SUBSCRIPTION_ID=$(az account list --query "[?isDefault].id" -o tsv) && echo "AZURE_SUBSCRIPTION_ID: \"$AZURE_SUBSCRIPTION_ID\""

   Copy to Clipboard Toggle word wrap

2. Generate the RBAC content. This generates the client ID, client secret, and the tenant ID:

   $ az ad sp create-for-rbac --role Contributor --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID --query "{ client_id: appId, client_secret: password, tenant_id: tenant }

   Copy to Clipboard Toggle word wrap

   The following output appears:

   {
     "client_id": `AZURE_CLIENT_ID`,
     "client_secret": `AZURE_CLIENT_SECRET`,
     "tenant_id": `AZURE_TENANT_ID`
   }

   Copy to Clipboard Toggle word wrap
3. Save the values in the RBAC output to use in the secret object.

4. Collect the additional parameter values for the secret object. Make sure to write down each value.

   1. Retrieve the resource group:

      $ AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster -o jsonpath='{.status.platformStatus.azure.resourceGroupName}') && echo "AZURE_RESOURCE_GROUP: \"$AZURE_RESOURCE_GROUP\""

      Copy to Clipboard Toggle word wrap

   2. Retrieve the Azure region:

      $ AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP} --query "{Location:location}" --output tsv) && echo "AZURE_REGION: \"$AZURE_REGION\""

      Copy to Clipboard Toggle word wrap

5. Create a YAML file with the following manifest:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     AZURE_CLIENT_ID: "<enter value>" 

   1

   
     AZURE_CLIENT_SECRET: "<enter value>" 

   2

   
     AZURE_TENANT_ID: "<enter value>" 

   3

   
     AZURE_SUBSCRIPTION_ID: "<enter value>" 

   4

   
     AZURE_REGION: "<enter value>" 

   5

   
     AZURE_RESOURCE_GROUP: "<enter value>" 

   6

   Copy to Clipboard Toggle word wrap

   1

   Enter the AZURE_CLIENT_ID value you generated in the RBAC file.

   2

   Enter the AZURE_CLIENT_SECRET value you generated in the RBAC file.

   3

   Enter the AZURE_TENANT_ID value you generated in the RBAC file.

   4

   Enter the AZURE_SUBSCRIPTION_ID value you retrieved.

   5

   Enter the AZURE_REGION value you retrieved.

   6

   Enter the AZURE_RESOURCE_GROUP value you retrieved.

6. Apply the secret object:

   $ oc apply -f peer-pods-secret.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Collect the configuration values for the Azure peer-pod ConfigMap. Make sure to write down each value.

   1. Retrieve the Azure VNet name:

      $ AZURE_VNET_NAME=$(az network vnet list --resource-group ${AZURE_RESOURCE_GROUP} --query "[].{Name:name}" --output tsv)

      Copy to Clipboard Toggle word wrap

      This value is not needed for the ConfigMap, but is used to retrieve the Azure subnet ID.

   2. Retrieve the Azure subnet ID:

      $ AZURE_SUBNET_ID=$(az network vnet subnet list --resource-group ${AZURE_RESOURCE_GROUP} --vnet-name $AZURE_VNET_NAME --query "[].{Id:id} | [? contains(Id, 'worker')]" --output tsv) && echo "AZURE_SUBNET_ID: \"$AZURE_SUBNET_ID\""

      Copy to Clipboard Toggle word wrap

   3. Retrieve the Azure network security group (NSG) ID:

      $ AZURE_NSG_ID=$(az network nsg list --resource-group ${AZURE_RESOURCE_GROUP} --query "[].{Id:id}" --output tsv) && echo "AZURE_NSG_ID: \"$AZURE_NSG_ID\""

      Copy to Clipboard Toggle word wrap

2. Create a YAML file with the following manifest:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: peer-pods-cm
     namespace: openshift-sandboxed-containers-operator
   data:
     CLOUD_PROVIDER: "azure"
     VXLAN_PORT: "9000"
     AZURE_INSTANCE_SIZE: "Standard_B2als_v2" 

   1

   
     AZURE_INSTANCE_SIZES: "Standard_B2als_v2,Standard_D2as_v5,Standard_D4as_v5,Standard_D2ads_v5" 

   2

   
     AZURE_SUBNET_ID: "<enter value>" 

   3

   
     AZURE_NSG_ID: "<enter value>" 

   4

   
     PROXY_TIMEOUT: "5m"
     DISABLECVM: "true"

   Copy to Clipboard Toggle word wrap

   1

   Defines the default instance size that is used when an instance is not defined in the workload.

   2

   List all of the instance sizes you can specify when creating the pod. This allows you to define smaller instances for workloads that need less memory and CPU, or larger instances for larger workloads.

   3

   Enter the AZURE_SUBNET_ID value that you retrieved.

   4

   Enter the AZURE_NSG_ID value that you retrieved.

3. Apply the ConfigMap:

   $ oc apply -f peer-pods-cm.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Generate the SSH key:

   $ ssh-keygen -f ./id_rsa -N ""

   Copy to Clipboard Toggle word wrap

2. Create the SSH secret object:

   $ oc create secret generic ssh-key-secret -n openshift-sandboxed-containers-operator --from-file=id_rsa.pub=./id_rsa.pub --from-file=id_rsa=./id_rsa

   Copy to Clipboard Toggle word wrap

Procedure

1. Log in to IBM Z KVM host and set the shell variables to be used by libvirt for storage management.

   1. Set the name of the libvirt pool by running the following command:

      $ export LIBVIRT_POOL=<name_of_libvirt_pool_to_create>

      Copy to Clipboard Toggle word wrap

   2. Set the name of the libvirt pool by running the following command:

      $ export LIBVIRT_VOL_NAME=<name_of_libvirt_volume_to_create>

      Copy to Clipboard Toggle word wrap

   3. Set the path of the default storage pool location, by running the following command:

      $ export LIBVIRT_POOL_DIRECTORY=<name_of_target_directory> 

      1

      Copy to Clipboard Toggle word wrap

      1

      To ensure libvirt has read and write access permissions, use a subdirectory of libvirt’s storage directory. The default is /var/lib/libvirt/images/.

2. Create a libvirt pool by running the following command:

   $ virsh pool-define-as $LIBVIRT_POOL --type dir --target "$LIBVIRT_POOL_DIRECTORY"

   Copy to Clipboard Toggle word wrap

3. Start the libvirt pool by running the following command:

   $ virsh pool-start $LIBVIRT_POOL

   Copy to Clipboard Toggle word wrap

4. Create a libvirt volume for the pool by running the following command:

   $ virsh -c qemu:///system \
     vol-create-as --pool $LIBVIRT_POOL \
     --name $LIBVIRT_VOL_NAME \
     --capacity 20G \
     --allocation 2G \
     --prealloc-metadata \
     --format qcow2

   Copy to Clipboard Toggle word wrap

Procedure

1. Set the RHEL ORG_ID and ACTIVATION_KEY shell variables for your system.

   1. When using a subscribed RHEL system, set the shell environment variables to the files holding the organization ID and the activation key for Red Hat Subscription Management (RHSM) by running the following commands:

      $ export ORG_ID=$(cat ~/.rh_subscription/orgid)
      
      $ export ACTIVATION_KEY=$(cat ~/.rh_subscription/activation_key)

      Copy to Clipboard Toggle word wrap

   2. When using an unsubscribed RHEL system, set the appropriate subscription values by running the following commands:

      $ export ORG_ID=<RHEL_ORGID_VALUE>
      
      $ export ACTIVATION_KEY=<RHEL_ACVTIVATION_KEY>

      Copy to Clipboard Toggle word wrap

2. Log in to your IBM Z system and perform the following steps:

   1. Download an s390x RHEL KVM guest image from the Red Hat Customer Portal to your libvirt storage directory to grant libvirt correct access. The default directory is /var/lib/libvirt/images. The image is used to generate the peer pod VM image, which will include the relevant binaries.

   2. Set the IMAGE_URL shell environment variable for the downloaded image by running the following command:

      $ export IMAGE_URL=<location_of_downloaded_KVM_guest_image> 

      1

      Copy to Clipboard Toggle word wrap

      1

      Enter the path of the KVM guest image that you downloaded in the previous step.

   3. Register the guest KVM image by running the following command:

      $ export REGISTER_CMD="subscription-manager register --org=${ORG_ID} \
        --activationkey=${ACTIVATION_KEY}"

      Copy to Clipboard Toggle word wrap

   4. Customize the guest KVM image by running the following command:

      $ virt-customize -v -x -a ${IMAGE_URL} --run-command "${REGISTER_CMD}"

      Copy to Clipboard Toggle word wrap

   5. Set the checksum of the image by running the following command:

      $ export IMAGE_CHECKSUM=$(sha256sum ${IMAGE_URL} | awk '{ print $1 }')

      Copy to Clipboard Toggle word wrap

Procedure

1. Create a YAML file peer-pods-secret.yaml with the following manifest:

   apiVersion: v1
   kind: Secret
   metadata:
     name: peer-pods-secret
     namespace: openshift-sandboxed-containers-operator
   type: Opaque
   stringData:
     CLOUD_PROVIDER: "libvirt" 

   1

   
     LIBVIRT_URI: "<libvirt_gateway_uri>" 

   2

   
     LIBVIRT_POOL: "<libvirt_pool>" 

   3

   
     LIBVIRT_VOL_NAME: "<libvirt_volume>" 

   4

   Copy to Clipboard Toggle word wrap

   1

   Enter libvirt as the cloud provider.

   2

   Enter the libvirt_gateway_uri value you retrieved.

   3

   Enter the libvirt_pool value you retrieved.

   4

   Enter the libvirt_volume value you retrieved.

2. Create the secret object:

   $ oc apply -f peer-pods-secret.yaml

   Copy to Clipboard Toggle word wrap