Chapter 2. Deploying confidential containers on Azure
You deploy confidential containers on a Red Hat OpenShift Container Platform cluster on Microsoft Azure Cloud Computing Services for your workloads.
You deploy confidential containers by performing the following steps:
- Configure outbound connections.
- Install the OpenShift sandboxed containers Operator.
- Enable the confidential containers feature gate.
-
Optional: If you pull a peer pod VM image from a private registry such as
registry.access.redhat.com
, configure the pull secret for peer pods. - Create initdata. See About initdata for details.
- Create the peer pods config map.
- Optional: Apply initdata to a peer pod.
-
Create the
KataConfig
CR. - Verify the attestation process.
2.1. Prerequisites Copy linkLink copied to clipboard!
- You have installed Red Hat OpenShift Container Platform 4.16 or later on the cluster where you are running your confidential containers workload.
- You have deployed Red Hat build of Trustee on an OpenShift Container Platform cluster in a trusted environment. For more information, see Deploying Red Hat build of Trustee.
- You have enabled ports 15150 and 9000 for communication in the subnet used for worker nodes and the pod virtual machine (VM). The ports enable communication between the Kata shim running on the worker node and the Kata agent running on the pod VM.
- You have configured outbound connectivity for the pod VM subnet.
2.2. Configuring outbound connections Copy linkLink copied to clipboard!
To enable peer pods to communicate with external networks, such as the public internet, you must configure outbound connectivity for the pod virtual machine (VM) subnet. This involves setting up a NAT gateway and, optionally, defining how the subnet integrates with your cluster’s virtual network (VNet) in Azure.
- Peer pods and subnets
- Peer pods operate in a dedicated Azure subnet that requires explicit configuration for outbound access. This subnet can either be the default worker subnet used by OpenShift Container Platform nodes or a separate, custom subnet created specifically for peer pods.
- VNet peering
- When using a separate subnet, VNet peering connects the peer pod VNet to the cluster’s VNet, ensuring internal communication while maintaining isolation. This requires non-overlapping CIDR ranges between the VNets.
You can configure outbound connectivity in two ways:
- Default worker subnet: Modify the existing worker subnet to include a NAT gateway. This is simpler and reuses cluster resources, but it offers less isolation.
- Peer pod VNet: Set up a dedicated VNet and subnet for peer pods, attach a NAT gateway, and peer it with the cluster VNet. This provides greater isolation and flexibility at the cost of additional complexity.
2.2.1. Configuring the default worker subnet for outbound connections Copy linkLink copied to clipboard!
You can configure the default worker subnet with a NAT gateway.
Prerequisites
-
The Azure CLI (
az
) is installed and authenticated. - You have administrator access to the Azure resource group and the VNet.
Procedure
Set the
AZURE_RESOURCE_GROUP
environment variable by running the following command:AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}')
$ AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
AZURE_REGION
environment variable by running the following command:AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP}\ --query "{Location:location}" --output tsv) && \ echo "AZURE_REGION: \"$AZURE_REGION\""
$ AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP}\ --query "{Location:location}" --output tsv) && \ echo "AZURE_REGION: \"$AZURE_REGION\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
AZURE_VNET_NAME
environment variable by running the following command:AZURE_VNET_NAME=$(az network vnet list \ -g "${AZURE_RESOURCE_GROUP}" --query '[].name' -o tsv)
$ AZURE_VNET_NAME=$(az network vnet list \ -g "${AZURE_RESOURCE_GROUP}" --query '[].name' -o tsv)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
AZURE_SUBNET_ID
environment variable by running the following command:AZURE_SUBNET_ID=$(az network vnet subnet list \ --resource-group "${AZURE_RESOURCE_GROUP}" \ --vnet-name "${AZURE_VNET_NAME}" --query "[].{Id:id} \ | [? contains(Id, 'worker')]" --output tsv)
$ AZURE_SUBNET_ID=$(az network vnet subnet list \ --resource-group "${AZURE_RESOURCE_GROUP}" \ --vnet-name "${AZURE_VNET_NAME}" --query "[].{Id:id} \ | [? contains(Id, 'worker')]" --output tsv)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the NAT gateway environment variables for the peer pod subnet by running the following commands:
export PEERPOD_NAT_GW=peerpod-nat-gw
$ export PEERPOD_NAT_GW=peerpod-nat-gw
Copy to Clipboard Copied! Toggle word wrap Toggle overflow export PEERPOD_NAT_GW_IP=peerpod-nat-gw-ip
$ export PEERPOD_NAT_GW_IP=peerpod-nat-gw-ip
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public IP address for the NAT gateway by running the following command:
az network public-ip create -g "${AZURE_RESOURCE_GROUP}" \ -n "${PEERPOD_NAT_GW_IP}" -l "${AZURE_REGION}" --sku Standard
$ az network public-ip create -g "${AZURE_RESOURCE_GROUP}" \ -n "${PEERPOD_NAT_GW_IP}" -l "${AZURE_REGION}" --sku Standard
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the NAT gateway and associate it with the public IP address by running the following command:
az network nat gateway create -g "${AZURE_RESOURCE_GROUP}" \ -l "${AZURE_REGION}" --public-ip-addresses "${PEERPOD_NAT_GW_IP}" \ -n "${PEERPOD_NAT_GW}"
$ az network nat gateway create -g "${AZURE_RESOURCE_GROUP}" \ -l "${AZURE_REGION}" --public-ip-addresses "${PEERPOD_NAT_GW_IP}" \ -n "${PEERPOD_NAT_GW}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the VNet subnet to use the NAT gateway by running the following command:
az network vnet subnet update --nat-gateway "${PEERPOD_NAT_GW}" \ --ids "${AZURE_SUBNET_ID}"
$ az network vnet subnet update --nat-gateway "${PEERPOD_NAT_GW}" \ --ids "${AZURE_SUBNET_ID}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Confirm the NAT gateway is attached to the VNet subnet by running the following command:
az network vnet subnet show --ids "${AZURE_SUBNET_ID}" \ --query "natGateway.id" -o tsv
$ az network vnet subnet show --ids "${AZURE_SUBNET_ID}" \ --query "natGateway.id" -o tsv
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The output contains the NAT gateway resource ID. If no NAT gateway is attached, the output is empty.
Example output
/subscriptions/12345678-1234-1234-1234-1234567890ab/resourceGroups/myResourceGroup/providers/Microsoft.Network/natGateways/myNatGateway
/subscriptions/12345678-1234-1234-1234-1234567890ab/resourceGroups/myResourceGroup/providers/Microsoft.Network/natGateways/myNatGateway
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.2.2. Creating a peer pod VNet for outbound connections Copy linkLink copied to clipboard!
To enable public internet access, you can create a dedicated virtual network (VNet) for peer pods, attach a network address translation (NAT) gateway, create a subnet, and enable VNet peering with non-overlapping address spaces.
Prerequisites
-
The Azure CLI (
az
) is installed - You have signed in to Azure. See Authenticate to Azure using Azure CLI.
- You have administrator access to the Azure resource group and VNet hosting the cluster.
-
You have verified the cluster VNet classless inter-domain routing (CIDR) address. The default value is
10.0.0.0/14
. If you overrode the default value, you have ensured that you chose a non-overlapping CIDR address for the peer pod VNet. For example,192.168.0.0/16
.
Procedure
Set the environmental variables for the peer pod network:
Set the peer pod VNet environment variables by running the following commands:
export PEERPOD_VNET_NAME="${PEERPOD_VNET_NAME:-peerpod-vnet}"
$ export PEERPOD_VNET_NAME="${PEERPOD_VNET_NAME:-peerpod-vnet}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow export PEERPOD_VNET_CIDR="${PEERPOD_VNET_CIDR:-192.168.0.0/16}"
$ export PEERPOD_VNET_CIDR="${PEERPOD_VNET_CIDR:-192.168.0.0/16}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the peer pod subnet environment variables by running the following commands:
export PEERPOD_SUBNET_NAME="${PEERPOD_SUBNET_NAME:-peerpod-subnet}"
$ export PEERPOD_SUBNET_NAME="${PEERPOD_SUBNET_NAME:-peerpod-subnet}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow export PEERPOD_SUBNET_CIDR="${PEERPOD_SUBNET_CIDR:-192.168.0.0/16}"
$ export PEERPOD_SUBNET_CIDR="${PEERPOD_SUBNET_CIDR:-192.168.0.0/16}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Set the environmental variables for Azure:
AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}')
$ AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}')
Copy to Clipboard Copied! Toggle word wrap Toggle overflow AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP}\ --query "{Location:location}" --output tsv) && \ echo "AZURE_REGION: \"$AZURE_REGION\""
$ AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP}\ --query "{Location:location}" --output tsv) && \ echo "AZURE_REGION: \"$AZURE_REGION\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow AZURE_VNET_NAME=$(az network vnet list \ -g "${AZURE_RESOURCE_GROUP}" --query '[].name' -o tsv)
$ AZURE_VNET_NAME=$(az network vnet list \ -g "${AZURE_RESOURCE_GROUP}" --query '[].name' -o tsv)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the peer pod NAT gateway environment variables by running the following commands:
export PEERPOD_NAT_GW="${PEERPOD_NAT_GW:-peerpod-nat-gw}"
$ export PEERPOD_NAT_GW="${PEERPOD_NAT_GW:-peerpod-nat-gw}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow export PEERPOD_NAT_GW_IP="${PEERPOD_NAT_PUBLIC_IP:-peerpod-nat-gw-ip}"
$ export PEERPOD_NAT_GW_IP="${PEERPOD_NAT_PUBLIC_IP:-peerpod-nat-gw-ip}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure the VNET:
Create the peer pod VNet by running the following command:
az network vnet create --resource-group "${AZURE_RESOURCE_GROUP}" \ --name "${PEERPOD_VNET_NAME}" \ --address-prefixes "${PEERPOD_VNET_CIDR}"
$ az network vnet create --resource-group "${AZURE_RESOURCE_GROUP}" \ --name "${PEERPOD_VNET_NAME}" \ --address-prefixes "${PEERPOD_VNET_CIDR}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public IP address for the peer pod VNet by running the following command:
az network public-ip create -g "${AZURE_RESOURCE_GROUP}" \ -n "${PEERPOD_NAT_GW_IP}" -l "${AZURE_REGION}"
$ az network public-ip create -g "${AZURE_RESOURCE_GROUP}" \ -n "${PEERPOD_NAT_GW_IP}" -l "${AZURE_REGION}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a NAT gateway for the peer pod VNet by running the following command:
az network nat gateway create -g "${AZURE_RESOURCE_GROUP}" \ -l "${AZURE_REGION}" \ --public-ip-addresses "${PEERPOD_NAT_GW_IP}" \ -n "${PEERPOD_NAT_GW}"
$ az network nat gateway create -g "${AZURE_RESOURCE_GROUP}" \ -l "${AZURE_REGION}" \ --public-ip-addresses "${PEERPOD_NAT_GW_IP}" \ -n "${PEERPOD_NAT_GW}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a subnet in the peer pod VNet and attach the NAT gateway by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Configure the virtual network peering connection:
Create the peering connection by running the following command:
az network vnet peering create -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}" \ --remote-vnet "${PEERPOD_VNET_NAME}" --allow-vnet-access \ --allow-forwarded-traffic
$ az network vnet peering create -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}" \ --remote-vnet "${PEERPOD_VNET_NAME}" --allow-vnet-access \ --allow-forwarded-traffic
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Sync the peering connection by running the following command:
az network vnet peering sync -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}"
$ az network vnet peering sync -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Complete the peering connection by running the following command:
az network vnet peering create -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-peerpod-vnet-to-azure-vnet \ --vnet-name "${PEERPOD_VNET_NAME}" \ --remote-vnet "${AZURE_VNET_NAME}" --allow-vnet-access \ --allow-forwarded-traffic
$ az network vnet peering create -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-peerpod-vnet-to-azure-vnet \ --vnet-name "${PEERPOD_VNET_NAME}" \ --remote-vnet "${AZURE_VNET_NAME}" --allow-vnet-access \ --allow-forwarded-traffic
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Check the peering connection status from the cluster VNet by running the following command:
az network vnet peering show -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}" \ --query "peeringState" -o tsv
$ az network vnet peering show -g "${AZURE_RESOURCE_GROUP}" \ -n peerpod-azure-vnet-to-peerpod-vnet \ --vnet-name "${AZURE_VNET_NAME}" \ --query "peeringState" -o tsv
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This should return
Connected
.Verify that the NAT gateway is attached to the peer pod subnet by running the following command:
az network vnet subnet show --resource-group "${AZURE_RESOURCE_GROUP}" \ --vnet-name "${PEERPOD_VNET_NAME}" --name "${PEERPOD_SUBNET_NAME}" \ --query "natGateway.id" -o tsv
$ az network vnet subnet show --resource-group "${AZURE_RESOURCE_GROUP}" \ --vnet-name "${PEERPOD_VNET_NAME}" --name "${PEERPOD_SUBNET_NAME}" \ --query "natGateway.id" -o tsv
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.3. Installing the OpenShift sandboxed containers Operator Copy linkLink copied to clipboard!
You install the OpenShift sandboxed containers Operator by using the command line interface (CLI).
Prerequisites
-
You have access to the cluster as a user with the
cluster-admin
role.
Procedure
Create an
osc-namespace.yaml
manifest file:apiVersion: v1 kind: Namespace metadata: name: openshift-sandboxed-containers-operator
apiVersion: v1 kind: Namespace metadata: name: openshift-sandboxed-containers-operator
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the namespace by running the following command:
oc apply -f osc-namespace.yaml
$ oc apply -f osc-namespace.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an
osc-operatorgroup.yaml
manifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the operator group by running the following command:
oc apply -f osc-operatorgroup.yaml
$ oc apply -f osc-operatorgroup.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an
osc-subscription.yaml
manifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the subscription by running the following command:
oc apply -f osc-subscription.yaml
$ oc apply -f osc-subscription.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the Operator is correctly installed by running the following command:
oc get csv -n openshift-sandboxed-containers-operator
$ oc get csv -n openshift-sandboxed-containers-operator
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This command can take several minutes to complete.
Watch the process by running the following command:
watch oc get csv -n openshift-sandboxed-containers-operator
$ watch oc get csv -n openshift-sandboxed-containers-operator
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME DISPLAY VERSION REPLACES PHASE openshift-sandboxed-containers openshift-sandboxed-containers-operator 1.10.1 1.9.0 Succeeded
NAME DISPLAY VERSION REPLACES PHASE openshift-sandboxed-containers openshift-sandboxed-containers-operator 1.10.1 1.9.0 Succeeded
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.4. Enabling the confidential containers feature gate Copy linkLink copied to clipboard!
You enable the confidential containers feature gate by creating the osc-feature-gates
config map.
Procedure
Create a
cc-feature-gate.yaml
manifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
osc-feature-gates
config map by running the following command:oc create -f cc-feature-gate.yaml
$ oc create -f cc-feature-gate.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.5. Configuring the pull secret for peer pods Copy linkLink copied to clipboard!
To pull pod VM images from a private registry, you must configure the pull secret for peer pods.
Then, you can link the pull secret to the default service account or you can specify the pull secret in the peer pod manifest.
Procedure
Set the
NS
variable to the namespace where you deploy your peer pods:NS=<namespace>
$ NS=<namespace>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the pull secret to the peer pod namespace:
oc get secret pull-secret -n openshift-config -o yaml \ | sed "s/namespace: openshift-config/namespace: ${NS}/" \ | oc apply -n "${NS}" -f -
$ oc get secret pull-secret -n openshift-config -o yaml \ | sed "s/namespace: openshift-config/namespace: ${NS}/" \ | oc apply -n "${NS}" -f -
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can use the cluster pull secret, as in this example, or a custom pull secret.
Optional: Link the pull secret to the default service account:
oc secrets link default pull-secret --for=pull -n ${NS}
$ oc secrets link default pull-secret --for=pull -n ${NS}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Alternatively, add the pull secret to the peer pod manifest:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.6. Creating initdata Copy linkLink copied to clipboard!
You create an initdata.toml
file and convert it to a Base64-encoded string in gzip format.
You specify this string as the INITDATA
value in the peer pods config map, for global configuration, or as an annotation in a peer pod manifest, for a specific pod.
Then, you generate a Platform Configuration Register (PCR) 8 hash from the initdata.toml
file for the Reference Value Provider Service (RVPS) config map for Red Hat build of Trustee.
Red Hat build of Trustee uses the RVPS to validate attestation evidence sent by confidential workloads. The RVPS contains trusted reference values, such as file hashes, that are compared to the PCR measurements included in attestation requests. These hashes are not generated by Red Hat build of Trustee.
You must delete the kbs_cert
setting if you configure insecure_http = true
in the kbs-config
config map.
Procedure
Obtain the Red Hat build of Trustee URL by running the following command:
TRUSTEE_URL=$(oc get route kbs-service \ -n trustee-operator-system -o jsonpath='{.spec.host}') \ && echo $TRUSTEE_URL
$ TRUSTEE_URL=$(oc get route kbs-service \ -n trustee-operator-system -o jsonpath='{.spec.host}') \ && echo $TRUSTEE_URL
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
initdata.toml
file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
<trustee-url>
: Specify the Red Hat build of Trustee URL. If you configure the Red Hat build of Trustee withinsecure_http
for testing purposes, use HTTP. Otherwise, use HTTPS. For production systems, avoid usinginsecure_http
unless you configure your environment to handle TLS externally, for example, with a proxy. -
<kbs_certificate>
: Specify the Base64-encoded TLS certificate for the attestation agent. -
kbs_cert
: Delete thekbs_cert
setting if you configureinsecure_http = true
in thekbs-config
config map.
-
Convert the
initdata.toml
file to a Base64-encoded string in gzip format in a text file by running the following command:cat initdata.toml | gzip | base64 -w0 > initdata.txt
$ cat initdata.toml | gzip | base64 -w0 > initdata.txt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Record this string for the peer pods config map or a peer pod manifest.
Calculate the SHA-256 hash of an
initdata.toml
file and assign its value to thehash
variable by running the following command:hash=$(sha256sum initdata.toml | cut -d' ' -f1)
$ hash=$(sha256sum initdata.toml | cut -d' ' -f1)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Assign 32 bytes of 0s to the
initial_pcr
variable by running the following command:initial_pcr=0000000000000000000000000000000000000000000000000000000000000000
$ initial_pcr=0000000000000000000000000000000000000000000000000000000000000000
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Calculate the SHA-256 hash of
hash
andinitial_pcr
and assign its value to thePCR8_HASH
variable by running the following command:PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH
$ PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Record the
PCR8_HASH
value for the. Calculate the SHA-256 hash of aninitdata.toml
file and assign its value to thehash
variable by running the following command:hash=$(sha256sum initdata.toml | cut -d' ' -f1)
$ hash=$(sha256sum initdata.toml | cut -d' ' -f1)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Assign 32 bytes of 0s to the
initial_pcr
variable by running the following command:initial_pcr=0000000000000000000000000000000000000000000000000000000000000000
$ initial_pcr=0000000000000000000000000000000000000000000000000000000000000000
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Calculate the SHA-256 hash of
hash
andinitial_pcr
and assign its value to thePCR8_HASH
variable by running the following command:PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH
$ PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Record the
PCR8_HASH
value for the RVPS config map.
2.7. Creating the peer pods config map Copy linkLink copied to clipboard!
You must create the peer pods config map.
Procedure
Obtain the following values from your Azure instance:
Retrieve and record the Azure resource group:
AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}') \ && echo "AZURE_RESOURCE_GROUP: \"$AZURE_RESOURCE_GROUP\""
$ AZURE_RESOURCE_GROUP=$(oc get infrastructure/cluster \ -o jsonpath='{.status.platformStatus.azure.resourceGroupName}') \ && echo "AZURE_RESOURCE_GROUP: \"$AZURE_RESOURCE_GROUP\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve and record the Azure VNet name:
AZURE_VNET_NAME=$(az network vnet list \ --resource-group ${AZURE_RESOURCE_GROUP} \ --query "[].{Name:name}" --output tsv)
$ AZURE_VNET_NAME=$(az network vnet list \ --resource-group ${AZURE_RESOURCE_GROUP} \ --query "[].{Name:name}" --output tsv)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This value is used to retrieve the Azure subnet ID.
Retrieve and record the Azure subnet ID:
AZURE_SUBNET_ID=$(az network vnet subnet list \ --resource-group ${AZURE_RESOURCE_GROUP} --vnet-name $AZURE_VNET_NAME \ --query "[].{Id:id} | [? contains(Id, 'worker')]" --output tsv) \ && echo "AZURE_SUBNET_ID: \"$AZURE_SUBNET_ID\""
$ AZURE_SUBNET_ID=$(az network vnet subnet list \ --resource-group ${AZURE_RESOURCE_GROUP} --vnet-name $AZURE_VNET_NAME \ --query "[].{Id:id} | [? contains(Id, 'worker')]" --output tsv) \ && echo "AZURE_SUBNET_ID: \"$AZURE_SUBNET_ID\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve and record the Azure network security group (NSG) ID:
AZURE_NSG_ID=$(az network nsg list --resource-group ${AZURE_RESOURCE_GROUP} \ --query "[].{Id:id}" --output tsv) && echo "AZURE_NSG_ID: \"$AZURE_NSG_ID\""
$ AZURE_NSG_ID=$(az network nsg list --resource-group ${AZURE_RESOURCE_GROUP} \ --query "[].{Id:id}" --output tsv) && echo "AZURE_NSG_ID: \"$AZURE_NSG_ID\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve and record the Azure region:
AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP} \ --query "{Location:location}" --output tsv) \ && echo "AZURE_REGION: \"$AZURE_REGION\""
$ AZURE_REGION=$(az group show --resource-group ${AZURE_RESOURCE_GROUP} \ --query "{Location:location}" --output tsv) \ && echo "AZURE_REGION: \"$AZURE_REGION\""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Create a
peer-pods-cm.yaml
manifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow AZURE_INSTANCE_SIZE
-
Defines the default instance size that is used if the instance size is not defined in the workload object.
"Standard_DC2as_v5"
is for AMD SEV-SNP. If your TEE is Intel TDX, specifyStandard_EC4eds_v5
. AZURE_IMAGE_ID
- Leave this value empty. When you install the Operator, a Job is scheduled to download the default pod VM image from the Red Hat Ecosystem Catalog and upload it to the Azure Image Gallery within the same Azure Resource Group as the OpenShift Container Platform cluster. This image provides root disk integrity protection (dm-verity) and encrypted container storage. See Confidential VMs: The core of confidential containers for details.
AZURE_INSTANCE_SIZES
- Specify the instance sizes, without spaces, for creating the pod. You can define smaller instance sizes for workloads that need less memory and fewer CPUs or larger instance sizes for larger workloads.
TAGS
-
You can configure custom tags as
key:value
pairs for pod VM instances to track peer pod costs or to identify peer pods in different clusters. PEERPODS_LIMIT_PER_NODE
-
You can increase this value to run more peer pods on a node. The default value is
10
. ROOT_VOLUME_SIZE
- You can increase this value for pods with larger container images. Specify the root volume size in gigabytes for the pod VM. The default and minimum size is 6 GB.
Create the config map by running the following command:
oc create -f peer-pods-cm.yaml
$ oc create -f peer-pods-cm.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.8. Applying initdata to a pod Copy linkLink copied to clipboard!
You can override the global INITDATA
setting you applied in the peer pods config map by applying customized initdata to a specific pod for special use cases, such as development and testing with a relaxed policy, or when using different Red Hat build of Trustee configurations. You can customize initdata by adding an annotation to the workload pod YAML.
Prerequisite
- You have created an initdata string.
Procedure
Add the initdata string to the pod manifest:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Apply the pod manifest by running the following command:
oc apply -f my-pod.yaml
$ oc apply -f my-pod.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.9. Selecting a custom peer pod VM image Copy linkLink copied to clipboard!
You can select a custom peer pod virtual machine (VM) image, tailored to your workload requirements by adding an annotation to the pod manifest. The custom image overrides the default image specified in the peer pods config map.
Prerequisites
- You have the ID of a custom pod VM image, which is compatible with your cloud provider or hypervisor.
Procedure
Create a
my-pod-manifest.yaml
file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the pod by running the following command:
oc create -f my-pod-manifest.yaml
$ oc create -f my-pod-manifest.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.10. Creating the KataConfig custom resource Copy linkLink copied to clipboard!
You must create the KataConfig
custom resource (CR) to install kata-remote
as a runtime class on your worker nodes.
OpenShift sandboxed containers installs kata-remote
as a secondary, optional runtime on the cluster and not as the primary runtime.
Creating the KataConfig
CR automatically reboots the worker nodes. The reboot can take from 10 to more than 60 minutes. The following factors can increase the reboot time:
- A large OpenShift Container Platform deployment with a greater number of worker nodes.
- Activation of the BIOS and Diagnostics utility.
- Deployment on a hard disk drive rather than an SSD.
- Deployment on physical nodes such as bare metal, rather than on virtual nodes.
- A slow CPU and network.
Procedure
Create an
example-kataconfig.yaml
manifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Optional: If you have applied node labels to install
kata-remote
on specific nodes, specify the key and value, for example,cc: 'true'
.
Create the
KataConfig
CR by running the following command:oc apply -f example-kataconfig.yaml
$ oc apply -f example-kataconfig.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The new
KataConfig
CR is created and installskata-remote
as a runtime class on the worker nodes.Wait for the
kata-remote
installation to complete and the worker nodes to reboot before verifying the installation.Monitor the installation progress by running the following command:
watch "oc describe kataconfig | sed -n /^Status:/,/^Events/p"
$ watch "oc describe kataconfig | sed -n /^Status:/,/^Events/p"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow When the status of all workers under
kataNodes
isinstalled
and the conditionInProgress
isFalse
without specifying a reason, thekata-remote
is installed on the cluster.Verify the daemon set by running the following command:
oc get -n openshift-sandboxed-containers-operator ds/osc-caa-ds
$ oc get -n openshift-sandboxed-containers-operator ds/osc-caa-ds
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the runtime classes by running the following command:
oc get runtimeclass
$ oc get runtimeclass
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME HANDLER AGE kata-remote kata-remote 152m
NAME HANDLER AGE kata-remote kata-remote 152m
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.11. Verifying attestation Copy linkLink copied to clipboard!
You can verify the attestation process by creating a test pod with a relaxed Kata agent policy and retrieving its key.
This procedure is an example to verify that attestation is working. Do not write sensitive data to standard I/O, because the data can be captured by using a memory dump. Only data written to memory is encrypted.
Procedure
Create a
test-pod.yaml
manifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Optional: Setting initdata in a pod annotation overrides the global
INITDATA
setting in the peer pods config map.
Create the pod by running the following command:
oc create -f test-pod.yaml
$ oc create -f test-pod.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Log in to the pod by running the following command:
oc exec -it ocp-cc-pod -- bash
$ oc exec -it ocp-cc-pod -- bash
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Fetch the pod secret by running the following command:
curl http://127.0.0.1:8006/cdh/resource/default/kbsres1/key1
$ curl http://127.0.0.1:8006/cdh/resource/default/kbsres1/key1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
res1val1/ #
res1val1/ #
Copy to Clipboard Copied! Toggle word wrap Toggle overflow