Deploying Red Hat build of Trustee
Secure management and attestation of confidential containers workloads
Abstract
Preface Copy linkLink copied to clipboard!
Chapter 1. Overview Copy linkLink copied to clipboard!
Learn about Red Hat build of Trustee features and terminology. You must ensure that your OpenShift Container Platform environment is compatible.
1.1. About Red Hat build of Trustee Copy linkLink copied to clipboard!
Red Hat build of Trustee is a critical component of the confidential containers solution for OpenShift sandboxed containers. Red Hat build of Trustee enables secure management and attestation of confidential workloads running within Trusted Execution Environments (TEEs) on a Red Hat OpenShift Container Platform cluster.
Red Hat build of Trustee is a service that facilitates the deployment and management of confidential containers. It provides attestation and key management services to ensure the integrity and confidentiality of workloads running in TEEs.
You deploy Red Hat build of Trustee on a separate OpenShift Container Platform cluster in a trusted environment, not on the cluster that hosts the primary workload. This separation enhances security by isolating sensitive operations from the untrusted cloud infrastructure.
Red Hat build of Trustee performs the following key functions:
- Verifies the integrity of the TEE and the workloads running within it, ensuring that it only executes trusted code and data.
- Securely manages cryptographic keys and secrets required by confidential containers, protecting sensitive data from unauthorized access.
-
Simplifies the configuration of TEEs through the
KbsConfigcustom resource, enabling seamless integration with OpenShift Container Platform workflows.
By leveraging Red Hat build of Trustee, OpenShift Container Platform users can deploy confidential workloads using familiar tools while maintaining strong security guarantees, even on shared or third-party infrastructure.
1.2. Providing feedback on Red Hat documentation Copy linkLink copied to clipboard!
You can provide feedback or report an error by submitting the Create Issue form in Jira:
- Ensure that you are logged in to Jira. If you do not have a Jira account, you must create a Red Hat Jira account.
- Launch the Create Issue form.
Complete the Summary, Description, and Reporter fields.
In the Description field, include the documentation URL, chapter or section number, and a detailed description of the issue.
- Click Create.
Chapter 2. Deploying Red Hat build of Trustee for workloads running on bare metal servers Copy linkLink copied to clipboard!
You can deploy Red Hat build of Trustee for confidential containers workloads running on bare metal servers. bare metal servers in a disconnected network environment.
Optionally, you can deploy Red Hat build of Trustee with a cluster-wide proxy. For details, see Configuring the cluster-wide proxy in the OpenShift Container Platform documentation.
2.1. Prerequisites Copy linkLink copied to clipboard!
- You have installed the latest version of Red Hat OpenShift Container Platform in a trusted environment. For more information, see Installing OpenShift Container Platform on bare metal.
2.2. Deployment overview Copy linkLink copied to clipboard!
You deploy Red Hat build of Trustee by performing the following steps:
- Install the Red Hat build of Trustee Operator.
-
Optional: Create the
kbs-configconfig map if you are using Intel® Trust Domain Extensions (TDX) remote attestation. - Create the Reference Value Provider Service (RVPS) config map.
- Create the attestation policy config map.
- Optional: Create a config map for Intel® TDX.
- Optional: Create a secret for custom keys clients.
- Optional: Create a secret for container image signature verification.
- Create the container image signature verification policy. The container image signature verification policy is disabled by default. For production workloads, you must use signature verification to ensure container images are not tampered with.
- Create the resource policy config map.
-
Create the
KBSConfigCR. - Create the cluster route.
- Create the authentication secret.
- Verify the Red Hat build of Trustee configuration.
2.3. Installing the Red Hat build of Trustee Operator Copy linkLink copied to clipboard!
You install the Red Hat build of Trustee Operator on an OpenShift Container Platform cluster in a trusted environment.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI tool (
oc).
Procedure
Create a
trustee-namespace.yamlmanifest file:apiVersion: v1 kind: Namespace metadata: name: trustee-operator-system
apiVersion: v1 kind: Namespace metadata: name: trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
trustee-operator-systemnamespace by running the following command:oc create -f trustee-namespace.yaml
$ oc create -f trustee-namespace.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-operatorgroup.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the operator group by running the following command:
oc create -f trustee-operatorgroup.yaml
$ oc create -f trustee-operatorgroup.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-subscription.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the subscription by running the following command:
oc create -f trustee-subscription.yaml
$ oc create -f trustee-subscription.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the Operator is correctly installed by running the following command:
oc get csv -n trustee-operator-system
$ oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command can take several minutes to complete.
Watch the process by running the following command:
watch oc get csv -n trustee-operator-system
$ watch oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 Succeeded
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 SucceededCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.4. Creating the kbs-config config map Copy linkLink copied to clipboard!
You create the kbs-config config map to configure Red Hat build of Trustee.
Procedure
Create a
kbs-config-cm.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the config map by running the following command:
oc create -f kbs-config-cm.yaml
$ oc create -f kbs-config-cm.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.5. Creating the RVPS config map Copy linkLink copied to clipboard!
You create the Reference Value Provider Service (RVPS) config map, which specifies the reference values for your Trusted Execution Environment (TEE).
The client collects measurements from the running software, the TEE hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to Red Hat build of Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.
The data.reference-values.json stanza must be present, but it can be empty.
Do not use this configuration example in a production environment. Initially, you create an empty RVPS config map. Then, you update the RVPS config map with reference values for your TEE.
Procedure
Create an
rvps-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the RVPS config map by running the following command:
oc create -f rvps-configmap.yaml
$ oc create -f rvps-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.6. Creating the attestation policy config map Copy linkLink copied to clipboard!
You create an attestation policy config map to define attestation policies for Red Hat build of Trustee.
The attestation policy follows the Open Policy Agent specification.
This policy checks the Platform Configuration Register (PCR) values 03, 08, 09, 11, and 12 values against the reference values to ensure that the confidential containers pod uses the specified restrictive Kata agent policy and that the Red Hat pod VM image has not been altered. For details, see Linux TPM PCR Registry in the UAPI Group Specifications documentation.
Procedure
Create an
attestation-policy.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the attestation policy config map by running the following command:
oc create -f attestation-policy.yaml
$ oc create -f attestation-policy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.7. Creating a tdx-config config map Copy linkLink copied to clipboard!
Create a config map for Intel® Trust Domain Extensions (TDX).
Procedure
Create a
tdx-config.yamlmanifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
tdx-configconfig map by running the following command:oc create -f tdx-config.yaml
$ oc create -f tdx-config.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.8. Creating a secret with custom keys for clients Copy linkLink copied to clipboard!
You can create a secret that contains one or more custom keys for Red Hat build of Trustee clients.
In this example, the attestation-status secret has two entries (key1, key2), which the clients retrieve. You can add additional secrets according to your requirements by using the same format.
Prerequisites
- You have created one or more custom keys.
Procedure
Create a secret for the custom keys according to the following example:
oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \ --from-literal key2=<custom_key2> \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \1 --from-literal key2=<custom_key2> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Specify a custom key.
You specify the
attestation-statussecret in thespec.kbsSecretResourceskey of theKbsConfigcustom resource manifest.
2.9. Creating a secret for container image signature verification Copy linkLink copied to clipboard!
If you use container image signature verification, you must create a secret that contains the public container image signing key.
The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.
You can use Red Hat Trusted Artifact Signer or other tools to sign container images.
Procedure
Create a secret for container image signature verification by running the following command:
oc create secret generic <type> \ --from-file=<tag>=./<public_key_file> \ -n trustee-operator-system
$ oc create secret generic <type> \1 --from-file=<tag>=./<public_key_file> \2 -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the
<type>value. You must add this value to thespec.kbsSecretResourceskey when you create theKbsConfigcustom resource.
2.10. Creating the container image signature verification policy Copy linkLink copied to clipboard!
You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.
Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself. For more information about signature keys, see Creating the attestation token secret.
Procedure
Create a
security-policy-config.jsonfile according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- By default, the policy rejects all images and all signatures. The transports section specifies which images the policy explicitly approves and verifies through their signatures.
- 2
- Specify the image repository for
transport, for example,"docker":. For more information, see containers-transports 5. - 3
- Specify the container registry and image, for example, "quay.io/my-image".
- 4
- Specify the type and tag of the container image signature verification secret that you created, for example,
img-sig/pub-key.
Create the security policy by running the following command:
oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-system
$ oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
<security-policy-name>secret is specified in thespec.kbsSecretResourceskey of theKbsConfigcustom resource.
2.11. Creating the resource policy config map Copy linkLink copied to clipboard!
You configure the resource policy config map for the policy engine. This policy determines which resources are accessible to Red Hat build of Trustee.
This policy engine is different from the Attestation Service policy engine, which determines the validity of TEE evidence.
Procedure
Create a
resourcepolicy-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - policy.rego
-
The name of the resource policy,
policy.rego, must match the resource policy defined in thekbs-configconfig map. - package policy
- The resource policy follows the Open Policy Agent specification.
Create the resource policy config map by running the following command:
oc create -f resourcepolicy-configmap.yaml
$ oc create -f resourcepolicy-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.12. Creating the cluster route Copy linkLink copied to clipboard!
You create a secure route with edge TLS termination for the cluster where you installed Red Hat build of Trustee.
External ingress traffic reaches the router pods as HTTPS and passes on to the pods running in the trustee-operator-system namespace as HTTP.
Procedure
Create an edge route by running the following command:
oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-system
$ oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
TRUSTEE_HOSTvariable by running the following command:TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})$ TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the route by running the following command:
echo $TRUSTEE_HOST
$ echo $TRUSTEE_HOSTCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.13. Creating the authentication secret Copy linkLink copied to clipboard!
You create the authentication secret for Red Hat build of Trustee.
Procedure
Create a private key by running the following command:
openssl genpkey -algorithm ed25519 > privateKey
$ openssl genpkey -algorithm ed25519 > privateKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public key by running the following command:
openssl pkey -in privateKey -pubout -out publicKey
$ openssl pkey -in privateKey -pubout -out publicKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a secret by running the following command:
oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-system
$ oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the secret by running the following command:
oc get secret -n trustee-operator-system
$ oc get secret -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.14. Creating the KbsConfig custom resource Copy linkLink copied to clipboard!
Create the KbsConfig custom resource (CR) to launch Red Hat build of Trustee.
Procedure
Create a
kbsconfig-cr.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
kbsSecretResources: Specify thetypevalue of the container image signature verification secret if you created the secret, for example,img-sig. -
Uncomment
tdxConfigSpec.kbsTdxConfigMapName: tdx-configfor Intel Trust Domain Extensions. -
Uncomment
kbsServiceType: <service_type>if you create a service type, other than the defaultClusterIPservice, to expose applications within the cluster external traffic. You can specifyNodePort,LoadBalancer, orExternalName.
-
Create the
KbsConfigCR by running the following command:oc create -f kbsconfig-cr.yaml
$ oc create -f kbsconfig-cr.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.15. Verifying the configuration Copy linkLink copied to clipboard!
You verify the Red Hat build of Trustee configuration by checking its pods and logs.
Procedure
Set the default project by running the following command:
oc project trustee-operator-system
$ oc project trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pods by running the following command:
oc get pods -n trustee-operator-system
$ oc get pods -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59m
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
POD_NAMEenvironmental variable by running the following command:POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pod logs by running the following command:
oc logs -n trustee-operator-system $POD_NAME
$ oc logs -n trustee-operator-system $POD_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16. Installing Red Hat build of Trustee in a disconnected environment Copy linkLink copied to clipboard!
You can deploy Red Hat build of Trustee for confidential containers workloads running on
2.16.1. Prerequisites Copy linkLink copied to clipboard!
You have installed the latest version of Red Hat OpenShift Container Platform in a trusted environment. For more information, see Installing a user-provisioned bare metal cluster on a disconnected environment
2.16.2. Deployment overview Copy linkLink copied to clipboard!
You deploy Red Hat build of Trustee by performing the following steps:
- Install the Red Hat build of Trustee Operator.
-
Optional: Create the
kbs-configconfig map if you are using Intel® Trust Domain Extensions (TDX) remote attestation. - Optional: Download VCEK certificates for AMD SEV-SNP nodes.
- Create the Reference Value Provider Service (RVPS) config map.
- Create the attestation policy config map.
- Optional: Create a config map for Intel® TDX.
- Optional: Create a secret for custom keys clients.
- Optional: Create a secret for container image signature verification.
- Create the container image signature verification policy. The container image signature verification policy is disabled by default. For production workloads, you must use signature verification to ensure container images are not tampered with.
- Create the resource policy config map.
-
Create the
KBSConfigCR. - Create the cluster route.
- Create the authentication secret.
- Verify the Red Hat build of Trustee configuration.
2.16.3. Installing the Red Hat build of Trustee Operator Copy linkLink copied to clipboard!
You install the Red Hat build of Trustee Operator on an OpenShift Container Platform cluster in a trusted environment.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI tool (
oc). - You have disabled the default catalog sources and mirrored the Operator catalog. For details, see Using Operator Lifecycle Manager in disconnected environments in the OpenShift Container Platform documentation.
Procedure
Create a
trustee-namespace.yamlmanifest file:apiVersion: v1 kind: Namespace metadata: name: trustee-operator-system
apiVersion: v1 kind: Namespace metadata: name: trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
trustee-operator-systemnamespace by running the following command:oc create -f trustee-namespace.yaml
$ oc create -f trustee-namespace.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-operatorgroup.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the operator group by running the following command:
oc create -f trustee-operatorgroup.yaml
$ oc create -f trustee-operatorgroup.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-subscription.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the subscription by running the following command:
oc create -f trustee-subscription.yaml
$ oc create -f trustee-subscription.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the Operator is correctly installed by running the following command:
oc get csv -n trustee-operator-system
$ oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command can take several minutes to complete.
Watch the process by running the following command:
watch oc get csv -n trustee-operator-system
$ watch oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 Succeeded
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 SucceededCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.4. Creating HTTPS secrets Copy linkLink copied to clipboard!
Generate keys to securely launch Red Hat build of Trustee and enables services to use HTTPS.
Procedure
Set the
DOMAINvariable for the cluster by running the following command:DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')$ DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
NSvariable for the Red Hat build of Trustee namespace by running the following command:NS=trustee-operator-system
$ NS=trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
ROUTE_NAMEvariable by running the following command:ROUTE_NAME=kbs-service
$ ROUTE_NAME=kbs-serviceCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
ROUTEvariable to the full DNS name by running the following command:ROUTE="${ROUTE_NAME}-${NS}.${DOMAIN}"$ ROUTE="${ROUTE_NAME}-${NS}.${DOMAIN}"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a private SSL/TLS key and certificate for Red Hat build of Trustee by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
<custom_cn>is a custom CN. For example:kbs-trustee-operator-system. -
<custom_org>is a name of your organization.
-
Create the
kbs-https-certificatesecret in thetrustee-operator-systemnamespace by running the following command:oc create secret generic kbs-https-certificate --from-file=tls.crt -n trustee-operator-system
$ oc create secret generic kbs-https-certificate --from-file=tls.crt -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
kbs-https-keysecret in thetrustee-operator-systemnamespace by running the following command:oc create secret generic kbs-https-key --from-file=tls.key -n trustee-operator-system
$ oc create secret generic kbs-https-key --from-file=tls.key -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.5. Creating the attestation token secrets Copy linkLink copied to clipboard!
Generate an attestation token key and certificate for Red Hat build of Trustee.
Procedure
Generate a private elliptic curve SSL key called
token.keyby running the following command:openssl ecparam -name prime256v1 -genkey -noout -out token.key
$ openssl ecparam -name prime256v1 -genkey -noout -out token.keyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
attestation-keysecret from the SSL/TLS key in thetrustee-operator-systemnamespace:oc create secret generic attestation-key \ --from-file=token.key \ -n trustee-operator-system
$ oc create secret generic attestation-key \ --from-file=token.key \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a self-signed SSL/TLS certificate from the private SSL key by running the following command:
openssl req -new -x509 -key token.key -out token.crt -days 365 \ -subj "/CN=<custom_cn>/O=<custom_org>"
$ openssl req -new -x509 -key token.key -out token.crt -days 365 \ -subj "/CN=<custom_cn>/O=<custom_org>"Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
<custom_cn>: Specify the Common Name. For example:kbs-trustee-operator-system. -
<custom_org>: Specify your organization name.
-
Create the
attestation-certsecret from the SSL/TLS key and certificate in thetrustee-operator-systemnamespace:oc create secret generic attestation-cert \ --from-file=token.crt \ -n trustee-operator-system
$ oc create secret generic attestation-cert \ --from-file=token.crt \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
attestation-statussecret used for verifying the attestation process:oc create secret generic attestation-status \ --from-literal=status=success \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal=status=success \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.6. Creating the kbs-config config map Copy linkLink copied to clipboard!
You create the kbs-config config map to configure Red Hat build of Trustee.
Procedure
Create a
kbs-config-cm.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the config map by running the following command:
oc create -f kbs-config-cm.yaml
$ oc create -f kbs-config-cm.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.7. Downloading VCEK certificates for AMD SEV-SNP Copy linkLink copied to clipboard!
For each AMD SEV-SNP node, you must download a Versioned Chip Endorsement Key (VCEK) certificate so that Red Hat build of Trustee can sign the attestation report generated by the confidential containers virtual machine (VM).
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI tool (
oc). -
You have installed the
podmanutility. -
You have installed the
jqutility.
Procedure
Perform the following steps on the worker node:
Set the
nodenamevariable by running the following command:export nodename=<node_name>
$ export nodename=<node_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve the VCEK URL by running the following command:
sudo podman run --privileged \ quay.io/openshift_sandboxed_containers/coco-tools:latest \ /tools/snphost show vcek-url \ | cut -d '?' -f 1 > vcek_base_url_$nodename.txt
$ sudo podman run --privileged \ quay.io/openshift_sandboxed_containers/coco-tools:latest \ /tools/snphost show vcek-url \ | cut -d '?' -f 1 > vcek_base_url_$nodename.txtCopy to Clipboard Copied! Toggle word wrap Toggle overflow Obtain the VCEK URL by running the following command:
VCEK_BASE_URL=$(cat vcek_base_url_$nodename.txt) \ && echo $VCEK_BASE_URL
$ VCEK_BASE_URL=$(cat vcek_base_url_$nodename.txt) \ && echo $VCEK_BASE_URLCopy to Clipboard Copied! Toggle word wrap Toggle overflow The command prints the full VCEK URL, including query parameters.
Example output
https://kdsintf.amd.com/vcek/v1/Genoa/126A06A956658EF040D5DF52D438BB78DBB8E8C1C6244B420513B00ECCF1E6E2456C925FD913E8F5A80C209F83052077871919E4F1EBB62B296B36E084B57095
https://kdsintf.amd.com/vcek/v1/Genoa/126A06A956658EF040D5DF52D438BB78DBB8E8C1C6244B420513B00ECCF1E6E2456C925FD913E8F5A80C209F83052077871919E4F1EBB62B296B36E084B57095Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
my-pod.yamlpod manifest according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - <data>
- Base64-encoded initialization data used by the hypervisor to securely launch the confidential virtual machine (VM). For more information, see Creating initdata.
Create the pod by running the following command:
oc create pod -f my-pod.yaml
$ oc create pod -f my-pod.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Obtain the pod attestation report by running the following command:
oc exec -it coco-guest -- curl http://127.0.0.1:8006/aa/evidence?runtime_data=test > <attestation_report>-$nodename.txt
$ oc exec -it coco-guest -- curl http://127.0.0.1:8006/aa/evidence?runtime_data=test > <attestation_report>-$nodename.txtCopy to Clipboard Copied! Toggle word wrap Toggle overflow <attestation_report>Specify the attestation report file name.
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Set the
bootloader,tee,snp, andmicrocodevalues by running the following command:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
VCEK_URL_NODE1variable by running the following command:export VCEK_URL_NODE1="$VCEK_BASE_URL?blSPL=$bootloader&teeSPL=$tee&snpSPL=$snp&ucodeSPL=$microcode"
$ export VCEK_URL_NODE1="$VCEK_BASE_URL?blSPL=$bootloader&teeSPL=$tee&snpSPL=$snp&ucodeSPL=$microcode"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a separate directory for each processor type by running the following commands:
mkdir -p ek/<processor_type>
$ mkdir -p ek/<processor_type>Copy to Clipboard Copied! Toggle word wrap Toggle overflow <processor_type>-
Specify
genoa,milan, orturin.
Download certificates in separate directories for each processor type:
curl -L -o ek/<processor_type>/vcek_node1.crt $VCEK_URL_NODE1
$ curl -L -o ek/<processor_type>/vcek_node1.crt $VCEK_URL_NODE1Copy to Clipboard Copied! Toggle word wrap Toggle overflow <processor_type>-
Specify
genoa,milan, orturin.
Create a secret for each processor from the VCEK certificates by running the following command:
oc create secret generic <processor_type>-secret \ --from-file ./ek/<processor_type> \ -n trustee-operator-system
$ oc create secret generic <processor_type>-secret \ --from-file ./ek/<processor_type> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow <processor_type>-
Specify
genoa,milan, orturin.
2.16.8. Creating the RVPS config map Copy linkLink copied to clipboard!
You create the Reference Value Provider Service (RVPS) config map, which specifies the reference values for your Trusted Execution Environment (TEE).
The client collects measurements from the running software, the TEE hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to Red Hat build of Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.
The data.reference-values.json stanza must be present, but it can be empty.
Do not use this configuration example in a production environment. Initially, you create an empty RVPS config map. Then, you update the RVPS config map with reference values for your TEE.
Procedure
Create an
rvps-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the RVPS config map by running the following command:
oc create -f rvps-configmap.yaml
$ oc create -f rvps-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.9. Creating the attestation policy config map Copy linkLink copied to clipboard!
You create an attestation policy config map to define attestation policies for Red Hat build of Trustee.
The attestation policy follows the Open Policy Agent specification.
This policy checks the Platform Configuration Register (PCR) values 03, 08, 09, 11, and 12 values against the reference values to ensure that the confidential containers pod uses the specified restrictive Kata agent policy and that the Red Hat pod VM image has not been altered. For details, see Linux TPM PCR Registry in the UAPI Group Specifications documentation.
Procedure
Create an
attestation-policy.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the attestation policy config map by running the following command:
oc create -f attestation-policy.yaml
$ oc create -f attestation-policy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.10. Creating a tdx-config config map Copy linkLink copied to clipboard!
Create a config map for Intel® Trust Domain Extensions (TDX).
Procedure
Create a
tdx-config.yamlmanifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
tdx-configconfig map by running the following command:oc create -f tdx-config.yaml
$ oc create -f tdx-config.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.11. Creating a secret with custom keys for clients Copy linkLink copied to clipboard!
You can create a secret that contains one or more custom keys for Red Hat build of Trustee clients.
In this example, the attestation-status secret has two entries (key1, key2), which the clients retrieve. You can add additional secrets according to your requirements by using the same format.
Prerequisites
- You have created one or more custom keys.
Procedure
Create a secret for the custom keys according to the following example:
oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \ --from-literal key2=<custom_key2> \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \1 --from-literal key2=<custom_key2> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Specify a custom key.
You specify the
attestation-statussecret in thespec.kbsSecretResourceskey of theKbsConfigcustom resource manifest.
2.16.12. Creating a secret for container image signature verification Copy linkLink copied to clipboard!
If you use container image signature verification, you must create a secret that contains the public container image signing key.
The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.
You can use Red Hat Trusted Artifact Signer or other tools to sign container images.
Procedure
Create a secret for container image signature verification by running the following command:
oc create secret generic <type> \ --from-file=<tag>=./<public_key_file> \ -n trustee-operator-system
$ oc create secret generic <type> \1 --from-file=<tag>=./<public_key_file> \2 -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the
<type>value. You must add this value to thespec.kbsSecretResourceskey when you create theKbsConfigcustom resource.
2.16.13. Creating the container image signature verification policy Copy linkLink copied to clipboard!
You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.
Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself. For more information about signature keys, see Creating the attestation token secret.
Procedure
Create a
security-policy-config.jsonfile according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- By default, the policy rejects all images and all signatures. The transports section specifies which images the policy explicitly approves and verifies through their signatures.
- 2
- Specify the image repository for
transport, for example,"docker":. For more information, see containers-transports 5. - 3
- Specify the container registry and image, for example, "quay.io/my-image".
- 4
- Specify the type and tag of the container image signature verification secret that you created, for example,
img-sig/pub-key.
Create the security policy by running the following command:
oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-system
$ oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
<security-policy-name>secret is specified in thespec.kbsSecretResourceskey of theKbsConfigcustom resource.
2.16.14. Creating the resource policy config map Copy linkLink copied to clipboard!
You configure the resource policy config map for the policy engine. This policy determines which resources are accessible to Red Hat build of Trustee.
This policy engine is different from the Attestation Service policy engine, which determines the validity of TEE evidence.
Procedure
Create a
resourcepolicy-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - policy.rego
-
The name of the resource policy,
policy.rego, must match the resource policy defined in thekbs-configconfig map. - package policy
- The resource policy follows the Open Policy Agent specification.
Create the resource policy config map by running the following command:
oc create -f resourcepolicy-configmap.yaml
$ oc create -f resourcepolicy-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.15. Creating the cluster route Copy linkLink copied to clipboard!
You create a secure route with edge TLS termination for the cluster where you installed Red Hat build of Trustee.
External ingress traffic reaches the router pods as HTTPS and passes on to the pods running in the trustee-operator-system namespace as HTTP.
Procedure
Create an edge route by running the following command:
oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-system
$ oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
TRUSTEE_HOSTvariable by running the following command:TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})$ TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the route by running the following command:
echo $TRUSTEE_HOST
$ echo $TRUSTEE_HOSTCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.16. Creating the authentication secret Copy linkLink copied to clipboard!
You create the authentication secret for Red Hat build of Trustee.
Procedure
Create a private key by running the following command:
openssl genpkey -algorithm ed25519 > privateKey
$ openssl genpkey -algorithm ed25519 > privateKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public key by running the following command:
openssl pkey -in privateKey -pubout -out publicKey
$ openssl pkey -in privateKey -pubout -out publicKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a secret by running the following command:
oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-system
$ oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the secret by running the following command:
oc get secret -n trustee-operator-system
$ oc get secret -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.17. Creating the KbsConfig custom resource Copy linkLink copied to clipboard!
Create the KbsConfig custom resource (CR) to launch Red Hat build of Trustee.
Procedure
Create a
kbsconfig-cr.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
kbsSecretResources: Specify thetypevalue of the container image signature verification secret if you created the secret, for example,img-sig. -
Uncomment
tdxConfigSpec.kbsTdxConfigMapName: tdx-configfor Intel Trust Domain Extensions. -
Uncomment
kbsServiceType: <service_type>if you create a service type, other than the defaultClusterIPservice, to expose applications within the cluster external traffic. You can specifyNodePort,LoadBalancer, orExternalName. -
kbsLocalCertCacheSpec: For workloads running on AMD SEV-SNP nodes, specify the VCEK certificates in thekbsLocalCertCacheSpecstanza.
-
Create the
KbsConfigCR by running the following command:oc create -f kbsconfig-cr.yaml
$ oc create -f kbsconfig-cr.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.16.18. Verifying the configuration Copy linkLink copied to clipboard!
You verify the Red Hat build of Trustee configuration by checking its pods and logs.
Procedure
Set the default project by running the following command:
oc project trustee-operator-system
$ oc project trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pods by running the following command:
oc get pods -n trustee-operator-system
$ oc get pods -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59m
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
POD_NAMEenvironmental variable by running the following command:POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pod logs by running the following command:
oc logs -n trustee-operator-system $POD_NAME
$ oc logs -n trustee-operator-system $POD_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 3. Deploying Red Hat build of Trustee for workloads running on Microsoft Azure Red Hat OpenShift Copy linkLink copied to clipboard!
You can deploy Red Hat build of Trustee for confidential containers workloads running on Azure Red Hat OpenShift.
3.1. Prerequisites Copy linkLink copied to clipboard!
- You have installed the latest version of Red Hat OpenShift Container Platform in a trusted environment. For more information, see Installing OpenShift Container Platform on bare metal.
3.2. Deployment overview Copy linkLink copied to clipboard!
You deploy Red Hat build of Trustee by performing the following steps:
- Install the Red Hat build of Trustee Operator.
- Create HTTPS secrets.
- Create the attestation token secret.
-
Optional: Create the
kbs-configconfig map if you are using Intel® Trust Domain Extensions (TDX) remote attestation. -
Create the Reference Value Provider Service (RVPS) config map. Initially, you create an empty config map for the reference values. You update the values after you create
KBSConfigcustom resource (CR). - Create the attestation policy config map.
- Optional: Create a config map for Intel® TDX.
- Optional: Create a secret for custom keys clients.
- Optional: Create a secret for container image signature verification.
- Create the container image signature verification policy. The container image signature verification policy is disabled by default. For production workloads, you must use signature verification to ensure container images are not tampered with.
- Create the resource policy config map.
-
Create the
KBSConfigCR. - Create the cluster route.
- Create the authentication secret.
- Update the RVPS config map with the reference values.
- Verify the Red Hat build of Trustee configuration.
3.3. Installing the Red Hat build of Trustee Operator Copy linkLink copied to clipboard!
You install the Red Hat build of Trustee Operator on an OpenShift Container Platform cluster in a trusted environment.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI tool (
oc).
Procedure
Create a
trustee-namespace.yamlmanifest file:apiVersion: v1 kind: Namespace metadata: name: trustee-operator-system
apiVersion: v1 kind: Namespace metadata: name: trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
trustee-operator-systemnamespace by running the following command:oc create -f trustee-namespace.yaml
$ oc create -f trustee-namespace.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-operatorgroup.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the operator group by running the following command:
oc create -f trustee-operatorgroup.yaml
$ oc create -f trustee-operatorgroup.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-subscription.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the subscription by running the following command:
oc create -f trustee-subscription.yaml
$ oc create -f trustee-subscription.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the Operator is correctly installed by running the following command:
oc get csv -n trustee-operator-system
$ oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command can take several minutes to complete.
Watch the process by running the following command:
watch oc get csv -n trustee-operator-system
$ watch oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 Succeeded
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 SucceededCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.4. Creating HTTPS secrets Copy linkLink copied to clipboard!
Generate keys to securely launch Red Hat build of Trustee and enables services to use HTTPS.
Procedure
Set the
DOMAINvariable for the cluster by running the following command:DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')$ DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
NSvariable for the Red Hat build of Trustee namespace by running the following command:NS=trustee-operator-system
$ NS=trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
ROUTE_NAMEvariable by running the following command:ROUTE_NAME=kbs-service
$ ROUTE_NAME=kbs-serviceCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
ROUTEvariable to the full DNS name by running the following command:ROUTE="${ROUTE_NAME}-${NS}.${DOMAIN}"$ ROUTE="${ROUTE_NAME}-${NS}.${DOMAIN}"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a private SSL/TLS key and certificate for Red Hat build of Trustee by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
<custom_cn>is a custom CN. For example:kbs-trustee-operator-system. -
<custom_org>is a name of your organization.
-
Create the
kbs-https-certificatesecret in thetrustee-operator-systemnamespace by running the following command:oc create secret generic kbs-https-certificate --from-file=tls.crt -n trustee-operator-system
$ oc create secret generic kbs-https-certificate --from-file=tls.crt -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
kbs-https-keysecret in thetrustee-operator-systemnamespace by running the following command:oc create secret generic kbs-https-key --from-file=tls.key -n trustee-operator-system
$ oc create secret generic kbs-https-key --from-file=tls.key -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.5. Creating the attestation token secrets Copy linkLink copied to clipboard!
Generate an attestation token key and certificate for Red Hat build of Trustee.
Procedure
Generate a private elliptic curve SSL key called
token.keyby running the following command:openssl ecparam -name prime256v1 -genkey -noout -out token.key
$ openssl ecparam -name prime256v1 -genkey -noout -out token.keyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
attestation-keysecret from the SSL/TLS key in thetrustee-operator-systemnamespace:oc create secret generic attestation-key \ --from-file=token.key \ -n trustee-operator-system
$ oc create secret generic attestation-key \ --from-file=token.key \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a self-signed SSL/TLS certificate from the private SSL key by running the following command:
openssl req -new -x509 -key token.key -out token.crt -days 365 \ -subj "/CN=<custom_cn>/O=<custom_org>"
$ openssl req -new -x509 -key token.key -out token.crt -days 365 \ -subj "/CN=<custom_cn>/O=<custom_org>"Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
<custom_cn>: Specify the Common Name. For example:kbs-trustee-operator-system. -
<custom_org>: Specify your organization name.
-
Create the
attestation-certsecret from the SSL/TLS key and certificate in thetrustee-operator-systemnamespace:oc create secret generic attestation-cert \ --from-file=token.crt \ -n trustee-operator-system
$ oc create secret generic attestation-cert \ --from-file=token.crt \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
attestation-statussecret used for verifying the attestation process:oc create secret generic attestation-status \ --from-literal=status=success \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal=status=success \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.6. Creating the kbs-config config map Copy linkLink copied to clipboard!
You create the kbs-config config map to configure Red Hat build of Trustee.
Procedure
Create a
kbs-config-cm.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the config map by running the following command:
oc create -f kbs-config-cm.yaml
$ oc create -f kbs-config-cm.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.7. Creating the RVPS config map Copy linkLink copied to clipboard!
You create the Reference Value Provider Service (RVPS) config map, which specifies the reference values for your Trusted Execution Environment (TEE).
The client collects measurements from the running software, the TEE hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to Red Hat build of Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.
The data.reference-values.json stanza must be present, but it can be empty.
Do not use this configuration example in a production environment. Initially, you create an empty RVPS config map. Then, you update the RVPS config map with reference values for your TEE.
Procedure
Create an
rvps-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the RVPS config map by running the following command:
oc create -f rvps-configmap.yaml
$ oc create -f rvps-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.8. Creating the attestation policy config map Copy linkLink copied to clipboard!
You create an attestation policy config map to define attestation policies for Red Hat build of Trustee.
The attestation policy follows the Open Policy Agent specification.
This policy checks the Platform Configuration Register (PCR) values 03, 08, 09, 11, and 12 values against the reference values to ensure that the confidential containers pod uses the specified restrictive Kata agent policy and that the Red Hat pod VM image has not been altered. For details, see Linux TPM PCR Registry in the UAPI Group Specifications documentation.
Procedure
Create an
attestation-policy.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the attestation policy config map by running the following command:
oc create -f attestation-policy.yaml
$ oc create -f attestation-policy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.9. Creating a tdx-config config map Copy linkLink copied to clipboard!
Create a config map for Intel® Trust Domain Extensions (TDX).
Procedure
Create a
tdx-config.yamlmanifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
tdx-configconfig map by running the following command:oc create -f tdx-config.yaml
$ oc create -f tdx-config.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.10. Creating a secret with custom keys for clients Copy linkLink copied to clipboard!
You can create a secret that contains one or more custom keys for Red Hat build of Trustee clients.
In this example, the attestation-status secret has two entries (key1, key2), which the clients retrieve. You can add additional secrets according to your requirements by using the same format.
Prerequisites
- You have created one or more custom keys.
Procedure
Create a secret for the custom keys according to the following example:
oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \ --from-literal key2=<custom_key2> \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \1 --from-literal key2=<custom_key2> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Specify a custom key.
You specify the
attestation-statussecret in thespec.kbsSecretResourceskey of theKbsConfigcustom resource manifest.
3.11. Creating a secret for container image signature verification Copy linkLink copied to clipboard!
If you use container image signature verification, you must create a secret that contains the public container image signing key.
The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.
You can use Red Hat Trusted Artifact Signer or other tools to sign container images.
Procedure
Create a secret for container image signature verification by running the following command:
oc create secret generic <type> \ --from-file=<tag>=./<public_key_file> \ -n trustee-operator-system
$ oc create secret generic <type> \1 --from-file=<tag>=./<public_key_file> \2 -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the
<type>value. You must add this value to thespec.kbsSecretResourceskey when you create theKbsConfigcustom resource.
3.12. Creating the container image signature verification policy Copy linkLink copied to clipboard!
You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.
Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself. For more information about signature keys, see Creating the attestation token secret.
Procedure
Create a
security-policy-config.jsonfile according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- By default, the policy rejects all images and all signatures. The transports section specifies which images the policy explicitly approves and verifies through their signatures.
- 2
- Specify the image repository for
transport, for example,"docker":. For more information, see containers-transports 5. - 3
- Specify the container registry and image, for example, "quay.io/my-image".
- 4
- Specify the type and tag of the container image signature verification secret that you created, for example,
img-sig/pub-key.
Create the security policy by running the following command:
oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-system
$ oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
<security-policy-name>secret is specified in thespec.kbsSecretResourceskey of theKbsConfigcustom resource.
3.13. Creating the resource policy config map Copy linkLink copied to clipboard!
You configure the resource policy config map for the policy engine. This policy determines which resources are accessible to Red Hat build of Trustee.
This policy engine is different from the Attestation Service policy engine, which determines the validity of TEE evidence.
Procedure
Create a
resourcepolicy-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - policy.rego
-
The name of the resource policy,
policy.rego, must match the resource policy defined in thekbs-configconfig map. - package policy
- The resource policy follows the Open Policy Agent specification.
Create the resource policy config map by running the following command:
oc create -f resourcepolicy-configmap.yaml
$ oc create -f resourcepolicy-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.14. Creating the cluster route Copy linkLink copied to clipboard!
You create a secure route with edge TLS termination for the cluster where you installed Red Hat build of Trustee.
External ingress traffic reaches the router pods as HTTPS and passes on to the pods running in the trustee-operator-system namespace as HTTP.
Procedure
Create an edge route by running the following command:
oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-system
$ oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
TRUSTEE_HOSTvariable by running the following command:TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})$ TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the route by running the following command:
echo $TRUSTEE_HOST
$ echo $TRUSTEE_HOSTCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.15. Creating the authentication secret Copy linkLink copied to clipboard!
You create the authentication secret for Red Hat build of Trustee.
Procedure
Create a private key by running the following command:
openssl genpkey -algorithm ed25519 > privateKey
$ openssl genpkey -algorithm ed25519 > privateKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public key by running the following command:
openssl pkey -in privateKey -pubout -out publicKey
$ openssl pkey -in privateKey -pubout -out publicKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a secret by running the following command:
oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-system
$ oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the secret by running the following command:
oc get secret -n trustee-operator-system
$ oc get secret -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.16. Creating the KbsConfig custom resource Copy linkLink copied to clipboard!
Create the KbsConfig custom resource (CR) to launch Red Hat build of Trustee.
Procedure
Create a
kbsconfig-cr.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
kbsSecretResources: Specify thetypevalue of the container image signature verification secret if you created the secret, for example,img-sig. -
Uncomment
tdxConfigSpec.kbsTdxConfigMapName: tdx-configfor Intel Trust Domain Extensions. -
Uncomment
kbsServiceType: <service_type>if you create a service type, other than the defaultClusterIPservice, to expose applications within the cluster external traffic. You can specifyNodePort,LoadBalancer, orExternalName.
-
Create the
KbsConfigCR by running the following command:oc create -f kbsconfig-cr.yaml
$ oc create -f kbsconfig-cr.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.17. Updating the RVPS config map Copy linkLink copied to clipboard!
You update the Reference Value Provider Service (RVPS) config map with expected measurements, including the Platform Configuration Register (PCR) 8 value, for the trusted execution environment. Red Hat build of Trustee uses these measurements to verify the attestation evidence.
The workload cluster administrator calculates the PCR8 value by performing a SHA256 hash on a configuration or policy file such as initdata:
hash=$(sha256sum <config_file> | cut -d' ' -f1) initial_pcr=0000000000000000000000000000000000000000000000000000000000000000 PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1)
$ hash=$(sha256sum <config_file> | cut -d' ' -f1)
$ initial_pcr=0000000000000000000000000000000000000000000000000000000000000000
$ PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1)
Prerequisites
- PCR8 value, expiration, and algorithm, created by the workload cluster administrator
Procedure
Create an
rvps-configmap-update.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteDo not use this configuration example in a production environment.
The
"value"can be any JSON type (string, number, boolean, array, object). The JSON type must be combined with the operand used in theattestation-policy. See the following examples for valid"value"types:If the attestation rule is:
input.sample.platform_version.major == data.reference.major_version
input.sample.platform_version.major == data.reference.major_versionCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
==operand expects to match an integer type in the reference values:{ "name": "major_version", "expiration": "2027-01-01T00:00:00Z", "value" : 1 }{ "name": "major_version", "expiration": "2027-01-01T00:00:00Z", "value" : 1 }Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the attestation rule is:
input.sample.svn in data.reference.svn
input.sample.svn in data.reference.svnCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
inoperand expects to match an array type in the reference values:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Update the RVPS config map by running the following command:
oc apply -f rvps-configmap-update.yaml
$ oc apply -f rvps-configmap-update.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.18. Verifying the configuration Copy linkLink copied to clipboard!
You verify the Red Hat build of Trustee configuration by checking its pods and logs.
Procedure
Set the default project by running the following command:
oc project trustee-operator-system
$ oc project trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pods by running the following command:
oc get pods -n trustee-operator-system
$ oc get pods -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59m
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
POD_NAMEenvironmental variable by running the following command:POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pod logs by running the following command:
oc logs -n trustee-operator-system $POD_NAME
$ oc logs -n trustee-operator-system $POD_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 4. Deploying Red Hat build of Trustee for workloads running on IBM Z and IBM LinuxONE Copy linkLink copied to clipboard!
You can deploy Red Hat build of Trustee for confidential containers workloads running on IBM Z® and IBM® LinuxONE.
4.1. Prerequisites Copy linkLink copied to clipboard!
- You have installed the latest version of Red Hat OpenShift Container Platform in a trusted environment. For more information, see Installing OpenShift Container Platform on bare metal.
- You are using LinuxONE Emperor 4.
- You have enabled Secure Unpack Facility on your Logical Partition (LPAR), which is necessary for the IBM Secure Execution. For more information, see Enabling the KVM host for IBM Secure Execution.
4.2. Deployment overview Copy linkLink copied to clipboard!
You deploy Red Hat build of Trustee by performing the following steps:
- Install the Red Hat build of Trustee Operator.
-
Optional: Create the
kbs-configconfig map if you are using Intel® Trust Domain Extensions (TDX) remote attestation. - Create the Reference Value Provider Service (RVPS) config map.
- Configure the IBM Secure Execution certificates and keys.
- Create the IBM persistent storage components.
- Create the attestation policy config map.
- Optional: Create a secret for custom keys clients.
- Optional: Create a secret for container image signature verification.
- Create the container image signature verification policy. The container image signature verification policy is disabled by default. For production workloads, you must use signature verification to ensure container images are not tampered with.
- Create the resource policy config map.
-
Create the
KBSConfigCR. - Create the cluster route.
- Create the authentication secret.
- Verify the Red Hat build of Trustee configuration.
4.3. Installing the Red Hat build of Trustee Operator Copy linkLink copied to clipboard!
You install the Red Hat build of Trustee Operator on an OpenShift Container Platform cluster in a trusted environment.
Prerequisites
-
You have access to the cluster as a user with the
cluster-adminrole. -
You have installed the OpenShift CLI tool (
oc).
Procedure
Create a
trustee-namespace.yamlmanifest file:apiVersion: v1 kind: Namespace metadata: name: trustee-operator-system
apiVersion: v1 kind: Namespace metadata: name: trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
trustee-operator-systemnamespace by running the following command:oc create -f trustee-namespace.yaml
$ oc create -f trustee-namespace.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-operatorgroup.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the operator group by running the following command:
oc create -f trustee-operatorgroup.yaml
$ oc create -f trustee-operatorgroup.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
trustee-subscription.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the subscription by running the following command:
oc create -f trustee-subscription.yaml
$ oc create -f trustee-subscription.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the Operator is correctly installed by running the following command:
oc get csv -n trustee-operator-system
$ oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command can take several minutes to complete.
Watch the process by running the following command:
watch oc get csv -n trustee-operator-system
$ watch oc get csv -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 Succeeded
NAME DISPLAY PHASE trustee-operator.v1.0.0 Trustee Operator 1.0.0 SucceededCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.4. Creating the kbs-config config map Copy linkLink copied to clipboard!
You create the kbs-config config map to configure Red Hat build of Trustee.
The following configuration example turns off security features. Do not use this example in a production environment.
Procedure
Create a
kbs-config-cm.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the config map by running the following command:
oc create -f kbs-config-cm.yaml
$ oc create -f kbs-config-cm.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.5. Creating the RVPS config map Copy linkLink copied to clipboard!
You create the Reference Value Provider Service (RVPS) config map, which specifies the reference values for your Trusted Execution Environment (TEE).
The client collects measurements from the running software, the TEE hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to Red Hat build of Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.
The data.reference-values.json stanza must be present, but it can be empty.
Do not use this configuration example in a production environment. Initially, you create an empty RVPS config map. Then, you update the RVPS config map with reference values for your TEE.
Procedure
Create an
rvps-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the RVPS config map by running the following command:
oc create -f rvps-configmap.yaml
$ oc create -f rvps-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6. Configuring the IBM Secure Execution certificates and keys Copy linkLink copied to clipboard!
You must configure the IBM Secure Execution (SE) certificates and keys for your worker nodes.
Prerequisites
- You have the IP address of the bastion node.
- You have the internal IP addresses of the worker nodes.
Procedure
- Generate the Key Broker Service (KBS) certificate and key.
- Obtain the attestation policy fields.
- Download the certificates and certificate revocation lists (CRLs).
- Generate the RSA keys.
- Verify and copy files to the OpenShift Container Platform worker nodes.
- Create the secrets in the cluster with the KBS key and certificate.
4.6.1. Generating the Key Broker Service certificate and key Copy linkLink copied to clipboard!
You must generate the Key Broker Service (KBS) certificate and key.
Procedure
Create the
kbs.confconfiguration file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow <worker_node_ip>Obtain the IP address of a worker node by running the following command:
oc get node $(oc get pod -n trustee-operator-system -o jsonpath='{.items[0].spec.nodeName}') -o jsonpath='{.status.addresses[?(@.type=="InternalIP")].address}'$ oc get node $(oc get pod -n trustee-operator-system -o jsonpath='{.items[0].spec.nodeName}') -o jsonpath='{.status.addresses[?(@.type=="InternalIP")].address}'Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Generate the KBS key and self-signed certificate by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the KBS key to the
ibmsedirectory by running the following command:cp kbs.key /tmp/ibmse/kbs.key
$ cp kbs.key /tmp/ibmse/kbs.keyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the KBS certificate to the
ibmsedirectory by running the following command:cp kbs.crt /tmp/ibmse/kbs.crt
$ cp kbs.crt /tmp/ibmse/kbs.crtCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6.2. Obtaining the attestation policy fields Copy linkLink copied to clipboard!
You must obtain the attestation policy fields by using Reference Value Provider Service (RVPS).
Procedure
Create a directory to download the
GetRvps.shscript by running the following command:mkdir -p Rvps-Extraction/
$ mkdir -p Rvps-Extraction/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Download the script by running the following command:
wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/GetRvps.sh -O $PWD/GetRvps.sh
$ wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/GetRvps.sh -O $PWD/GetRvps.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a subdirectory by running the following command:
mkdir -p Rvps-Extraction/static-files
$ mkdir -p Rvps-Extraction/static-filesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Go to the
static-filesdirectory by running the following command:cd Rvps-Extraction/static-files
$ cd Rvps-Extraction/static-filesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
pvextract-hdrtool by running the following command:wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/static-files/pvextract-hdr -O $PWD/pvextract-hdr
$ wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/static-files/pvextract-hdr -O $PWD/pvextract-hdrCopy to Clipboard Copied! Toggle word wrap Toggle overflow Make the tool executable by running the following command:
chmod +x pvextract-hdr
$ chmod +x pvextract-hdrCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
se_parse_hdr.pyscript by running the following command:wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/static-files/se_parse_hdr.py -O $PWD/se_parse_hdr.py
$ wget https://github.com/openshift/sandboxed-containers-operator/raw/devel/scripts/rvps-extraction/static-files/se_parse_hdr.py -O $PWD/se_parse_hdr.pyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy your Host Key Document (HKD) certificate to the
static-filesdirectory by running the following command:cp ~/path/to/<hkd_cert.crt> .
$ cp ~/path/to/<hkd_cert.crt> .Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
static-filesdirectory contains the following files:-
HKD.crt -
pvextract-hdr -
se_parse_hdr.py
-
Go to the
Rvps-Extractiondirectory by running the following command:cd ..
$ cd ..Copy to Clipboard Copied! Toggle word wrap Toggle overflow Make the
GetRvps.shscript executable by running the following command:chmod +x GetRvps.sh
$ chmod +x GetRvps.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the script:
./GetRvps.sh
$ ./GetRvps.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enter
2to generate the Reference Value Provider Service from the volume:Please enter your choice: 2
Please enter your choice: 2Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enter
fa-ppfor the libvirt pool name:Enter the Libvirt Pool Name: fa-pp
Enter the Libvirt Pool Name: fa-ppCopy to Clipboard Copied! Toggle word wrap Toggle overflow Enter the libvirt gateway URI:
Enter the Libvirt URI Name: <libvirt-uri>
Enter the Libvirt URI Name: <libvirt-uri>1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Specify the
LIBVIRT_URIvalue that you used to create the peer pods secret.
Enter
fa-pp-volfor the libvirt volume name:Enter the Libvirt Volume Name: fa-pp-vol
Enter the Libvirt Volume Name: fa-pp-volCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Obtain the attestation policy field values by running the following command:
cat /root/Rvps-Extraction/output-files/se-sample
$ cat /root/Rvps-Extraction/output-files/se-sampleCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6.3. Downloading the certificates and certificate revocation lists Copy linkLink copied to clipboard!
You must download the IBM certificates and Certificate Revocation Lists (CRLs).
Procedure
Create a temporary directory for certificates by running the following command:
mkdir /tmp/ibmse/certs
$ mkdir /tmp/ibmse/certsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
ibm-z-host-key-signing-gen2.crtcertificate by running the following command:wget https://www.ibm.com/support/resourcelink/api/content/public/ibm-z-host-key-signing-gen2.crt -O /tmp/ibmse/certs/ibm-z-host-key-signing-gen2.crt
$ wget https://www.ibm.com/support/resourcelink/api/content/public/ibm-z-host-key-signing-gen2.crt -O /tmp/ibmse/certs/ibm-z-host-key-signing-gen2.crtCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
DigiCertCA.crtcertificate by running the following command:wget https://www.ibm.com/support/resourcelink/api/content/public/DigiCertCA.crt -O /tmp/ibmse/certs/DigiCertCA.crt
$ wget https://www.ibm.com/support/resourcelink/api/content/public/DigiCertCA.crt -O /tmp/ibmse/certs/DigiCertCA.crtCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a temporary directory for the CRLs by running the following command:
mkdir /tmp/ibmse/crls
$ mkdir /tmp/ibmse/crlsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
ibm-z-host-key-gen2.crlfile by running the following command:wget https://www.ibm.com/support/resourcelink/api/content/public/ibm-z-host-key-gen2.crl -O /tmp/ibmse/crls/ibm-z-host-key-gen2.crl
$ wget https://www.ibm.com/support/resourcelink/api/content/public/ibm-z-host-key-gen2.crl -O /tmp/ibmse/crls/ibm-z-host-key-gen2.crlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
DigiCertTrustedRootG4.crlfile by running the following command:wget http://crl3.digicert.com/DigiCertTrustedRootG4.crl -O /tmp/ibmse/crls/DigiCertTrustedRootG4.crl
$ wget http://crl3.digicert.com/DigiCertTrustedRootG4.crl -O /tmp/ibmse/crls/DigiCertTrustedRootG4.crlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Download the
DigiCertTrustedG4CodeSigningRSA4096SHA3842021CA1.crlfile by running the following command:wget http://crl3.digicert.com/DigiCertTrustedG4CodeSigningRSA4096SHA3842021CA1.crl -O /tmp/ibmse/crls/DigiCertTrustedG4CodeSigningRSA4096SHA3842021CA1.crl
$ wget http://crl3.digicert.com/DigiCertTrustedG4CodeSigningRSA4096SHA3842021CA1.crl -O /tmp/ibmse/crls/DigiCertTrustedG4CodeSigningRSA4096SHA3842021CA1.crlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a temporary directory for the
hdr.binfile by running the following command:mkdir -p /tmp/ibmse/hdr/
$ mkdir -p /tmp/ibmse/hdr/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the
hdr.binfile to thehdrdirectory by running the following command:cp /root/Rvps-Extraction/output-files/hdr.bin /tmp/ibmse/hdr/
$ cp /root/Rvps-Extraction/output-files/hdr.bin /tmp/ibmse/hdr/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a temporary directory for Host Key Document (HKD) certificate by running the following command:
mkdir -p /tmp/ibmse/hkds
$ mkdir -p /tmp/ibmse/hkdsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy your HKD certificate to the
hkdsdirectory by running the following command:cp ~/path/to/<hkd_cert.crt> /tmp/ibmse/hkds/
$ cp ~/path/to/<hkd_cert.crt> /tmp/ibmse/hkds/Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6.4. Generating the RSA keys Copy linkLink copied to clipboard!
You must generate the RSA key pair for encryption.
Procedure
Generate an RSA key pair by running the following command:
openssl genrsa -aes256 -passout pass:<password> -out /tmp/encrypt_key-psw.pem 4096
$ openssl genrsa -aes256 -passout pass:<password> -out /tmp/encrypt_key-psw.pem 40961 Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a temporary directory for the RSA keys by running the following command:
mkdir -p /tmp/ibmse/rsa
$ mkdir -p /tmp/ibmse/rsaCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create an
encrypt_key.pubkey by running the following command:openssl rsa -in /tmp/encrypt_key-psw.pem -passin pass:<password> -pubout -out /tmp/ibmse/rsa/encrypt_key.pub
$ openssl rsa -in /tmp/encrypt_key-psw.pem -passin pass:<password> -pubout -out /tmp/ibmse/rsa/encrypt_key.pubCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create an
encrypt_key.pemkey by running the following command:openssl rsa -in /tmp/encrypt_key-psw.pem -passin pass:<password> -out /tmp/ibmse/rsa/encrypt_key.pem
$ openssl rsa -in /tmp/encrypt_key-psw.pem -passin pass:<password> -out /tmp/ibmse/rsa/encrypt_key.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6.5. Verifying and copying files to the worker nodes Copy linkLink copied to clipboard!
You must verify the /tmp/ibmse folder structure and copy the files to the OpenShift Container Platform worker nodes.
Procedure
Verify the structure of the
/tmp/ibmsedirectory by running the following command:tree /tmp/ibmse
$ tree /tmp/ibmseCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a compressed file from the
/tmp/ibmsedirectory by running the following command:tar -czf ibmse.tar.gz -C /tmp/ibmse
$ tar -czf ibmse.tar.gz -C /tmp/ibmseCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the
.tar.gzfile to the bastion node in your cluster by running the following command:scp /tmp/ibmse.tar.gz root@<ocp_bastion_ip>:/tmp
$ scp /tmp/ibmse.tar.gz root@<ocp_bastion_ip>:/tmp1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow Connect to the bastion node over SSH by running the following command:
ssh root@<ocp_bastion_ip>
$ ssh root@<ocp_bastion_ip>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the
.tar.gzfile to each worker node by running the following command:scp /tmp/ibmse.tar.gz core@<worker_node_ip>:/tmp
$ scp /tmp/ibmse.tar.gz core@<worker_node_ip>:/tmp1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow Extract the
.tar.gzon each worker node by running the following command:ssh core@<worker_node_ip> 'sudo mkdir -p /opt/confidential-containers/ && sudo tar -xzf /tmp/ibmse.tar.gz -C /opt/confidential-containers/'
$ ssh core@<worker_node_ip> 'sudo mkdir -p /opt/confidential-containers/ && sudo tar -xzf /tmp/ibmse.tar.gz -C /opt/confidential-containers/'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the
ibmsefolder permissions by running the following command:ssh core@<worker_node_ip> 'sudo chmod -R 755 /opt/confidential-containers/ibmse/'
$ ssh core@<worker_node_ip> 'sudo chmod -R 755 /opt/confidential-containers/ibmse/'Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6.6. Creating the secrets for Key Broker Service Copy linkLink copied to clipboard!
You must create the secrets in the cluster with the Key Broker Service (KBS) key and certificate.
Procedure
Convert the
kbs.crtfile to a Base64-encoded string by running the following command:cat /tmp/ibmse/kbs.crt | base64 -w 0
$ cat /tmp/ibmse/kbs.crt | base64 -w 0Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the string for the
kbs-https-certificate.yamlmanifest. Create a
kbs-https-certificate.yamlmanifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow <kbs_crt_value>-
Specify the Base64-encoded string you created from the
kbs.crtfile.
Create the secret with the KBS certificate by running the following command:
oc create -f kbs-https-certificate.yaml
$ oc create -f kbs-https-certificate.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Convert the
kbs.keyfile to a Base64-encoded string by running the following command:cat /tmp/ibmse/kbs.key | base64 -w 0
$ cat /tmp/ibmse/kbs.key | base64 -w 0Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the string for the
kbs-https-key.yamlmanifest. Create a
kbs-https-key.yamlmanifest file according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow <kbs_key_value>-
Specify the Base64-encoded string you created from the
kbs.keyfile.
Create the secret with the KBS key by running the following command:
oc create -f kbs-https-key.yaml
$ oc create -f kbs-https-key.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.7. Creating the IBM persistent storage components Copy linkLink copied to clipboard!
You must create the persistent volume (PV) and persistent volume claim (PVC) to mount the ibmse folder on the trustee-deployment pod.
Procedure
Create a
persistent-volume.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the persistent volume by running the following command:
oc create -f persistent-volume.yaml
$ oc create -f persistent-volume.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
persistent-volume-claim.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the persistent volume claim by running the following command:
oc create -f persistent-volume-claim.yaml
$ oc create -f persistent-volume-claim.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.8. Creating the attestation policy config map Copy linkLink copied to clipboard!
You create an attestation policy config map to define attestation policies for Red Hat build of Trustee.
The attestation policy follows the Open Policy Agent specification.
Procedure
Create an
attestation-policy.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow default.rego- Do not modify the policy name.
Create the attestation policy config map by running the following command:
oc create -f attestation-policy.yaml
$ oc create -f attestation-policy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.9. Creating a secret with custom keys for clients Copy linkLink copied to clipboard!
You can create a secret that contains one or more custom keys for Red Hat build of Trustee clients.
In this example, the attestation-status secret has two entries (key1, key2), which the clients retrieve. You can add additional secrets according to your requirements by using the same format.
Prerequisites
- You have created one or more custom keys.
Procedure
Create a secret for the custom keys according to the following example:
oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \ --from-literal key2=<custom_key2> \ -n trustee-operator-system
$ oc create secret generic attestation-status \ --from-literal key1=<custom_key1> \1 --from-literal key2=<custom_key2> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow You specify the
attestation-statussecret in thespec.kbsSecretResourceskey of theKbsConfigcustom resource manifest.
4.10. Creating a secret for container image signature verification Copy linkLink copied to clipboard!
If you use container image signature verification, you must create a secret that contains the public container image signing key.
The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.
You can use Red Hat Trusted Artifact Signer or other tools to sign container images.
Procedure
Create a secret for container image signature verification by running the following command:
oc create secret generic <type> \ --from-file=<tag>=./<public_key_file> \ -n trustee-operator-system
$ oc create secret generic <type> \1 --from-file=<tag>=./<public_key_file> \2 -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Record the
<type>value. You must add this value to thespec.kbsSecretResourceskey when you create theKbsConfigcustom resource.
4.11. Creating the container image signature verification policy Copy linkLink copied to clipboard!
You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.
Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself. For more information about signature keys, see Creating the attestation token secret.
Procedure
Create a
security-policy-config.jsonfile according to the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the security policy by running the following command:
oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-system
$ oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
<security-policy-name>secret is specified in thespec.kbsSecretResourceskey of theKbsConfigcustom resource.
4.12. Creating the resource policy config map Copy linkLink copied to clipboard!
You configure the resource policy config map for the policy engine. This policy determines which resources are accessible to Red Hat build of Trustee.
This policy engine is different from the Attestation Service policy engine, which determines the validity of TEE evidence.
Procedure
Create a
resourcepolicy-configmap.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - policy.rego
-
The name of the resource policy,
policy.rego, must match the resource policy defined in thekbs-configconfig map. - package policy
- The resource policy follows the Open Policy Agent specification.
Create the resource policy config map by running the following command:
oc create -f resourcepolicy-configmap.yaml
$ oc create -f resourcepolicy-configmap.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.13. Creating the cluster route Copy linkLink copied to clipboard!
You create a secure route with edge TLS termination for the cluster where you installed Red Hat build of Trustee.
External ingress traffic reaches the router pods as HTTPS and passes on to the pods running in the trustee-operator-system namespace as HTTP.
Procedure
Create an edge route by running the following command:
oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-system
$ oc create route passthrough --service=kbs-service --port kbs-port \ -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
TRUSTEE_HOSTvariable by running the following command:TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})$ TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \ -o jsonpath={.spec.host})Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the route by running the following command:
echo $TRUSTEE_HOST
$ echo $TRUSTEE_HOSTCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io
kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.14. Creating the authentication secret Copy linkLink copied to clipboard!
You create the authentication secret for Red Hat build of Trustee.
Procedure
Create a private key by running the following command:
openssl genpkey -algorithm ed25519 > privateKey
$ openssl genpkey -algorithm ed25519 > privateKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a public key by running the following command:
openssl pkey -in privateKey -pubout -out publicKey
$ openssl pkey -in privateKey -pubout -out publicKeyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a secret by running the following command:
oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-system
$ oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the secret by running the following command:
oc get secret -n trustee-operator-system
$ oc get secret -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.15. Creating the KbsConfig custom resource Copy linkLink copied to clipboard!
Create the KbsConfig custom resource (CR) to launch Red Hat build of Trustee.
Procedure
Create a
kbsconfig-cr.yamlmanifest file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
kbsSecretResources: Specify thetypevalue of the container image signature verification secret if you created the secret, for example,img-sig. -
SE_SKIP_CERTS_VERIFICATION: Set to
trueonly for testing purposes.
-
Create the
KbsConfigCR by running the following command:oc create -f kbsconfig-cr.yaml
$ oc create -f kbsconfig-cr.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.16. Verifying the configuration Copy linkLink copied to clipboard!
You verify the Red Hat build of Trustee configuration by checking its pods and logs.
Procedure
Set the default project by running the following command:
oc project trustee-operator-system
$ oc project trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pods by running the following command:
oc get pods -n trustee-operator-system
$ oc get pods -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59m
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
POD_NAMEenvironmental variable by running the following command:POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the pod logs by running the following command:
oc logs -n trustee-operator-system $POD_NAME
$ oc logs -n trustee-operator-system $POD_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 5. Uninstalling Copy linkLink copied to clipboard!
You uninstall by performing the following tasks:
-
Delete the
KbsConfigcustom resource. - Uninstall the Red Hat build of Trustee Operator.
-
Delete the
KbsConfigcustom resource definition.
5.1. Deleting the KbsConfig custom resource Copy linkLink copied to clipboard!
You delete the KbsConfig custom resource (CR) by using the command line.
Procedure
Delete the
KbsConfigCR by running the following command:oc delete kbsconfig kbsconfig
$ oc delete kbsconfig kbsconfigCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify the CR removal by running the following command:
oc get kbsconfig kbsconfig
$ oc get kbsconfig kbsconfigCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
No kbsconfig instances exist
No kbsconfig instances existCopy to Clipboard Copied! Toggle word wrap Toggle overflow
You must ensure that all pods are deleted. Any remaining pod resources might result in an unexpected bill from your cloud provider.
5.2. Uninstalling the Red Hat build of Trustee Operator Copy linkLink copied to clipboard!
You uninstall the Red Hat build of Trustee Operator by using the command line.
Procedure
Delete the subscription by running the following command:
oc delete subscription trustee-operator -n trustee-operator-system
$ oc delete subscription trustee-operator -n trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Delete the namespace by running the following command:
oc delete namespace trustee-operator-system
$ oc delete namespace trustee-operator-systemCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.3. Deleting the KbsConfig CRD Copy linkLink copied to clipboard!
You delete the KbsConfig custom resource definition (CRD) by using the command line.
Prerequisites
-
You have deleted the
KbsConfigcustom resource. - You have uninstalled the Red Hat build of Trustee Operator.
Procedure
Delete the
KbsConfigCRD by running the following command:oc delete crd kataconfigs.kataconfiguration.openshift.io
$ oc delete crd kataconfigs.kataconfiguration.openshift.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the CRD was deleted by running the following command:
oc get crd kataconfigs.kataconfiguration.openshift.io
$ oc get crd kataconfigs.kataconfiguration.openshift.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Unknown CRD kataconfigs.kataconfiguration.openshift.io
Unknown CRD kataconfigs.kataconfiguration.openshift.ioCopy to Clipboard Copied! Toggle word wrap Toggle overflow