Chapter 3. Configure

Procedure

1. Generate an HTTPS certificate and key for secure communication by running the following command:

   $ DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')
   $ ROUTE="kbs-route-trustee-operator-system.${DOMAIN}"
   $ openssl req -x509 -nodes -days 365 \
     -newkey rsa:2048 \
     -keyout tls.key \
     -out tls.crt \
     -subj "/CN=kbs-trustee-operator-system/O=Red Hat" \
     -addext "subjectAltName=DNS:${ROUTE}"

   This command creates tls.crt (the HTTPS certificate) and tls.key (the HTTPS private key).

2. Create a Kubernetes TLS secret from the certificate and key by running the following command:

   $ oc create secret tls trustee-tls-cert -n trustee-operator-system \
     --cert=tls.crt \
     --key=tls.key

3. Create a TrusteeConfig resource with the Restricted profile by running the following command:

   $ cat <<EOF | oc apply -f -
   apiVersion: trustee.confidentialcontainers.org/v1
   kind: TrusteeConfig
   metadata:
     name: trustee-config
     namespace: trustee-operator-system
   spec:
     profileType: Restricted
     kbsServiceType: ClusterIP
     httpsSpec:
       tlsSecretName: trustee-tls-cert
   EOF

   The Restricted profile provides production-ready security controls with HTTPS encryption and stricter resource policies.

   You can modify the YAML configuration to customize the deployment:

   • Change profileType: Restricted to Permissive for testing or staging environments where you want to test a permissive policy first.
   • Change kbsServiceType: ClusterIP to NodePort or LoadBalancer for different network exposure requirements.

4. Verify that TrusteeConfig was created successfully by running the following command:

   $ oc get trusteeconfig -n trustee-operator-system

   NAME              AGE
   trustee-config    10s

5. Verify that the operator automatically generated the required resources by running the following command:

   $ oc get secrets,cm,kbsconfig -n trustee-operator-system

   NAME                                       TYPE                 DATA   AGE
   secret/kbs-auth                            Opaque               1      15s
   
   NAME                                            DATA   AGE
   configmap/trustee-config-kbs-config             3      15s
   configmap/trustee-config-rvps-reference-values  1      15s
   configmap/trustee-config-resource-policy        1      15s
   configmap/trustee-config-attestation-policy-cpu 1      15s
   configmap/trustee-config-attestation-policy-gpu 1      15s
   
   NAME                                              AGE
   kbsconfig.trustee.confidentialcontainers.org/trustee-config   15s

Verification

1. Verify that the key broker service (KBS) pod is running by running the following command:

   $ oc get pods -n trustee-operator-system -l app=kbs

   NAME                                  READY   STATUS    RESTARTS   AGE
   trustee-deployment-768fd9bf4d-kb6x6   1/1     Running   0          14h

2. Verify that the Trustee service is accessible by running the following command:

   $ oc get svc -n trustee-operator-system kbs-service

   NAME          TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
   kbs-service   ClusterIP  10.x.x.x        <none>        8080/TCP   1m

Procedure

1. To customize the autogenerated KbsConfig resource, edit the resource directly by running the following command:

   $ oc edit kbsconfig <kbsconfig-name> -n trustee-operator-system

   Common KbsConfig customizations include:

   • Adjusting the number of Trustee replicas:

     spec:
       KbsDeploymentSpec:
         replicas: 3

   • Adding secret resources for workload access:

     spec:
       kbsSecretResources:
         - kbsres1
         - kbsres2

   • Configuring disconnected environment support:

     spec:
       kbsLocalCertCacheSpec:
         # Add disconnected environment configuration

2. To customize the resource access policy, edit the autogenerated config map by running the following command:

   $ oc edit cm -n trustee-operator-system <trusteeconfig-name>-resource-policy

3. To customize the CPU attestation policy, edit the autogenerated config map by running the following command:

   $ oc edit cm -n trustee-operator-system <trusteeconfig-name>-attestation-policy-cpu

4. To customize the GPU attestation policy, edit the autogenerated config map by running the following command:

   $ oc edit cm -n trustee-operator-system <trusteeconfig-name>-attestation-policy-gpu

5. To customize Reference Value Provider Service (RVPS) reference values, edit the autogenerated config map by running the following command:

   $ oc edit cm -n trustee-operator-system <trusteeconfig-name>-rvps-reference-values

6. To configure container image signature verification, edit the autogenerated KbsConfig resource to add secret references by running the following command:

   $ oc edit kbsconfig <kbsconfig-name> -n trustee-operator-system

   Add the secret names to the kbsSecretResources list:

   spec:
     kbsSecretResources:
       - img-sig
       - security-policy

   where:

   • img-sig is the secret containing the public container image signing key
   • security-policy is the secret containing the signature verification policy

   For more information about creating these secrets, see Creating a secret for container image signature verification and Create the container image signature verification policy.

Verification

1. Verify that the Trustee pods restart after configuration changes by running the following command:

   $ oc get pods -n trustee-operator-system

2. Check the Trustee operator logs for any configuration errors by running the following command:

   $ oc logs -n trustee-operator-system deployment/trustee-operator-controller-manager

Procedure

1. Retrieve the ConfigMap name and verify the NVIDIA verifier configuration:

   $ NAMESPACE="trustee-operator-system"
   $ KBSCONFIG_NAME=$(kubectl get kbsconfig -n "$NAMESPACE" \
       -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
   $ oc get configmap $KBSCONFIG_NAME -n trustee-operator-system -o yaml

   The ConfigMap should include the following configuration:

   [attestation_service.verifier_config.nvidia_verifier]
   type = "Remote"

   The type = "Remote" setting configures Red Hat build of Trustee to forward GPU attestation evidence to NRAS for verification. The verifier_url parameter is optional and defaults to https://nras.attestation.nvidia.com/v4/attest.

2. Verify the NVIDIA verifier initialization in the Red Hat build of Trustee logs:

   $ oc logs -n trustee-operator-system deployment/trustee-deployment | grep -i nvidia

   You should see log entries indicating that the NVIDIA verifier initialized in remote mode.

   Example output

   INFO nvidia_verifier: Initialized NVIDIA verifier in Remote mode

Procedure

1. Create a secret for container image signature verification by running the following command:

   $ oc create secret generic <type> \
     --from-file=<tag>=./<public_key_file> \
     -n trustee-operator-system

   where:

   <type>

   Specifies the secret type, for example, img-sig.

   <public_key_file>

   Specifies the secret tag, for example, pub-key, and the public container image signing key.

2. Record the <type> value. You must add this value to the spec.kbsSecretResources key when you create the KbsConfig custom resource.

Procedure

1. Update the rvps-configmap.yaml manifest file by running the following command:

   $ podman run \
     -v ./pull-secret.json:/pull-secret.json \
     -v ./initdata.toml:/initdata.toml \
     quay.io/openshift_sandboxed_containers/coco-tools:1.12 \
     veritas --platform baremetal --tee <tee> \
     --authfile /pull-secret.json \
     --initdata /initdata.toml > rvps-configmap.yaml

   <tee>

   Specifies the Intel® Trust Domain Extensions (TDX) or AMD SEV-SNP Trusted Execution Environments (TEEs)

2. Update the RVPS config map by running the following command:

   $ oc apply -f rvps-configmap.yaml

   Note

   After you apply the updated RVPS config map, the new values are available to Red Hat build of Trustee automatically. You do not need to restart the service. This synchronization process might take several minutes to propagate throughout the system. Review the generated RVPS config map to ensure the values match your platform and software versions. You can regenerate and update the RVPS values at any time, for example when upgrading OpenShift Container Platform or Red Hat OpenShift sandboxed containers. For details, see Updating the RVPS config map.

Procedure

1. Create a security-policy-config.json file according to the following example:

   {
     "default": [
         {
         "type": "reject"
         }
     ],
     "transports": {
         "<transport>": {
             "<registry>/<image>":
             [
                 {
                     "type": "<type>",
                     "keyPath": "kbs:///default/<type>/<tag>"
                 }
             ]
         }
     }
   }

   where:

   transports

   Specifies which images the policy explicitly approves and verifies through their signatures. By default, the policy rejects all images and all signatures. For more information, see containers-transports 5.

   keyPath

   Specifies the type and tag of the container image signature verification secret that you created, for example, img-sig/pub-key.

2. Create the security policy by running the following command:

   $ oc create secret generic <security-policy-name> \
     --from-file=<osc-key>=./<security-policy-config.json> \
     -n trustee-operator-system

   where:

   <security-policy-name>

   Specifies the value of the spec.kbsSecretResources key of the KbsConfig custom resource.

Procedure

1. Switch to the Red Hat build of Trustee Operator project by running the following command:

   $ oc project trustee-operator-system

2. Check the pods by running the following command:

   $ oc get pods -n trustee-operator-system

   Example output

   NAME                                                   READY   STATUS    RESTARTS   AGE
   trustee-deployment-8585f98449-9bbgl                    1/1     Running   0          22m
   trustee-operator-controller-manager-5fbd44cd97-55dlh   2/2     Running   0          59m

3. Set the POD_NAME environmental variable by running the following command:

   $ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)

4. Check the pod logs by running the following command:

   $ oc logs -n trustee-operator-system $POD_NAME

   Example output

   [2024-05-30T13:44:24Z INFO  kbs] Using config file /etc/kbs-config/kbs-config.json
   [2024-05-30T13:44:24Z WARN  attestation_service::rvps] No RVPS address provided and will launch a built-in rvps
   [2024-05-30T13:44:24Z INFO  attestation_service::token::simple] No Token Signer key in config file, create an ephemeral key and without CA pubkey cert
   [2024-05-30T13:44:24Z INFO  api_server] Starting HTTPS server at [0.0.0.0:8080]
   [2024-05-30T13:44:24Z INFO  actix_server::builder] starting 12 workers
   [2024-05-30T13:44:24Z INFO  actix_server::server] Tokio runtime found; starting in existing Tokio runtime