Chapter 3. Deploying Red Hat build of Trustee for workloads running on Microsoft Azure Red Hat OpenShift


You can deploy Red Hat build of Trustee for confidential containers workloads running on Azure Red Hat OpenShift.

3.1. Prerequisites

3.2. Deployment overview

You deploy Red Hat build of Trustee by performing the following steps:

  1. Install the Red Hat build of Trustee Operator.
  2. Create HTTPS secrets.
  3. Create the attestation token secret.
  4. Optional: Create the kbs-config config map if you are using Intel® Trust Domain Extensions (TDX) remote attestation.
  5. Create the Reference Value Provider Service (RVPS) config map. Initially, you create an empty config map for the reference values. You update the values after you create KBSConfig custom resource (CR).
  6. Create the attestation policy config map.
  7. Optional: Create a config map for Intel® TDX.
  8. Optional: Create a secret for custom keys clients.
  9. Optional: Create a secret for container image signature verification.
  10. Create the container image signature verification policy. The container image signature verification policy is disabled by default. For production workloads, you must use signature verification to ensure container images are not tampered with.
  11. Create the resource policy config map.
  12. Create the KBSConfig CR.
  13. Create the cluster route.
  14. Create the authentication secret.
  15. Update the RVPS config map with the reference values.
  16. Verify the Red Hat build of Trustee configuration.

You install the Red Hat build of Trustee Operator on an OpenShift Container Platform cluster in a trusted environment.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.
  • You have installed the OpenShift CLI tool (oc).

Procedure

  1. Create a trustee-namespace.yaml manifest file:

    apiVersion: v1
    kind: Namespace
    metadata:
      name: trustee-operator-system
  2. Create the trustee-operator-system namespace by running the following command:

    $ oc create -f trustee-namespace.yaml
  3. Create a trustee-operatorgroup.yaml manifest file:

    apiVersion: operators.coreos.com/v1
    kind: OperatorGroup
    metadata:
      name: trustee-operator-group
      namespace: trustee-operator-system
    spec:
      targetNamespaces:
      - trustee-operator-system
  4. Create the operator group by running the following command:

    $ oc create -f trustee-operatorgroup.yaml
  5. Create a trustee-subscription.yaml manifest file:

    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: trustee-operator-system
      namespace: trustee-operator-system
    spec:
      channel: stable
      installPlanApproval: Automatic
      name: trustee-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
  6. Create the subscription by running the following command:

    $ oc create -f trustee-subscription.yaml
  7. Verify that the Operator is correctly installed by running the following command:

    $ oc get csv -n trustee-operator-system

    This command can take several minutes to complete.

  8. Watch the process by running the following command:

    $ watch oc get csv -n trustee-operator-system

    Example output

    NAME                      DISPLAY                        PHASE
    trustee-operator.v1.0.0   Trustee Operator  1.0.0        Succeeded

3.4. Creating HTTPS secrets

Generate keys to securely launch Red Hat build of Trustee and enables services to use HTTPS.

Procedure

  1. Set the DOMAIN variable for the cluster by running the following command:

    $ DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}')
  2. Set the NS variable for the Red Hat build of Trustee namespace by running the following command:

    $ NS=trustee-operator-system
  3. Set the ROUTE_NAME variable by running the following command:

    $ ROUTE_NAME=kbs-service
  4. Set the ROUTE variable to the full DNS name by running the following command:

    $ ROUTE="${ROUTE_NAME}-${NS}.${DOMAIN}"
  5. Generate a private SSL/TLS key and certificate for Red Hat build of Trustee by running the following command:

    $ openssl req -x509 -nodes -days 365 \
      -newkey rsa:2048 \
      -keyout tls.key \
      -out tls.crt \
      -subj "/CN=<custom_cn>/O=<custom_org>" \
      -addext "subjectAltName=DNS:${ROUTE}"
    • <custom_cn> is a custom CN. For example: kbs-trustee-operator-system.
    • <custom_org> is a name of your organization.
  6. Create the kbs-https-certificate secret in the trustee-operator-system namespace by running the following command:

    $ oc create secret generic kbs-https-certificate --from-file=tls.crt -n trustee-operator-system
  7. Create the kbs-https-key secret in the trustee-operator-system namespace by running the following command:

    $ oc create secret generic kbs-https-key --from-file=tls.key -n trustee-operator-system

3.5. Creating the attestation token secrets

Generate an attestation token key and certificate for Red Hat build of Trustee.

Procedure

  1. Generate a private elliptic curve SSL key called token.key by running the following command:

    $ openssl ecparam -name prime256v1 -genkey -noout -out token.key
  2. Create the attestation-key secret from the SSL/TLS key in the trustee-operator-system namespace:

    $ oc create secret generic attestation-key \
      --from-file=token.key \
      -n trustee-operator-system
  3. Generate a self-signed SSL/TLS certificate from the private SSL key by running the following command:

    $ openssl req -new -x509 -key token.key -out token.crt -days 365 \
      -subj "/CN=<custom_cn>/O=<custom_org>"
    • <custom_cn>: Specify the Common Name. For example: kbs-trustee-operator-system.
    • <custom_org>: Specify your organization name.
  4. Create the attestation-cert secret from the SSL/TLS key and certificate in the trustee-operator-system namespace:

    $ oc create secret generic attestation-cert \
      --from-file=token.crt \
      -n trustee-operator-system
  5. Create the attestation-status secret used for verifying the attestation process:

    $ oc create secret generic attestation-status \
      --from-literal=status=success \
      -n trustee-operator-system

3.6. Creating the kbs-config config map

You create the kbs-config config map to configure Red Hat build of Trustee.

Procedure

  1. Create a kbs-config-cm.yaml manifest file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: kbs-config-cm
      namespace: trustee-operator-system
    data:
      kbs-config.toml: |
        [http_server]
        sockets = ["0.0.0.0:8080"]
        insecure_http = false
        private_key = "/etc/https-key/tls.key"
        certificate = "/etc/https-cert/tls.crt"
        worker_count = 4
    
        [admin]
        insecure_api = false
    
        auth_public_key = "/etc/auth-secret/publicKey"
    
        [attestation_token]
        insecure_key = false
        trusted_certs_paths = ["/etc/attestation-cert/token.crt"]
        attestation_token_type = "CoCo"
    
        [attestation_service]
        type = "coco_as_builtin"
        work_dir = "/opt/confidential-containers/attestation-service"
        policy_engine = "opa"
    
        [attestation_service.attestation_token_broker]
        type = "Ear"
        policy_dir = "/opt/confidential-containers/attestation-service/policies"
    
        [attestation_service.attestation_token_broker.signer]
        key_path = "/etc/attestation-key/token.key"
        cert_path = "/etc/attestation-cert/token.crt"
    
        [attestation_service.attestation_token_config]
        duration_min = 5
    
        [attestation_service.rvps_config]
        type = "BuiltIn"
    
        [attestation_service.rvps_config.storage]
        type = "LocalJson"
        file_path = "/opt/confidential-containers/rvps/reference-values/reference-values.json"
    
        [[plugins]]
        name = "resource"
        type = "LocalFs"
        dir_path = "/opt/confidential-containers/kbs/repository"
    
        [policy_engine]
        policy_path = "/opt/confidential-containers/opa/policy.rego"
  2. Create the config map by running the following command:

    $ oc create -f kbs-config-cm.yaml

3.7. Creating the RVPS config map

You create the Reference Value Provider Service (RVPS) config map, which specifies the reference values for your Trusted Execution Environment (TEE).

The client collects measurements from the running software, the TEE hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to Red Hat build of Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.

The data.reference-values.json stanza must be present, but it can be empty.

Note

Do not use this configuration example in a production environment. Initially, you create an empty RVPS config map. Then, you update the RVPS config map with reference values for your TEE.

Procedure

  1. Create an rvps-configmap.yaml manifest file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: rvps-reference-values
      namespace: trustee-operator-system
    data:
      reference-values.json: |
        [
        ]
  2. Create the RVPS config map by running the following command:

    $ oc create -f rvps-configmap.yaml

3.8. Creating the attestation policy config map

You create an attestation policy config map to define attestation policies for Red Hat build of Trustee.

The attestation policy follows the Open Policy Agent specification.

This policy checks the Platform Configuration Register (PCR) values 03, 08, 09, 11, and 12 values against the reference values to ensure that the confidential containers pod uses the specified restrictive Kata agent policy and that the Red Hat pod VM image has not been altered. For details, see Linux TPM PCR Registry in the UAPI Group Specifications documentation.

Procedure

  1. Create an attestation-policy.yaml manifest file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: attestation-policy
      namespace: trustee-operator-system
    data:
      default_cpu.rego: |
        package policy
    
        import rego.v1
        default executables := 33
        default hardware := 97
        default configuration := 36
    
        ##### Azure vTPM SNP
        executables := 3 if {
          input.azsnpvtpm.measurement in data.reference.measurement
          input.azsnpvtpm.tpm.pcr11 in data.reference.snp_pcr11
        }
    
        hardware := 2 if {
          input.azsnpvtpm.reported_tcb_bootloader in data.reference.tcb_bootloader
          input.azsnpvtpm.reported_tcb_microcode in data.reference.tcb_microcode
          input.azsnpvtpm.reported_tcb_snp in data.reference.tcb_snp
          input.azsnpvtpm.reported_tcb_tee in data.reference.tcb_tee
        }
    
        configuration := 2 if {
          input.azsnpvtpm.platform_smt_enabled in data.reference.smt_enabled
          input.azsnpvtpm.platform_tsme_enabled in data.reference.tsme_enabled
          input.azsnpvtpm.policy_abi_major in data.reference.abi_major
          input.azsnpvtpm.policy_abi_minor in data.reference.abi_minor
          input.azsnpvtpm.policy_single_socket in data.reference.single_socket
          input.azsnpvtpm.policy_smt_allowed in data.reference.smt_allowed
        }
    
        ##### Azure vTPM TDX
        executables := 3 if {
          input.aztdxvtpm.tpm.pcr03 in data.reference.tdx_pcr03
          input.aztdxvtpm.tpm.pcr08 in data.reference.tdx_pcr08
          input.aztdxvtpm.tpm.pcr09 in data.reference.tdx_pcr09
          input.aztdxvtpm.tpm.pcr11 in data.reference.tdx_pcr11
          input.aztdxvtpm.tpm.pcr12 in data.reference.tdx_pcr12
        }
    
        hardware := 2 if {
          # Check the quote is a TDX quote signed by Intel SGX Quoting Enclave
          input.aztdxvtpm.quote.header.tee_type == "81000000"
          input.aztdxvtpm.quote.header.vendor_id == "939a7233f79c4ca9940a0db3957f0607"
    
          # Check TDX Module version and its hash. Also check OVMF code hash.
          # input.aztdxvtpm.quote.body.mr_seam in data.reference.mr_seam
          # input.aztdxvtpm.quote.body.tcb_svn in data.reference.tcb_svn
          # input.aztdxvtpm.quote.body.mr_td in data.reference.mr_td
        }
    
        configuration := 2 if {
          # input.aztdxvtpm.quote.body.xfam in data.reference.xfam
        }
  2. Create the attestation policy config map by running the following command:

    $ oc create -f attestation-policy.yaml

3.9. Creating a tdx-config config map

Create a config map for Intel® Trust Domain Extensions (TDX).

Procedure

  1. Create a tdx-config.yaml manifest file according to the following example:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: tdx-config
      namespace: trustee-operator-system
    data:
      sgx_default_qcnl.conf: |
        {
          "collateral_service": "https://api.trustedservices.intel.com/sgx/certification/v4/"
        }
  2. Create the tdx-config config map by running the following command:

    $ oc create -f tdx-config.yaml

You can create a secret that contains one or more custom keys for Red Hat build of Trustee clients.

In this example, the attestation-status secret has two entries (key1, key2), which the clients retrieve. You can add additional secrets according to your requirements by using the same format.

Prerequisites

  • You have created one or more custom keys.

Procedure

  • Create a secret for the custom keys according to the following example:

    $ oc create secret generic attestation-status \
      --from-literal key1=<custom_key1> \ 
    1
    
      --from-literal key2=<custom_key2> \
      -n trustee-operator-system
    1
    Specify a custom key.

    You specify the attestation-status secret in the spec.kbsSecretResources key of the KbsConfig custom resource manifest.

If you use container image signature verification, you must create a secret that contains the public container image signing key.

The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.

You can use Red Hat Trusted Artifact Signer or other tools to sign container images.

Procedure

  1. Create a secret for container image signature verification by running the following command:

    $ oc create secret generic <type> \ 
    1
    
      --from-file=<tag>=./<public_key_file> \ 
    2
    
      -n trustee-operator-system
    1
    Specify the KBS secret type, for example, img-sig.
    2
    Specify the secret tag, for example, pub-key, and the public container image signing key.
  2. Record the <type> value. You must add this value to the spec.kbsSecretResources key when you create the KbsConfig custom resource.

You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.

Note

Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself. For more information about signature keys, see Creating the attestation token secret.

Procedure

  1. Create a security-policy-config.json file according to the following example:

    {
      "default": [
          {
          "type": "reject" 
    1
    
          }
      ],
      "transports": {
          "<transport>": { 
    2
    
              "<registry>/<image>": 
    3
    
              [
                  {
                      "type": "sigstoreSigned",
                      "keyPath": "kbs:///default/<type>/<tag>" 
    4
    
                  }
              ]
          }
      }
    }
    1
    By default, the policy rejects all images and all signatures. The transports section specifies which images the policy explicitly approves and verifies through their signatures.
    2
    Specify the image repository for transport, for example, "docker":. For more information, see containers-transports 5.
    3
    Specify the container registry and image, for example, "quay.io/my-image".
    4
    Specify the type and tag of the container image signature verification secret that you created, for example, img-sig/pub-key.
  2. Create the security policy by running the following command:

    $ oc create secret generic <security-policy-name> \
      --from-file=<osc-key>=./<security-policy-config.json> \
      -n trustee-operator-system

    The <security-policy-name> secret is specified in the spec.kbsSecretResources key of the KbsConfig custom resource.

3.13. Creating the resource policy config map

You configure the resource policy config map for the policy engine. This policy determines which resources are accessible to Red Hat build of Trustee.

Note

This policy engine is different from the Attestation Service policy engine, which determines the validity of TEE evidence.

Procedure

  1. Create a resourcepolicy-configmap.yaml manifest file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: resource-policy
      namespace: trustee-operator-system
    data:
      policy.rego: |
        package policy
        default allow = true
        allow {
          input["submods"]["cpu0"]["ear.status"] == "affirming"
        }
    policy.rego
    The name of the resource policy, policy.rego, must match the resource policy defined in the kbs-config config map.
    package policy
    The resource policy follows the Open Policy Agent specification.
  2. Create the resource policy config map by running the following command:

    $ oc create -f resourcepolicy-configmap.yaml

3.14. Creating the cluster route

You create a secure route with edge TLS termination for the cluster where you installed Red Hat build of Trustee.

External ingress traffic reaches the router pods as HTTPS and passes on to the pods running in the trustee-operator-system namespace as HTTP.

Procedure

  1. Create an edge route by running the following command:

    $ oc create route passthrough --service=kbs-service --port kbs-port \
      -n trustee-operator-system
  2. Set the TRUSTEE_HOST variable by running the following command:

    $ TRUSTEE_HOST=$(oc get route -n trustee-operator-system kbs-service \
      -o jsonpath={.spec.host})
  3. Verify the route by running the following command:

    $ echo $TRUSTEE_HOST

    Example output

    kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io

3.15. Creating the authentication secret

You create the authentication secret for Red Hat build of Trustee.

Procedure

  1. Create a private key by running the following command:

    $ openssl genpkey -algorithm ed25519 > privateKey
  2. Create a public key by running the following command:

    $ openssl pkey -in privateKey -pubout -out publicKey
  3. Create a secret by running the following command:

    $ oc create secret generic kbs-auth-public-key \
      --from-file=publicKey -n trustee-operator-system
  4. Verify the secret by running the following command:

    $ oc get secret -n trustee-operator-system

3.16. Creating the KbsConfig custom resource

Create the KbsConfig custom resource (CR) to launch Red Hat build of Trustee.

Procedure

  1. Create a kbsconfig-cr.yaml manifest file:

    apiVersion: confidentialcontainers.org/v1alpha1
    kind: KbsConfig
    metadata:
      labels:
        app.kubernetes.io/name: kbsconfig
        app.kubernetes.io/instance: kbsconfig
        app.kubernetes.io/part-of: trustee-operator
        app.kubernetes.io/managed-by: kustomize
        app.kubernetes.io/created-by: trustee-operator
      name: kbsconfig
      namespace: trustee-operator-system
    spec:
      kbsConfigMapName: kbs-config-cm
      kbsAuthSecretName: kbs-auth-public-key
      kbsDeploymentType: AllInOneDeployment
      kbsRvpsRefValuesConfigMapName: rvps-reference-values
      kbsSecretResources:
      - attestation-status
      - <security-policy-name>
      kbsResourcePolicyConfigMapName: resource-policy
      kbsHttpsKeySecretName: kbs-https-key
      kbsHttpsCertSecretName: kbs-https-certificate
      kbsAttestationCertSecretName: attestation-cert
      kbsAttestationKeySecretName: attestation-key
    # tdxConfigSpec:
    #   kbsTdxConfigMapName: tdx-config
    # kbsServiceType: <service_type>
    • kbsSecretResources: Specify the type value of the container image signature verification secret if you created the secret, for example, img-sig.
    • Uncomment tdxConfigSpec.kbsTdxConfigMapName: tdx-config for Intel Trust Domain Extensions.
    • Uncomment kbsServiceType: <service_type> if you create a service type, other than the default ClusterIP service, to expose applications within the cluster external traffic. You can specify NodePort, LoadBalancer, or ExternalName.
  2. Create the KbsConfig CR by running the following command:

    $ oc create -f kbsconfig-cr.yaml

3.17. Updating the RVPS config map

You update the Reference Value Provider Service (RVPS) config map with expected measurements, including the Platform Configuration Register (PCR) 8 value, for the trusted execution environment. Red Hat build of Trustee uses these measurements to verify the attestation evidence.

The workload cluster administrator calculates the PCR8 value by performing a SHA256 hash on a configuration or policy file such as initdata:

$ hash=$(sha256sum <config_file> | cut -d' ' -f1)

$ initial_pcr=0000000000000000000000000000000000000000000000000000000000000000

$ PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1)

Prerequisites

  • PCR8 value, expiration, and algorithm, created by the workload cluster administrator

Procedure

  1. Create an rvps-configmap-update.yaml manifest file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: rvps-reference-values
      namespace: trustee-operator-system
    data:
      reference-values.json: |
        [
         {
            "name": "svn",
            "expiration": "2027-01-01T00:00:00Z",
            "value" : 1
          },
          {
            "name": "major_version",
            "expiration": "2027-01-01T00:00:00Z",
            "value" : 1
          },
          {
            "name": "minimum_minor_version",
            "expiration": "2027-01-01T00:00:00Z",
            "value" : 4
          }
        ]
    Note

    Do not use this configuration example in a production environment.

    The "value" can be any JSON type (string, number, boolean, array, object). The JSON type must be combined with the operand used in the attestation-policy. See the following examples for valid "value" types:

    • If the attestation rule is:

      input.sample.platform_version.major == data.reference.major_version

      The == operand expects to match an integer type in the reference values:

      {
      "name": "major_version",
      "expiration": "2027-01-01T00:00:00Z",
      "value" : 1
      }
    • If the attestation rule is:

      input.sample.svn in data.reference.svn

      The in operand expects to match an array type in the reference values:

      {
      "name": "svn",
      "expiration": "2027-01-01T00:00:00Z",
      "value" : [
      1
      ]
      }
  2. Update the RVPS config map by running the following command:

    $ oc apply -f rvps-configmap-update.yaml

3.18. Verifying the configuration

You verify the Red Hat build of Trustee configuration by checking its pods and logs.

Procedure

  1. Set the default project by running the following command:

    $ oc project trustee-operator-system
  2. Check the pods by running the following command:

    $ oc get pods -n trustee-operator-system

    Example output

    NAME                                                   READY   STATUS    RESTARTS   AGE
    trustee-deployment-8585f98449-9bbgl                    1/1     Running   0          22m
    trustee-operator-controller-manager-5fbd44cd97-55dlh   2/2     Running   0          59m

  3. Set the POD_NAME environmental variable by running the following command:

    $ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)
  4. Check the pod logs by running the following command:

    $ oc logs -n trustee-operator-system $POD_NAME

    Example output

    [2024-05-30T13:44:24Z INFO  kbs] Using config file /etc/kbs-config/kbs-config.json
    [2024-05-30T13:44:24Z WARN  attestation_service::rvps] No RVPS address provided and will launch a built-in rvps
    [2024-05-30T13:44:24Z INFO  attestation_service::token::simple] No Token Signer key in config file, create an ephemeral key and without CA pubkey cert
    [2024-05-30T13:44:24Z INFO  api_server] Starting HTTPS server at [0.0.0.0:8080]
    [2024-05-30T13:44:24Z INFO  actix_server::builder] starting 12 workers
    [2024-05-30T13:44:24Z INFO  actix_server::server] Tokio runtime found; starting in existing Tokio runtime

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red Hat
Back to top