Chapter 3. Configure
You can configure Red Hat build of Trustee for confidential containers workloads running on Microsoft Azure with the Intel® Trust Domain Extensions (TDX) or AMD SEV-SNP Trusted Execution Environments (TEEs) using the TrusteeConfig custom resource. The TrusteeConfig custom resource simplifies Red Hat build of Trustee deployment by reducing configuration complexity.
A single custom resource defines all configuration while the operator automatically generates secrets, config maps, and the KbsConfig resource.
Permissive and Restricted profile types provide pre-configured security levels, while platform-specific extensions are integrated into the specification.
Use TrusteeConfig for new deployments and existing deployments planning to simplify configuration. To get started, see Create TrusteeConfig custom resources.
After creating a TrusteeConfig resource, you can customize the autogenerated resources if you need advanced configuration. For more information, see Customize TrusteeConfig autogenerated resources.
3.1. Understanding TrusteeConfig custom resources Copy linkLink copied to clipboard!
You can simplify Red Hat build of Trustee configuration by using the TrusteeConfig custom resource. TrusteeConfig is a Kubernetes custom resource that simplifies Red Hat build of Trustee deployment by automating resource creation. It includes the secrets, config maps, and lower-level resources required for deployment.
TrusteeConfig uses profile types to determine security and configuration settings.
- Permissive profile
The Permissive profile provides a simplified configuration suitable for development and testing environments. Use the Permissive profile when you need a quick evaluation for deployment or development.
With Permissive profile:
- Key Broker Service (KBS) service is exposed by using a Kubernetes service with ClusterIP
- Resource policy is set to allow all requests
- Configuration is minimal and focuses on essential settings
- Restricted profile
The Restricted profile enforces stricter security controls suitable for production environments. Use the Restricted profile for production deployments where security and compliance are critical.
With Restricted profile:
-
You must provide HTTPS certificate and key in the
TrusteeConfigspecification - KBS service is exposed by using a secure HTTPS route with proper certificate validation
- Resource policy is configured to enforce specific access controls
- Attestation token verification is enabled and enforced
-
You must provide HTTPS certificate and key in the
- Service type configuration
The
kbsServiceTypefield determines how the KBS service is exposed in your cluster.Expand Service type Use case Default ClusterIP
Services within the cluster; typical for OpenShift with routes
Yes
NodePort
Direct access to worker nodes; use when routes are not available
No
LoadBalancer
External load balancer integration; use with cloud provider load balancers
No
Most deployments use the default ClusterIP with OpenShift routes for secure external access.
- HTTPS configuration
For the Restricted profile, you must provide HTTPS certificates for secure communication. For Permissive profile deployments, HTTPS configuration is optional.
You create a Kubernetes TLS secret containing your certificate and private key, then reference it in the
TrusteeConfigspecification:- Create the TLS secret using cert-manager, OpenShift’s certificate management, or manually
-
Reference the secret name in
httpsSpec.tlsSecretName - The Operator uses this to configure secure HTTPS communication
- Attestation token verification
You can optionally configure attestation token verification for enhanced security.
Production Restricted profile deployments require the following setting.
TrusteeConfigsupports optional platform-specific configurations that you can specify in theTrusteeConfigresource. Each platform has documentation covering platform-specific extensions and configuration options.Create a Kubernetes TLS secret containing the attestation token verification certificate, then reference it in the
TrusteeConfigspecification:- Create the TLS secret with the attestation token verification certificate
-
Reference the secret name in
attestationTokenVerificationSpec.tlsSecretName - This establishes trust between clients and the Trustee service
- Generated resources
-
When you create a
TrusteeConfigresource, the Trustee Operator automatically creates the following resources: - Config maps
-
<name>-kbs-config- Contains KBS service configuration -
<name>-rvps-reference-values- Contains Reference Value Provider Service (RVPS) reference value settings -
<name>-attestation-policy-cpu- Contains CPU attestation policies -
<name>-attestation-policy-gpu- Contains GPU attestation policies -
<name>-resource-policy- Contains the resource access policy
where
<name>is yourTrusteeConfigresource name.-
- Secrets
-
<name>-kbs-auth- Contains authentication credentials -
<name>-https- Contains HTTPS certificate and key (whenhttpsSpecis provided) -
<name>-attestation-token- Contains attestation token verification certificate (whenattestationTokenSpecis provided)
where
<name>is yourTrusteeConfigresource name.-
- Custom resources
-
<name>-kbsconfig- TheKbsConfigcustom resource that configures the Trustee services
where
<name>is yourTrusteeConfigresource name.-
- Services and routes
The Operator creates Kubernetes services and OpenShift routes based on your service specification:
-
Service type (ClusterIP, NodePort, or LoadBalancer) as specified in
kbsServiceType - OpenShift route when using ClusterIP service type
-
HTTPS configuration when
httpsSpecis provided
The Operator generates these resources based on your
TrusteeConfigspecification. You do not manually create them when usingTrusteeConfig. You can customize these autogenerated resources after creation if you need advanced configuration.-
Service type (ClusterIP, NodePort, or LoadBalancer) as specified in
TrusteeConfigas the starting pointUse
TrusteeConfigas the starting point for all Red Hat build of Trustee deployments.TrusteeConfigprovides automated resource generation and simplifies configuration management.After creating a
TrusteeConfigresource, you can customize the autogenerated resources if you need advanced configuration beyond theTrusteeConfigspecification.Use the manual KbsConfig approach only for advanced use cases where you need extensive customization beyond
TrusteeConfigcapabilities or when migrating from existing KbsConfig deployments.
Additional resources
3.2. Create TrusteeConfig custom resources Copy linkLink copied to clipboard!
You can create a TrusteeConfig custom resource to deploy Red Hat build of Trustee with simplified configuration. TrusteeConfig automatically generates the required secrets, config maps, and KbsConfig resource.
Prerequisites
- The Red Hat build of Trustee Operator is installed in the cluster.
- You have access to the cluster with administrator privileges.
-
The Red Hat build of Trustee namespace (
trustee-operator-system) exists.
Procedure
Generate an HTTPS certificate and key for secure communication by running the following command:
$ DOMAIN=$(oc get ingress.config/cluster -o jsonpath='{.spec.domain}') $ ROUTE="kbs-route-trustee-operator-system.${DOMAIN}" $ openssl req -x509 -nodes -days 365 \ -newkey rsa:2048 \ -keyout tls.key \ -out tls.crt \ -subj "/CN=kbs-trustee-operator-system/O=Red Hat" \ -addext "subjectAltName=DNS:${ROUTE}"This command creates
tls.crt(the HTTPS certificate) andtls.key(the HTTPS private key).Create a Kubernetes TLS secret from the certificate and key by running the following command:
$ oc create secret tls trustee-tls-cert -n trustee-operator-system \ --cert=tls.crt \ --key=tls.keyCreate a
TrusteeConfigresource with the Restricted profile by running the following command:$ cat <<EOF | oc apply -f - apiVersion: trustee.confidentialcontainers.org/v1 kind: TrusteeConfig metadata: name: trustee-config namespace: trustee-operator-system spec: profileType: Restricted kbsServiceType: ClusterIP httpsSpec: tlsSecretName: trustee-tls-cert EOFThe Restricted profile provides production-ready security controls with HTTPS encryption and stricter resource policies.
You can modify the YAML configuration to customize the deployment:
-
Change
profileType: RestrictedtoPermissivefor testing or staging environments where you want to test a permissive policy first. -
Change
kbsServiceType: ClusterIPtoNodePortorLoadBalancerfor different network exposure requirements.
-
Change
Verify that
TrusteeConfigwas created successfully by running the following command:$ oc get trusteeconfig -n trustee-operator-systemNAME AGE trustee-config 10sVerify that the operator automatically generated the required resources by running the following command:
$ oc get secrets,cm,kbsconfig -n trustee-operator-systemNAME TYPE DATA AGE secret/kbs-auth Opaque 1 15s NAME DATA AGE configmap/trustee-config-kbs-config 3 15s configmap/trustee-config-rvps-reference-values 1 15s configmap/trustee-config-resource-policy 1 15s configmap/trustee-config-attestation-policy-cpu 1 15s configmap/trustee-config-attestation-policy-gpu 1 15s NAME AGE kbsconfig.trustee.confidentialcontainers.org/trustee-config 15s
Verification
Verify that the key broker service (KBS) pod is running by running the following command:
$ oc get pods -n trustee-operator-system -l app=kbsNAME READY STATUS RESTARTS AGE trustee-deployment-768fd9bf4d-kb6x6 1/1 Running 0 14hVerify that the Trustee service is accessible by running the following command:
$ oc get svc -n trustee-operator-system kbs-serviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kbs-service ClusterIP 10.x.x.x <none> 8080/TCP 1m
3.3. TrusteeConfig custom resource field reference Copy linkLink copied to clipboard!
This reference provides complete documentation for all fields in the TrusteeConfig custom resource specification. Use this reference when creating or modifying TrusteeConfig resources.
TrusteeConfig uses the following specification structure:
apiVersion: trustee.confidentialcontainers.org/v1
kind: TrusteeConfig
metadata:
name: trustee-config
namespace: trustee-operator-system
spec:
profileType: Permissive | Restricted
kbsServiceType: ClusterIP | NodePort | LoadBalancer
httpsSpec:
tlsSecretName: <tls-secret-name>
attestationTokenVerificationSpec:
tlsSecretName: <token-verification-secret-name>
Core specification fields
| Field | Type | Description | Required |
|---|---|---|---|
|
| String |
Determines the security profile and configuration level. Values: | Yes |
|
| String |
Type of Kubernetes service for exposing the Key Broker Service (KBS). Values: | No |
|
| Object |
HTTPS configuration containing certificate and private key for secure communication. Required for | Depends on profile |
|
| Object | Attestation token verification settings for enhanced security and trust establishment. | No |
profileType values
| Value | Description |
|---|---|
|
| Development and testing profile that auto-generates self-signed HTTPS certificates and uses permissive resource policies. Suitable for evaluation and non-production environments. The KBS service is exposed internally via ClusterIP. |
|
| Production profile that enforces stricter security controls. Requires manual provision of HTTPS certificate and key. Enables attestation token verification and enforces resource access policies. The KBS service is exposed via secure HTTPS routes with certificate validation. |
kbsServiceType values
| Value | Description | Use case |
|---|---|---|
|
| Kubernetes internal service accessible only from within the cluster. Uses OpenShift routes for external access. | Standard deployments on OpenShift. Default value. |
|
| Service accessible via a port on each worker node. Direct node access without routes. | Deployments requiring direct worker node access or when routes are unavailable. |
|
| Service uses cloud provider load balancer for external access. | Deployments on cloud platforms with integrated load balancer support. |
httpsSpec fields
Required for Restricted profile deployments. Optional for Permissive profile (autogenerated if not provided).
| Field | Type | Description |
|---|---|---|
|
| String | Name of a Kubernetes TLS secret containing the HTTPS certificate and private key. This is the recommended approach for providing HTTPS certificates. |
Example:
httpsSpec:
tlsSecretName: trustee-tls-cert
Create the TLS secret before referencing it:
$ oc create secret tls trustee-tls-cert -n trustee-operator-system \
--cert=tls.crt \
--key=tls.key
attestationTokenVerificationSpec fields
Optional specification for attestation token verification and enhanced security.
| Field | Type | Description |
|---|---|---|
|
| String | Name of a Kubernetes TLS secret containing the certificate for verifying attestation tokens from clients. |
Example:
attestationTokenVerificationSpec:
tlsSecretName: trustee-token-cert
Complete example: Permissive profile
apiVersion: trustee.confidentialcontainers.org/v1
kind: TrusteeConfig
metadata:
name: trustee-config
namespace: trustee-operator-system
spec:
profileType: Permissive
kbsServiceType: ClusterIP
Complete example: Restricted profile
apiVersion: trustee.confidentialcontainers.org/v1
kind: TrusteeConfig
metadata:
name: trustee-config
namespace: trustee-operator-system
spec:
profileType: Restricted
kbsServiceType: ClusterIP
httpsSpec:
tlsSecretName: trustee-tls-cert
attestationTokenVerificationSpec:
tlsSecretName: trustee-token-cert
Field validation rules
TrusteeConfig enforces the following validation rules:
-
profileTypemust be eitherPermissiveorRestricted(required). -
kbsServiceTypemust beClusterIP,NodePort, orLoadBalancer. -
For
Restrictedprofile,httpsSpec.tlsSecretNameis required. - TLS secrets must contain valid certificate and key data in PEM format.
Default values
| Field | Default |
|---|---|
|
|
|
Additional resources
3.4. Customize TrusteeConfig autogenerated resources Copy linkLink copied to clipboard!
For production deployments with specific scaling, security policy, or disconnected environment requirements, you can customize the autogenerated TrusteeConfig resources.
Prerequisites
-
You have created a
TrusteeConfigcustom resource. - The Trustee operator has generated the required resources.
Procedure
To customize the autogenerated
KbsConfigresource, edit the resource directly by running the following command:$ oc edit kbsconfig <kbsconfig-name> -n trustee-operator-systemCommon KbsConfig customizations include:
Adjusting the number of Trustee replicas:
spec: KbsDeploymentSpec: replicas: 3Adding secret resources for workload access:
spec: kbsSecretResources: - kbsres1 - kbsres2Configuring disconnected environment support:
spec: kbsLocalCertCacheSpec: # Add disconnected environment configuration
To customize the resource access policy, edit the autogenerated config map by running the following command:
$ oc edit cm -n trustee-operator-system <trusteeconfig-name>-resource-policyTo customize the CPU attestation policy, edit the autogenerated config map by running the following command:
$ oc edit cm -n trustee-operator-system <trusteeconfig-name>-attestation-policy-cpuTo customize the GPU attestation policy, edit the autogenerated config map by running the following command:
$ oc edit cm -n trustee-operator-system <trusteeconfig-name>-attestation-policy-gpuTo customize Reference Value Provider Service (RVPS) reference values, edit the autogenerated config map by running the following command:
$ oc edit cm -n trustee-operator-system <trusteeconfig-name>-rvps-reference-valuesTo configure container image signature verification, edit the autogenerated
KbsConfigresource to add secret references by running the following command:$ oc edit kbsconfig <kbsconfig-name> -n trustee-operator-systemAdd the secret names to the
kbsSecretResourceslist:spec: kbsSecretResources: - img-sig - security-policywhere:
-
img-sigis the secret containing the public container image signing key -
security-policyis the secret containing the signature verification policy
For more information about creating these secrets, see Creating a secret for container image signature verification and Create the container image signature verification policy.
-
Verification
Verify that the Trustee pods restart after configuration changes by running the following command:
$ oc get pods -n trustee-operator-systemCheck the Trustee operator logs for any configuration errors by running the following command:
$ oc logs -n trustee-operator-system deployment/trustee-operator-controller-manager
3.5. Creating a secret for container image signature verification Copy linkLink copied to clipboard!
If you use container image signature verification, you must create a secret that contains the public container image signing key.
The Red Hat build of Trustee Operator uses the secret to verify the signature, ensuring that only trusted and authenticated container images are deployed in your environment.
You can use Red Hat Trusted Artifact Signer or other tools to sign container images.
Procedure
Create a secret for container image signature verification by running the following command:
$ oc create secret generic <type> \ --from-file=<tag>=./<public_key_file> \ -n trustee-operator-systemwhere:
<type>-
Specifies the secret type, for example,
img-sig. <public_key_file>-
Specifies the secret tag, for example,
pub-key, and the public container image signing key.
-
Record the
<type>value. You must add this value to thespec.kbsSecretResourceskey when you create theKbsConfigcustom resource.
3.6. Create the container image signature verification policy Copy linkLink copied to clipboard!
You configure the container image signature verification policy. Signature verification is disabled by default. To enable signature verification for your container images, follow the procedure. For more information, see containers-policy.json 5.
Both the signature keys and the corresponding policy must be added to Red Hat build of Trustee. The following procedure describes only how to add the policy itself.
Procedure
Create a
security-policy-config.jsonfile according to the following example:{ "default": [ { "type": "reject" } ], "transports": { "<transport>": { "<registry>/<image>": [ { "type": "<type>", "keyPath": "kbs:///default/<type>/<tag>" } ] } } }where:
transports- Specifies which images the policy explicitly approves and verifies through their signatures. By default, the policy rejects all images and all signatures. For more information, see containers-transports 5.
keyPath-
Specifies the type and tag of the container image signature verification secret that you created, for example,
img-sig/pub-key.
Create the security policy by running the following command:
$ oc create secret generic <security-policy-name> \ --from-file=<osc-key>=./<security-policy-config.json> \ -n trustee-operator-systemwhere:
<security-policy-name>-
Specifies the value of the
spec.kbsSecretResourceskey of theKbsConfigcustom resource.
3.7. Creating the authentication secret Copy linkLink copied to clipboard!
You create the authentication secret for Red Hat build of Trustee.
Procedure
Create a private key by running the following command:
$ openssl genpkey -algorithm ed25519 > privateKeyCreate a public key by running the following command:
$ openssl pkey -in privateKey -pubout -out publicKeyCreate a secret by running the following command:
$ oc create secret generic kbs-auth-public-key \ --from-file=publicKey -n trustee-operator-systemVerify the secret by running the following command:
$ oc get secret -n trustee-operator-system
3.8. Verifying the configuration Copy linkLink copied to clipboard!
You verify the Red Hat build of Trustee configuration by checking its pods and logs.
Procedure
Switch to the Red Hat build of Trustee Operator project by running the following command:
$ oc project trustee-operator-systemCheck the pods by running the following command:
$ oc get pods -n trustee-operator-systemExample output
NAME READY STATUS RESTARTS AGE trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59mSet the
POD_NAMEenvironmental variable by running the following command:$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)Check the pod logs by running the following command:
$ oc logs -n trustee-operator-system $POD_NAMEExample output
[2024-05-30T13:44:24Z INFO kbs] Using config file /etc/kbs-config/kbs-config.json [2024-05-30T13:44:24Z WARN attestation_service::rvps] No RVPS address provided and will launch a built-in rvps [2024-05-30T13:44:24Z INFO attestation_service::token::simple] No Token Signer key in config file, create an ephemeral key and without CA pubkey cert [2024-05-30T13:44:24Z INFO api_server] Starting HTTPS server at [0.0.0.0:8080] [2024-05-30T13:44:24Z INFO actix_server::builder] starting 12 workers [2024-05-30T13:44:24Z INFO actix_server::server] Tokio runtime found; starting in existing Tokio runtime