Administration Guide

Procedure

1. Open a terminal on your workstation, and log in to OpenShift:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

2. Create a new bucket:

   Syntax

   Copy to Clipboard Toggle word wrap

   export BUCKET=NEW_BUCKET_NAME
   export REGION=AWS_REGION_ID
   export USER=OADP_USER_NAME
   
   aws s3api create-bucket \
   --bucket $BUCKET \
   --region $REGION \
   --create-bucket-configuration LocationConstraint=$REGION

   Example

   Copy to Clipboard Toggle word wrap

   export BUCKET=example-bucket-name
   export REGION=us-east-1
   export USER=velero
   
   aws s3api create-bucket \
   --bucket $BUCKET \
   --region $REGION \
   --create-bucket-configuration LocationConstraint=$REGION

3. Create a new user:

   Example

   Copy to Clipboard Toggle word wrap

   aws iam create-user --user-name $USER

4. Create a new policy:

   Example

   Copy to Clipboard Toggle word wrap

   cat > velero-policy.json <<EOF
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "ec2:DescribeVolumes",
                   "ec2:DescribeSnapshots",
                   "ec2:CreateTags",
                   "ec2:CreateVolume",
                   "ec2:CreateSnapshot",
                   "ec2:DeleteSnapshot"
               ],
               "Resource": "*"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject",
                   "s3:DeleteObject",
                   "s3:PutObject",
                   "s3:AbortMultipartUpload",
                   "s3:ListMultipartUploadParts"
               ],
               "Resource": [
                   "arn:aws:s3:::${BUCKET}/*"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetBucketLocation",
                   "s3:ListBucketMultipartUploads"
               ],
               "Resource": [
                   "arn:aws:s3:::${BUCKET}"
               ]
           }
       ]
   }
   EOF

5. Associate this policy to the new user:

   Example

   Copy to Clipboard Toggle word wrap

   aws iam put-user-policy \
   --user-name $USER \
   --policy-name velero \
   --policy-document file://velero-policy.json

6. Create an access key:

   Example

   Copy to Clipboard Toggle word wrap

   aws iam create-access-key --user-name $USER --output=json | jq -r '.AccessKey | [ "export AWS_ACCESS_KEY_ID=" + .AccessKeyId, "export AWS_SECRET_ACCESS_KEY=" + .SecretAccessKey ] | join("\n")'

7. Create a credentials file with your AWS secret key information:

   Syntax

   Copy to Clipboard Toggle word wrap

   cat << EOF > ./credentials-velero
   [default]
   aws_access_key_id=$AWS_ACCESS_KEY_ID
   aws_secret_access_key=$AWS_SECRET_ACCESS_KEY
   EOF

8. Log in to the OpenShift web console with a user that has the cluster-admin role.
9. From the Administrator perspective, expand the Operators navigation menu, and click OperatorHub.
10. In the search field, type oadp, and click the OADP Operator tile provided by Red Hat.
11. Click the Install button to show the operator details.
12. Accept the default values, click Install on the Install Operator page, and wait for the installation to finish.

13. After the operator installation finishes, from your workstation terminal, create a secret resource for OpenShift with your AWS credentials:

    Example

    Copy to Clipboard Toggle word wrap

    oc create secret generic cloud-credentials -n openshift-adp --from-file cloud=credentials-velero

14. From the OpenShift web console, click the View Operator button.
15. Click Create instance on the DataProtectionApplication (DPA) tile.
16. On the Create DataProtectionApplication page, select YAML view.

17. Edit the following values in the resource file:

    1. Under the metadata section, replace velero-sample with velero.
    2. Under the spec.configuration.nodeAgent section, replace restic with kopia.
    3. Under the spec.configuration.velero section, add resourceTimeout: 10m.
    4. Under the spec.configuration.velero.defaultPlugins section, add - csi.
    5. Under the spec.snapshotLocations section, replace the us-west-2 value with your AWS regional value.
    6. Under the spec.backupLocations section, replace the us-east-1 value with your AWS regional value.
    7. Under the spec.backupLocations.objectStorage section, replace my-bucket-name with your bucket name. Replace velero with your bucket prefix name, if you use a different prefix.
18. Click the Create button.

Procedure

1. Open a terminal on your workstation, and log in to OpenShift:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

2. Find and edit the VolumeSnapshotClass resource:

   Example

   Copy to Clipboard Toggle word wrap

   oc get VolumeSnapshotClass -n openshift-adp
   oc edit VolumeSnapshotClass csi-aws-vsc -n openshift-adp

3. Update the following values in the resource file:

   1. Under the metadata.labels section, add the velero.io/csi-volumesnapshot-class: "true" label.
   2. Save your changes, and quit the editor.

4. Create a Backup resource:

   Example

   Copy to Clipboard Toggle word wrap

   cat <<EOF | oc apply -f -
   apiVersion: velero.io/v1
   kind: Backup
   metadata:
     name: rhtas-backup
     labels:
       velero.io/storage-location: velero-1
     namespace: openshift-adp
   spec:
     schedule: 0 7 * * *
     hooks: {}
     includedNamespaces:
     - trusted-artifact-signer
     includedResources: []
     excludedResources: []
     snapshotMoveData: true
     storageLocation: velero-1
     ttl: 720h0m0s
   EOF

   Add the schedule property to enable Cron scheduling for running this backup. In the example, this backup resource runs everyday at 7:00 a.m.

   By default, all resources are backed up within the trusted-artifact-signer namespace. You can specify what resources you want to include or exclude by using the includeResources or excludedResources properties respectively.

   Important

   Depending on the storage class of the backup target, persistent volumes cannot be actively in-use for the backup to be successful.

Procedure

1. Disable the RHTAS operator:

   Example

   Copy to Clipboard Toggle word wrap

   oc scale deploy rhtas-operator-controller-manager --replicas=0 -n openshift-operators

2. Create the Restore resource:

   Example

   Copy to Clipboard Toggle word wrap

   cat <<EOF | oc apply -f -
   apiVersion: velero.io/v1
   kind: Restore
   metadata:
     name: rhtas-restore
     namespace: openshift-adp
   spec:
     backupName: rhtas-backup
     includedResources: []
     restoreStatus:
       includedResources:
         - securesign.rhtas.redhat.com
         - trillian.rhtas.redhat.com
         - ctlog.rhtas.redhat.com
         - fulcio.rhtas.redhat.com
         - rekor.rhtas.redhat.com
         - tuf.rhtas.redhat.com
         - timestampauthority.rhtas.redhat.com
     excludedResources:
     - pod
     - deployment
     - nodes
     - route
     - service
     - replicaset
     - events
     - cronjob
     - events.events.k8s.io
     - backups.velero.io
     - restores.velero.io
     - resticrepositories.velero.io
     - pods
     - deployments
     restorePVs: true
     existingResourcePolicy: update
   EOF

3. If restoring your RHTAS data to a different OpenShift cluster, do the following steps.

   1. Delete the secret for the Trillian database:

      Example

      Copy to Clipboard Toggle word wrap

      oc delete secret securesign-sample-trillian-db-tls
      oc delete pod trillian-db-xxx

      Note

      The RHTAS operator recreates the secret and restarts the pod.

   2. Run the restoreOwnerReferences.sh script.

4. Enable the RHTAS operator:

   Example

   Copy to Clipboard Toggle word wrap

   oc scale deploy rhtas-operator-controller-manager --replicas=1 -n openshift-operators

   Important

   Immediately starting the RHTAS operator after starting the restore ensures the claim of the persistent volume.

Procedure

1. Download the tuftool binary from the OpenShift cluster to your workstation.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the home page, click the ? icon, click Command line tools, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Log in to OpenShift from the command line:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

3. Switch to the RHTAS project:

   Example

   Copy to Clipboard Toggle word wrap

   oc project trusted-artifact-signer

4. Configure your shell environment:

   Example

   Copy to Clipboard Toggle word wrap

   export WORK="${HOME}/trustroot-example"
   export ROOT="${WORK}/root/root.json"
   export KEYDIR="${WORK}/keys"
   export INPUT="${WORK}/input"
   export TUF_REPO="${WORK}/tuf-repo"
   export TUF_SERVER_POD="$(oc get pod --selector=app.kubernetes.io/component=tuf --no-headers -o custom-columns=":metadata.name")"
   export TIMESTAMP_EXPIRATION="in 10 days"
   export SNAPSHOT_EXPIRATION="in 26 weeks"
   export TARGETS_EXPIRATION="in 26 weeks"
   export ROOT_EXPIRATION="in 26 weeks"

   Set the expiration durations according to your requirements.

5. Create a temporary TUF directory structure:

   Example

   Copy to Clipboard Toggle word wrap

   mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

6. Download the TUF contents to the temporary TUF directory structure:

   Example

   Copy to Clipboard Toggle word wrap

   oc extract --to "${KEYDIR}/" secret/tuf-root-keys
   oc cp "${TUF_SERVER_POD}:/var/www/html" "${TUF_REPO}"
   cp "${TUF_REPO}/root.json" "${ROOT}"

7. You can update the timestamp, snapshot, and targets metadata all in one command:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool update \
     --root "${ROOT}" \
     --key "${KEYDIR}/timestamp.pem" \
     --key "${KEYDIR}/snapshot.pem" \
     --key "${KEYDIR}/targets.pem" \
     --timestamp-expires "${TIMESTAMP_EXPIRATION}" \
     --snapshot-expires "${SNAPSHOT_EXPIRATION}" \
     --targets-expires "${TARGETS_EXPIRATION}" \
     --outdir "${TUF_REPO}" \
     --metadata-url "file://${TUF_REPO}"

   Note

   You can also run the TUF metadata update on a subset of TUF metadata files. For example, the timestamp.json metadata file expires more often than the other metadata files. Therefore, you can just update the timestamp metadata file by running the following command:

   Copy to Clipboard Toggle word wrap

   tuftool update \
     --root "${ROOT}" \
     --key "${KEYDIR}/timestamp.pem" \
     --timestamp-expires "${TIMESTAMP_EXPIRATION}" \
     --outdir "${TUF_REPO}" \
     --metadata-url "file://${TUF_REPO}"

8. Only update the root expiration date if it is about to expire:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool root expire "${ROOT}" "${ROOT_EXPIRATION}"

   Note

   You can skip this step if the root file is not close to expiring.

9. Update the root version:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool root bump-version "${ROOT}"

10. Sign the root metadata file again:

    Example

    Copy to Clipboard Toggle word wrap

    tuftool root sign "${ROOT}" -k "${KEYDIR}/root.pem"

11. Set the new root version, and copy the root metadata file in place:

    Example

    Copy to Clipboard Toggle word wrap

    export NEW_ROOT_VERSION=$(cat "${ROOT}" | jq -r ".signed.version")
    cp "${ROOT}" "${TUF_REPO}/root.json"
    cp "${ROOT}" "${TUF_REPO}/${NEW_ROOT_VERSION}.root.json"

12. Upload these changes to the TUF server:

    Example

    Copy to Clipboard Toggle word wrap

    oc rsync "${TUF_REPO}/" "${TUF_SERVER_POD}:/var/www/html"

Procedure

1. Download the rekor-cli binary from the OpenShift cluster to your workstation.

   1. Login to the OpenShift web console. From the home page, click the ? icon, click Command line tools, go to the rekor-cli download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip rekor-cli-amd64.gz
      chmod +x rekor-cli-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv rekor-cli-amd64 /usr/local/bin/rekor-cli

2. Download the tuftool binary from the OpenShift cluster to your workstation.

   Important

   The tuftool binary is only available for Linux operating systems.

   1. From the home page, click the ? icon, click Command line tools, go to the tuftool download section, and click the link for your platform.

   2. From a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

3. Log in to OpenShift from the command line:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

4. Switch to the RHTAS project:

   Example

   Copy to Clipboard Toggle word wrap

   oc project trusted-artifact-signer

5. Get the Rekor URL:

   Example

   Copy to Clipboard Toggle word wrap

   export REKOR_URL=$(oc get rekor -o jsonpath='{.items[0].status.url}')

6. Get the log tree identifier for the active shard:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_TREE_ID=$(rekor-cli loginfo --rekor_server $REKOR_URL --format json | jq -r .TreeID)

7. Set the log tree to the DRAINING state:

   Example

   Copy to Clipboard Toggle word wrap

   oc run --image registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- updatetree --admin_server=trillian-logserver:8091 --tree_id=${OLD_TREE_ID} --tree_state=DRAINING

   While draining, the tree log will not accept any new entries. Watch and wait for the queue to empty.

   Important

   You must wait for the queues to be empty before proceeding to the next step. If leaves are still integrating while draining, then freezing the log tree during this process can cause the log path to exceed the maximum merge delay (MMD) threshold.

8. Freeze the log tree:

   Example

   Copy to Clipboard Toggle word wrap

   oc run --image registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- updatetree --admin_server=trillian-logserver:8091 --tree_id=${OLD_TREE_ID} --tree_state=FROZEN

9. Get the length of the frozen log tree:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_SHARD_LENGTH=$(rekor-cli loginfo --rekor_server $REKOR_URL --format json | jq -r .ActiveTreeSize)

10. Get Rekor’s public key for the old shard:

    Example

    Copy to Clipboard Toggle word wrap

    export OLD_PUBLIC_KEY=$(curl -s $REKOR_URL/api/v1/log/publicKey | base64 | tr -d '\n')

11. Create a new log tree:

    Example

    Copy to Clipboard Toggle word wrap

    export NEW_TREE_ID=$(oc run createtree --image registry.redhat.io/rhtas/createtree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- -logtostderr=false --admin_server=trillian-logserver:8091 --display_name=rekor-tree)

    Now you have two log trees, one frozen tree, and a new tree that will become the active shard.

12. Create a new private key:

    Example

    Copy to Clipboard Toggle word wrap

    openssl ecparam -genkey -name secp384r1 -noout -out new-rekor.pem

    Important

    The new key must have a unique file name.

13. Create a new secret resource with the new signer key:

    Example

    Copy to Clipboard Toggle word wrap

    oc create secret generic rekor-signer-key --from-file=private=new-rekor.pem

14. Update the Securesign Rekor configuration with the new tree identifier and the old sharding information:

    Example

    Copy to Clipboard Toggle word wrap

    read -r -d '' SECURESIGN_PATCH_1 <<EOF
    [
        {
            "op": "replace",
            "path": "/spec/rekor/treeID",
            "value": $NEW_TREE_ID
        },
        {
            "op": "add",
            "path": "/spec/rekor/sharding/-",
            "value": {
                "treeID": $OLD_TREE_ID,
                "treeLength": $OLD_SHARD_LENGTH,
                "encodedPublicKey": "$OLD_PUBLIC_KEY"
            }
        },
        {
            "op": "replace",
            "path": "/spec/rekor/signer/keyRef",
            "value": {"name": "rekor-signer-key", "key": "private"}
        }
    ]
    EOF

    Note

    If you have /spec/rekor/signer/keyPasswordRef set with a value, then create a new separate update to remove it:

    Example

    Copy to Clipboard Toggle word wrap

    read -r -d '' SECURESIGN_PATCH_2 <<EOF
    [
        {
            "op": "remove",
            "path": "/spec/rekor/signer/keyPasswordRef"
        }
    ]
    EOF

    Apply this update after applying the first update.

15. Update the Securesign instance:

    Example

    Copy to Clipboard Toggle word wrap

    oc patch Securesign securesign-sample --type='json' -p="$SECURESIGN_PATCH_1"

16. Wait for the Rekor server to redeploy with the new signer key:

    Example

    Copy to Clipboard Toggle word wrap

    oc wait pod -l app.kubernetes.io/name=rekor-server --for=condition=Ready

17. Get the new public key:

    Example

    Copy to Clipboard Toggle word wrap

    export NEW_KEY_NAME=new-rekor.pub
    curl $(oc get rekor -o jsonpath='{.items[0].status.url}')/api/v1/log/publicKey -o $NEW_KEY_NAME

18. Configure The Update Framework (TUF) service to use the new Rekor public key.

    1. Configure your shell environment:

       Example

       Copy to Clipboard Toggle word wrap

       export WORK="${HOME}/trustroot-example"
       export ROOT="${WORK}/root/root.json"
       export KEYDIR="${WORK}/keys"
       export INPUT="${WORK}/input"
       export TUF_REPO="${WORK}/tuf-repo"
       export TUF_SERVER_POD="$(oc get pod --selector=app.kubernetes.io/component=tuf --no-headers -o custom-columns=":metadata.name")"

    2. Create a temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

    3. Download the TUF contents to the temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       oc extract --to "${KEYDIR}/" secret/tuf-root-keys
       oc cp "${TUF_SERVER_POD}:/var/www/html" "${TUF_REPO}"
       cp "${TUF_REPO}/root.json" "${ROOT}"

    4. Find the active Rekor signer key file name. Open the latest target file, for example, 1.target.json, within the local TUF repository. In this file you will find the active Rekor signer key file name, for example, rekor.pub. Set an environment variable with this active Rekor signer key file name:

       Example

       Copy to Clipboard Toggle word wrap

       export ACTIVE_KEY_NAME=rekor.pub

    5. Update the Rekor signer key with the old public key:

       Example

       Copy to Clipboard Toggle word wrap

       echo $OLD_PUBLIC_KEY | base64 -d > $ACTIVE_KEY_NAME

    6. Expire the old Rekor signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-rekor-target "${ACTIVE_KEY_NAME}" \
         --rekor-uri "${REKOR_URL}" \
         --rekor-status "Expired" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    7. Add the new Rekor signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-rekor-target "${NEW_KEY_NAME}" \
         --rekor-uri "${REKOR_URL}" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    8. Upload these changes to the TUF server:

       Example

       Copy to Clipboard Toggle word wrap

       oc rsync "${TUF_REPO}/" "${TUF_SERVER_POD}:/var/www/html"

    9. Delete the working directory:

       Example

       Copy to Clipboard Toggle word wrap

       rm -r $WORK

19. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts with the new Rekor signer key.

Procedure

1. Download the tuftool binary from the OpenShift cluster to your workstation.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the home page, click the ? icon, click Command line tools, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Log in to OpenShift from the command line:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

3. Switch to the RHTAS project:

   Example

   Copy to Clipboard Toggle word wrap

   oc project trusted-artifact-signer

4. Make a backup of the current CT log configuration, and keys:

   Example

   Copy to Clipboard Toggle word wrap

   export SERVER_CONFIG_NAME=$(oc get ctlog -o jsonpath='{.items[0].status.serverConfigRef.name}')
   oc get secret $SERVER_CONFIG_NAME -o jsonpath="{.data.config}" | base64 --decode > config.txtpb
   oc get secret $SERVER_CONFIG_NAME -o jsonpath="{.data.fulcio-0}" | base64 --decode > fulcio-0.pem
   oc get secret $SERVER_CONFIG_NAME -o jsonpath="{.data.private}" | base64 --decode > private.pem
   oc get secret $SERVER_CONFIG_NAME -o jsonpath="{.data.public}" | base64 --decode > public.pem

5. Capture the current tree identifier:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_TREE_ID=$(oc get ctlog -o jsonpath='{.items[0].status.treeID}')

6. Set the log tree to the DRAINING state:

   Example

   Copy to Clipboard Toggle word wrap

   oc run --image registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- updatetree --admin_server=trillian-logserver:8091 --tree_id=${OLD_TREE_ID} --tree_state=DRAINING

   While draining, the tree log will not accept any new entries. Watch and wait for the queue to empty.

   Important

   You must wait for the queues to be empty before proceeding to the next step. If leaves are still integrating while draining, then freezing the log tree during this process can cause the log path to exceed the maximum merge delay (MMD) threshold.

7. Once the queue has been fully drained, freeze the log:

   Example

   Copy to Clipboard Toggle word wrap

   oc run --image registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- updatetree --admin_server=trillian-logserver:8091 --tree_id=${OLD_TREE_ID} --tree_state=FROZEN

8. Create a new Merkle tree, and capture the new tree identifier:

   Example

   Copy to Clipboard Toggle word wrap

   export NEW_TREE_ID=$(kubectl run createtree --image registry.redhat.io/rhtas/createtree-rhel9:1.1.0 --restart=Never --attach=true --rm=true -q -- -logtostderr=false --admin_server=trillian-logserver:8091 --display_name=ctlog-tree)

9. Generate a new certificate, along with new public and private keys:

   Example

   Copy to Clipboard Toggle word wrap

   openssl ecparam -genkey -name prime256v1 -noout -out new-ctlog.pem
   openssl ec -in new-ctlog.pem -pubout -out new-ctlog-public.pem
   openssl ec -in new-ctlog.pem -out new-ctlog.pass.pem -des3 -passout pass:"CHANGE_ME"

   Replace CHANGE_ME with a new password.

   Important

   The certificate and new keys must have unique file names.

10. Update the CT log configuration.

    1. Open the config.txtpb file for editing.

    2. For the frozen log, add the not_after_limit field to the frozen log entry, rename the prefix value to a unique name, and replace the old path to the private key with ctfe-keys/private-0:

       Example

       Copy to Clipboard Toggle word wrap

       ...
       log_configs:{
         # frozen log
         config:{
           log_id:2066075212146181968
           prefix:"trusted-artifact-signer-0"
           roots_pem_file:"/ctfe-keys/fulcio-0"
           private_key:{[type.googleapis.com/keyspb.PEMKeyFile]:{path:"/ctfe-keys/private-0" password:"Example123"}}
           public_key:{der:"0Y0\x13\x06\x07*\x86H\xce=\x02\x01\x06\x08*\x86H\xce=\x03\x01\x07\x03B\x00\x04)'.\xffUJ\xe2s)\xefR\x8a\xfcO\xdcewȶy\xa7\x9d<\x13\xb0\x1c\x99\x96\xe4'\xe3v\x07:\xc8I+\x08J\x9d\x8a\xed\x06\xe4\xaeI:q\x98\xf4\xbc<o4VD\x0cr\xf9\x9c\xecxT\x84"}
           not_after_limit:{seconds:1728056285 nanos:012111000}
           ext_key_usages:"CodeSigning"
           log_backend_name:"trillian"
         }

       Note

       You can get the current time value for seconds and nanoseconds, by running the following commands: date +%s, and date +%N.

       Important

       The not_after_limit field defines the end of the timestamp range for the frozen log only. Certificates beyond this point in time are no longer accepted for inclusion in this log.

    3. Copy and paste the frozen log config block, appending it to the configuration file to create a new entry.

    4. Change the following lines in the new config block. Set the log_id to the new tree identifier, change the prefix to trusted-artifact-signer, change the private_key path to ctfe-keys/private, remove the public_key line, and change not_after_limit to not_after_start and set the timestamp range:

       Example

       Copy to Clipboard Toggle word wrap

       ...
       log_configs:{
         # frozen log
         ...
         # new active log
         config:{
       	  log_id: NEW_TREE_ID
       	  prefix:"trusted-artifact-signer"
       	  roots_pem_file:"/ctfe-keys/fulcio-0"
       	  private_key:{[type.googleapis.com/keyspb.PEMKeyFile]:{path:"ctfe-keys/private" password:"CHANGE_ME"}}
       	  ext_key_usages:"CodeSigning"
       	  not_after_start:{seconds:1713201754 nanos:155663000}
       	  log_backend_name:"trillian"
         }

       Add the NEW_TREE_ID, and replace CHANGE_ME with the new private key password. The password here must match the password used for generating the new private and public keys.

       Important

       The not_after_start field defines the beginning of the timestamp range inclusively. This means the log will start accepting certificates at this point in time.

11. Create a new secret resource:

    Example

    Copy to Clipboard Toggle word wrap

    oc create secret generic ctlog-config \
    --from-file=config=config.txtpb \
    --from-file=private=new-ctlog.pass.pem \
    --from-file=public=new-ctlog-public.pem \
    --from-file=fulcio-0=fulcio-0.pem \
    --from-file=private-0=private.pem \
    --from-file=public-0=public.pem \
    --from-literal=password=CHANGE_ME

    Replace CHANGE_ME with the new private key password.

12. Configure The Update Framework (TUF) service to use the new CT log public key.

    1. Configure your shell environment:

       Example

       Copy to Clipboard Toggle word wrap

       export WORK="${HOME}/trustroot-example"
       export ROOT="${WORK}/root/root.json"
       export KEYDIR="${WORK}/keys"
       export INPUT="${WORK}/input"
       export TUF_REPO="${WORK}/tuf-repo"
       export TUF_SERVER_POD="$(oc get pod --selector=app.kubernetes.io/component=tuf --no-headers -o custom-columns=":metadata.name")"

    2. Create a temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

    3. Download the TUF contents to the temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       oc extract --to "${KEYDIR}/" secret/tuf-root-keys
       oc cp "${TUF_SERVER_POD}:/var/www/html" "${TUF_REPO}"
       cp "${TUF_REPO}/root.json" "${ROOT}"

    4. Find the active CT log public key file name. Open the latest target file, for example, 1.targets.json, within the local TUF repository. In this target file you will find the active CT log public key file name, for example, ctfe.pub. Set an environment variable with this active CT log public key file name:

       Example

       Copy to Clipboard Toggle word wrap

       export ACTIVE_CTFE_NAME=ctfe.pub

    5. Extract the active CT log public key from OpenShift:

       Example

       Copy to Clipboard Toggle word wrap

       oc get secret $(oc get ctlog securesign-sample -o jsonpath='{.status.publicKeyRef.name}') -o jsonpath='{.data.public}' | base64 -d > $ACTIVE_CTFE_NAME

    6. Expire the old CT log signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-ctlog-target "$ACTIVE_CTFE_NAME" \
         --ctlog-uri "https://ctlog.rhtas" \
         --ctlog-status "Expired" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    7. Add the new CT log signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-ctlog-target "new-ctlog-public.pem" \
         --ctlog-uri "https://ctlog.rhtas" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    8. Upload these changes to the TUF server:

       Example

       Copy to Clipboard Toggle word wrap

       oc rsync "${TUF_REPO}/" "${TUF_SERVER_POD}:/var/www/html"

13. Update the Securesign CT log configuration with the new tree identifier:

    Example

    Copy to Clipboard Toggle word wrap

    read -r -d '' SECURESIGN_PATCH <<EOF
    [
    	{
        	"op": "replace",
        	"path": "/spec/ctlog/serverConfigRef",
        	"value": {"name": "ctlog-config"}
    	},
        {
            "op": "replace",
            "path": "/spec/ctlog/treeID",
            "value": $NEW_TREE_ID
        },
    	{
        	"op": "replace",
        	"path": "/spec/ctlog/privateKeyRef",
        	"value": {"name": "ctlog-config", "key": "private"}
    	},
        {
            "op": "replace",
            "path": "/spec/ctlog/privateKeyPasswordRef",
            "value": {"name": "ctlog-config", "key": "password"}
        },
    	{
        	"op": "replace",
        	"path": "/spec/ctlog/publicKeyRef",
        	"value": {"name": "ctlog-config", "key": "public"}
    	}
    ]
    EOF

14. Patch the Securesign instance:

    Example

    Copy to Clipboard Toggle word wrap

    oc patch Securesign securesign-sample --type='json' -p="$SECURESIGN_PATCH"

15. Wait for the CT log server to redeploy:

    Example

    Copy to Clipboard Toggle word wrap

    oc wait pod -l app.kubernetes.io/name=ctlog --for=condition=Ready

16. Delete the working directory:

    Example

    Copy to Clipboard Toggle word wrap

    rm -r $WORK

17. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts with the new CT log signer key.

Procedure

1. Download the tuftool binary from the OpenShift cluster to your workstation.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the home page, click the ? icon, click Command line tools, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Log in to OpenShift from the command line:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

3. Switch to the RHTAS project:

   Example

   Copy to Clipboard Toggle word wrap

   oc project trusted-artifact-signer

4. Generate a new certificate, along with new public and private keys:

   Example

   Copy to Clipboard Toggle word wrap

   openssl ecparam -genkey -name prime256v1 -noout -out new-fulcio.pem
   openssl ec -in new-fulcio.pem -pubout -out new-fulcio-public.pem
   openssl ec -in new-fulcio.pem -out new-fulcio.pass.pem -des3 -passout pass:"CHANGE_ME"
   openssl req -new -x509 -key new-fulcio.pass.pem -out new-fulcio.cert.pem

   Replace CHANGE_ME with a new password.

   Important

   The certificate and new keys must have unique file names.

5. Create a new secret:

   Example

   Copy to Clipboard Toggle word wrap

   oc create secret generic fulcio-config \
   --from-file=private=new-fulcio.pass.pem \
   --from-file=cert=new-fulcio.cert.pem \
   --from-literal=password=CHANGE_ME

   Replace CHANGE_ME with a new password.

   Note

   The password here must match the password used for generating the new private and public keys.

6. Configure The Update Framework (TUF) service to use the new Fulcio certificate.

   1. Set up your shell environment:

      Example

      Copy to Clipboard Toggle word wrap

      export WORK="${HOME}/trustroot-example"
      export ROOT="${WORK}/root/root.json"
      export KEYDIR="${WORK}/keys"
      export INPUT="${WORK}/input"
      export TUF_REPO="${WORK}/tuf-repo"
      export TUF_SERVER_POD="$(oc get pod --selector=app.kubernetes.io/component=tuf --no-headers -o custom-columns=":metadata.name")"

   2. Create a temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

   3. Download the TUF contents to the temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      oc extract --to "${KEYDIR}/" secret/tuf-root-keys
      oc cp "${TUF_SERVER_POD}:/var/www/html" "${TUF_REPO}"
      cp "${TUF_REPO}/root.json" "${ROOT}"

   4. Find the active Fulcio certificate file name. Open the latest target file, for example, 1.targets.json, within the local TUF repository. In this file you will find the active Fulcio certificate file name, for example, fulcio_v1.crt.pem. Set an environment variable with this active Fulcio certificate file name:

      Example

      Copy to Clipboard Toggle word wrap

      export ACTIVE_CERT_NAME=fulcio_v1.crt.pem

   5. Extract the active Fulcio certificate from OpenShift:

      Example

      Copy to Clipboard Toggle word wrap

      oc get secret $(oc get fulcio securesign-sample -o jsonpath='{.status.certificate.caRef.name}') -o jsonpath='{.data.cert}' | base64 -d > $ACTIVE_CERT_NAME

   6. Expire the old certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-fulcio-target "$ACTIVE_CERT_NAME" \
        --fulcio-uri "https://fulcio.rhtas" \
        --fulcio-status "Expired" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   7. Add the new Fulcio certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-fulcio-target "new-fulcio.cert.pem" \
        --fulcio-uri "https://fulcio.rhtas" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   8. Upload these changes to the TUF server:

      Example

      Copy to Clipboard Toggle word wrap

      oc rsync "${TUF_REPO}/" "${TUF_SERVER_POD}:/var/www/html"

   9. Delete the working directory:

      Example

      Copy to Clipboard Toggle word wrap

      rm -r $WORK

7. Update the Securesign Fulcio configuration:

   Example

   Copy to Clipboard Toggle word wrap

   read -r -d '' SECURESIGN_PATCH <<EOF
   [
   {
       "op": "replace",
       "path": "/spec/fulcio/certificate/privateKeyRef",
       "value": {"name": "fulcio-config", "key": "private"}
   },
   {
       "op": "replace",
       "path": "/spec/fulcio/certificate/privateKeyPasswordRef",
       "value": {"name": "fulcio-config", "key": "password"}
   },
   {
       "op": "replace",
       "path": "/spec/fulcio/certificate/caRef",
       "value": {"name": "fulcio-config", "key": "cert"}
   },
   {
       "op": "replace",
       "path": "/spec/ctlog/rootCertificates",
       "value": [{"name": "fulcio-config", "key": "cert"}]
   }
   ]
   EOF

8. Patch the Securesign instance:

   Example

   Copy to Clipboard Toggle word wrap

   oc patch Securesign securesign-sample --type='json' -p="$SECURESIGN_PATCH"

9. Wait for the Fulcio server to redeploy:

   Example

   Copy to Clipboard Toggle word wrap

   oc wait pod -l app.kubernetes.io/name=fulcio-server --for=condition=Ready
   oc wait pod -l app.kubernetes.io/name=ctlog --for=condition=Ready

10. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts with the new Fulcio certificate.

Procedure

1. Download the tuftool binary from the OpenShift cluster to your workstation.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the home page, click the ? icon, click Command line tools, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Log in to OpenShift from the command line:

   Syntax

   Copy to Clipboard Toggle word wrap

   oc login --token=TOKEN --server=SERVER_URL_AND_PORT

   Example

   Copy to Clipboard Toggle word wrap

   oc login --token=sha256~ZvFDBvoIYAbVECixS4-WmkN4RfnNd8Neh3y1WuiFPXC --server=https://example.com:6443

   Note

   You can find your login token and URL for use on the command line from the OpenShift web console. Log in to the OpenShift web console. Click your user name, and click Copy login command. Offer your user name and password again, if asked, and click Display Token to view the command.

3. Switch to the RHTAS project:

   Example

   Copy to Clipboard Toggle word wrap

   oc project trusted-artifact-signer

4. Generate a new certificate chain, and a new signer key.

   Important

   The new certificate and keys must have unique file names.

   1. Create a temporary working directory:

      Example

      Copy to Clipboard Toggle word wrap

      mkdir certs && cd certs

   2. Create the root certificate authority (CA) private key, and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -x509 -newkey rsa:2048 -days 365 -sha256 -nodes \
      -keyout rootCA.key.pem -out rootCA.crt.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=RootCA/CN=RootCA" \
      -addext "basicConstraints=CA:true" -addext "keyUsage=cRLSign, keyCertSign"

      Replace CHANGE_ME with a new password.

   3. Create the intermediate CA private key and certificate signing request (CSR), and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -newkey rsa:2048 -sha256 \
      -keyout intermediateCA.key.pem -out intermediateCA.csr.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=IntermediateCA/CN=IntermediateCA"

      Replace CHANGE_ME with a new password.

   4. Sign the intermediate CA certificate with the root CA:

      Example

      Copy to Clipboard Toggle word wrap

      openssl x509 -req -in intermediateCA.csr.pem -CA rootCA.crt.pem -CAkey rootCA.key.pem \
      -CAcreateserial -out intermediateCA.crt.pem -days 365 -sha256 \
      -extfile <(echo -e "basicConstraints=CA:true\nkeyUsage=cRLSign, keyCertSign\nextendedKeyUsage=critical,timeStamping") \
      -passin pass:"CHANGE_ME"

      Replace CHANGE_ME with the root CA private key password to sign the intermediate CA certificate.

   5. Create the leaf CA private key and CSR, and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -newkey rsa:2048 -sha256 \
      -keyout leafCA.key.pem -out leafCA.csr.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=LeafCA/CN=LeafCA"

   6. Sign the leaf CA certificate with the intermediate CA:

      Example

      Copy to Clipboard Toggle word wrap

      openssl x509 -req -in leafCA.csr.pem -CA intermediateCA.crt.pem -CAkey intermediateCA.key.pem \
        -CAcreateserial -out leafCA.crt.pem -days 365 -sha256 \
        -extfile <(echo -e "basicConstraints=CA:false\nkeyUsage=cRLSign, keyCertSign\nextendedKeyUsage=critical,timeStamping") \
        -passin pass:"CHANGE_ME"

      Replace CHANGE_ME with the intermediate CA private key password to sign the leaf CA certificate.

   7. Create the certificate chain by combining the newly created certificates together:

      Example

      Copy to Clipboard Toggle word wrap

      cat leafCA.crt.pem intermediateCA.crt.pem rootCA.crt.pem > new-tsa.certchain.pem

5. Create a new secret resource with the signer key:

   Example

   Copy to Clipboard Toggle word wrap

   oc create secret generic rotated-signer-key --from-file=rotated-signer-key=certs/leafCA.key.pem

6. Create a new secret resource with the new certificate chain:

   Example

   Copy to Clipboard Toggle word wrap

   oc create secret generic rotated-cert-chain --from-file=rotated-cert-chain=certs/new-tsa.certchain.pem

7. Create a new secret resource with for the password:

   Example

   Copy to Clipboard Toggle word wrap

   oc create secret generic rotated-password --from-literal=rotated-password=CHANGE_ME

   Replace CHANGE_ME with the intermediate CA private key password.

8. Find your active TSA certificate file name, the TSA URL string, and configure your shell environment with these values:

   Example

   Copy to Clipboard Toggle word wrap

   export ACTIVE_CERT_CHAIN_NAME=tsa.certchain.pem
   export TSA_URL=$(oc get timestampauthority securesign-sample -o jsonpath='{.status.url}')/api/v1/timestamp
   curl $TSA_URL/certchain -o $ACTIVE_CERT_CHAIN_NAME

9. Update the Securesign TSA configuration:

   Example

   Copy to Clipboard Toggle word wrap

   read -r -d '' SECURESIGN_PATCH <<EOF
   [
       {
           "op": "replace",
           "path": "/spec/tsa/signer/certificateChain",
           "value": {
               "certificateChainRef" : {"name": "rotated-cert-chain", "key": "rotated-cert-chain"}
           }
       },
       {
           "op": "replace",
           "path": "/spec/tsa/signer/file",
           "value": {
                   "privateKeyRef": {"name": "rotated-signer-key", "key": "rotated-signer-key"},
                   "passwordRef": {"name": "rotated-password", "key": "rotated-password"}
               }
       }
   ]
   EOF

10. Patch the Securesign instance:

    Example

    Copy to Clipboard Toggle word wrap

    oc patch Securesign securesign-sample --type='json' -p="$SECURESIGN_PATCH"

11. Wait for the TSA server to redeploy with the new signer key and certificate chain:

    Example

    Copy to Clipboard Toggle word wrap

    oc get pods -w -l app.kubernetes.io/name=tsa-server

12. Get the new certificate chain:

    Example

    Copy to Clipboard Toggle word wrap

    export NEW_CERT_CHAIN_NAME=new-tsa.certchain.pem
    curl $TSA_URL/certchain -o $NEW_CERT_CHAIN_NAME

13. Configure The Update Framework (TUF) service to use the new TSA certificate chain.

    1. Set up your shell environment:

       Example

       Copy to Clipboard Toggle word wrap

       export WORK="${HOME}/trustroot-example"
       export ROOT="${WORK}/root/root.json"
       export KEYDIR="${WORK}/keys"
       export INPUT="${WORK}/input"
       export TUF_REPO="${WORK}/tuf-repo"
       export TUF_SERVER_POD="$(oc get pod --selector=app.kubernetes.io/component=tuf --no-headers -o custom-columns=":metadata.name")"

    2. Create a temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

    3. Download the TUF contents to the temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       oc extract --to "${KEYDIR}/" secret/tuf-root-keys
       oc cp "${TUF_SERVER_POD}:/var/www/html" "${TUF_REPO}"
       cp "${TUF_REPO}/root.json" "${ROOT}"

    4. Expire the old TSA certificate:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-tsa-target "$ACTIVE_CERT_CHAIN_NAME" \
         --tsa-uri "$TSA_URL" \
         --tsa-status "Expired" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    5. Add the new TSA certificate:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-tsa-target "$NEW_CERT_CHAIN_NAME" \
         --tsa-uri "$TSA_URL" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    6. Upload these changes to the TUF server:

       Example

       Copy to Clipboard Toggle word wrap

       oc rsync "${TUF_REPO}/" "${TUF_SERVER_POD}:/var/www/html"

    7. Delete the working directory:

       Example

       Copy to Clipboard Toggle word wrap

       rm -r $WORK

14. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts that uses the new TSA signer key, and certificate.

Procedure

1. Open for editing the RHTAS Ansible Playbook.

2. Under the tas_single_node_backup_restore.backup section, set the enabled variable to true:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_backup_restore:
     backup:
       enabled: true

3. By default, a daily backup job runs at midnight every day. You can change this to better fit your schedule.

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_backup_restore:
     backup:
       enabled: true
       schedule: "*-*-* 00:00:00"

4. Set a passphrase, and specify the local backup directory:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_backup_restore:
     backup:
       enabled: true
       schedule: "*-*-* 00:00:00"
       force_run: false
       passphrase: "example123"
       directory: /root/tas_backups

5. Optional. To start an immediate backup job, set the force_run variable to true.
6. Save the changes, and quit the editor.

7. Run the RHTAS Ansible Playbook to apply the changes:

   Example

   Copy to Clipboard Toggle word wrap

   ansible-playbook -i inventory play.yml

   After the backup finishes, the resulting encrypted, and compressed file name format is, BACKUP-<date-and-time>-UTC.tar.gz.enc.

Procedure

1. Copy the backup data file to a directory on the Ansible control node.
2. Open for editing the RHTAS Ansible Playbook.

3. Under the tas_single_node_backup_restore.restore section, set the enabled variable to true:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_backup_restore:
     ...
     restore:
       enabled: true

4. Specify the source location of the backup file, and give the correct passphrase:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_backup_restore:
     ...
     restore:
       enabled: true
       source: "PATH_TO_BACKUP_FILE"
       passphrase: "example123"

5. Under the tas_single_node_backup_restore.backup section, verify that the force_run variable to false. If the force_run variable to true, then set it to false.

6. . Run the RHTAS Ansible Playbook to apply the changes:

   Example

   Copy to Clipboard Toggle word wrap

   ansible-playbook -i inventory play.yml

   The restoration process starts, and does a re-execution of all tasks to validate the integrity of the RHTAS service.

Procedure

1. Download the tuftool binary from the local command-line interface (CLI) tool download page to your workstation.

   Note

   The URL address is the configured node as defined by the tas_single_node_base_hostname variable. An example URL address would be, https://cli-server.example.com, given the tas_single_node_base_hostname value as example.com.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the download page, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Configure your shell environment:

   Example

   Copy to Clipboard Toggle word wrap

   export WORK="${HOME}/trustroot-example"
   export ROOT="${WORK}/root/root.json"
   export KEYDIR="${WORK}/keys"
   export INPUT="${WORK}/input"
   export TUF_REPO="${WORK}/tuf-repo"
   export MANAGED_NODE_IP=IP_OF_ANSIBLE_MANAGED_NODE
   export MANAGED_NODE_SSH_USER=USER_TO_CONNECT_TO_MANAGED_NODE
   export REMOTE_KEYS_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-signing-keys" | tr -d '[:space:]')
   export REMOTE_TUF_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-repository" | tr -d '[:space:]')
   export TIMESTAMP_EXPIRATION="in 10 days"
   export SNAPSHOT_EXPIRATION="in 26 weeks"
   export TARGETS_EXPIRATION="in 26 weeks"
   export ROOT_EXPIRATION="in 26 weeks"

   Replace IP_OF_ANSIBLE_MANAGED_NODE and USER_TO_CONNECT_TO_MANAGED_NODE with your relevant values.

   Set the expiration durations according to your requirements.

3. Create a temporary TUF directory structure:

   Example

   Copy to Clipboard Toggle word wrap

   mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

4. Download the TUF contents to the temporary TUF directory structure:

   Example

   Copy to Clipboard Toggle word wrap

   rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_KEYS_VOLUME}/" "${KEYDIR}"
   rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_TUF_VOLUME}/" "${TUF_REPO}"
   cp "${TUF_REPO}/root.json" "${ROOT}"

5. You can update the timestamp, snapshot, and targets metadata all in one command:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool update \
     --root "${ROOT}" \
     --key "${KEYDIR}/timestamp.pem" \
     --key "${KEYDIR}/snapshot.pem" \
     --key "${KEYDIR}/targets.pem" \
     --timestamp-expires "${TIMESTAMP_EXPIRATION}" \
     --snapshot-expires "${SNAPSHOT_EXPIRATION}" \
     --targets-expires "${TARGETS_EXPIRATION}" \
     --outdir "${TUF_REPO}" \
     --metadata-url "file://${TUF_REPO}"

   Note

   You can also run the TUF metadata update on a subset of TUF metadata files. For example, the timestamp.json metadata file expires more often than the other metadata files. Therefore, you can just update the timestamp metadata file by running the following command:

   Copy to Clipboard Toggle word wrap

   tuftool update \
     --root "${ROOT}" \
     --key "${KEYDIR}/timestamp.pem" \
     --timestamp-expires "${TIMESTAMP_EXPIRATION}" \
     --outdir "${TUF_REPO}" \
     --metadata-url "file://${TUF_REPO}"

6. Only update the root expiration date if it is about to expire:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool root expire "${ROOT}" "${ROOT_EXPIRATION}"

   Note

   You can skip this step if the root file is not close to expiring.

7. Update the root version:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool root bump-version "${ROOT}"

8. Sign the root metadata file again:

   Example

   Copy to Clipboard Toggle word wrap

   tuftool root sign "${ROOT}" -k "${KEYDIR}/root.pem"

9. Set the new root version, and copy the root metadata file in place:

   Example

   Copy to Clipboard Toggle word wrap

   export NEW_ROOT_VERSION=$(cat "${ROOT}" | jq -r ".signed.version")
   cp "${ROOT}" "${TUF_REPO}/root.json"
   cp "${ROOT}" "${TUF_REPO}/${NEW_ROOT_VERSION}.root.json"

10. Upload these changes to the TUF server.

    1. Create a compressed archive of the TUF repository:

       Example

       Copy to Clipboard Toggle word wrap

       tar -C "${WORK}" -czvf repository.tar.gz tuf-repo

    2. Update the RHTAS Ansible Playbook with these two lines:

       Example

       Copy to Clipboard Toggle word wrap

       tas_single_node_trust_root:
         full_archive: "{{ lookup('file', 'repository.tar.gz') | b64encode }}"

    3. Run the RHTAS Anisble Playbook to apply the changes:

       Example

       Copy to Clipboard Toggle word wrap

       ansible-playbook -i inventory play.yml

Procedure

1. Download the rekor-cli binary from the local command-line interface (CLI) tool download page to your workstation.

   1. Open a web browser, and go to the CLI server web page.

      Note

      The URL address is the configured node as defined by the tas_single_node_base_hostname variable. An example URL address would be, https://cli-server.example.com, given that the value of tas_single_node_base_hostname is example.com.

   2. From the download page, go to the rekor-cli download section, and click the link for your platform.

   3. From a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip rekor-cli-amd64.gz
      chmod +x rekor-cli-amd64

   4. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv rekor-cli-amd64 /usr/local/bin/rekor-cli

2. Download the tuftool binary from the local command-line interface (CLI) tool download page to your workstation.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the download page, go to the tuftool download section, and click the link for your platform.

   2. From a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

3. Assign shell variables to the base hostname, and the Rekor URL:

   Example

   Copy to Clipboard Toggle word wrap

   export BASE_HOSTNAME=BASE_HOSTNAME_OF_RHTAS_SERVICE
   export REKOR_URL=https://rekor.${BASE_HOSTNAME}

   Replace BASE_HOSTNAME_OF_RHTAS_SERVICE with the value of the tas_single_node_base_hostname variable.

4. Get the log tree identifier for the active shard:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_TREE_ID=$(rekor-cli loginfo --rekor_server $REKOR_URL --format json | jq -r .TreeID)

5. Configure your shell environment:

   Example

   Copy to Clipboard Toggle word wrap

   export MANAGED_NODE_IP=IP_OF_ANSIBLE_MANAGED_NODE
   export MANAGED_NODE_SSH_USER=USER_TO_CONNECT_TO_MANAGED_NODE
   export REMOTE_KEYS_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-signing-keys" | tr -d '[:space:]')
   export REMOTE_TUF_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-repository" | tr -d '[:space:]')

   Replace IP_OF_ANSIBLE_MANAGED_NODE and USER_TO_CONNECT_TO_MANAGED_NODE with values for your environment.

6. Set the log tree to the DRAINING state:

   Example

   Copy to Clipboard Toggle word wrap

   ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run --network=rhtas --rm registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --admin_server=trillian-logserver-pod:8091 --tree_id=${OLD_TREE_ID} --tree_state=DRAINING"

   While draining, the tree log will not accept any new entries. Watch and wait for the queue to empty.

   Important

   You must wait for the queues to be empty before proceeding to the next step. If leaves are still integrating while draining, then freezing the log tree during this process can cause the log path to exceed the maximum merge delay (MMD) threshold.

7. Freeze the log tree:

   Example

   Copy to Clipboard Toggle word wrap

   ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run --network=rhtas --rm registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --tree_id=${OLD_TREE_ID} --admin_server=trillian-logserver-pod:8091 --tree_state=FROZEN"

8. Get the length of the frozen log tree:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_SHARD_LENGTH=$(rekor-cli loginfo --rekor_server $REKOR_URL --format json | jq -r .ActiveTreeSize)

9. Get Rekor’s public key for the old shard:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_PUBLIC_KEY=$(curl -s $REKOR_URL/api/v1/log/publicKey | base64 | tr -d '\n')

10. Create a new log tree:

    Example

    Copy to Clipboard Toggle word wrap

    export NEW_TREE_ID=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run -q --network=rhtas --rm registry.redhat.io/rhtas/createtree-rhel9:1.1.0 --logtostderr=false --admin_server=trillian-logserver-pod:8091 --display_name=rekor-tree | tr -d '[:punct:][:blank:][:cntrl:]'")

    Now you have two log trees, one frozen tree, and a new tree that will become the active shard.

11. Create a new private key and an associated public key:

    Example

    Copy to Clipboard Toggle word wrap

    openssl ecparam -genkey -name secp384r1 -noout -out new-rekor.pem
    openssl ec -in new-rekor.pem -pubout -out new-rekor.pub
    export NEW_KEY_NAME=new-rekor.pub

    Important

    The new key must have a unique file name.

12. Get the active Rekor signing key, and save the key to a file:

    Example

    Copy to Clipboard Toggle word wrap

    rsync --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/certs/rekor-signer0.key ./rekor-signer0.key
    echo "$OLD_PUBLIC_KEY" | base64 -d > rekor.pub

13. Update the Rekor configuration in the RHTAS Ansible playbook:

    Example

    Copy to Clipboard Toggle word wrap

    tas_single_node_rekor:
      active_signer_id: "new-rekor-key"
      active_tree_id: NEW_TREE_ID
      private_keys:
        - id: "new-rekor-key"
          key: |
            {{ lookup('file', 'new-rekor.pem') }}
        - id: "private-0"
          key: |
            {{ lookup('file', 'rekor-signer0.key') }}
      public_keys:
        - id: "new-rekor-pubkey"
          key: |
            {{ lookup('file', 'new-rekor.pub') }}
        - id: "public-0"
          key: |
            {{ lookup('file', 'rekor.pub') }}
      sharding_config:
        - tree_id: OLD_TREE_ID
          tree_length: OLD_SHARD_LENGTH
          pem_pub_key: "public-0"

14. Configure The Update Framework (TUF) service to use the new Rekor public key.

    1. Configure your shell environment:

       Example

       Copy to Clipboard Toggle word wrap

       export WORK="${HOME}/trustroot-example"
       export ROOT="${WORK}/root/root.json"
       export KEYDIR="${WORK}/keys"
       export INPUT="${WORK}/input"
       export TUF_REPO="${WORK}/tuf-repo"
       export TUF_URL="https://tuf.${BASE_HOSTNAME}"

    2. Create a temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

    3. Download the TUF contents to the temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_KEYS_VOLUME}/" "${KEYDIR}"
       rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_TUF_VOLUME}/" "${TUF_REPO}"
       cp "${TUF_REPO}/root.json" "${ROOT}"

    4. Assign an environment variable to the active Rekor signer key file name:

       Example

       Copy to Clipboard Toggle word wrap

       export ACTIVE_KEY_NAME=rekor.pub

    5. Expire the old Rekor signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-rekor-target "${ACTIVE_KEY_NAME}" \
         --rekor-uri "${REKOR_URL}" \
         --rekor-status "Expired" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    6. Add the new Rekor signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-rekor-target "${NEW_KEY_NAME}" \
         --rekor-uri "${REKOR_URL}" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    7. Create a compressed archive file of the updated TUF repository:

       Example

       Copy to Clipboard Toggle word wrap

       tar -C "${WORK}" -czvf repository.tar.gz tuf-repo

    8. Update the RHTAS Ansible playbook by adding the new compressed archive file name to the tas_single_node_trust_root variable:

       Example

       Copy to Clipboard Toggle word wrap

       tas_single_node_trust_root:
         full_archive: "{{ lookup('file', 'repository.tar.gz') | b64encode }}"

    9. Delete the working directory:

       Example

       Copy to Clipboard Toggle word wrap

       rm -r $WORK

15. Run the RHTAS Ansible Playbook to apply the changes:

    Example

    Copy to Clipboard Toggle word wrap

    ansible-playbook -i inventory play.yml

16. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts with the new Rekor signer key.

Procedure

1. Download the tuftool binary from the local command-line interface (CLI) tool download page to your workstation.

   Note

   The URL address is the configured node as defined by the tas_single_node_base_hostname variable. An example URL address would be, https://cli-server.example.com, given the tas_single_node_base_hostname value as example.com.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the download page, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Configure your shell environment:

   Example

   Copy to Clipboard Toggle word wrap

   export MANAGED_NODE_IP=IP_OF_ANSIBLE_MANAGED_NODE
   export MANAGED_NODE_SSH_USER=USER_TO_CONNECT_TO_MANAGED_NODE
   export REMOTE_KEYS_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-signing-keys" | tr -d '[:space:]')
   export REMOTE_TUF_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-repository" | tr -d '[:space:]')
   export BASE_HOSTNAME=BASE_HOSTNAME_OF_RHTAS_SERVICE

   Replace BASE_HOSTNAME_OF_RHTAS_SERVICE with the value of the tas_single_node_base_hostname variable.

3. Download the CTlog configuration map, the CTlog keys, and the Fulcio root certificate to your workstation:

   Example

   Copy to Clipboard Toggle word wrap

   rsync --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/configs/ctlog-config.yaml ./ctlog-config.yaml
   rsync --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/certs/ctlog0.key ./ctfe.key
   rsync --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/certs/ctlog0.pub ./ctfe.pub
   rsync --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/certs/fulcio.pem ./fulcio-0.pem

4. Capture the current tree identifier:

   Example

   Copy to Clipboard Toggle word wrap

   export OLD_TREE_ID=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo cat /etc/rhtas/configs/ctlog-treeid-config.yaml | grep 'tree_id:' | awk '{print \$2}'" | tr -d '[:punct:][:blank:][:cntrl:]')

5. Set the log tree to the DRAINING state:

   Example

   Copy to Clipboard Toggle word wrap

   ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run --network=rhtas --rm registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --tree_id=${OLD_TREE_ID} --admin_server=trillian-logserver-pod:8091 --tree_state=DRAINING"

   While draining, the tree log will not accept any new entries. Watch and wait for the queue to empty.

   Important

   You must wait for the queues to be empty before proceeding to the next step. If leaves are still integrating while draining, then freezing the log tree during this process can cause the log path to exceed the maximum merge delay (MMD) threshold.

6. Once the queue has been fully drained, freeze the log:

   Example

   Copy to Clipboard Toggle word wrap

   ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run --network=rhtas --rm registry.redhat.io/rhtas/updatetree-rhel9:1.1.0 --tree_id=${OLD_TREE_ID} --admin_server=trillian-logserver-pod:8091 --tree_state=FROZEN"

7. Create a new Merkle tree, and capture the new tree identifier:

   Example

   Copy to Clipboard Toggle word wrap

   export NEW_TREE_ID=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman run -q --network=rhtas --rm registry.redhat.io/rhtas/createtree-rhel9:1.1.0 --logtostderr=false --admin_server=trillian-logserver-pod:8091 --display_name=ctlog-tree" | tr -d '[:punct:][:blank:][:cntrl:]')

8. Generate a new certificate, along with new public and private keys:

   Example

   Copy to Clipboard Toggle word wrap

   openssl ecparam -genkey -name prime256v1 -noout -out new-ctlog.pem
   openssl ec -in new-ctlog.pem -pubout -out new-ctlog-public.pem
   openssl ec -in new-ctlog.pem -out new-ctlog.pass.pem -des3 -passout pass:"CHANGE_ME"

   Replace CHANGE_ME with a new password.

   Important

   The certificate and new keys must have unique file names.

9. Update the CT log configuration.

   1. Open the RHTAS Ansible playbook for editing.

   2. Configuring the CTlog signer key rotation for the first time, you need to add the following to the tas_single_node_ctlog.sharding_config section:

      Example

      Copy to Clipboard Toggle word wrap

      tas_single_node_ctlog:
        sharding_config:
          - treeid: OLD_TREE_ID # frozen log
            prefix: "rhtasansible"
            private_key: "private-0"
            password: "rhtas"
            root_pem_file: "/ctfe-keys/fulcio-0"
            not_after_limit:
              seconds: 1728056285
              nanos: 012111000

      Replace OLD_TREE_ID with the contents contained in the $OLD_TREE_ID environment variable.

      Note

      You can get the current time value for seconds and nanoseconds, by running the following commands: date +%s, and date +%N.

      Important

      The not_after_limit field defines the end of the timestamp range for the frozen log only. Certificates beyond this point in time are no longer accepted for inclusion in this log.

   3. Copy and paste the frozen log block, appending it to the tas_single_node_ctlog.sharding_config section, creating a new entry.

   4. Change the following lines in the new log block. Set the treeid to the new tree identifier, change the prefix to trusted-artifact-signer, change the private_key path to private-1, change not_after_limit to not_after_start, set the timestamp range, and update tas_single_node_fulcio.ct_log_prefix for Fulcio to make use of the new log:

      Example

      Copy to Clipboard Toggle word wrap

      tas_single_node_ctlog:
        sharding_config:
           ... # frozen log
           - treeid: NEW_TREE_ID # new active log
             prefix: "trusted-artifact-signer"
             private_key: "private-1"
             password: "CHANGE_ME"
             root_pem_file: "/ctfe-keys/fulcio-0"
             not_after_start:
               seconds: 1713201754
               nanos: 155663000
      tas_single_node_fulcio:
        ct_log_prefix: "trusted-artifact-signer"

      Replace CHANGE_ME with the new private key password. The password here must match the password used for generating the new private and public keys.

      Important

      The not_after_start field defines the beginning of the timestamp range inclusively. This means the log will start accepting certificates at this point in time.

10. Update the tas_single_node_ctlog section for CTlog to distribute the new keys to the managed node:

    Example

    Copy to Clipboard Toggle word wrap

    tas_single_node_ctlog:
      ...
      private_keys:
        - id: private-0
          key: |
            {{ lookup('file', 'ctfe.key') }}
        - id: private-1
          key: |
            {{ lookup('file', 'new-ctlog.pass.pem') }}
      public_keys:
        - id: public-0
          key: |
            {{ lookup('file', 'ctfe.pub') }}
        - id: public-1
          key: |
            {{ lookup('file', 'new-ctlog-public.pem') }}

11. Configure The Update Framework (TUF) service to use the new CT log public key.

    1. Configure your shell environment:

       Example

       Copy to Clipboard Toggle word wrap

       export WORK="${HOME}/trustroot-example"
       export ROOT="${WORK}/root/root.json"
       export KEYDIR="${WORK}/keys"
       export INPUT="${WORK}/input"
       export TUF_REPO="${WORK}/tuf-repo"
       export TUF_URL="https://tuf.${BASE_HOSTNAME}"

    2. Create a temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

    3. Download the TUF contents to the temporary TUF directory structure:

       Example

       Copy to Clipboard Toggle word wrap

       rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_KEYS_VOLUME}/" "${KEYDIR}"
       rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_TUF_VOLUME}/" "${TUF_REPO}"
       cp "${TUF_REPO}/root.json" "${ROOT}"

    4. Assign an environment variable to the active CT log signer key file name:

       Example

       Copy to Clipboard Toggle word wrap

       export ACTIVE_CTFE_NAME=ctfe.pub

    5. Expire the old CT log signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-ctlog-target "$ACTIVE_CTFE_NAME" \
         --ctlog-uri "https://ctlog.rhtas" \
         --ctlog-status "Expired" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    6. Add the new CT log signer key:

       Example

       Copy to Clipboard Toggle word wrap

       tuftool rhtas \
         --root "${ROOT}" \
         --key "${KEYDIR}/snapshot.pem" \
         --key "${KEYDIR}/targets.pem" \
         --key "${KEYDIR}/timestamp.pem" \
         --set-ctlog-target "new-ctlog-public.pem" \
         --ctlog-uri "https://ctlog.rhtas" \
         --outdir "${TUF_REPO}" \
         --metadata-url "file://${TUF_REPO}"

    7. Create a compressed archive file of the updated TUF repository:

       Example

       Copy to Clipboard Toggle word wrap

       tar -C "${WORK}" -czvf repository.tar.gz tuf-repo

    8. Update the RHTAS Ansible playbook by adding the new compressed archive file name to the tas_single_node_trust_root variable:

       Example

       Copy to Clipboard Toggle word wrap

       tas_single_node_trust_root:
         full_archive: "{{ lookup('file', 'repository.tar.gz') | b64encode }}"

    9. Save the changes to the playbook, and close your text editor.

12. Run the RHTAS Ansible playbook to apply the changes:

    Example

    Copy to Clipboard Toggle word wrap

    ansible-playbook -i inventory play.yml

13. Delete the working directory:

    Example

    Copy to Clipboard Toggle word wrap

    rm -r $WORK

14. Update the cosign configuration with the updated TUF configuration:

    Example

    Copy to Clipboard Toggle word wrap

    cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

    Now, you are ready to sign and verify your artifacts with the new CT log signer key.

Procedure

1. Download the tuftool binary from the local command-line interface (CLI) tool download page to your workstation.

   Note

   The URL address is the configured node as defined by the tas_single_node_base_hostname variable. An example URL address would be, https://cli-server.example.com, given the tas_single_node_base_hostname value as example.com.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the download page, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Generate a new certificate, along with new public and private keys:

   Example

   Copy to Clipboard Toggle word wrap

   openssl ecparam -genkey -name prime256v1 -noout -out new-fulcio.pem
   openssl ec -in new-fulcio.pem -pubout -out new-fulcio-public.pem
   openssl ec -in new-fulcio.pem -out new-fulcio.pass.pem -des3 -passout pass:"CHANGE_ME"
   openssl req -new -x509 -key new-fulcio.pass.pem -out new-fulcio.cert.pem

   Replace CHANGE_ME with a new password.

   Important

   The certificate and new keys must have unique file names.

3. Update the RHTAS Ansible playbook by adding the new private key file name, the new certificate content, and the password to the tas_single_node_fulcio variable:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_fulcio:
     root_ca: "{{ lookup('file', 'new-fulcio.cert.pem') }}"
     private_key: "{{ lookup('file', 'new-fulcio.pass.pem') }}"
     ca_passphrase: CHANGE_ME

   Replace CHANGE_ME with a new password.

   Note

   The password here must match the password used for generating the new private and public keys.

   Note

   We recommend sourcing the passphrase either from a file or encrypted by using Ansible Vault.

4. Configure The Update Framework (TUF) service to use the new Fulcio certificate.

   1. Set up your shell environment:

      Example

      Copy to Clipboard Toggle word wrap

      export WORK="${HOME}/trustroot-example"
      export ROOT="${WORK}/root/root.json"
      export KEYDIR="${WORK}/keys"
      export INPUT="${WORK}/input"
      export TUF_REPO="${WORK}/tuf-repo"
      export TUF_URL="https://tuf.${BASE_HOSTNAME}"
      export MANAGED_NODE_IP=IP_OF_ANSIBLE_MANAGED_NODE
      export MANAGED_NODE_SSH_USER=USER_TO_CONNECT_TO_MANAGED_NODE
      export REMOTE_KEYS_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-signing-keys" | tr -d '[:space:]')
      export REMOTE_TUF_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-repository" | tr -d '[:space:]')

   2. Create a temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

   3. Download the TUF contents to the temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_KEYS_VOLUME}/" "${KEYDIR}"
      rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_TUF_VOLUME}/" "${TUF_REPO}"
      cp "${TUF_REPO}/root.json" "${ROOT}"

   4. Find the active Fulcio certificate file name. Open the latest target file, for example, 1.targets.json, within the local TUF repository. In this file you will find the active Fulcio certificate file name, for example, fulcio_v1.crt.pem. Set an environment variable with this active Fulcio certificate file name:

      Example

      Copy to Clipboard Toggle word wrap

      export ACTIVE_CERT_NAME=fulcio_v1.crt.pem

   5. Get the active Fulico certificate from the managed node:

      Example

      Copy to Clipboard Toggle word wrap

      rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:/etc/rhtas/certs/fulcio.pem "${ACTIVE_CERT_NAME}"

   6. Expire the old certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-fulcio-target "$ACTIVE_CERT_NAME" \
        --fulcio-uri "https://fulcio.rhtas" \
        --fulcio-status "Expired" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   7. Add the new Fulcio certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-fulcio-target "new-fulcio.cert.pem" \
        --fulcio-uri "https://fulcio.rhtas" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   8. Create a compressed archive file of the updated TUF repository:

      Example

      Copy to Clipboard Toggle word wrap

      tar -C "${WORK}" -czvf repository.tar.gz tuf-repo

   9. Update the RHTAS Ansible playbook by adding the new compressed archive file content to the tas_single_node_trust_root variable:

      Example

      Copy to Clipboard Toggle word wrap

      tas_single_node_trust_root:
        full_archive: "{{ lookup('file', 'repository.tar.gz') | b64encode }}"

   10. Delete the working directory:

       Example

       Copy to Clipboard Toggle word wrap

       rm -r $WORK

5. Run the RHTAS Ansible Playbook to apply the changes:

   Example

   Copy to Clipboard Toggle word wrap

   ansible-playbook -i inventory play.yml

6. Update the cosign configuration with the updated TUF configuration:

   Example

   Copy to Clipboard Toggle word wrap

   cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

   Now, you are ready to sign and verify your artifacts with the new Fulcio certificate.

Procedure

1. Download the tuftool binary from the local command-line interface (CLI) tool download page to your workstation.

   Note

   The URL address is the configured node as defined by the tas_single_node_base_hostname variable. An example URL address would be, https://cli-server.example.com, given that the value of tas_single_node_base_hostname is example.com.

   Important

   Currently, the tuftool binary is only available for Linux operating systems on the x86_64 architecture.

   1. From the download page, go to the tuftool download section, and click the link for your platform.

   2. Open a terminal on your workstation, decompress the binary .gz file, and set the execute bit:

      Example

      Copy to Clipboard Toggle word wrap

      gunzip tuftool-amd64.gz
      chmod +x tuftool-amd64

   3. Move and rename the binary to a location within your $PATH environment:

      Example

      Copy to Clipboard Toggle word wrap

      sudo mv tuftool-amd64 /usr/local/bin/tuftool

2. Generate a new certificate chain, and a new signer key.

   Important

   The new certificate and keys must have unique file names.

   1. Create a temporary working directory:

      Example

      Copy to Clipboard Toggle word wrap

      mkdir certs && cd certs

   2. Create the root certificate authority (CA) private key, and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -x509 -newkey rsa:2048 -days 365 -sha256 -nodes \
      -keyout rootCA.key.pem -out rootCA.crt.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=RootCA/CN=RootCA" \
      -addext "basicConstraints=CA:true" -addext "keyUsage=cRLSign, keyCertSign"

      Replace CHANGE_ME with a new password.

   3. Create the intermediate CA private key and certificate signing request (CSR), and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -newkey rsa:2048 -sha256 \
      -keyout intermediateCA.key.pem -out intermediateCA.csr.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=IntermediateCA/CN=IntermediateCA"

      Replace CHANGE_ME with a new password.

   4. Sign the intermediate CA certificate with the root CA:

      Example

      Copy to Clipboard Toggle word wrap

      openssl x509 -req -in intermediateCA.csr.pem -CA rootCA.crt.pem -CAkey rootCA.key.pem \
      -CAcreateserial -out intermediateCA.crt.pem -days 365 -sha256 \
      -extfile <(echo -e "basicConstraints=CA:true\nkeyUsage=cRLSign, keyCertSign\nextendedKeyUsage=critical,timeStamping") \
      -passin pass:"CHANGE_ME"

      Replace CHANGE_ME with the root CA private key password to sign the intermediate CA certificate.

   5. Create the leaf CA private key and CSR, and set a password:

      Example

      Copy to Clipboard Toggle word wrap

      openssl req -newkey rsa:2048 -sha256 \
      -keyout leafCA.key.pem -out leafCA.csr.pem \
      -passout pass:"CHANGE_ME" \
      -subj "/C=CC/ST=state/L=Locality/O=RH/OU=LeafCA/CN=LeafCA"

   6. Sign the leaf CA certificate with the intermediate CA:

      Example

      Copy to Clipboard Toggle word wrap

      openssl x509 -req -in leafCA.csr.pem -CA intermediateCA.crt.pem -CAkey intermediateCA.key.pem \
        -CAcreateserial -out leafCA.crt.pem -days 365 -sha256 \
        -extfile <(echo -e "basicConstraints=CA:false\nkeyUsage=cRLSign, keyCertSign\nextendedKeyUsage=critical,timeStamping") \
        -passin pass:"CHANGE_ME"

      Replace CHANGE_ME with the intermediate CA private key password to sign the leaf CA certificate.

   7. Create the certificate chain by combining the newly created certificates together:

      Example

      Copy to Clipboard Toggle word wrap

      cat leafCA.crt.pem intermediateCA.crt.pem rootCA.crt.pem > new-tsa.certchain.pem

3. Update the RHTAS playbook with the new certificate chain, private key, and password:

   Example

   Copy to Clipboard Toggle word wrap

   tas_single_node_tsa:
     certificate_chain: "{{ lookup('file', 'new-tsa.certchain.pem') }}"
     signer_private_key: "{{ lookup('file', 'leafCA.key.pem') }}"
     ca_passphrase: CHANGE_ME

   Replace CHANGE_ME with the leaf CA private key password.

   Note

   Red Hat recommends sourcing the passphrase either from a file or encrypted by using Ansible Vault.

4. Find your active TSA certificate file name, the TSA URL string, and configure your shell environment with these values:

   Example

   Copy to Clipboard Toggle word wrap

   export BASE_HOSTNAME=BASE_HOSTNAME_OF_RHTAS_SERVICE
   export ACTIVE_CERT_CHAIN_NAME=tsa.certchain.pem
   export TSA_URL=https://tsa.${BASE_HOSTNAME}/api/v1/timestamp
   curl $TSA_URL/certchain -o $ACTIVE_CERT_CHAIN_NAME

5. Configure The Update Framework (TUF) service to use the new TSA certificate chain.

   1. Set up your shell environment:

      Example

      Copy to Clipboard Toggle word wrap

      export WORK="${HOME}/trustroot-example"
      export ROOT="${WORK}/root/root.json"
      export KEYDIR="${WORK}/keys"
      export INPUT="${WORK}/input"
      export TUF_REPO="${WORK}/tuf-repo"
      export TUF_URL="https://tuf.${BASE_HOSTNAME}"
      export MANAGED_NODE_IP=IP_OF_ANSIBLE_MANAGED_NODE
      export MANAGED_NODE_SSH_USER=USER_TO_CONNECT_TO_MANAGED_NODE
      export NEW_CERT_CHAIN_NAME=new-tsa.certchain.pem

   2. Create a temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      mkdir -p "${WORK}/root/" "${KEYDIR}" "${INPUT}" "${TUF_REPO}"

   3. Download the TUF contents to the temporary TUF directory structure:

      Example

      Copy to Clipboard Toggle word wrap

      export REMOTE_KEYS_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-signing-keys" | tr -d '[:space:]')
      export REMOTE_TUF_VOLUME=$(ssh ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP} -t "sudo podman volume mount tuf-repository" | tr -d '[:space:]')
      rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_KEYS_VOLUME}/" "${KEYDIR}"
      rsync -r --rsync-path="sudo rsync" ${MANAGED_NODE_SSH_USER}@${MANAGED_NODE_IP}:"${REMOTE_TUF_VOLUME}/" "${TUF_REPO}"
      cp "${TUF_REPO}/root.json" "${ROOT}"

   4. Expire the old TSA certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-tsa-target "$ACTIVE_CERT_CHAIN_NAME" \
        --tsa-uri "$TSA_URL" \
        --tsa-status "Expired" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   5. Add the new TSA certificate:

      Example

      Copy to Clipboard Toggle word wrap

      tuftool rhtas \
        --root "${ROOT}" \
        --key "${KEYDIR}/snapshot.pem" \
        --key "${KEYDIR}/targets.pem" \
        --key "${KEYDIR}/timestamp.pem" \
        --set-tsa-target "$NEW_CERT_CHAIN_NAME" \
        --tsa-uri "$TSA_URL" \
        --outdir "${TUF_REPO}" \
        --metadata-url "file://${TUF_REPO}"

   6. Create a compressed archive file of the updated TUF repository:

      Example

      Copy to Clipboard Toggle word wrap

      tar -C "${WORK}" -czvf repository.tar.gz tuf-repo

   7. Update the RHTAS Ansible playbook by adding the new compressed archive file name to the tas_single_node_trust_root variable:

      Example

      Copy to Clipboard Toggle word wrap

      tas_single_node_trust_root:
        full_archive: "{{ lookup('file', 'repository.tar.gz') | b64encode }}"

   8. Delete the working directory:

      Example

      Copy to Clipboard Toggle word wrap

      rm -r $WORK

6. Run the RHTAS Ansible Playbook to apply the changes:

   Example

   Copy to Clipboard Toggle word wrap

   ansible-playbook -i inventory play.yml

7. Update the cosign configuration with the updated TUF configuration:

   Example

   Copy to Clipboard Toggle word wrap

   cosign initialize --mirror=$TUF_URL --root=$TUF_URL/root.json

   Now, you are ready to sign and verify your artifacts that uses the new TSA signer key, and certificate.