Backing up and restoring applications

Procedure

1. Create the following environment variables by running the following commands:

   Note

   Change the cluster name to match your ROSA cluster, and ensure you are logged into the cluster as an administrator. Ensure that all fields are outputted correctly before continuing.

   $ export CLUSTER_NAME=my-cluster 1
   export ROSA_CLUSTER_ID=$(rosa describe cluster -c ${CLUSTER_NAME} --output json | jq -r .id)
   export REGION=$(rosa describe cluster -c ${CLUSTER_NAME} --output json | jq -r .region.id)
   export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o jsonpath='{.spec.serviceAccountIssuer}' | sed 's|^https://||')
   export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
   export CLUSTER_VERSION=$(rosa describe cluster -c ${CLUSTER_NAME} -o json | jq -r .version.raw_id | cut -f -2 -d '.')
   export ROLE_NAME="${CLUSTER_NAME}-openshift-oadp-aws-cloud-credentials"
   export SCRATCH="/tmp/${CLUSTER_NAME}/oadp"
   mkdir -p ${SCRATCH}
   echo "Cluster ID: ${ROSA_CLUSTER_ID}, Region: ${REGION}, OIDC Endpoint:
   ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"

   1

   Replace my-cluster with your ROSA cluster name.

2. On the AWS account, create an IAM policy to allow access to S3.

   1. Check to see if the policy exists by running the following command:

      $ POLICY_ARN=$(aws iam list-policies --query "Policies[?PolicyName=='RosaOadpVer1'].{ARN:Arn}" --output text) 1

      1

      Replace RosaOadp with your policy name.

   2. Use the following command to create the policy JSON file and then create the policy in ROSA.

      Note

      If the policy ARN is not found, the command will create the policy. If the policy ARN already exists, the if statement will intentionally skip the policy creation.

      $ if [[ -z "${POLICY_ARN}" ]]; then
      cat << EOF > ${SCRATCH}/policy.json 1
      {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "s3:CreateBucket",
            "s3:DeleteBucket",
            "s3:PutBucketTagging",
            "s3:GetBucketTagging",
            "s3:PutEncryptionConfiguration",
            "s3:GetEncryptionConfiguration",
            "s3:PutLifecycleConfiguration",
            "s3:GetLifecycleConfiguration",
            "s3:GetBucketLocation",
            "s3:ListBucket",
            "s3:GetObject",
            "s3:PutObject",
            "s3:DeleteObject",
            "s3:ListBucketMultipartUploads",
            "s3:AbortMultipartUpload",
            "s3:ListMultipartUploadParts",
            "ec2:DescribeSnapshots",
            "ec2:DescribeVolumes",
            "ec2:DescribeVolumeAttribute",
            "ec2:DescribeVolumesModifications",
            "ec2:DescribeVolumeStatus",
            "ec2:CreateTags",
            "ec2:CreateVolume",
            "ec2:CreateSnapshot",
            "ec2:DeleteSnapshot"
          ],
          "Resource": "*"
        }
       ]}
      EOF
      
      POLICY_ARN=$(aws iam create-policy --policy-name "RosaOadpVer1" \
      --policy-document file:///${SCRATCH}/policy.json --query Policy.Arn \
      --tags Key=rosa_openshift_version,Value=${CLUSTER_VERSION} Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-oadp Key=operator_name,Value=openshift-oadp \
      --output text)
      fi

      1

      SCRATCH is a name for a temporary directory created for the environment variables.

   3. View the policy ARN by running the following command:

      $ echo ${POLICY_ARN}

3. Create an IAM role trust policy for the cluster:

   1. Create the trust policy file by running the following command:

      $ cat <<EOF > ${SCRATCH}/trust-policy.json
      {
          "Version": "2012-10-17",
          "Statement": [{
            "Effect": "Allow",
            "Principal": {
              "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_ENDPOINT}"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
              "StringEquals": {
                "${OIDC_ENDPOINT}:sub": [
                  "system:serviceaccount:openshift-adp:openshift-adp-controller-manager",
                  "system:serviceaccount:openshift-adp:velero"]
              }
            }
          }]
      }
      EOF

   2. Create the role by running the following command:

      $ ROLE_ARN=$(aws iam create-role --role-name \
        "${ROLE_NAME}" \
        --assume-role-policy-document file://${SCRATCH}/trust-policy.json \
        --tags Key=rosa_cluster_id,Value=${ROSA_CLUSTER_ID}
      Key=rosa_openshift_version,Value=${CLUSTER_VERSION}
      Key=rosa_role_prefix,Value=ManagedOpenShift
      Key=operator_namespace,Value=openshift-adp
      Key=operator_name,Value=openshift-oadp \
         --query Role.Arn --output text)

   3. View the role ARN by running the following command:

      $ echo ${ROLE_ARN}

4. Attach the IAM policy to the IAM role by running the following command:

   $ aws iam attach-role-policy --role-name "${ROLE_NAME}" \
     --policy-arn ${POLICY_ARN}

Procedure

1. Create an Red Hat OpenShift Service on AWS secret from your AWS token file by entering the following commands:

   1. Create the credentials file:

      $ cat <<EOF > ${SCRATCH}/credentials
        [default]
        role_arn = ${ROLE_ARN}
        web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
      EOF

   2. Create a namespace for OADP:

      $ oc create namespace openshift-adp

   3. Create the Red Hat OpenShift Service on AWS secret:

      $ oc -n openshift-adp create secret generic cloud-credentials \
        --from-file=${SCRATCH}/credentials

      Note

      In Red Hat OpenShift Service on AWS versions 4.15 and later, the OADP Operator supports a new standardized STS workflow through the Operator Lifecycle Manager (OLM) and Cloud Credentials Operator (CCO). In this workflow, you do not need to create the above secret, you only need to supply the role ARN during the installation of OLM-managed operators using the Red Hat OpenShift Service on AWS web console, for more information see Installing from OperatorHub using the web console.

      The preceding secret is created automatically by CCO.

2. Install the OADP Operator:

   1. In the Red Hat OpenShift Service on AWS web console, browse to Operators → OperatorHub.
   2. Search for the OADP Operator.
   3. In the role_ARN field, paste the role_arn that you created previously and click Install.

3. Create AWS cloud storage using your AWS credentials by entering the following command:

   $ cat << EOF | oc create -f -
     apiVersion: oadp.openshift.io/v1alpha1
     kind: CloudStorage
     metadata:
       name: ${CLUSTER_NAME}-oadp
       namespace: openshift-adp
     spec:
       creationSecret:
         key: credentials
         name: cloud-credentials
       enableSharedConfig: true
       name: ${CLUSTER_NAME}-oadp
       provider: aws
       region: $REGION
   EOF

4. Check your application’s storage default storage class by entering the following command:

   $ oc get pvc -n <namespace>

   Example output

   NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
   applog   Bound    pvc-351791ae-b6ab-4e8b-88a4-30f73caf5ef8   1Gi        RWO            gp3-csi        4d19h
   mysql    Bound    pvc-16b8e009-a20a-4379-accc-bc81fedd0621   1Gi        RWO            gp3-csi        4d19h

5. Get the storage class by running the following command:

   $ oc get storageclass

   Example output

   NAME                PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
   gp2                 kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   true                   4d21h
   gp2-csi             ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
   gp3                 ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
   gp3-csi (default)   ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h

   Note

   The following storage classes will work:

   • gp3-csi
   • gp2-csi
   • gp3
   • gp2

   If the application or applications that are being backed up are all using persistent volumes (PVs) with Container Storage Interface (CSI), it is advisable to include the CSI plugin in the OADP DPA configuration.

6. Create the DataProtectionApplication resource to configure the connection to the storage where the backups and volume snapshots are stored:

   1. If you are using only CSI volumes, deploy a Data Protection Application by entering the following command:

      $ cat << EOF | oc create -f -
        apiVersion: oadp.openshift.io/v1alpha1
        kind: DataProtectionApplication
        metadata:
          name: ${CLUSTER_NAME}-dpa
          namespace: openshift-adp
        spec:
          backupImages: true 1
          features:
            dataMover:
              enable: false
          backupLocations:
          - bucket:
              cloudStorageRef:
                name: ${CLUSTER_NAME}-oadp
              credential:
                key: credentials
                name: cloud-credentials
              prefix: velero
              default: true
              config:
                region: ${REGION}
          configuration:
            velero:
              defaultPlugins:
              - openshift
              - aws
              - csi
            restic:
              enable: false
      EOF

      1

      ROSA supports internal image backup. Set this field to false if you do not want to use image backup.

1. If you are using CSI or non-CSI volumes, deploy a Data Protection Application by entering the following command:

   $ cat << EOF | oc create -f -
     apiVersion: oadp.openshift.io/v1alpha1
     kind: DataProtectionApplication
     metadata:
       name: ${CLUSTER_NAME}-dpa
       namespace: openshift-adp
     spec:
       backupImages: true 1
       features:
         dataMover:
            enable: false
       backupLocations:
       - bucket:
           cloudStorageRef:
             name: ${CLUSTER_NAME}-oadp
           credential:
             key: credentials
             name: cloud-credentials
           prefix: velero
           default: true
           config:
             region: ${REGION}
       configuration:
         velero:
           defaultPlugins:
           - openshift
           - aws
         nodeAgent: 2
           enable: false
           uploaderType: restic
       snapshotLocations:
         - velero:
             config:
               credentialsFile: /tmp/credentials/openshift-adp/cloud-credentials-credentials 3
               enableSharedConfig: "true" 4
               profile: default 5
               region: ${REGION} 6
             provider: aws
   EOF

   1

   ROSA supports internal image backup. Set this field to false if you do not want to use image backup.

   2

   See the important note regarding the nodeAgent attribute.

   3

   The credentialsFile field is the mounted location of the bucket credential on the pod.

   4

   The enableSharedConfig field allows the snapshotLocations to share or reuse the credential defined for the bucket.

   5

   Use the profile name set in the AWS credentials file.

   6

   Specify region as your AWS region. This must be the same as the cluster region.

   You are now ready to back up and restore Red Hat OpenShift Service on AWS applications, as described in Backing up applications.