Chapter 8. Backup and recovery

Procedure

1. Create a Secret containing the repo-password key with a password:

   apiVersion: v1
   kind: Secret
   metadata:
     name: backup-encryption-password-secret
     namespace: eclipse-che
   type: Opaque
   stringData:
     repo-password: <password>

   (The CodeReady Workspaces Operator will set up this password for the backup repository that the CodeReady Workspaces Operator will create from this custom object on the backup server.)

   Warning

   The backup repository password is used to encrypt the backup data. If you lose this password, you will lose the backup data.

2. Create a Secret (for example, name: ssh-key-secret) containing the ssh-privatekey key with a private SSH key for logging in to the SFTP server without a password.

3. Create the CheBackupServerConfiguration custom object:

   apiVersion: org.eclipse.che/v1
   kind: CheBackupServerConfiguration
   metadata:
     name: backup-server-configuration
     namespace: openshift-workspaces
   spec: 1
     sftp:
       username: <username> 2
       hostname: <hostname> 3
       port: 1234 4
       repositoryPath: red-hat-codeready-workspaces-backups 5
       repositoryPasswordSecretRef: backup-encryption-password-secret 6
       sshKeySecretRef: ssh-key-secret 7

   1

   Must only contain one section (such as sftp).

   2

   User name on the remote server to log in using the SSH protocol.

   3

   Remote server hostname.

   4

   Optional property that specifies the port on which an SFTP server is running. The default value is 22.

   5

   Absolute or relative path on the server where backup snapshots are stored.

   6

   Secret created in step 1.

   7

   Secret created in step 2.

4. Optional: To configure multiple backup servers, create a separate CheBackupServerConfiguration custom object for each backup server.

Procedure

1. Create the CheClusterBackup custom object, which creates a backup snapshot:

   apiVersion: org.eclipse.che/v1
   kind: CheClusterBackup
   metadata:
     name: red-hat-codeready-workspaces-backup
     namespace: openshift-workspaces
   spec:
     backupServerConfigRef: backup-server-configuration 1
     useInternalBackupServer: false 2

   1

   Name of the CheBackupServerConfiguration custom object defining what backup server to use.

   2

   Configures the Operator through this custom resource to use the CodeReady Workspaces-managed internal backup server or an administrator-managed external backup server (SFTP, Amazon S3 or S3 API compatible storage, or REST).

   Tip

   If you intend to reuse a name for CheClusterBackup custom objects, first delete any existing custom object with the same name. To delete it on the command line, use oc:

   $ oc delete CheClusterBackup <name> -n openshift-workspaces

2. Read the status section of the CheClusterBackup custom object to verify the backup process, for example:

   status:
     message: 'Backup is in progress. Start time: <timestamp>' 1
     stage: Collecting CodeReady Workspaces installation data 2
     state: InProgress 3

   1

   Summary of the latest state of the backup process.

   2

   Current stage of the backup process in a human-readable format.

   3

   Backup process state: InProgress or Succeeded or Failed.

   The CodeReady Workspaces instance is backed up in a snapshot when state is Succeeded:

   status:
     ...
     message: >-
       Backup successfully finished at 2021-12-03 10:07:51.151090621 +0000 UTC
       m=+999.553272281 1
     snapshotId: 9f0adce2 2
     state: Succeeded 3

   1

   Summary of the latest state of the backup process.

   2

   ID of the created backup snapshot. This field appears only when state is Succeeded.

   3

   Backup process state: Succeeded (or Failed).

Procedure

1. Enter the crwctl server:backup command with the following arguments:

   $ crwctl server:backup \
   --repository-url=<repository_url> \ 1
   --repository-password=<repository_password> \ 2
   --ssh-key-file=<ssh_key_file> 3

   1

   Backup repository URL as an argument using the --repository-url or -r option. Syntax for the backup repository URL: sftp:<user>@<host>:/<repository_path_on_sftp_server>. An example of a repository path on the SFTP server: /srv/restic-repo. For more details about repository URL syntax, see Restic Documentation.

   2

   Backup repository password as an argument using the --repository-password or -p option.

   3

   Path to a private SSH key file for authenticating on the SFTP server.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --repository-url and --repository-password options with the crwctl server:backup command just once and omit them onward when entering crwctl server:backup or crwctl server:restore.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Run the following command on a command line:

   $ crwctl server:backup \
   --backup-server-config-name=<name_of_CheBackupServerConfiguration> 1

   1

   This option points crwctl to a CheBackupServerConfiguration custom object. You can find the name of the CheBackupServerConfiguration custom object under metadata in the custom object.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --backup-server-config-name option with the crwctl server:backup command just once and omit this option onward when entering crwctl server:backup.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Create a Secret containing the repo-password key with a password:

   apiVersion: v1
   kind: Secret
   metadata:
     name: backup-encryption-password-secret
     namespace: eclipse-che
   type: Opaque
   stringData:
     repo-password: <password>

   (The CodeReady Workspaces Operator will set up this password for the backup repository that the CodeReady Workspaces Operator will create from this custom object on the backup server.)

   Warning

   The backup repository password is used to encrypt the backup data. If you lose this password, you will lose the backup data.

2. Create a Secret (for example, name: aws-user-credentials-secret) containing:

   • awsAccessKeyId key with the AWS access key ID of the user
   • awsSecretAccessKey key with the AWS secret access key of the user

3. Create the CheBackupServerConfiguration custom object:

   apiVersion: org.eclipse.che/v1
   kind: CheBackupServerConfiguration
   metadata:
     name: backup-server-configuration
     namespace: openshift-workspaces
   spec: 1
     awss3:
       protocol: https 2
       hostname: s3.amazonaws.com 3
       port: 443 4
       repositoryPath: red-hat-codeready-workspaces-backups 5
       repositoryPasswordSecretRef: backup-encryption-password-secret 6
       awsAccessKeySecretRef: aws-user-credentials-secret 7

   1

   Must only contain one section (such as awss3).

   2

   Optional property that specifies the protocol to be used. https is the default value; http is a permitted value.

   3

   Optional property that specifies the S3 hostname. The default value is s3.amazonaws.com.

   4

   Optional property that specifies the port on which the backup server is running. The default value is 443.

   5

   Name of the bucket resource that stores the backup snapshots. You must manually create the bucket resource before configuring it here.

   6

   Secret created in step 1.

   7

   Secret created in step 2.

4. Optional: To configure multiple backup servers, create a separate CheBackupServerConfiguration custom object for each backup server.

Procedure

1. Create the CheClusterBackup custom object, which creates a backup snapshot:

   apiVersion: org.eclipse.che/v1
   kind: CheClusterBackup
   metadata:
     name: red-hat-codeready-workspaces-backup
     namespace: openshift-workspaces
   spec:
     backupServerConfigRef: backup-server-configuration 1
     useInternalBackupServer: false 2

   1

   Name of the CheBackupServerConfiguration custom object defining what backup server to use.

   2

   Configures the Operator through this custom resource to use the CodeReady Workspaces-managed internal backup server or an administrator-managed external backup server (SFTP, Amazon S3 or S3 API compatible storage, or REST).

   Tip

   If you intend to reuse a name for CheClusterBackup custom objects, first delete any existing custom object with the same name. To delete it on the command line, use oc:

   $ oc delete CheClusterBackup <name> -n openshift-workspaces

2. Read the status section of the CheClusterBackup custom object to verify the backup process, for example:

   status:
     message: 'Backup is in progress. Start time: <timestamp>' 1
     stage: Collecting CodeReady Workspaces installation data 2
     state: InProgress 3

   1

   Summary of the latest state of the backup process.

   2

   Current stage of the backup process in a human-readable format.

   3

   Backup process state: InProgress or Succeeded or Failed.

   The CodeReady Workspaces instance is backed up in a snapshot when state is Succeeded:

   status:
     ...
     message: >-
       Backup successfully finished at 2021-12-03 10:07:51.151090621 +0000 UTC
       m=+999.553272281 1
     snapshotId: 9f0adce2 2
     state: Succeeded 3

   1

   Summary of the latest state of the backup process.

   2

   ID of the created backup snapshot. This field appears only when state is Succeeded.

   3

   Backup process state: Succeeded (or Failed).

Procedure

1. Enter the crwctl server:backup command with the following arguments:

   $ crwctl server:backup \
   --repository-url=<repository_url> \ 1
   --repository-password=<repository_password> \ 2
   --aws-access-key-id=<aws-access-key-id> \ 3
   --aws-secret-access-key==<aws-secret-access-key> 4

   1

   Backup repository URL as an argument using the --repository-url or -r option. Syntax for the backup repository URL: s3:s3.amazonaws.com/<bucket_name>. For more details about repository URL syntax, see Restic Documentation.

   2

   Backup repository password as an argument using the --repository-password or -p option.

   3

   User’s AWS access key ID.

   4

   User’s AWS secret access key.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --repository-url and --repository-password options with the crwctl server:backup command just once and omit them onward when entering crwctl server:backup or crwctl server:restore.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Run the following command on a command line:

   $ crwctl server:backup \
   --backup-server-config-name=<name_of_CheBackupServerConfiguration> 1

   1

   This option points crwctl to a CheBackupServerConfiguration custom object. You can find the name of the CheBackupServerConfiguration custom object under metadata in the custom object.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --backup-server-config-name option with the crwctl server:backup command just once and omit this option onward when entering crwctl server:backup.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Create a Secret containing the repo-password key with a password:

   apiVersion: v1
   kind: Secret
   metadata:
     name: backup-encryption-password-secret
     namespace: eclipse-che
   type: Opaque
   stringData:
     repo-password: <password>

   (The CodeReady Workspaces Operator will set up this password for the backup repository that the CodeReady Workspaces Operator will create from this custom object on the backup server.)

   Warning

   The backup repository password is used to encrypt the backup data. If you lose this password, you will lose the backup data.

2. Optional: Create a Secret (for example, name: rest-server-auth-secret) containing the username and password keys for the REST server user credentials.

   Note

   Skip this step if not using the REST server user credentials.

3. Create the CheBackupServerConfiguration custom object:

   apiVersion: org.eclipse.che/v1
   kind: CheBackupServerConfiguration
   metadata:
     name: backup-server-configuration
     namespace: openshift-workspaces
   spec: 1
     rest:
       protocol: https 2
       hostname: <hostname> 3
       port: 1234 4
       repositoryPath: red-hat-codeready-workspaces-backups 5
       repositoryPasswordSecretRef: backup-encryption-password-secret 6
       credentialsSecretRef: rest-server-auth-secret 7

   1

   Must only contain one section (such as rest).

   2

   Optional property that specifies the protocol to be used: https is the default value; http is a permitted value.

   3

   Backup server hostname.

   4

   Optional property that specifies the port on which the backup server is running. The default value is 8000.

   5

   Path on the backup server where the backup snapshots are stored.

   6

   Secret created in step 1.

   7

   Optional property that specifies the Secret created in step 2.

4. Optional: To configure multiple backup servers, create a separate CheBackupServerConfiguration custom object for each backup server.

Procedure

1. Create the CheClusterBackup custom object, which creates a backup snapshot:

   apiVersion: org.eclipse.che/v1
   kind: CheClusterBackup
   metadata:
     name: red-hat-codeready-workspaces-backup
     namespace: openshift-workspaces
   spec:
     backupServerConfigRef: backup-server-configuration 1
     useInternalBackupServer: false 2

   1

   Name of the CheBackupServerConfiguration custom object defining what backup server to use.

   2

   Configures the Operator through this custom resource to use the CodeReady Workspaces-managed internal backup server or an administrator-managed external backup server (SFTP, Amazon S3 or S3 API compatible storage, or REST).

   Tip

   If you intend to reuse a name for CheClusterBackup custom objects, first delete any existing custom object with the same name. To delete it on the command line, use oc:

   $ oc delete CheClusterBackup <name> -n openshift-workspaces

2. Read the status section of the CheClusterBackup custom object to verify the backup process, for example:

   status:
     message: 'Backup is in progress. Start time: <timestamp>' 1
     stage: Collecting CodeReady Workspaces installation data 2
     state: InProgress 3

   1

   Summary of the latest state of the backup process.

   2

   Current stage of the backup process in a human-readable format.

   3

   Backup process state: InProgress or Succeeded or Failed.

   The CodeReady Workspaces instance is backed up in a snapshot when state is Succeeded:

   status:
     ...
     message: >-
       Backup successfully finished at 2021-12-03 10:07:51.151090621 +0000 UTC
       m=+999.553272281 1
     snapshotId: 9f0adce2 2
     state: Succeeded 3

   1

   Summary of the latest state of the backup process.

   2

   ID of the created backup snapshot. This field appears only when state is Succeeded.

   3

   Backup process state: Succeeded (or Failed).

Procedure

1. Enter the crwctl server:backup command with the following arguments:

   $ crwctl server:backup \
   --repository-url=<repository_url> \ 1
   --repository-password=<repository_password> \ 2
   --username=<username> \ 3
   --password=<password> 4

   1

   Backup repository URL as an argument using the --repository-url or -r option. Syntax for the backup repository URL: rest:http://<host>:8000/ or rest:https://<user>:<password>@<host>:8000/ or rest:https://<user>:<password>@<host>:8000/<repository_path_on_rest_server>/. For more details about repository URL syntax, see Restic Documentation.

   2

   Backup repository password as an argument using the --repository-password or -p option.

   3

   When authentication is required, the username for authenticating to the backup REST server as the --username=<username> argument.

   4

   When authentication is required, the password for authenticating to the backup REST server as the --password=<password> argument.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --repository-url and --repository-password options with the crwctl server:backup command just once and omit them onward when entering crwctl server:backup or crwctl server:restore.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Run the following command on a command line:

   $ crwctl server:backup \
   --backup-server-config-name=<name_of_CheBackupServerConfiguration> 1

   1

   This option points crwctl to a CheBackupServerConfiguration custom object. You can find the name of the CheBackupServerConfiguration custom object under metadata in the custom object.

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --backup-server-config-name option with the crwctl server:backup command just once and omit this option onward when entering crwctl server:backup.

2. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling backup...OK
     ✔ Waiting until backup process finishes...OK
   Backup snapshot ID: 9f0adce2
   Command server:backup has completed successfully in 00:10.

Procedure

1. Run the crwctl server:restore command with the following arguments:

   $ crwctl server:restore \
   --snapshot-id=latest \ 1
   --repository-url=<repository-url> \ 2
   --repository-password=<repository-password> 3

   1

   The latest snapshot.

   2

   The backup repository URL as an argument using the --repository-url or -r option. (Omit if using the internal backup server!)

   3

   The backup repository password as an argument using the --repository-password or -p option. (Omit if using the internal backup server!)

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --repository-url and --repository-password options with the crwctl server:backup command just once and omit them onward when entering crwctl server:backup or crwctl server:restore.

2. When prompted in the output, enter y to confirm:

   ...
    ✔ Snapshot:          ...
    Asking for restore confirmation: Do you want to proceed? [y/n]
   ...

   Tip

   To skip this question, add the --batch option to the server:restore command.

3. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling restore...OK
     ✔ Waiting until restore process finishes...OK
   Command server:restore has completed successfully in 05:59.

Procedure

1. Run the crwctl server:restore command with the backup snapshot ID specified using the --snapshot-id= option:

   $ crwctl server:restore \
   --snapshot-id=<snapshot-id> \ 1
   --repository-url=<repository-url> \ 2
   --repository-password=<repository-password> 3

   1

   The snapshot ID.

   2

   The backup repository URL as an argument using the --repository-url or -r option. (Omit if using the internal backup server!)

   3

   The backup repository password as an argument using the --repository-password or -p option. (Omit if using the internal backup server!)

   Tip

   Last used backup server information is stored in a Secret inside the CodeReady Workspaces cluster. When consistently using the same backup server, you can enter the --repository-url and --repository-password options with the crwctl server:backup command just once and omit them onward when entering crwctl server:backup or crwctl server:restore.

2. When prompted in the output, enter y to confirm:

   ...
    ✔ Snapshot:          ...
    Asking for restore confirmation: Do you want to proceed? [y/n]
   ...

   Tip

   To skip this question, add the --batch option to the server:restore command.

3. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling restore...OK
     ✔ Waiting until restore process finishes...OK
   Command server:restore has completed successfully in 05:59.

Procedure

1. Run the following command on a command line:

   $ crwctl server:restore \
   --backup-cr-name=<name_of_custom_object> 1

   1

   The name of the CheClusterBackup custom object.

2. When prompted in the output, enter y to confirm:

   ...
    Asking for restore confirmation: Do you want to proceed? [y/n]
   ...

   Tip

   To skip this question, add the --batch option to the server:restore command.

3. Verify the output of the entered command. For example:

   ...
     ✔ Scheduling restore...OK
     ✔ Waiting until restore process finishes...OK
   Command server:restore has completed successfully in 05:59.