Chapter 9. 3scale backup and restore

Procedure

1. Scale down the zync DeploymentConfig to 0 pods:

   oc scale dc zync --replicas=0
   oc scale dc zync-que --replicas=0

2. Copy the Zync database dump to the zync-database pod:

   oc cp ./zync-database-backup.gz $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq '.items[0].metadata.name' -r):/var/lib/pgsql/

3. Decompress the backup file:

   oc rsh $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq -r '.items[0].metadata.name') bash -c 'gzip -d ${HOME}/zync-database-backup.gz'

4. Restore the PostgreSQL DB backup file:

   oc rsh $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq -r '.items[0].metadata.name') bash -c 'psql -f ${HOME}/zync-database-backup'

5. Restore to the original count of replicas, by replacing ${ZYNC_REPLICAS} with the number of replicas, in the commands below:

   oc scale dc zync --replicas=${ZYNC_REPLICAS}
   oc scale dc zync-que --replicas=${ZYNC_REPLICAS}

Procedure

1. Store the number of replicas, by replacing ${DEPLOYMENT_NAME} with the name you defined when you created your 3scale deployment:

   ZYNC_SPEC=`oc get APIManager/${DEPLOYMENT_NAME} -o json | jq -r '.spec.zync'`

2. Scale down the zync DeploymentConfig to 0 pods:

   oc patch APIManager/${DEPLOYMENT_NAME} --type merge -p '{"spec": {"zync": {"appSpec": {"replicas": 0}, "queSpec": {"replicas": 0}}}}'

3. Copy the Zync database dump to the zync-database pod:

   oc cp ./zync-database-backup.gz $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq '.items[0].metadata.name' -r):/var/lib/pgsql/

4. Decompress the backup file:

   oc rsh $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq -r '.items[0].metadata.name') bash -c 'gzip -d ${HOME}/zync-database-backup.gz'

5. Restore the PostgreSQL DB backup file:

   oc rsh $(oc get pods -l 'deploymentConfig=zync-database' -o json | jq -r '.items[0].metadata.name') bash -c 'psql -f ${HOME}/zync-database-backup'

6. Restore to the original count of replicas:

   oc patch APIManager ${DEPLOYMENT_NAME} --type merge -p '{"spec": {"zync":'"${ZYNC_SPEC}"'}}'

Procedure

1. Edit the redis-config configmap:

   oc edit configmap redis-config

2. Comment SAVE commands in the redis-config configmap:

    #save 900 1
    #save 300 10
    #save 60 10000

3. Set appendonly to no in the redis-config configmap:

   appendonly no

4. Redeploy backend-redis to load the new configurations:

   oc rollout latest dc/backend-redis

5. Check the status of the rollout to ensure it has finished:

   oc rollout status dc/backend-redis

6. Rename the dump.rdb file:

   oc rsh $(oc get pods -l 'deploymentConfig=backend-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'mv ${HOME}/data/dump.rdb ${HOME}/data/dump.rdb-old'

7. Rename the appendonly.aof file:

   oc rsh $(oc get pods -l 'deploymentConfig=backend-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'mv ${HOME}/data/appendonly.aof ${HOME}/data/appendonly.aof-old'

8. Move the backup file to the POD:

   oc cp ./backend-redis-dump.rdb $(oc get pods -l 'deploymentConfig=backend-redis' -o json | jq '.items[0].metadata.name' -r):/var/lib/redis/data/dump.rdb

9. Redeploy backend-redis to load the backup:

   oc rollout latest dc/backend-redis

10. Check the status of the rollout to ensure it has finished:

    oc rollout status dc/backend-redis

11. Create the appendonly file:

    oc rsh $(oc get pods -l 'deploymentConfig=backend-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'redis-cli BGREWRITEAOF'

12. After a while, ensure that the AOF rewrite is complete:

    oc rsh $(oc get pods -l 'deploymentConfig=backend-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'redis-cli info' | grep aof_rewrite_in_progress

    • While aof_rewrite_in_progress = 1, the execution is in progress.
    • Check periodically until aof_rewrite_in_progress = 0. Zero indicates that the execution is complete.

13. Edit the redis-config configmap:

    oc edit configmap redis-config

14. Uncomment SAVE commands in the redis-config configmap:

     save 900 1
     save 300 10
     save 60 10000

15. Set appendonly to yes in the redis-config configmap:

    appendonly yes

16. Redeploy backend-redis to reload the default configurations:

    oc rollout latest dc/backend-redis

17. Check the status of the rollout to ensure it has finished:

    oc rollout status dc/backend-redis

Procedure

1. Edit the redis-config configmap:

   oc edit configmap redis-config

2. Comment SAVE commands in the redis-config configmap:

    #save 900 1
    #save 300 10
    #save 60 10000

3. Set appendonly to no in the redis-config configmap:

   appendonly no

4. Redeploy system-redis to load the new configurations:

   oc rollout latest dc/system-redis

5. Check the status of the rollout to ensure it has finished:

   oc rollout status dc/system-redis

6. Rename the dump.rdb file:

   oc rsh $(oc get pods -l 'deploymentConfig=system-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'mv ${HOME}/data/dump.rdb ${HOME}/data/dump.rdb-old'

7. Rename the appendonly.aof file:

   oc rsh $(oc get pods -l 'deploymentConfig=system-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'mv ${HOME}/data/appendonly.aof ${HOME}/data/appendonly.aof-old'

8. Move the Backup file to the POD:

   oc cp ./system-redis-dump.rdb $(oc get pods -l 'deploymentConfig=system-redis' -o json | jq '.items[0].metadata.name' -r):/var/lib/redis/data/dump.rdb

9. Redeploy system-redis to load the backup:

   oc rollout latest dc/system-redis

10. Check the status of the rollout to ensure it has finished:

    oc rollout status dc/system-redis

11. Create the appendonly file:

    oc rsh $(oc get pods -l 'deploymentConfig=system-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'redis-cli BGREWRITEAOF'

12. After a while, ensure that the AOF rewrite is complete:

    oc rsh $(oc get pods -l 'deploymentConfig=system-redis' -o json | jq '.items[0].metadata.name' -r) bash -c 'redis-cli info' | grep aof_rewrite_in_progress

    • While aof_rewrite_in_progress = 1, the execution is in progress.
    • Check periodically until aof_rewrite_in_progress = 0. Zero indicates that the execution is complete.

13. Edit the redis-config configmap:

    oc edit configmap redis-config

14. Uncomment SAVE commands in the redis-config configmap:

     save 900 1
     save 300 10
     save 60 10000

15. Set appendonly to yes in the redis-config configmap:

    appendonly yes

16. Redeploy system-redis to reload the default configurations:

    oc rollout latest dc/system-redis

17. Check the status of the rollout to ensure it has finished:

    oc rollout status dc/system-redis

1. Restore to the latest version of system-sphinx:

   oc rollout latest dc/system-sphinx

2. Check the status of the rollout to ensure it has finished:

   oc rollout status dc/system-sphinx

1. Force Zync to recreate missing OpenShift routes:

   oc rsh $(oc get pods -l 'deploymentConfig=system-sidekiq' -o json | jq '.items[0].metadata.name' -r) bash -c 'bundle exec rake zync:resync:domains'