10.5. Synchronizing versions in Multicloud Object Gateway bucket replication

Prerequisites

• A Multicloud Object Gateway (MCG) source bucket, which is created from an object bucket claim (OBC) and any MCG target bucket. For example, you can create the two buckets using OBCs using the MCG command line interface (CLI):

  • Create a source bucket using an OBC:

    $ mcg-cli obc create source-bucket --exact
    $ mcg-cli obc create target-bucket --exact

    where --exact is optional.

• Ensure that S3 client aliases with MCG credentials and endpoint are set up.

  $ NOOBAA_ACCESS_KEY=$(oc extract secret/noobaa-admin -n openshift-storage --keys=AWS_ACCESS_KEY_ID --to=- 2>/dev/null); \
  NOOBAA_SECRET_KEY=$(oc extract secret/noobaa-admin -n openshift-storage --keys=AWS_SECRET_ACCESS_KEY --to=- 2>/dev/null); \
  S3_ENDPOINT=https://$(oc get route s3 -n openshift-storage -o json | jq -r ".spec.host")

  $ alias common_s3='AWS_ACCESS_KEY_ID=$NOOBAA_ACCESS_KEY AWS_SECRET_ACCESS_KEY=$NOOBAA_SECRET_KEY aws --endpoint $S3_ENDPOINT --no-verify-ssl'; \
  alias s3_alias='common_s3 s3'; \
  alias s3api_alias='common_s3 s3api'

• Make sure to enable versioning on both the source and target bucket by using the put-bucket-versioning command in the AWS S3 client:

  $ s3api_alias put-bucket-versioning --bucket source-bucket --versioning-configuration Status=Enabled

  $ s3api_alias put-bucket-versioning --bucket target-bucket --versioning-configuration Status=Enabled

  Procedure

  1. Patch or edit a replication policy with sync_versions: true to the source bucket’s OBC:

     $ oc patch obc source-bucket -n openshift-storage --type=merge -p '{"spec": {"additionalConfig": {"replicationPolicy": "{\"rules\":[{\"rule_id\":\"replication-rule-a\",\"destination_bucket\":\"target-bucket\", \"sync_versions\": true}]}"}}}'

     Normal bucket replication replicates only the latest version, and the use of sync_versions add the functionality of replicating even the older versions, and in their original order.

Verification steps

• Verify the replication to the target bucket by adding a few versions under an object key and waiting for them to get replicated to the target bucket:

  $ echo 'version_a' | s3_alias cp - s3://source-bucket/versioned_obj.txt

  $ echo 'version_b' | s3_alias cp - s3://source-bucket/versioned_obj.txt

  $ echo 'version_c' | s3_alias cp - s3://source-bucket/versioned_obj.txt

  After a few minutes, compare the versions of the object on both the buckets:

  $ s3api_alias list-object-versions --bucket source-bucket --prefix versioned_obj.txt | jq -r ".Versions[].ETag"
  "aaabf266d38a8e995cef03c13ee9a7f1"
  "181d8a23f59939c0cddec1692e05cdf3"
  "ebc538cc6ffa04a39263f4b1be2f832f"

  $ s3api_alias list-object-versions --bucket target-bucket --prefix versioned_obj.txt | jq -r ".Versions[].ETag"
  "aaabf266d38a8e995cef03c13ee9a7f1"
  "181d8a23f59939c0cddec1692e05cdf3"
  "ebc538cc6ffa04a39263f4b1be2f832f"