Chapter 9. Administration

Procedure

1. Calculate the recommended number of shards:

   number of objects expected in a bucket / 100,000

   Note

   The maximum number of bucket index shards currently supported is 65,521.

2. Set the rgw_override_bucket_index_max_shards option accordingly:

   Syntax

   ceph config set client.rgw rgw_override_bucket_index_max_shards VALUE

   Replace VALUE with the recommended number of shards calculated:

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_override_bucket_index_max_shards 12

   • To configure bucket index resharding for all instances of the Ceph Object Gateway, set the rgw_override_bucket_index_max_shards parameter with the global option.
   • To configure bucket index resharding only for a particular instance of the Ceph Object Gateway, add rgw_override_bucket_index_max_shards parameter under the instance.

3. Restart the Ceph Object Gateways on all nodes in the cluster to take effect:

   Syntax

   ceph orch restart SERVICE_TYPE

   Example

   [ceph: root#host01 /]# ceph orch restart rgw

Procedure

1. Calculate the recommended number of shards:

   number of objects expected in a bucket / 100,000

   Note

   The maximum number of bucket index shards currently supported is 65,521.

2. Extract the zone group configuration to the zonegroup.json file:

   Example

   [ceph: root@host01 /]# radosgw-admin zonegroup get > zonegroup.json

3. In the zonegroup.json file, set the bucket_index_max_shards parameter for each named zone:

   Syntax

   bucket_index_max_shards = VALUE

   Replace VALUE with the recommended number of shards calculated:

   Example

   bucket_index_max_shards = 12

4. Reset the zone group:

   Example

   [ceph: root@host01 /]# radosgw-admin zonegroup set < zonegroup.json

5. Update the period:

   Example

   [ceph: root@host01 /]# radosgw-admin period update --commit

6. Check if resharding is complete:

   Syntax

   radosgw-admin reshard status --bucket BUCKET_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin reshard status --bucket data

Procedure

1. Set the rgw_dynamic_resharding parameter is set to true.

   Example

   [ceph: root@host01 /]# radosgw-admin period get

2. Optional: Customize Ceph configuration using the following command:

   Syntax

   ceph config set client.rgw OPTION VALUE

   Replace OPTION with the following options:

   • rgw_reshard_num_logs: The number of shards for the resharding log. The default value is 16.
   • rgw_reshard_bucket_lock_duration: The duration of the lock on a bucket during resharding. The default value is 360 seconds.
   • rgw_dynamic_resharding: Enables or disables dynamic resharding. The default value is true.
   • rgw_max_objs_per_shard: The maximum number of objects per shard. The default value is 100000 objects per shard.
   • rgw_reshard_thread_interval: The maximum time between rounds of reshard thread processing. The default value is 600 seconds.

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_reshard_num_logs 23

3. Add a bucket to the resharding queue:

   Syntax

   radosgw-admin reshard add --bucket BUCKET --num-shards NUMBER

   Example

   [ceph: root@host01 /]# radosgw-admin reshard add --bucket data --num-shards 10

4. List the resharding queue:

   Example

   [ceph: root@host01 /]# radosgw-admin reshard list

5. Check the bucket log generations and shards:

   Example

   [ceph: root@host01 /]# radosgw-admin bucket layout --bucket data
   {
       "layout": {
           "resharding": "None",
           "current_index": {
               "gen": 1,
               "layout": {
                   "type": "Normal",
                   "normal": {
                       "num_shards": 23,
                       "hash_type": "Mod"
                   }
               }
           },
           "logs": [
               {
                   "gen": 0,
                   "layout": {
                       "type": "InIndex",
                       "in_index": {
                           "gen": 0,
                           "layout": {
                               "num_shards": 11,
                               "hash_type": "Mod"
                           }
                       }
                   }
               },
               {
                   "gen": 1,
                   "layout": {
                       "type": "InIndex",
                       "in_index": {
                           "gen": 1,
                           "layout": {
                               "num_shards": 23,
                               "hash_type": "Mod"
                           }
                       }
                   }
               }
           ]
       }
   }

6. Check bucket resharding status:

   Syntax

   radosgw-admin reshard status --bucket BUCKET

   Example

   [ceph: root@host01 /]# radosgw-admin reshard status --bucket data

7. Process entries on the resharding queue immediately:

   [ceph: root@host01 /]# radosgw-admin reshard process

8. Cancel pending bucket resharding:

   Warning

   You can only cancel pending resharding operations. Do not cancel ongoing resharding operations.

   Syntax

   radosgw-admin reshard cancel --bucket BUCKET

   Example

   [ceph: root@host01 /]# radosgw-admin reshard cancel --bucket data

Procedure

1. Check if resharding is enabled on the zonegroup:

   Example

   [ceph: root@host01 /]# radosgw-admin sync status

   If zonegroup features enabled is not enabled for resharding on the zonegroup, then continue with the procedure.

2. Enable the resharding feature on all the zonegroups in the multi-site configuration where Ceph Object Gateway is installed:

   Syntax

   radosgw-admin zonegroup modify --rgw-zonegroup=ZONEGROUP_NAME --enable-feature=resharding

   Example

   [ceph: root@host01 /]# radosgw-admin zonegroup modify --rgw-zonegroup=us --enable-feature=resharding

3. Update the period and commit:

   Example

   [ceph: root@host01 /]# radosgw-admin period update --commit

4. Enable the resharding feature on all the zones in the multi-site configuration where Ceph Object Gateway is installed:

   Syntax

   radosgw-admin zone modify --rgw-zone=ZONE_NAME --enable-feature=resharding

   Example

   [ceph: root@host01 /]# radosgw-admin zone modify --rgw-zone=us-east --enable-feature=resharding

5. Update the period and commit:

   Example

   [ceph: root@host01 /]# radosgw-admin period update --commit

6. Verify the resharding feature is enabled on the zones and zonegroups. You can see that each zone lists its supported_features and the zonegroups lists its enabled_features

   Example

   [ceph: root@host01 /]# radosgw-admin period get
   
   "zones": [
                       {
                           "id": "505b48db-6de0-45d5-8208-8c98f7b1278d",
                           "name": "us_east",
                           "endpoints": [
                               "http://10.0.208.11:8080"
                           ],
                           "log_meta": "false",
                           "log_data": "true",
                           "bucket_index_max_shards": 11,
                           "read_only": "false",
                           "tier_type": "",
                           "sync_from_all": "true",
                           "sync_from": [],
                           "redirect_zone": "",
                           "supported_features": [
                               "resharding"
                           ]
                   "default_placement": "default-placement",
                   "realm_id": "26cf6f23-c3a0-4d57-aae4-9b0010ee55cc",
                   "sync_policy": {
                       "groups": []
                   },
                   "enabled_features": [
                       "resharding"
                   ]

7. Check the sync status:

   Example

   [ceph: root@host01 /]# radosgw-admin sync status
             realm 26cf6f23-c3a0-4d57-aae4-9b0010ee55cc (usa)
         zonegroup 33a17718-6c77-493e-99fe-048d3110a06e (us)
              zone 505b48db-6de0-45d5-8208-8c98f7b1278d (us_east)
   zonegroup features enabled: resharding

   In this example, the resharding feature is enabled for the us zonegroup.

8. Optional: You can disable the resharding feature for the zonegroups:

   Important

   To disable resharding on any singular zone, set the rgw_dynamic_resharding configuration option to false on that specific zone.

   1. Disable the feature on all the zonegroups in the multi-site where Ceph Object Gateway is installed:

      Syntax

      radosgw-admin zonegroup modify --rgw-zonegroup=ZONEGROUP_NAME --disable-feature=resharding

      Example

      [ceph: root@host01 /]# radosgw-admin zonegroup modify --rgw-zonegroup=us --disable-feature=resharding

   2. Update the period and commit:

      Example

      [ceph: root@host01 /]# radosgw-admin period update --commit

Procedure

1. Back up the original bucket index:

   Syntax

   radosgw-admin bi list --bucket=BUCKET > BUCKET.list.backup

   Example

   [ceph: root@host01 /]# radosgw-admin bi list --bucket=data > data.list.backup

2. Reshard the bucket index:

   Syntax

   radosgw-admin bucket reshard --bucket=BUCKET --num-shards=NUMBER

   Example

   [ceph: root@host01 /]# radosgw-admin bucket reshard --bucket=data --num-shards=100

Procedure

1. List stale instances:

   [ceph: root@host01 /]# radosgw-admin reshard stale-instances list

2. Clean the stale instances of the bucket entries:

   [ceph: root@host01 /]# radosgw-admin reshard stale-instances rm

Procedure

1. Rename a user:

   Syntax

   radosgw-admin user rename --uid=CURRENT_USER_NAME --new-uid=NEW_USER_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin user rename --uid=user1 --new-uid=user2
   
   {
       "user_id": "user2",
       "display_name": "user 2",
       "email": "",
       "suspended": 0,
       "max_buckets": 1000,
       "auid": 0,
       "subusers": [],
       "keys": [
           {
               "user": "user2",
               "access_key": "59EKHI6AI9F8WOW8JQZJ",
               "secret_key": "XH0uY3rKCUcuL73X0ftjXbZqUbk0cavD11rD8MsA"
           }
       ],
       "swift_keys": [],
       "caps": [],
       "op_mask": "read, write, delete",
       "default_placement": "",
       "placement_tags": [],
       "bucket_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "user_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "temp_url_keys": [],
       "type": "rgw"
   }

   If a user is inside a tenant, specify both the user name and the tenant:

   Syntax

   radosgw-admin user rename --uid USER_NAME --new-uid NEW_USER_NAME --tenant TENANT

   Example

   [ceph: root@host01 /]# radosgw-admin user rename --uid=test$user1 --new-uid=test$user2 --tenant test
   
   1000 objects processed in tvtester1. Next marker 80_tVtester1_99
   2000 objects processed in tvtester1. Next marker 64_tVtester1_44
   3000 objects processed in tvtester1. Next marker 48_tVtester1_28
   4000 objects processed in tvtester1. Next marker 2_tVtester1_74
   5000 objects processed in tvtester1. Next marker 14_tVtester1_53
   6000 objects processed in tvtester1. Next marker 87_tVtester1_61
   7000 objects processed in tvtester1. Next marker 6_tVtester1_57
   8000 objects processed in tvtester1. Next marker 52_tVtester1_91
   9000 objects processed in tvtester1. Next marker 34_tVtester1_74
   9900 objects processed in tvtester1. Next marker 9_tVtester1_95
   1000 objects processed in tvtester2. Next marker 82_tVtester2_93
   2000 objects processed in tvtester2. Next marker 64_tVtester2_9
   3000 objects processed in tvtester2. Next marker 48_tVtester2_22
   4000 objects processed in tvtester2. Next marker 32_tVtester2_42
   5000 objects processed in tvtester2. Next marker 16_tVtester2_36
   6000 objects processed in tvtester2. Next marker 89_tVtester2_46
   7000 objects processed in tvtester2. Next marker 70_tVtester2_78
   8000 objects processed in tvtester2. Next marker 51_tVtester2_41
   9000 objects processed in tvtester2. Next marker 33_tVtester2_32
   9900 objects processed in tvtester2. Next marker 9_tVtester2_83
   {
       "user_id": "test$user2",
       "display_name": "User 2",
       "email": "",
       "suspended": 0,
       "max_buckets": 1000,
       "auid": 0,
       "subusers": [],
       "keys": [
           {
               "user": "test$user2",
               "access_key": "user2",
               "secret_key": "123456789"
           }
       ],
       "swift_keys": [],
       "caps": [],
       "op_mask": "read, write, delete",
       "default_placement": "",
       "placement_tags": [],
       "bucket_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "user_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "temp_url_keys": [],
       "type": "rgw"
   }

2. Verify that the user has been renamed successfully:

   Syntax

   radosgw-admin user info --uid=NEW_USER_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin user info --uid=user2

   If a user is inside a tenant, use the TENANT$USER_NAME format:

   Syntax

   radosgw-admin user info --uid= TENANT$USER_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin user info --uid=test$user2

1. Find the access key for the specific user:

   Example

   [root@host01 ~]# radosgw-admin user info --uid=johndoe

   The access key is the "access_key" value in the output:

   Example

   [root@host01 ~]# radosgw-admin user info --uid=johndoe
   {
       "user_id": "johndoe",
       ...
       "keys": [
           {
               "user": "johndoe",
               "access_key": "0555b35654ad1656d804",
               "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
           }
       ],
       ...
   }

2. Specify the user ID and the access key from the previous step to remove the access key:

   Syntax

   radosgw-admin key rm --uid=USER_ID --access-key ACCESS_KEY

   Example

   [root@host01 ~]# radosgw-admin key rm --uid=johndoe --access-key 0555b35654ad1656d804

Procedure

1. Delete the policy attached to the role:

   Syntax

   radosgw-admin role policy delete --role-name=ROLE_NAME --policy-name=POLICY_NAME

   Example

   [root@host01 ~]# radosgw-admin role policy delete --role-name=S3Access1 --policy-name=Policy1

2. Delete the role:

   Syntax

   radosgw-admin role delete --role-name=ROLE_NAME

   Example

   [root@host01 ~]# radosgw-admin role delete --role-name=S3Access1

Procedure

1. List the buckets:

   Example

   [ceph: root@host01 /]# radosgw-admin bucket list
   [
       "34150b2e9174475db8e191c188e920f6/swcontainer",
       "s3bucket1",
       "34150b2e9174475db8e191c188e920f6/swimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/ec2container",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten1",
       "c278edd68cfb4705bb3e07837c7ad1a8/demo-ct",
       "c278edd68cfb4705bb3e07837c7ad1a8/demopostup",
       "34150b2e9174475db8e191c188e920f6/postimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten2",
       "c278edd68cfb4705bb3e07837c7ad1a8/postupsw"
   ]

2. Rename the bucket:

   Syntax

   radosgw-admin bucket link --bucket=ORIGINAL_NAME --bucket-new-name=NEW_NAME --uid=USER_ID

   Example

   [ceph: root@host01 /]# radosgw-admin bucket link --bucket=s3bucket1 --bucket-new-name=s3newb --uid=testuser

   If the bucket is inside a tenant, specify the tenant as well:

   Syntax

   radosgw-admin bucket link --bucket=tenant/ORIGINAL_NAME --bucket-new-name=NEW_NAME --uid=TENANT$USER_ID

   Example

   [ceph: root@host01 /]# radosgw-admin bucket link --bucket=test/s3bucket1 --bucket-new-name=s3newb --uid=test$testuser

3. Verify the bucket was renamed:

   Example

   [ceph: root@host01 /]# radosgw-admin bucket list
   [
       "34150b2e9174475db8e191c188e920f6/swcontainer",
       "34150b2e9174475db8e191c188e920f6/swimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/ec2container",
       "s3newb",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten1",
       "c278edd68cfb4705bb3e07837c7ad1a8/demo-ct",
       "c278edd68cfb4705bb3e07837c7ad1a8/demopostup",
       "34150b2e9174475db8e191c188e920f6/postimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten2",
       "c278edd68cfb4705bb3e07837c7ad1a8/postupsw"
   ]

Procedure

1. List the buckets.

   Example

   [ceph: root@host01 /]# radosgw-admin bucket list
   [
       "34150b2e9174475db8e191c188e920f6/swcontainer",
       "s3bucket1",
       "34150b2e9174475db8e191c188e920f6/swimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/ec2container",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten1",
       "c278edd68cfb4705bb3e07837c7ad1a8/demo-ct",
       "c278edd68cfb4705bb3e07837c7ad1a8/demopostup",
       "34150b2e9174475db8e191c188e920f6/postimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten2",
       "c278edd68cfb4705bb3e07837c7ad1a8/postupsw"
   ]

2. Remove the bucket.

   Syntax

   radosgw-admin bucket rm --bucket=BUCKET_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin bucket rm --bucket=s3bucket1

3. If the bucket has objects, then run the following command:

   Syntax

   radosgw-admin bucket rm --bucket=BUCKET --purge-objects --bypass-gc

   Example

   [ceph: root@host01 /]# radosgw-admin bucket rm --bucket=s3bucket1 --purge-objects --bypass-gc

   The --purge-objects option purges the objects and --bypass-gc option triggers deletion of objects without the garbage collector to make the process more efficient.

4. Verify the bucket was removed.

   Example

   [ceph: root@host01 /]# radosgw-admin bucket list
   [
       "34150b2e9174475db8e191c188e920f6/swcontainer",
       "34150b2e9174475db8e191c188e920f6/swimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/ec2container",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten1",
       "c278edd68cfb4705bb3e07837c7ad1a8/demo-ct",
       "c278edd68cfb4705bb3e07837c7ad1a8/demopostup",
       "34150b2e9174475db8e191c188e920f6/postimpfalse",
       "c278edd68cfb4705bb3e07837c7ad1a8/demoten2",
       "c278edd68cfb4705bb3e07837c7ad1a8/postupsw"
   ]

Procedure

1. Create a JSON file for lifecycle configuration:

   Example

   [user@client ~]$ vi lifecycle.json

2. Add the specific lifecycle configuration rules in the file:

   Example

   {
   	"Rules": [
           {
   		    "Filter": {
   			    "Prefix": "images/"
   		    },
   		    "Status": "Enabled",
   		    "Expiration": {
   			    "Days": 1
   		    },
   		    "ID": "ImageExpiration"
   	    }
       ]
   }

   The lifecycle configuration example expires objects in the images directory after 1 day.

3. Set the lifecycle configuration on the bucket:

   Syntax

   aws --endpoint-url=RADOSGW_ENDPOINT_URL:PORT s3api put-bucket-lifecycle-configuration --bucket BUCKET_NAME --lifecycle-configuration file://PATH_TO_LIFECYCLE_CONFIGURATION_FILE/LIFECYCLE_CONFIGURATION_FILE.json

   Example

   [user@client ~]$ aws --endpoint-url=http://host01:80 s3api put-bucket-lifecycle-configuration --bucket testbucket --lifecycle-configuration file://lifecycle.json

   In this example, the lifecycle.json file exists in the current directory.

Procedure

1. Create a JSON file for the lifecycle configuration:

   Example

   [user@client ~]$ vi lifecycle.json

2. Add the specific lifecycle configuration rules to the file:

   Example

   {
   	"Rules": [
           {
   		    "Filter": {
   			    "Prefix": "images/"
   		    },
   		    "Status": "Enabled",
   		    "Expiration": {
   			    "Days": 1
   		    },
   		    "ID": "ImageExpiration"
   	    },
   		{
   			"Filter": {
   				"Prefix": "docs/"
   			},
   			"Status": "Enabled",
   			"Expiration": {
   				"Days": 30
   			},
   			"ID": "DocsExpiration"
   		}
   	]
   }

3. Update the lifecycle configuration on the bucket:

   Syntax

   aws --endpoint-url=RADOSGW_ENDPOINT_URL:PORT s3api put-bucket-lifecycle-configuration --bucket BUCKET_NAME --lifecycle-configuration file://PATH_TO_LIFECYCLE_CONFIGURATION_FILE/LIFECYCLE_CONFIGURATION_FILE.json

   Example

   [user@client ~]$ aws --endpoint-url=http://host01:80 s3api put-bucket-lifecycle-configuration --bucket testbucket --lifecycle-configuration file://lifecycle.json

Procedure

1. Log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. List bucket lifecycle progress:

   Example

   [ceph: root@host01 /]# radosgw-admin lc list
   
   [
      {
            “bucket”: “:testbucket:8b63d584-9ea1-4cf3-8443-a6a15beca943.54187.1”,
            “started”: “Thu, 01 Jan 1970 00:00:00 GMT”,
            “status” : “UNINITIAL”
      },
      {
            “bucket”: “:testbucket1:8b635499-9e41-4cf3-8443-a6a15345943.54187.2”,
            “started”: “Thu, 01 Jan 1970 00:00:00 GMT”,
            “status” : “UNINITIAL”
      }
   ]

   The bucket lifecycle processing status can be one of the following:

   • UNINITIAL - The process has not run yet.
   • PROCESSING - The process is currently running.
   • COMPLETE - The process has completed.

3. Optional: You can manually process bucket lifecycle policies:

   1. Process the lifecycle policy for a single bucket:

      Syntax

      radosgw-admin lc process --bucket=BUCKET_NAME

      Example

      [ceph: root@host01 /]# radosgw-admin lc process --bucket=testbucket1

   2. Process all bucket lifecycle policies immediately:

      Example

      [ceph: root@host01 /]# radosgw-admin lc process

Procedure

1. Log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. Set the lifecycle expiration time:

   Syntax

   ceph config set client.rgw rgw_lifecycle_work_time %D:%D-%D:%D

   Replace %d:%d-%d:%d with start_hour:start_minute-end_hour:end_minute.

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_lifecycle_work_time 06:00-08:00

Procedure

1. Create a new data pool:

   Syntax

   ceph osd pool create POOL_NAME

   Example

   [ceph: root@host01 /]# ceph osd pool create test.hot.data

2. Add a new storage class:

   Syntax

   radosgw-admin zonegroup placement add  --rgw-zonegroup default --placement-id PLACEMENT_TARGET --storage-class STORAGE_CLASS

   Example

   [ceph: root@host01 /]# radosgw-admin zonegroup placement add  --rgw-zonegroup default --placement-id default-placement --storage-class hot.test
   {
           "key": "default-placement",
           "val": {
               "name": "default-placement",
               "tags": [],
               "storage_classes": [
                   "STANDARD",
                   "hot.test"
               ]
           }
       }

3. Provide the zone placement information for the new storage class:

   Syntax

   radosgw-admin zone placement add --rgw-zone default --placement-id PLACEMENT_TARGET --storage-class STORAGE_CLASS --data-pool DATA_POOL

   Example

   [ceph: root@host01 /]# radosgw-admin zone placement add --rgw-zone default --placement-id default-placement --storage-class hot.test --data-pool test.hot.data
   {
              "key": "default-placement",
              "val": {
                  "index_pool": "test_zone.rgw.buckets.index",
                  "storage_classes": {
                      "STANDARD": {
                          "data_pool": "test.hot.data"
                      },
                      "hot.test": {
                          "data_pool": "test.hot.data",
                     }
                  },
                  "data_extra_pool": "",
                  "index_type": 0
              }

   Note

   Consider setting the compression_type when creating cold or archival data storage pools with write once.

4. Enable the rgw application on the data pool:

   Syntax

   ceph osd pool application enable POOL_NAME rgw

   Example

   [ceph: root@host01 /]# ceph osd pool application enable test.hot.data rgw
   enabled application 'rgw' on pool 'test.hot.data'

5. Restart all the rgw daemons.

6. Create a bucket:

   Example

   [ceph: root@host01 /]# aws s3api create-bucket --bucket testbucket10 --create-bucket-configuration LocationConstraint=default:default-placement --endpoint-url http://10.0.0.80:8080

7. Add the object:

   Example

   [ceph: root@host01 /]# aws --endpoint=http://10.0.0.80:8080 s3api put-object --bucket testbucket10  --key compliance-upload --body /root/test2.txt

8. Create a second data pool:

   Syntax

   ceph osd pool create POOL_NAME

   Example

   [ceph: root@host01 /]# ceph osd pool create test.cold.data

9. Add a new storage class:

   Syntax

   radosgw-admin zonegroup placement add  --rgw-zonegroup default --placement-id PLACEMENT_TARGET --storage-class STORAGE_CLASS

   Example

   [ceph: root@host01 /]# radosgw-admin zonegroup placement add  --rgw-zonegroup default --placement-id default-placement --storage-class cold.test
   {
           "key": "default-placement",
           "val": {
               "name": "default-placement",
               "tags": [],
               "storage_classes": [
                   "STANDARD",
                   "cold.test"
               ]
           }
       }

10. Provide the zone placement information for the new storage class:

    Syntax

    radosgw-admin zone placement add --rgw-zone default --placement-id PLACEMENT_TARGET --storage-class STORAGE_CLASS --data-pool DATA_POOL

    Example

    [ceph: root@host01 /]# radosgw-admin zone placement add --rgw-zone default --placement-id default-placement --storage-class cold.test --data-pool test.cold.data

11. Enable rgw application on the data pool:

    Syntax

    ceph osd pool application enable POOL_NAME rgw

    Example

    [ceph: root@host01 /]# ceph osd pool application enable test.cold.data rgw
    enabled application 'rgw' on pool 'test.cold.data'

12. Restart all the rgw daemons.

13. To view the zone group configuration, run the following command:

    Syntax

    radosgw-admin zonegroup get
    {
        "id": "3019de59-ddde-4c5c-b532-7cdd29de09a1",
        "name": "default",
        "api_name": "default",
        "is_master": "true",
        "endpoints": [],
        "hostnames": [],
        "hostnames_s3website": [],
        "master_zone": "adacbe1b-02b4-41b8-b11d-0d505b442ed4",
        "zones": [
            {
                "id": "adacbe1b-02b4-41b8-b11d-0d505b442ed4",
                "name": "default",
                "endpoints": [],
                "log_meta": "false",
                "log_data": "false",
                "bucket_index_max_shards": 11,
                "read_only": "false",
                "tier_type": "",
                "sync_from_all": "true",
                "sync_from": [],
                "redirect_zone": ""
            }
        ],
        "placement_targets": [
            {
                "name": "default-placement",
                "tags": [],
                "storage_classes": [
                    "hot.test",
                    "cold.test",
                    "STANDARD"
                ]
            }
        ],
        "default_placement": "default-placement",
        "realm_id": "",
        "sync_policy": {
            "groups": []
        }
    }

14. To view the zone configuration, run the following command:

    Syntax

    radosgw-admin zone get
    {
        "id": "adacbe1b-02b4-41b8-b11d-0d505b442ed4",
        "name": "default",
        "domain_root": "default.rgw.meta:root",
        "control_pool": "default.rgw.control",
        "gc_pool": "default.rgw.log:gc",
        "lc_pool": "default.rgw.log:lc",
        "log_pool": "default.rgw.log",
        "intent_log_pool": "default.rgw.log:intent",
        "usage_log_pool": "default.rgw.log:usage",
        "roles_pool": "default.rgw.meta:roles",
        "reshard_pool": "default.rgw.log:reshard",
        "user_keys_pool": "default.rgw.meta:users.keys",
        "user_email_pool": "default.rgw.meta:users.email",
        "user_swift_pool": "default.rgw.meta:users.swift",
        "user_uid_pool": "default.rgw.meta:users.uid",
        "otp_pool": "default.rgw.otp",
        "system_key": {
            "access_key": "",
            "secret_key": ""
        },
        "placement_pools": [
            {
                "key": "default-placement",
                "val": {
                    "index_pool": "default.rgw.buckets.index",
                    "storage_classes": {
                        "cold.test": {
                            "data_pool": "test.cold.data"
                        },
                        "hot.test": {
                            "data_pool": "test.hot.data"
                        },
                        "STANDARD": {
                            "data_pool": "default.rgw.buckets.data"
                        }
                    },
                    "data_extra_pool": "default.rgw.buckets.non-ec",
                    "index_type": 0
                }
            }
        ],
        "realm_id": "",
        "notif_pool": "default.rgw.log:notif"
    }

15. Create a bucket:

    Example

    [ceph: root@host01 /]# aws s3api create-bucket --bucket testbucket10 --create-bucket-configuration LocationConstraint=default:default-placement --endpoint-url http://10.0.0.80:8080

16. List the objects prior to transition:

    Example

    [ceph: root@host01 /]# radosgw-admin bucket list --bucket testbucket10
    
            {
                "ETag": "\"211599863395c832a3dfcba92c6a3b90\"",
                "Size": 540,
                "StorageClass": "STANDARD",
                "Key": "obj1",
                "VersionId": "W95teRsXPSJI4YWJwwSG30KxSCzSgk-",
                "IsLatest": true,
                "LastModified": "2023-11-23T10:38:07.214Z",
                "Owner": {
                    "DisplayName": "test-user",
                    "ID": "test-user"
                }
            }

17. Create a JSON file for lifecycle configuration:

    Example

    [ceph: root@host01 /]# vi lifecycle.json

18. Add the specific lifecycle configuration rule in the file:

    Example

    {
        "Rules": [
            {
                "Filter": {
                    "Prefix": ""
                },
                "Status": "Enabled",
                "Transitions": [
                    {
                        "Days": 5,
                        "StorageClass": "hot.test"
                    },
     {
                        "Days": 20,
                        "StorageClass": "cold.test"
                    }
                ],
                "Expiration": {
                    "Days": 365
                },
                "ID": "double transition and expiration"
            }
        ]
    }

    The lifecycle configuration example shows an object that will transition from the default STANDARD storage class to the hot.test storage class after 5 days, again transitions after 20 days to the cold.test storage class, and finally expires after 365 days in the cold.test storage class.

19. Set the lifecycle configuration on the bucket:

    Example

    [ceph: root@host01 /]# aws s3api put-bucket-lifecycle-configuration --bucket testbucket10 --lifecycle-configuration file://lifecycle.json

20. Retrieve the lifecycle configuration on the bucket:

    Example

    [ceph: root@host01 /]# aws s3api get-bucket-lifecycle-configuration --bucket testbucke10
    {
        "Rules": [
            {
                "Expiration": {
                    "Days": 365
                },
                "ID": "double transition and expiration",
                "Prefix": "",
                "Status": "Enabled",
                "Transitions": [
                    {
                        "Days": 20,
                        "StorageClass": "cold.test"
                    },
                    {
                        "Days": 5,
                        "StorageClass": "hot.test"
                    }
                ]
            }
        ]
    }

21. Verify that the object is transitioned to the given storage class:

    Example

    [ceph: root@host01 /]# radosgw-admin bucket list --bucket testbucket10
    
            {
                "ETag": "\"211599863395c832a3dfcba92c6a3b90\"",
                "Size": 540,
                "StorageClass": "cold.test",
                "Key": "obj1",
                "VersionId": "W95teRsXPSJI4YWJwwSG30KxSCzSgk-",
                "IsLatest": true,
                "LastModified": "2023-11-23T10:38:07.214Z",
                "Owner": {
                    "DisplayName": "test-user",
                    "ID": "test-user"
                }
            }

Procedure

1. Create a bucket with object lock enabled:

   Syntax

   aws --endpoint=http://RGW_PORT:8080 s3api create-bucket --bucket BUCKET_NAME --object-lock-enabled-for-bucket

   Example

   [root@rgw-2 ~]# aws --endpoint=http://rgw.ceph.com:8080 s3api create-bucket --bucket worm-bucket --object-lock-enabled-for-bucket

2. Set a retention period for the bucket:

   Syntax

   aws --endpoint=http://RGW_PORT:8080 s3api put-object-lock-configuration --bucket BUCKET_NAME --object-lock-configuration '{ "ObjectLockEnabled": "Enabled", "Rule": { "DefaultRetention": { "Mode": "RETENTION_MODE", "Days": NUMBER_OF_DAYS }}}'

   Example

   [root@rgw-2 ~]# aws --endpoint=http://rgw.ceph.com:8080 s3api put-object-lock-configuration --bucket worm-bucket --object-lock-configuration '{ "ObjectLockEnabled": "Enabled", "Rule": { "DefaultRetention": { "Mode": "COMPLIANCE", "Days": 10 }}}'

   Note

   You can choose either the GOVERNANCE or COMPLIANCE mode for the RETENTION_MODE in S3 object lock, to apply different levels of protection to any object version that is protected by object lock.

   In GOVERNANCE mode, users cannot overwrite or delete an object version or alter its lock settings unless they have special permissions.

   In COMPLIANCE mode, a protected object version cannot be overwritten or deleted by any user, including the root user in your AWS account. When an object is locked in COMPLIANCE mode, its RETENTION_MODE cannot be changed, and its retention period cannot be shortened. COMPLIANCE mode helps ensure that an object version cannot be overwritten or deleted for the duration of the period.

3. Put the object into the bucket with a retention time set:

   Syntax

   aws --endpoint=http://RGW_PORT:8080 s3api put-object --bucket BUCKET_NAME --object-lock-mode RETENTION_MODE --object-lock-retain-until-date "DATE" --key compliance-upload --body TEST_FILE

   Example

   [root@rgw-2 ~]# aws --endpoint=http://rgw.ceph.com:8080 s3api put-object --bucket worm-bucket --object-lock-mode COMPLIANCE --object-lock-retain-until-date "2022-05-31" --key compliance-upload --body test.dd
   {
       "ETag": "\"d560ea5652951637ba9c594d8e6ea8c1\"",
       "VersionId": "Nhhk5kRS6Yp6dZXVWpZZdRcpSpBKToD"
   }

4. Upload a new object using the same key:

   Syntax

   aws --endpoint=http://RGW_PORT:8080 s3api put-object --bucket BUCKET_NAME --object-lock-mode RETENTION_MODE --object-lock-retain-until-date "DATE" --key compliance-upload --body PATH

   Example

   [root@rgw-2 ~]# aws --endpoint=http://rgw.ceph.com:8080 s3api put-object --bucket worm-bucket --object-lock-mode COMPLIANCE --object-lock-retain-until-date "2022-05-31" --key compliance-upload --body /etc/fstab
   {
       "ETag": "\"d560ea5652951637ba9c594d8e6ea8c1\"",
       "VersionId": "Nhhk5kRS6Yp6dZXVWpZZdRcpSpBKToD"
   }

Procedure

1. Set the rate limit for the user:

   Syntax

   radosgw-admin ratelimit set --ratelimit-scope=user --uid=USER_ID [--max-read-ops=NUMBER_OF_OPERATIONS] [--max-read-bytes=NUMBER_OF_BYTES]
   [--max-write-ops=NUMBER_OF_OPERATIONS] [--max-write-bytes=NUMBER_OF_BYTES]

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit set --ratelimit-scope=user --uid=testing --max-read-ops=1024 --max-write-bytes=10240

   A value of 0 for NUMBER_OF_OPERATIONS or NUMBER_OF_BYTES means that the specific rate limit attribute check is disabled.

2. Get the user rate limit:

   Syntax

   radosgw-admin ratelimit get --ratelimit-scope=user --uid=USER_ID

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit get --ratelimit-scope=user --uid=testing
   
   {
       "user_ratelimit": {
           "max_read_ops": 1024,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 10240,
           "enabled": false
       }
   }

3. Enable user rate limit:

   Syntax

   radosgw-admin ratelimit enable --ratelimit-scope=user --uid=USER_ID

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit enable --ratelimit-scope=user --uid=testing
   
   {
       "user_ratelimit": {
           "max_read_ops": 1024,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 10240,
           "enabled": true
       }
   }

4. Optional: Disable user rate limit:

   Syntax

   radosgw-admin ratelimit disable --ratelimit-scope=user --uid=USER_ID

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit disable --ratelimit-scope=user --uid=testing

Procedure

1. Set the rate limit for the bucket:

   Syntax

   radosgw-admin ratelimit set --ratelimit-scope=bucket --bucket= BUCKET_NAME [--max-read-ops=NUMBER_OF_OPERATIONS] [--max-read-bytes=NUMBER_OF_BYTES]
   [--max-write-ops=NUMBER_OF_OPERATIONS] [--max-write-bytes=NUMBER_OF_BYTES]

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit set --ratelimit-scope=bucket --bucket=mybucket --max-read-ops=1024 --max-write-bytes=10240

   A value of 0 for NUMBER_OF_OPERATIONS or NUMBER_OF_BYTES means that the specific rate limit attribute check is disabled.

2. Get the bucket rate limit:

   Syntax

   radosgw-admin ratelimit get --ratelimit-scope=bucket --bucket=BUCKET_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit get --ratelimit-scope=bucket --bucket=mybucket
   
   {
       "bucket_ratelimit": {
           "max_read_ops": 1024,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 10240,
           "enabled": false
       }
   }

3. Enable bucket rate limit:

   Syntax

   radosgw-admin ratelimit enable --ratelimit-scope=bucket --bucket=BUCKET_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit enable --ratelimit-scope=bucket --bucket=mybucket
   
   {
       "bucket_ratelimit": {
           "max_read_ops": 1024,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 10240,
           "enabled": true
       }
   }

4. Optional: Disable bucket rate limit:

   Syntax

   radosgw-admin ratelimit disable --ratelimit-scope=bucket --bucket=BUCKET_NAME

   Example

   [ceph: root@host01 /]# radosgw-admin ratelimit disable --ratelimit-scope=bucket --bucket=mybucket

Procedure

1. View the global rate limit settings:

   Syntax

   radosgw-admin global ratelimit get

   Example

   [ceph: root@host01 /]# radosgw-admin global ratelimit get
   
   {
       "bucket_ratelimit": {
           "max_read_ops": 1024,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 0,
           "enabled": false
       },
       "user_ratelimit": {
           "max_read_ops": 0,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 0,
           "enabled": false
       },
       "anonymous_ratelimit": {
           "max_read_ops": 0,
           "max_write_ops": 0,
           "max_read_bytes": 0,
           "max_write_bytes": 0,
           "enabled": false
       }
   }

2. Configure and enable rate limit scope for the buckets:

   1. Set the global rate limits for bucket:

      Syntax

      radosgw-admin global ratelimit set --ratelimit-scope=bucket [--max-read-ops=NUMBER_OF_OPERATIONS] [--max-read-bytes=NUMBER_OF_BYTES]
      [--max-write-ops=NUMBER_OF_OPERATIONS] [--max-write-bytes=NUMBER_OF_BYTES]

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit set --ratelimit-scope bucket --max-read-ops=1024

   2. Enable bucket rate limit:

      Syntax

      radosgw-admin global ratelimit enable --ratelimit-scope=bucket

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit enable --ratelimit-scope bucket

3. Configure and enable rate limit scope for authenticated users:

   1. Set the global rate limits for users:

      Syntax

      radosgw-admin global ratelimit set --ratelimit-scope=user [--max-read-ops=NUMBER_OF_OPERATIONS] [--max-read-bytes=NUMBER_OF_BYTES]
      [--max-write-ops=NUMBER_OF_OPERATIONS] [--max-write-bytes=NUMBER_OF_BYTES]

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit set --ratelimit-scope=user --max-read-ops=1024

   2. Enable user rate limit:

      Syntax

      radosgw-admin global ratelimit enable --ratelimit-scope=user

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit enable --ratelimit-scope=user

4. Configure and enable rate limit scope for unauthenticated users:

   1. Set the global rate limits for unauthenticated users:

      Syntax

      radosgw-admin global ratelimit set --ratelimit-scope=anonymous [--max-read-ops=NUMBER_OF_OPERATIONS] [--max-read-bytes=NUMBER_OF_BYTES]
      [--max-write-ops=NUMBER_OF_OPERATIONS] [--max-write-bytes=NUMBER_OF_BYTES]

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit set --ratelimit-scope=anonymous --max-read-ops=1024

   2. Enable user rate limit:

      Syntax

      radosgw-admin global ratelimit enable --ratelimit-scope=anonymous

      Example

      [ceph: root@host01 /]# radosgw-admin global ratelimit enable --ratelimit-scope=anonymous

Procedure

1. Set the value of rgw_gc_max_concurrent_io to 20, and the value of rgw_gc_max_trim_chunk to 64:

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_gc_max_concurrent_io 20
   [ceph: root@host01 /]# ceph config set client.rgw rgw_gc_max_trim_chunk 64

2. Restart the Ceph Object Gateway to allow the changed settings to take effect.
3. Monitor the storage cluster during GC activity to verify that the increased values do not adversely affect performance.

Procedure

1. To increase the number of threads to run in parallel, set the value of rgw_lc_max_worker to a value between 3 and 9:

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_lc_max_worker 7

2. To increase the number of threads in each thread’s work pool, set the value of rgw_lc_max_wp_worker to a value between 3 and 9:

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_lc_max_wp_worker 7

3. Restart the Ceph Object Gateway to allow the changed settings to take effect.
4. Monitor the storage cluster to verify that the increased values do not adversely affect performance.