Chapter 6. Object Gateway S3 API
Red Hat Ceph Object Gateway supports a RESTful API that is compatible with the basic data access model of the Amazon S3 API.
6.1. API
6.1.1. Feature Support
The following table describes the support status for current Amazon S3 functional features:
Feature | Status | Remarks |
---|---|---|
Supported | ||
Supported | Different set of canned ACLs | |
Supported | ||
Supported | ||
Supported | ||
Supported | Different set of canned ACLs | |
Supported | ||
Get Bucket Info (HEAD) | Supported | |
Supported | ||
Bucket Lifecycle | Not Supported | |
Policy (Buckets, Objects) | Not Supported | ACLs are supported |
Bucket Website | Not Supported | |
Bucket Notification | Not Supported | |
Bucket Request Payment | Not Supported | |
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Supported | ||
Multipart Uploads | Supported | (missing Copy Part) |
6.1.2. Unsupported Header Fields
The following common request header fields are not supported:
Name | Type |
---|---|
x-amz-security-token | Request |
Server | Response |
x-amz-delete-marker | Response |
x-amz-id-2 | Response |
x-amz-request-id | Response |
x-amz-version-id | Response |
6.2. Common
6.2.1. Bucket and Host Name
There are two different modes of accessing the buckets. The first (preferred) method identifies the bucket as the top-level directory in the URI.
GET /mybucket HTTP/1.1 Host: cname.domain.com
The second method identifies the bucket via a virtual bucket host name. For example:
GET / HTTP/1.1 Host: mybucket.cname.domain.com
We prefer the first method, because the second method requires expensive domain certification and DNS wild cards.
6.2.2. Common Request Headers
Request Header | Description |
---|---|
| Length of the request body. |
| Request time and date (in UTC). |
| The name of the host server. |
| Authorization token. |
6.2.3. Common Response Status
HTTP Status | Response Code |
---|---|
| Continue |
| Success |
| Created |
| Accepted |
| NoContent |
| Partial content |
| NotModified |
| InvalidArgument |
| InvalidDigest |
| BadDigest |
| InvalidBucketName |
| InvalidObjectName |
| UnresolvableGrantByEmailAddress |
| InvalidPart |
| InvalidPartOrder |
| RequestTimeout |
| EntityTooLarge |
| AccessDenied |
| UserSuspended |
| RequestTimeTooSkewed |
| NoSuchKey |
| NoSuchBucket |
| NoSuchUpload |
| MethodNotAllowed |
| RequestTimeout |
| BucketAlreadyExists |
| BucketNotEmpty |
| MissingContentLength |
| PreconditionFailed |
| InvalidRange |
| UnprocessableEntity |
| InternalError |
6.3. Authentication and ACLs
Requests to the Ceph Object Gateway can be either authenticated or unauthenticated. Ceph Object Gateway assumes unauthenticated requests are sent by an anonymous user. Ceph Object Gateway supports canned ACLs.
6.3.1. Authentication
For most use cases, clients use existing open source libraries like the Amazon SDK’s AmazonS3Client
for Java, Python Boto, etc. where you simply pass in the access key and secret key, and the library builds the request header and authentication signature for you. However, you can create your own requests and sign them too.
Authenticating a request requires including an access key and a base 64-encoded Hash-based Message Authentication Code (HMAC) in the request before it is sent to the Ceph Object Gateway server. Ceph Object Gateway uses an S3-compatible authentication approach.
HTTP/1.1 PUT /buckets/bucket/object.mpeg Host: cname.domain.com Date: Mon, 2 Jan 2012 00:01:01 +0000 Content-Encoding: mpeg Content-Length: 9999999 Authorization: AWS {access-key}:{hash-of-header-and-secret}
In the foregoing example, replace {access-key}
with the value for your access key ID followed by a colon (:
). Replace {hash-of-header-and-secret}
with a hash of a canonicalized header string and the secret corresponding to the access key ID.
To generate the hash of the header string and secret, you must:
- Get the value of the header string.
- Normalize the request header string into canonical form.
- Generate an HMAC using a SHA-1 hashing algorithm.
-
Encode the
hmac
result as base-64.
To normalize the header into canonical form:
-
Get all
content-
headers. -
Remove all
content-
headers except forcontent-type
andcontent-md5
. -
Ensure the
content-
header names are lowercase. -
Sort the
content-
headers lexicographically. -
Ensure you have a
Date
header AND ensure the specified date uses GMT and not an offset. -
Get all headers beginning with
x-amz-
. -
Ensure that the
x-amz-
headers are all lowercase. -
Sort the
x-amz-
headers lexicographically. - Combine multiple instances of the same field name into a single field and separate the field values with a comma.
- Replace white space and line breaks in header values with a single space.
- Remove white space before and after colons.
- Append a new line after each header.
- Merge the headers back into the request header.
Replace the {hash-of-header-and-secret}
with the base-64 encoded HMAC string.
For additional details, consult the Signing and Authenticating REST Requests section of Amazon Simple Storage Service documentation.
6.3.2. Access Control Lists (ACLs)
Ceph Object Gateway supports S3-compatible ACL functionality. An ACL is a list of access grants that specify which operations a user can perform on a bucket or on an object. Each grant has a different meaning when applied to a bucket versus applied to an object:
Permission | Bucket | Object |
---|---|---|
| Grantee can list the objects in the bucket. | Grantee can read the object. |
| Grantee can write or delete objects in the bucket. | N/A |
| Grantee can read bucket ACL. | Grantee can read the object ACL. |
| Grantee can write bucket ACL. | Grantee can write to the object ACL. |
| Grantee has full permissions for object in the bucket. | Grantee can read or write to the object ACL. |
6.4. Service Operations
6.4.1. List Buckets
GET /
returns a list of buckets created by the user making the request. GET /
only returns buckets created by an authenticated user. You cannot make an anonymous request.
6.4.1.1. Syntax
GET / HTTP/1.1 Host: cname.domain.com Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.4.1.2. Response Entities
Name | Type | Description |
---|---|---|
| Container | Container for list of buckets. |
| Container | Container for bucket information. |
| String | Bucket name. |
| Date | UTC time when the bucket was created. |
| Container | A container for the result. |
| Container |
A container for the bucket owner’s |
| String | The bucket owner’s ID. |
| String | The bucket owner’s display name. |
6.5. Bucket Operations
6.5.1. PUT Bucket
Creates a new bucket. To create a bucket, you must have a user ID and a valid AWS Access Key ID to authenticate requests. You may not create buckets as an anonymous user.
6.5.1.1. Constraints
In general, bucket names should follow domain name constraints.
- Bucket names must be unique.
- Bucket names must begin and end with a lowercase letter.
- Bucket names may contain a dash (-).
6.5.1.2. Syntax
PUT /{bucket} HTTP/1.1 Host: cname.domain.com x-amz-acl: public-read-write Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.5.1.3. Parameters
Name | Description | Valid Values | Required |
---|---|---|---|
| Canned ACLs. |
| No |
6.5.1.4. HTTP Response
If the bucket name is unique, within constraints and unused, the operation will succeed. If a bucket with the same name already exists and the user is the bucket owner, the operation will succeed. If the bucket name is already in use, the operation will fail.
HTTP Status | Status Code | Description |
---|---|---|
| BucketAlreadyExists | Bucket already exists under different user’s ownership. |
6.5.2. DELETE Bucket
Deletes a bucket. You can reuse bucket names following a successful bucket removal.
6.5.2.1. Syntax
DELETE /{bucket} HTTP/1.1 Host: cname.domain.com Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.5.2.2. HTTP Response
HTTP Status | Status Code | Description |
---|---|---|
| No Content | Bucket removed. |
6.5.3. GET Bucket
Returns a list of bucket objects.
6.5.3.1. Syntax
GET /{bucket}?max-keys=25 HTTP/1.1 Host: cname.domain.com
6.5.3.2. Parameters
Name | Type | Description |
---|---|---|
| String | Only returns objects that contain the specified prefix. |
| String | The delimiter between the prefix and the rest of the object name. |
| String | A beginning index for the list of objects returned. |
| Integer | The maximum number of keys to return. Default is 1000. |
6.5.3.3. HTTP Response
HTTP Status | Status Code | Description |
---|---|---|
| OK | Buckets retrieved |
6.5.3.4. Bucket Response Entities
GET /{bucket}
returns a container for buckets with the following fields.
Name | Type | Description |
---|---|---|
| Entity | The container for the list of objects. |
| String | The name of the bucket whose contents will be returned. |
| String | A prefix for the object keys. |
| String | A beginning index for the list of objects returned. |
| Integer | The maximum number of keys returned. |
| String |
If set, objects with the same prefix will appear in the |
| Boolean |
If |
| Container | If multiple objects contain the same prefix, they will appear in this list. |
6.5.3.5. Object Response Entities
The ListBucketResult
contains objects, where each object is within a Contents
container.
Name | Type | Description |
---|---|---|
| Object | A container for the object. |
| String | The object’s key. |
| Date | The object’s last-modified date/time. |
| String | An MD-5 hash of the object. (entity tag) |
| Integer | The object’s size. |
| String |
Should always return |
6.5.4. Get Bucket Location
Retrieves the bucket’s region. The user needs to be the bucket owner to call this. A bucket can be constrained to a region by providing LocationConstraint
during a PUT request.
6.5.4.1. Syntax
Add the location
subresource to bucket resource as shown below.
GET /{bucket}?location HTTP/1.1 Host: cname.domain.com Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.5.4.2. Response Entities
Name | Type | Description |
---|---|---|
| String | The region where bucket resides, empty string for defult region |
6.5.5. Get Bucket ACLs
Retrieves the bucket access control list. The user needs to be the bucket owner or to have been granted READ_ACP
permission on the bucket.
6.5.5.1. Syntax
Add the acl
subresource to the bucket request as shown below.
GET /{bucket}?acl HTTP/1.1 Host: cname.domain.com Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.5.5.2. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the response. |
| Container | A container for the ACL information. |
| Container |
A container for the bucket owner’s |
| String | The bucket owner’s ID. |
| String | The bucket owner’s display name. |
| Container |
A container for |
| Container |
A container for the |
| String |
The permission given to the |
6.5.6. PUT Bucket ACLs
Sets an access control to an existing bucket. The user needs to be the bucket owner or to have been granted WRITE_ACP
permission on the bucket.
6.5.6.1. Syntax
Add the acl
subresource to the bucket request as shown below.
PUT /{bucket}?acl HTTP/1.1
6.5.6.2. Request Entities
Name | Type | Description |
---|---|---|
| Container | A container for the request. |
| Container | A container for the ACL information. |
| Container |
A container for the bucket owner’s |
| String | The bucket owner’s ID. |
| String | The bucket owner’s display name. |
| Container |
A container for |
| Container |
A container for the |
| String |
The permission given to the |
6.5.7. List Bucket Object Versions
Returns a list of metadata about all the version of objects within a bucket. Requires READ access to the bucket.
6.5.7.1. Syntax
Add the versions
subresource to the bucket request as shown below.
GET /{bucket}?versions HTTP/1.1 Host: cname.domain.com Authorization: AWS {access-key}:{hash-of-header-and-secret}
6.5.7.2. Parameters
You may specify parameters for GET /{bucket}?versions
, but none of them are required.
Name | Type | Description |
---|---|---|
| String | Returns in-progress uploads whose keys contains the specified prefix. |
| String | The delimiter between the prefix and the rest of the object name. |
| String | The beginning marker for the list of uploads. |
| Integer | The maximum number of in-progress uploads. The default is 1000. |
| String | Specifies the object version to begin the list. |
6.5.7.3. Response Entities
Name | Type | Description |
---|---|---|
| String |
The key marker specified by the |
| String |
The key marker to use in a subsequent request if |
| String |
The upload ID marker to use in a subsequent request if |
| Boolean |
If |
| Integer | The size of the uploaded part. |
| String | The owners’s display name. |
| String | The owners’s ID. |
| Container |
A container for the |
| String |
The method used to store the resulting object. |
| Container | Container for the version information. |
| String | Version ID of an object. |
| String | The last version of the key in a truncated response. |
6.5.8. List Bucket Multipart Uploads
GET /?uploads
returns a list of the current in-progress multipart uploads—i.e., the application initiates a multipart upload, but the service hasn’t completed all the uploads yet.
6.5.8.1. Syntax
GET /{bucket}?uploads HTTP/1.1
6.5.8.2. Parameters
You may specify parameters for GET /{bucket}?uploads
, but none of them are required.
Name | Type | Description |
---|---|---|
| String | Returns in-progress uploads whose keys contains the specified prefix. |
| String | The delimiter between the prefix and the rest of the object name. |
| String | The beginning marker for the list of uploads. |
| Integer | The maximum number of in-progress uploads. The default is 1000. |
| Integer | The maximum number of multipart uploads. The range from 1-1000. The default is 1000. |
| String |
Ignored if |
6.5.8.3. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the results. |
| String |
The prefix specified by the |
| String | The bucket that will receive the bucket contents. |
| String |
The key marker specified by the |
| String |
The marker specified by the |
| String |
The key marker to use in a subsequent request if |
| String |
The upload ID marker to use in a subsequent request if |
| Integer |
The max uploads specified by the |
| String |
If set, objects with the same prefix will appear in the |
| Boolean |
If |
| Container |
A container for |
| String | The key of the object once the multipart upload is complete. |
| String |
The |
| Container |
Contains the |
| String | The initiator’s display name. |
| String | The initiator’s ID. |
| Container |
A container for the |
| String |
The method used to store the resulting object. |
| Date | The date and time the user initiated the upload. |
| Container | If multiple objects contain the same prefix, they will appear in this list. |
| String |
The substring of the key after the prefix as defined by the |
6.6. Object Operations
6.6.1. PUT Object
Adds an object to a bucket. You must have write permissions on the bucket to perform this operation.
6.6.1.1. Syntax
PUT /{bucket}/{object} HTTP/1.1
6.6.1.2. Request Headers
Name | Description | Valid Values | Required |
---|---|---|---|
content-md5 | A base64 encoded MD-5 hash of the message. | A string. No defaults or constraints. | No |
content-type | A standard MIME type. |
Any MIME type. Default: | No |
x-amz-meta-<…> | User metadata. Stored with the object. | A string up to 8kb. No defaults. | No |
x-amz-acl | A canned ACL. |
| No |
6.6.1.3. Response Headers
Name | Description |
---|---|
x-amz-version-id | Returns the version ID or null. |
6.6.2. Copy Object
To copy an object, use PUT
and specify a destination bucket and the object name.
6.6.2.1. Syntax
PUT /{dest-bucket}/{dest-object} HTTP/1.1 x-amz-copy-source: {source-bucket}/{source-object}
6.6.2.2. Request Headers
Name | Description | Valid Values | Required |
---|---|---|---|
x-amz-copy-source | The source bucket name + object name. | {bucket}/{obj} | Yes |
x-amz-acl | A canned ACL. |
| No |
x-amz-copy-if-modified-since | Copies only if modified since the timestamp. | Timestamp | No |
x-amz-copy-if-unmodified-since | Copies only if unmodified since the timestamp. | Timestamp | No |
x-amz-copy-if-match | Copies only if object ETag matches ETag. | Entity Tag | No |
x-amz-copy-if-none-match | Copies only if object ETag doesn’t match. | Entity Tag | No |
6.6.2.3. Response Entities
Name | Type | Description |
---|---|---|
CopyObjectResult | Container | A container for the response elements. |
LastModified | Date | The last modified date of the source object. |
Etag | String | The ETag of the new object. |
6.6.3. Remove Object
Removes an object. Requires WRITE permission set on the containing bucket.
6.6.3.1. Syntax
Deletes an object. If object versioning is on, it creates a marker.
DELETE /{bucket}/{object} HTTP/1.1
To delete an object when versioning is on, you must specify the versionId
subresource and the version of the object to delete.
DELETE /{bucket}/{object}?versionId={versionID} HTTP/1.1
6.6.4. Get Object
Retrieves an object from a bucket.
6.6.4.1. Syntax
GET /{bucket}/{object} HTTP/1.1
Add the versionId
subresource to retrieve a particular version of the object.
GET /{bucket}/{object}?versionId={versionID} HTTP/1.1
6.6.4.2. Request Headers
Name | Description | Valid Values | Required |
---|---|---|---|
range | The range of the object to retrieve. | Range: bytes=beginbyte-endbyte | No |
if-modified-since | Gets only if modified since the timestamp. | Timestamp | No |
if-unmodified-since | Gets only if not modified since the timestamp. | Timestamp | No |
if-match | Gets only if object ETag matches ETag. | Entity Tag | No |
if-none-match | Gets only if object ETag matches ETag. | Entity Tag | No |
6.6.4.3. Response Headers
Name | Description |
---|---|
Content-Range | Data range, will only be returned if the range header field was specified in the request |
x-amz-version-id | Returns the version ID or null. |
6.6.5. Get Object Information
Returns information about an object. This request will return the same header information as with the Get Object request, but will include the metadata only, not the object data payload.
6.6.5.1. Syntax
Retrieves the current version of the object.
HEAD /{bucket}/{object} HTTP/1.1
Add the versionId
subresource to retrieve info for a particular version.
HEAD /{bucket}/{object}?versionId={versionID} HTTP/1.1
6.6.5.2. Request Headers
Name | Description | Valid Values | Required |
---|---|---|---|
range | The range of the object to retrieve. | Range: bytes=beginbyte-endbyte | No |
if-modified-since | Gets only if modified since the timestamp. | Timestamp | No |
if-unmodified-since | Gets only if not modified since the timestamp. | Timestamp | No |
if-match | Gets only if object ETag matches ETag. | Entity Tag | No |
if-none-match | Gets only if object ETag matches ETag. | Entity Tag | No |
6.6.5.3. Response Headers
Name | Description |
---|---|
x-amz-version-id | Returns the version ID or null. |
6.6.6. Get Object ACL
6.6.6.1. Syntax
Returns the ACL for the current version of the object.
GET /{bucket}/{object}?acl HTTP/1.1
Add the versionId
subresource to retrieve the ACL for a particular version.
GET /{bucket}/{object}versionId={versionID}&acl HTTP/1.1
6.6.6.2. Response Headers
Name | Description |
---|---|
x-amz-version-id | Returns the version ID or null. |
6.6.6.3. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the response. |
| Container | A container for the ACL information. |
| Container |
A container for the object owner’s |
| String | The object owner’s ID. |
| String | The object owner’s display name. |
| Container |
A container for |
| Container |
A container for the |
| String |
The permission given to the |
6.6.7. Set Object ACL
Sets an object ACL for the current version of the object.
6.6.7.1. Syntax
PUT /{bucket}/{object}?acl
6.6.7.2. Request Entities
Name | Type | Description |
---|---|---|
| Container | A container for the response. |
| Container | A container for the ACL information. |
| Container |
A container for the object owner’s |
| String | The object owner’s ID. |
| String | The object owner’s display name. |
| Container |
A container for |
| Container |
A container for the |
| String |
The permission given to the |
6.6.8. Initiate Multipart Upload
Initiates a multi-part upload process. Returns a UploadId
, which you may specify when adding additional parts, listing parts, and completing or abandoning a multi-part upload.
6.6.8.1. Syntax
POST /{bucket}/{object}?uploads
6.6.8.2. Request Headers
Name | Description | Valid Values | Required |
---|---|---|---|
| A base64 encoded MD-5 hash of the message. | A string. No defaults or constraints. | No |
| A standard MIME type. |
Any MIME type. Default: | No |
| User metadata. Stored with the object. | A string up to 8kb. No defaults. | No |
| A canned ACL. |
| No |
6.6.8.3. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the results. |
| String | The bucket that will receive the object contents. |
| String |
The key specified by the |
| String |
The ID specified by the |
6.6.9. Multipart Upload Part
Adds a part to a multi-part upload.
6.6.9.1. Syntax
Specify the uploadId
subresource and the upload ID to add a part to a multi-part upload.
PUT /{bucket}/{object}?partNumber=&uploadId={upload-id} HTTP/1.1
6.6.9.2. HTTP Response
The following HTTP response may be returned:
HTTP Status | Status Code | Description |
---|---|---|
| NoSuchUpload | Specified upload-id does not match any initiated upload on this object |
6.6.10. List Multipart Upload Parts
6.6.10.1. Syntax
Specify the uploadId
subresource and the upload ID to list the parts of a multi-part upload.
GET /{bucket}/{object}?uploadId={upload-id} HTTP/1.1
6.6.10.2. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the results. |
| String | The bucket that will receive the object contents. |
| String |
The key specified by the |
| String |
The ID specified by the |
| Container |
Contains the |
| String | The initiator’s ID. |
| String | The initiator’s display name. |
| Container |
A container for the |
| String |
The method used to store the resulting object. |
| String |
The part marker to use in a subsequent request if |
| String |
The next part marker to use in a subsequent request if |
| Integer |
The max parts allowed in the response as specified by the |
| Boolean |
If |
| Container |
A container for |
| Integer | The identification number of the part. |
| String | The part’s entity tag. |
| Integer | The size of the uploaded part. |
6.6.11. Complete Multipart Upload
Assembles uploaded parts and creates a new object, thereby completing a multipart upload.
6.6.11.1. Syntax
Specify the uploadId
subresource and the upload ID to complete a multi-part upload.
POST /{bucket}/{object}?uploadId= HTTP/1.1
6.6.11.2. Request Entities
Name | Type | Description | Required |
---|---|---|---|
| Container | A container consisting of one or more parts. | Yes |
| Container |
A container for the | Yes |
| Integer | The identifier of the part. | Yes |
| String | The part’s entity tag. | Yes |
6.6.11.3. Response Entities
Name | Type | Description |
---|---|---|
| Container | A container for the response. |
| URI | The resource identifier (path) of the new object. |
| String | The name of the bucket that contains the new object. |
| String | The object’s key. |
| String | The entity tag of the new object. |
6.6.12. Abort Multipart Upload
Aborts a multipart upload.
6.6.12.1. Syntax
Specify the uploadId
subresource and the upload ID to abort a multi-part upload.
DELETE /{bucket}/{object}?uploadId={upload-id} HTTP/1.1
6.7. Accessing the Gateway
You can use various programming languages to create a connection with the gateway server and do the bucket management tasks. There are different open source libraries available for these programming languages that are used for authentication with the gateway.
The sections mentioned below will describe the procedure for some of the popular programmimg languages.
6.7.1. Prerequisites
You have to follow some pre-requisites on your gateway node
before attempting to access the gateway server. The pre-requisites are as follows:
- Set up your gateway server properly by following the instructions mentioned in Install RHCS v1.3 Ceph Object Gateway for RHEL.
-
DO NOT modify the Ceph configuration file to use port
80
and letCivetweb
use the default port7480
. Port80
is required byApache
and it needs to be running and enabled. Install and start
Apache
.sudo yum install httpd -y sudo systemctl start httpd sudo systemctl enable httpd.service
Open port
80
on firewall.sudo firewall-cmd --zone=public --add-port=80/tcp --permanent sudo firewall-cmd --reload
Disable
ssl
.If you have
ssl
enabled in yourgateway node
, please follow the steps mentioned in Disable SSL to disable it.Add a wildcard to your DNS server that you are using for your gateway as mentioned in Add wildcard to DNS.
You can also set up your gateway node for local DNS caching. To do so, execute the following steps:
Install and setup
dnsmasq
.sudo yum install dnsmasq -y echo "address=/.<FQDN_of_gateway_node>/<IP_of_gateway_node>" | sudo tee --append /etc/dnsmasq.conf sudo systemctl start dnsmasq sudo systemctl enable dnsmasq
Replace
<IP_of_gateway_node>
and<FQDN_of_gateway_node>
with the IP address and FQDN of your gateway node.Stop NetworkManager.
sudo systemctl stop NetworkManager sudo systemctl disable NetworkManager
Set your gateway server’s IP as the nameserver.
echo "DNS1=<IP_of_gateway_node>" | sudo tee --append /etc/sysconfig/network-scripts/ifcfg-eth0 echo "<IP_of_gateway_node> <FQDN_of_gateway_node>" | sudo tee --append /etc/hosts sudo systemctl restart network sudo systemctl enable network sudo systemctl restart dnsmasq
Replace
<IP_of_gateway_node>
and<FQDN_of_gateway_node>
with the IP address and FQDN of your gateway node.Verify subdomain requests:
ping mybucket.<FQDN_of_gateway_node>
Replace
<FQDN_of_gateway_node>
with the FQDN of your gateway node.WarningSetting up your gateway server for local DNS caching is for testing purposes only. You won’t be able to access outside network after doing this. It is strongly recommended to use a proper DNS server for your Ceph cluster and gateway node.
-
Create the
radosgw
user forS3
access carefully as mentioned in Create Radosgw user for S3 access and copy the generatedaccess_key
andsecret_key
. You will need these keys forS3
access and subsequent bucket management tasks.
6.7.2. Ruby AWS::S3 Examples (aws-s3 gem)
You can use Ruby
programming language alongwith aws-s3
gem for S3
access. Execute the steps mentioned below in your gateway node
for accessing the gateway with Ruby AWS::S3
.
6.7.2.1. Setup Ruby
Execute the following steps to setup Ruby
:
Install
Ruby
:sudo yum install ruby -y
NoteThe above command will install
ruby
and it’s essential dependencies likerubygems
andruby-libs
too. If somehow the command doesn’t install all the dependencies, install them separately.Install
aws-s3
:sudo gem install aws-s3
6.7.2.2. Creating a connection
Create a project directory:
mkdir ruby_aws_s3 cd ruby_aws_s3
Create the connection file:
vim conn.rb
Paste the following contents in the
conn.rb
file:#!/usr/bin/env ruby require 'aws/s3' require 'resolv-replace' AWS::S3::Base.establish_connection!( :server => '<FQDN_of_gateway_node>', :port => '7480', :access_key_id => 'my-access-key', :secret_access_key => 'my-secret-key' )
Replace
<FQDN_of_gateway_node>
with the FQDN of you gateway node. Replacemy-access-key
andmy-secret-key
with theaccess_key
andsecret_key
that was generated when you created theradosgw
user forS3
access as mentioned in Create Radosgw user for S3 access.An example connection file will look like the following:
#!/usr/bin/env ruby require 'aws/s3' require 'resolv-replace' AWS::S3::Base.establish_connection!( :server => 'testclient.englab.pnq.redhat.com', :port => '7480', :access_key_id => '98J4R9P22P5CDL65HKP8', :secret_access_key => '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049' )
Save the file and exit the editor.
Make the file executable:
chmod +x conn.rb
Run the file:
./conn.rb | echo $?
If you have provided the values correctly in the file, the output of the command will be
0
.
6.7.2.3. Creating a bucket
Create a new file:
vim create_bucket.rb
Paste the following contents into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::Bucket.create('my-new-bucket1')
Save the file and exit the editor.
Make the file executable:
chmod +x create_bucket.rb
Run the file:
./create_bucket.rb
If the output of the command is
true
it would mean that bucketmy-new-bucket1
was created successfully.
6.7.2.4. Listing owned buckets
Create a new file:
vim list_owned_buckets.rb
Paste the following content into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::Service.buckets.each do |bucket| puts "#{bucket.name}\t#{bucket.creation_date}" end
Save the file and exit the editor.
Make the file executable:
chmod +x list_owned_buckets.rb
Run the file:
./list_owned_buckets.rb
The output should look something like this:
my-new-bucket1 2016-01-21 10:33:19 UTC
6.7.2.5. Creating an object
Create a new file:
vim create_object.rb
Paste the following contents into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::S3Object.store( 'hello.txt', 'Hello World!', 'my-new-bucket1', :content_type => 'text/plain' )
Save the file and exit the editor.
Make the file executable:
chmod +x create_object.rb
Run the file:
./create_object.rb
This will create a file
hello.txt
with the stringHello World!
.
6.7.2.6. Listing a Bucket’s Content
Create a new file:
vim list_bucket_content.rb
Paste the following content into the file:
#!/usr/bin/env ruby load 'conn.rb' new_bucket = AWS::S3::Bucket.find('my-new-bucket1') new_bucket.each do |object| puts "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}" end
Save the file and exit the editor.
Make the file executable.
chmod +x list_bucket_content.rb
Run the file:
./list_bucket_content.rb
The output will look something like this:
hello.txt 12 Fri, 22 Jan 2016 15:54:52 GMT
6.7.2.7. Deleting a empty bucket
Create a new file:
vim del_empty_bucket.rb
Paste the following contents into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::Bucket.delete('my-new-bucket1')
Save the file and exit the editor.
Make the file executable:
chmod +x del_empty_bucket.rb
Run the file:
./del_empty_bucket.rb | echo $?
If the bucket is successfully deleted, the command will return
0
as output.NotePlease edit the
create_bucket.rb
file to create empty buckets likemy-new-bucket9
,my-new-bucket10
etc and edit the above mentioneddel_empty_bucket.rb
file accordingly before trying to delete empty buckets.
6.7.2.8. Deleting a non-empty bucket (forcefully)
Create a new file:
vim del_non_empty_bucket.rb
Paste the following contents into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::Bucket.delete('my-new-bucket1', :force => true)
Save the file and exit the editor.
Make the file executable:
chmod +x del_non_empty_bucket.rb
Run the file:
./del_non_empty_bucket.rb | echo $?
If the bucket is successfully deleted, the command will return
0
as output.
6.7.2.9. Deleting an object
Create a new file:
vim delete_object.rb
Paste the following contents into the file:
#!/usr/bin/env ruby load 'conn.rb' AWS::S3::S3Object.delete('hello.txt', 'my-new-bucket1')
Save the file and exit the editor.
Make the file executable:
chmod +x delete_object.rb
Run the file:
./delete_object.rb
This will delete the object
hello.txt
.