Chapter 2. Object Gateway S3 Application Programming Interface (API)

Red Hat Ceph Object Gateway supports a RESTful API that is compatible with the basic data access model of the Amazon S3 API.

The following table describes the support status for current Amazon S3 functional features.

Table 2.1. Features
Feature	Status	Remarks
List Buckets	Supported
Create Bucket	Supported	Different set of canned ACLs
Get Bucket	Supported
Get Bucket Location	Supported
Get Bucket Versioning	Supported
Delete Bucket	Supported
Bucket ACLs (Get, Put)	Supported	Different set of canned ACLs
Bucket cors (Get, Put, Delete)	Supported
Bucket Object Versions	Supported
Get Bucket Info (HEAD)	Supported
List Bucket Multipart Uploads	Supported
Bucket Lifecycle	Not Supported
Policy (Buckets, Objects)	Not Supported	ACLs are supported
Bucket Website	Supported
Bucket Notification	Not Supported
Bucket Request Payment (Get, Put)	Supported
Put Object	Supported
Delete Object	Supported
Get Object	Supported
Object ACLs (Get, Put)	Supported
Get Object Info (HEAD)	Supported
Copy Object	Supported
Post Object	Supported
Options Object	Supported
Delete Multiple Objects	Supported
Initiate Multipart Upload	Supported
Initiage Multipart Upload Part	Supported
List Multipart Upload Parts	Supported
Complete Multipart Upload	Supported
Abort Multipart Upload	Supported
Multipart Uploads	Supported
Copy Multipart Upload	Supported
Multi Tenancy	Supported

The following table lists the common request header fields that are not supported.

Table 2.2. Unsupported Header Fields
Name	Type
x-amz-security-token	Request
Server	Response
x-amz-delete-marker	Response
x-amz-id-2	Response
x-amz-request-id	Response
x-amz-version-id	Response

2.1. Authentication and Access Control Lists

Requests to the Ceph Object Gateway can be either authenticated or unauthenticated. Ceph Object Gateway assumes unauthenticated requests are sent by an anonymous user. Ceph Object Gateway supports canned ACLs.

2.1.1. Authentication

For most use cases, clients use existing open source libraries like the Amazon SDK’s AmazonS3Client for Java, and Python Boto. where you simply pass in the access key and secret key, and the library builds the request header and authentication signature for you. However, you can create requests and sign them too.

Authenticating a request requires including an access key and a base 64-encoded Hash-based Message Authentication Code (HMAC) in the request before it is sent to the Ceph Object Gateway server. Ceph Object Gateway uses an S3-compatible authentication approach.

Example

HTTP/1.1
PUT /buckets/bucket/object.mpeg
Host: cname.domain.com
Date: Mon, 2 Jan 2012 00:01:01 +0000
Content-Encoding: mpeg
Content-Length: 9999999

Authorization: AWS <access_key>:<hash_of_header_and_secret>

In the foregoing example, replace <access_key> with the value for the access key ID followed by a colon (:). Replace <hash_of_header_and_secret> with a hash of a canonicalized header string and the secret corresponding to the access key ID.

To generate the hash of the header string and secret, you must:

Get the value of the header string.
Normalize the request header string into canonical form.
Generate an HMAC using a SHA-1 hashing algorithm.
Encode the hmac result as base-64.

To normalize the header into canonical form:

Get all content- headers.
Remove all content- headers except for content-type and content-md5.
Ensure the content- header names are lowercase.
Sort the content- headers lexicographically.
Ensure you have a Date header AND ensure the specified date uses GMT and not an offset.
Get all headers beginning with x-amz-.
Ensure that the x-amz- headers are all lowercase.
Sort the x-amz- headers lexicographically.
Combine multiple instances of the same field name into a single field and separate the field values with a comma.
Replace white space and line breaks in header values with a single space.
Remove white space before and after colons.
Append a new line after each header.
Merge the headers back into the request header.

Replace the <hash_of_header_and_secret> with the base-64 encoded HMAC string.

For additional details, consult the Signing and Authenticating REST Requests section of Amazon Simple Storage Service documentation.

2.1.2. Access Control Lists (ACLs)

Ceph Object Gateway supports S3-compatible ACL functionality. An ACL is a list of access grants that specify which operations a user can perform on a bucket or on an object. Each grant has a different meaning when applied to a bucket versus applied to an object:

Table 2.3. User Operations
Permission	Bucket	Object
`READ`	Grantee can list the objects in the bucket.	Grantee can read the object.
`WRITE`	Grantee can write or delete objects in the bucket.	N/A
`READ_ACP`	Grantee can read bucket ACL.	Grantee can read the object ACL.
`WRITE_ACP`	Grantee can write bucket ACL.	Grantee can write to the object ACL.
`FULL_CONTROL`	Grantee has full permissions for object in the bucket.	Grantee can read or write to the object ACL.

2.2. Accessing the Gateway

You can use various programming languages to create a connection with the gateway server and do the bucket management tasks. There are different open source libraries available for these programming languages that are used for authentication with the gateway.

The sections mentioned below will describe the procedure for some of the popular programming languages.

2.2.1. Prerequisites

You have to follow some pre-requisites on the Ceph Object Gateway node before attempting to access the gateway server. The pre-requisites are as follows:

Set up the gateway server properly by following the instructions based on the operating system:
1. For Red Hat Enterprise Linux, see the Ceph Object Gateway Installation chapter in the Red Hat Ceph Storage 2 Installation Guide.
2. For Ubuntu, see the Ceph Object Gateway Installation chapter in the Red Hat Ceph Storage 2 Installation Guide.
DO NOT modify the Ceph configuration file to use port 80 and let Civetweb use the default port 7480.

As root, open port 7480 on firewall:

# firewall-cmd --zone=public --add-port=7480/tcp --permanent
# firewall-cmd --reload

Add a wildcard to the DNS server that you are using for the gateway as mentioned in the Red Hat Ceph Storage Object Gateway Guide.
You can also set up the gateway node for local DNS caching. To do so, execute the following steps:
1. As root, install and setup dnsmasq:
```
# yum install dnsmasq
# echo "address=/.<FQDN_of_gateway_node>/<IP_of_gateway_node>" | tee --append /etc/dnsmasq.conf
# systemctl start dnsmasq
# systemctl enable dnsmasq
```
  Replace <IP_of_gateway_node> and <FQDN_of_gateway_node> with the IP address and FQDN of the gateway node.
2. As root, stop NetworkManager:
```
# systemctl stop NetworkManager
# systemctl disable NetworkManager
```
3. As root, set the gateway server’s IP as the nameserver:
```
# echo "DNS1=<IP_of_gateway_node>" | tee --append /etc/sysconfig/network-scripts/ifcfg-eth0
# echo "<IP_of_gateway_node> <FQDN_of_gateway_node>" | tee --append /etc/hosts
# systemctl restart network
# systemctl enable network
# systemctl restart dnsmasq
```
  Replace <IP_of_gateway_node> and <FQDN_of_gateway_node> with the IP address and FQDN of the gateway node.
4. Verify subdomain requests:
```
$ ping mybucket.<FQDN_of_gateway_node>
```
  Replace <FQDN_of_gateway_node> with the FQDN of the gateway node.
  Warning
  Setting up the gateway server for local DNS caching is for testing purposes only. You won’t be able to access outside network after doing this. It is strongly recommended to use a proper DNS server for the Ceph cluster and gateway node.
Create the radosgw user for S3 access carefully as mentioned in the Red Hat Ceph Storage Object Gateway Guide and copy the generated access_key and secret_key. You will need these keys for S3 access and subsequent bucket management tasks.

2.2.2. Ruby AWS::S3 Examples (aws-s3 gem)

You can use Ruby programming language along with aws-s3 gem for S3 access. Execute the steps mentioned below on the node used for accessing the Ceph Object Gateway server with Ruby AWS::S3.

Setup Ruby

Execute the following steps to setup Ruby:

As root, install ruby:
```
# yum install ruby
```
Note
The above command will install ruby and it’s essential dependencies like rubygems and ruby-libs too. If somehow the command doesn’t install all the dependencies, install them separately.
As root, install aws-s3:
```
# gem install aws-s3
```

Creating a connection

Create a project directory:
```
$ mkdir ruby_aws_s3
$ cd ruby_aws_s3
```
Create the connection file:
```
$ vim conn.rb
```

Paste the following contents into the conn.rb file:

#!/usr/bin/env ruby

require 'aws/s3'
require 'resolv-replace'

AWS::S3::Base.establish_connection!(
        :server            => '<FQDN_of_gateway_node>',
        :port           => '7480',
        :access_key_id     => 'my-access-key',
        :secret_access_key => 'my-secret-key'
)

Replace <FQDN_of_gateway_node> with the FQDN of you gateway node. Replace my-access-key and my-secret-key with the access_key and secret_key that was generated when you created the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Guide.

An example connection file will look like the following:

#!/usr/bin/env ruby

require 'aws/s3'

require 'resolv-replace'

AWS::S3::Base.establish_connection!(
        :server            => 'testclient.englab.pnq.redhat.com',
        :port           => '7480',
        :access_key_id     => '98J4R9P22P5CDL65HKP8',
        :secret_access_key => '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049'
)

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x conn.rb
```
Run the file:
```
$ ./conn.rb | echo $?
```
If you have provided the values correctly in the file, the output of the command will be 0.

Creating a bucket

Create a new file:

$ vim create_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.create('my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x create_bucket.rb
```
Run the file:
```
$ ./create_bucket.rb
```
If the output of the command is true it would mean that bucket my-new-bucket1 was created successfully.

Listing owned buckets

Create a new file:

$ vim list_owned_buckets.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Service.buckets.each do |bucket|
        puts "#{bucket.name}\t#{bucket.creation_date}"
end

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x list_owned_buckets.rb
```

Run the file:

$ ./list_owned_buckets.rb

The output should look something like this:

my-new-bucket1 2016-01-21 10:33:19 UTC

Creating an object

Create a new file:

$ vim create_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::S3Object.store(
        'hello.txt',
        'Hello World!',
        'my-new-bucket1',
        :content_type => 'text/plain'
)

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x create_object.rb
```
Run the file:
```
$ ./create_object.rb
```
This will create a file hello.txt with the string Hello World!.

Listing a Bucket’s Content

Create a new file:

$ vim list_bucket_content.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

new_bucket = AWS::S3::Bucket.find('my-new-bucket1')
new_bucket.each do |object|
        puts "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}"
end

Save the file and exit the editor.

Make the file executable.
```
$ chmod +x list_bucket_content.rb
```

Run the file:

$ ./list_bucket_content.rb

The output will look something like this:

hello.txt    12    Fri, 22 Jan 2016 15:54:52 GMT

Deleting a empty bucket

Create a new file:

$ vim del_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.delete('my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x del_empty_bucket.rb
```
Run the file:
```
$ ./del_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.
Note
Please edit the create_bucket.rb file to create empty buckets like my-new-bucket9, my-new-bucket10 etc and edit the above mentioned del_empty_bucket.rb file accordingly before trying to delete empty buckets.

Deleting a non-empty bucket (forcefully)

Create a new file:

$ vim del_non_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.delete('my-new-bucket1', :force => true)

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x del_non_empty_bucket.rb
```
Run the file:
```
$ ./del_non_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.

Deleting an object

Create a new file:

$ vim delete_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::S3Object.delete('hello.txt', 'my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
$ chmod +x delete_object.rb
```
Run the file:
```
$ ./delete_object.rb
```
This will delete the object hello.txt.

2.2.3. Ruby AWS::SDK Examples (aws-sdk gem ~>2)

You can use the Ruby programming language along with aws-sdk gem for S3 access. Execute the steps mentioned below on the node used for accessing the Ceph Object Gateway server with Ruby AWS::SDK.

Setup Ruby

Execute the following steps to setup Ruby:

As root, install ruby:
```
# yum install ruby
```
Note
The above command will install ruby and it’s essential dependencies like rubygems and ruby-libs too. If somehow the command doesn’t install all the dependencies, install them separately.
As root, install aws-sdk:
```
# gem install aws-sdk
```

Creating a connection

Create a project directory:
```
$ mkdir ruby_aws_sdk
$ cd ruby_aws_sdk
```
Create the connection file:
```
$ vim conn.rb
```

Paste the following contents into the conn.rb file:

#!/usr/bin/env ruby

require 'aws-sdk'
require 'resolv-replace'

Aws.config.update(
        endpoint: 'http://<FQDN_of_gateway_node>:7480',
        access_key_id: 'my-access-key',
        secret_access_key: 'my-secret-key',
        force_path_style: true,
        region: 'us-east-1'
)

An example connection file will look like the following:

#!/usr/bin/env ruby

require 'aws-sdk'
require 'resolv-replace'

Aws.config.update(
        endpoint: 'http://testclient.englab.pnq.redhat.com:7480',
        access_key_id: '98J4R9P22P5CDL65HKP8',
        secret_access_key: '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049',
        force_path_style: true,
        region: 'us-east-1'
)

Save the file and exit the editor.

Make the file executable:
```
chmod +x conn.rb
```
Run the file:
```
./conn.rb | echo $?
```
If you have provided the values correctly in the file, the output of the command will be 0.

Creating a bucket

Create a new file:

vim create_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.create_bucket(bucket: 'my-new-bucket2')

Save the file and exit the editor.

Make the file executable:
```
chmod +x create_bucket.rb
```
Run the file:
```
./create_bucket.rb
```
If the output of the command is true it would mean that bucket my-new-bucket2 was created successfully.

Listing owned buckets

Create a new file:

vim list_owned_buckets.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.list_buckets.buckets.each do |bucket|
        puts "#{bucket.name}\t#{bucket.creation_date}"
end

Save the file and exit the editor.

Make the file executable:
```
chmod +x list_owned_buckets.rb
```

Run the file:

./list_owned_buckets.rb

The output should look something like this:

my-new-bucket2 2016-01-21 10:33:19 UTC

Creating an object

Create a new file:

vim create_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.put_object(
        key: 'hello.txt',
        body: 'Hello World!',
        bucket: 'my-new-bucket2',
        content_type: 'text/plain'
)

Save the file and exit the editor.

Make the file executable:
```
chmod +x create_object.rb
```
Run the file:
```
./create_object.rb
```
This will create a file hello.txt with the string Hello World!.

Listing a Bucket’s Content

Create a new file:

vim list_bucket_content.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.list_objects(bucket: 'my-new-bucket2').contents.each do |object|
        puts "#{object.key}\t#{object.size}"
end

Save the file and exit the editor.

Make the file executable.
```
chmod +x list_bucket_content.rb
```

Run the file:

./list_bucket_content.rb

The output will look something like this:

hello.txt    12    Fri, 22 Jan 2016 15:54:52 GMT

Deleting a empty bucket

Create a new file:

vim del_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.delete_bucket(bucket: 'my-new-bucket2')

Save the file and exit the editor.

Make the file executable:
```
chmod +x del_empty_bucket.rb
```
Run the file:
```
./del_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.
Note
Please edit the create_bucket.rb file to create empty buckets like my-new-bucket6, my-new-bucket7 etc and edit the above mentioned del_empty_bucket.rb file accordingly before trying to delete empty buckets.

Deleting a non-empty bucket (forcefully)

Create a new file:

vim del_non_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
Aws::S3::Bucket.new('my-new-bucket2', client: s3_client).clear!
s3_client.delete_bucket(bucket: 'my-new-bucket2')

Save the file and exit the editor.

Make the file executable:
```
chmod +x del_non_empty_bucket.rb
```
Run the file:
```
./del_non_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.

Deleting an object

Create a new file:

vim delete_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

s3_client = Aws::S3::Client.new
s3_client.delete_object(key: 'hello.txt', bucket: 'my-new-bucket2')

Save the file and exit the editor.

Make the file executable:
```
chmod +x delete_object.rb
```
Run the file:
```
./delete_object.rb
```
This will delete the object hello.txt.

2.2.4. PHP S3 Examples

You can use PHP scripts too for S3 access. Execute the steps mentioned below on the node used for accessing the Ceph Object Gateway server with PHP.

Important

The examples given below are tested against php v5.4.16 and aws-sdk v2.8.24. DO NOT use the latest version of aws-sdk for php as it requires php >= 5.5+. php 5.5 is not available in the default repos of RHEL 7. If you want to use php 5.5, you will have to enable epel and other third party repos. Also, the configuration options for php 5.5 and latest version of aws-sdk are different.

Setup PHP/AWS SDK

Execute the following steps to setup PHP:

As root, install php:
```
# yum install php
```
Install aws-sdk for php:
Download the zip archive of aws-sdk for php and extract it.

Creating a connection

Create a project directory:
```
$ mkdir php_s3
$ cd php_s3
```
Copy the extracted aws directory to the project directory. For example:
```
$ cp -r ~/Downloads/aws/ ~/php_s3/
```
Create the connection file:
```
$ vim conn.php
```

Paste the following contents in the conn.php file:

<?php
define('AWS_KEY', 'my_access_key');
define('AWS_SECRET_KEY', 'my_secret_key');
define('HOST', '<FQDN_of_gateway_node>');
define('PORT', '7480');

// require the AWS SDK for php library
require '/path_to_aws/aws-autoloader.php';

use Aws\S3\S3Client;

// Establish connection with host using S3 Client
$client = S3Client::factory(array(
    'base_url' => HOST,
    'port' => PORT,
    'key'      => AWS_KEY,
    'secret'   => AWS_SECRET_KEY
));
?>

Replace <FQDN_of_gateway_node> with the FQDN of the gateway node. Replace my-access-key and my-secret-key with the access_key and secret_key that was generated when you created the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Guide. Also, replace path_to_aws with absolute path to the extracted aws directory that you copied to the php project directory.

An example connection file will look like the following:

<?php
define('AWS_KEY', '{key}');
define('AWS_SECRET_KEY', '{secret}');
define('HOST', 'http://{hostname}');

// require the AWS SDK for php library
require '/home/ceph/php_s3/aws/aws-autoloader.php';

use Aws\S3\S3Client;

// Establish connection with host using S3 Client
$client = S3Client::factory(array(
    'base_url' => HOST,
    'port' => PORT,
    'key'      => AWS_KEY,
    'secret'   => AWS_SECRET_KEY
));
?>

Save the file and exit the editor.

Run the file:
```
$ php -f conn.php | echo $?
```
If you have provided the values correctly in the file, the output of the command will be 0.

Creating a bucket

Create a new file:

vim create_bucket.php

Paste the following contents into the file:

<?php

include 'conn.php';

$client->createBucket(array('Bucket' => 'my-new-bucket3'));

?>

Save the file and exit the editor.

Run the file:
```
php -f create_bucket.php
```

Listing owned buckets

Create a new file:

vim list_owned_buckets.php

Paste the following content into the file:

<?php

include 'conn.php';

$blist = $client->listBuckets();
echo "   Buckets belonging to " . $blist['Owner']['ID'] . ":\n";
foreach ($blist['Buckets'] as $b) {
    echo "{$b['Name']}\t{$b['CreationDate']}\n";
}

?>

Save the file and exit the editor.

Run the file:

php -f list_owned_buckets.php

The output should look something like this:

my-new-bucket3 2016-01-21 10:33:19 UTC

Creating an object

Create a source file hello.txt:
```
echo "Hello World!" > hello.txt
```

Create a new php file:

vim create_object.php

Paste the following contents into the file:

<?php

include 'conn.php';

$key         = 'hello.txt';
$source_file = './hello.txt';
$acl         = 'private';
$bucket      = 'my-new-bucket3';
$client->upload($bucket, $key, fopen($source_file, 'r'), $acl);

?>

Save the file and exit the editor.

Run the file:
```
php -f create_object.php
```
This will create the object hello.txt in bucket my-new-bucket3.

Listing a Bucket’s Content

Create a new file:

vim list_bucket_content.php

Paste the following content into the file:

<?php

include 'conn.php';

$o_iter = $client->getIterator('ListObjects', array(
    'Bucket' => 'my-new-bucket3'
));
foreach ($o_iter as $o) {
    echo "{$o['Key']}\t{$o['Size']}\t{$o['LastModified']}\n";
}
?>

Save the file and exit the editor.

Run the file:

php -f list_bucket_content.php

The output will look something like this:

hello.txt    12    Fri, 22 Jan 2016 15:54:52 GMT

Deleting an empty bucket

Create a new file:

vim del_empty_bucket.php

Paste the following contents into the file:

<?php

include 'conn.php';

$client->deleteBucket(array('Bucket' => 'my-new-bucket3'));
?>

Save the file and exit the editor.

Run the file:
```
php -f del_empty_bucket.php | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.
Note
Edit the create_bucket.php file to create empty buckets like my-new-bucket4, my-new-bucket5 etc and edit the above mentioned del_empty_bucket.php file accordingly before trying to delete empty buckets.

Deleting a non-empty bucket (forcefully)

Deleting a non-empty bucket is not currently supported in php 2 and newer versions of aws-sdk.

Deleting an object

Create a new file:

vim delete_object.php

Paste the following contents into the file:

<?php

include 'conn.php';

$client->deleteObject(array(
    'Bucket' => 'my-new-bucket3',
    'Key'    => 'hello.txt',
));
?>

Save the file and exit the editor.

Run the file:
```
php -f delete_object.php
```
This will delete the object hello.txt.

2.3. Common Operations

2.3.1. Bucket and Host Name

There are two different modes of accessing the buckets. The first, and preferred method identifies the bucket as the top-level directory in the URI.

Example

GET /mybucket HTTP/1.1
Host: cname.domain.com

The second method identifies the bucket via a virtual bucket host name.

Example

GET / HTTP/1.1
Host: mybucket.cname.domain.com

Tip

Red Hat prefers the first method, because the second method requires expensive domain certification and DNS wild cards.

2.3.2. Common Request Headers

The following table lists the valid common request headers and their descriptions.

Table 2.4. Request Headers
Request Header	Description
`CONTENT_LENGTH`	Length of the request body.
`DATE`	Request time and date (in UTC).
`HOST`	The name of the host server.
`AUTHORIZATION`	Authorization token.

2.3.3. Common Response Status

The following table lists the valid common HTTP response status and its corresponding code.

Table 2.5. Response Status
HTTP Status	Response Code
`100`	Continue
`200`	Success
`201`	Created
`202`	Accepted
`204`	NoContent
`206`	Partial content
`304`	NotModified
`400`	InvalidArgument
`400`	InvalidDigest
`400`	BadDigest
`400`	InvalidBucketName
`400`	InvalidObjectName
`400`	UnresolvableGrantByEmailAddress
`400`	InvalidPart
`400`	InvalidPartOrder
`400`	RequestTimeout
`400`	EntityTooLarge
`403`	AccessDenied
`403`	UserSuspended
`403`	RequestTimeTooSkewed
`404`	NoSuchKey
`404`	NoSuchBucket
`404`	NoSuchUpload
`405`	MethodNotAllowed
`408`	RequestTimeout
`409`	BucketAlreadyExists
`409`	BucketNotEmpty
`411`	MissingContentLength
`412`	PreconditionFailed
`416`	InvalidRange
`422`	UnprocessableEntity
`500`	InternalError

2.4. Service Operations

2.4.1. List Buckets

GET / returns a list of buckets created by the user making the request. GET / only returns buckets created by an authenticated user. You cannot make an anonymous request.

Syntax

GET / HTTP/1.1
Host: cname.domain.com

Authorization: AWS <access_key>:<hash_of_header_and_secret>

Table 2.6. Response Entities
Name	Type	Description
`Buckets`	Container	Container for list of buckets.
`Bucket`	Container	Container for bucket information.
`Name`	String	Bucket name.
`CreationDate`	Date	UTC time when the bucket was created.
`ListAllMyBucketsResult`	Container	A container for the result.
`Owner`	Container	A container for the bucket owner’s `ID` and `DisplayName`.
`ID`	String	The bucket owner’s ID.
`DisplayName`	String	The bucket owner’s display name.

Name	Type	Description
`prefix`	String	Only returns objects that contain the specified prefix.
`delimiter`	String	The delimiter between the prefix and the rest of the object name.
`marker`	String	A beginning index for the list of objects returned.
`max-keys`	Integer	The maximum number of keys to return. Default is 1000.

Name	Type	Description
`ListBucketResult`	Entity	The container for the list of objects.
`Name`	String	The name of the bucket whose contents will be returned.
`Prefix`	String	A prefix for the object keys.
`Marker`	String	A beginning index for the list of objects returned.
`MaxKeys`	Integer	The maximum number of keys returned.
`Delimiter`	String	If set, objects with the same prefix will appear in the `CommonPrefixes` list.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s contents were returned.
`CommonPrefixes`	Container	If multiple objects contain the same prefix, they will appear in this list.

Name	Type	Description
`Contents`	Object	A container for the object.
`Key`	String	The object’s key.
`LastModified`	Date	The object’s last-modified date/time.
`ETag`	String	An MD-5 hash of the object. (entity tag)
`Size`	Integer	The object’s size.
`StorageClass`	String	Should always return `STANDARD`.

Name	Type	Description
`VersioningConfiguration`	container	A container for the request.
`Status`	String	Sets the versioning state of the bucket. Valid Values: Suspended/Enabled

Name	Type	Description
`AccessControlPolicy`	Container	A container for the response.
`AccessControlList`	Container	A container for the ACL information.
`Owner`	Container	A container for the bucket owner’s `ID` and `DisplayName`.
`ID`	String	The bucket owner’s ID.
`DisplayName`	String	The bucket owner’s display name.
`Grant`	Container	A container for `Grantee` and `Permission`.
`Grantee`	Container	A container for the `DisplayName` and `ID` of the user receiving a grant of permission.
`Permission`	String	The permission given to the `Grantee` bucket.

Name	Type	Description
`prefix`	String	Returns in-progress uploads whose keys contains the specified prefix.
`delimiter`	String	The delimiter between the prefix and the rest of the object name.
`key-marker`	String	The beginning marker for the list of uploads.
`max-keys`	Integer	The maximum number of in-progress uploads. The default is 1000.
`version-id-marker`	String	Specifies the object version to begin the list.

Name	Type	Description
`KeyMarker`	String	The key marker specified by the `key-marker` request parameter (if any).
`NextKeyMarker`	String	The key marker to use in a subsequent request if `IsTruncated` is `true`.
`NextUploadIdMarker`	String	The upload ID marker to use in a subsequent request if `IsTruncated` is `true`.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s upload contents were returned.
`Size`	Integer	The size of the uploaded part.
`DisplayName`	String	The owners’s display name.
`ID`	String	The owners’s ID.
`Owner`	Container	A container for the `ID` and `DisplayName` of the user who owns the object.
`StorageClass`	String	The method used to store the resulting object. `STANDARD` or `REDUCED_REDUNDANCY`
`Version`	Container	Container for the version information.
`versionId`	String	Version ID of an object.
`versionIdMarker`	String	The last version of the key in a truncated response.

Name	Type	Description
`ListMultipartUploadsResult`	Container	A container for the results.
`ListMultipartUploadsResult.Prefix`	String	The prefix specified by the `prefix` request parameter (if any).
`Bucket`	String	The bucket that will receive the bucket contents.
`KeyMarker`	String	The key marker specified by the `key-marker` request parameter (if any).
`UploadIdMarker`	String	The marker specified by the `upload-id-marker` request parameter (if any).
`NextKeyMarker`	String	The key marker to use in a subsequent request if `IsTruncated` is `true`.
`NextUploadIdMarker`	String	The upload ID marker to use in a subsequent request if `IsTruncated` is `true`.
`MaxUploads`	Integer	The max uploads specified by the `max-uploads` request parameter.
`Delimiter`	String	If set, objects with the same prefix will appear in the `CommonPrefixes` list.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s upload contents were returned.
`Upload`	Container	A container for `Key`, `UploadId`, `InitiatorOwner`, `StorageClass`, and `Initiated` elements.
`Key`	String	The key of the object once the multipart upload is complete.
`UploadId`	String	The `ID` that identifies the multipart upload.
`Initiator`	Container	Contains the `ID` and `DisplayName` of the user who initiated the upload.
`DisplayName`	String	The initiator’s display name.
`ID`	String	The initiator’s ID.
`Owner`	Container	A container for the `ID` and `DisplayName` of the user who owns the uploaded object.
`StorageClass`	String	The method used to store the resulting object. `STANDARD` or `REDUCED_REDUNDANCY`
`Initiated`	Date	The date and time the user initiated the upload.
`CommonPrefixes`	Container	If multiple objects contain the same prefix, they will appear in this list.
`CommonPrefixes.Prefix`	String	The substring of the key after the prefix as defined by the `prefix` request parameter.

Name	Type	Description
`Payer`	Enum	Specifies who pays for the download and request fees.
`RequestPaymentConfiguration`	Container	A container for `Payer`.

Name	Description	Valid Values	Required
content-md5	A base64 encoded MD-5 hash of the message.	A string. No defaults or constraints.	No
content-type	A standard MIME type.	Any MIME type. Default: `binary/octet-stream`	No
x-amz-meta-<…>	User metadata. Stored with the object.	A string up to 8kb. No defaults.	No
x-amz-acl	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No

Table 2.26. Request Headers
Name	Description	Valid Values	Required
range	The range of the object to retrieve.	Range: bytes=beginbyte-endbyte	No
if-modified-since	Gets only if modified since the timestamp.	Timestamp	No
if-unmodified-since	Gets only if not modified since the timestamp.	Timestamp	No
if-match	Gets only if object ETag matches ETag.	Entity Tag	No
if-none-match	Gets only if object ETag matches ETag.	Entity Tag	No

Table 2.28. Request Headers
Name	Description	Valid Values	Required
range	The range of the object to retrieve.	Range: bytes=beginbyte-endbyte	No
if-modified-since	Gets only if modified since the timestamp.	Timestamp	No
if-unmodified-since	Gets only if not modified since the timestamp.	Timestamp	No
if-match	Gets only if object ETag matches ETag.	Entity Tag	No
if-none-match	Gets only if object ETag matches ETag.	Entity Tag	No

Name	Description	Valid Values	Required
x-amz-copy-source	The source bucket name + object name.	<bucket>/<object>	Yes
x-amz-acl	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No
x-amz-copy-if-modified-since	Copies only if modified since the timestamp.	Timestamp	No
x-amz-copy-if-unmodified-since	Copies only if unmodified since the timestamp.	Timestamp	No
x-amz-copy-if-match	Copies only if object ETag matches ETag.	Entity Tag	No
x-amz-copy-if-none-match	Copies only if object ETag doesn’t match.	Entity Tag	No

Name	Type	Description
CopyObjectResult	Container	A container for the response elements.
LastModified	Date	The last modified date of the source object.
Etag	String	The ETag of the new object.

Name	Description
Content-Range	Data range, will only be returned if the range header field was specified in the request
x-amz-version-id	Returns the version ID or null.

Name	Description	Valid Values	Required
`content-md5`	A base64 encoded MD-5 hash of the message.	A string. No defaults or constraints.	No
`content-type`	A standard MIME type.	Any MIME type. Default: `binary/octet-stream`	No
`x-amz-meta-<…>`	User metadata. Stored with the object.	A string up to 8kb. No defaults.	No
`x-amz-acl`	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No

Name	Type	Description
`InitiatedMultipartUploadsResult`	Container	A container for the results.
`Bucket`	String	The bucket that will receive the object contents.
`Key`	String	The key specified by the `key` request parameter (if any).
`UploadId`	String	The ID specified by the `upload-id` request parameter identifying the multipart upload (if any).

Name	Type	Description	Required
`CompleteMultipartUpload`	Container	A container consisting of one or more parts.	Yes
`Part`	Container	A container for the `PartNumber` and `ETag`.	Yes
`PartNumber`	Integer	The identifier of the part.	Yes
`ETag`	String	The part’s entity tag.	Yes

Name	Type	Description
`CompleteMultipartUploadResult`	Container	A container for the response.
`Location`	URI	The resource identifier (path) of the new object.
`Bucket`	String	The name of the bucket that contains the new object.
`Key`	String	The object’s key.
`ETag`	String	The entity tag of the new object.

Name	Description	Valid Values	Required
`x-amz-copy-source`	The source bucket name and object name.	<bucket>/<object>	Yes
`x-amz-copy-source-range`	The range of bytes to copy from the source object.	Range: `bytes=first-last`, where the first and last are the zero-based byte offsets to copy. For example, `bytes=0-9` indicates that you want to copy the first ten bytes of the source.	No

Name	Type	Description
`CopyPartResult`	Container	A container for all response elements.
`ETag`	String	Returns the ETag of the new part.
`LastModified`	String	Returns the date the part was last modified.

Chapter 2. Object Gateway S3 Application Programming Interface (API)

2.1. Authentication and Access Control Lists

2.1.1. Authentication

2.1.2. Access Control Lists (ACLs)

2.2. Accessing the Gateway

2.2.1. Prerequisites

2.2.2. Ruby AWS::S3 Examples (aws-s3 gem)

2.2.3. Ruby AWS::SDK Examples (aws-sdk gem ~>2)

2.2.4. PHP S3 Examples

2.3. Common Operations

2.3.1. Bucket and Host Name

2.3.2. Common Request Headers

2.3.3. Common Response Status

2.4. Service Operations

2.4.1. List Buckets

2.5. Bucket Operations

2.5.1. Bucket Operations with Multi Tenancy

2.5.2. PUT Bucket

2.5.3. DELETE Bucket

2.5.4. GET Bucket

2.5.5. Get Bucket Location

2.5.6. Get Bucket Versioning

2.5.7. PUT Bucket Versioning

2.5.8. Get Bucket ACLs

2.5.9. PUT Bucket ACLs

2.5.10. GET Bucket cors

2.5.11. PUT Bucket cors

2.5.12. DELETE Bucket cors

2.5.13. List Bucket Object Versions

2.5.14. List Bucket Multipart Uploads

2.5.15. PUT Bucket Request Payment

2.5.16. GET Bucket Request Payment

2.6. Object Operations

2.6.1. PUT Object

2.6.2. Copy Object

2.6.3. POST Object

2.6.4. OPTIONS Object

2.6.5. Delete Multiple Objects

2.6.6. Remove Object

2.6.7. Get Object

2.6.8. Get Object Information

2.6.9. Get Object ACL

2.6.10. Set Object ACL

2.6.11. Initiate Multipart Upload

2.6.12. Multipart Upload Part

2.6.13. List Multipart Upload Parts

2.6.14. Complete Multipart Upload

2.6.15. Abort Multipart Upload

2.6.16. Copy Multipart Upload

2.7. Hadoop S3A Interoperability

2.8. S3 Limitations

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

Making open source more inclusive

About Red Hat

Red Hat legal and privacy links

Red Hat legal and privacy links