このコンテンツは選択した言語では利用できません。
6.5. Managing Object Store
6.5.1. Architecture Overview
- OpenStack Object Storage environment.For detailed information on Object Storage, see OpenStack Object Storage Administration Guide available at: http://docs.openstack.org/admin-guide-cloud/content/ch_admin-openstack-object-storage.html.
- Red Hat Gluster Storage environment.Red Hat Gluster Storage environment consists of bricks that are used to build volumes. For more information on bricks and volumes, see Section 5.4, “Formatting and Mounting Bricks”.
Figure 6.2. Object Store Architecture
Important
# firewall-cmd --get-active-zones
# firewall-cmd --zone=zone_name --add-port=6010/tcp --add-port=6011/tcp --add-port=6012/tcp --add-port=8080/tcp # firewall-cmd --zone=zone_name --add-port=6010/tcp --add-port=6011/tcp --add-port=6012/tcp --add-port=8080/tcp --permanent
only
if your swift proxy server is configured with SSL. To add the port number, run the following commands:
# firewall-cmd --zone=zone_name --add-port=443/tcp # firewall-cmd --zone=zone_name --add-port=443/tcp --permanent
6.5.2. Components of Object Store
- Authenticate Object Store against an external OpenStack Keystone server.Each Red Hat Gluster Storage volume is mapped to a single account. Each account can have multiple users with different privileges based on the group and role they are assigned to. After authenticating using accountname:username and password, user is issued a token which will be used for all subsequent REST requests.Integration with Keystone
When you integrate Red Hat Gluster Storage Object Store with Keystone authentication, you must ensure that the Swift account name and Red Hat Gluster Storage volume name are the same. It is common that Red Hat Gluster Storage volumes are created before exposing them through the Red Hat Gluster Storage Object Store.
When working with Keystone, account names are defined by Keystone as thetenant id
. You must create the Red Hat Gluster Storage volume using the Keystonetenant id
as the name of the volume. This means, you must create the Keystone tenant before creating a Red Hat Gluster Storage Volume.Important
Red Hat Gluster Storage does not contain any Keystone server components. It only acts as a Keystone client. After you create a volume for Keystone, ensure to export this volume for accessing it using the object storage interface. For more information on exporting volume, see Section 6.5.7.8, “Exporting the Red Hat Gluster Storage Volumes”.Integration with GSwauthGSwauth is a Web Server Gateway Interface (WGSI) middleware that uses a Red Hat Gluster Storage Volume itself as its backing store to maintain its metadata. The benefit in this authentication service is to have the metadata available to all proxy servers and saving the data to a Red Hat Gluster Storage volume.
To protect the metadata, the Red Hat Gluster Storage volume should only be able to be mounted by the systems running the proxy servers. For more information on mounting volumes, see Chapter 6, Creating Access to Volumes.Integration with TempAuthYou can also use the
TempAuth
authentication service to test Red Hat Gluster Storage Object Store in the data center.
6.5.3. Advantages of using Object Store
- Default object size limit of 1 TiB
- Unified view of data across NAS and Object Storage technologies
- High availability
- Scalability
- Replication
- Elastic Volume Management
6.5.4. Limitations
- Object NameObject Store imposes the following constraints on the object name to maintain the compatibility with network file access:
- Object names must not be prefixed or suffixed by a '/' character. For example,
a/b/
- Object names must not have contiguous multiple '/' characters. For example,
a//b
- Account Management
- Object Store does not allow account management even though OpenStack Swift allows the management of accounts. This limitation is because Object Store treats
accounts
equivalent to the Red Hat Gluster Storage volumes. - Object Store does not support account names (i.e. Red Hat Gluster Storage volume names) having an underscore.
- In Object Store, every account must map to a Red Hat Gluster Storage volume.
- Subdirectory ListingHeaders
X-Content-Type: application/directory
andX-Content-Length: 0
can be used to create subdirectory objects under a container, but GET request on a subdirectory would not list all the objects under it.
6.5.5. Swift API Support Matrix
Feature | Status |
---|---|
Authentication | Supported |
Get Account Metadata | Supported |
Swift ACLs | Supported |
List Containers | Supported |
Delete Container | Supported |
Create Container | Supported |
Get Container Metadata | Supported |
Update Container Metadata | Supported |
Delete Container Metadata | Supported |
List Objects | Supported |
Static Website | Supported |
Create/Update an Object | Supported |
Create Large Object | Supported |
Delete Object | Supported |
Get Object | Supported |
Copy Object | Supported |
Get Object Metadata | Supported |
Add/Update Object Metadata | Supported |
Temp URL Operations | Supported |
Expiring Objects | Supported |
Object Versioning | Supported |
Cross-Origin Resource Sharing (CORS) | Supported |
Bulk Upload | Supported |
Account Quota | Unsupported |
Container Quota | Unsupported |
6.5.6. Prerequisites
- Ensure that the openstack-swift-* and swiftonfile packages have matching version numbers.
# rpm -qa | grep swift openstack-swift-container-1.13.1-6.el7ost.noarch openstack-swift-object-1.13.1-6.el7ost.noarch swiftonfile-1.13.1-6.el7rhgs.noarch openstack-swift-proxy-1.13.1-6.el7ost.noarch openstack-swift-doc-1.13.1-6.el7ost.noarch openstack-swift-1.13.1-6.el7ost.noarch openstack-swift-account-1.13.1-6.el7ost.noarch
- Ensure that the gluster-swift services are owned by and run as the
root
user, not theswift
user as in a typical OpenStack installation.# cd /usr/lib/systemd/system # sed -i s/User=swift/User=root/ openstack-swift-proxy.service openstack-swift-account.service openstack-swift-container.service openstack-swift-object.service openstack-swift-object-expirer.service
- Start the
memcached
service:# service memcached start
- Ensure that the ports for the Object, Container, Account, and Proxy servers are open. Note that the ports used for these servers are configurable. The ports listed in Table 6.11, “Ports required for Red Hat Gluster Storage Object Store” are the default values.
Table 6.11. Ports required for Red Hat Gluster Storage Object Store Server Port Object Server 6010 Container Server 6011 Account Server 6012 Proxy Server (HTTPS) 443 Proxy Server (HTTP) 8080 - Create and mount a Red Hat Gluster Storage volume for use as a Swift Account. For information on creating Red Hat Gluster Storage volumes, see Chapter 5, Setting Up Storage Volumes . For information on mounting Red Hat Gluster Storage volumes, see Chapter 6, Creating Access to Volumes .
6.5.7. Configuring the Object Store
Warning
/etc/swift
directory would contain both *.conf
extension and *.conf-gluster
files. You must delete the *.conf
files and create new configuration files based on *.conf-gluster
template. Otherwise, inappropriate python packages will be loaded and the component may not work as expected.
.rpmnew
extension. You must ensure to delete .conf
files and folders (account-server, container-server, and object-server) for better understanding of the loaded configuration.
6.5.7.1. Configuring a Proxy Server
etc/swift/proxy-server.conf
by referencing the template file available at /etc/swift/proxy-server.conf-gluster
.
6.5.7.1.1. Configuring a Proxy Server for HTTPS
- Create self-signed cert for SSL using the following commands:
# cd /etc/swift # openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
- Add the following lines to
/etc/swift/proxy-server.conf
under [DEFAULT]bind_port = 443 cert_file = /etc/swift/cert.crt key_file = /etc/swift/cert.key
Important
memcache_servers
configuration option in the proxy-server.conf
and list all memcached servers.
proxy-server.conf
file.
[filter:cache] use = egg:swift#memcache memcache_servers = 192.168.1.20:11211,192.168.1.21:11211,192.168.1.22:11211
6.5.7.2. Configuring the Authentication Service
Keystone, GSwauth,
and TempAuth
authentication services.
6.5.7.2.1. Integrating with the Keystone Authentication Service
- To configure Keystone, add
authtoken
andkeystoneauth
to/etc/swift/proxy-server.conf
pipeline as shown below:[pipeline:main] pipeline = catch_errors healthcheck proxy-logging cache authtoken keystoneauth proxy-logging proxy-server
- Add the following sections to
/etc/swift/proxy-server.conf
file by referencing the example below as a guideline. You must substitute the values according to your setup:[filter:authtoken] paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory signing_dir = /etc/swift auth_host = keystone.server.com auth_port = 35357 auth_protocol = http auth_uri = http://keystone.server.com:5000 # if its defined admin_tenant_name = services admin_user = swift admin_password = adminpassword delay_auth_decision = 1 [filter:keystoneauth] use = egg:swift#keystoneauth operator_roles = admin, SwiftOperator is_admin = true cache = swift.cache
Verify that the Red Hat Gluster Storage Object Store has been configured successfully by running the following command:
$ swift -V 2 -A http://keystone.server.com:5000/v2.0 -U tenant_name:user -K password stat
6.5.7.2.2. Integrating with the GSwauth Authentication Service
Perform the following steps to integrate GSwauth:
- Create and start a Red Hat Gluster Storage volume to store metadata.
# gluster volume create NEW-VOLNAME NEW-BRICK # gluster volume start NEW-VOLNAME
For example:# gluster volume create gsmetadata server1:/rhgs/brick1 # gluster volume start gsmetadata
- Run
gluster-swift-gen-builders
tool with all the volumes to be accessed using the Swift client includinggsmetadata
volume:# gluster-swift-gen-builders gsmetadata other volumes
- Edit the
/etc/swift/proxy-server.conf
pipeline as shown below:[pipeline:main] pipeline = catch_errors cache gswauth proxy-server
- Add the following section to
/etc/swift/proxy-server.conf
file by referencing the example below as a guideline. You must substitute the values according to your setup.[filter:gswauth] use = egg:gluster_swift#gswauth set log_name = gswauth super_admin_key = gswauthkey metadata_volume = gsmetadata auth_type = sha1 auth_type_salt = swauthsalt
Important
You must ensure to secure theproxy-server.conf
file and thesuper_admin_key
option to prevent unprivileged access. - Restart the proxy server by running the following command:
#
swift-init proxy restart
You can set the following advanced options for GSwauth WSGI filter:
- default-swift-cluster: The default storage-URL for the newly created accounts. When you attempt to authenticate for the first time, the access token and the storage-URL where data for the given account is stored will be returned.
- token_life: The set default token life. The default value is 86400 (24 hours).
- max_token_life: The maximum token life. You can set a token lifetime when requesting a new token with header
x-auth-token-lifetime
. If the passed in value is greater than themax_token_life
, then themax_token_life
value will be used.
GSwauth provides CLI tools to facilitate managing accounts and users. All tools have some options in common:
- -A, --admin-url: The URL to the auth. The default URL is
http://127.0.0.1:8080/auth/
. - -U, --admin-user: The user with administrator rights to perform action. The default user role is
.super_admin
. - -K, --admin-key: The key for the user with administrator rights to perform the action. There is no default value.
Prepare the Red Hat Gluster Storage volume for gswauth
to save its metadata by running the following command:
# gswauth-prep [option]
# gswauth-prep -A http://10.20.30.40:8080/auth/ -K gswauthkey
6.5.7.2.2.1. Managing Account Services in GSwauth
Create an account for GSwauth. This account is mapped to a Red Hat Gluster Storage volume.
# gswauth-add-account [option] <account_name>
# gswauth-add-account -K gswauthkey <account_name>
You must ensure that all users pertaining to this account must be deleted before deleting the account. To delete an account:
# gswauth-delete-account [option] <account_name>
# gswauth-delete-account -K gswauthkey test
Sets a service URL for an account. User with reseller admin
role only can set the service URL. This command can be used to change the default storage URL for a given account. All accounts will have the same storage-URL as default value, which is set using default-swift-cluster
option.
# gswauth-set-account-service [options] <account> <service> <name> <value>
# gswauth-set-account-service -K gswauthkey test storage local http://newhost:8080/v1/AUTH_test
6.5.7.2.2.2. Managing User Services in GSwauth
The following user roles are supported in GSwauth:
- A regular user has no rights. Users must be given both read and write privileges using Swift ACLs.
- The
admin
user is a super-user at the account level. This user can create and delete users for that account. These members will have both write and read privileges to all stored objects in that account. - The
reseller admin
user is a super-user at the cluster level. This user can create and delete accounts and users and has read and write privileges to all accounts under that cluster. - GSwauth maintains its own swift account to store all of its metadata on accounts and users. The
.super_admin
role provides access to GSwauth own swift account and has all privileges to act on any other account or user.
Role/Group | Allowed Actions |
---|---|
.super_admin (username) |
|
.reseller_admin (group) |
|
.admin (group) |
|
regular user (type) | No administrative actions. |
You can create an user for an account that does not exist. The account will be created before creating the user.
-r
flag to create a reseller admin
user and -a
flag to create an admin
user. To change the password or role of the user, you can run the same command with the new option.
# gswauth-add-user [option] <account_name> <user> <password>
# gswauth-add-user -K gswauthkey -a test ana anapwd
Delete a user by running the following command:
# gswauth-delete-user [option] <account_name> <user>
# gwauth-delete-user -K gswauthkey test ana
There are two methods to access data using the Swift client. The first and simple method is by providing the user name and password everytime. The swift client will acquire the token from gswauth.
$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:ana -K anapwd upload container1 README.md
# curl -v -H 'X-Storage-User: test:ana' -H 'X-Storage-Pass: anapwd' -k http://localhost:8080/auth/v1.0 ... < X-Auth-Token: AUTH_tk7e68ef4698f14c7f95af07ab7b298610 < X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test ...
$ swift --os-auth-token=AUTH_tk7e68ef4698f14c7f95af07ab7b298610 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 README.md README.md bash-4.2$ bash-4.2$ swift --os-auth-token=AUTH_tk7e68ef4698f14c7f95af07ab7b298610 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test list container1 README.md
Important
Reseller admins
must always use the second method to acquire a token to get access to other accounts other than his own. The first method of using the username and password will give them access only to their own accounts.
6.5.7.2.2.3. Managing Accounts and Users Information
You can obtain the accounts and users information including stored password.
# gswauth-list [options] [account] [user]
# gswauth-list -K gswauthkey test ana +----------+ | Groups | +----------+ | test:ana | | test | | .admin | +----------+
- If [account] and [user] are omitted, all the accounts will be listed.
- If [account] is included but not [user], a list of users within that account will be listed.
- If [account] and [user] are included, a list of groups that the user belongs to will be listed.
- If the [user] is .groups, the active groups for that account will be listed.
-p
option provides the output in plain text format, -j
provides the output in JSON format.
You can change the password of the user, account administrator, and reseller_admin roles.
- Change the password of a regular user by running the following command:
# gswauth-add-user -U account1:user1 -K old_passwd account1 user1 new_passwd
- Change the password of an
account administrator
by running the following command:# gswauth-add-user -U account1:admin -K old_passwd -a account1 admin new_passwd
- Change the password of the
reseller_admin
by running the following command:# gswauth-add-user -U account1:radmin -K old_passwd -r account1 radmin new_passwd
Users with .super_admin
role can delete the expired tokens.
# gswauth-cleanup-tokens [options]
# gswauth-cleanup-tokens -K gswauthkey --purge test
- -t, --token-life: The expected life of tokens. The token objects modified before the give number of seconds will be checked for expiration (default: 86400).
- --purge: Purges all the tokens for a given account whether the tokens have expired or not.
- --purge-all: Purges all the tokens for all the accounts and users whether the tokens have expired or not.
6.5.7.2.3. Integrating with the TempAuth Authentication Service
Warning
cleartext
in a single proxy-server.conf
file. In your /etc/swift/proxy-server.conf
file, enable TempAuth in pipeline and add user information in TempAuth
section by referencing the below example.
[pipeline:main] pipeline = catch_errors healthcheck proxy-logging cache tempauth proxy-logging proxy-server [filter:tempauth] use = egg:swift#tempauth user_admin_admin = admin.admin.reseller_admin user_test_tester = testing .admin user_test_tester2 = testing2
user_accountname_username = password [.admin]
accountname
is the Red Hat Gluster Storage volume used to store objects.
6.5.7.3. Configuring Object Servers
etc/swift/object.server.conf
by referencing the template file available at /etc/swift/object-server.conf-gluster
.
6.5.7.4. Configuring Container Servers
etc/swift/container-server.conf
by referencing the template file available at /etc/swift/container-server.conf-gluster
.
6.5.7.5. Configuring Account Servers
etc/swift/account-server.conf
by referencing the template file available at /etc/swift/account-server.conf-gluster
.
6.5.7.6. Configuring Swift Object and Container Constraints
/etc/swift/swift.conf
by referencing the template file available at /etc/swift/swift.conf-gluster
.
6.5.7.7. Configuring Object Expiration
Note
object-expirer
daemon. This is an expected behavior.
6.5.7.7.1. Setting Up Object Expiration
gsexpiring
for managing object expiration. Hence, you must create a Red Hat Gluster Storage volume and name it as gsexpiring
.
/etc/swift/object.expirer.conf
by referencing the template file available at /etc/swift/object-expirer.conf-gluster
.
6.5.7.7.2. Using Object Expiration
The X-Delete-At header requires a UNIX epoch timestamp, in integer form. For example, 1418884120 represents Thu, 18 Dec 2014 06:27:31 GMT. By setting the header to a specific epoch time, you indicate when you want the object to expire, not be served, and be deleted completely from the Red Hat Gluster Storage volume. The current time in Epoch notation can be found by running this command:
$ date +%s
- Set the object expiry time during an object PUT with X-Delete-At header using cURL:
# curl -v -X PUT -H 'X-Delete-At: 1392013619' http://127.0.0.1:8080/v1/AUTH_test/container1/object1 -T ./localfile
Set the object expiry time during an object PUT with X-Delete-At header using swift client:# swift --os-auth-token=AUTH_tk99a39aecc3dd4f80b2b1e801d00df846 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 ./localfile --header 'X-Delete-At: 1392013619'
The X-Delete-After header takes an integer number of seconds that represents the amount of time from now when you want the object to be deleted.
- Set the object expiry time with an object PUT with X-Delete-After header using cURL:
# curl -v -X PUT -H 'X-Delete-After: 3600' http://127.0.0.1:8080/v1/AUTH_test/container1/object1 -T ./localfile
Set the object expiry time with an object PUT with X-Delete-At header using swift client:# swift --os-auth-token=AUTH_tk99a39aecc3dd4f80b2b1e801d00df846 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 ./localfile --header 'X-Delete-After: 3600'
6.5.7.7.3. Running Object Expirer Service
interval
option in /etc/swift/object-expirer.conf
file. For every pass it makes, it queries the gsexpiring account for tracker objects. Based on the timestamp and path present in the name of tracker objects, object-expirer deletes the actual object and the corresponding tracker object.
# swift-init object-expirer start
# swift-object-expirer -o -v /etc/swift/object-expirer.conf
6.5.7.8. Exporting the Red Hat Gluster Storage Volumes
Swift on File
component.
# cd /etc/swift # gluster-swift-gen-builders VOLUME [VOLUME...]
# cd /etc/swift # gluster-swift-gen-builders testvol1 testvol2 testvol3
/mnt/gluster-object
). The default value can be changed to a different path by changing the devices
configurable option across all account, container, and object configuration files. The path must contain Red Hat Gluster Storage volumes mounted under directories having the same names as volume names. For example, if devices
option is set to /home
, it is expected that the volume named testvol1
be mounted at /home/testvol1
.
gluster-swift-gen-builders
tool even if it was previously added. The gluster-swift-gen-builders
tool creates new ring files every time it runs successfully.
gluster-swift-gen-builders
only with the volumes which are required to be accessed using the Swift interface.
testvol2
volume, run the following command:
# gluster-swift-gen-builders testvol1 testvol3
6.5.7.9. Starting and Stopping Server
- To start the server, run the following command:
# swift-init main start
- To stop the server, run the following command:
# swift-init main stop
- To restart the server, run the following command:
# swift-init main restart
6.5.8. Starting the Services Automatically
# chkconfig memcached on # chkconfig openstack-swift-proxy on # chkconfig openstack-swift-account on # chkconfig openstack-swift-container on # chkconfig openstack-swift-object on # chkconfig openstack-swift-object-expirer on
# systemctl enable openstack-swift-proxy.service # systemctl enable openstack-swift-account.service # systemctl enable openstack-swift-container.service # systemctl enable openstack-swift-object.service # systemctl enable openstack-swift-object-expirer.service # systemctl enable openstack-swift-object-expirer.service
systemctl
command may require additional configuration. Refer to https://access.redhat.com/solutions/2043773 for details if you encounter problems.
Important
6.5.9. Working with the Object Store
6.5.9.1. Creating Containers and Objects
6.5.9.2. Creating Subdirectory under Containers
Content-Type: application/directory
and Content-Length: 0
. However, the current behavior of Object Store returns 200 OK
on a GET
request on subdirectory but this does not list all the objects under that subdirectory.