Chapter 2. Configure Ceph Object Gateway

Each instance must have a user name and key to communicate with a Ceph Storage Cluster. In the following steps, we use an admin node to create a keyring. Then, we create a client user name and key. Next, we add the key to the Ceph Storage Cluster. Finally, we distribute the key ring to the node containing the gateway instance i.e, gateway host.

Monitor Key CAPS

When you provide CAPS to the key, you MUST provide read capability. However, you have the option of providing write capability for the monitor. This is an important choice. If you provide write capability to the key, the Ceph Object Gateway will have the ability to create pools automatically; however, it will create pools with either the default number of placement groups (not ideal) or the number of placement groups you specified in your Ceph configuration file. If you allow the Ceph Object Gateway to create pools automatically, ensure that you have reasonable defaults for the number of placement groups first.

Execute the following steps on the admin node of your cluster:

Ceph Object Gateways require Ceph Storage Cluster pools to store specific gateway data. If the user you created in the preceding section has permissions, the gateway will create the pools automatically. However, you should ensure that you have set an appropriate default number of placement groups per pool in your Ceph configuration file, or create the pools manually with an appropriate number of placement groups. See Ceph Placement Groups (PGs) per Pool Calculator for details.

The Ceph Object Gateway requires multiple pools, and they can use the same CRUSH ruleset. The mon_pg_warn_max_per_osd setting warns you if assign too many placement groups to a pool (i.e., 300 by default). You may adjust the value to suit your needs and the capabilities of your hardware where n is the maximum number of PGs per OSD.

mon_pg_warn_max_per_osd = n

mon_pg_warn_max_per_osd = n

When configuring a gateway with the default region and zone, the naming convention for pools typically omits region and zone naming, but you can use any naming convention you prefer. For example:

As already said, if write permission is given, Ceph Object Gateway will create pools automatically. To create a pool manually, execute the following:

ceph osd pool create {poolname} {pg-num} {pgp-num}

ceph osd pool create {poolname} {pg-num} {pgp-num}

When you have completed this step, execute the following to ensure that you have created all of the foregoing pools:

rados lspools

rados lspools

Add the Ceph Object Gateway configuration to your Ceph Configuration file in admin node. The Ceph Object Gateway configuration requires you to identify the Ceph Object Gateway instance. Then, you must specify the host name where you installed the Ceph Object Gateway daemon, a keyring (for use with cephx), the socket path for FastCGI and a log file.

For RHEL 6, append the following configuration to /etc/ceph/ceph.conf in your admin node:

[client.radosgw.gateway]
host = {hostname}
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = ""
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw frontends = fastcgi socket_port=9000 socket_host=0.0.0.0
rgw print continue = false

[client.radosgw.gateway]
host = {hostname}
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = ""
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw frontends = fastcgi socket_port=9000 socket_host=0.0.0.0
rgw print continue = false

For RHEL 7, append the following configuration to /etc/ceph/ceph.conf in your admin node:

[client.radosgw.gateway]
host = {hostname}
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw print continue = false

[client.radosgw.gateway]
host = {hostname}
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw print continue = false

Here, {hostname} is the short hostname (output of command hostname -s) of the node that is going to provide the gateway service i.e, the gateway host.

The [client.radosgw.gateway] portion of the gateway instance identifies this portion of the Ceph configuration file as configuring a Ceph Storage Cluster client where the client type is a Ceph Object Gateway (i.e., radosgw).

Once you finish the setup procedure, if you encounter issues with your configuration, you can add debugging to the [global] section of your Ceph configuration file and restart the gateway to help troubleshoot any configuration issues. For example:

[global]
#append the following in the global section.
debug ms = 1
debug rgw = 20

[global]
#append the following in the global section.
debug ms = 1
debug rgw = 20

The updated Ceph configuration file needs to be distributed to all Ceph cluster nodes from the admin node.

It involves the following steps:

As the gateway host can be a different node that is not part of the cluster, the ceph.client.admin.keyring needs to be copied from the admin node to the gateway host. To do so, execute the following on admin node:

sudo scp /etc/ceph/ceph.client.admin.keyring  ceph@{hostname}:/home/ceph
ssh {hostname}
sudo mv ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring

sudo scp /etc/ceph/ceph.client.admin.keyring  ceph@{hostname}:/home/ceph
ssh {hostname}
sudo mv ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring

Deployment scripts may not create the default Ceph Object Gateway data directory. Create data directories for each instance of a radosgw daemon (if you haven’t done so already). The host variables in the Ceph configuration file determine which host runs each instance of a radosgw daemon. The typical form specifies the radosgw daemon, the cluster name and the daemon ID.

To create the directory on the gateway host, execute the following:

sudo mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway

sudo mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway

The radosgw daemon runs as the unprivileged apache UID, and this UID must have write access to the location where it will write its socket file.

To grant permissions to the default socket location, execute the following on the gateway host:

sudo chown apache:apache /var/run/ceph

sudo chown apache:apache /var/run/ceph

The Ceph Object gateway daemon needs to be started. To do so, execute the following on the gateway host:

On RHEL 7:

sudo systemctl start ceph-radosgw
sudo chkconfig ceph-radosgw on

sudo systemctl start ceph-radosgw
sudo chkconfig ceph-radosgw on

On RHEL 6:

sudo service ceph-radosgw start
sudo chkconfig ceph-radosgw on

sudo service ceph-radosgw start
sudo chkconfig ceph-radosgw on

The radosgw daemon runs as the unprivileged apache UID, but the root user owns the log file by default. You must change it to the apache user so that Apache can populate the log file.

sudo chown apache:apache /var/log/radosgw/client.radosgw.gateway.log

sudo chown apache:apache /var/log/radosgw/client.radosgw.gateway.log

On the host where you installed the Ceph Object Gateway i.e, gateway host, create an rgw.conf file. Place the file in the /etc/httpd/conf.d directory. It is a httpd configuration file which is needed for the radosgw service. This file must be readable by the web server.

Execute the following steps:

The httpd service needs to be restarted to accept the new configuration.

On RHEL 7, execute:

sudo systemctl restart httpd
sudo systemctl enable httpd

sudo systemctl restart httpd
sudo systemctl enable httpd

On RHEL 6, execute:

sudo service httpd restart
sudo chkconfig httpd on

sudo service httpd restart
sudo chkconfig httpd on