Installation Guide for Ubuntu

Check the current pid_max settings:

# cat /proc/sys/kernel/pid_max

As root, consider setting kernel.pid_max to a higher number of threads. The theoretical maximum is 4,194,303 threads. For example, add the following to the /etc/sysctl.conf file to set it to the maximum value:

kernel.pid_max = 4194303

As root, to load the changes without a rebooting:

# sysctl -p

As root, verify the changes:

# sysctl -a | grep kernel.pid_max

Install the ceph-ansible package:

$ sudo apt-get install ceph-ansible

As root, add the Ceph hosts to the /etc/ansible/hosts file. Remember to comment out example hosts.

If the Ceph hosts have sequential naming, consider using a range:

As the Ansible user, ensure that Ansible can reach the Ceph hosts:

$ ansible all -m ping

Create a directory under the home directory so Ansible can write the keys:

# cd ~
# mkdir ceph-ansible-keys

As root, create a symbolic link to the Ansible group_vars directory in the /etc/ansible/ directory:

# ln -s /usr/share/ceph-ansible/group_vars /etc/ansible/group_vars

As root, create an all.yml file from the all.yml.sample file and open it for editing:

# cd /etc/ansible/group_vars
# cp all.yml.sample all.yml
# vim all.yml

Uncomment the fetch_directory setting under the GENERAL section. Then, point it to the directory you created in step 1:

fetch_directory: ~/ceph-ansible-keys

Uncomment the ceph_repository_type setting and set it to either cdn or iso:

ceph_repository_type: cdn

Select the installation method. There are two approaches:

For RCHS 2.5 and later versions, uncomment and set ceph_rhcs_cdn_debian_repo and ceph_rhcs_cdn_debian_repo_version so that Ansible can automatically enable and access Ubuntu online repositories:

ceph_rhcs_cdn_debian_repo: <repo-path>
ceph_rhcs_cdn_debian_repo_version: <repo-version>

ceph_rhcs_cdn_debian_repo: https://<login>:<pwd>@rhcs.download.redhat.com
ceph_rhcs_cdn_debian_repo_version: /2-release/

Where <login> is the RHN user login and <pwd> is the RHN user’s password.

Set the generate_fsid setting to false:

generate_fsid: false

fsid: <generated_uuid>

To enable authentication, uncomment the cephx setting under the Ceph Configuration section. Red Hat recommends running Ceph with authentication enabled:

cephx: true

Uncomment the monitor_interface setting and specify the network interface:

monitor_interface:

monitor_interface: eth0

If not using IPv6, then skip this step. Uncomment and set the ip_version option:

ip_version: ipv6

Set journal size:

journal_size: <size_in_MB>

If not filled, the default journal size will be 5 GB. See Journal Settings for additional details.

Set the public network. Optionally, set the cluster network, too:

public_network: <public_network>
cluster_network: <cluster_network>

See Section 2.4, “Configuring Network” and Network Configuration Reference for additional details.

If not using IPv6, then skip this step. Uncomment and set the radosgw_civetweb_bind_ip option:

radosgw_civetweb_bind_ip: "[{{ ansible_default_ipv6.address }}]"

radosgw_civetweb_bind_ip: “[<left_curly_bracket><left_curly_bracket> ansible_default_ipv6.address <right_curly_bracket><right_curly_bracket>]"

Navigate to the /etc/ansible/group_vars/ directory:

# cd /etc/ansible/group_vars/

As root, create an mons.yml file from mons.yml.sample file and open it for editing:

# cp mons.yml.sample mons.yml
# vim mons.yml

To enable the Calamari API, uncomment the calamari setting and set it to true:

calamari: true

Navigate to the /etc/ansible/group_vars/ directory:

# cd /etc/ansible/group_vars/

As root, create a new osds.yml file from the osds.yml.sample file and open it for editing:

# cp osds.yml.sample osds.yml
# vim osds.yml

Navigate to the Ansible configuration directory:

# cd /usr/share/ceph-ansible

As root, create a site.yml file from the site.yml.sample file:

# cp site.yml.sample site.yml

As the Ansible user, run the Ansible playbook from within the directory where the playbook exists:

$ ansible-playbook site.yml [-u <user_name>]

Once the playbook runs, it creates a running Ceph cluster.

As root, on the Ceph Monitor nodes, create a Calamari user:

# calamari-ctl add_user --password <password> --email <email_address> <user_name>

# calamari-ctl add_user --password abc123 --email user1@example.com user1

On your initial Monitor node, install the ceph-mon package as root:

$ sudo apt-get install ceph-mon

As root, create a Ceph configuration file in the /etc/ceph/ directory. By default, Ceph uses ceph.conf, where ceph reflects the cluster name:

# touch /etc/ceph/<cluster_name>.conf

# touch /etc/ceph/ceph.conf

As root, generate the unique identifier for your cluster and add the unique identifier to the [global] section of the Ceph configuration file:

# echo "[global]" > /etc/ceph/<cluster_name>.conf
# echo "fsid = `uuidgen`" >> /etc/ceph/<cluster_name>.conf

# echo "[global]" > /etc/ceph/ceph.conf
# echo "fsid = `uuidgen`" >> /etc/ceph/ceph.conf

View the current Ceph configuration file:

$ cat /etc/ceph/ceph.conf
[global]
fsid = a7f64266-0894-4f1e-a635-d0aeaca0e993

As root, add the initial Monitor to the Ceph configuration file:

# echo "mon initial members = <monitor_host_name>[,<monitor_host_name>]" >> /etc/ceph/<cluster_name>.conf

# echo "mon initial members = node1" >> /etc/ceph/ceph.conf

As root, add the IP address of the initial Monitor to the Ceph configuration file:

# echo "mon host = <ip-address>[,<ip-address>]" >> /etc/ceph/<cluster_name>.conf

# echo "mon host = 192.168.0.120" >> /etc/ceph/ceph.conf

As root, create the keyring for the cluster and generate the Monitor secret key:

# ceph-authtool --create-keyring /tmp/<cluster_name>.mon.keyring --gen-key -n mon. --cap mon '<capabilites>'

# ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'
creating /tmp/ceph.mon.keyring

As root, generate an administrator keyring, generate a <cluster_name>.client.admin.keyring user and add the user to the keyring:

# ceph-authtool --create-keyring /etc/ceph/<cluster_name>.client.admin.keyring --gen-key -n client.admin --set-uid=0 --cap mon '<capabilites>' --cap osd '<capabilites>' --cap mds '<capabilites>'

# ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow'
creating /etc/ceph/ceph.client.admin.keyring

As root, add the <cluster_name>.client.admin.keyring key to the <cluster_name>.mon.keyring:

# ceph-authtool /tmp/<cluster_name>.mon.keyring --import-keyring /etc/ceph/<cluster_name>.client.admin.keyring

# ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring
importing contents of /etc/ceph/ceph.client.admin.keyring into /tmp/ceph.mon.keyring

Generate the Monitor map. Specify using the node name, IP address and the fsid, of the initial Monitor and save it as /tmp/monmap:

$ monmaptool --create --add <monitor_host_name> <ip-address> --fsid <uuid> /tmp/monmap

$ monmaptool --create --add node1 192.168.0.120 --fsid a7f64266-0894-4f1e-a635-d0aeaca0e993 /tmp/monmap
monmaptool: monmap file /tmp/monmap
monmaptool: set fsid to a7f64266-0894-4f1e-a635-d0aeaca0e993
monmaptool: writing epoch 0 to /tmp/monmap (1 monitors)

As root on the initial Monitor node, create a default data directory:

# mkdir /var/lib/ceph/mon/<cluster_name>-<monitor_host_name>

# mkdir /var/lib/ceph/mon/ceph-node1

As root, populate the initial Monitor daemon with the Monitor map and keyring:

# ceph-mon [--cluster <cluster_name>] --mkfs -i <monitor_host_name> --monmap /tmp/monmap --keyring /tmp/<cluster_name>.mon.keyring

# ceph-mon --mkfs -i node1 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
ceph-mon: set fsid to a7f64266-0894-4f1e-a635-d0aeaca0e993
ceph-mon: created monfs at /var/lib/ceph/mon/ceph-node1 for mon.node1

View the current Ceph configuration file:

# cat /etc/ceph/ceph.conf
[global]
fsid = a7f64266-0894-4f1e-a635-d0aeaca0e993
mon_initial_members = node1
mon_host = 192.168.0.120

For more details on the various Ceph configuration settings, see the Red Hat Ceph Storage Configuration Guide. The following example of a Ceph configuration file lists some of the most common configuration settings:

[global]
fsid = <cluster-id>
mon initial members = <monitor_host_name>[, <monitor_host_name>]
mon host = <ip-address>[, <ip-address>]
public network = <network>[, <network>]
cluster network = <network>[, <network>]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd journal size = <n>
filestore xattr use omap = true
osd pool default size = <n>  # Write an object n times.
osd pool default min size = <n> # Allow writing n copy in a degraded state.
osd pool default pg num = <n>
osd pool default pgp num = <n>
osd crush chooseleaf type = <n>

As root, create the done file:

# touch /var/lib/ceph/mon/<cluster_name>-<monitor_host_name>/done

# touch /var/lib/ceph/mon/ceph-node1/done

As root, update the owner and group permissions on the newly created directory and files:

# chown -R <owner>:<group> <path_to_directory>

# chown -R ceph:ceph /var/lib/ceph/mon
# chown -R ceph:ceph /var/log/ceph
# chown -R ceph:ceph /var/run/ceph
# chown ceph:ceph /etc/ceph/ceph.client.admin.keyring
# chown ceph:ceph /etc/ceph/ceph.conf
# chown ceph:ceph /etc/ceph/rbdmap

# ls -l /etc/ceph/
...
-rw-------.  1 glance glance      64 <date> ceph.client.glance.keyring
-rw-------.  1 cinder cinder      64 <date> ceph.client.cinder.keyring
...

For storage clusters with custom names, as root, add the the following line:

$ sudo echo "CLUSTER=<custom_cluster_name>" >> /etc/default/ceph

$ sudo echo "CLUSTER=test123" >> /etc/default/ceph

As root, start and enable the ceph-mon process on the initial Monitor node:

$ sudo systemctl enable ceph-mon.target
$ sudo systemctl enable ceph-mon@<monitor_host_name>
$ sudo systemctl start ceph-mon@<monitor_host_name>

$ sudo systemctl enable ceph-mon.target
$ sudo systemctl enable ceph-mon@node1
$ sudo systemctl start ceph-mon@node1

Verify that Ceph created the default pools:

$ ceph osd lspools
0 rbd,

Verify that the Monitor is running. The status output will look similar to the following example. The Monitor is up and running, but the cluster health will be in a HEALTH_ERR state. This error is indicating that placement groups are stuck and inactive. Once OSDs are added to the cluster and active, the placement group health errors will disappear.

$ ceph -s
cluster a7f64266-0894-4f1e-a635-d0aeaca0e993
health HEALTH_ERR 192 pgs stuck inactive; 192 pgs stuck unclean; no osds
monmap e1: 1 mons at {node1=192.168.0.120:6789/0}, election epoch 1, quorum 0 node1
osdmap e1: 0 osds: 0 up, 0 in
pgmap v2: 192 pgs, 3 pools, 0 bytes data, 0 objects
0 kB used, 0 kB / 0 kB avail
192 creating

As root, install the ceph-osd package on the Ceph OSD node:

$ sudo apt-get install ceph-osd

Copy the Ceph configuration file and administration keyring file from the initial Monitor node to the OSD node:

# scp <user_name>@<monitor_host_name>:<path_on_remote_system> <path_to_local_file>

# scp root@node1:/etc/ceph/ceph.conf /etc/ceph
# scp root@node1:/etc/ceph/ceph.client.admin.keyring /etc/ceph

Generate the Universally Unique Identifier (UUID) for the OSD:

$ uuidgen
b367c360-b364-4b1d-8fc6-09408a9cda7a

As root, create the OSD instance:

# ceph osd create <uuid> [<osd_id>]

# ceph osd create b367c360-b364-4b1d-8fc6-09408a9cda7a
0

As root, create the default directory for the new OSD:

# mkdir /var/lib/ceph/osd/<cluster_name>-<osd_id>

# mkdir /var/lib/ceph/osd/ceph-0

As root, prepare the drive for use as an OSD, and mount it to the directory you just created. Create a partition for the Ceph data and journal. The journal and the data partitions can be located on the same disk. This example is using a 15 GB disk:

# parted <path_to_disk> mklabel gpt
# parted <path_to_disk> mkpart primary 1 10000
# mkfs -t <fstype> <path_to_partition>
# mount -o noatime <path_to_partition> /var/lib/ceph/osd/<cluster_name>-<osd_id>
# echo "<path_to_partition>  /var/lib/ceph/osd/<cluster_name>-<osd_id>   xfs defaults,noatime 1 2" >> /etc/fstab

# parted /dev/sdb mklabel gpt
# parted /dev/sdb mkpart primary 1 10000
# parted /dev/sdb mkpart primary 10001 15000
# mkfs -t xfs /dev/sdb1
# mount -o noatime /dev/sdb1 /var/lib/ceph/osd/ceph-0
# echo "/dev/sdb1 /var/lib/ceph/osd/ceph-0  xfs defaults,noatime 1 2" >> /etc/fstab

As root, initialize the OSD data directory:

# ceph-osd -i <osd_id> --mkfs --mkkey --osd-uuid <uuid>

# ceph-osd -i 0 --mkfs --mkkey --osd-uuid b367c360-b364-4b1d-8fc6-09408a9cda7a
... auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory
... created new key in keyring /var/lib/ceph/osd/ceph-0/keyring

As root, register the OSD authentication key. If your cluster name differs from ceph, insert your cluster name instead:

# ceph auth add osd.<osd_id> osd 'allow *' mon 'allow profile osd' -i /var/lib/ceph/osd/<cluster_name>-<osd_id>/keyring

# ceph auth add osd.0 osd 'allow *' mon 'allow profile osd' -i /var/lib/ceph/osd/ceph-0/keyring
added key for osd.0

As root, add the OSD node to the CRUSH map:

# ceph [--cluster <cluster_name>] osd crush add-bucket <host_name> host

# ceph osd crush add-bucket node2 host

As root, place the OSD node under the default CRUSH tree:

# ceph [--cluster <cluster_name>] osd crush move <host_name> root=default

# ceph osd crush move node2 root=default

As root, add the OSD disk to the CRUSH map

# ceph [--cluster <cluster_name>] osd crush add osd.<osd_id> <weight> [<bucket_type>=<bucket-name> ...]

# ceph osd crush add osd.0 1.0 host=node2
add item id 0 name 'osd.0' weight 1 at location {host=node2} to crush map

As root, update the owner and group permissions on the newly created directory and files:

# chown -R <owner>:<group> <path_to_directory>

# chown -R ceph:ceph /var/lib/ceph/osd
# chown -R ceph:ceph /var/log/ceph
# chown -R ceph:ceph /var/run/ceph
# chown -R ceph:ceph /etc/ceph

For storage clusters with custom names, as root, add the following line to the /etc/default/ceph file:

$ sudo echo "CLUSTER=<custom_cluster_name>" >> /etc/default/ceph

$ sudo echo "CLUSTER=test123" >> /etc/default/ceph

The OSD node is in your Ceph storage cluster configuration. However, the OSD daemon is down and in. The new OSD must be up before it can begin receiving data. As root, enable and start the OSD process:

$ sudo systemctl enable ceph-osd.target
$ sudo systemctl enable ceph-osd@<osd_id>
$ sudo systemctl start ceph-osd@<osd_id>

$ sudo systemctl enable ceph-osd.target
$ sudo systemctl enable ceph-osd@0
$ sudo systemctl start ceph-osd@0

Once you start the OSD daemon, it is up and in.

As root, install calamari-server:

$ sudo apt-get install calamari-server

As root, initialize the calamari-server:

$ sudo calamari-ctl clear --yes-i-am-sure
$ sudo calamari-ctl initialize --admin-username <uid> --admin-password <pwd> --admin-email <email>

$ sudo calamari-ctl clear --yes-i-am-sure
$ sudo calamari-ctl initialize --admin-username admin --admin-password admin --admin-email cephadm@example.com

Under the [calamari_web] section, modify ssl_cert and ssl_key to point to the respective certificate and key path, for example:

[calamari_web]
...
ssl_cert = /etc/calamari/ssl/certs/calamari-lite-bundled.crt
ssl_key = /etc/calamari/ssl/private/calamari-lite.key

As root, re-initialize Calamari:

$ sudo calamari-ctl initialize

On the Object Gateway node, install the radosgw package:

$ sudo apt-get install radosgw

On the initial Monitor node, do the following steps.

On the Object Gateway node, create the data directory:

$ sudo mkdir -p /var/lib/ceph/radosgw/<cluster_name>-rgw.`hostname -s`

$ sudo mkdir -p /var/lib/ceph/radosgw/ceph-rgw.`hostname -s`

On the Object Gateway node, add a user and keyring to bootstrap the object gateway:

$ sudo ceph auth get-or-create client.rgw.`hostname -s` osd 'allow rwx' mon 'allow rw' -o /var/lib/ceph/radosgw/<cluster_name>-rgw.`hostname -s`/keyring

$ sudo ceph auth get-or-create client.rgw.`hostname -s` osd 'allow rwx' mon 'allow rw' -o /var/lib/ceph/radosgw/ceph-rgw.`hostname -s`/keyring

On the Object Gateway node, create the done file:

$ sudo touch /var/lib/ceph/radosgw/<cluster_name>-rgw.`hostname -s`/done

$ sudo touch /var/lib/ceph/radosgw/ceph-rgw.`hostname -s`/done

On the Object Gateway node, change the owner and group permissions:

$ sudo chown -R ceph:ceph /var/lib/ceph/radosgw
$ sudo chown -R ceph:ceph /var/log/ceph
$ sudo chown -R ceph:ceph /var/run/ceph
$ sudo chown -R ceph:ceph /etc/ceph

For storage clusters with custom names, as root, add the the following line:

$ sudo echo "CLUSTER=<custom_cluster_name>" >> /etc/default/ceph

$ sudo echo "CLUSTER=test123" >> /etc/default/ceph

On the Object Gateway node, open TCP port 80:

$ sudo iptables -I INPUT 1 -i <network_interface> -p tcp -s <ip_address>/<netmask> --dport 80 -j ACCEPT

On the Object Gateway node, start and enable the ceph-radosgw process:

$ sudo systemctl enable ceph-radosgw.target
$ sudo systemctl enable ceph-radosgw@rgw.<rgw_hostname>
$ sudo systemctl start ceph-radosgw@rgw.<rgw_hostname>

$ sudo systemctl enable ceph-radosgw.target
$ sudo systemctl enable ceph-radosgw@rgw.node1
$ sudo systemctl start ceph-radosgw@rgw.node1

As root, disable any Red Hat Ceph Storage 1.3.x repositories:

If the following lines exist in the /etc/apt/sources.list or in the /etc/apt/sources.list.d/ceph.list files, then comment out the online repositories for Red Hat Ceph Storage 1.3 by adding a # to the beginning of the line.

deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Installer
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Calamari
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Tools

Also, check for the following files in /etc/apt/sources.list.d/:

Calamari.list
ceph-mon.list
ceph-osd.list
Installer.list
Tools.list

As root, stop the Monitor process:

$ sudo stop ceph-mon id=<monitor_host_name>

$ sudo stop ceph-mon id=node1

As root, update the ceph-mon package:

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo apt-get install ceph-mon

As root, update the owner and group permissions:

# chown -R <owner>:<group> <path_to_directory>

# chown -R ceph:ceph /var/lib/ceph/mon
# chown -R ceph:ceph /var/log/ceph
# chown -R ceph:ceph /var/run/ceph
# chown ceph:ceph /etc/ceph/ceph.client.admin.keyring
# chown ceph:ceph /etc/ceph/ceph.conf
# chown ceph:ceph /etc/ceph/rbdmap

# ls -l /etc/ceph/
...
-rw-------.  1 glance glance      64 <date> ceph.client.glance.keyring
-rw-------.  1 cinder cinder      64 <date> ceph.client.cinder.keyring
...

Remove packages that are no longer needed:

$ sudo apt-get purge ceph ceph-osd

As root, replay device events from the kernel:

# udevadm trigger

Upgrade from Ubuntu 14.04 Trusty to Ubuntu 16.04 Xenial.

As root, disable the Red Hat Ceph Storage 1.3 repositories:

If the following lines exist in the /etc/apt/sources.list or in the /etc/apt/sources.list.d/ceph.list files, then comment out the online repositories for Red Hat Ceph Storage 1.3 by adding a # to the beginning of the line.

deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Installer
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Calamari
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Tools

Also, check for the following files in /etc/apt/sources.list.d/:

Calamari.list
ceph-mon.list
ceph-osd.list
Installer.list
Tools.list

As root, stop any running OSD process:

$ sudo stop ceph-osd id=<osd_id>

$ sudo stop ceph-osd id=0

As root, update the ceph-osd package:

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo apt-get install ceph-osd

As root, update the owner and group permissions on the newly created directory and files:

# chown -R <owner>:<group> <path_to_directory>

# chown -R ceph:ceph /var/lib/ceph/osd
# chown -R ceph:ceph /var/log/ceph
# chown -R ceph:ceph /var/run/ceph
# chown -R ceph:ceph /etc/ceph

# find /var/lib/ceph/osd -maxdepth 1 -mindepth 1 -print | xargs -P12 -n1 chown -R ceph:ceph

Remove packages that are no longer needed:

$ sudo apt-get purge ceph ceph-mon

As root, replay device events from the kernel:

# udevadm trigger

Upgrade from Ubuntu 14.04 Trusty to Ubuntu 16.04 Xenial.

Move to the next OSD node.

$ ceph health
HEALTH_WARN noout,norebalance flag(s) set

If you used online repositories to install Red Hat Ceph Storage, disable the 1.3 repositories.

Stop the Ceph Object Gateway process (ceph-radosgw):

$ sudo stop radosgw id=rgw.<hostname>

Replace <hostname> with the name of Ceph Object Gateway host, for example gateway-node.

$ sudo stop radosgw id=rgw.node

Update the ceph-radosgw package:

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo apt-get install radosgw

Change the owner and group permissions on the newly created /var/lib/ceph/radosgw/ and /var/log/ceph/ directories and their content to ceph.

# chown -R ceph:ceph /var/lib/ceph/radosgw
# chown -R ceph:ceph /var/log/ceph

Remove packages that are no longer needed.

$ sudo apt-get purge ceph

Upgrade from Ubuntu 14.04 to 16.04.

Enable the ceph-radosgw process:

$ sudo systemctl enable ceph-radosgw.target
$ sudo systemctl enable ceph-radosgw@rgw.<hostname>

Replace <hostname> with the name of the Ceph Object Gateway host, for example gateway-node.

$ sudo systemctl enable ceph-radosgw.target
$ sudo systemctl enable ceph-radosgw@rgw.gateway-node

Reboot the Ceph Object Gateway node:

# shutdown -r now

As root, disable any Red Hat Ceph Storage 1.3 repositories:

If the following lines exist in the /etc/apt/sources.list or in the /etc/apt/sources.list.d/ceph.list files, then comment out the online repositories for Red Hat Ceph Storage 1.3 by adding a # to the beginning of the line.

deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Installer
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Calamari
deb https://<customer_name>:<customer_password>@rhcs.download.redhat.com/ubuntu/1.3-updates/Tools

Also, check for the following files in /etc/apt/sources.list.d/:

Calamari.list
ceph-mon.list
ceph-osd.list
Installer.list
Tools.list

On the client node, update the ceph-common package:

$ sudo apt-get install ceph-common