Chapter 7. Recommended Minimum Hardware for Containerized Ceph Clusters

ceph-osd-container

Processor

1x AMD64 or Intel 64 CPU CORE per OSD container

Nodes

Minimum of three nodes required.

RAM

Minimum of 5 GB of RAM per OSD container

OS Disk

1x OS disk per host

OSD Storage

1x storage drive per OSD container. Cannot be shared with OS Disk.

Journal

1x SSD/NVME partition per daemon if using dedicated device

block.db

Optional, but Red Hat recommended, 1x SSD or NVMe or Optane partition or lvm per daemon. Sizing is 4% of block.data for BlueStore

block.wal

Optionally, 1x SSD or NVMe or Optane partition or logical volume per daemon. Use a small size, for example 10 GB, and only if it’s faster than the block.db device.

Network

2x 1GB Ethernet NICs, 10GB Recommended

ceph-mon-container

Processor

1x AMD64 or Intel 64 CPU CORE per mon-container

Nodes

Minimum of three nodes required. These daemons can run colocated when containerized, thus we only require a minimum of 3 physical nodes when running containerized.

RAM

3 GB per mon-container

Disk Space

10 GB per mon-container, 50GB Recommended

Monitor Disk

Optionally, 1x SSD disk for Monitor rocksdb data

Network

2x 1GB Ethernet NICs, 10GB Recommended

ceph-mgr-container

Processor

1x AMD64 or Intel 64 CPU CORE per mgr-container

RAM

3 GB per mgr-container

Network

2x 1GB Ethernet NICs, 10GB Recommended

ceph-radosgw-container

Processor

1x AMD64 or Intel 64 CPU CORE per radosgw-container

RAM

1 GB per daemon

Disk Space

5 GB per daemon

Network

1x 1GB Ethernet NICs

ceph-mds-container

Processor

1x AMD64 or Intel 64 CPU CORE per mds-container

RAM

3 GB per mds-container

This number is highly dependent on the configurable MDS cache size. The RAM requirement is typically twice as much as the amount set in the mds_cache_memory_limit configuration setting. Note also that this is the memory for your daemon, not the overall system memory.

Disk Space

2GB per mds-container, plus taking into consideration any additional space required for possible debug logging (20GB is a good start).

Network

2x 1GB Ethernet NICs, 10GB Recommended

Note that this is the same network as the OSD containers. If you have a 10 GB network on your OSDs you should use the same on your MDS so that the MDS is not disadvantaged when it comes to latency.