Chapter 3. Major Updates

A new structure for detecting duplicate operations

Under certain circumstances, it is better to recover by using the backfill process instead of log-based recovery. The most direct way to force backfilling is to reduce the size of the placement group log. This setting, however, undermines the detection of duplicate operations. This update introduces a separate structure to detect duplicate operations beyond the placement group log entries. As a result, duplicate operations are detected as expected.

RocksDB is enabled as an option to replace levelDB

This update enables an option to use the RocksDB back end for the omap database as opposed to levelDB. RocksDB uses the multi-threading mechanism in compaction so that it better handles the situation when the omap directories become very large (more than 40 G). LevelDB compaction takes a lot of time in such a situation and causes OSD daemons to time out.

Scrubbing is blocked for any PG if the primary or any replica OSDs are recovering

The osd_scrub_during_recovery parameter now defaults to false, so that when an OSD is recovering, the scrubbing process is not initialized on it. Previously, osd_scrub_during_recovery was set to true by default allowing scrubbing and recovery to run simultaneously. In addition, in previous releases if the user set osd_scrub_during_recovery to false, only the primary OSD was checked for recovery activity.

A new compact command

With this update, the OSD administration socket supports the compact command. A large number of omap create and delete operations can cause the normal compaction of the levelDB database during those operations to be too slow to keep up with the workload. As a result, levelDB can grow very large and inhibit performance. The compact command compacts the omap database (levelDB or RocksDB) to a smaller size to provide more consistent performance.

Improved delete handling

With this update, delete handling has been improved and it is enabled by default in new clusters. To enable this improvement in previously deployed clusters, use the ceph osd set recovery_deletes command after upgrading to version 2.4:

A new Compatibility Guide

A new Compatibility Guide is now available. The guide provides a matrix that lists which versions of various products are compatible with this version of Red Hat Ceph Storage. See the Compatibility Guide for details.