Chapter 3. New features
This section lists all major updates, enhancements, and new features introduced in this release of Red Hat Ceph Storage.
The main features added by this release are:
- Vault support
- Support for co-locating the Grafana container alongside the OSD container
- Ability to clone subvolumes from snapshots in CephFS
Vault support
With the Red Hat Ceph Storage 4.1 release, the Ceph object storage gateway (RGW) can now interoperate with the Hashicorp Vault secure key management service. As a storage administrator, you can securely store keys, passwords and certificates in the HashiCorp Vault for use with the Ceph Object Gateway. The HashiCorp Vault provides a secure key management service for server-side encryption used by the Ceph Object Gateway.
For more information, see the The HashiCorp Vault section in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide.
Support for co-locating the Grafana container
The Red Hat Ceph Storage 4.1 release supports the co-location of the Grafana container with Ceph OSD and an additional scale-out daemon, cardinality 2. Cardinality 2 was previously only supported with the Ceph Object Gateway in the Red Hat Ceph Storage 4.0 release.
For more information, see the Red Hat Ceph Supported Configurations article.
Cloning subvolumes from snapshots in CephFS
Subvolumes can be created by cloning subvolume snapshots. It is an asynchronous operation involving copying data from a snapshot to a subvolume.
For information about cloning subvolumes from snapshots in CephFS, see the Red Hat Ceph Storage File System Guide.
3.1. The Ceph Ansible utility
ceph-ansible now supports multisite deployments with multiple realms
Previously, ceph-ansible multisite deployments supported a single RGW realm. With this update, ceph-ansible now supports multiple realms with their associated zones, zonegroups, and endpoints.
For more information, see Configuring multisite Ceph Object Gateways in the Red Hat Ceph Storage Installation Guide.
The dedicated journal devices retain their configuration when migrating from Filestore OSD to Bluestore
Previously, dedicated journal devices for Filestore OSD could not be reused when migrating to Bluestore OSD DB. An example of a dedicated device configuration is using a HDD for data and an SSD for journaling.
With this update, dedicated journal devices retain their configuration during the migration, so that they can be reused with the Bluestore OSD DB.
For more information, see How to Migrate the Object Store from FileStore to BlueStore in the Administration Guide.
purge-container-cluster.yml playbook now supports clusters with three-digit IDs
Previously, purge-container-cluster only supported Red Hat Ceph Storage clusters with up to 99 OSDs. This is because the playbook supported ceph-osd services with two-digit indices. With this update, you can properly purge clusters with three-digit IDs.
OpenStack users can deploy Ceph Dashboard with a default admin account with read-only privileges
Previously, changes made from Ceph Dashboard by OpenStack users with full admin privileges could override cluster settings or status. With this feature, Ceph Dashboard admin account can only monitor Ceph cluster status and retrieve information and settings.
Added support for logrotate in containerized Red Hat Ceph Storage deployments
With this release, containerized Ceph logs are rotated using the logrotate program. This can help prevent the file system from filling up with log data.
3.2. Ceph File System
Independent life-cycle operations for subvolumes and subvolume snapshots
Because the CSI protocol treats snapshots as first class objects, this requires source subvolumes and subvolume snapshots to operate independently of each other. Since the Kubernetes storage interface uses the CSI protocol, subvolume removal with a snapshot retention option (--retain-snapshots) has been implemented. This allows other life-cycle operations on a retained snapshot to proceed appropriately.
3.3. The Ceph Volume utility
Ceph OSD encryption support when using ceph-volume in raw mode
With this release, the ceph-volume command can prepare Ceph OSDs for encryption using raw mode.
3.4. Distribution
An S3 client is included in the Ceph tools repository
Starting with this release, an S3 command-line client, s3cmd is included in the Red Hat Ceph Storage 4 Tools software repository. To install the S3 command-line client package, enable the rhceph-4-tools-for-rhel-8-x86_64-rpms repository first.
3.5. Ceph Object Gateway
Support for Amazon S3 resources in Ceph Object Gateway
AWS provides the Secure Token Service (STS) to allow secure federation with existing OpenID Connect/ OAuth2.0 compliant identity services such as Keycloak. STS is a standalone REST service that provides temporary tokens for an application or user to access a Simple Storage Service (S3) endpoint after the user authenticates against an identity provider (IDP).
Previously, users without permanent Amazon Web Services (AWS) credentials could not access S3 resources through Ceph Object Gateway. With this update, Ceph Object Gateway supports STS AssumeRoleWithWebIdentity. This service allows web application users who have been authenticated with an OpenID Connect/OAuth 2.0 compliant IDP to access S3 resources through Ceph Object Gateway.
For more information, see Secure Token Service in the Developer Guide.
AWS S3 ListObjects v2 operation provides an improved mechanism to list the objects in the S3 bucket without additional configuration
Previously, S3 protocol clients, like S3A and the awscli command-line tool, had to be configured with the older ListObjects method. With this feature, AWS S3 ListObjects v2 operation is implemented, that provides an improved mechanism to list objects in an S3 bucket.
Ceph Object Gateway’s default bucket-index shards increased to 11
The default number of bucket-index shards for new buckets has been increased, from 1 to 11. This increases the amount of write throughput for small buckets and delays the onset of dynamic resharding. This change only affects new deployments and zones.
For existing deployments, you can change this default value using the radosgw-admin zonegroup modify --bucket-index-max-shards=11 command. If the zonegroup is part of a realm, the change must be committed with radosgw-admin period update --commit command. If a commit is not done, then the change will take effect until after the Ceph Object Gateways are restarted.
After an upgrade, the new default value has to be increased manually, but on new Red Hat Ceph Storage 4.1 deployments the new default value is set automatically.
The Ceph Object Gateway log includes access log for Beast
With this release, Beast, the front-end web server, now includes an Apache-style access log line in the Ceph Object Gateway log. This update to the log helps diagnose connection and client network issues.
The minimum value of a session token’s expiration is configurable
The rgw_sts_min_session_duration option can now have a value lower than the default value of 900 seconds.
Listing the contents of large buckets
With this release, the ordered listing when delimited, and possible prefix, are specified more efficiently by skipping over object in lower pseudo-directories. This allows fewer interactions between the Ceph client and the Ceph Object Gateway, and also between the Ceph Object Gateway and the Ceph OSDs. This enhancement, along with configuration changes to HA proxy allows the listing of contents in large buckets.
The Ceph Object Gateway log includes the access log for Beast
With this release, Beast, the front-end web server, now includes an Apache-style access log line in the Ceph Object Gateway log. This update to the log helps diagnose connection and client network issues.
3.6. RADOS
Update to use ping times to track network performance
Previously, when network problems occur, it was difficult to distinguish from other performance issues. With this release, a heath warning is generated if the average Red Hat Ceph Storage OSD heartbeat exceeds a configurable threshold for any computed intervals. The Red Hat Ceph Storage OSD computes 1 minute,5 minute and 15 minute intervals with the average, minimum and maximum values.
BlueStore compression stats added to the dashboard
With this release, compression related performance metrics for BlueStore OSDs will now be visible in the dashboard.
For more information about the dashboard, see the Dashboard Guide.
The storage cluster status changes when a Ceph OSD encounters an I/O error
With this release, the Ceph Monitor now has a mon_osd_warn_num_repaired option, which is set to 10 by default. If any Ceph OSD has repaired more than this many I/O errors in stored data, a OSD_TOO_MANY_REPAIRS health warning status is generated. To clear this warning, the new clear_shards_repaired option has been added to the ceph tell command. For example:
ceph tell osd.NUMBER clear_shards_repaired [COUNT]
By default, the clear_shards_repaired option sets the repair count to 0. To be warned again if additional Ceph OSD repairs are performed, you can specify the value of the mon_osd_warn_num_repaired option.
Update to the heartbeat grace period
Previously, when there were no Ceph OSD failures for more than 48 hours, there was no mechanism to reset the grace timer back to the default value. With this release, the heartbeat grace timer is reset to the default value of 20 seconds, if there have been no failures on a Ceph OSD for 48 hours. When the failure interval between the last failure and the latest failure exceeds 48 hours, the grace timer is reset to the default value of 20 seconds.
The grace time is the interval in which a Ceph storage cluster considers a Ceph OSD as down by the absence of a heartbeat. The grace time is scaled based on lag estimations or on how frequently a Ceph ODS is experiencing failures.
The osd_client_message_cap option has been added back
Previously, the osd_client_message_cap option was removed, but with this release, the osd_client_message_cap option has been re-introduced. This option helps control the maximum number of in-flight client requests by throttling those requests. Doing this can be helpful when a Ceph OSD flaps due to overwhelming amount of client-based traffic.
