Chapter 4. Customizing the Red Hat Ceph Storage cluster for HCI
Red Hat OpenStack Platform (RHOSP) director uses a default configuration to deploy containerized Red Hat Ceph Storage. You can customize Ceph Storage by overriding the default settings.
Prerequisites
- The servers are deployed and their storage networks configured.
-
The deployed bare metal file as output by
openstack overcloud node provision -o ~/deployed_metal.yaml …
.
4.1. Configuration options
There are several options for configuring the Red Hat Ceph Storage cluster.
Procedure
-
Log in to the undercloud node as the
stack
user. Optional: Use a standard format initialization (ini) file to configure the Ceph cluster.
Create the file with configuration options.
The following is an example of a simple configuration file:
[global] osd_crush_chooseleaf type = 0 log_file = /var/log/ceph/$cluster-$type.$id.log [mon] mon_cluster_log_to_syslog = true
- Save the configuration file.
Use the
openstack overcloud ceph deploy --config <configuration_file_name>
command to deploy the configuration.Replace
<configuration_file_name>
with the name of the file you created.$ openstack overcloud ceph deploy --config initial-ceph.conf
Optional: Send configuration values to the
cephadm bootstrap
command:openstack overcloud ceph deploy --force \ --cephadm-extra-args '<optional_arguments>' \
Replace
<optional_arguments>
with the configuration values to provide to the underlying command.NoteWhen using the arguments
--log-to-file
and--skip-prepare-host
, the commandopenstack overcloud ceph deploy --force \ --cephadm-extra-args '--log-to-file --skip-prepare-host' \
is used.
4.2. Generating the service specification (optional)
The Red Hat Ceph Storage cluster service specification is a YAML file that describes the deployment of Ceph Storage services. It is automatically generated by tripleo
before the Ceph Storage cluster is deployed. It does not usually have to be generated separately.
A custom service specification can be created to customize the Red Hat Ceph Storage cluster.
Procedure
-
Log in to the undercloud node as the
stack
user. Generate the specification file:
openstack overcloud ceph spec deployed_metal.yaml -o <specification_file>
Replace
<specification_file>
with the name of the file to generate with the current service specification.NoteThe
deployed_metal.yaml
comes from the output of theopenstack overcloud node provision
command.
- Edit the generated file with the required configuration.
Deploy the custom service specification:
openstack overcloud ceph deploy \ deployed_metal.yaml \ -o deployed_ceph.yaml \ --ceph-spec <specification_file>
-
Replace
<specification_file>
with the name of the custom service specification file.
-
Replace
4.3. Ceph containers for Red Hat OpenStack Platform with Red Hat Ceph Storage
You must have a Ceph Storage container to configure Red Hat Openstack Platform (RHOSP) to use Red Hat Ceph Storage with NFS Ganesha. You do not require a Ceph Storage container if the external Ceph Storage cluster only provides Block (through RBD), Object (through RGW), or File (through native CephFS) storage.
RHOSP 17.1 will deploy Red Hat Ceph Storage 6.x (Ceph package 17.x). The Ceph Storage 6.x containers are hosted at registry.redhat.io
, a registry that requires authentication. For more information, see Container image preparation parameters.
4.4. Configuring advanced OSD specifications
Configure an advanced OSD specification when the default specification does not provide the necessary functionality for your Ceph Storage cluster.
Procedure
-
Log in to the undercloud node as the
stack
user. Create a YAML format file that defines the advanced OSD specification.
The following is an example of a custom OSD specification.
data_devices: rotational: 1 db_devices: rotational: 0
This example would create an OSD specification where all rotating devices will be data devices and all non-rotating devices will be used as shared devices. When the dynamic Ceph service specification is built, whatever is in the specification file is appended to the section of the specification if the
service_type
isosd
.- Save the specification file.
Deploy the specification:
openstack overcloud ceph deploy \ --osd-spec <osd_specification_file>
Replace
<osd_specification_file>
with the name of the specification file you created.$ openstack overcloud ceph deploy \ --osd-spec osd_spec.yaml \
Additional resources
For a list of OSD-related attributes used to configure OSDs in the service specification, see Advanced service specifications and filters for deploying OSDs in the Red Hat Ceph Storage Operations Guide.
4.5. Migrating from node-specific overrides
Node-specific overrides were used to manage non-homogenous server hardware before Red Hat OpenStack Platform 17.0. This is now done with a custom OSD specification file. See Configuring advanced OSD specifications for information on how to create a custom OSD specification file.
4.6. Enabling Ceph on-wire encryption
Enable encryption for all Ceph Storage traffic using the secure mode
of the messenger version 2 protocol. Configure Ceph Storage as described in Encryption and Key Management in the Red Hat Ceph Storage Data Hardening Red Hat OpenStack Platform to enable Ceph on-wire encryption.
Additional resources
For more information about Ceph on-wire encryption, see Ceph on-wire encryption in the Red Hat Ceph Storage Architecture Guide.