Chapter 8. Configuring the Shared File Systems service (manila)
When you deploy the Shared File Systems service (manila), you can choose one or more supported back ends, such as native CephFS, CephFS-NFS, NetApp, and others.
For a complete list of supported back-end appliances and drivers, see the Manila section of the Red Hat Knowledge Article, Component, Plug-In, and Driver Support in Red Hat OpenStack Platform.
Prerequisites
-
You have the
oc
command line tool installed on your workstation. -
You are logged on to a workstation that has access to the RHOSO control plane as a user with
cluster-admin
privileges. - You have planned networking for the Shared File Systems service. For more information, see Planning networking for the Shared File Systems service in Planning your deployment.
- You have enabled the Shared File Systems service. For more information, see Enabling the Shared File Systems service.
For native CephFS or CephFS-NFS:
- A CephFS file system must exist on the Red Hat Ceph Storage cluster. For more information, see Integrating Red Hat Ceph Storage.
- A Ceph user must exist that has CephX capabilities (caps) to perform operations on the CephFS file system. For more information, see Integrating Red Hat Ceph Storage.
For CephFS-NFS:
-
A
ceph nfs
service must exist in the Ceph Storage cluster. For more information, see Integrating Red Hat Ceph Storage. - You have created an isolated StorageNFS network for NFS exports and a corresponding StorageNFS shared provider network in the Networking service (neutron). The StorageNFS shared provider network maps to the isolated StorageNFS network of the data center.
The NFS service is isolated on a network that you can share with all Red Hat OpenStack Services on OpenShift (RHOSO) users. For more information about customizing the NFS service, see NFS cluster and export management in the Red Hat Ceph Storage File System Guide.
ImportantWhen you deploy an NFS service for the Shared File Systems service, do not select a custom port to expose NFS. Only the default NFS port of 2049 is supported. You must enable the Red Hat Ceph Storage
ingress
service and set theingress-mode
tohaproxy-protocol
. Otherwise, you cannot use IP-based access rules with the Shared File Systems service. For security in production environments, Red Hat does not recommend providing access to0.0.0.0/0
on shares to mount them on client machines.
-
A
8.1. Enabling the Shared File Systems service
You can enable the Shared File Systems service (manila) to provision remote, shareable file systems in your Red Hat OpenStack Services on OpenShift (RHOSO) deployment. These file systems are known as shares, and they allow projects in the cloud to share POSIX compliant storage. Shares can be mounted to multiple compute instances, baremetal computes, containers or pods of containers at the same time with read/write access mode.
When you enable the Shared File Systems service, you can configure the service with the following back ends:
- Red Hat Ceph Storage CephFS
- Red Hat Ceph Storage CephFS-NFS
- NFS or CIFS through third party vendor storage systems
Prerequisites
- You have planned networking for the Shared File Systems service. For more information, see Planning networking for the Shared File Systems service in Planning your deployment.
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to thespec
section to enable the Shared File Systems service:spec: ... manila: enabled: true apiOverride: route: {} template: databaseInstance: openstack secret: osp-secret override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer manilaAPI: replicas: 3 manilaScheduler: replicas: 3 manilaShares: share1: networkAttachments: - storage replicas: 0 # backend needs to be configured
NoteYou must configure a back end for the Shared File Systems service. If you do not configure a back end for the Shared File Systems service, then the service is deployed but not activated (
replicas: 0
).Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.2. Configuring a native CephFS back end
You can configure the Shared File Systems service (manila) with native CephFS as the storage back end.
Limitations
You can expose a native CephFS back end to trusted users, but take the following security measures:
- Configure the storage network as a provider network.
- Apply role-based access control (RBAC) policies to secure the storage provider network.
- Create a private share type.
Prerequisites
- An isolated storage network.
- You have planned networking for storage to ensure connectivity between the storage back end, the control plane, and the Compute nodes on the data plane. For more information, see Storage networks in Planning your deployment and Preparing networks for Red Hat OpenStack Services on OpenShift in Deploying Red Hat OpenStack Services on OpenShift.
- You have created a Red Hat Ceph Storage secret. For more information, see Integrating Red Hat Ceph Storage
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add theextraMounts
parameter in thespec
section to present the Ceph configuration files:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: extraMounts: - name: v1 region: r1 extraVol: - propagation: - ManilaShare extraVolType: Ceph volumes: - name: ceph projected: sources: - secret: name: <ceph-conf-files> mounts: - name: ceph mountPath: "/etc/ceph" readOnly: true
Add the following parameters to the
manila
template to configure the native CephFS back end:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaAPI: replicas: 3 customServiceConfig: | [DEFAULT] debug = true enabled_share_protocols=cephfs manilaScheduler: replicas: 3 manilaShares: cephfsnative: replicas: 1 networkAttachments: - storage customServiceConfig: | [DEFAULT] enabled_share_backends=cephfs [cephfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_volume_mode=0755 cephfs_protocol_helper_type=CEPHFS ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.3. Configuring a CephFS-NFS back end
You can configure the Shared File Systems service (manila) with CephFS-NFS as the storage back end.
Prerequisites
- The isolated storage network is configured on the share manager pod on OpenShift so that the Shared File Systems service can communicate with the Red Hat Ceph Storage cluster.
- Use an isolated NFS network for NFS traffic. This network does not need to be available to the share manager pod for the Shared File Systems service on OpenShift, but it must be available to Compute instances owned by end users.
- You have planned networking for storage to ensure connectivity between the storage back end, the control plane, and the Compute nodes on the data plane. For more information, see Storage networks in Planning your deployment and Preparing networks for Red Hat OpenStack Services on OpenShift in Deploying Red Hat OpenStack Services on OpenShift.
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to themanila
template:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaAPI: replicas: 3 customServiceConfig: | [DEFAULT] debug = true enabled_share_protocols=nfs manilaScheduler: replicas: 3 manilaShares: share1: customServiceConfig: | [DEFAULT] enabled_share_backends=cephfsnfs [cephfsnfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_nfs_cluster_id=cephfs cephfs_protocol_helper_type=NFS networkAttachments: - storage ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.4. Configuring alternative back ends
To configure the Shared File Systems service (manila) with an alternative back end, for example, NetApp or Pure Storage, complete the following high level tasks:
- Create the server connection secret.
-
Configure the
OpenStackControlPlane
CR to use the alternative storage system as the back end for the Shared File Systems service.
Prerequisites
- You have prepared the alternative storage system for consumption by Red Hat OpenStack Services on OpenShift (RHOSO).
- Network connectivity between the Red Hat OpenShift cluster, the Compute nodes, and the alternative storage system.
8.4.1. Creating the server connection secret
Create a server connection secret for an alternative back end to prevent placing server connection information directly in the OpenStackControlPlane
CR.
Procedure
Create a configuration file that contains the server connection information for your alternative back end. In this example, you are creating the
secret
for a NetApp back end.The following is an example of the contents of a configuration file:
[netapp] netapp_server_hostname = <netapp_ip> netapp_login = <netapp_user> netapp_password = <netapp_password> netapp_vserver = <netappvserver>
-
Replace
<netapp_ip>
with the IP address of the server. -
Replace
<netapp_user>
with the login user name. -
Replace
<netapp_password>
with the login password. -
Replace
<netappvserver>
with the vserver name. You do not need this option if configuring thedriver_handles_share_servers=True
mode.
-
Replace
- Save the configuration file.
Create the secret based on the configuration file:
$ oc create secret generic <secret_name> --from-file=<configuration_file_name>
-
Replace
<secret_name>
with the name you want to assign to the secret. -
Replace
<configuration_file_name>
with the name of the configuration file you created.
-
Replace
- Delete the configuration file.
8.4.2. Configuring an alternative back end
You can configure the Shared File Systems service (manila) with an alternative storage back end, for example, a NetApp back end.
Prerequisites
- You have planned networking for storage to ensure connectivity between the storage back end, the control plane, and the Compute nodes on the data plane. For more information, see Storage networks in Planning your deployment and Preparing networks for Red Hat OpenStack Services on OpenShift in Deploying Red Hat OpenStack Services on OpenShift.
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to themanila
template:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaAPI: replicas: 3 customServiceConfig: | [DEFAULT] debug = true enabled_share_protocols=cifs manilaScheduler: replicas: 3 manilaShares: share1: networkAttachments: - storage customServiceConfigSecrets: - manila_netapp_secret customServiceConfig: | [DEFAULT] debug = true enabled_share_backends=netapp [netapp] driver_handles_share_servers=False share_backend_name=netapp share_driver=manila.share.drivers.netapp.common.NetAppDriver netapp_storage_family=ontap_cluster ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.4.3. Custom configuration files
When you configure an alternative back end for the Shared File Systems service (manila), you might need to use additional configuration files. You can use the extraMounts
parameter in your OpenStackControlPlane
CR file to present these configuration files as OpenShift configMap
or secret
objects in the relevant share manager pod.
Example:
apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... extraMounts: - name: v1 region: r1 extraVol: - propagation: - sharepod1 extraVolType: Undefined volumes: - name: backendconfig projected: sources: - secret: name: manila-sharepod1-secrets mounts: - name: backendconfig mountPath: /etc/manila/drivers readOnly: true ...
8.4.4. Custom storage driver images
When you configure an alternative back end for the Shared File Systems service (manila), you might need to use a custom manilaShares
container image from the vendor on the Red Hat Ecosystem Catalog.
You can add the path to the container image to your OpenStackVersion
CR file with the customContainerImages
parameter.
For more information, see Deploying partner container images in Integrating partner content.
8.5. Configuring multiple back ends
You can deploy multiple back ends for the Shared File Systems service (manila), for example, a CephFS-NFS back end, a native CephFS back end, and a third-party back end. Add one back end only per pod.
Prerequisites
- When you use a back-end driver from a storage vendor that requires external software components, you must override the standard container image for the Shared File Systems service during deployment. You can find custom container images, for example, the Dell EMC Unity container image for a Dell EMC Unity storage system, at Red Hat Ecosystem Catalog.
- You have planned networking for storage to ensure connectivity between the storage back end, the control plane, and the Compute nodes on the data plane. For more information, see Storage networks in Planning your deployment and Preparing networks for Red Hat OpenStack Services on OpenShift in Deploying Red Hat OpenStack Services on OpenShift.
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to themanila
template to configure the back ends. In this example, there is a CephFS-NFS back end, a native CephFS back end, and a Pure Storage back end:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaAPI: replicas: 3 customServiceConfig: | [DEFAULT] debug = true enabled_share_protocols=nfs,cephfs,cifs manilaScheduler: replicas: 3 ...
Add the configuration for each back end you want to use:
Add the configuration for the CephFS-NFS back end:
... customServiceConfig: | ... manilaShares: cephfsnfs: networkAttachments: - storage customServiceConfig: | [DEFAULT] enabled_share_backends=cephfsnfs [cephfsnfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_nfs_cluster_id=cephfs cephfs_protocol_helper_type=NFS replicas: 1 ...
Add the configuration for the native CephFS back end:
... customServiceConfig: | ... manilaShares: cephfsnfs: ... cephfs: networkAttachments: - storage customServiceConfig: | [DEFAULT] enabled_share_backends=cephfs [cephfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_protocol_helper_type=CEPHFS replicas: 1 ...
Add the configuration for the Pure Storage back end:
... customContainerImages: manilaShareImages: pure: registry.connect.redhat.com/purestorage/openstack-manila-share-pure:rhoso18 manilaShares: cephfsnfs: ... cephfs: ... pure: networkAttachments: - storage customServiceConfigSecret: | - manila-pure-secret customServiceConfig: | [DEFAULT] debug = true enabled_share_backends=pure [pure] driver_handles_share_servers=False share_backend_name=pure share_driver=manila.share.drivers.purestorage.flashblade.FlashBladeShareDriver ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.6. Verifying the deployment of the Shared File Systems service
After a deployment or when troubleshooting issues, verify that the services for the Shared File Systems service (manila) are running and that they are up
.
Verify that the manila
pods are running. The number of pods depends on the number of replicas you have configured for the different components of the Shared File Systems service.
When you have verified that the pods are running, you can use the Shared File Systems service API to check the status of the services.
Procedure
List the
manila
pods to verify that they are running:$ oc -n openstack get pod -l service=manila
Example output:
NAME READY STATUS RESTARTS AGE manila-api-0 2/2 Running 0 43h manila-api-1 2/2 Running 0 43h manila-api-2 2/2 Running 0 43h manila-db-purge-28696321-tkl9g 0/1 Completed 0 41h manila-db-purge-28697761-zxxzc 0/1 Completed 0 17h manila-scheduler-0 2/2 Running 0 43h manila-scheduler-1 2/2 Running 0 43h manila-scheduler-2 2/2 Running 0 43h manila-share-share1-0 2/2 Running 0 43h
Access the remote shell for the
OpenStackClient
pod from your workstation:$ oc rsh -n openstack openstackclient
Run the
openstack share service list
command:$ openstack share service list
Example output:
----
---------------------------------------
---------------
-------------------------------------
| ID | Binary | Host | Zone | Status | State | Updated At |----
---------------------------------------
---------------
-------------------------------------
| 1 | manila-scheduler | hostgroup | nova | enabled | up | 2024-07-25T17:40:27.323342 | | 4 | manila-share | hostgroup@cephfsnfs | nova | enabled | up | 2024-07-25T17:40:49.115386 |----
---------------------------------------
---------------
-------------------------------------
-
Verify that the
Status
entry of every service isup
. If not, examine the relevant log files. Exit the
openstackclient
pod:$ exit
8.7. Verifying the deployment of multiple back ends
Use the openstack share service list
command to verify that the storage back ends for the Shared File Systems service (manila) deployed successfully. If you use a health check on multiple back ends, a ping test returns a response even if one of the back ends is unresponsive, so this is not a reliable way to verify your deployment.
Procedure
Access the remote shell for the
OpenStackClient
pod from your workstation:$ oc rsh -n openstack openstackclient
Confirm the list of Shared File Systems service back ends:
$ openstack share service list
The status of each successfully deployed back end shows as
enabled
and the state shows asup
.Exit the
openstackclient
pod:$ exit
8.8. Creating availability zones for back ends
You can create availability zones (AZs) for Shared File Systems service back ends to group cloud infrastructure and services logically for users. Map the AZs to failure domains and compute resources for high availability, fault tolerance, and resource scheduling. For example, you can create an AZ of Compute nodes that have specific hardware that users can specify when they create an instance that requires that hardware.
Post deployment, use the availability_zones
share type extra specification to limit share types to one or more AZs. Users can create a share directly in an AZ as long as the share type does not restrict them.
Procedure
The following example deploys two back ends where CephFS
is zone 1 and NetApp
is zone 2.
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to themanila
template:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaShares: cephfs: customServiceConfig: | [cephfs] backend_availability_zone = zone_1 ... netapp: customServiceConfig: | [netapp] backend_availability_zone = zone_2 ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.9. Changing the allowed NAS protocols
You can use the Shared File Systems service (manila) to export shares in the NFS, CephFS, or CIFS network attached storage (NAS) protocols. By default, the Shared File Systems service enables NFS and CIFS, and this may not be supported by the back ends in your deployment.
You can change the enabled_share_protocols
parameter and list only the protocols that you want to allow in your cloud. For example, if back ends in your deployment support both NFS and CIFS, you can change the default value and enable only one protocol. The NAS protocols that you assign must be supported by the back ends in your Shared File Systems service deployment.
Not all storage back-end drivers support the CIFS protocol. For information about which certified storage systems support CIFS, see the Red Hat Ecosystem Catalog.
Procedure
Open your
OpenStackControlPlane
CR file,openstack_control_plane.yaml
, and add the following parameters to themanila
template. In this example, you enable the NFS protocol:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane spec: ... manila: enabled: true template: manilaAPI: customServiceConfig: | [DEFAULT] enabled_share_protocols = NFS ...
Update the control plane:
$ oc apply -f openstack_control_plane.yaml -n openstack
Wait until RHOCP creates the resources related to the
OpenStackControlPlane
CR. Run the following command to check the status:$ oc get openstackcontrolplane -n openstack
The
OpenStackControlPlane
resources are created when the status is "Setup complete".TipAppend the
-w
option to the end of theget
command to track deployment progress.
8.10. Viewing back-end storage capacity
The scheduler component of the Shared File Systems service (manila) makes intelligent placement decisions based on several factors such as capacity, provisioning configuration, placement hints, and the capabilities that the back-end storage system driver detects and exposes. You can use share types and extra specifications to modify placement decisions.
Procedure
Access the remote shell for the
OpenStackClient
pod from your workstation:$ oc rsh -n openstack openstackclient
Run the following command to view the available back-end storage capacity:
$ openstack share pool list --detail
Exit the
openstackclient
pod:$ exit
8.11. Configuring automatic database cleanup
The Shared File Systems (manila) service automatically purges database entries marked for deletion for a set number of days. By default, records are marked for deletion for 30 days. You can configure a different record age and schedule for purge jobs.
Procedure
-
Open your
openstack_control_plane.yaml
file to edit theOpenStackControlPlane
CR. Add the
dbPurge
parameter to themanila
template to configure database cleanup.The following is an example of using the
dbPurge
parameter to configure the Shared File Systems service:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: openstack spec: manila: template: dbPurge: age: 20 1 schedule: 1 0 * * 0 2
Update the control plane:
$ oc apply -f openstack_control_plane.yaml