Chapter 3. Administering GFS2 file systems
There are a variety of commands and options that you use to create, mount, grow, and manage GFS2 file systems.
3.1. GFS2 file system creation
You create a GFS2 file system with the mkfs.gfs2 command. A file system is created on an activated LVM volume.
3.1.1. The GFS2 mkfs command
The following information is required to run the mkfs.gfs2 command to create a clustered GFS2 file system:
-
Lock protocol/module name, which is
lock_dlmfor a cluster - Cluster name
- Number of journals (one journal required for each node that may be mounting the file system)
Once you have created a GFS2 file system with the mkfs.gfs2 command, you cannot decrease the size of the file system. You can, however, increase the size of an existing file system with the gfs2_grow command.
The format for creating a clustered GFS2 file system is as follows. Note that Red Hat does not support the use of GFS2 as a single-node file system.
mkfs.gfs2 -p lock_dlm -t ClusterName:FSName -j NumberJournals BlockDevice
If you prefer, you can create a GFS2 file system by using the mkfs command with the -t parameter specifying a file system of type gfs2, followed by the GFS2 file system options.
mkfs -t gfs2 -p lock_dlm -t ClusterName:FSName -j NumberJournals BlockDevice
Improperly specifying the ClusterName:FSName parameter may cause file system or lock space corruption.
ClusterName- The name of the cluster for which the GFS2 file system is being created.
FSName-
The file system name, which can be 1 to 16 characters long. The name must be unique for all
lock_dlmfile systems over the cluster. NumberJournals-
Specifies the number of journals to be created by the
mkfs.gfs2command. One journal is required for each node that mounts the file system. For GFS2 file systems, more journals can be added later without growing the file system. BlockDevice- Specifies a logical or other block device
The following table describes the mkfs.gfs2 command options (flags and parameters).
| Flag | Parameter | Description |
|---|---|---|
|
|
|
Sets the initial size of each journal’s quota change file to |
|
| Enables debugging output. | |
|
| Help. Displays available options. | |
|
|
| Specifies the size of the journal in megabytes. Default journal size is 128 megabytes. The minimum size is 8 megabytes. Larger journals improve performance, although they use more memory than smaller journals. |
|
|
|
Specifies the number of journals to be created by the |
|
|
Prevents the | |
|
|
| * Specifies the name of the locking protocol to use. Recognized locking protocols include:
*
* |
|
| Quiet. Do not display anything. | |
|
|
|
Specifies the size of the resource groups in megabytes. The minimum resource group size is 32 megabytes. The maximum resource group size is 2048 megabytes. A large resource group size may increase performance on very large file systems. If this is not specified, |
|
|
|
* A unique identifier that specifies the lock table field when you use the
* This parameter has two parts separated by a colon (no spaces) as follows:
*
* |
|
| Displays command version information. |
3.1.2. Creating a GFS2 file system
The following example creates two GFS2 file systems. For both of these file systems, lock_dlm` is the locking protocol that the file system uses, since this is a clustered file system. Both file systems can be used in the cluster named alpha.
For the first file system, file system name is mydata1. it contains eight journals and is created on /dev/vg01/lvol0. For the second file system, the file system name is mydata2. It contains eight journals and is created on /dev/vg01/lvol1.
# mkfs.gfs2 -p lock_dlm -t alpha:mydata1 -j 8 /dev/vg01/lvol0 # mkfs.gfs2 -p lock_dlm -t alpha:mydata2 -j 8 /dev/vg01/lvol1
3.2. Mounting a GFS2 file system
Before you can mount a GFS2 file system, the file system must exist, the volume where the file system exists must be activated, and the supporting clustering and locking systems must be started. After those requirements have been met, you can mount the GFS2 file system as you would any Linux file system.
You should always use Pacemaker to manage the GFS2 file system in a production environment rather than manually mounting the file system with a mount command, as this may cause issues at system shutdown.
To manipulate file ACLs, you must mount the file system with the -o acl mount option. If a file system is mounted without the -o acl mount option, users are allowed to view ACLs (with getfacl), but are not allowed to set them (with setfacl).
3.2.1. Mounting a GFS2 file system with no options specified
In this example, the GFS2 file system on /dev/vg01/lvol0 is mounted on the /mygfs2 directory.
# mount /dev/vg01/lvol0 /mygfs23.2.2. Mounting a GFS2 file system that specifies mount options
The following is the format for the command to mount a GFS2 file system that specifies mount options.
mount BlockDevice MountPoint -o option
BlockDevice- Specifies the block device where the GFS2 file system resides.
MountPoint- Specifies the directory where the GFS2 file system should be mounted.
The -o option argument consists of GFS2-specific options or acceptable standard Linux mount -o options, or a combination of both. Multiple option parameters are separated by a comma and no spaces.
The mount command is a Linux system command. In addition to using these GFS2-specific options, you can use other, standard, mount command options (for example, -r). For information about other Linux mount command options, see the Linux mount man page.
The following table describes the available GFS2-specific -o option values that can be passed to GFS2 at mount time.
This table includes descriptions of options that are used with local file systems only. Note, however, that Red Hat does not support the use of GFS2 as a single-node file system. Red Hat will continue to support single-node GFS2 file systems for mounting snapshots of cluster file systems (for example, for backup purposes).
| Option | Description |
|---|---|
|
|
Allows manipulating file ACLs. If a file system is mounted without the |
|
|
When |
|
|
Forces GFS2 to treat the file system as a multi-host file system. By default, using |
|
|
Tells GFS2 to let the VFS (virtual file system) layer do all flock and fcntl. The |
|
|
Allows the user to specify which locking protocol to use with the file system. If |
|
| Allows the user to specify which locking table to use with the file system. |
|
|
Turns quotas on or off for a file system. Setting the quotas to be in the |
|
|
When |
|
| Causes GFS2 to generate "discard" I/O requests for blocks that have been freed. These can be used by suitable hardware to implement thin provisioning and similar schemes. |
|
|
Causes GFS2 to send I/O barriers when flushing the journal. The default value is |
|
| Sets the number of seconds for which a change in the quota information may sit on one node before being written to the quota file. This is the preferred way to set this parameter. The value is an integer number of seconds greater than zero. The default is 60 seconds. Shorter settings result in faster updates of the lazy quota information and less likelihood of someone exceeding their quota. Longer settings make file system operations involving quotas faster and more efficient. |
|
|
Setting |
|
|
Provides a bound on the maximum percentage change in the |
3.2.3. Unmounting a GFS2 file system
GFS2 file systems that have been mounted manually rather than automatically through Pacemaker will not be known to the system when file systems are unmounted at system shutdown. As a result, the GFS2 resource agent will not unmount the GFS2 file system. After the GFS2 resource agent is shut down, the standard shutdown process kills off all remaining user processes, including the cluster infrastructure, and tries to unmount the file system. This unmount will fail without the cluster infrastructure and the system will hang.
To prevent the system from hanging when the GFS2 file systems are unmounted, you should do one of the following:
- Always use Pacemaker to manage the GFS2 file system.
-
If a GFS2 file system has been mounted manually with the
mountcommand, be sure to unmount the file system manually with theumountcommand before rebooting or shutting down the system.
If your file system hangs while it is being unmounted during system shutdown under these circumstances, perform a hardware reboot. It is unlikely that any data will be lost since the file system is synced earlier in the shutdown process.
The GFS2 file system can be unmounted the same way as any Linux file system, by using the umount command.
The umount command is a Linux system command. Information about this command can be found in the Linux umount command man pages.
Usage
umount MountPointMountPoint- Specifies the directory where the GFS2 file system is currently mounted.
3.3. Backing up a GFS2 file system
It is important to make regular backups of your GFS2 file system in case of emergency, regardless of the size of your file system. Many system administrators feel safe because they are protected by RAID, multipath, mirroring, snapshots, and other forms of redundancy, but there is no such thing as safe enough.
It can be a problem to create a backup since the process of backing up a node or set of nodes usually involves reading the entire file system in sequence. If this is done from a single node, that node will retain all the information in cache until other nodes in the cluster start requesting locks. Running this type of backup program while the cluster is in operation will negatively impact performance.
Dropping the caches once the backup is complete reduces the time required by other nodes to regain ownership of their cluster locks and caches. This is still not ideal, however, because the other nodes will have stopped caching the data that they were caching before the backup process began. You can drop caches using the following command after the backup is complete:
echo -n 3 > /proc/sys/vm/drop_caches
It is faster if each node in the cluster backs up its own files so that the task is split between the nodes. You might be able to accomplish this with a script that uses the rsync command on node-specific directories.
Red Hat recommends making a GFS2 backup by creating a hardware snapshot on the SAN, presenting the snapshot to another system, and backing it up there. The backup system should mount the snapshot with -o lockproto=lock_nolock since it will not be in a cluster. Note, however, that Red Hat does not support the use of GFS2 as a single-node file system in a production environment. This option should be used only for the purposes of backup or for a secondary-site Disaster Recovery node, as described in Minimum cluster size. When using this option, you must ensure that the GFS2 file system is being used by only one system at a time.
3.4. Suspending activity on a GFS2 file system
You can suspend write activity to a file system by using the dmsetup suspend command. Suspending write activity allows hardware-based device snapshots to be used to capture the file system in a consistent state. The dmsetup resume command ends the suspension.
The format for the command to suspend activity on a GFS2 file system is as follows.
dmsetup suspend MountPoint
This example suspends writes to file system /mygfs2.
# dmsetup suspend /mygfs2The format for the command to end suspension of activity on a GFS2 file system is as follows.
dmsetup resume MountPoint
This example ends suspension of writes to file system /mygfs2.
# dmsetup resume /mygfs23.5. Growing a GFS2 file system
The gfs2_grow command is used to expand a GFS2 file system after the device where the file system resides has been expanded. Running the gfs2_grow command on an existing GFS2 file system fills all spare space between the current end of the file system and the end of the device with a newly initialized GFS2 file system extension. All nodes in the cluster can then use the extra storage space that has been added.
You cannot decrease the size of a GFS2 file system.
The gfs2_grow command must be run on a mounted file system. The following procedure increases the size of the GFS2 file system in a cluster that is mounted on the logical volume shared_vg/shared_lv1 with a mount point of /mnt/gfs2.
Procedure
- Perform a backup of the data on the file system.
If you do not know the logical volume that is used by the file system to be expanded, you can determine this by running the
df mountpointcommand. This will display the device name in the following format:/dev/mapper/vg-lvFor example, the device name
/dev/mapper/shared_vg-shared_lv1indicates that the logical volume isshared_vg/shared_lv1.On one node of the cluster, expand the underlying cluster volume with the
lvextendcommand. If you are running RHEL 8.0, use the--lockopt skiplvoption to override normal logical volume locking. This is not necessary for systems running RHEL 8.1 or later.For RHEL 8.1 and later use the following command.
# lvextend -L+1G shared_vg/shared_lv1 Size of logical volume shared_vg/shared_lv1 changed from 5.00 GiB (1280 extents) to 6.00 GiB (1536 extents). WARNING: extending LV with a shared lock, other hosts may require LV refresh. Logical volume shared_vg/shared_lv1 successfully resized.For RHEL 8.0, use the following command.
# lvextend --lockopt skiplv -L+1G shared_vg/shared_lv1 WARNING: skipping LV lock in lvmlockd. Size of logical volume shared_vg/shared_lv1 changed from 5.00 GiB (1280 extents) to 6.00 GiB (1536 extents). WARNING: extending LV with a shared lock, other hosts may require LV refresh. Logical volume shared_vg/shared_lv1 successfully resized.If you are running RHEL 8.0, on every additional node of the cluster refresh the logical volume to update the active logical volume on that node. This step is not necessary on systems running RHEL 8.1 and later as the step is automated when the logical volume is extended.
# lvchange --refresh shared_vg/shared_lv1One one node of the cluster, increase the size of the GFS2 file system. Do not extend the file system if the logical volume was not refreshed on all of the nodes, otherwise the file system data may become unavailable throughout the cluster.
# gfs2_grow /mnt/gfs2 FS: Mount point: /mnt/gfs2 FS: Device: /dev/mapper/shared_vg-shared_lv1 FS: Size: 1310719 (0x13ffff) DEV: Length: 1572864 (0x180000) The file system will grow by 1024MB. gfs2_grow complete.Run the
dfcommand on all nodes to check that the new space is now available in the file system. Note that it may take up to 30 seconds for thedfcommand on all nodes to show the same file system size# df -h /mnt/gfs2] Filesystem Size Used Avail Use% Mounted on /dev/mapper/shared_vg-shared_lv1 6.0G 4.5G 1.6G 75% /mnt/gfs2
3.6. Adding journals to a GFS2 file system
GFS2 requires one journal for each node in a cluster that needs to mount the file system. If you add additional nodes to the cluster, you can add journals to a GFS2 file system with the gfs2_jadd command. You can add journals to a GFS2 file system dynamically at any point without expanding the underlying logical volume. The gfs2_jadd command must be run on a mounted file system, but it needs to be run on only one node in the cluster. All the other nodes sense that the expansion has occurred.
If a GFS2 file system is full, the gfs2_jadd command will fail, even if the logical volume containing the file system has been extended and is larger than the file system. This is because in a GFS2 file system, journals are plain files rather than embedded metadata, so simply extending the underlying logical volume will not provide space for the journals.
Before adding journals to a GFS2 file system, you can find out how many journals the GFS2 file system currently contains with the gfs2_edit -p jindex command, as in the following example:
# gfs2_edit -p jindex /dev/sasdrives/scratch|grep journal
3/3 [fc7745eb] 4/25 (0x4/0x19): File journal0
4/4 [8b70757d] 5/32859 (0x5/0x805b): File journal1
5/5 [127924c7] 6/65701 (0x6/0x100a5): File journal2The format for the basic command to add journals to a GFS2 file system is as follows.
gfs2_jadd -j Number MountPoint
Number- Specifies the number of new journals to be added.
MountPoint- Specifies the directory where the GFS2 file system is mounted.
In this example, one journal is added to the file system on the /mygfs2 directory.
# gfs2_jadd -j 1 /mygfs2