25.9. Removing a Storage Device
Before removing access to the storage device itself, it is advisable to back up data from the device first. Afterwards, flush I/O and remove all operating system references to the device (as described below). If the device uses multipathing, then do this for the multipath "pseudo device" (Section 25.8.2, “World Wide Identifier (WWID)”) and each of the identifiers that represent a path to the device. If you are only removing a path to a multipath device, and other paths will remain, then the procedure is simpler, as described in Section 25.11, “Adding a Storage Device or Path”.
Removal of a storage device is not recommended when the system is under memory pressure, since the I/O flush will add to the load. To determine the level of memory pressure, run the command
vmstat 1 100; device removal is not recommended if:
- Free memory is less than 5% of the total memory in more than 10 samples per 100 (the command
freecan also be used to display the total memory). - Swapping is active (non-zero
siandsocolumns in thevmstatoutput).
The general procedure for removing all access to a device is as follows:
Procedure 25.11. Ensuring a Clean Device Removal
- Close all users of the device and backup device data as needed.
- Use
umountto unmount any file systems that mounted the device. - Remove the device from any
mdand LVM volume using it. If the device is a member of an LVM Volume group, then it may be necessary to move data off the device using thepvmovecommand, then use thevgreducecommand to remove the physical volume, and (optionally)pvremoveto remove the LVM metadata from the disk. - Run
multipath -lcommand to find the list of devices which are configured asmultipathdevice. If the device is configured as a multipath device, runmultipath -f devicecommand to flush any outstanding I/O and to remove the multipath device. - Flush any outstanding I/O to the used paths. This is important for raw devices, where there is no
umountorvgreduceoperation to cause an I/O flush. You need to do this step only if:- the device is not configured as a multipath device, or
- the device is configured as a multipath device and I/O has been issued directly to its individual paths at some point in the past.
Use the following command to flush any outstanding I/O:#blockdev --flushbufs device - Remove any reference to the device's path-based name, like
/dev/sd,/dev/disk/by-pathor themajor:minornumber, in applications, scripts, or utilities on the system. This is important in ensuring that different devices added in the future will not be mistaken for the current device. - Finally, remove each path to the device from the SCSI subsystem. To do so, use the command
echo 1 > /sys/block/device-name/device/deletewheredevice-namemay besde, for example.Another variation of this operation isecho 1 > /sys/class/scsi_device/h:c:t:l/device/delete, wherehis the HBA number,cis the channel on the HBA,tis the SCSI target ID, andlis the LUN.Note
The older form of these commands,echo "scsi remove-single-device 0 0 0 0" > /proc/scsi/scsi, is deprecated.
You can determine the
device-name, HBA number, HBA channel, SCSI target ID and LUN for a device from various commands, such as lsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*.
After performing Procedure 25.11, “Ensuring a Clean Device Removal”, a device can be physically removed safely from a running system. It is not necessary to stop I/O to other devices while doing so.
Other procedures, such as the physical removal of the device, followed by a rescan of the SCSI bus (as described in Section 25.12, “Scanning Storage Interconnects”) to cause the operating system state to be updated to reflect the change, are not recommended. This will cause delays due to I/O timeouts, and devices may be removed unexpectedly. If it is necessary to perform a rescan of an interconnect, it must be done while I/O is paused, as described in Section 25.12, “Scanning Storage Interconnects”.
