Red Hat SummitChapter 10. Using Fibre Channel devices
Red Hat Enterprise Linux 8 provides the following native Fibre Channel drivers:
-
lpfc -
qla2xxx -
zfcp
10.1. Re-scanning Fibre Channel logical units after resizing a LUN
If you changed the logical unit number (LUN) size on the external storage, use the echo command to update the kernel’s view of the size.
Procedure
Determine which devices are paths for a
multipathlogical unit:Copy to Clipboard Copied! Toggle word wrap Toggle overflow multipath -ll
# multipath -llRe-scan Fibre Channel logical units on a system that uses multipathing:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow echo 1 > /sys/block/sdX/device/rescan
$ echo 1 > /sys/block/sdX/device/rescan
Additional resources
-
multipath(8)man page on your system
10.2. Determining the link loss behavior of device using Fibre Channel
If a driver implements the Transport dev_loss_tmo callback, access attempts to a device through a link will be blocked when a transport problem is detected.
Procedure
Determine the state of a remote port:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow cat /sys/class/fc_remote_ports/rport-host:bus:remote-port/port_state
$ cat /sys/class/fc_remote_ports/rport-host:bus:remote-port/port_stateThis command returns one of the following output:
-
Blockedwhen the remote port along with devices accessed through it are blocked. Onlineif the remote port is operating normallyIf the problem is not resolved within
dev_loss_tmoseconds, therportand devices will be unblocked. All I/O running on that device along with any new I/O sent to that device will fail.
-
When a link loss exceeds dev_loss_tmo, the scsi_device and sd_N_ devices are removed. Typically, the Fibre Channel class will leave the device as is, that is /dev/sdx will remain /dev/sdx. This is because the target binding is saved by the Fibre Channel driver and when the target port returns, the SCSI addresses are recreated faithfully. However, this cannot be guaranteed, the sdx device will be restored only if no additional change on in-storage box configuration of LUNs is made.
Additional resources
-
multipath.conf(5)man page on your system - Recommended tuning at scsi,multipath and at application layer while configuring Oracle RAC cluster (Red Hat Knowledgebase)
10.3. Fibre Channel configuration files
The following is the list of configuration files in the /sys/class/ directory that provide the user-space API to Fibre Channel.
The items use the following variables:
H- Host number
B- Bus number
T- Target
L- Logical unit (LUNs)
R- Remote port number
Consult your hardware vendor before changing any of the values described in this section, if your system is using multipath software.
Transport configuration in /sys/class/fc_transport/targetH:B:T/
port_id- 24-bit port ID/address
node_name- 64-bit node name
port_name- 64-bit port name
Remote port configuration in /sys/class/fc_remote_ports/rport-H:B-R/
-
port_id -
node_name -
port_name dev_loss_tmoControls when the scsi device gets removed from the system. After
dev_loss_tmotriggers, the scsi device is removed. In themultipath.conffile , you can setdev_loss_tmotoinfinity.In Red Hat Enterprise Linux 8, if you do not set the
fast_io_fail_tmooption,dev_loss_tmois capped to600seconds. By default,fast_io_fail_tmois set to5seconds in Red Hat Enterprise Linux 8 if themultipathdservice is running; otherwise, it is set tooff.fast_io_fail_tmoSpecifies the number of seconds to wait before it marks a link as "bad". Once a link is marked bad, existing running I/O or any new I/O on its corresponding path fails.
If I/O is in a blocked queue, it will not be failed until
dev_loss_tmoexpires and the queue is unblocked.If
fast_io_fail_tmois set to any value except off,dev_loss_tmois uncapped. Iffast_io_fail_tmois set to off, no I/O fails until the device is removed from the system. Iffast_io_fail_tmois set to a number, I/O fails immediately when thefast_io_fail_tmotimeout triggers.
Host configuration in /sys/class/fc_host/hostH/
-
port_id -
node_name -
port_name issue_lipInstructs the driver to rediscover remote ports.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}