Red Hat Summit Red Hat SummitSupportoConsoleSviluppatoriInizia una provaContatti

Questo contenuto non è disponibile nella lingua selezionata.

8.10. Enabling pNFS SCSI Layouts in NFS

You can configure the NFS server and client to use the pNFS SCSI layout for accessing data. pNFS SCSI is beneficial in use cases that involve longer-duration single-client access to a file.

Prerequisites

  • Both the client and the server must be able to send SCSI commands to the same block device. That is, the block device must be on a shared SCSI bus.
  • The block device must contain an XFS file system.
  • The SCSI device must support SCSI Persistent Reservations as described in the SCSI-3 Primary Commands specification.

8.10.1. pNFS SCSI Layouts
Copia collegamento

The SCSI layout builds on the work of pNFS block layouts. The layout is defined across SCSI devices. It contains a sequential series of fixed-size blocks as logical units (LUs) that must be capable of supporting SCSI persistent reservations. The LU devices are identified by their SCSI device identification.
pNFS SCSI performs well in use cases that involve longer-duration single-client access to a file. An example might be a mail server or a virtual machine housing a cluster.

Operations Between the Client and the Server

When an NFS client reads from a file or writes to it, the client performs a LAYOUTGET operation. The server responds with the location of the file on the SCSI device. The client might need to perform an additional operation of GETDEVICEINFO to determine which SCSI device to use. If these operations work correctly, the client can issue I/O requests directly to the SCSI device instead of sending READ and WRITE operations to the server.
Errors or contention between clients might cause the server to recall layouts or not issue them to the clients. In those cases, the clients fall back to issuing READ and WRITE operations to the server instead of sending I/O requests directly to the SCSI device.
To monitor the operations, see Section 8.10.6, “Monitoring pNFS SCSI Layouts Functionality”.

Device Reservations

pNFS SCSI handles fencing through the assignment of reservations. Before the server issues layouts to clients, it reserves the SCSI device to ensure that only registered clients may access the device. If a client can issue commands to that SCSI device but is not registered with the device, many operations from the client on that device fail. For example, the blkid command on the client fails to show the UUID of the XFS file system if the server has not given a layout for that device to the client.
The server does not remove its own persistent reservation. This protects the data within the file system on the device across restarts of clients and servers. In order to repurpose the SCSI device, you might need to manually remove the persistent reservation on the NFS server.

8.10.2. Checking for a SCSI Device Compatible with pNFS
Copia collegamento

This procedure checks if a SCSI device supports the pNFS SCSI layout.

Prerequisites

  • Install the sg3_utils package: 
    # yum install sg3_utils
    Copy to Clipboard Toggle word wrap

Procedure 8.4. Checking for a SCSI Device Compatible with pNFS

  • On both the server and client, check for the proper SCSI device support: 
    # sg_persist --in --report-capabilities --verbose path-to-scsi-device
    Copy to Clipboard Toggle word wrap
    Ensure that the Persist Through Power Loss Active (PTPL_A) bit is set.

    Example 8.8.  A SCSI device that supports pNFS SCSI

    The following is an example of sg_persist output for a SCSI device that supports pNFS SCSI. The PTPL_A bit reports 1. 
        inquiry cdb: 12 00 00 00 24 00
    Persistent Reservation In cmd: 5e 02 00 00 00 00 00 20 00 00
  LIO-ORG   block11           4.0
  Peripheral device type: disk
Report capabilities response:
  Compatible Reservation Handling(CRH): 1
  Specify Initiator Ports Capable(SIP_C): 1
  All Target Ports Capable(ATP_C): 1
  Persist Through Power Loss Capable(PTPL_C): 1
  Type Mask Valid(TMV): 1
  Allow Commands: 1
  Persist Through Power Loss Active(PTPL_A): 1
    Support indicated in Type mask:
      Write Exclusive, all registrants: 1
      Exclusive Access, registrants only: 1
      Write Exclusive, registrants only: 1
      Exclusive Access: 1
      Write Exclusive: 1
      Exclusive Access, all registrants: 1
    Copy to Clipboard Toggle word wrap

Additional Resources

  • The sg_persist(8) man page

8.10.3. Setting up pNFS SCSI on the Server
Copia collegamento

This procedure configures an NFS server to export a pNFS SCSI layout.

Procedure 8.5. Setting up pNFS SCSI on the Server

  1. On the server, mount the XFS file system created on the SCSI device.
  2. Configure the NFS server to export NFS version 4.1 or higher. Set the following option in the [nfsd] section of the /etc/nfs.conf file: 
    [nfsd]

vers4.1=y
    Copy to Clipboard Toggle word wrap
  3. Configure the NFS server to export the XFS file system over NFS with the pnfs option:

    Example 8.9. An Entry in /etc/exports to Export pNFS SCSI

    The following entry in the /etc/exports configuration file exports the file system mounted at /exported/directory/ to the allowed.example.com client as a pNFS SCSI layout: 
    /exported/directory allowed.example.com(pnfs)
    Copy to Clipboard Toggle word wrap

Additional Resources

8.10.4. Setting up pNFS SCSI on the Client
Copia collegamento

This procedure configures an NFS client to mount a pNFS SCSI layout.

Prerequisites

Procedure 8.6. Setting up pNFS SCSI on the Client

  • On the client, mount the exported XFS file system using NFS version 4.1 or higher: 
    # mount -t nfs -o nfsvers=4.1 host:/remote/export /local/directory
    Copy to Clipboard Toggle word wrap
    Do not mount the XFS file system directly without NFS.

Additional Resources

8.10.5. Releasing the pNFS SCSI Reservation on the Server
Copia collegamento

This procedure releases the persistent reservation that an NFS server holds on a SCSI device. This enables you to repurpose the SCSI device when you no longer need to export pNFS SCSI.
You must remove the reservation from the server. It cannot be removed from a different IT Nexus.

Prerequisites

  • Install the sg3_utils package: 
    # yum install sg3_utils
    Copy to Clipboard Toggle word wrap

Procedure 8.7. Releasing the pNFS SCSI Reservation on the Server

  1. Query an existing reservation on the server: 
    # sg_persist --read-reservation path-to-scsi-device
    Copy to Clipboard Toggle word wrap

    Example 8.10. Querying a Reservation on /dev/sda

    # sg_persist --read-reservation /dev/sda

  LIO-ORG   block_1           4.0
  Peripheral device type: disk
  PR generation=0x8, Reservation follows:
    Key=0x100000000000000
    scope: LU_SCOPE,  type: Exclusive Access, registrants only
    Copy to Clipboard Toggle word wrap
  2. Remove the existing registration on the server: 
    # sg_persist --out \
             --release \
             --param-rk=reservation-key \
             --prout-type=6 \
             path-to-scsi-device
    Copy to Clipboard Toggle word wrap

    Example 8.11. Removing a Reservation on /dev/sda

    # sg_persist --out \
             --release \
             --param-rk=0x100000000000000 \
             --prout-type=6 \
             /dev/sda

  LIO-ORG   block_1           4.0
  Peripheral device type: disk
    Copy to Clipboard Toggle word wrap

Additional Resources

  • The sg_persist(8) man page

8.10.6. Monitoring pNFS SCSI Layouts Functionality
Copia collegamento

You can monitor if the pNFS client and server exchange proper pNFS SCSI operations or if they fall back on regular NFS operations.

Prerequisites

  • A pNFS SCSI client and server are configured.

8.10.6.1. Checking pNFS SCSI Operations from the Server Using nfsstat
Copia collegamento

This procedure uses the nfsstat utility to monitor pNFS SCSI operations from the server.

Procedure 8.8. Checking pNFS SCSI Operations from the Server Using nfsstat

  1. Monitor the operations serviced from the server: 
    # watch --differences \
        "nfsstat --server | egrep --after-context=1 read\|write\|layout"

Every 2.0s: nfsstat --server | egrep --after-context=1 read\|write\|layout

putrootfh    read         readdir      readlink     remove	 rename
2         0% 0         0% 1         0% 0         0% 0         0% 0         0%
--
setcltidconf verify	  write        rellockowner bc_ctl	 bind_conn
0         0% 0         0% 0         0% 0         0% 0         0% 0         0%
--
getdevlist   layoutcommit layoutget    layoutreturn secinfononam sequence
0         0% 29        1% 49        1% 5         0% 0         0% 2435     86%
    Copy to Clipboard Toggle word wrap
  2. The client and server use pNFS SCSI operations when:
    • The layoutget, layoutreturn, and layoutcommit counters increment. This means that the server is serving layouts.
    • The server read and write counters do not increment. This means that the clients are performing I/O requests directly to the SCSI devices.

8.10.6.2. Checking pNFS SCSI Operations from the Client Using mountstats
Copia collegamento

This procedure uses the /proc/self/mountstats file to monitor pNFS SCSI operations from the client.

Procedure 8.9. Checking pNFS SCSI Operations from the Client Using mountstats

  1. List the per-mount operation counters: 
    # cat /proc/self/mountstats \
      | awk /scsi_lun_0/,/^$/ \
      | egrep device\|READ\|WRITE\|LAYOUT

device 192.168.122.73:/exports/scsi_lun_0 mounted on /mnt/rhel7/scsi_lun_0 with fstype nfs4 statvers=1.1
    nfsv4:  bm0=0xfdffbfff,bm1=0x40f9be3e,bm2=0x803,acl=0x3,sessions,pnfs=LAYOUT_SCSI
            READ: 0 0 0 0 0 0 0 0
           WRITE: 0 0 0 0 0 0 0 0
        READLINK: 0 0 0 0 0 0 0 0
         READDIR: 0 0 0 0 0 0 0 0
       LAYOUTGET: 49 49 0 11172 9604 2 19448 19454
    LAYOUTCOMMIT: 28 28 0 7776 4808 0 24719 24722
    LAYOUTRETURN: 0 0 0 0 0 0 0 0
     LAYOUTSTATS: 0 0 0 0 0 0 0 0
    Copy to Clipboard Toggle word wrap
  2. In the results:
    • The LAYOUT statistics indicate requests where the client and server use pNFS SCSI operations.
    • The READ and WRITE statistics indicate requests where the client and server fall back to NFS operations.
Torna in cima
Red Hat logoGithubredditYoutubeTwitter

Formazione

Prova, acquista e vendi

Community

Informazioni sulla documentazione di Red Hat

Aiutiamo gli utenti Red Hat a innovarsi e raggiungere i propri obiettivi con i nostri prodotti e servizi grazie a contenuti di cui possono fidarsi. Esplora i nostri ultimi aggiornamenti.

Rendiamo l’open source più inclusivo

Red Hat si impegna a sostituire il linguaggio problematico nel codice, nella documentazione e nelle proprietà web. Per maggiori dettagli, visita il Blog di Red Hat.

Informazioni su Red Hat

Forniamo soluzioni consolidate che rendono più semplice per le aziende lavorare su piattaforme e ambienti diversi, dal datacenter centrale all'edge della rete.

Theme

© 2025 Red Hat