18.7. Creating a file system DAX namespace on an NVDIMM
Configure an NVDIMM device that is attached to your system, in file system DAX mode to support a file system with direct access capabilities.
To do this, consider the following options:
- Reconfiguring an existing namespace to file system DAX mode.
- Creating a new file system DAX namespace if there is available space.
18.7.1. NVDIMM in file system direct access mode 复制链接链接已复制到粘贴板!
When an NVDIMM device is configured in file system direct access (file system DAX, fsdax) mode, you can create a file system on top of it. Any application that performs an mmap() operation on a file on this file system gets direct access to its storage. This enables the direct access programming model on NVDIMM.
The following new -o dax options are now available, and direct access behavior can be controlled through a file attribute if required:
-o dax=inodeThis is the default option when you do not specify any dax option while mounting a file system. Using this option, you can set an attribute flag on files to control if the dax mode can be activated. If required, you can set this flag on individual files.
You can also set this flag on a directory and any files in that directory will be created with the same flag. You can set this attribute flag by using the
xfs_io -c 'chattr +x'directory-name command.-o dax=never-
With this option, the dax mode will not be enabled even if the dax flag is set to an
inodemode. This means that the per-inode dax attribute flag is ignored, and files set with this flag will never be direct-access enabled. -o dax=alwaysThis option is equivalent to the old
-o daxbehavior. With this option, you can activate direct access mode for any file on the file system, regardless of the dax attribute flag.警告In further releases,
-o daxmight not be supported and if required, you can use-o dax=alwaysinstead. In this mode, every file might be in the direct-access mode.- Per-page metadata allocation
This mode requires allocating per-page metadata in the system DRAM or on the NVDIMM device itself. The overhead of this data structure is 64 bytes per each 4-KiB page:
- On small devices, the amount of overhead is small enough to fit in DRAM with no problems. For example, a 16-GiB namespace only requires 256 MiB for page structures. Since NVDIMM devices are usually small and expensive, storing the page tracking data structures in DRAM is preferable.
On NVDIMM devices that are terabytes in size or larger, the amount of memory required to store the page tracking data structures might exceed the amount of DRAM in the system. One TiB of NVDIMM requires 16 GiB for page structures. As a result, storing the data structures on the NVDIMM itself is preferable in such cases.
You can configure where per-page metadata are stored using the
--mapoption when configuring a namespace:-
To allocate in the system RAM, use
--map=mem. -
To allocate on the NVDIMM, use
--map=dev.
You can reconfigure an existing Non-Volatile Dual In-line Memory Modules (NVDIMM) namespace to file system DAX mode using ndctl. You can also Improve storage performance by enabling DAX, and to safely convert a namespace, understanding that reconfiguration will remove all previously stored data.
Reconfiguring a namespace deletes previously stored data on the namespace.
Prerequisites
-
The
ndctlutility is installed. For more information, see Installing ndctl.
Procedure
List all namespaces on your system:
# ndctl list --namespaces --idle [ { "dev":"namespace1.0", "mode":"raw", "size":34359738368, "uuid":"ac951312-b312-4e76-9f15-6e00c8f2e6f4" "state":"disabled", "numa_node":1 }, { "dev":"namespace0.0", "mode":"raw", "size":38615912448, "uuid":"ff5a0a16-3495-4ce8-b86b-f0e3bd9d1817", "state":"disabled", "numa_node":0 } ]Reconfigure any namespace:
# ndctl create-namespace --force --mode=fsdax --reconfig=namespace-ID例 18.2. Reconfiguring a namespace as file system DAX
To use
namespace0.0for a file system that supports DAX, use the following command:# ndctl create-namespace --force --mode=fsdax --reconfig=namespace0.0 { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":"11.81 GiB (12.68 GB)", "uuid":"f8153ee3-c52d-4c6e-bc1d-197f5be38483", "sector_size":512, "align":2097152, "blockdev":"pmem0" }The namespace is now available at the
/dev/pmem0path.
Verification
Verify if the existing namespaces on your system is reconfigured:
# ndctl list --namespace namespace0.0 [ { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":12681478144, "uuid":"f8153ee3-c52d-4c6e-bc1d-197f5be38483", "sector_size":512, "align":2097152, "blockdev":"pmem0" } ]
You can create a new file system DAX namespace on a Non-Volatile Dual In-line Memory Modules (NVDIMM) device if there is available space in the region.
Prerequisites
-
The
ndctlutility is installed. For more information, see Installing ndctl. The NVDIMM device supports labels to create multiple namespaces in a region. You can check this using the following command:
# ndctl read-labels nmem0 >/dev/null read 1 nmemThis indicates that it read the label of one NVDIMM device. If the value is
0, it implies that your device does not support labels.
Procedure
List the
pmemregions on your system that have available space. In the following example, space is available in the region1 and region0 regions:# ndctl list --regions [ { "dev":"region1", "size":2156073582592, "align":16777216, "available_size":2117418876928, "max_available_extent":2117418876928, "type":"pmem", "iset_id":-9102197055295954944, "badblock_count":1, "persistence_domain":"memory_controller" }, { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2143188680704, "max_available_extent":2143188680704, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller" } ]Allocate one or more namespaces on any of the available regions:
# ndctl create-namespace --mode=fsdax --region=regionN --size=namespace-sizeFor example, the following command creates a 36-GiB file system DAX namespace on region0:
# ndctl create-namespace --mode=fsdax --region=region0 --size=36G { "dev":"namespace0.3", "mode":"fsdax", "map":"dev", "size":"35.44 GiB (38.05 GB)", "uuid":"99e77865-42eb-4b82-9db6-c6bc9b3959c2", "sector_size":512, "align":2097152, "blockdev":"pmem0.3" }The namespace is now available as
/dev/pmem0.3.
Verification
Verify if the new namespace is created in file system DAX mode:
# ndctl list -RN -n namespace0.3 { "regions":[ { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2027224563712, "max_available_extent":2027224563712, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller", "namespaces":[ { "dev":"namespace0.3", "mode":"fsdax", "map":"dev", "size":38048628736, "uuid":"99e77865-42eb-4b82-9db6-c6bc9b3959c2", "sector_size":512, "align":2097152, "blockdev":"pmem0.3" } ] } ] }
You can create a file system on a file system DAX device and mount the file system. After creating a file system, applications can use persistent memory and create files in the mount-point directory, open the files, and use the mmap operation to map the files for direct access.
Procedure
Optional: Create a partition on the file system DAX device. For more information, see Creating a partition with parted.
注意When creating partitions on an
fsdaxdevice, partitions must be aligned on page boundaries. On the Intel 64 and AMD64 architecture, at least 4 KiB alignment is required for the start and end of the partition. 2 MiB is the preferred alignment.By default, the
partedtool aligns partitions on 1 MiB boundaries. For the first partition, specify 2 MiB as the start of the partition. If the size of the partition is a multiple of 2 MiB, all other partitions are also aligned.Create an XFS or ext4 file system on the partition or the NVDIMM device:
# mkfs.xfs -d su=2m,sw=1 fsdax-partition-or-device注意The dax-capable and reflinked files can now co-exist on the file system. However, for an individual file, dax and reflink are mutually exclusive.
Also, to increase the likelihood of large page mappings, set the stripe unit and stripe width.
Mount the file system:
# mount f_sdax-partition-or-device mount-point_There is no need to mount a file system with the dax option to enable direct access mode. When you do not specify any dax option while mounting, the file system is in the
dax=inodemode. Set the dax option on the file before direct access mode is activated.