Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 12. Configuring huge pages

The system manages physical memory in fixed-size pages. The default 4KiB page size on x86_64 is suitable for general workloads, but applications with large data sets benefit from huge pages. Configure huge pages in Red Hat Enterprise Linux 10 to improve performance and reduce resource usage.

12.1. Available huge page features
Copy link

With Red Hat Enterprise Linux, you can use huge pages for applications that work with big data sets, and improve the performance of such applications.

The following are the huge page methods, which are supported in RHEL:

HugeTLB pages

HugeTLB pages are also called static huge pages. There are two ways of reserving HugeTLB pages:

  • At boot time: It increases the possibility of success because the memory has not yet been significantly fragmented. However, on NUMA machines, the number of pages is automatically split among the NUMA nodes.
  • At runtime: It allows you to reserve the huge pages per NUMA node. If the runtime reservation is done as early as possible in the boot process, the probability of memory fragmentation is lower.
Transparent HugePages (THP)

With THP, the kernel automatically assigns huge pages to processes, and therefore there is no need to manually reserve the static huge pages. The following are the two modes of operation in THP:

  • system-wide: Here, the kernel tries to assign huge pages to a process whenever it is possible to allocate the huge pages and the process is using a large contiguous virtual memory area.

  • per-process: Here, the kernel only assigns huge pages to the memory areas of individual processes which you can specify using the madvise() system call.

    Note

    The THP feature only supports 2 MB pages.

Influence HugeTLB page behavior at boot time by using certain kernel parameters.

For more information about how to use these parameters to configure HugeTLB pages at boot time, see Configuring HugeTLB at boot time.

Expand
Table 12.1. Parameters used to configure HugeTLB pages at boot time
ParameterDescriptionDefault value

hugepages

Defines the number of persistent huge pages configured in the kernel at boot time.

In a NUMA system, huge pages, that have this parameter defined, are divided equally between nodes.

You can assign huge pages to specific nodes at runtime by changing the value of the nodes in the /sys/devices/system/node/node_id/hugepages/hugepages-size/nr_hugepages file.

The default value is 0.

To update this value at boot, change the value of this parameter in the /proc/sys/vm/nr_hugepages file.

hugepagesz

Defines the size of persistent huge pages configured in the kernel at boot time.

Valid values are 2 MB and 1 GB. The default value is 2 MB.

default_hugepagesz

Defines the default size of persistent huge pages configured in the kernel at boot time.

Valid values are 2 MB and 1 GB. The default value is 2 MB.

12.3. Configuring HugeTLB at boot time
Copy link

With the Huge Translation Lookaside Buffer (HugeTLB), you can use huge pages by reserving them at boot time. You can minimize the memory fragmentation to ensure that sufficient resources are available for workloads that can benefit from larger memory pages.

To reserve Huge Translation Lookaside Buffer (HugeTLB) pages at the earliest boot stage, you can use kernel command-line parameters. This method ensures the highest chance of successfully reserving the required number and size of huge pages, as memory is allocated during the kernel boot.

Prefer reserving HugeTLB pages by using kernel boot parameters, as this method ensures allocation of larger contiguous memory regions compared to using a systemd unit.

Note

The examples in the procedure demonstrate how to use the command-line options for configuring HugeTLB pages. These examples do not necessarily apply to your system configuration. Review your system requirements and objectives before applying these settings in your environment.

Prerequisites

  • You must have root privileges on your system.

Procedure

  • Update the kernel command line to include HugeLTB options.

    • To reserve HugeTLB pages of the default size for your architecture, enter: 

      # grubby --update-kernel=ALL --args="hugepages=10"

      This command reserves 10 HugeTLB pages of the default pool size. For example, on x86_64, the default pool size is 2 MB. On systems with Non-Uniform Memory Access (NUMA), the allocation is distributed evenly across NUMA nodes. If the system has two NUMA nodes, each node reserves five pages.

    • To reserve different sizes of HugeTLB pages, specify the hugepagesz and hugepages options in the kernel command line, enter: 

      # grubby --update-kernel=ALL --args="hugepagesz=2M hugepages=10 hugepagesz=1G hugepages=1"

      This command reserves 10 pages of 2 MB each and 1 page of 1 GB.

      The system reserves the specified number and size of HugeTLB pages at boot time, ensuring that memory is allocated before the operating system begins normal operation.

      Important

      The order of the options is significant. Each hugepagesz= option must be immediately followed by its corresponding hugepages= option.

To configure Huge Translation Lookaside Buffer (HugeTLB) pages during user-space booting, use a systemd service unit. This reserves memory after kernel initialization but before most services start, ensuring applications have access to the required pages.

Prerequisites

  • You must have root privileges on your system.

Procedure

  1. Create a new file called hugetlb-gigantic-pages.service in the /usr/lib/systemd/system/ directory and add the following content: 

    [Unit]
Description=HugeTLB Gigantic Pages Reservation
DefaultDependencies=no
Before=dev-hugepages.mount
ConditionPathExists=/sys/devices/system/node
ConditionKernelCommandLine=hugepagesz=1G

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/lib/systemd/hugetlb-reserve-pages.sh

[Install]
WantedBy=sysinit.target

  2. Create a new file called hugetlb-reserve-pages.sh in the /usr/lib/systemd/ directory and add the following content: 

    #!/bin/sh

nodes_path=/sys/devices/system/node/
if [ ! -d $nodes_path ]; then
    echo "ERROR: $nodes_path does not exist"
    exit 1
fi

reserve_pages()
{
    echo $1 > $nodes_path/$2/hugepages/hugepages-1048576kB/nr_hugepages
}

reserve_pages <number_of_pages> <node>

    Replace <number_of_pages> with the number of 1GB pages you want to reserve, and <node> with the node name on which to reserve these pages. For example, to reserve two 1 GB pages on node0 and one 1GB page on node1, replace <number_of_pages> with 2 for node0 and 1 for node1.

  3. Create an executable script: 

    # chmod +x /usr/lib/systemd/hugetlb-reserve-pages.sh

  4. Enable early boot reservation: 

    # systemctl enable hugetlb-gigantic-pages.service
    Note
    • You can try reserving more 1 GB pages at runtime by writing to the nr_hugepages attribute at any time. However, to prevent failures due to memory fragmentation, reserve 1 GB pages early during the boot process.
    • Reserving static huge pages can effectively reduce the amount of memory available to the system, and prevent it from using its full memory capacity. Although a properly sized pool of reserved huge pages can be beneficial to applications that use it, an oversized or unused pool of reserved huge pages will eventually be detrimental to the overall system performance. When setting a reserved huge page pool, ensure that the system can properly use its full memory capacity.

Influence HugeTLB page behavior at run time by using certain kernel parameters.

For more information about how to use these parameters to configure HugeTLB pages at run time, see Configuring HugeTLB at run time.

Expand
Table 12.2. Parameters used to configure HugeTLB pages at run time
ParameterDescriptionFile name

nr_hugepages

Defines the number of huge pages of a specified size assigned to a specified NUMA node.

/sys/devices/system/node/node_id/hugepages/hugepages-size/nr_hugepages

nr_overcommit_hugepages

Defines the maximum number of additional huge pages that can be created and used by the system through overcommitting memory.

Writing any nonzero value into this file indicates that the system obtains that number of huge pages from the kernel’s normal page pool if the persistent huge page pool is exhausted. As these surplus huge pages become unused, they are then freed and returned to the kernel’s normal page pool.

/proc/sys/vm/nr_overcommit_hugepages

12.5. Configuring HugeTLB at run time
Copy link

Configure HugeTLB pages at runtime to improve memory performance for applications that require large memory allocations. Reserve huge pages on specific NUMA nodes with specified sizes.

Procedure

  1. Display the memory statistics: 

    # numastat -cm | egrep 'Node|Huge'
     
                     Node 0 Node 1 Node 2 Node 3  Total add
AnonHugePages         0      2      0      8     10
HugePages_Total       0      0      0      0      0
HugePages_Free        0      0      0      0      0
HugePages_Surp        0      0      0      0      0

  2. Add the number of huge pages of a specified size to the node: 

    # echo 20 > /sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages

    Replace the following values:

    • 20 with the number of huge pages you want to reserve
    • 2048 with the size of the huge pages in KiB

    • node2 with the NUMA node on which you want to reserve the pages

      Note

      The directory name format in the path is hugepages-size_KiB where size is specified in KiB. The actual directory name might appear as hugepages-2048KiB or hugepages-2048kB depending on your system.

Verification

  • Ensure that the number of huge pages are added: 

    # numastat -cm | egrep 'Node|Huge'
     
                     Node 0 Node 1 Node 2 Node 3  Total
AnonHugePages         0      2      0      8     10
HugePages_Total       0      0     40      0     40
HugePages_Free        0      0     40      0     40
HugePages_Surp        0      0      0      0      0

    See numastat(8) man page for more information.

12.6. Managing transparent huge pages
Copy link

Transparent huge pages (THP) are enabled by default in Red Hat Enterprise Linux 10. However, you can enable, disable, or set the transparent huge pages to madvise with runtime configuration, TuneD profiles, kernel command line parameters, or systemd unit file.

To optimize memory usage, manage transparent huge pages (THP) at runtime. Note that runtime configuration does not persist across reboots.

Procedure

  1. Check the status of THP: 

    $ cat /sys/kernel/mm/transparent_hugepage/enabled

  2. Configure THP.

    • Enabling THP: 

      $ echo always > /sys/kernel/mm/transparent_hugepage/enabled

    • Disabling THP: 

      $ echo never > /sys/kernel/mm/transparent_hugepage/enabled

    • Setting THP to madvise: 

      $ echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

      To prevent applications from allocating more memory resources than necessary, disable the system-wide transparent huge pages and only enable them for the applications that explicitly request it through the madvise system call.

      Note

      Sometimes, providing low latency to short-lived allocations has higher priority than immediately achieving the best performance with long-lived allocations. In such cases, you can disable direct compaction while leaving THP enabled.

      Direct compaction is a synchronous memory compaction during the huge page allocation. Disabling direct compaction provides no guarantee of saving memory, but can decrease the risk of higher latencies during frequent page faults. Also, disabling direct compaction allows synchronous compaction of Virtual Memory Areas (VMAs) highlighted in madvise only. Note that if the workload benefits significantly from THP, the performance decreases. Disable direct compaction:

      $ echo never > /sys/kernel/mm/transparent_hugepage/defrag

      See the madvise(2) man page on your system for more information.

To manage transparent huge pages (THP) persistently, use TuneD profiles. Configure these profiles in the tuned.conf file.

Prerequisites

  • TuneD package is installed.
  • TuneD service is enabled.

Procedure

  1. Copy the active profile file to the same directory: 

    $ sudo cp -R /usr/lib/tuned/my_profile /usr/lib/tuned/my_copied_profile

  2. Edit the tune.conf file: 

    $ sudo vi /usr/lib/tuned/my_copied_profile/tuned.conf

    • To enable THP, add the line: 

      [bootloader]

cmdline = transparent_hugepage=always

    • To disable THP, add the line: 

      [bootloader]

cmdline = transparent_hugepage=never

    • To set THP to madvise, add the line: 

      [bootloader]

cmdline = transparent_hugepage=madvise

  3. Restart the TuneD service: 

    $ sudo systemctl restart tuned

  4. Set the new profile active: 

    $ sudo tuned-adm profile my_copied_profile

Verification

  1. Verify that the new profile is active: 

    $ sudo tuned-adm active

  2. Verify that the required mode of THP is set: 

    $ cat /sys/kernel/mm/transparent_hugepage/enabled

To manage transparent huge pages (THP) at boot time, modify the kernel parameters. This configuration persists across reboots.

Prerequisites

  • You have root permissions on the system.

Procedure

  1. Get the current kernel command line parameters: 

    # grubby --info=$(grubby --default-kernel)
     
    kernel="/boot/vmlinuz-6.12.X-XXX.XX.X.el10_0.x86_64"
args="ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXX console=tty0 console=ttyS0"
root="UUID=XXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
initrd="/boot/initramfs-6.12.X-XXX.XX.X.el10_0.x86_64.img"
title="Red Hat Enterprise Linux (6.12.X-XXX.XX.X.el10_0.x86_64) 10.0"
id="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-6.12.X-XXX.XX.X.el10_0.x86_64"

  2. Configure THP by adding kernel parameters.

    • To enable THP: 

      # grubby --args="transparent_hugepage=always" --update-kernel=DEFAULT

    • To disable THP: 

      # grubby --args="transparent_hugepage=never" --update-kernel=DEFAULT

    • To set THP to madvise: 

      # grubby --args="transparent_hugepage=madvise" --update-kernel=DEFAULT

  3. Reboot the system for changes to take effect: 

    # reboot

Verification

  • To verify the status of THP, view the following files: 

    # cat /sys/kernel/mm/transparent_hugepage/enabled
     
    always madvise [never]
     
    # grep AnonHugePages: /proc/meminfo
     
    AnonHugePages:         0 kB
     
    # grep nr_anon_transparent_hugepages /proc/vmstat
     
    nr_anon_transparent_hugepages 0

To manage transparent huge pages (THP) at system startup, use systemd unit files. Creating a service ensures consistent THP configuration across reboots.

Prerequisites

  • You have root permissions on the system.

Procedure

  1. Create new systemd service files for enabling, disabling and setting THP to madvise. For example, /etc/systemd/system/disable-thp.service.

  2. Configure THP by adding the following contents to a new systemd service file.

    • To enable THP, add the following content to <new_thp_file>.service file: 

      [Unit]
Description=Enable Transparent Hugepages
After=local-fs.target
Before=sysinit.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c 'echo always > /sys/kernel/mm/transparent_hugepage/enabled

[Install]
WantedBy=multi-user.target

    • To disable THP, add the following content to <new_thp_file>.service file: 

      [Unit]
Description=Disable Transparent Hugepages
After=local-fs.target
Before=sysinit.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled

[Install]
WantedBy=multi-user.target

    • To set THP to madvise, add the following content to <new_thp_file>.service file: 

      [Unit]
Description=Madvise Transparent Hugepages
After=local-fs.target
Before=sysinit.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

[Install]
WantedBy=multi-user.target

  3. Enable and start the service: 

    # systemctl enable <new_thp_file>.service
     
    # systemctl start <new_thp_file>.service

Verification

  • To verify the status of THP, view the following files: 

    $ cat /sys/kernel/mm/transparent_hugepage/enabled

Reading address mappings from the page table is time-consuming and resource-expensive, so CPUs are built with a cache for recently-used addresses, called the Translation Lookaside Buffer (TLB). However, the default TLB can only cache a certain number of address mappings.

If a requested address mapping is not in the TLB, called a TLB miss, the system still needs to read the page table to determine the physical to virtual address mapping. Because of the relationship between application memory requirements and the size of pages used to cache address mappings, applications with large memory requirements are more likely to suffer performance degradation from TLB misses than applications with minimal memory requirements. It is therefore important to avoid TLB misses wherever possible.

Both HugeTLB and Transparent Huge Page features allow applications to use pages larger than 4 KB. This allows addresses stored in the TLB to reference more memory, which reduces TLB misses and improves application performance.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top