8.3. Kernel Same-page Merging (KSM)
Kernel same-page Merging (KSM), used by the KVM hypervisor, allows KVM guests to share identical memory pages. These shared pages are usually common libraries or other identical, high-use data. KSM allows for greater guest density of identical or similar guest operating systems by avoiding memory duplication.
The concept of shared memory is common in modern operating systems. For example, when a program is first started, it shares all of its memory with the parent program. When either the child or parent program tries to modify this memory, the kernel allocates a new memory region, copies the original contents and allows the program to modify this new region. This is known as copy on write.
KSM is a Linux feature which uses this concept in reverse. KSM enables the kernel to examine two or more already running programs and compare their memory. If any memory regions or pages are identical, KSM reduces multiple identical memory pages to a single page. This page is then marked copy on write. If the contents of the page is modified by a guest virtual machine, a new page is created for that guest.
This is useful for virtualization with KVM. When a guest virtual machine is started, it only inherits the memory from the host
qemu-kvm process. Once the guest is running, the contents of the guest operating system image can be shared when guests are running the same operating system or applications. KSM allows KVM to request that these identical guest memory regions be shared.
KSM provides enhanced memory speed and utilization. With KSM, common process data is stored in cache or in main memory. This reduces cache misses for the KVM guests, which can improve performance for some applications and operating systems. Secondly, sharing memory reduces the overall memory usage of guests, which allows for higher densities and greater utilization of resources.
Note
In Red Hat Enterprise Linux 7, KSM is NUMA aware. This allows it to take NUMA locality into account while coalescing pages, thus preventing performance drops related to pages being moved to a remote node. Red Hat recommends avoiding cross-node memory merging when KSM is in use. If KSM is in use, change the
/sys/kernel/mm/ksm/merge_across_nodes tunable to 0 to avoid merging pages across NUMA nodes. This can be done with the virsh node-memory-tune --shm-merge-across-nodes 0 command. Kernel memory accounting statistics can eventually contradict each other after large amounts of cross-node merging. As such, numad can become confused after the KSM daemon merges large amounts of memory. If your system has a large amount of free memory, you may achieve higher performance by turning off and disabling the KSM daemon. See Chapter 9, NUMA" for more information on NUMA.
Important
Ensure the swap size is sufficient for the committed RAM even without taking KSM into account. KSM reduces the RAM usage of identical or similar guests. Overcommitting guests with KSM without sufficient swap space may be possible, but is not recommended because guest virtual machine memory use can result in pages becoming unshared.
Red Hat Enterprise Linux uses two separate methods for controlling KSM:
- The
ksmservice starts and stops the KSM kernel thread. - The
ksmtunedservice controls and tunes theksmservice, dynamically managing same-page merging.ksmtunedstarts theksmservice and stops theksmservice if memory sharing is not necessary. When new guests are created or destroyed,ksmtunedmust be instructed with theretuneparameter to run.
Both of these services are controlled with the standard service management tools.
Note
KSM is off by default on Red Hat Enterprise Linux 6.7.
8.3.1. The KSM Service
- The
ksmservice is included in the qemu-kvm package. - When the
ksmservice is not started, Kernel same-page merging (KSM) shares only 2000 pages. This default value provides limited memory-saving benefits. - When the
ksmservice is started, KSM will share up to half of the host system's main memory. Start theksmservice to enable KSM to share more memory.
# systemctl start ksm
Starting ksm: [ OK ]
The
ksm service can be added to the default startup sequence. Make the ksm service persistent with the systemctl command.
# systemctl enable ksm
8.3.2. The KSM Tuning Service
The
ksmtuned service fine-tunes the kernel same-page merging (KSM) configuration by looping and adjusting ksm. In addition, the ksmtuned service is notified by libvirt when a guest virtual machine is created or destroyed. The ksmtuned service has no options.
# systemctl start ksmtuned
Starting ksmtuned: [ OK ]
The
ksmtuned service can be tuned with the retune parameter, which instructs ksmtuned to run tuning functions manually.
The
/etc/ksmtuned.conf file is the configuration file for the ksmtuned service. The file output below is the default ksmtuned.conf file:
# Configuration file for ksmtuned. # How long ksmtuned should sleep between tuning adjustments # KSM_MONITOR_INTERVAL=60 # Millisecond sleep between ksm scans for 16Gb server. # Smaller servers sleep more, bigger sleep less. # KSM_SLEEP_MSEC=10 # KSM_NPAGES_BOOST - is added to the `npages` value, when `free memory` is less than `thres`. # KSM_NPAGES_BOOST=300 # KSM_NPAGES_DECAY - is the value given is subtracted to the `npages` value, when `free memory` is greater than `thres`. # KSM_NPAGES_DECAY=-50 # KSM_NPAGES_MIN - is the lower limit for the `npages` value. # KSM_NPAGES_MIN=64 # KSM_NPAGES_MAX - is the upper limit for the `npages` value. # KSM_NPAGES_MAX=1250 # KSM_THRES_COEF - is the RAM percentage to be calculated in parameter `thres`. # KSM_THRES_COEF=20 # KSM_THRES_CONST - If this is a low memory system, and the `thres` value is less than `KSM_THRES_CONST`, then reset `thres` value to `KSM_THRES_CONST` value. # KSM_THRES_CONST=2048 # uncomment the following to enable ksmtuned debug information # LOGFILE=/var/log/ksmtuned # DEBUG=1
Within the
/etc/ksmtuned.conf file, npages sets how many pages ksm will scan before the ksmd daemon becomes inactive. This value will also be set in the /sys/kernel/mm/ksm/pages_to_scan file.
The
KSM_THRES_CONST value represents the amount of available memory used as a threshold to activate ksm. ksmd is activated if either of the following occurs:
- The amount of free memory drops below the threshold, set in
KSM_THRES_CONST. - The amount of committed memory plus the threshold,
KSM_THRES_CONST, exceeds the total amount of memory.
8.3.3. KSM Variables and Monitoring
Kernel same-page merging (KSM) stores monitoring data in the
/sys/kernel/mm/ksm/ directory. Files in this directory are updated by the kernel and are an accurate record of KSM usage and statistics.
The variables in the list below are also configurable variables in the
/etc/ksmtuned.conf file, as noted above.
Files in /sys/kernel/mm/ksm/:
- full_scans
- Full scans run.
- merge_across_nodes
- Whether pages from different NUMA nodes can be merged.
- Total pages shared.
- pages_sharing
- Pages currently shared.
- pages_to_scan
- Pages not scanned.
- Pages no longer shared.
- pages_volatile
- Number of volatile pages.
- run
- Whether the KSM process is running.
- sleep_millisecs
- Sleep milliseconds.
These variables can be manually tuned using the
virsh node-memory-tune command. For example, the following specifies the number of pages to scan before the shared memory service goes to sleep:
# virsh node-memory-tune --shm-pages-to-scan number
KSM tuning activity is stored in the
/var/log/ksmtuned log file if the DEBUG=1 line is added to the /etc/ksmtuned.conf file. The log file location can be changed with the LOGFILE parameter. Changing the log file location is not advised and may require special configuration of SELinux settings.
8.3.4. Deactivating KSM
Kernel same-page merging (KSM) has a performance overhead which may be too large for certain environments or host systems. KSM may also introduce side channels that could be potentially used to leak information across guests. If this is a concern, KSM can be disabled on per-guest basis.
KSM can be deactivated by stopping the
ksmtuned and the ksm services. However, this action does not persist after restarting. To deactivate KSM, run the following in a terminal as root:
# systemctl stop ksmtuned Stopping ksmtuned: [ OK ] # systemctl stop ksm Stopping ksm: [ OK ]
Stopping the
ksmtuned and the ksm deactivates KSM, but this action does not persist after restarting. Persistently deactivate KSM with the systemctl commands:
# systemctl disable ksm
# systemctl disable ksmtuned
When KSM is disabled, any memory pages that were shared prior to deactivating KSM are still shared. To delete all of the PageKSM in the system, use the following command:
# echo 2 >/sys/kernel/mm/ksm/run
After this is performed, the
khugepaged daemon can rebuild transparent hugepages on the KVM guest physical memory. Using # echo 0 >/sys/kernel/mm/ksm/run stops KSM, but does not unshare all the previously created KSM pages (this is the same as the # systemctl stop ksmtuned command).
