Chapter 7. KSM
The concept of shared memory is common in modern operating systems. For example, when a program is first started it shares all of its memory with the parent program. When either the child or parent program tries to modify this memory, the kernel allocates a new memory region, copies the original contents and allows the program to modify this new region. This is known as copy on write.
KSM is a new Linux feature which uses this concept in reverse. KSM enables the kernel to examine two or more already running programs and compare their memory. If any memory regions or pages are identical, KSM reduces multiple identical memory pages to a single page. This page is then marked copy on write. If the contents of the page is modified by a guest virtual machine, a new page is created for that guest virtual machine.
This is useful for virtualization with KVM. When a guest virtual machine is started, it inherits only the memory from the parent
qemu-kvm process. Once the guest virtual machine is running, the contents of the guest virtual machine operating system image can be shared when guests are running the same operating system or applications.
Note
The page deduplication technology (used also by the KSM implementation) may introduce side channels that could potentially be used to leak information across multiple guests. In case this is a concern, KSM can be disabled on a per-guest basis.
KSM provides enhanced memory speed and utilization. With KSM, common process data is stored in cache or in main memory. This reduces cache misses for the KVM guests which can improve performance for some applications and operating systems. Secondly, sharing memory reduces the overall memory usage of guests which allows for higher densities and greater utilization of resources.
Note
Starting in Red Hat Enterprise Linux 6.5, KSM is NUMA aware. This allows it to take NUMA locality into account while coalescing pages, thus preventing performance drops related to pages being moved to a remote node. Red Hat recommends avoiding cross-node memory merging when KSM is in use. If KSM is in use, change the
/sys/kernel/mm/ksm/merge_across_nodes tunable to 0 to avoid merging pages across NUMA nodes. Kernel memory accounting statistics can eventually contradict each other after large amounts of cross-node merging. As such, numad can become confused after the KSM daemon merges large amounts of memory. If your system has a large amount of free memory, you may achieve higher performance by turning off and disabling the KSM daemon. Refer to the Red Hat Enterprise Linux Performance Tuning Guide for more information on NUMA.
Red Hat Enterprise Linux uses two separate methods for controlling KSM:
- The
ksmservice starts and stops the KSM kernel thread. - The
ksmtunedservice controls and tunes theksm, dynamically managing same-page merging. Theksmtunedservice startsksmand stops theksmservice if memory sharing is not necessary. Theksmtunedservice must be told with theretuneparameter to run when new guests are created or destroyed.
Both of these services are controlled with the standard service management tools.
The KSM Service
The ksm service is included in the qemu-kvm package. KSM is off by default on Red Hat Enterprise Linux 6. When using Red Hat Enterprise Linux 6 as a KVM host physical machine, however, it is likely turned on by the ksm/ksmtuned services.
When the
ksm service is not started, KSM shares only 2000 pages. This default is low and provides limited memory saving benefits.
When the
ksm service is started, KSM will share up to half of the host physical machine system's main memory. Start the ksm service to enable KSM to share more memory.
# service ksm start Starting ksm: [ OK ]
The
ksm service can be added to the default startup sequence. Make the ksm service persistent with the chkconfig command.
# chkconfig ksm on
The KSM Tuning Service
The ksmtuned service does not have any options. The ksmtuned service loops and adjusts ksm. The ksmtuned service is notified by libvirt when a guest virtual machine is created or destroyed.
# service ksmtuned start Starting ksmtuned: [ OK ]
The
ksmtuned service can be tuned with the retune parameter. The retune parameter instructs ksmtuned to run tuning functions manually.
Before changing the parameters in the file, there are a few terms that need to be clarified:
thres- Activation threshold, in kbytes. A KSM cycle is triggered when thethresvalue added to the sum of allqemu-kvmprocesses RSZ exceeds total system memory. This parameter is the equivalent in kbytes of the percentage defined inKSM_THRES_COEF.
The
/etc/ksmtuned.conf file is the configuration file for the ksmtuned service. The file output below is the default ksmtuned.conf file.
# Configuration file for ksmtuned. # How long ksmtuned should sleep between tuning adjustments # KSM_MONITOR_INTERVAL=60 # Millisecond sleep between ksm scans for 16Gb server. # Smaller servers sleep more, bigger sleep less. # KSM_SLEEP_MSEC=10 # KSM_NPAGES_BOOST is added to thenpagesvalue, whenfree memoryis less thanthres. # KSM_NPAGES_BOOST=300 # KSM_NPAGES_DECAY Value given is subtracted to thenpagesvalue, whenfree memoryis greater thanthres. # KSM_NPAGES_DECAY=-50 # KSM_NPAGES_MIN is the lower limit for thenpagesvalue. # KSM_NPAGES_MIN=64 # KSM_NAGES_MAX is the upper limit for thenpagesvalue. # KSM_NPAGES_MAX=1250 # KSM_TRES_COEF - is the RAM percentage to be calculated in parameterthres. # KSM_THRES_COEF=20 # KSM_THRES_CONST - If this is a low memory system, and thethresvalue is less thanKSM_THRES_CONST, then resetthresvalue toKSM_THRES_CONSTvalue. # KSM_THRES_CONST=2048 # uncomment the following to enable ksmtuned debug information # LOGFILE=/var/log/ksmtuned # DEBUG=1
KSM Variables and Monitoring
KSM stores monitoring data in the /sys/kernel/mm/ksm/ directory. Files in this directory are updated by the kernel and are an accurate record of KSM usage and statistics.
The variables in the list below are also configurable variables in the
/etc/ksmtuned.conf file as noted below.
The /sys/kernel/mm/ksm/ files
- full_scans
- Full scans run.
- pages_shared
- Total pages shared.
- pages_sharing
- Pages presently shared.
- pages_to_scan
- Pages not scanned.
- pages_unshared
- Pages no longer shared.
- pages_volatile
- Number of volatile pages.
- run
- Whether the KSM process is running.
- sleep_millisecs
- Sleep milliseconds.
KSM tuning activity is stored in the
/var/log/ksmtuned log file if the DEBUG=1 line is added to the /etc/ksmtuned.conf file. The log file location can be changed with the LOGFILE parameter. Changing the log file location is not advised and may require special configuration of SELinux settings.
Deactivating KSM
KSM has a performance overhead which may be too large for certain environments or host physical machine systems.
KSM can be deactivated by stopping the
ksmtuned and the ksm service. Stopping the services deactivates KSM but does not persist after restarting.
# service ksmtuned stop Stopping ksmtuned: [ OK ] # service ksm stop Stopping ksm: [ OK ]
Persistently deactivate KSM with the
chkconfig command. To turn off the services, run the following commands:
# chkconfig ksm off # chkconfig ksmtuned off
Important
Ensure the swap size is sufficient for the committed RAM even with KSM. KSM reduces the RAM usage of identical or similar guests. Overcommitting guests with KSM without sufficient swap space may be possible but is not recommended because guest virtual machine memory use can result in pages becoming unshared.
