Red Hat SummitChapter 21. Managing Out of Memory states
Out-of-memory (OOM) is a computing state where all available memory, including swap space, has been allocated. Normally this causes the system to panic and stop functioning as expected. The provided instructions help in avoiding OOM states on your system.
21.1. Prerequisites
- You have root permissions on the system.
21.2. Changing the Out of Memory value
The /proc/sys/vm/panic_on_oom file contains a value which is the switch that controls Out of Memory (OOM) behavior. When the file contains 1, the kernel panics on OOM and stops functioning as expected.
The default value is 0, which instructs the kernel to call the oom_killer() function when the system is in an OOM state. Usually, oom_killer() terminates unnecessary processes, which allows the system to survive.
You can change the value of /proc/sys/vm/panic_on_oom.
Prerequisites
- You have root permissions on the system.
Procedure
Display the current value of
/proc/sys/vm/panic_on_oom.# cat /proc/sys/vm/panic_on_oom 0To change the value in
/proc/sys/vm/panic_on_oom:Echo the new value to
/proc/sys/vm/panic_on_oom.# echo 1 > /proc/sys/vm/panic_on_oomNoteMake the Real-Time kernel panic on OOM (
1). Otherwise, when the system encounters an OOM state, it no longer remains deterministic.
Verification
Display the value of
/proc/sys/vm/panic_on_oom.# cat /proc/sys/vm/panic_on_oom 1- Verify that the displayed value matches the value specified.
21.3. Prioritizing processes to end when in an Out of Memory state
You can prioritize which processes the oom_killer() function ends by adjusting the oom_score_adj value for each process, ensuring that high-priority processes keep running during an Out of Memory (OOM) state.
Each process has a directory at /proc/PID/ that contains the following files:
-
oom_score_adj- Valid scores foroom_score_adjare in the range -16 to +15. This value is used to calculate the performance footprint of the process, using an algorithm that also takes into account how long the process has been running, among other factors. -
oom_score- Contains the result of the algorithm calculated using the value inoom_score_adj.
In an OOM state, the oom_killer() function ends processes with the highest oom_score. You can prioritize which processes are ended by editing the oom_score_adj file for the process.
Prerequisites
- Know the process ID (PID) of the process you want to prioritize.
Procedure
Display the current
oom_scorefor a process.# cat /proc/12465/oom_score 79872Display the contents of
oom_score_adjfor the process.# *cat /proc/12465/oom_score_adj * 13Edit the value in
oom_score_adj.# *echo -5 > /proc/12465/oom_score_adj *
Verification
Display the current
oom_scorefor the process.# cat /proc/12465/oom_score 78- Verify that the displayed value is lower than the previous value.
21.4. Disabling the Out of Memory killer for a process
You can disable the oom_killer() function for a process by setting oom_score_adj to the reserved value of -17. This keeps the process alive, even in an OOM state.
Prerequisites
- You have root permissions on the system.
Procedure
Set the value in
oom_score_adjto-17.# echo -17 > /proc/12465/oom_score_adj
Verification
Display the current
oom_scorefor the process.# cat /proc/12465/oom_score 0-
Verify that the displayed value is
0.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}