Chapter 28. Setting system resource limits for applications by using control groups
Use control groups (cgroups) kernel functionality to control application resource usage. Set limits for system resource allocation, prioritize hardware resources for specific processes, and isolate processes from obtaining hardware resources.
28.1. Introducing control groups Copy linkLink copied to clipboard!
Using the control groups Linux kernel feature, you can organize processes into hierarchically ordered groups called cgroups. You define the hierarchy by providing structure to the cgroups virtual file system, mounted by default on the /sys/fs/cgroup/ directory.
The systemd service manager uses cgroups to organize all units and services that it governs. Manually, you can manage the hierarchies of cgroups by creating and removing sub-directories in the /sys/fs/cgroup/ directory.
The resource controllers in the kernel then modify the behavior of processes in cgroups by limiting, prioritizing or allocating system resources, of those processes. These resources include the following:
- CPU time
- Memory
- Network bandwidth
- Combinations of these resources
The primary use case of cgroups is aggregating system processes and dividing hardware resources among applications and users. This makes it possible to increase the efficiency, stability, and security of your environment.
- Control groups version 1
Control groups version 1 (
cgroups-v1) provides a separate hierarchy for each resource controller. Resources such as CPU, memory, or I/O has its own control group hierarchy. You can combine different control group hierarchies so that one controller can coordinate with another in managing their individual resources. However, when the two controllers belong to different process hierarchies, the coordination is limited.The
cgroups-v1controllers were developed across a large time span, resulting in inconsistent behavior and naming of their control files.- Control groups version 2
Control groups version 2 (
cgroups-v2) provides a single control group hierarchy against which all resource controllers are mounted.The control file behavior and naming is consistent among different controllers.
ImportantRHEL 10, by default, mounts and uses
cgroups-v2.
For more details about cgroups-v1 and cgroups-v2, install the kernel-doc RPM package. After installation, the documentation is in the /usr/share/doc/kernel-doc-<version>/Documentation directory on the local system. The cgroups-v1 documentation files are in the Documentation/admin-guide/cgroup-v1/ directory. This directory has multiple files for different controllers. The cgroups-v2 documentation is in the Documentation/admin-guide/cgroup-v2.rst file.
28.2. Introducing kernel resource controllers Copy linkLink copied to clipboard!
Kernel resource controllers provide the functionality of control groups. RHEL 10 supports various controllers for control groups version 1 (cgroups-v1) and control groups version 2 (cgroups-v2).
A resource controller, also called a control group subsystem, is a kernel subsystem that represents a single resource, such as CPU time, memory, network bandwidth or disk I/O. The Linux kernel provides a range of resource controllers that are mounted automatically by the systemd service manager.
You can find a list of the currently mounted resource controllers in the /proc/cgroups file.
- Controllers available for
cgroups-v1 -
blkio: Sets limits on input/output access to and from block devices. -
cpu: Adjusts the parameters of the default scheduler for a control group’s tasks. Thecpucontroller is mounted together with thecpuacctcontroller on the same mount. -
cpuacct: Creates automatic reports on CPU resources used by tasks in a control group. Thecpuacctcontroller is mounted together with thecpucontroller on the same mount. -
cpuset:Restricts control group tasks to run only on a specified subset of CPUs and to direct the tasks to use memory only on specified memory nodes. -
devices: Controls access to devices for tasks in a control group. -
freezer: Suspends or resumes tasks in a control group. -
memory: Sets limits on memory use by tasks in a control group and generates automatic reports on memory resources used by those tasks. -
net_cls: Tags network packets with a class identifier (classid) that enables the Linux traffic controller (thetccommand) to identify packets that originate from a particular control group task. A subsystem ofnet_cls, thenet_filter(iptables), can also use this tag to perform actions on such packets. -
net_filter: Tags network sockets with a firewall identifier (fwid) that allows the Linux firewall to identify packets that originate from a particular control group task (by using theiptablescommand). -
net_prio: Sets the priority of network traffic. -
pids: Sets limits for multiple processes and their children in a control group. -
perf_event: Groups tasks for monitoring by theperfperformance monitoring and reporting utility. -
rdma: Sets limits on Remote Direct Memory Access/InfiniBand specific resources in a control group. -
hugetlb: Limits the usage of large size virtual memory pages by tasks in a control group.
-
- Controllers available for
cgroups-v2 -
io: Sets limits on input/output access to and from block devices. -
memory: Sets limits on memory use by tasks in a control group and generates automatic reports on memory resources used by those tasks. -
pids: Sets limits for multiple processes and their children in a control group. -
rdma: Sets limits on Remote Direct Memory Access/InfiniBand specific resources in a control group. -
cpu: Adjusts the parameters of the default scheduler for a control group’s tasks and creates automatic reports on CPU resources used by tasks in a control group. -
cpuset: Restricts control group tasks to run only on a specified subset of CPUs and to direct the tasks to use memory only on specified memory nodes. Supports only the core functionality (cpus{,.effective},mems{,.effective}) with a new partition feature. -
perf_event: Groups tasks for monitoring by theperfperformance monitoring and reporting utility.perf_eventis enabled automatically on the v2 hierarchy.
-
A resource controller can be used either in a cgroups-v1 hierarchy or a cgroups-v2 hierarchy, not simultaneously in both.
28.3. Introducing namespaces Copy linkLink copied to clipboard!
Namespaces create separate spaces for organizing and identifying software objects. This keeps them from affecting each other. As a result, each software object contains its own set of resources, for example, a mount point, a network device, or a hostname, even though they are sharing the same system.
One of the most common technologies that use namespaces are containers.
Changes to a particular global resource are visible only to processes in that namespace and do not affect the rest of the system or other namespaces.
To inspect which namespaces a process is a member of, you can check the symbolic links in the /proc/<PID>/ns/ directory.
| Namespace | Isolates |
|---|---|
| Mount | Mount points |
| UTS | Hostname and NIS domain name |
| IPC | SysV IPC, POSIX message queues |
| PID | Process IDs |
| Network | Network devices, stacks, ports, and so on |
| User | User and group IDs |
| Control groups | Control group root directory |
See namespaces(7) and cgroup_namespaces(7) man pages on your system for more information.