6.5. Managing the maximum number of pods per node
In OpenShift Container Platform, you can configure the number of pods that can run on a node based on the number of processor cores on the node, a hard limit, or both. If you use both options, the lower of the two limits the number of pods on a node. Setting a maximum number of pods can prevent a node from running more pods than its underlying hardware can handle.
When both options are in use, the lower of the two values limits the number of pods on a node. Exceeding these values can result in the following conditions:
- Increased CPU utilization.
- Slow pod scheduling.
- Potential out-of-memory scenarios, depending on the amount of memory in the node.
- Exhausting the pool of IP addresses.
- Resource overcommitting, leading to poor user application performance.
In Kubernetes, a pod that is holding a single container actually uses two containers. The second container is used to set up networking prior to the actual container starting. Therefore, a system running 10 pods will actually have 20 containers running.
Disk IOPS throttling from the cloud provider might have an impact on CRI-O and kubelet. They might get overloaded when there are large number of I/O intensive pods running on the nodes. It is recommended that you monitor the disk I/O on the nodes and use volumes with sufficient throughput for the workload.
The podsPerCore parameter sets the number of pods that the node can run based on the number of processor cores on the node. For example, if podsPerCore is set to 10 on a node with 4 processor cores, the maximum number of pods allowed on the node is 40.
kubeletConfig:
podsPerCore: 10
Setting podsPerCore to 0 disables this limit. The default is 0. The value of the podsPerCore parameter cannot exceed the value of the maxPods parameter.
The maxPods parameter sets the number of pods that the node can run to a fixed value, regardless of the properties of the node.
kubeletConfig:
maxPods: 250
6.5.1. Configuring the maximum number of pods per node 링크 복사링크가 클립보드에 복사되었습니다!
You can use the podsPerCore and maxPods parameters in a kubelet configuration to control the maximum number of pods that can be scheduled to a node. If you use both options, the lower of the two limits the number of pods on a node. Setting an appropriate maximum can help ensure your nodes run efficiently.
For example, if podsPerCore is set to 10 on a node with 4 processor cores, the maximum number of pods allowed on the node will be 40.
Prerequisites
-
You have the label associated with the static
MachineConfigPoolCRD for the type of node you want to configure.
Procedure
Create a custom resource (CR) for your configuration change.
Sample configuration for a
max-podsCRapiVersion: machineconfiguration.openshift.io/v1 kind: KubeletConfig metadata: name: set-max-pods spec: machineConfigPoolSelector: matchLabels: pools.operator.machineconfiguration.openshift.io/worker: "" kubeletConfig: podsPerCore: 10 maxPods: 250 #...where:
metadata.name- Specifies a name for the CR.
spec.machineConfigPoolSelector.matchLabels- Specifies the label from the machine config pool.
spec.kubeletConfig.podsPerCore- Specifies the number of pods the node can run based on the number of processor cores on the node.
spec.kubeletConfig.maxPodsSpecifies the number of pods the node can run to a fixed value, regardless of the properties of the node.
참고Setting
podsPerCoreto0disables this limit.In the above example, the default value for
podsPerCoreis10and the default value formaxPodsis250. This means that unless the node has 25 cores or more, by default,podsPerCorewill be the limiting factor.
Run the following command to create the CR:
$ oc create -f <file_name>.yaml
Verification
List the
MachineConfigPoolCRDs to check if the change is applied. TheUPDATINGcolumn reportsTrueif the change is picked up by the Machine Config Controller:$ oc get machineconfigpoolsExample output
NAME CONFIG UPDATED UPDATING DEGRADED master master-9cc2c72f205e103bb534 False False False worker worker-8cecd1236b33ee3f8a5e False True FalseAfter the change is complete, the
UPDATEDcolumn reportsTrue.$ oc get machineconfigpoolsExample output
NAME CONFIG UPDATED UPDATING DEGRADED master master-9cc2c72f205e103bb534 False True False worker worker-8cecd1236b33ee3f8a5e True False False