13.2. 设置 CPU Manager

另外，还可为节点添加标签：

oc label node perf-node.example.com cpumanager=true

# oc label node perf-node.example.com cpumanager=true

Copy to Clipboard

Toggle word wrap

在目标节点上启用 CPU Manager 支持：

oc edit configmap <name> -n openshift-node

# oc edit configmap <name> -n openshift-node

Copy to Clipboard

Toggle word wrap

例如：

oc edit cm node-config-compute -n openshift-node

# oc edit cm node-config-compute -n openshift-node

Copy to Clipboard

Toggle word wrap

输出示例

...
kubeletArguments:
...
  feature-gates:
  - CPUManager=true
  cpu-manager-policy:
  - static
  cpu-manager-reconcile-period:
  - 5s
  system-reserved: 
  - cpu=500m

...
kubeletArguments:
...
  feature-gates:
  - CPUManager=true
  cpu-manager-policy:
  - static
  cpu-manager-reconcile-period:
  - 5s
  system-reserved:

1


  - cpu=500m

Copy to Clipboard

Toggle word wrap

systemctl restart atomic-openshift-node

# systemctl restart atomic-openshift-node

Copy to Clipboard

Toggle word wrap

1: system-reserved 是一个必需的设置。可能需要根据您的环境调整值。

创建请求一个或多个内核的 pod。限制和请求都必须将其 CPU 值设置为一个整数。这是专用于此 pod 的内核数：

cat cpumanager.yaml

# cat cpumanager.yaml

Copy to Clipboard

Toggle word wrap

输出示例

apiVersion: v1
kind: Pod
metadata:
  generateName: cpumanager-
spec:
  containers:
  - name: cpumanager
    image: gcr.io/google_containers/pause-amd64:3.0
    resources:
      requests:
        cpu: 1
        memory: "1G"
      limits:
        cpu: 1
        memory: "1G"
  nodeSelector:
    cpumanager: "true"

apiVersion: v1
kind: Pod
metadata:
  generateName: cpumanager-
spec:
  containers:
  - name: cpumanager
    image: gcr.io/google_containers/pause-amd64:3.0
    resources:
      requests:
        cpu: 1
        memory: "1G"
      limits:
        cpu: 1
        memory: "1G"
  nodeSelector:
    cpumanager: "true"

Copy to Clipboard

Toggle word wrap

创建 pod：

oc create -f cpumanager.yaml

# oc create -f cpumanager.yaml

Copy to Clipboard

Toggle word wrap

确定为您标记的节点调度了 pod：

oc describe pod cpumanager

# oc describe pod cpumanager

Copy to Clipboard

Toggle word wrap

输出示例

Name:         cpumanager-4gdtn
Namespace:    test
Node:         perf-node.example.com/172.31.62.105
...
    Limits:
      cpu:     1
      memory:  1G
    Requests:
      cpu:        1
      memory:     1G
...
QoS Class:       Guaranteed
Node-Selectors:  cpumanager=true
                 region=primary

Name:         cpumanager-4gdtn
Namespace:    test
Node:         perf-node.example.com/172.31.62.105
...
    Limits:
      cpu:     1
      memory:  1G
    Requests:
      cpu:        1
      memory:     1G
...
QoS Class:       Guaranteed
Node-Selectors:  cpumanager=true
                 region=primary

Copy to Clipboard

Toggle word wrap

确认正确配置了 cgroups。获取暂停进程的 PID：

systemd-cgls -l

# systemd-cgls -l

Copy to Clipboard

Toggle word wrap

输出示例

├─1 /usr/lib/systemd/systemd --system --deserialize 20
├─kubepods.slice
│ ├─kubepods-pod0ec1ab8b_e1c4_11e7_bb22_027b30990a24.slice
│ │ ├─docker-b24e29bc4021064057f941dc5f3538595c317d294f2c8e448b5e61a29c026d1c.scope
│ │ │ └─44216 /pause

├─1 /usr/lib/systemd/systemd --system --deserialize 20
├─kubepods.slice
│ ├─kubepods-pod0ec1ab8b_e1c4_11e7_bb22_027b30990a24.slice
│ │ ├─docker-b24e29bc4021064057f941dc5f3538595c317d294f2c8e448b5e61a29c026d1c.scope
│ │ │ └─44216 /pause

Copy to Clipboard

Toggle word wrap

QoS 等级为 Guaranteed 的 pod 放置在 kubepods.slice 中。其它 QoS 等级的 Pod 会位于 kubepods 的子 cgroups 中。

cd /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-pod0ec1ab8b_e1c4_11e7_bb22_027b30990a24.slice/docker-b24e29bc4021064057f941dc5f3538595c317d294f2c8e448b5e61a29c026d1c.scope
for i in `ls cpuset.cpus tasks` ; do echo -n "$i "; cat $i ; done

# cd /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-pod0ec1ab8b_e1c4_11e7_bb22_027b30990a24.slice/docker-b24e29bc4021064057f941dc5f3538595c317d294f2c8e448b5e61a29c026d1c.scope
# for i in `ls cpuset.cpus tasks` ; do echo -n "$i "; cat $i ; done

Copy to Clipboard

Toggle word wrap

输出示例

cpuset.cpus 2
tasks 44216

cpuset.cpus 2
tasks 44216

Copy to Clipboard

Toggle word wrap

检查任务允许的 CPU 列表：

grep ^Cpus_allowed_list /proc/44216/status

# grep ^Cpus_allowed_list /proc/44216/status

Copy to Clipboard

Toggle word wrap

输出示例

Cpus_allowed_list:      2

Cpus_allowed_list:      2

Copy to Clipboard

Toggle word wrap

验证系统中的另一个 pod（在这个示例中，burstable QoS 等级为 burstable 的 pod）无法在为 Guaranteed pod 分配的内核中运行：

cat /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe76ff22_dead_11e7_b99e_027b30990a24.slice/docker-da621bea7569704fc39f84385a179923309ab9d832f6360cccbff102e73f9557.scope/cpuset.cpus
0-1,3

# cat /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe76ff22_dead_11e7_b99e_027b30990a24.slice/docker-da621bea7569704fc39f84385a179923309ab9d832f6360cccbff102e73f9557.scope/cpuset.cpus
0-1,3

Copy to Clipboard

Toggle word wrap

oc describe node perf-node.example.com

# oc describe node perf-node.example.com

Copy to Clipboard

Toggle word wrap

输出示例

...
Capacity:
 cpu:     4
 memory:  16266720Ki
 pods:    40
Allocatable:
 cpu:     3500m
 memory:  16164320Ki
 pods:    40
---
  Namespace                  Name                      CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----                      ------------  ----------  ---------------  -------------
  test                        cpumanager-4gdtn          1 (28%)       1 (28%)     1G (6%)          1G (6%)
  test                        cpumanager-hczts          1 (28%)       1 (28%)     1G (6%)          1G (6%)
  test                        cpumanager-r9wrq          1 (28%)       1 (28%)     1G (6%)          1G (6%)
...
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ------------  ----------  ---------------  -------------
  3 (85%)       3 (85%)     5437500k (32%)   9250M (55%)

...
Capacity:
 cpu:     4
 memory:  16266720Ki
 pods:    40
Allocatable:
 cpu:     3500m
 memory:  16164320Ki
 pods:    40
---
  Namespace                  Name                      CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----                      ------------  ----------  ---------------  -------------
  test                        cpumanager-4gdtn          1 (28%)       1 (28%)     1G (6%)          1G (6%)
  test                        cpumanager-hczts          1 (28%)       1 (28%)     1G (6%)          1G (6%)
  test                        cpumanager-r9wrq          1 (28%)       1 (28%)     1G (6%)          1G (6%)
...
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ------------  ----------  ---------------  -------------
  3 (85%)       3 (85%)     5437500k (32%)   9250M (55%)

Copy to Clipboard

Toggle word wrap

此虚拟机有四个 CPU 内核。将 system-reserved 设置为 500 毫秒，即从节点的总容量中减去一个内核的一半，以达到 Node Allocatable 的数量。

您可以看到，Allocatable CPU 为 3500 millicore。这意味着，我们可以运行三个 CPU Manager pod，因为每个 pod 都需要一个完整的内核。一个完整的内核等于 1000 毫秒。

如果您尝试调度第四个 pod，系统将接受该 pod，但不会调度它：

oc get pods --all-namespaces |grep test

# oc get pods --all-namespaces |grep test

Copy to Clipboard

Toggle word wrap

输出示例

test              cpumanager-4gdtn               1/1       Running            0          8m
test              cpumanager-hczts               1/1       Running            0          8m
test              cpumanager-nb9d5               0/1       Pending            0          8m
test              cpumanager-r9wrq               1/1       Running            0          8m

test              cpumanager-4gdtn               1/1       Running            0          8m
test              cpumanager-hczts               1/1       Running            0          8m
test              cpumanager-nb9d5               0/1       Pending            0          8m
test              cpumanager-r9wrq               1/1       Running            0          8m

Copy to Clipboard

Toggle word wrap

学习

尝试、购买和销售

社区

关于红帽文档

让开源更具包容性

關於紅帽

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links