8.4. 将资源移到基础架构机器集

流程

1. 查看路由器 Operator 的 IngressController 自定义资源：

   $ oc get ingresscontroller default -n openshift-ingress-operator -o yaml

   命令输出类似于以下文本：

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     creationTimestamp: 2019-04-18T12:35:39Z
     finalizers:
     - ingresscontroller.operator.openshift.io/finalizer-ingresscontroller
     generation: 1
     name: default
     namespace: openshift-ingress-operator
     resourceVersion: "11341"
     selfLink: /apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default
     uid: 79509e05-61d6-11e9-bc55-02ce4781844a
   spec: {}
   status:
     availableReplicas: 2
     conditions:
     - lastTransitionTime: 2019-04-18T12:36:15Z
       status: "True"
       type: Available
     domain: apps.<cluster>.example.com
     endpointPublishingStrategy:
       type: LoadBalancerService
     selector: ingresscontroller.operator.openshift.io/deployment-ingresscontroller=default

2. 编辑 ingresscontroller 资源，并更改 nodeSelector 以使用infra 标签：

   $ oc edit ingresscontroller default -n openshift-ingress-operator

     spec:
       nodePlacement:
         nodeSelector: 1
           matchLabels:
             node-role.kubernetes.io/infra: ""
         tolerations:
         - effect: NoSchedule
           key: node-role.kubernetes.io/infra
           value: reserved
         - effect: NoExecute
           key: node-role.kubernetes.io/infra
           value: reserved

   1

   添加 nodeSelector 参数，并设为适用于您想要移动的组件的值。您可以根据为节点指定的值，按所示格式使用 nodeSelector 或使用 <key>: <value> 对。如果您在 infrasructure 节点中添加了污点，还要添加匹配的容限。

3. 确认路由器 Pod 在 infra 节点上运行。

   1. 查看路由器 Pod 列表，并记下正在运行的 Pod 的节点名称：

      $ oc get pod -n openshift-ingress -o wide

      输出示例

      NAME                              READY     STATUS        RESTARTS   AGE       IP           NODE                           NOMINATED NODE   READINESS GATES
      router-default-86798b4b5d-bdlvd   1/1      Running       0          28s       10.130.2.4   ip-10-0-217-226.ec2.internal   <none>           <none>
      router-default-955d875f4-255g8    0/1      Terminating   0          19h       10.129.2.4   ip-10-0-148-172.ec2.internal   <none>           <none>

      在本例中，正在运行的 Pod 位于 ip-10-0-217-226.ec2.internal 节点上。

   2. 查看正在运行的 Pod 的节点状态：

      $ oc get node <node_name> 1

      1

      指定从 Pod 列表获得的 <node_name>。

      输出示例

      NAME                          STATUS  ROLES         AGE   VERSION
      ip-10-0-217-226.ec2.internal  Ready   infra,worker  17h   v1.30.3

      由于角色列表包含 infra，因此 Pod 在正确的节点上运行。

流程

1. 查看 config/instance 对象：

   $ oc get configs.imageregistry.operator.openshift.io/cluster -o yaml

   输出示例

   apiVersion: imageregistry.operator.openshift.io/v1
   kind: Config
   metadata:
     creationTimestamp: 2019-02-05T13:52:05Z
     finalizers:
     - imageregistry.operator.openshift.io/finalizer
     generation: 1
     name: cluster
     resourceVersion: "56174"
     selfLink: /apis/imageregistry.operator.openshift.io/v1/configs/cluster
     uid: 36fd3724-294d-11e9-a524-12ffeee2931b
   spec:
     httpSecret: d9a012ccd117b1e6616ceccb2c3bb66a5fed1b5e481623
     logging: 2
     managementState: Managed
     proxy: {}
     replicas: 1
     requests:
       read: {}
       write: {}
     storage:
       s3:
         bucket: image-registry-us-east-1-c92e88cad85b48ec8b312344dff03c82-392c
         region: us-east-1
   status:
   ...

2. 编辑 config/instance 对象：

   $ oc edit configs.imageregistry.operator.openshift.io/cluster

   spec:
     affinity:
       podAntiAffinity:
         preferredDuringSchedulingIgnoredDuringExecution:
         - podAffinityTerm:
             namespaces:
             - openshift-image-registry
             topologyKey: kubernetes.io/hostname
           weight: 100
     logLevel: Normal
     managementState: Managed
     nodeSelector: 1
       node-role.kubernetes.io/infra: ""
     tolerations:
     - effect: NoSchedule
       key: node-role.kubernetes.io/infra
       value: reserved
     - effect: NoExecute
       key: node-role.kubernetes.io/infra
       value: reserved

   1

   添加 nodeSelector 参数，并设为适用于您想要移动的组件的值。您可以根据为节点指定的值，按所示格式使用 nodeSelector 或使用 <key>: <value> 对。如果您在 infrasructure 节点中添加了污点，还要添加匹配的容限。

3. 验证 registry pod 已移至基础架构节点。

   1. 运行以下命令，以识别 registry pod 所在的节点：

      $ oc get pods -o wide -n openshift-image-registry

   2. 确认节点具有您指定的标签：

      $ oc describe node <node_name>

      查看命令输出，并确认 node-role.kubernetes.io/infra 列在 LABELS 列表中。

流程

1. 编辑 cluster-monitoring-config 配置映射，并更改 nodeSelector 以使用 infra 标签：

   $ oc edit configmap cluster-monitoring-config -n openshift-monitoring

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cluster-monitoring-config
     namespace: openshift-monitoring
   data:
     config.yaml: |+
       alertmanagerMain:
         nodeSelector: 1
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       prometheusK8s:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       prometheusOperator:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       metricsServer:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       kubeStateMetrics:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       telemeterClient:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       openshiftStateMetrics:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       thanosQuerier:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute
       monitoringPlugin:
         nodeSelector:
           node-role.kubernetes.io/infra: ""
         tolerations:
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoSchedule
         - key: node-role.kubernetes.io/infra
           value: reserved
           effect: NoExecute

   1

   添加 nodeSelector 参数，并设为适用于您想要移动的组件的值。您可以根据为节点指定的值，按所示格式使用 nodeSelector 或使用 <key>: <value> 对。如果您在 infrasructure 节点中添加了污点，还要添加匹配的容限。

2. 观察监控 pod 移至新机器：

   $ watch 'oc get pod -n openshift-monitoring -o wide'

3. 如果组件没有移到 infra 节点，请删除带有这个组件的 pod:

   $ oc delete pod -n openshift-monitoring <pod>

   已删除 pod 的组件在 infra 节点上重新创建。

流程

1. 通过将节点选择器添加到 VPA Operator 的 Subscription 自定义资源 (CR)中来移动 VPA Operator pod：

   1. 编辑 CR：

      $ oc edit Subscription vertical-pod-autoscaler -n openshift-vertical-pod-autoscaler

   2. 添加节点选择器以匹配 infra 节点上的节点角色标签：

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        labels:
          operators.coreos.com/vertical-pod-autoscaler.openshift-vertical-pod-autoscaler: ""
        name: vertical-pod-autoscaler
      # ...
      spec:
        config:
          nodeSelector:
            node-role.kubernetes.io/infra: "" 1

      1

      指定 infra 节点的节点角色。

      注意

      如果 infra 节点使用污点，则需要为 Subscription CR 添加容限。

      例如：

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        labels:
          operators.coreos.com/vertical-pod-autoscaler.openshift-vertical-pod-autoscaler: ""
        name: vertical-pod-autoscaler
      # ...
      spec:
        config:
          nodeSelector:
            node-role.kubernetes.io/infra: ""
          tolerations: 1
          - key: "node-role.kubernetes.io/infra"
            operator: "Exists"
            effect: "NoSchedule"

      1

      为 infra 节点上的污点指定容限。

2. 通过将节点选择器添加到 VerticalPodAutoscaler 自定义资源 (CR) 来移动每个 VPA 组件：

   1. 编辑 CR：

      $ oc edit VerticalPodAutoscalerController default -n openshift-vertical-pod-autoscaler

   2. 添加节点选择器以匹配 infra 节点上的节点角色标签：

      apiVersion: autoscaling.openshift.io/v1
      kind: VerticalPodAutoscalerController
      metadata:
       name: default
        namespace: openshift-vertical-pod-autoscaler
      # ...
      spec:
        deploymentOverrides:
          admission:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: "" 1
          recommender:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: "" 2
          updater:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: "" 3

      1

      可选：指定 VPA 准入 pod 的节点角色。

      2

      可选：指定 VPA recommender pod 的节点角色。

      3

      可选：指定 VPA updater pod 的节点角色。

      注意

      如果目标节点使用污点，则需要为 VerticalPodAutoscalerController CR 添加容限。

      例如：

      apiVersion: autoscaling.openshift.io/v1
      kind: VerticalPodAutoscalerController
      metadata:
       name: default
        namespace: openshift-vertical-pod-autoscaler
      # ...
      spec:
        deploymentOverrides:
          admission:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: ""
            tolerations: 1
            - key: "my-example-node-taint-key"
              operator: "Exists"
              effect: "NoSchedule"
          recommender:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: ""
            tolerations: 2
            - key: "my-example-node-taint-key"
              operator: "Exists"
              effect: "NoSchedule"
          updater:
            container:
              resources: {}
            nodeSelector:
              node-role.kubernetes.io/infra: ""
            tolerations: 3
            - key: "my-example-node-taint-key"
              operator: "Exists"
              effect: "NoSchedule"

      1

      为 infra 节点上的污点指定准入控制器 pod 的容限。

      2

      为 infra 节点上的污点指定 recommender pod 的容限。

      3

      为 infra 节点上的污点指定 updater pod 的容限。

流程

1. 通过将节点选择器添加到 Cluster Resource Override Operator 的 Subscription 自定义资源 (CR) 中来移动 Cluster Resource Override Operator pod。

   1. 编辑 CR：

      $ oc edit -n clusterresourceoverride-operator subscriptions.operators.coreos.com clusterresourceoverride

   2. 添加节点选择器以匹配您要安装 Cluster Resource Override Operator pod 的节点上的节点角色标签：

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: clusterresourceoverride
        namespace: clusterresourceoverride-operator
      # ...
      spec:
        config:
          nodeSelector:
            node-role.kubernetes.io/infra: "" 1
      # ...

      1

      指定您要部署 Cluster Resource Override Operator pod 的节点的角色。

      注意

      如果 infra 节点使用污点，则需要为 Subscription CR 添加容限。

      例如：

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: clusterresourceoverride
        namespace: clusterresourceoverride-operator
      # ...
      spec:
        config:
          nodeSelector:
            node-role.kubernetes.io/infra: ""
          tolerations: 1
          - key: "node-role.kubernetes.io/infra"
            operator: "Exists"
            effect: "NoSchedule"

      1

      为 infra 节点上的污点指定容限。

2. 通过将节点选择器添加到 ClusterResourceOverride 自定义资源(CR) 来移动 Cluster Resource Override pod：

   1. 编辑 CR：

      $ oc edit ClusterResourceOverride cluster -n clusterresourceoverride-operator

   2. 添加节点选择器以匹配 infra 节点上的节点角色标签：

      apiVersion: operator.autoscaling.openshift.io/v1
      kind: ClusterResourceOverride
      metadata:
        name: cluster
        resourceVersion: "37952"
      spec:
        podResourceOverride:
          spec:
            cpuRequestToLimitPercent: 25
            limitCPUToMemoryPercent: 200
            memoryRequestToLimitPercent: 50
        deploymentOverrides:
          replicas: 1 1
          nodeSelector:
            node-role.kubernetes.io/infra: "" 2
      # ...

      1

      可选：指定要部署的 Cluster Resource Override pod 的数量。默认值为 2。每个节点只允许一个 pod。

      2

      可选：指定要部署 Cluster Resource Override pod 的节点的角色。

      注意

      如果 infra 节点使用污点，则需要为 ClusterResourceOverride CR 添加容限。

      例如：

      apiVersion: operator.autoscaling.openshift.io/v1
      kind: ClusterResourceOverride
      metadata:
        name: cluster
      # ...
      spec:
        podResourceOverride:
          spec:
            memoryRequestToLimitPercent: 50
            cpuRequestToLimitPercent: 25
            limitCPUToMemoryPercent: 200
        deploymentOverrides:
          replicas: 3
          nodeSelector:
            node-role.kubernetes.io/worker: ""
          tolerations: 1
          - key: "key"
            operator: "Equal"
            value: "value"
            effect: "NoSchedule"

      1

      为 infra 节点上的污点指定容限。