검색

12.5. 정상 클러스터에 기본 컨트롤 플레인 노드 설치

download PDF

다음 절차에서는 정상적인 OpenShift Container Platform 클러스터에 기본 컨트롤 플레인 노드를 설치하는 방법을 설명합니다.

클러스터가 비정상이면 추가 작업을 관리해야 관리할 수 있습니다. 자세한 내용은 비정상 클러스터에 기본 컨트롤 플레인 노드 설치를 참조하십시오.

사전 요구 사항

  • 올바른 etcd-operator 버전으로 OpenShift Container Platform 4.11 이상을 사용하고 있습니다.
  • 최소 3개의 노드가 있는 정상 클러스터가 설치되어 있습니다.
  • 단일 노드에 role: master할당되어 있습니다.

절차

  1. CSR 검토 및 승인

    1. CSR( CertificateSigningRequests )을 검토합니다.

      $ oc get csr | grep Pending

      출력 예

      csr-5sd59   8m19s   kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   <none>              Pending
      csr-xzqts   10s     kubernetes.io/kubelet-serving                 system:node:worker-6                                                   <none>              Pending

    2. 보류 중인 모든 CSR을 승인합니다.

      $ oc get csr -o go-template='{{range .items}}{{if not .status}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}' | xargs --no-run-if-empty oc adm certificate approve
      중요

      설치를 완료하려면 CSR을 승인해야 합니다.

  2. 기본 노드가 Ready 상태에 있는지 확인합니다.

    $ oc get nodes

    출력 예

    NAME       STATUS   ROLES    AGE     VERSION
    master-0   Ready    master   4h42m   v1.24.0+3882f8f
    worker-1   Ready    worker   4h29m   v1.24.0+3882f8f
    master-2   Ready    master   4h43m   v1.24.0+3882f8f
    master-3   Ready    master   4h27m   v1.24.0+3882f8f
    worker-4   Ready    worker   4h30m   v1.24.0+3882f8f
    master-5   Ready    master   105s    v1.24.0+3882f8f

    참고

    etcd-operator 에는 클러스터가 기능적인 Machine API로 실행될 때 새 노드를 참조하는 Machine Custom Resource(CR)가 필요합니다.

  3. Machine CR을 BareMetalHostNode:에 연결합니다.

    1. 고유한 .metadata.name 값을 사용하여 BareMetalHost CR을 생성합니다.

      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: custom-master3
        namespace: openshift-machine-api
        annotations:
      spec:
        automatedCleaningMode: metadata
        bootMACAddress: 00:00:00:00:00:02
        bootMode: UEFI
        customDeploy:
          method: install_coreos
        externallyProvisioned: true
        online: true
        userData:
          name: master-user-data-managed
          namespace: openshift-machine-api
      $ oc create -f <filename>
    2. BareMetalHost CR을 적용합니다.

      $ oc apply -f <filename>
    3. 고유한 .machine.name 값을 사용하여 Machine CR을 생성합니다.

      apiVersion: machine.openshift.io/v1beta1
      kind: Machine
      metadata:
        annotations:
          machine.openshift.io/instance-state: externally provisioned
          metal3.io/BareMetalHost: openshift-machine-api/custom-master3
        finalizers:
        - machine.machine.openshift.io
        generation: 3
        labels:
          machine.openshift.io/cluster-api-cluster: test-day2-1-6qv96
          machine.openshift.io/cluster-api-machine-role: master
          machine.openshift.io/cluster-api-machine-type: master
        name: custom-master3
        namespace: openshift-machine-api
      spec:
        metadata: {}
        providerSpec:
          value:
            apiVersion: baremetal.cluster.k8s.io/v1alpha1
            customDeploy:
              method: install_coreos
            hostSelector: {}
            image:
              checksum: ""
              url: ""
            kind: BareMetalMachineProviderSpec
            metadata:
              creationTimestamp: null
            userData:
              name: master-user-data-managed
      $ oc create -f <filename>
    4. Machine CR을 적용합니다.

      $ oc apply -f <filename>
    5. link-machine-and-node.sh 스크립트를 사용하여 BareMetalHost,Machine, Node 를 연결합니다.

      #!/bin/bash
      
      # Credit goes to https://bugzilla.redhat.com/show_bug.cgi?id=1801238.
      # This script will link Machine object and Node object. This is needed
      # in order to have IP address of the Node present in the status of the Machine.
      
      set -x
      set -e
      
      machine="$1"
      node="$2"
      
      if [ -z "$machine" -o -z "$node" ]; then
          echo "Usage: $0 MACHINE NODE"
          exit 1
      fi
      
      uid=$(echo $node | cut -f1 -d':')
      node_name=$(echo $node | cut -f2 -d':')
      
      oc proxy &
      proxy_pid=$!
      function kill_proxy {
          kill $proxy_pid
      }
      trap kill_proxy EXIT SIGINT
      
      HOST_PROXY_API_PATH="http://localhost:8001/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts"
      
      function wait_for_json() {
          local name
          local url
          local curl_opts
          local timeout
      
          local start_time
          local curr_time
          local time_diff
      
          name="$1"
          url="$2"
          timeout="$3"
          shift 3
          curl_opts="$@"
          echo -n "Waiting for $name to respond"
          start_time=$(date +%s)
          until curl -g -X GET "$url" "${curl_opts[@]}" 2> /dev/null | jq '.' 2> /dev/null > /dev/null; do
              echo -n "."
              curr_time=$(date +%s)
              time_diff=$(($curr_time - $start_time))
              if [[ $time_diff -gt $timeout ]]; then
                  echo "\nTimed out waiting for $name"
                  return 1
              fi
              sleep 5
          done
          echo " Success!"
          return 0
      }
      wait_for_json oc_proxy "${HOST_PROXY_API_PATH}" 10 -H "Accept: application/json" -H "Content-Type: application/json"
      
      addresses=$(oc get node -n openshift-machine-api ${node_name} -o json | jq -c '.status.addresses')
      
      machine_data=$(oc get machine -n openshift-machine-api -o json ${machine})
      host=$(echo "$machine_data" | jq '.metadata.annotations["metal3.io/BareMetalHost"]' | cut -f2 -d/ | sed 's/"//g')
      
      if [ -z "$host" ]; then
          echo "Machine $machine is not linked to a host yet." 1>&2
          exit 1
      fi
      
      # The address structure on the host doesn't match the node, so extract
      # the values we want into separate variables so we can build the patch
      # we need.
      hostname=$(echo "${addresses}" | jq '.[] | select(. | .type == "Hostname") | .address' | sed 's/"//g')
      ipaddr=$(echo "${addresses}" | jq '.[] | select(. | .type == "InternalIP") | .address' | sed 's/"//g')
      
      host_patch='
      {
        "status": {
          "hardware": {
            "hostname": "'${hostname}'",
            "nics": [
              {
                "ip": "'${ipaddr}'",
                "mac": "00:00:00:00:00:00",
                "model": "unknown",
                "speedGbps": 10,
                "vlanId": 0,
                "pxe": true,
                "name": "eth1"
              }
            ],
            "systemVendor": {
              "manufacturer": "Red Hat",
              "productName": "product name",
              "serialNumber": ""
            },
            "firmware": {
              "bios": {
                "date": "04/01/2014",
                "vendor": "SeaBIOS",
                "version": "1.11.0-2.el7"
              }
            },
            "ramMebibytes": 0,
            "storage": [],
            "cpu": {
              "arch": "x86_64",
              "model": "Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz",
              "clockMegahertz": 2199.998,
              "count": 4,
              "flags": []
            }
          }
        }
      }
      '
      
      echo "PATCHING HOST"
      echo "${host_patch}" | jq .
      
      curl -s \
           -X PATCH \
           ${HOST_PROXY_API_PATH}/${host}/status \
           -H "Content-type: application/merge-patch+json" \
           -d "${host_patch}"
      
      oc get baremetalhost -n openshift-machine-api -o yaml "${host}"
      $ bash link-machine-and-node.sh custom-master3 worker-5
  4. etcd 멤버를 확인합니다.

    $ oc rsh -n openshift-etcd etcd-worker-2
    etcdctl member list -w table

    출력 예

    +--------+---------+--------+--------------+--------------+---------+
    |   ID   |  STATUS |  NAME  |  PEER ADDRS  | CLIENT ADDRS | LEARNER |
    +--------+---------+--------+--------------+--------------+---------+
    |2c18942f| started |worker-3|192.168.111.26|192.168.111.26|  false  |
    |61e2a860| started |worker-2|192.168.111.25|192.168.111.25|  false  |
    |ead4f280| started |worker-5|192.168.111.28|192.168.111.28|  false  |
    +--------+---------+--------+--------------+--------------+---------+

  5. etcd-operator 구성이 모든 노드에 적용되는지 확인합니다.

    $ oc get clusteroperator etcd

    출력 예

    NAME   VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    etcd   4.11.5    True        False         False      5h54m

  6. etcd-operator 상태를 확인합니다.

    $ oc rsh -n openshift-etcd etcd-worker-0
    etcdctl endpoint health

    출력 예

    192.168.111.26 is healthy: committed proposal: took = 11.297561ms
    192.168.111.25 is healthy: committed proposal: took = 13.892416ms
    192.168.111.28 is healthy: committed proposal: took = 11.870755ms

  7. 노드 상태를 확인합니다.

    $ oc get Nodes

    출력 예

    NAME       STATUS   ROLES    AGE     VERSION
    master-0   Ready    master   6h20m   v1.24.0+3882f8f
    worker-1   Ready    worker   6h7m    v1.24.0+3882f8f
    master-2   Ready    master   6h20m   v1.24.0+3882f8f
    master-3   Ready    master   6h4m    v1.24.0+3882f8f
    worker-4   Ready    worker   6h7m    v1.24.0+3882f8f
    master-5   Ready    master   99m     v1.24.0+3882f8f

  8. ClusterOperators 상태를 확인합니다.

    $ oc get ClusterOperators

    출력 예

    NAME                                      VERSION AVAILABLE PROGRESSING DEGRADED SINCE MSG
    authentication                            4.11.5  True      False       False    5h57m
    baremetal                                 4.11.5  True      False       False    6h19m
    cloud-controller-manager                  4.11.5  True      False       False    6h20m
    cloud-credential                          4.11.5  True      False       False    6h23m
    cluster-autoscaler                        4.11.5  True      False       False    6h18m
    config-operator                           4.11.5  True      False       False    6h19m
    console                                   4.11.5  True      False       False    6h4m
    csi-snapshot-controller                   4.11.5  True      False       False    6h19m
    dns                                       4.11.5  True      False       False    6h18m
    etcd                                      4.11.5  True      False       False    6h17m
    image-registry                            4.11.5  True      False       False    6h7m
    ingress                                   4.11.5  True      False       False    6h6m
    insights                                  4.11.5  True      False       False    6h12m
    kube-apiserver                            4.11.5  True      False       False    6h16m
    kube-controller-manager                   4.11.5  True      False       False    6h16m
    kube-scheduler                            4.11.5  True      False       False    6h16m
    kube-storage-version-migrator             4.11.5  True      False       False    6h19m
    machine-api                               4.11.5  True      False       False    6h15m
    machine-approver                          4.11.5  True      False       False    6h19m
    machine-config                            4.11.5  True      False       False    6h18m
    marketplace                               4.11.5  True      False       False    6h18m
    monitoring                                4.11.5  True      False       False    6h4m
    network                                   4.11.5  True      False       False    6h20m
    node-tuning                               4.11.5  True      False       False    6h18m
    openshift-apiserver                       4.11.5  True      False       False    6h8m
    openshift-controller-manager              4.11.5  True      False       False    6h7m
    openshift-samples                         4.11.5  True      False       False    6h12m
    operator-lifecycle-manager                4.11.5  True      False       False    6h18m
    operator-lifecycle-manager-catalog        4.11.5  True      False       False    6h19m
    operator-lifecycle-manager-pkgsvr         4.11.5  True      False       False    6h12m
    service-ca                                4.11.5  True      False       False    6h19m
    storage                                   4.11.5  True      False       False    6h19m

  9. ClusterVersion 확인 :

    $ oc get ClusterVersion

    출력 예

    NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
    version   4.11.5    True        False         5h57m   Cluster version is 4.11.5

  10. 이전 컨트롤 플레인 노드를 제거합니다.

    1. BareMetalHost CR을 삭제합니다.

      $ oc delete bmh -n openshift-machine-api custom-master3
    2. Machine 이 비정상인지 확인합니다.

      $ oc get machine -A

      출력 예

      NAMESPACE              NAME                              PHASE    AGE
      openshift-machine-api  custom-master3                    Running  14h
      openshift-machine-api  test-day2-1-6qv96-master-0        Failed   20h
      openshift-machine-api  test-day2-1-6qv96-master-1        Running  20h
      openshift-machine-api  test-day2-1-6qv96-master-2        Running  20h
      openshift-machine-api  test-day2-1-6qv96-worker-0-8w7vr  Running  19h
      openshift-machine-api  test-day2-1-6qv96-worker-0-rxddj  Running  19h

    3. Machine CR을 삭제합니다.

      $ oc delete machine -n openshift-machine-api   test-day2-1-6qv96-master-0
      machine.machine.openshift.io "test-day2-1-6qv96-master-0" deleted
    4. Node CR 제거를 확인합니다.

      $ oc get nodes

      출력 예

      NAME       STATUS   ROLES    AGE   VERSION
      worker-1   Ready    worker   19h   v1.24.0+3882f8f
      master-2   Ready    master   20h   v1.24.0+3882f8f
      master-3   Ready    master   19h   v1.24.0+3882f8f
      worker-4   Ready    worker   19h   v1.24.0+3882f8f
      master-5   Ready    master   15h   v1.24.0+3882f8f

  11. etcd-operator 로그를 확인하여 etcd 클러스터의 상태를 확인합니다.

    $ oc logs -n openshift-etcd-operator etcd-operator-8668df65d-lvpjf

    출력 예

    E0927 07:53:10.597523       1 base_controller.go:272] ClusterMemberRemovalController reconciliation failed: cannot remove member: 192.168.111.23 because it is reported as healthy but it doesn't have a machine nor a node resource

  12. etcd-operator 가 클러스터 멤버를 조정할 수 있도록 물리적 머신을 제거합니다.

    $ oc rsh -n openshift-etcd etcd-worker-2
    etcdctl member list -w table; etcdctl endpoint health

    출력 예

    +--------+---------+--------+--------------+--------------+---------+
    |   ID   |  STATUS |  NAME  |  PEER ADDRS  | CLIENT ADDRS | LEARNER |
    +--------+---------+--------+--------------+--------------+---------+
    |2c18942f| started |worker-3|192.168.111.26|192.168.111.26|  false  |
    |61e2a860| started |worker-2|192.168.111.25|192.168.111.25|  false  |
    |ead4f280| started |worker-5|192.168.111.28|192.168.111.28|  false  |
    +--------+---------+--------+--------------+--------------+---------+
    192.168.111.26 is healthy: committed proposal: took = 10.458132ms
    192.168.111.25 is healthy: committed proposal: took = 11.047349ms
    192.168.111.28 is healthy: committed proposal: took = 11.414402ms

Red Hat logoGithubRedditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

© 2024 Red Hat, Inc.