Red Hat Summit Red Hat Summit지원콘솔개발자평가판 시작연락처

12.6. 정상 클러스터에 기본 컨트롤 플레인 노드 설치

다음 절차에서는 정상적인 OpenShift Container Platform 클러스터에 기본 컨트롤 플레인 노드를 설치하는 방법을 설명합니다.

클러스터가 비정상이면 추가 작업을 관리해야 관리할 수 있습니다. 자세한 내용은 추가 리소스 를 참조하십시오.

사전 요구 사항

  • 올바른 etcd-operator 버전으로 OpenShift Container Platform 4.11 이상을 사용하고 있습니다.
  • 최소 3개의 노드가 있는 정상 클러스터가 설치되어 있습니다.
  • 단일 노드에 role: master 가 할당되어 있습니다.

프로세스

  1. CSR 검토 및 승인

    1. CSR( CertificateSigningRequests )을 검토합니다. 

      $ oc get csr | grep Pending
      Copy to Clipboard Toggle word wrap

      출력 예

      csr-5sd59   8m19s   kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   <none>              Pending
csr-xzqts   10s     kubernetes.io/kubelet-serving                 system:node:worker-6                                                   <none>              Pending
      Copy to Clipboard Toggle word wrap

    2. 보류 중인 모든 CSR을 승인합니다. 

      $ oc get csr -o go-template='{{range .items}}{{if not .status}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}' | xargs --no-run-if-empty oc adm certificate approve
      Copy to Clipboard Toggle word wrap
      중요

      설치를 완료하려면 CSR을 승인해야 합니다.

  2. 기본 노드가 Ready 상태에 있는지 확인합니다. 

    $ oc get nodes
    Copy to Clipboard Toggle word wrap

    출력 예

    NAME       STATUS   ROLES    AGE     VERSION
master-0   Ready    master   4h42m   v1.24.0+3882f8f
worker-1   Ready    worker   4h29m   v1.24.0+3882f8f
master-2   Ready    master   4h43m   v1.24.0+3882f8f
master-3   Ready    master   4h27m   v1.24.0+3882f8f
worker-4   Ready    worker   4h30m   v1.24.0+3882f8f
master-5   Ready    master   105s    v1.24.0+3882f8f
    Copy to Clipboard Toggle word wrap

    참고

    etcd-operator 에는 클러스터가 기능적인 Machine API로 실행될 때 새 노드를 참조하는 Machine Custom Resource(CR)가 필요합니다.

  3. Machine CR을 BareMetalHost 및 Node:에 연결합니다.

    1. 고유한 .metadata.name 값을 사용하여 BareMetalHost CR을 생성합니다. 

      apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  name: custom-master3
  namespace: openshift-machine-api
  annotations:
spec:
  automatedCleaningMode: metadata
  bootMACAddress: 00:00:00:00:00:02
  bootMode: UEFI
  customDeploy:
    method: install_coreos
  externallyProvisioned: true
  online: true
  userData:
    name: master-user-data-managed
    namespace: openshift-machine-api
      Copy to Clipboard Toggle word wrap 
      $ oc create -f <filename>
      Copy to Clipboard Toggle word wrap

    2. BareMetalHost CR을 적용합니다. 

      $ oc apply -f <filename>
      Copy to Clipboard Toggle word wrap

    3. 고유한 .machine.name 값을 사용하여 Machine CR을 생성합니다. 

      apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  annotations:
    machine.openshift.io/instance-state: externally provisioned
    metal3.io/BareMetalHost: openshift-machine-api/custom-master3
  finalizers:
  - machine.machine.openshift.io
  generation: 3
  labels:
    machine.openshift.io/cluster-api-cluster: test-day2-1-6qv96
    machine.openshift.io/cluster-api-machine-role: master
    machine.openshift.io/cluster-api-machine-type: master
  name: custom-master3
  namespace: openshift-machine-api
spec:
  metadata: {}
  providerSpec:
    value:
      apiVersion: baremetal.cluster.k8s.io/v1alpha1
      customDeploy:
        method: install_coreos
      hostSelector: {}
      image:
        checksum: ""
        url: ""
      kind: BareMetalMachineProviderSpec
      metadata:
        creationTimestamp: null
      userData:
        name: master-user-data-managed
      Copy to Clipboard Toggle word wrap 
      $ oc create -f <filename>
      Copy to Clipboard Toggle word wrap

    4. Machine CR을 적용합니다. 

      $ oc apply -f <filename>
      Copy to Clipboard Toggle word wrap

    5. link-machine-and-node.sh 스크립트를 사용하여 BareMetalHost,Machine, Node 를 연결합니다. 

      #!/bin/bash

# Credit goes to https://bugzilla.redhat.com/show_bug.cgi?id=1801238.
# This script will link Machine object and Node object. This is needed
# in order to have IP address of the Node present in the status of the Machine.

set -x
set -e

machine="$1"
node="$2"

if [ -z "$machine" -o -z "$node" ]; then
    echo "Usage: $0 MACHINE NODE"
    exit 1
fi

uid=$(echo $node | cut -f1 -d':')
node_name=$(echo $node | cut -f2 -d':')

oc proxy &
proxy_pid=$!
function kill_proxy {
    kill $proxy_pid
}
trap kill_proxy EXIT SIGINT

HOST_PROXY_API_PATH="http://localhost:8001/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts"

function wait_for_json() {
    local name
    local url
    local curl_opts
    local timeout

    local start_time
    local curr_time
    local time_diff

    name="$1"
    url="$2"
    timeout="$3"
    shift 3
    curl_opts="$@"
    echo -n "Waiting for $name to respond"
    start_time=$(date +%s)
    until curl -g -X GET "$url" "${curl_opts[@]}" 2> /dev/null | jq '.' 2> /dev/null > /dev/null; do
        echo -n "."
        curr_time=$(date +%s)
        time_diff=$(($curr_time - $start_time))
        if [[ $time_diff -gt $timeout ]]; then
            echo "\nTimed out waiting for $name"
            return 1
        fi
        sleep 5
    done
    echo " Success!"
    return 0
}
wait_for_json oc_proxy "${HOST_PROXY_API_PATH}" 10 -H "Accept: application/json" -H "Content-Type: application/json"

addresses=$(oc get node -n openshift-machine-api ${node_name} -o json | jq -c '.status.addresses')

machine_data=$(oc get machine -n openshift-machine-api -o json ${machine})
host=$(echo "$machine_data" | jq '.metadata.annotations["metal3.io/BareMetalHost"]' | cut -f2 -d/ | sed 's/"//g')

if [ -z "$host" ]; then
    echo "Machine $machine is not linked to a host yet." 1>&2
    exit 1
fi

# The address structure on the host doesn't match the node, so extract
# the values we want into separate variables so we can build the patch
# we need.
hostname=$(echo "${addresses}" | jq '.[] | select(. | .type == "Hostname") | .address' | sed 's/"//g')
ipaddr=$(echo "${addresses}" | jq '.[] | select(. | .type == "InternalIP") | .address' | sed 's/"//g')

host_patch='
{
  "status": {
    "hardware": {
      "hostname": "'${hostname}'",
      "nics": [
        {
          "ip": "'${ipaddr}'",
          "mac": "00:00:00:00:00:00",
          "model": "unknown",
          "speedGbps": 10,
          "vlanId": 0,
          "pxe": true,
          "name": "eth1"
        }
      ],
      "systemVendor": {
        "manufacturer": "Red Hat",
        "productName": "product name",
        "serialNumber": ""
      },
      "firmware": {
        "bios": {
          "date": "04/01/2014",
          "vendor": "SeaBIOS",
          "version": "1.11.0-2.el7"
        }
      },
      "ramMebibytes": 0,
      "storage": [],
      "cpu": {
        "arch": "x86_64",
        "model": "Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz",
        "clockMegahertz": 2199.998,
        "count": 4,
        "flags": []
      }
    }
  }
}
'

echo "PATCHING HOST"
echo "${host_patch}" | jq .

curl -s \
     -X PATCH \
     ${HOST_PROXY_API_PATH}/${host}/status \
     -H "Content-type: application/merge-patch+json" \
     -d "${host_patch}"

oc get baremetalhost -n openshift-machine-api -o yaml "${host}"
      Copy to Clipboard Toggle word wrap 
      $ bash link-machine-and-node.sh custom-master3 worker-5
      Copy to Clipboard Toggle word wrap

  4. etcd 멤버를 확인합니다. 

    $ oc rsh -n openshift-etcd etcd-worker-2
etcdctl member list -w table
    Copy to Clipboard Toggle word wrap

    출력 예

    +--------+---------+--------+--------------+--------------+---------+
|   ID   |  STATUS |  NAME  |  PEER ADDRS  | CLIENT ADDRS | LEARNER |
+--------+---------+--------+--------------+--------------+---------+
|2c18942f| started |worker-3|192.168.111.26|192.168.111.26|  false  |
|61e2a860| started |worker-2|192.168.111.25|192.168.111.25|  false  |
|ead4f280| started |worker-5|192.168.111.28|192.168.111.28|  false  |
+--------+---------+--------+--------------+--------------+---------+
    Copy to Clipboard Toggle word wrap

  5. etcd-operator 구성이 모든 노드에 적용되는지 확인합니다. 

    $ oc get clusteroperator etcd
    Copy to Clipboard Toggle word wrap

    출력 예

    NAME   VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
etcd   4.11.5    True        False         False      5h54m
    Copy to Clipboard Toggle word wrap

  6. etcd-operator 상태를 확인합니다. 

    $ oc rsh -n openshift-etcd etcd-worker-0
etcdctl endpoint health
    Copy to Clipboard Toggle word wrap

    출력 예

    192.168.111.26 is healthy: committed proposal: took = 11.297561ms
192.168.111.25 is healthy: committed proposal: took = 13.892416ms
192.168.111.28 is healthy: committed proposal: took = 11.870755ms
    Copy to Clipboard Toggle word wrap

  7. 노드 상태를 확인합니다. 

    $ oc get Nodes
    Copy to Clipboard Toggle word wrap

    출력 예

    NAME       STATUS   ROLES    AGE     VERSION
master-0   Ready    master   6h20m   v1.24.0+3882f8f
worker-1   Ready    worker   6h7m    v1.24.0+3882f8f
master-2   Ready    master   6h20m   v1.24.0+3882f8f
master-3   Ready    master   6h4m    v1.24.0+3882f8f
worker-4   Ready    worker   6h7m    v1.24.0+3882f8f
master-5   Ready    master   99m     v1.24.0+3882f8f
    Copy to Clipboard Toggle word wrap

  8. ClusterOperators 상태를 확인합니다. 

    $ oc get ClusterOperators
    Copy to Clipboard Toggle word wrap

    출력 예

    NAME                                      VERSION AVAILABLE PROGRESSING DEGRADED SINCE MSG
authentication                            4.11.5  True      False       False    5h57m
baremetal                                 4.11.5  True      False       False    6h19m
cloud-controller-manager                  4.11.5  True      False       False    6h20m
cloud-credential                          4.11.5  True      False       False    6h23m
cluster-autoscaler                        4.11.5  True      False       False    6h18m
config-operator                           4.11.5  True      False       False    6h19m
console                                   4.11.5  True      False       False    6h4m
csi-snapshot-controller                   4.11.5  True      False       False    6h19m
dns                                       4.11.5  True      False       False    6h18m
etcd                                      4.11.5  True      False       False    6h17m
image-registry                            4.11.5  True      False       False    6h7m
ingress                                   4.11.5  True      False       False    6h6m
insights                                  4.11.5  True      False       False    6h12m
kube-apiserver                            4.11.5  True      False       False    6h16m
kube-controller-manager                   4.11.5  True      False       False    6h16m
kube-scheduler                            4.11.5  True      False       False    6h16m
kube-storage-version-migrator             4.11.5  True      False       False    6h19m
machine-api                               4.11.5  True      False       False    6h15m
machine-approver                          4.11.5  True      False       False    6h19m
machine-config                            4.11.5  True      False       False    6h18m
marketplace                               4.11.5  True      False       False    6h18m
monitoring                                4.11.5  True      False       False    6h4m
network                                   4.11.5  True      False       False    6h20m
node-tuning                               4.11.5  True      False       False    6h18m
openshift-apiserver                       4.11.5  True      False       False    6h8m
openshift-controller-manager              4.11.5  True      False       False    6h7m
openshift-samples                         4.11.5  True      False       False    6h12m
operator-lifecycle-manager                4.11.5  True      False       False    6h18m
operator-lifecycle-manager-catalog        4.11.5  True      False       False    6h19m
operator-lifecycle-manager-pkgsvr         4.11.5  True      False       False    6h12m
service-ca                                4.11.5  True      False       False    6h19m
storage                                   4.11.5  True      False       False    6h19m
    Copy to Clipboard Toggle word wrap

  9. ClusterVersion 확인 : 

    $ oc get ClusterVersion
    Copy to Clipboard Toggle word wrap

    출력 예

    NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.5    True        False         5h57m   Cluster version is 4.11.5
    Copy to Clipboard Toggle word wrap

  10. 이전 컨트롤 플레인 노드를 제거합니다.

    1. BareMetalHost CR을 삭제합니다. 

      $ oc delete bmh -n openshift-machine-api custom-master3
      Copy to Clipboard Toggle word wrap

    2. Machine 이 비정상인지 확인합니다. 

      $ oc get machine -A
      Copy to Clipboard Toggle word wrap

      출력 예

      NAMESPACE              NAME                              PHASE    AGE
openshift-machine-api  custom-master3                    Running  14h
openshift-machine-api  test-day2-1-6qv96-master-0        Failed   20h
openshift-machine-api  test-day2-1-6qv96-master-1        Running  20h
openshift-machine-api  test-day2-1-6qv96-master-2        Running  20h
openshift-machine-api  test-day2-1-6qv96-worker-0-8w7vr  Running  19h
openshift-machine-api  test-day2-1-6qv96-worker-0-rxddj  Running  19h
      Copy to Clipboard Toggle word wrap

    3. Machine CR을 삭제합니다. 

      $ oc delete machine -n openshift-machine-api   test-day2-1-6qv96-master-0
machine.machine.openshift.io "test-day2-1-6qv96-master-0" deleted
      Copy to Clipboard Toggle word wrap

    4. Node CR 제거를 확인합니다. 

      $ oc get nodes
      Copy to Clipboard Toggle word wrap

      출력 예

      NAME       STATUS   ROLES    AGE   VERSION
worker-1   Ready    worker   19h   v1.24.0+3882f8f
master-2   Ready    master   20h   v1.24.0+3882f8f
master-3   Ready    master   19h   v1.24.0+3882f8f
worker-4   Ready    worker   19h   v1.24.0+3882f8f
master-5   Ready    master   15h   v1.24.0+3882f8f
      Copy to Clipboard Toggle word wrap

  11. etcd-operator 로그를 확인하여 etcd 클러스터의 상태를 확인합니다. 

    $ oc logs -n openshift-etcd-operator etcd-operator-8668df65d-lvpjf
    Copy to Clipboard Toggle word wrap

    출력 예

    E0927 07:53:10.597523       1 base_controller.go:272] ClusterMemberRemovalController reconciliation failed: cannot remove member: 192.168.111.23 because it is reported as healthy but it doesn't have a machine nor a node resource
    Copy to Clipboard Toggle word wrap

  12. etcd-operator 가 클러스터 멤버를 조정할 수 있도록 물리적 머신을 제거합니다. 

    $ oc rsh -n openshift-etcd etcd-worker-2
etcdctl member list -w table; etcdctl endpoint health
    Copy to Clipboard Toggle word wrap

    출력 예

    +--------+---------+--------+--------------+--------------+---------+
|   ID   |  STATUS |  NAME  |  PEER ADDRS  | CLIENT ADDRS | LEARNER |
+--------+---------+--------+--------------+--------------+---------+
|2c18942f| started |worker-3|192.168.111.26|192.168.111.26|  false  |
|61e2a860| started |worker-2|192.168.111.25|192.168.111.25|  false  |
|ead4f280| started |worker-5|192.168.111.28|192.168.111.28|  false  |
+--------+---------+--------+--------------+--------------+---------+
192.168.111.26 is healthy: committed proposal: took = 10.458132ms
192.168.111.25 is healthy: committed proposal: took = 11.047349ms
192.168.111.28 is healthy: committed proposal: took = 11.414402ms
    Copy to Clipboard Toggle word wrap

맨 위로 이동
Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2025 Red Hat