5.2.


참고

5.2.1.

5.2.2.

  1. $ oc get etcd -o=jsonpath='{range .items[0].status.conditions[?(@.type=="EtcdMembersAvailable")]}{.message}{"\n"}'
    Copy to Clipboard Toggle word wrap
  2. 2 of 3 members are available, ip-10-0-131-183.ec2.internal is unhealthy
    Copy to Clipboard Toggle word wrap

5.2.3.

참고

  1. $ oc get machines -A -ojsonpath='{range .items[*]}{@.status.nodeRef.name}{"\t"}{@.status.providerStatus.instanceState}{"\n"}' | grep -v running
    Copy to Clipboard Toggle word wrap

    ip-10-0-131-183.ec2.internal  stopped 
    1
    Copy to Clipboard Toggle word wrap

    1

    • $ oc get nodes -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{"\t"}{range .spec.taints[*]}{.key}{" "}' | grep unreachable
      Copy to Clipboard Toggle word wrap

      ip-10-0-131-183.ec2.internal	node-role.kubernetes.io/master node.kubernetes.io/unreachable node.kubernetes.io/unreachable 
      1
      Copy to Clipboard Toggle word wrap

      1
    • $ oc get nodes -l node-role.kubernetes.io/master | grep "NotReady"
      Copy to Clipboard Toggle word wrap

      ip-10-0-131-183.ec2.internal   NotReady   master   122m   v1.29.4 
      1
      Copy to Clipboard Toggle word wrap

      1

    1. $ oc get nodes -l node-role.kubernetes.io/master
      Copy to Clipboard Toggle word wrap

      NAME                           STATUS   ROLES    AGE     VERSION
      ip-10-0-131-183.ec2.internal   Ready    master   6h13m   v1.29.4
      ip-10-0-164-97.ec2.internal    Ready    master   6h13m   v1.29.4
      ip-10-0-154-204.ec2.internal   Ready    master   6h13m   v1.29.4
      Copy to Clipboard Toggle word wrap

    2. $ oc -n openshift-etcd get pods -l k8s-app=etcd
      Copy to Clipboard Toggle word wrap

      etcd-ip-10-0-131-183.ec2.internal                2/3     Error       7          6h9m 
      1
      
      etcd-ip-10-0-164-97.ec2.internal                 3/3     Running     0          6h6m
      etcd-ip-10-0-154-204.ec2.internal                3/3     Running     0          6h6m
      Copy to Clipboard Toggle word wrap

      1

5.2.4.

5.2.4.1.

참고

  • 중요

  • 중요

    1. $ oc -n openshift-etcd get pods -l k8s-app=etcd
      Copy to Clipboard Toggle word wrap

      etcd-ip-10-0-131-183.ec2.internal                3/3     Running     0          123m
      etcd-ip-10-0-164-97.ec2.internal                 3/3     Running     0          123m
      etcd-ip-10-0-154-204.ec2.internal                3/3     Running     0          124m
      Copy to Clipboard Toggle word wrap

    2. $ oc rsh -n openshift-etcd etcd-ip-10-0-154-204.ec2.internal
      Copy to Clipboard Toggle word wrap
    3. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+------------------------------+---------------------------+---------------------------+
      |        ID        | STATUS  |             NAME             |        PEER ADDRS         |       CLIENT ADDRS        |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      | 6fc1e7c9db35841d | started | ip-10-0-131-183.ec2.internal | https://10.0.131.183:2380 | https://10.0.131.183:2379 |
      | 757b6793e2408b6c | started |  ip-10-0-164-97.ec2.internal |  https://10.0.164.97:2380 |  https://10.0.164.97:2379 |
      | ca8c2990a0aa29d1 | started | ip-10-0-154-204.ec2.internal | https://10.0.154.204:2380 | https://10.0.154.204:2379 |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      Copy to Clipboard Toggle word wrap

    4. sh-4.2# etcdctl member remove 6fc1e7c9db35841d
      Copy to Clipboard Toggle word wrap

      Member 6fc1e7c9db35841d removed from cluster ead669ce1fbfb346
      Copy to Clipboard Toggle word wrap

    5. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+------------------------------+---------------------------+---------------------------+
      |        ID        | STATUS  |             NAME             |        PEER ADDRS         |       CLIENT ADDRS        |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      | 757b6793e2408b6c | started |  ip-10-0-164-97.ec2.internal |  https://10.0.164.97:2380 |  https://10.0.164.97:2379 |
      | ca8c2990a0aa29d1 | started | ip-10-0-154-204.ec2.internal | https://10.0.154.204:2380 | https://10.0.154.204:2379 |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      Copy to Clipboard Toggle word wrap

  1. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": {"useUnsupportedUnsafeNonHANonProductionUnstableEtcd": true}}}'
    Copy to Clipboard Toggle word wrap

    중요

    참고

  2. $ oc delete node <node_name>
    Copy to Clipboard Toggle word wrap

    $ oc delete node ip-10-0-131-183.ec2.internal
    Copy to Clipboard Toggle word wrap

    1. $ oc get secrets -n openshift-etcd | grep ip-10-0-131-183.ec2.internal 
      1
      Copy to Clipboard Toggle word wrap
      1

      etcd-peer-ip-10-0-131-183.ec2.internal              kubernetes.io/tls                     2      47m
      etcd-serving-ip-10-0-131-183.ec2.internal           kubernetes.io/tls                     2      47m
      etcd-serving-metrics-ip-10-0-131-183.ec2.internal   kubernetes.io/tls                     2      47m
      Copy to Clipboard Toggle word wrap

      1. $ oc delete secret -n openshift-etcd etcd-peer-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
      2. $ oc delete secret -n openshift-etcd etcd-serving-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
      3. $ oc delete secret -n openshift-etcd etcd-serving-metrics-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
    1. $ oc get machines -n openshift-machine-api -o wide
      Copy to Clipboard Toggle word wrap

      NAME                                        PHASE     TYPE        REGION      ZONE         AGE     NODE                           PROVIDERID                              STATE
      clustername-8qw5l-master-0                  Running   m4.xlarge   us-east-1   us-east-1a   3h37m   ip-10-0-131-183.ec2.internal   aws:///us-east-1a/i-0ec2782f8287dfb7e   stopped 
      1
      
      clustername-8qw5l-master-1                  Running   m4.xlarge   us-east-1   us-east-1b   3h37m   ip-10-0-154-204.ec2.internal   aws:///us-east-1b/i-096c349b700a19631   running
      clustername-8qw5l-master-2                  Running   m4.xlarge   us-east-1   us-east-1c   3h37m   ip-10-0-164-97.ec2.internal    aws:///us-east-1c/i-02626f1dba9ed5bba   running
      clustername-8qw5l-worker-us-east-1a-wbtgd   Running   m4.large    us-east-1   us-east-1a   3h28m   ip-10-0-129-226.ec2.internal   aws:///us-east-1a/i-010ef6279b4662ced   running
      clustername-8qw5l-worker-us-east-1b-lrdxb   Running   m4.large    us-east-1   us-east-1b   3h28m   ip-10-0-144-248.ec2.internal   aws:///us-east-1b/i-0cb45ac45a166173b   running
      clustername-8qw5l-worker-us-east-1c-pkg26   Running   m4.large    us-east-1   us-east-1c   3h28m   ip-10-0-170-181.ec2.internal   aws:///us-east-1c/i-06861c00007751b0a   running
      Copy to Clipboard Toggle word wrap

      1
    2. $ oc delete machine -n openshift-machine-api clustername-8qw5l-master-0 
      1
      Copy to Clipboard Toggle word wrap
      1

    3. $ oc get machines -n openshift-machine-api -o wide
      Copy to Clipboard Toggle word wrap

      NAME                                        PHASE          TYPE        REGION      ZONE         AGE     NODE                           PROVIDERID                              STATE
      clustername-8qw5l-master-1                  Running        m4.xlarge   us-east-1   us-east-1b   3h37m   ip-10-0-154-204.ec2.internal   aws:///us-east-1b/i-096c349b700a19631   running
      clustername-8qw5l-master-2                  Running        m4.xlarge   us-east-1   us-east-1c   3h37m   ip-10-0-164-97.ec2.internal    aws:///us-east-1c/i-02626f1dba9ed5bba   running
      clustername-8qw5l-master-3                  Provisioning   m4.xlarge   us-east-1   us-east-1a   85s     ip-10-0-133-53.ec2.internal    aws:///us-east-1a/i-015b0888fe17bc2c8   running 
      1
      
      clustername-8qw5l-worker-us-east-1a-wbtgd   Running        m4.large    us-east-1   us-east-1a   3h28m   ip-10-0-129-226.ec2.internal   aws:///us-east-1a/i-010ef6279b4662ced   running
      clustername-8qw5l-worker-us-east-1b-lrdxb   Running        m4.large    us-east-1   us-east-1b   3h28m   ip-10-0-144-248.ec2.internal   aws:///us-east-1b/i-0cb45ac45a166173b   running
      clustername-8qw5l-worker-us-east-1c-pkg26   Running        m4.large    us-east-1   us-east-1c   3h28m   ip-10-0-170-181.ec2.internal   aws:///us-east-1c/i-06861c00007751b0a   running
      Copy to Clipboard Toggle word wrap

      1

  3. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": null}}'
    Copy to Clipboard Toggle word wrap
  4. $ oc get etcd/cluster -oyaml
    Copy to Clipboard Toggle word wrap
  5. EtcdCertSignerControllerDegraded: [Operation cannot be fulfilled on secrets "etcd-peer-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-metrics-sno-0": the object has been modified; please apply your changes to the latest version and try again]
    Copy to Clipboard Toggle word wrap

  1. $ oc -n openshift-etcd get pods -l k8s-app=etcd
    Copy to Clipboard Toggle word wrap

    etcd-ip-10-0-133-53.ec2.internal                 3/3     Running     0          7m49s
    etcd-ip-10-0-164-97.ec2.internal                 3/3     Running     0          123m
    etcd-ip-10-0-154-204.ec2.internal                3/3     Running     0          124m
    Copy to Clipboard Toggle word wrap

    $ oc patch etcd cluster -p='{"spec": {"forceRedeploymentReason": "recovery-'"$( date --rfc-3339=ns )"'"}}' --type=merge 
    1
    Copy to Clipboard Toggle word wrap
    1
    1. $ oc rsh -n openshift-etcd etcd-ip-10-0-154-204.ec2.internal
      Copy to Clipboard Toggle word wrap
    2. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+------------------------------+---------------------------+---------------------------+
      |        ID        | STATUS  |             NAME             |        PEER ADDRS         |       CLIENT ADDRS        |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      | 5eb0d6b8ca24730c | started |  ip-10-0-133-53.ec2.internal |  https://10.0.133.53:2380 |  https://10.0.133.53:2379 |
      | 757b6793e2408b6c | started |  ip-10-0-164-97.ec2.internal |  https://10.0.164.97:2380 |  https://10.0.164.97:2379 |
      | ca8c2990a0aa29d1 | started | ip-10-0-154-204.ec2.internal | https://10.0.154.204:2380 | https://10.0.154.204:2379 |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      Copy to Clipboard Toggle word wrap

      주의

5.2.4.2.

  • 중요

    1. $ oc debug node/ip-10-0-131-183.ec2.internal 
      1
      Copy to Clipboard Toggle word wrap
      1
    2. sh-4.2# chroot /host
      Copy to Clipboard Toggle word wrap
    3. sh-4.2# mkdir /var/lib/etcd-backup
      Copy to Clipboard Toggle word wrap
      sh-4.2# mv /etc/kubernetes/manifests/etcd-pod.yaml /var/lib/etcd-backup/
      Copy to Clipboard Toggle word wrap
    4. sh-4.2# mv /var/lib/etcd/ /tmp
      Copy to Clipboard Toggle word wrap

    1. $ oc -n openshift-etcd get pods -l k8s-app=etcd
      Copy to Clipboard Toggle word wrap

      etcd-ip-10-0-131-183.ec2.internal                2/3     Error       7          6h9m
      etcd-ip-10-0-164-97.ec2.internal                 3/3     Running     0          6h6m
      etcd-ip-10-0-154-204.ec2.internal                3/3     Running     0          6h6m
      Copy to Clipboard Toggle word wrap

    2. $ oc rsh -n openshift-etcd etcd-ip-10-0-154-204.ec2.internal
      Copy to Clipboard Toggle word wrap
    3. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+------------------------------+---------------------------+---------------------------+
      |        ID        | STATUS  |             NAME             |        PEER ADDRS         |       CLIENT ADDRS        |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      | 62bcf33650a7170a | started | ip-10-0-131-183.ec2.internal | https://10.0.131.183:2380 | https://10.0.131.183:2379 |
      | b78e2856655bc2eb | started |  ip-10-0-164-97.ec2.internal |  https://10.0.164.97:2380 |  https://10.0.164.97:2379 |
      | d022e10b498760d5 | started | ip-10-0-154-204.ec2.internal | https://10.0.154.204:2380 | https://10.0.154.204:2379 |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      Copy to Clipboard Toggle word wrap

    4. sh-4.2# etcdctl member remove 62bcf33650a7170a
      Copy to Clipboard Toggle word wrap

      Member 62bcf33650a7170a removed from cluster ead669ce1fbfb346
      Copy to Clipboard Toggle word wrap

    5. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+------------------------------+---------------------------+---------------------------+
      |        ID        | STATUS  |             NAME             |        PEER ADDRS         |       CLIENT ADDRS        |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      | b78e2856655bc2eb | started |  ip-10-0-164-97.ec2.internal |  https://10.0.164.97:2380 |  https://10.0.164.97:2379 |
      | d022e10b498760d5 | started | ip-10-0-154-204.ec2.internal | https://10.0.154.204:2380 | https://10.0.154.204:2379 |
      +------------------+---------+------------------------------+---------------------------+---------------------------+
      Copy to Clipboard Toggle word wrap

  1. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": {"useUnsupportedUnsafeNonHANonProductionUnstableEtcd": true}}}'
    Copy to Clipboard Toggle word wrap

    1. $ oc get secrets -n openshift-etcd | grep ip-10-0-131-183.ec2.internal 
      1
      Copy to Clipboard Toggle word wrap
      1

      etcd-peer-ip-10-0-131-183.ec2.internal              kubernetes.io/tls                     2      47m
      etcd-serving-ip-10-0-131-183.ec2.internal           kubernetes.io/tls                     2      47m
      etcd-serving-metrics-ip-10-0-131-183.ec2.internal   kubernetes.io/tls                     2      47m
      Copy to Clipboard Toggle word wrap

      1. $ oc delete secret -n openshift-etcd etcd-peer-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
      2. $ oc delete secret -n openshift-etcd etcd-serving-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
      3. $ oc delete secret -n openshift-etcd etcd-serving-metrics-ip-10-0-131-183.ec2.internal
        Copy to Clipboard Toggle word wrap
  2. $ oc patch etcd cluster -p='{"spec": {"forceRedeploymentReason": "single-master-recovery-'"$( date --rfc-3339=ns )"'"}}' --type=merge 
    1
    Copy to Clipboard Toggle word wrap
    1

  3. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": null}}'
    Copy to Clipboard Toggle word wrap
  4. $ oc get etcd/cluster -oyaml
    Copy to Clipboard Toggle word wrap
  5. EtcdCertSignerControllerDegraded: [Operation cannot be fulfilled on secrets "etcd-peer-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-metrics-sno-0": the object has been modified; please apply your changes to the latest version and try again]
    Copy to Clipboard Toggle word wrap

    1. $ oc rsh -n openshift-etcd etcd-ip-10-0-154-204.ec2.internal
      Copy to Clipboard Toggle word wrap
    2. sh-4.2# etcdctl endpoint health
      Copy to Clipboard Toggle word wrap

      https://10.0.131.183:2379 is healthy: successfully committed proposal: took = 16.671434ms
      https://10.0.154.204:2379 is healthy: successfully committed proposal: took = 16.698331ms
      https://10.0.164.97:2379 is healthy: successfully committed proposal: took = 16.621645ms
      Copy to Clipboard Toggle word wrap

5.2.4.3.

  • 중요

    1. $ oc -n openshift-etcd get pods -l k8s-app=etcd -o wide
      Copy to Clipboard Toggle word wrap

      etcd-openshift-control-plane-0   5/5   Running   11   3h56m   192.168.10.9   openshift-control-plane-0  <none>           <none>
      etcd-openshift-control-plane-1   5/5   Running   0    3h54m   192.168.10.10   openshift-control-plane-1   <none>           <none>
      etcd-openshift-control-plane-2   5/5   Running   0    3h58m   192.168.10.11   openshift-control-plane-2   <none>           <none>
      Copy to Clipboard Toggle word wrap

    2. $ oc rsh -n openshift-etcd etcd-openshift-control-plane-0
      Copy to Clipboard Toggle word wrap
    3. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+--------------------+---------------------------+---------------------------+---------------------+
      | ID               | STATUS  | NAME                      | PEER ADDRS                  | CLIENT ADDRS                | IS LEARNER |
      +------------------+---------+--------------------+---------------------------+---------------------------+---------------------+
      | 7a8197040a5126c8 | started | openshift-control-plane-2 | https://192.168.10.11:2380/ | https://192.168.10.11:2379/ | false |
      | 8d5abe9669a39192 | started | openshift-control-plane-1 | https://192.168.10.10:2380/ | https://192.168.10.10:2379/ | false |
      | cc3830a72fc357f9 | started | openshift-control-plane-0 | https://192.168.10.9:2380/ | https://192.168.10.9:2379/   | false |
      +------------------+---------+--------------------+---------------------------+---------------------------+---------------------+
      Copy to Clipboard Toggle word wrap

    4. 주의

      sh-4.2# etcdctl member remove 7a8197040a5126c8
      Copy to Clipboard Toggle word wrap

      Member 7a8197040a5126c8 removed from cluster b23536c33f2cdd1b
      Copy to Clipboard Toggle word wrap

    5. sh-4.2# etcdctl member list -w table
      Copy to Clipboard Toggle word wrap

      +------------------+---------+--------------------+---------------------------+---------------------------+-------------------------+
      | ID               | STATUS  | NAME                      | PEER ADDRS                  | CLIENT ADDRS                | IS LEARNER |
      +------------------+---------+--------------------+---------------------------+---------------------------+-------------------------+
      | cc3830a72fc357f9 | started | openshift-control-plane-2 | https://192.168.10.11:2380/ | https://192.168.10.11:2379/ | false |
      | 8d5abe9669a39192 | started | openshift-control-plane-1 | https://192.168.10.10:2380/ | https://192.168.10.10:2379/ | false |
      +------------------+---------+--------------------+---------------------------+---------------------------+-------------------------+
      Copy to Clipboard Toggle word wrap

      중요

  1. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": {"useUnsupportedUnsafeNonHANonProductionUnstableEtcd": true}}}'
    Copy to Clipboard Toggle word wrap

    1. $ oc get secrets -n openshift-etcd | grep openshift-control-plane-2
      Copy to Clipboard Toggle word wrap

      etcd-peer-openshift-control-plane-2             kubernetes.io/tls   2   134m
      etcd-serving-metrics-openshift-control-plane-2  kubernetes.io/tls   2   134m
      etcd-serving-openshift-control-plane-2          kubernetes.io/tls   2   134m
      Copy to Clipboard Toggle word wrap
      1. $ oc delete secret etcd-peer-openshift-control-plane-2 -n openshift-etcd
        
        secret "etcd-peer-openshift-control-plane-2" deleted
        Copy to Clipboard Toggle word wrap
      2. $ oc delete secret etcd-serving-metrics-openshift-control-plane-2 -n openshift-etcd
        
        secret "etcd-serving-metrics-openshift-control-plane-2" deleted
        Copy to Clipboard Toggle word wrap
      3. $ oc delete secret etcd-serving-openshift-control-plane-2 -n openshift-etcd
        
        secret "etcd-serving-openshift-control-plane-2" deleted
        Copy to Clipboard Toggle word wrap
  2. $ oc get machines -n openshift-machine-api -o wide
    Copy to Clipboard Toggle word wrap

    NAME                              PHASE     TYPE   REGION   ZONE   AGE     NODE                               PROVIDERID                                                                                              STATE
    examplecluster-control-plane-0    Running                          3h11m   openshift-control-plane-0   baremetalhost:///openshift-machine-api/openshift-control-plane-0/da1ebe11-3ff2-41c5-b099-0aa41222964e   externally provisioned 
    1
    
    examplecluster-control-plane-1    Running                          3h11m   openshift-control-plane-1   baremetalhost:///openshift-machine-api/openshift-control-plane-1/d9f9acbc-329c-475e-8d81-03b20280a3e1   externally provisioned
    examplecluster-control-plane-2    Running                          3h11m   openshift-control-plane-2   baremetalhost:///openshift-machine-api/openshift-control-plane-2/3354bdac-61d8-410f-be5b-6a395b056135   externally provisioned
    examplecluster-compute-0          Running                          165m    openshift-compute-0         baremetalhost:///openshift-machine-api/openshift-compute-0/3d685b81-7410-4bb3-80ec-13a31858241f         provisioned
    examplecluster-compute-1          Running                          165m    openshift-compute-1         baremetalhost:///openshift-machine-api/openshift-compute-1/0fdae6eb-2066-4241-91dc-e7ea72ab13b9         provisioned
    Copy to Clipboard Toggle word wrap

    1
  3. $ oc get clusteroperator baremetal
    Copy to Clipboard Toggle word wrap

    NAME        VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    baremetal   4.16.0    True        False         False      3d15h
    Copy to Clipboard Toggle word wrap

  4. $ oc delete bmh openshift-control-plane-2 -n openshift-machine-api
    Copy to Clipboard Toggle word wrap

    baremetalhost.metal3.io "openshift-control-plane-2" deleted
    Copy to Clipboard Toggle word wrap

  5. $ oc delete machine -n openshift-machine-api examplecluster-control-plane-2
    Copy to Clipboard Toggle word wrap

    중요

    1. $ oc edit machine -n openshift-machine-api examplecluster-control-plane-2
      Copy to Clipboard Toggle word wrap
    2. finalizers:
      - machine.machine.openshift.io
      Copy to Clipboard Toggle word wrap

      machine.machine.openshift.io/examplecluster-control-plane-2 edited
      Copy to Clipboard Toggle word wrap

  6. $ oc get machines -n openshift-machine-api -o wide
    Copy to Clipboard Toggle word wrap

    NAME                              PHASE     TYPE   REGION   ZONE   AGE     NODE                                 PROVIDERID                                                                                       STATE
    examplecluster-control-plane-0    Running                          3h11m   openshift-control-plane-0   baremetalhost:///openshift-machine-api/openshift-control-plane-0/da1ebe11-3ff2-41c5-b099-0aa41222964e   externally provisioned
    examplecluster-control-plane-1    Running                          3h11m   openshift-control-plane-1   baremetalhost:///openshift-machine-api/openshift-control-plane-1/d9f9acbc-329c-475e-8d81-03b20280a3e1   externally provisioned
    examplecluster-compute-0          Running                          165m    openshift-compute-0         baremetalhost:///openshift-machine-api/openshift-compute-0/3d685b81-7410-4bb3-80ec-13a31858241f         provisioned
    examplecluster-compute-1          Running                          165m    openshift-compute-1         baremetalhost:///openshift-machine-api/openshift-compute-1/0fdae6eb-2066-4241-91dc-e7ea72ab13b9         provisioned
    Copy to Clipboard Toggle word wrap

  7. $ oc get nodes
    
    NAME                     STATUS ROLES   AGE   VERSION
    openshift-control-plane-0 Ready master 3h24m v1.29.4
    openshift-control-plane-1 Ready master 3h24m v1.29.4
    openshift-compute-0       Ready worker 176m v1.29.4
    openshift-compute-1       Ready worker 176m v1.29.4
    Copy to Clipboard Toggle word wrap
  8. $ cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: openshift-control-plane-2-bmc-secret
      namespace: openshift-machine-api
    data:
      password: <password>
      username: <username>
    type: Opaque
    ---
    apiVersion: metal3.io/v1alpha1
    kind: BareMetalHost
    metadata:
      name: openshift-control-plane-2
      namespace: openshift-machine-api
    spec:
      automatedCleaningMode: disabled
      bmc:
        address: redfish://10.46.61.18:443/redfish/v1/Systems/1
        credentialsName: openshift-control-plane-2-bmc-secret
        disableCertificateVerification: true
      bootMACAddress: 48:df:37:b0:8a:a0
      bootMode: UEFI
      externallyProvisioned: false
      online: true
      rootDeviceHints:
        deviceName: /dev/disk/by-id/scsi-<serial_number>
      userData:
        name: master-user-data-managed
        namespace: openshift-machine-api
    EOF
    Copy to Clipboard Toggle word wrap
    참고

    중요

  9. $ oc get bmh -n openshift-machine-api
    
    NAME                      STATE                  CONSUMER                      ONLINE ERROR   AGE
    openshift-control-plane-0 externally provisioned examplecluster-control-plane-0 true         4h48m
    openshift-control-plane-1 externally provisioned examplecluster-control-plane-1 true         4h48m
    openshift-control-plane-2 available              examplecluster-control-plane-3 true         47m
    openshift-compute-0       provisioned            examplecluster-compute-0       true         4h48m
    openshift-compute-1       provisioned            examplecluster-compute-1       true         4h48m
    Copy to Clipboard Toggle word wrap
    1. $ oc get machines -n openshift-machine-api -o wide
      Copy to Clipboard Toggle word wrap

      NAME                                   PHASE     TYPE   REGION   ZONE   AGE     NODE                              PROVIDERID                                                                                            STATE
      examplecluster-control-plane-0         Running                          3h11m   openshift-control-plane-0   baremetalhost:///openshift-machine-api/openshift-control-plane-0/da1ebe11-3ff2-41c5-b099-0aa41222964e   externally provisioned 
      1
      
      examplecluster-control-plane-1         Running                          3h11m   openshift-control-plane-1   baremetalhost:///openshift-machine-api/openshift-control-plane-1/d9f9acbc-329c-475e-8d81-03b20280a3e1   externally provisioned
      examplecluster-control-plane-2         Running                          3h11m   openshift-control-plane-2   baremetalhost:///openshift-machine-api/openshift-control-plane-2/3354bdac-61d8-410f-be5b-6a395b056135   externally provisioned
      examplecluster-compute-0               Running                          165m    openshift-compute-0         baremetalhost:///openshift-machine-api/openshift-compute-0/3d685b81-7410-4bb3-80ec-13a31858241f         provisioned
      examplecluster-compute-1               Running                          165m    openshift-compute-1         baremetalhost:///openshift-machine-api/openshift-compute-1/0fdae6eb-2066-4241-91dc-e7ea72ab13b9         provisioned
      Copy to Clipboard Toggle word wrap

      1

    2. $ oc get bmh -n openshift-machine-api
      Copy to Clipboard Toggle word wrap

      $ oc get bmh -n openshift-machine-api
      NAME                      STATE                  CONSUMER                       ONLINE ERROR AGE
      openshift-control-plane-0 externally provisioned examplecluster-control-plane-0 true         4h48m
      openshift-control-plane-1 externally provisioned examplecluster-control-plane-1 true         4h48m
      openshift-control-plane-2 provisioned            examplecluster-control-plane-3 true          47m
      openshift-compute-0       provisioned            examplecluster-compute-0       true         4h48m
      openshift-compute-1       provisioned            examplecluster-compute-1       true         4h48m
      Copy to Clipboard Toggle word wrap

    3. $ oc get nodes
      Copy to Clipboard Toggle word wrap

      $ oc get nodes
      NAME                     STATUS ROLES   AGE   VERSION
      openshift-control-plane-0 Ready master 4h26m v1.29.4
      openshift-control-plane-1 Ready master 4h26m v1.29.4
      openshift-control-plane-2 Ready master 12m   v1.29.4
      openshift-compute-0       Ready worker 3h58m v1.29.4
      openshift-compute-1       Ready worker 3h58m v1.29.4
      Copy to Clipboard Toggle word wrap

  10. $ oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": null}}'
    Copy to Clipboard Toggle word wrap
  11. $ oc get etcd/cluster -oyaml
    Copy to Clipboard Toggle word wrap
  12. EtcdCertSignerControllerDegraded: [Operation cannot be fulfilled on secrets "etcd-peer-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-sno-0": the object has been modified; please apply your changes to the latest version and try again, Operation cannot be fulfilled on secrets "etcd-serving-metrics-sno-0": the object has been modified; please apply your changes to the latest version and try again]
    Copy to Clipboard Toggle word wrap

  1. $ oc -n openshift-etcd get pods -l k8s-app=etcd
    Copy to Clipboard Toggle word wrap

    etcd-openshift-control-plane-0      5/5     Running     0     105m
    etcd-openshift-control-plane-1      5/5     Running     0     107m
    etcd-openshift-control-plane-2      5/5     Running     0     103m
    Copy to Clipboard Toggle word wrap

    $ oc patch etcd cluster -p='{"spec": {"forceRedeploymentReason": "recovery-'"$( date --rfc-3339=ns )"'"}}' --type=merge 
    1
    Copy to Clipboard Toggle word wrap
    1

    $ oc rsh -n openshift-etcd etcd-openshift-control-plane-0
    Copy to Clipboard Toggle word wrap
  2. sh-4.2# etcdctl member list -w table
    Copy to Clipboard Toggle word wrap

    +------------------+---------+--------------------+---------------------------+---------------------------+-----------------+
    |        ID        | STATUS  |        NAME        |        PEER ADDRS         |       CLIENT ADDRS        |    IS LEARNER    |
    +------------------+---------+--------------------+---------------------------+---------------------------+-----------------+
    | 7a8197040a5126c8 | started | openshift-control-plane-2 | https://192.168.10.11:2380 | https://192.168.10.11:2379 |   false |
    | 8d5abe9669a39192 | started | openshift-control-plane-1 | https://192.168.10.10:2380 | https://192.168.10.10:2379 |   false |
    | cc3830a72fc357f9 | started | openshift-control-plane-0 | https://192.168.10.9:2380 | https://192.168.10.9:2379 |     false |
    +------------------+---------+--------------------+---------------------------+---------------------------+-----------------+
    Copy to Clipboard Toggle word wrap

    참고

  3. # etcdctl endpoint health --cluster
    Copy to Clipboard Toggle word wrap

    https://192.168.10.10:2379 is healthy: successfully committed proposal: took = 8.973065ms
    https://192.168.10.9:2379 is healthy: successfully committed proposal: took = 11.559829ms
    https://192.168.10.11:2379 is healthy: successfully committed proposal: took = 11.665203ms
    Copy to Clipboard Toggle word wrap

  4. $ oc get etcd -o=jsonpath='{range.items[0].status.conditions[?(@.type=="NodeInstallerProgressing")]}{.reason}{"\n"}{.message}{"\n"}'
    Copy to Clipboard Toggle word wrap
    AllNodesAtLatestRevision
    Copy to Clipboard Toggle word wrap
맨 위로 이동
Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2025 Red Hat