Chapter 18. Network reconfiguration for single-node OpenShift

Procedure

1. Verify that the IPConfig CR exists and check its current state by running the following command:

   $ oc get ipc ipconfig -o yaml

   If the CR does not exist, verify that you installed the Lifecycle Agent.

2. Verify that the spec.stage field is set to Idle, the Idle status condition is set to true, and review the current network configuration in the status fields.

   Note

   You can only modify spec fields when the CR is in the Idle stage.

3. Edit the IPConfig CR to specify the new network configuration by running the following command:

   $ oc edit ipc ipconfig

4. Update the spec fields with your new network configuration. The following example shows a dual-stack configuration with VLAN and DNS settings:

   apiVersion: lca.openshift.io/v1
   kind: IPConfig
   metadata:
     name: ipconfig
   spec:
     stage: Idle
     ipv4:
       address: 192.0.2.10
       machineNetwork: 192.0.2.0/24
       gateway: 192.0.2.1
     ipv6:
       address: 2001:db8::10
       machineNetwork: 2001:db8::/64
       gateway: 2001:db8::1
     dnsServers:
     - 192.0.2.53
     - 2001:4860:4860::8888
     vlanID: 100
     dnsFilterOutFamily: none
     autoRollbackOnFailure:
       initMonitorTimeoutSeconds: 1800

   where:

   spec.stage

   Set this field to Config when you are ready to apply the new network settings.

   spec.ipv4.address

   Specifies the target IPv4 address. Must be within the machine network CIDR.

   spec.ipv4.machineNetwork

   Specifies the target machine network CIDR.

   spec.ipv4.gateway

   Specifies the target default gateway.

   spec.dnsServers

   Specifies an ordered list of DNS servers. The first server in the list is used as the primary DNS server. Use a maximum of two servers.

   spec.vlanID

   Specifies an optional VLAN ID. Only specify if the cluster already has VLAN configuration.

   spec.dnsFilterOutFamily

   Specifies optional DNS filtering for dual-stack clusters. Set to ipv4 or ipv6 to filter out A or AAAA records respectively.

   spec.autoRollbackOnFailure.initMonitorTimeoutSeconds

   Specifies the timeout in seconds for automatic rollback if the configuration does not complete. The default value is 1800 seconds, or 30 minutes.

5. After saving the configuration, change the stage to Config to start the network reconfiguration by running the following command:

   $ oc patch ipc ipconfig --type merge -p '{"spec":{"stage":"Config"}}'

   Note

   After triggering the network reconfiguration, update your external DNS servers to resolve the cluster’s new API and ingress endpoints.

6. Monitor the progress of the configuration by running the following command:

   $ oc get ipc ipconfig -o yaml

   Watch for the following progression:

   • The controller sets the ConfigInProgress condition
   • The pre-pivot phase runs and triggers a reboot
   • After reboot, the post-pivot phase applies the network changes
   • The controller waits for cluster stabilization
   • The ConfigCompleted condition is set when successful
7. After the configuration completes successfully, verify that the status.validNextStages field includes Idle and Rollback.

8. Verify the new network configuration by running the following command:

   $ oc get nodes -o wide

9. Verify cluster health by running the following command:

   $ oc get clusteroperators

10. When you are satisfied with the new configuration, finalize the change by setting the stage to Idle. Run the following command:

    $ oc patch ipc ipconfig --type merge -p '{"spec":{"stage":"Idle"}}'

    Important

    After you finalize the configuration by transitioning to Idle, you cannot roll back to the previous network configuration. The old stateroot is removed during Idle state cleanup.

Procedure

1. Verify that rollback stage is available by checking the IPConfig CR. Run the following command:

   $ oc get ipc ipconfig -o jsonpath='{.status.validNextStages}'

   Verify that the output includes Rollback.

2. Check the rollback availability expiration timestamp by running the following command:

   $ oc get ipc ipconfig -o jsonpath='{.status.rollbackAvailabilityExpiration}'

   Note

   Plan your rollback before this timestamp. After this timestamp, rolling back requires manual recovery because of expired control plane or kubelet certificates in the rollback stateroot.

3. Trigger the rollback by setting the stage to Rollback by running the following command:

   $ oc patch ipc ipconfig --type merge -p '{"spec":{"stage":"Rollback"}}'

   The Lifecycle Agent reboots the node into the earlier stateroot.

4. After the node reboots, monitor the rollback progress by running the following command:

   $ oc get ipc ipconfig -o yaml

   Wait for the RollbackCompleted condition to be set.

5. Verify the node is using the original network configuration by running the following command:

   $ oc get nodes -o wide

6. Finalize the rollback by setting the stage to Idle by running the following command:

   $ oc patch ipc ipconfig --type merge -p '{"spec":{"stage":"Idle"}}'

Procedure

1. Inspect the IPConfig custom resource (CR) status by running the following command:

   $ oc get ipc ipconfig -o yaml

   Review the status.conditions field for the current state, reason, and message. Check status.validNextStages for possible stage transitions, and status.history for timestamps of stage progression.

2. View the Lifecycle Agent controller logs by running the following command:

   $ oc logs -n openshift-lifecycle-agent deployment/lifecycle-agent-controller-manager -c manager

3. Create a debug session on the target node by running the following command:

   $ oc debug node/<node_name>
   # chroot /host

   • Replace <node_name> with the name of your single-node OpenShift node.

4. View the relevant service logs depending on which phase you are troubleshooting by running one of the following commands:

   • View the logs for pre-pivot issues by running the following command:

     $ sudo journalctl -u lca-ipconfig-pre-pivot -b --no-pager

   • View the logs for post-pivot issues by running the following command:

     $ sudo journalctl -u ip-configuration.service -b --no-pager

   • View the logs for init-monitor watchdog issues by running the following command:

     $ sudo journalctl -u lca-init-monitor.service -b --no-pager

   • View the logs for rollback issues by running the following command:

     $ sudo journalctl -u lca-ipconfig-rollback -b --no-pager