Chapter 14. Rebooting Nodes
Some situations require a reboot of nodes in the undercloud and overcloud. The following procedures show how to reboot different node types. Be aware of the following notes:
- If rebooting all nodes in one role, it is advisable to reboot each node individually. This helps retain services for that role during the reboot.
- If rebooting all nodes in your OpenStack Platform environment, use the following list to guide the reboot order:
Recommended Node Reboot Order
- Reboot the undercloud node
- Reboot Controller and other composable nodes
- Reboot standalone Ceph MON nodes
- Reboot Ceph Storage nodes
- Reboot Object Storage service (swift) nodes
- Reboot Compute nodes
14.1. Rebooting the undercloud node Copy linkLink copied to clipboard!
The following procedure reboots the undercloud node.
Procedure
-
Log into the undercloud as the
stackuser. Reboot the undercloud:
$ sudo reboot- Wait until the node boots.
14.2. Rebooting controller and composable nodes Copy linkLink copied to clipboard!
The following procedure reboots controller nodes and standalone nodes based on composable roles. This excludes Compute nodes and Ceph Storage nodes.
Procedure
- Log in to the node that you want to reboot.
Optional: If the node uses Pacemaker resources, stop the cluster:
[heat-admin@overcloud-controller-0 ~]$ sudo pcs cluster stopReboot the node:
[heat-admin@overcloud-controller-0 ~]$ sudo reboot- Wait until the node boots.
Check the services. For example:
If the node uses Pacemaker services, check that the node has rejoined the cluster:
[heat-admin@overcloud-controller-0 ~]$ sudo pcs statusIf the node uses Systemd services, check that all services are enabled:
[heat-admin@overcloud-controller-0 ~]$ sudo systemctl status- Repeat these steps for all Controller and composable nodes.
14.3. Rebooting standalone Ceph MON nodes Copy linkLink copied to clipboard!
Procedure
- Log into a Ceph MON node.
Reboot the node:
$ sudo reboot- Wait until the node boots and rejoins the MON cluster.
Repeat these steps for each MON node in the cluster.
14.4. Rebooting a Ceph Storage (OSD) cluster Copy linkLink copied to clipboard!
The following procedure reboots a cluster of Ceph Storage (OSD) nodes.
Procedure
Log in to a Ceph MON or Controller node and disable Ceph Storage cluster rebalancing temporarily:
$ sudo ceph osd set noout $ sudo ceph osd set norebalance- Select the first Ceph Storage node to reboot and log into it.
Reboot the node:
$ sudo reboot- Wait until the node boots.
Log in to a Ceph MON or Controller node and check the cluster status:
$ sudo ceph -sCheck that the
pgmapreports allpgsas normal (active+clean).- Log out of the Ceph MON or Controller node, reboot the next Ceph Storage node, and check its status. Repeat this process until you have rebooted all Ceph storage nodes.
When complete, log into a Ceph MON or Controller node and enable cluster rebalancing again:
$ sudo ceph osd unset noout $ sudo ceph osd unset norebalancePerform a final status check to verify the cluster reports
HEALTH_OK:$ sudo ceph status
14.5. Rebooting Object Storage service (swift) nodes Copy linkLink copied to clipboard!
The following procedure reboots Object Storage service (swift) nodes. Complete the following steps for every Object Storage node in your cluster.
Procedure
- Log in to an Object Storage node.
Reboot the node:
$ sudo reboot- Wait until the node boots.
- Repeat the reboot for each Object Storage node in the cluster.
14.6. Rebooting Compute nodes Copy linkLink copied to clipboard!
Rebooting a Compute node involves the following workflow:
- Select a Compute node to reboot and disable it so that it does not provision new instances.
- Migrate the instances to another Compute node to minimise instance downtime.
- Reboot the empty Compute node and enable it.
Procedure
-
Log in to the undercloud as the
stackuser. To identify the Compute node that you intend to reboot, list all Compute nodes:
$ source ~/stackrc (undercloud) $ openstack server list --name computeFrom the overcloud, select a Compute Node and disable it:
$ source ~/overcloudrc (overcloud) $ openstack compute service list (overcloud) $ openstack compute service set <hostname> nova-compute --disableList all instances on the Compute node:
(overcloud) $ openstack server list --host <hostname> --all-projects- Migrate your instances. For more information on migration strategies, see Migrating virtual machines between Compute nodes.
Log into the Compute Node and reboot it:
[heat-admin@overcloud-compute-0 ~]$ sudo reboot- Wait until the node boots.
Enable the Compute node:
$ source ~/overcloudrc (overcloud) $ openstack compute service set <hostname> nova-compute --enableVerify that the Compute node is enabled:
(overcloud) $ openstack compute service list