Chapter 19. Scaling overcloud nodes

Procedure

1. Create a new JSON file called newnodes.json that contains details of the new node that you want to register:

   {
     "nodes":[
       {
           "mac":[
               "dd:dd:dd:dd:dd:dd"
           ],
           "cpu":"4",
           "memory":"6144",
           "disk":"40",
           "arch":"x86_64",
           "pm_type":"ipmi",
           "pm_user":"admin",
           "pm_password":"p@55w0rd!",
           "pm_addr":"192.168.24.207"
       },
       {
           "mac":[
               "ee:ee:ee:ee:ee:ee"
           ],
           "cpu":"4",
           "memory":"6144",
           "disk":"40",
           "arch":"x86_64",
           "pm_type":"ipmi",
           "pm_user":"admin",
           "pm_password":"p@55w0rd!",
           "pm_addr":"192.168.24.208"
       }
     ]
   }

   Copy to Clipboard Toggle word wrap

2. Register the new nodes:

   $ source ~/stackrc
   (undercloud)$ openstack overcloud node import newnodes.json

   Copy to Clipboard Toggle word wrap

3. Launch the introspection process for each new node:

   (undercloud)$ openstack overcloud node introspect \
    --provide <node_1> [node_2] [node_n]

   Copy to Clipboard Toggle word wrap
   • Use the --provide option to reset all the specified nodes to an available state after introspection.
   • Replace <node_1>, [node_2], and all nodes up to [node_n] with the UUID of each node that you want to introspect.

4. Configure the image properties for each new node:

   (undercloud)$ openstack overcloud node configure <node>

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the stackrc undercloud credential file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap
2. Open the overcloud-baremetal-deploy.yaml node definition file that you use to provision your bare-metal nodes.

3. Increment the count parameter for the roles that you want to scale up. For example, the following configuration increases the Object Storage node count to 4:

   - name: Controller
     count: 3
   - name: Compute
     count: 10
   - name: ObjectStorage
     count: 4

   Copy to Clipboard Toggle word wrap

4. Optional: Configure predictive node placement for the new nodes. For example, use the following configuration to provision a new Object Storage node on node03:

   - name: ObjectStorage
     count: 4
     instances:
     - hostname: overcloud-objectstorage-0
       name: node00
     - hostname: overcloud-objectstorage-1
       name: node01
     - hostname: overcloud-objectstorage-2
       name: node02
     - hostname: overcloud-objectstorage-3
       name: node03

   Copy to Clipboard Toggle word wrap
5. Optional: Define any other attributes that you want to assign to your new nodes. For more information about the properties you can use to configure node attributes in your node definition file, see Bare-metal node provisioning attributes.
6. If you use the Object Storage service (swift) and the whole disk overcloud image, overcloud-hardened-uefi-full, configure the size of the /srv partition based on the size of your disk and your storage requirements for /var and /srv. For more information, see Configuring whole disk partitions for the Object Storage service.

7. Provision the overcloud nodes:

   (undercloud)$ openstack overcloud node provision \
   --stack <stack> \
   --output <deployment_file> \
   /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.
   • Replace <deployment_file> with the name of the heat environment file to generate for inclusion in the deployment command, for example /home/stack/templates/overcloud-baremetal-deployed.yaml.

8. Monitor the provisioning progress in a separate terminal. When provisioning is successful, the node state changes from available to active:

   (undercloud)$ watch openstack baremetal node list

   Copy to Clipboard Toggle word wrap

9. Add the generated overcloud-baremetal-deployed.yaml file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/overcloud-baremetal-deployed.yaml \
     --deployed-server \
     --disable-validations \
     ...

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the stackrc undercloud credential file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap
2. Decrement the count parameter in the overcloud-baremetal-deploy.yaml file, for the roles that you want to scale down.
3. Define the hostname and name of each node that you want to remove from the stack, if they are not already defined in the instances attribute for the role.

4. Add the attribute provisioned: false to the node that you want to remove. For example, to remove the node overcloud-objectstorage-1 from the stack, include the following snippet in your overcloud-baremetal-deploy.yaml file:

   - name: ObjectStorage
     count: 3
     instances:
     - hostname: overcloud-objectstorage-0
       name: node00
     - hostname: overcloud-objectstorage-1
       name: node01
       # Removed from cluster due to disk failure
       provisioned: false
     - hostname: overcloud-objectstorage-2
       name: node02
     - hostname: overcloud-objectstorage-3
       name: node03

   Copy to Clipboard Toggle word wrap

   After you redeploy the overcloud, the nodes that you define with the provisioned: false attribute are no longer present in the stack. However, these nodes are still running in a provisioned state.

   Note

   To remove a node from the stack temporarily, deploy the overcloud with the attribute provisioned: false and then redeploy the overcloud with the attribute provisioned: true to return the node to the stack.

5. Delete the node from the overcloud:

   (undercloud)$ openstack overcloud node delete \
   --stack <stack> \
   --baremetal-deployment /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap

   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.

     Note

     Do not include the nodes that you want to remove from the stack as command arguments in the openstack overcloud node delete command.

6. Provision the overcloud nodes to generate an updated heat environment file for inclusion in the deployment command:

   (undercloud)$ openstack overcloud node provision \
   --stack <stack> \
   --output <deployment_file> \
   /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <deployment_file> with the name of the heat environment file to generate for inclusion in the deployment command, for example /home/stack/templates/overcloud-baremetal-deployed.yaml.

7. Add the overcloud-baremetal-deployed.yaml file generated by the provisioning command to the stack with your other environment files, and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy \
     ...
     -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-environment.yaml \
     -e /home/stack/templates/overcloud-baremetal-deployed.yaml \
     --deployed-server \
     --disable-validations \
     ...

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the undercloud configuration:

   (overcloud)$ source ~/stackrc

   Copy to Clipboard Toggle word wrap
2. Decrement the count parameter in the overcloud-baremetal-deploy.yaml file, for the roles that you want to scale down.
3. Define the hostname and name of each node that you want to remove from the stack, if they are not already defined in the instances attribute for the role.

4. Add the attribute provisioned: false to the node that you want to remove. For example, to remove the node overcloud-compute-1 from the stack, include the following snippet in your overcloud-baremetal-deploy.yaml file:

   - name: Compute
     count: 2
     instances:
     - hostname: overcloud-compute-0
       name: node00
     - hostname: overcloud-compute-1
       name: node01
       # Removed from cluster due to disk failure
       provisioned: false
     - hostname: overcloud-compute-2
       name: node02

   Copy to Clipboard Toggle word wrap

   After you redeploy the overcloud, the nodes that you define with the provisioned: false attribute are no longer present in the stack. However, these nodes are still running in a provisioned state.

   Note

   If you want to remove a node from the stack temporarily, you can deploy the overcloud with the attribute provisioned: false and then redeploy the overcloud with the attribute provisioned: true to return the node to the stack.

5. Delete the node from the overcloud:

   (undercloud)$ openstack overcloud node delete \
   --stack <stack> \
   --baremetal-deployment /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap

   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.

     Note

     Do not include the nodes that you want to remove from the stack as command arguments in the openstack overcloud node delete command.

6. Provision the overcloud nodes to generate an updated heat environment file for inclusion in the deployment command:

   (undercloud)$ openstack overcloud node provision \
   --stack <stack> \
   --output <deployment_file> \
   /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.
   • Replace <deployment_file> with the name of the heat environment file to generate for inclusion in the deployment command, for example /home/stack/templates/overcloud-baremetal-deployed.yaml.

7. If Instance HA is enabled, perform the following actions:

   1. Clean up the Pacemaker resources for the node:

      $ sudo pcs resource delete <scaled_down_node>
      $ sudo cibadmin -o nodes --delete --xml-text '<node id="<scaled_down_node>"/>'
      $ sudo cibadmin -o fencing-topology --delete --xml-text '<fencing-level target="<scaled_down_node>"/>'
      $ sudo cibadmin -o status --delete --xml-text '<node_state id="<scaled_down_node>"/>'
      $ sudo cibadmin -o status --delete-all --xml-text '<node id="<scaled_down_node>"/>' --force

      Copy to Clipboard Toggle word wrap
      • Replace <scaled_down_node> with the name of the removed node.

   2. Delete the STONITH device for the node:

      $ sudo pcs stonith delete <device-name>

      Copy to Clipboard Toggle word wrap
8. If you are replacing the removed Compute node on your overcloud deployment, see Replacing a removed Compute node.