Hyperconverged Infrastructure Guide

Procedure

1. Log in to the undercloud as the stack user.

2. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

3. Generate a new custom roles data file that includes the ComputeHCI role, along with other roles you intend to use for the overcloud. The following example generates the roles data file roles_data_hci.yaml that includes the roles Controller, ComputeHCI, Compute, and CephStorage:

   (undercloud)$ openstack overcloud roles \
    generate -o /home/stack/templates/roles_data_hci.yaml \
     Controller ComputeHCI Compute CephStorage

   Note

   The networks listed for the ComputeHCI role in the generated custom roles data file include the networks required for both Compute and Storage services, for example:

   - name: ComputeHCI
     description: |
       Compute node role hosting Ceph OSD
     tags:
       - compute
     networks:
       InternalApi:
         subnet: internal_api_subnet
       Tenant:
         subnet: tenant_subnet
       Storage:
         subnet: storage_subnet
       StorageMgmt:
         subnet: storage_mgmt_subnet

4. Create a local copy of the network_data.yaml file to add a composable network to your overcloud. The network_data.yaml file interacts with the default network environment files, /usr/share/openstack-tripleo-heat-templates/environments/*, to associate the networks you defined for your ComputeHCI role with the hyperconverged nodes. For more information, see Adding a composable network in the Advanced Overcloud Customization guide.
5. To improve the performance of Red Hat Ceph Storage, update the MTU setting for both the Storage and StorageMgmt networks to 9000, for jumbo frames, in your local copy of network_data.yaml. For more information, see Configuring MTU Settings in Director and Configuring jumbo frames.

6. Create the computeHCI overcloud flavor for hyperconverged nodes:

   (undercloud)$ openstack flavor create --id auto \
    --ram <ram_size_mb> --disk <disk_size_gb> \
    --vcpus <no_vcpus> computeHCI

   • Replace <ram_size_mb> with the RAM of the bare metal node, in MB.
   • Replace <disk_size_gb> with the size of the disk on the bare metal node, in GB.
   • Replace <no_vcpus> with the number of CPUs on the bare metal node.
   Note

   These properties are not used for scheduling instances. However, the Compute scheduler does use the disk size to determine the root partition size.

7. Retrieve a list of your nodes to identify their UUIDs:

   (undercloud)$ openstack baremetal node list

8. Tag each bare metal node that you want to designate as hyperconverged with a custom HCI resource class:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.HCI <node>

   Replace <node> with the ID of the bare metal node.

9. Associate the computeHCI flavor with the custom HCI resource class:

   (undercloud)$ openstack flavor set \
   --property resources:CUSTOM_BAREMETAL_HCI=1 \
   computeHCI

   To determine the name of a custom resource class that corresponds to a resource class of a Bare Metal service node, convert the resource class to uppercase, replace all punctuation with an underscore, and prefix with CUSTOM_.

   Note

   A flavor can request only one instance of a bare metal resource class.

10. Set the following flavor properties to prevent the Compute scheduler from using the bare metal flavor properties to schedule instances:

    (undercloud)$ openstack flavor set \
     --property resources:VCPU=0 \
     --property resources:MEMORY_MB=0 \
     --property resources:DISK_GB=0 computeHCI

11. Add the following parameters to the node-info.yaml file to specify the number of hyperconverged and Controller nodes, and the flavor to use for the hyperconverged and controller designated nodes:

    parameter_defaults:
      OvercloudComputeHCIFlavor: computeHCI
      ComputeHCICount: 3
      OvercloudControlFlavor: baremetal
      ControllerCount: 3

Procedure

1. Optional: Override the autogenerated Compute settings by adding the following parameters to a Compute environment file:

   parameter_defaults:
     ComputeHCIParameters:
       NovaReservedHostMemory: <ram>
       NovaCPUAllocationRatio: <ratio>

   • Replace <ram> with the amount of RAM to reserve for the Ceph OSD services and instance overhead on hyperconverged nodes, in MB.

   • Replace <ratio> with the ratio that the Compute scheduler should use when choosing which Compute node to deploy an instance on.

     For more information on the autogenerated Compute settings, see Process for autogenerating CPU and memory resources to reserve for the Compute service.

2. To reserve memory resources for Red Hat Ceph Storage, set the parameter is_hci to true in /home/stack/templates/storage-container-config.yaml:

   parameter_defaults:
     CephAnsibleExtraConfig:
       is_hci: true

   This allows ceph-ansible to reserve memory resources for Red Hat Ceph Storage, and reduce memory growth by Ceph OSDs, by automatically adjusting the osd_memory_target parameter setting for a HCI deployment.

   Warning

   Red Hat does not recommend directly overriding the ceph_osd_docker_memory_limit parameter.

   Note

   As of ceph-ansible 3.2, the ceph_osd_docker_memory_limit is set automatically to the maximum memory of the host, as discovered by Ansible, regardless of whether the FileStore or BlueStore back end is used.

3. Optional: By default, ceph-ansible reserves one vCPU for each Ceph OSD. If you require more than one CPU per Ceph OSD, add the following configuration to /home/stack/templates/storage-container-config.yaml:

   parameter_defaults:
     CephAnsibleExtraConfig:
       ceph_osd_docker_cpu_limit: <cpu_limit>

   Replace <cpu_limit> with the number of CPUs to reserve for each Ceph OSD.

   For more information on how to tune CPU resources based on your hardware and workload, see Red Hat Ceph Storage Hardware Selection Guide.

4. Optional: Reduce the priority of Red Hat Ceph Storage backfill and recovery operations when a Ceph OSD is removed by adding the following parameters to a Ceph environment file:

   parameter_defaults:
     CephConfigOverrides:
       osd_recovery_op_priority: <priority_value>
       osd_recovery_max_active: <no_active_recovery_requests>
       osd_max_backfills: <max_no_backfills>

   • Replace <priority_value> with the priority for recovery operations, relative to the OSD client OP priority.
   • Replace <no_active_recovery_requests> with the number of active recovery requests per OSD, at one time.

   • Replace <max_no_backfills> with the maximum number of backfills allowed to or from a single OSD.

     For more information on default Red Hat Ceph Storage backfill and recovery options, see Red Hat Ceph Storage backfill and recovery operations.

Procedure

1. Disable and rebalance the Ceph OSD services on the HCI node. This step is necessary because director does not automatically rebalance the Red Hat Ceph Storage cluster when you remove HCI or Red Hat Ceph Storage nodes.
2. Migrate the instances from the HCI nodes. For more information, see Migrating virtual machines between Compute nodes in the Configuring the Compute Service for Instance Creation guide.
3. Remove the Compute nodes from the overcloud. For more information, see Removing Compute nodes.