Configuring the Compute service for instance creation

Procedure

1. Log in to the undercloud as the stack user.

2. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

   Copy to Clipboard Toggle word wrap

3. Generate a new roles data file named roles_data_cpu_pinning.yaml that includes the Controller, Compute, and ComputeCPUPinning roles, along with any other roles that you need for the overcloud:

   (undercloud)$ openstack overcloud roles \
    generate -o /home/stack/templates/roles_data_cpu_pinning.yaml \
    Compute:ComputeCPUPinning Compute Controller

   Copy to Clipboard Toggle word wrap

4. Open roles_data_cpu_pinning.yaml and edit or add the following parameters and sections:

   Expand

   Section/Parameter	Current value	New value
   Role comment	Role: Compute	Role: ComputeCPUPinning
   Role name	name: Compute	name: ComputeCPUPinning
   description	Basic Compute Node role	CPU Pinning Compute Node role
   HostnameFormatDefault	%stackname%-novacompute-%index%	%stackname%-novacomputepinning-%index%
   deprecated_nic_config_name	compute.yaml	compute-cpu-pinning.yaml

   Show more
5. Register the CPU pinning Compute nodes for the overcloud by adding them to your node definition template, node.json or node.yaml. For more information, see Registering nodes for the overcloud in the Installing and managing Red Hat OpenStack Platform with director guide.

6. Inspect the node hardware:

   (undercloud)$ openstack overcloud node introspect \
    --all-manageable --provide

   Copy to Clipboard Toggle word wrap

   For more information, see Creating an inventory of the bare-metal node hardware in the Installing and managing Red Hat OpenStack Platform with director guide.

7. Tag each bare metal node that you want to designate for CPU pinning with a custom CPU pinning resource class:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.CPU-PINNING <node>

   Copy to Clipboard Toggle word wrap

   Replace <node> with the ID of the bare metal node.

8. Add the ComputeCPUPinning role to your node definition file, overcloud-baremetal-deploy.yaml, and define any predictive node placements, resource classes, network topologies, or other attributes that you want to assign to your nodes:

   - name: Controller
     count: 3
   - name: Compute
     count: 3
   - name: ComputeCPUPinning
     count: 1
     defaults:
       resource_class: baremetal.CPU-PINNING
       network_config:
         template: /home/stack/templates/nic-config/myRoleTopology.j2 

   1

   Copy to Clipboard Toggle word wrap

   1

   You can reuse an existing network topology or create a new custom network interface template for the role. For more information, see Custom network interface templates in the Installing and managing Red Hat OpenStack Platform with director guide. If you do not define the network definitions by using the network_config property, then the default network definitions are used.

   For more information about the properties you can use to configure node attributes in your node definition file, see Bare metal node provisioning attributes. For an example node definition file, see Example node definition file.

9. Run the provisioning command to provision the new nodes for your role:

   (undercloud)$ openstack overcloud node provision \
   --stack <stack> \
   [--network-config \]
   --output /home/stack/templates/overcloud-baremetal-deployed.yaml \
   /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.
   • Include the --network-config optional argument to provide the network definitions to the cli-overcloud-node-network-config.yaml Ansible playbook. If you do not define the network definitions by using the network_config property, then the default network definitions are used.

10. Monitor the provisioning progress in a separate terminal. When provisioning is successful, the node state changes from available to active:

    (undercloud)$ watch openstack baremetal node list

    Copy to Clipboard Toggle word wrap

11. If you did not run the provisioning command with the --network-config option, then configure the <Role>NetworkConfigTemplate parameters in your network-environment.yaml file to point to your NIC template files:

    parameter_defaults:
       ComputeNetworkConfigTemplate: /home/stack/templates/nic-configs/compute.j2
       ComputeCPUPinningNetworkConfigTemplate: /home/stack/templates/nic-configs/<cpu_pinning_net_top>.j2
       ControllerNetworkConfigTemplate: /home/stack/templates/nic-configs/controller.j2

    Copy to Clipboard Toggle word wrap

    Replace <cpu_pinning_net_top> with the name of the file that contains the network topology of the ComputeCPUPinning role, for example, compute.yaml to use the default network topology.

Procedure

1. Create an environment file to configure Compute nodes to reserve cores for pinned instances, floating instances, and host processes, for example, cpu_pinning.yaml.

2. To schedule instances with a NUMA topology on NUMA-capable Compute nodes, add NUMATopologyFilter to the NovaSchedulerEnabledFilters parameter in your Compute environment file, if not already present:

   parameter_defaults:
     NovaSchedulerEnabledFilters:
       - AvailabilityZoneFilter
       - ComputeFilter
       - ComputeCapabilitiesFilter
       - ImagePropertiesFilter
       - ServerGroupAntiAffinityFilter
       - ServerGroupAffinityFilter
       - PciPassthroughFilter
       - NUMATopologyFilter

   Copy to Clipboard Toggle word wrap

   For more information on NUMATopologyFilter, see Compute scheduler filters .

3. To reserve physical CPU cores for the dedicated instances, add the following configuration to cpu_pinning.yaml:

   parameter_defaults:
     ComputeCPUPinningParameters:
       NovaComputeCpuDedicatedSet: 1,3,5,7

   Copy to Clipboard Toggle word wrap

4. To reserve physical CPU cores for the shared instances, add the following configuration to cpu_pinning.yaml:

   parameter_defaults:
     ComputeCPUPinningParameters:
       ...
       NovaComputeCpuSharedSet: 2,6

   Copy to Clipboard Toggle word wrap

5. If you are not using file-backed memory, specify the amount of RAM to reserve for host processes:

   parameter_defaults:
     ComputeCPUPinningParameters:
       ...
       NovaReservedHugePages: <ram>

   Copy to Clipboard Toggle word wrap

   Replace <ram> with the amount of RAM to reserve in MB.

6. To ensure that host processes do not run on the CPU cores reserved for instances, set the parameter IsolCpusList to the CPU cores you have reserved for instances:

   parameter_defaults:
     ComputeCPUPinningParameters:
       ...
       IsolCpusList: 1-3,5-7

   Copy to Clipboard Toggle word wrap

   Specify the value of the IsolCpusList parameter using a list, or ranges, of CPU indices separated by a comma.

7. Ensure that TunedProfileName is set to cpu-partitioning:

   parameter_defaults:
     ComputeCPUPinningParameters:
       ...
       TunedProfileName: "cpu-partitioning"

   Copy to Clipboard Toggle word wrap

8. Add your new files to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -r /home/stack/templates/roles_data_cpu_pinning.yaml \
     -e /home/stack/templates/network-environment.yaml \
     -e /home/stack/templates/cpu_pinning.yaml \
     -e /home/stack/templates/overcloud-baremetal-deployed.yaml \
     -e /home/stack/templates/node-info.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the overcloudrc file:

   (undercloud)$ source ~/overcloudrc

   Copy to Clipboard Toggle word wrap

2. Create a flavor for instances that require CPU pinning:

   (overcloud)$ openstack flavor create --ram <size_mb> \
    --disk <size_gb> --vcpus <no_reserved_vcpus> pinned_cpus

   Copy to Clipboard Toggle word wrap

3. To request pinned CPUs, set the hw:cpu_policy property of the flavor to dedicated:

   (overcloud)$ openstack flavor set \
    --property hw:cpu_policy=dedicated pinned_cpus

   Copy to Clipboard Toggle word wrap

4. If you are not using file-backed memory, set the hw:mem_page_size property of the flavor to enable NUMA-aware memory allocation:

   (overcloud)$ openstack flavor set \
    --property hw:mem_page_size=<page_size> pinned_cpus

   Copy to Clipboard Toggle word wrap

   • Replace <page_size> with one of the following valid values:

     • large: Selects the largest page size supported on the host, which may be 2 MB or 1 GB on x86_64 systems.
     • small: (Default) Selects the smallest page size supported on the host. On x86_64 systems this is 4 kB (normal pages).
     • any: Selects the page size by using the hw_mem_page_size set on the image. If the page size is not specified by the image, selects the largest available page size, as determined by the libvirt driver.
     • <pagesize>: Set an explicit page size if the workload has specific requirements. Use an integer value for the page size in KB, or any standard suffix. For example: 4KB, 2MB, 2048, 1GB.
   Note

   To set hw:mem_page_size to small or any, you must have configured the amount of memory pages to reserve on each NUMA node for processes that are not instances. For more information, see Configuring huge pages on Compute nodes.

5. To place each vCPU on thread siblings, set the hw:cpu_thread_policy property of the flavor to require:

   (overcloud)$ openstack flavor set \
    --property hw:cpu_thread_policy=require pinned_cpus

   Copy to Clipboard Toggle word wrap
   Note
   • If the host does not have an SMT architecture or enough CPU cores with available thread siblings, scheduling fails. To prevent this, set hw:cpu_thread_policy to prefer instead of require. The prefer policy is the default policy that ensures that thread siblings are used when available.
   • If you use hw:cpu_thread_policy=isolate, you must have SMT disabled or use a platform that does not support SMT.

Verification

1. To verify the flavor creates an instance with dedicated CPUs, use your new flavor to launch an instance:

   (overcloud)$ openstack server create --flavor pinned_cpus \
    --image <image> pinned_cpu_instance

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the overcloudrc file:

   (undercloud)$ source ~/overcloudrc

   Copy to Clipboard Toggle word wrap

2. Create a flavor for instances that do not require CPU pinning:

   (overcloud)$ openstack flavor create --ram <size_mb> \
    --disk <size_gb> --vcpus <no_reserved_vcpus> floating_cpus

   Copy to Clipboard Toggle word wrap

3. To request floating CPUs, set the hw:cpu_policy property of the flavor to shared:

   (overcloud)$ openstack flavor set \
    --property hw:cpu_policy=shared floating_cpus

   Copy to Clipboard Toggle word wrap

4. If you are not using file-backed memory, set the hw:mem_page_size property of the flavor to enable NUMA-aware memory allocation:

   (overcloud)$ openstack flavor set \
    --property hw:mem_page_size=<page_size> pinned_cpus

   Copy to Clipboard Toggle word wrap

   • Replace <page_size> with one of the following valid values:

     • large: Selects the largest page size supported on the host, which may be 2 MB or 1 GB on x86_64 systems.
     • small: (Default) Selects the smallest page size supported on the host. On x86_64 systems this is 4 kB (normal pages).
     • any: Selects the page size by using the hw_mem_page_size set on the image. If the page size is not specified by the image, selects the largest available page size, as determined by the libvirt driver.
     • <pagesize>: Set an explicit page size if the workload has specific requirements. Use an integer value for the page size in KB, or any standard suffix. For example: 4KB, 2MB, 2048, 1GB.
   Note

   To set hw:mem_page_size to small or any, you must have configured the amount of memory pages to reserve on each NUMA node for processes that are not instances. For more information, see Configuring huge pages on Compute nodes.

Procedure

1. Source the overcloudrc file:

   (undercloud)$ source ~/overcloudrc

   Copy to Clipboard Toggle word wrap

2. Create a flavor for instances that require a mixed of dedicated and shared CPUs:

   (overcloud)$ openstack flavor create --ram <size_mb> \
    --disk <size_gb> --vcpus <number_of_reserved_vcpus> \
    --property hw:cpu_policy=mixed mixed_CPUs_flavor

   Copy to Clipboard Toggle word wrap

3. Specify which CPUs must be dedicated or shared:

   (overcloud)$ openstack flavor set \
    --property hw:cpu_dedicated_mask=<CPU_number> \
    mixed_CPUs_flavor

   Copy to Clipboard Toggle word wrap

   • Replace <CPU_number> with the CPUs that must be either dedicated or shared:

     • To specify dedicated CPUs, specify the CPU number or CPU range. For example, set the property to 2-3 to specify that CPUs 2 and 3 are dedicated and all the remaining CPUs are shared.
     • To specify shared CPUs, prepend the CPU number or CPU range with a caret (^). For example, set the property to ^0-1 to specify that CPUs 0 and 1 are shared and all the remaining CPUs are dedicated.

4. If you are not using file-backed memory, set the hw:mem_page_size property of the flavor to enable NUMA-aware memory allocation:

   (overcloud)$ openstack flavor set \
    --property hw:mem_page_size=<page_size> pinned_cpus

   Copy to Clipboard Toggle word wrap

   • Replace <page_size> with one of the following valid values:

     • large: Selects the largest page size supported on the host, which may be 2 MB or 1 GB on x86_64 systems.
     • small: (Default) Selects the smallest page size supported on the host. On x86_64 systems this is 4 kB (normal pages).
     • any: Selects the page size by using the hw_mem_page_size set on the image. If the page size is not specified by the image, selects the largest available page size, as determined by the libvirt driver.
     • <pagesize>: Set an explicit page size if the workload has specific requirements. Use an integer value for the page size in KB, or any standard suffix. For example: 4KB, 2MB, 2048, 1GB.
   Note

   To set hw:mem_page_size to small or any, you must have configured the amount of memory pages to reserve on each NUMA node for processes that are not instances. For more information, see Configuring huge pages on Compute nodes.

Procedure

1. Log in to the undercloud as the stack user.

2. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

   Copy to Clipboard Toggle word wrap
3. Open your Compute environment file.

4. Configure the instance CPU mode:

   parameter_defaults:
     ComputeParameters:
       NovaLibvirtCPUMode: <cpu_mode>

   Copy to Clipboard Toggle word wrap

   Replace <cpu_mode> with the CPU mode of each instance on the Compute node. Set to one of the following valid values:

   • host-model: (Default) Use the CPU model of the host Compute node. Use this CPU mode to automatically add critical CPU flags to the instance to provide mitigation from security flaws.

   • custom: Use to configure the specific CPU models each instance should use.

     Note

     You can also set the CPU mode to host-passthrough to use the same CPU model and feature flags as the Compute node for the instances hosted on that Compute node.

5. Optional: If you set NovaLibvirtCPUMode to custom, configure the instance CPU models that you want to customise:

   parameter_defaults:
     ComputeParameters:
       NovaLibvirtCPUMode: 'custom'
       NovaLibvirtCPUModels: <cpu_model>

   Copy to Clipboard Toggle word wrap

   • Replace <cpu_model> with a list of the CPU models that the host supports. List the CPU models in order, placing the more common and less advanced CPU models first in the list, and the more feature-rich CPU models last, for example:

     NovaLibvirtCPUModels:
           - SandyBridge
           - IvyBridge
           - Haswell-noTSX-IBRS

     Copy to Clipboard Toggle word wrap

     For a list of model names, see the /usr/share/libvirt/cpu_map.xml file, or use one of the following commands on the host Compute node:

     • For a RHEL version 8.4 Compute node:

       $ sudo podman exec -it nova_libvirt virsh cpu-models <arch>

       Copy to Clipboard Toggle word wrap

     • For a RHEL version 9.2 Compute node:

       $ sudo podman exec -it nova_virtqemud virsh cpu-models <arch>

       Copy to Clipboard Toggle word wrap
     • Replace <arch> with the name of the architecture of the Compute node, for example, x86_64.

6. Configure the CPU feature flags for instances with the specified CPU models:

   parameter_defaults:
     ComputeParameters:
       ...
       NovaLibvirtCPUModelExtraFlags: <cpu_feature_flags>

   Copy to Clipboard Toggle word wrap

   Replace <cpu_feature_flags> with a comma-separated list of feature flags to enable or disable. Prefix each flag with "+" to enable the flag, or "-" to disable it. If a prefix is not specified, the flag is enabled. For a list of the available feature flags for a given CPU model, see /usr/share/libvirt/cpu_map/*.xml.

   The following example enables the CPU feature flags pcid and ssbd for the IvyBridge and Cascadelake-Server models, and disables the feature flag mtrr.

   parameter_defaults:
     ComputeParameters:
       NovaLibvirtCPUMode: 'custom'
       NovaLibvirtCPUModels:
       - IvyBridge
       - Cascadelake-Server
       NovaLibvirtCPUModelExtraFlags: 'pcid,+ssbd,-mtrr'

   Copy to Clipboard Toggle word wrap

7. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
    -e [your environment files] \
    -e /home/stack/templates/<compute_environment_file>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a flavor for instances that require huge pages:

   $ openstack flavor create --ram <size_mb> --disk <size_gb> \
    --vcpus <no_reserved_vcpus> huge_pages

   Copy to Clipboard Toggle word wrap

2. To request huge pages, set the hw:mem_page_size property of the flavor to the required size:

   $ openstack flavor set huge_pages --property hw:mem_page_size=<page_size>

   Copy to Clipboard Toggle word wrap

   • Replace <page_size> with one of the following valid values:

     • large: Selects the largest page size supported on the host, which may be 2 MB or 1 GB on x86_64 systems.
     • small: (Default) Selects the smallest page size supported on the host. On x86_64 systems this is 4 kB (normal pages).
     • any: Selects the page size by using the hw_mem_page_size set on the image. If the page size is not specified by the image, selects the largest available page size, as determined by the libvirt driver.
     • <pagesize>: Set an explicit page size if the workload has specific requirements. Use an integer value for the page size in KB, or any standard suffix. For example: 4KB, 2MB, 2048, 1GB.

3. To verify the flavor creates an instance with huge pages, use your new flavor to launch an instance:

   $ openstack server create --flavor huge_pages \
    --image <image> huge_pages_instance

   Copy to Clipboard Toggle word wrap

   The Compute scheduler identifies a host with enough free huge pages of the required size to back the memory of the instance. If the scheduler is unable to find a host and NUMA node with enough pages, then the request will fail with a NoValidHost error.

Procedure

1. Create a first boot template file, hugepages.yaml, that runs a script to create the mounts for the huge page folders. You can use the OS::TripleO::MultipartMime resource type to send the configuration script:

   heat_template_version: <version>
   
   description: >
     Huge pages configuration
   
   resources:
     userdata:
       type: OS::Heat::MultipartMime
       properties:
         parts:
         - config: {get_resource: hugepages_config}
   
     hugepages_config:
       type: OS::Heat::SoftwareConfig
       properties:
         config: |
           #!/bin/bash
           hostname | grep -qiE 'co?mp' || exit 0
           systemctl mask dev-hugepages.mount || true
           for pagesize in 2M 1G;do
             if ! [ -d "/dev/hugepages${pagesize}" ]; then
               mkdir -p "/dev/hugepages${pagesize}"
               cat << EOF > /etc/systemd/system/dev-hugepages${pagesize}.mount
           [Unit]
           Description=${pagesize} Huge Pages File System
           Documentation=https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt
           Documentation=https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
           DefaultDependencies=no
           Before=sysinit.target
           ConditionPathExists=/sys/kernel/mm/hugepages
           ConditionCapability=CAP_SYS_ADMIN
           ConditionVirtualization=!private-users
   
           [Mount]
           What=hugetlbfs
           Where=/dev/hugepages${pagesize}
           Type=hugetlbfs
           Options=pagesize=${pagesize}
   
           [Install]
           WantedBy = sysinit.target
           EOF
             fi
           done
           systemctl daemon-reload
           for pagesize in 2M 1G;do
             systemctl enable --now dev-hugepages${pagesize}.mount
           done
   
   outputs:
     OS::stack_id:
       value: {get_resource: userdata}

   Copy to Clipboard Toggle word wrap

   The config script in this template performs the following tasks:

   1. Filters the hosts to create the mounts for the huge page folders on, by specifying hostnames that match 'co?mp'. You can update the filter grep pattern for specific computes as required.
   2. Masks the default dev-hugepages.mount systemd unit file to enable new mounts to be created using the page size.
   3. Ensures that the folders are created first.
   4. Creates systemd mount units for each pagesize.
   5. Runs systemd daemon-reload after the first loop, to include the newly created unit files.
   6. Enables each mount for 2M and 1G pagesizes. You can update this loop to include additional pagesizes, as required.

2. Optional: The /dev folder is automatically bind mounted to the nova_compute and nova_libvirt containers. If you have used a different destination for the huge page mounts, then you need to pass the mounts to the nova_compute and nova_libvirt containers:

   parameter_defaults
     NovaComputeOptVolumes:
       - /opt/dev:/opt/dev
     NovaLibvirtOptVolumes:
       - /opt/dev:/opt/dev

   Copy to Clipboard Toggle word wrap

3. Register your heat template as the OS::TripleO::NodeUserData resource type in your ~/templates/firstboot.yaml environment file:

   resource_registry:
     OS::TripleO::NodeUserData: ./hugepages.yaml

   Copy to Clipboard Toggle word wrap
   Important

   You can only register the NodeUserData resources to one heat template for each resource. Subsequent usage overrides the heat template to use.

4. Add your first boot environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
     -e [your environment files] \
     -e /home/stack/templates/firstboot.yaml \
     ...

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a file system on the alternative backing device. For example, enter the following command to create an ext4 filesystem on /dev/sdb:

   # mkfs.ext4 /dev/sdb

   Copy to Clipboard Toggle word wrap

2. Mount the backing device. For example, enter the following command to mount /dev/sdb on the default libvirt memory backing directory:

   # mount /dev/sdb /var/lib/libvirt/qemu/ram

   Copy to Clipboard Toggle word wrap
   Note

   The mount point must match the value of the QemuMemoryBackingDir parameter.

Procedure

1. Log in to the undercloud as the stack user.

2. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

   Copy to Clipboard Toggle word wrap

3. Generate a new roles data file that includes the ComputeAMDSEV role, along with any other roles that you need for the overcloud. The following example generates the roles data file roles_data_amd_sev.yaml, which includes the roles Controller and ComputeAMDSEV:

   (undercloud)$ openstack overcloud roles \
    generate -o /home/stack/templates/roles_data_amd_sev.yaml \
    Compute:ComputeAMDSEV Controller

   Copy to Clipboard Toggle word wrap

4. Open roles_data_amd_sev.yaml and edit or add the following parameters and sections:

   Expand

   Section/Parameter	Current value	New value
   Role comment	Role: Compute	Role: ComputeAMDSEV
   Role name	name: Compute	name: ComputeAMDSEV
   description	Basic Compute Node role	AMD SEV Compute Node role
   HostnameFormatDefault	%stackname%-novacompute-%index%	%stackname%-novacomputeamdsev-%index%
   deprecated_nic_config_name	compute.yaml	compute-amd-sev.yaml

   Show more
5. Register the AMD SEV Compute nodes for the overcloud by adding them to your node definition template, node.json or node.yaml. For more information, see Registering nodes for the overcloud in the Installing and managing Red Hat OpenStack Platform with director guide.

6. Inspect the node hardware:

   (undercloud)$ openstack overcloud node introspect \
    --all-manageable --provide

   Copy to Clipboard Toggle word wrap

   For more information, see Creating an inventory of the bare-metal node hardware in the Installing and managing Red Hat OpenStack Platform with director guide.

7. Tag each bare metal node that you want to designate for memory encryption with a custom AMD SEV resource class:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.AMD-SEV <node>

   Copy to Clipboard Toggle word wrap

   Replace <node> with the name or ID of the bare metal node.

8. Add the ComputeAMDSEV role to your node definition file, overcloud-baremetal-deploy.yaml, and define any predictive node placements, resource classes, network topologies, or other attributes that you want to assign to your nodes:

   - name: Controller
     count: 3
   - name: Compute
     count: 3
   - name: ComputeAMDSEV
     count: 1
     defaults:
       resource_class: baremetal.AMD-SEV
       network_config:
         template: /home/stack/templates/nic-config/myRoleTopology.j2 

   1

   Copy to Clipboard Toggle word wrap

   1

   You can reuse an existing network topology or create a new custom network interface template for the role. For more information, see Custom network interface templates in the Installing and managing Red Hat OpenStack Platform with director guide. If you do not define the network definitions by using the network_config property, then the default network definitions are used.

   For more information about the properties you can use to configure node attributes in your node definition file, see Bare metal node provisioning attributes. For an example node definition file, see Example node definition file.

9. Run the provisioning command to provision the new nodes for your role:

   (undercloud)$ openstack overcloud node provision \
   --stack <stack> \
   [--network-config \]
   --output /home/stack/templates/overcloud-baremetal-deployed.yaml \
   /home/stack/templates/overcloud-baremetal-deploy.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the stack for which the bare-metal nodes are provisioned. If not specified, the default is overcloud.
   • Include the --network-config optional argument to provide the network definitions to the cli-overcloud-node-network-config.yaml Ansible playbook. If you do not define the network definitions by using the network_config property, then the default network definitions are used.

10. Monitor the provisioning progress in a separate terminal. When provisioning is successful, the node state changes from available to active:

    (undercloud)$ watch openstack baremetal node list

    Copy to Clipboard Toggle word wrap

11. If you did not run the provisioning command with the --network-config option, then configure the <Role>NetworkConfigTemplate parameters in your network-environment.yaml file to point to your NIC template files:

    parameter_defaults:
       ComputeNetworkConfigTemplate: /home/stack/templates/nic-configs/compute.j2
       ComputeAMDSEVNetworkConfigTemplate: /home/stack/templates/nic-configs/<amd_sev_net_top>.j2
       ControllerNetworkConfigTemplate: /home/stack/templates/nic-configs/controller.j2

    Copy to Clipboard Toggle word wrap

    Replace <amd_sev_net_top> with the name of the file that contains the network topology of the ComputeAMDSEV role, for example, compute.yaml to use the default network topology.

Procedure

1. Open your Compute environment file.

2. Optional: Add the following configuration to your Compute environment file to specify the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently:

   parameter_defaults:
     ComputeAMDSEVExtraConfig:
       nova::config::nova_config:
         libvirt/num_memory_encrypted_guests:
           value: 15

   Copy to Clipboard Toggle word wrap
   Note

   The default value of the libvirt/num_memory_encrypted_guests parameter is none. If you do not set a custom value, the AMD SEV Compute nodes do not impose a limit on the number of memory-encrypted instances that the nodes can host concurrently. Instead, the hardware determines the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently, which might cause some memory-encrypted instances to fail to launch.

3. Optional: To specify that all x86_64 images use the q35 machine type by default, add the following configuration to your Compute environment file:

   parameter_defaults:
     ComputeAMDSEVParameters:
       NovaHWMachineType: x86_64=q35

   Copy to Clipboard Toggle word wrap

   If you specify this parameter value, you do not need to set the hw_machine_type property to q35 on every AMD SEV instance image.

4. To ensure that the AMD SEV Compute nodes reserve enough memory for host-level services to function, add 16MB for each potential AMD SEV instance:

   parameter_defaults:
     ComputeAMDSEVParameters:
       ...
       NovaReservedHostMemory: <libvirt/num_memory_encrypted_guests * 16>

   Copy to Clipboard Toggle word wrap

5. Configure the kernel parameters for the AMD SEV Compute nodes:

   parameter_defaults:
     ComputeAMDSEVParameters:
       ...
       KernelArgs: "hugepagesz=1GB hugepages=32 default_hugepagesz=1GB kvm_amd.sev=1"

   Copy to Clipboard Toggle word wrap
   Note

   When you first add the KernelArgs parameter to the configuration of a role, the overcloud nodes are automatically rebooted. If required, you can disable the automatic rebooting of nodes and instead perform node reboots manually after each overcloud deployment. For more information, see Configuring manual node reboot to define KernelArgs.

6. Save the updates to your Compute environment file.

7. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
    -e [your environment files] \
    -r /home/stack/templates/roles_data_amd_sev.yaml \
    -e /home/stack/templates/network-environment.yaml \
    -e /home/stack/templates/<compute_environment_file>.yaml \
    -e /home/stack/templates/overcloud-baremetal-deployed.yaml \
    -e /home/stack/templates/node-info.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a new image for memory encryption:

   (overcloud)$ openstack image create ...  \
    --property hw_firmware_type=uefi amd-sev-image

   Copy to Clipboard Toggle word wrap
   Note

   If you use an existing image, the image must have the hw_firmware_type property set to uefi.

2. Optional: Add the property hw_mem_encryption=True to the image to enable AMD SEV memory encryption on the image:

   (overcloud)$ openstack image set  \
    --property hw_mem_encryption=True amd-sev-image

   Copy to Clipboard Toggle word wrap
   Tip

   You can enable memory encryption on the flavor. For more information, see Creating a flavor for memory encryption.

3. Optional: Set the machine type to q35, if not already set in the Compute node configuration:

   (overcloud)$ openstack image set  \
    --property hw_machine_type=q35 amd-sev-image

   Copy to Clipboard Toggle word wrap

4. Optional: To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the image extra specs:

   (overcloud)$ openstack image set  \
    --property trait:HW_CPU_X86_AMD_SEV=required amd-sev-image

   Copy to Clipboard Toggle word wrap
   Tip

   You can also specify this trait on the flavor. For more information, see Creating a flavor for memory encryption.

Procedure

1. Create a flavor for memory encryption:

   (overcloud)$ openstack flavor create --vcpus 1 --ram 512 --disk 2  \
    --property hw:mem_encryption=True m1.small-amd-sev

   Copy to Clipboard Toggle word wrap

2. To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the flavor extra specs:

   (overcloud)$ openstack flavor set  \
    --property trait:HW_CPU_X86_AMD_SEV=required m1.small-amd-sev

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an instance by using an AMD SEV flavor or image. The following example creates an instance by using the flavor created in Creating a flavor for memory encryption and the image created in Creating an image for memory encryption:

   (overcloud)$ openstack server create --flavor m1.small-amd-sev \
    --image amd-sev-image amd-sev-instance

   Copy to Clipboard Toggle word wrap
2. Log in to the instance as a cloud user.

3. To verify that the instance uses memory encryption, enter the following command from the instance:

   $ dmesg | grep -i sev
   AMD Secure Encrypted Virtualization (SEV) active

   Copy to Clipboard Toggle word wrap

Procedure

1. Open your Compute environment file.
2. To exclude Compute nodes that do not support the disk format of the image used to launch an instance, set the NovaSchedulerQueryImageType parameter to True in the Compute environment file.
3. Save the updates to your Compute environment file.

4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
    -e [your environment files] \
    -e /home/stack/templates/<compute_environment_file>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Open your Compute environment file.
2. To isolate host aggregates to host only instances whose flavor and image traits match the aggregate metadata, set the NovaSchedulerEnableIsolatedAggregateFiltering parameter to True in the Compute environment file.
3. Save the updates to your Compute environment file.

4. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
    -e [your environment files] \
    -e /home/stack/templates/<compute_environment_file>.yaml

   Copy to Clipboard Toggle word wrap

5. Identify the traits you want to isolate the host aggregate for. You can select an existing trait, or create a new trait:

   • To use an existing trait, list the existing traits to retrieve the trait name:

     (overcloud)$ openstack --os-placement-api-version 1.6 trait list

     Copy to Clipboard Toggle word wrap

   • To create a new trait, enter the following command:

     (overcloud)$ openstack --os-placement-api-version 1.6 trait \
      create CUSTOM_TRAIT_NAME

     Copy to Clipboard Toggle word wrap

     Custom traits must begin with the prefix CUSTOM_ and contain only the letters A through Z, the numbers 0 through 9 and the underscore “_” character.

6. Collect the existing resource provider traits of each Compute node:

   (overcloud)$ existing_traits=$(openstack --os-placement-api-version 1.6 resource provider trait list -f value <host_uuid> | sed 's/^/--trait /')

   Copy to Clipboard Toggle word wrap

7. Check the existing resource provider traits for the traits you want to isolate the host aggregate for:

   (overcloud)$ echo $existing_traits

   Copy to Clipboard Toggle word wrap

8. If the traits you require are not already added to the resource provider, then add the existing traits and your required traits to the resource providers for each Compute node in the host aggregate:

   (overcloud)$ openstack --os-placement-api-version 1.6 \
    resource provider trait set $existing_traits \
    --trait <TRAIT_NAME> \
    <host_uuid>

   Copy to Clipboard Toggle word wrap

   Replace <TRAIT_NAME> with the name of the trait that you want to add to the resource provider. You can use the --trait option more than once to add additional traits, as required.

   Note

   This command performs a full replacement of the traits for the resource provider. Therefore, you must retrieve the list of existing resource provider traits on the host and set them again to prevent them from being removed.

9. Repeat steps 6 - 8 for each Compute node in the host aggregate.

10. Add the metadata property for the trait to the host aggregate:

    (overcloud)$ openstack --os-compute-api-version 2.53 aggregate set \
     --property trait:<TRAIT_NAME>=required <aggregate_name>

    Copy to Clipboard Toggle word wrap

11. Add the trait to a flavor or an image:

    (overcloud)$ openstack flavor set \
     --property trait:<TRAIT_NAME>=required <flavor>
    (overcloud)$ openstack image set \
     --property trait:<TRAIT_NAME>=required <image>

    Copy to Clipboard Toggle word wrap

Procedure

1. Open your Compute environment file.
2. To use the Placement service to filter by availability zone, set the NovaSchedulerQueryPlacementForAvailabilityZone parameter to True in the Compute environment file.
3. Remove the AvailabilityZoneFilter filter from the NovaSchedulerEnabledFilters parameter.
4. Save the updates to your Compute environment file.

5. Add your Compute environment file to the stack with your other environment files and deploy the overcloud:

   (undercloud)$ openstack overcloud deploy --templates \
    -e [your environment files] \
    -e /home/stack/templates/<compute_environment_file>.yaml

   Copy to Clipboard Toggle word wrap