Chapter 9. Tuning a Red Hat OpenStack Platform environment

1. Deploy an overcloud with NovaComputeCpuSharedSet defined for a given role. The value of NovaComputeCpuSharedSet applies to the cpu_shared_set parameter in the nova.conf file for hosts within that role.

   parameter_defaults:
       ComputeOvsDpdkParameters:
           OvsDpdkCoreList: “0-1,16-17”
           NovaComputeCpuSharedSet: “0-1,16-17”
           NovaComputeCpuDedicatedSet: “2-15,18-31”

2. Create a flavor to build instances with emulator threads separated into a shared pool.

   openstack flavor create --ram <size_mb> --disk <size_gb> --vcpus <vcpus> <flavor>

3. Add the hw:emulator_threads_policy extra specification, and set the value to share. Instances created with this flavor will use the instance CPUs defined in the cpu_share_set parameter in the nova.conf file.

   openstack flavor set <flavor> --property hw:emulator_threads_policy=share

1. Identify the host and name for a given instance.

   openstack server show <instance_id>

2. Use SSH to log on to the identified host as heat-admin.

   ssh heat-admin@compute-1
   [compute-1]$ sudo virsh dumpxml instance-00001 | grep `'emulatorpin cpuset'`

Building the real-time image

1. Install the libguestfs-tools package on the undercloud to get the virt-customize tool:

   (undercloud) [stack@undercloud-0 ~]$ sudo dnf install libguestfs-tools

   Important

   If you install the libguestfs-tools package on the undercloud, disable iscsid.socket to avoid port conflicts with the tripleo_iscsid service on the undercloud:

   $ sudo systemctl disable --now iscsid.socket

2. Extract the images:

   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/overcloud-full.tar
   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/ironic-python-agent.tar

3. Copy the default image:

   (undercloud) [stack@undercloud-0 ~]$ cp overcloud-full.qcow2 overcloud-realtime-compute.qcow2

4. Register your image to enable Red Hat repositories relevant to your customizations. Replace [username] and [password] with valid credentials in the following example.

   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'subscription-manager register --username=[username] --password=[password]' \
   subscription-manager release --set 8.1

   Note

   For security, you can remove credentials from the history file if they are used on the command prompt. You can delete individual lines in history using the history -d command followed by the line number.

5. Find a list of pool IDs from your account’s subscriptions, and attach the appropriate pool ID to your image.

   sudo subscription-manager list --all --available | less
   ...
   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'subscription-manager attach --pool [pool-ID]'

6. Add the repositories necessary for Red Hat OpenStack Platform with NFV.

   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms \
   --enable=rhel-8-for-x86_64-appstream-eus-rpms \
   --enable=rhel-8-for-x86_64-highavailability-eus-rpms \
   --enable=ansible-2.8-for-rhel-8-x86_64-rpms \
   --enable=openstack-16-for-rhel-8-x86_64-rpms \
   --enable=rhel-8-for-x86_64-nfv-rpms \
   --enable=advanced-virt-for-rhel-8-x86_64-rpms \
   --enable=fast-datapath-for-rhel-8-x86_64-rpms'

7. Create a script to configure real-time capabilities on the image.

   (undercloud) [stack@undercloud-0 ~]$ cat <<'EOF' > rt.sh
     #!/bin/bash
   
     set -eux
   
     dnf -v -y --setopt=protected_packages= erase kernel.$(uname -m)
     dnf -v -y install kernel-rt kernel-rt-kvm tuned-profiles-nfv-host
     EOF

8. Run the script to configure the real-time image:

   (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 -v --run rt.sh 2>&1 | tee virt-customize.log

   Note

   If you see the following line in the rt.sh script output, "grubby fatal error: unable to find a suitable template", you can ignore this error.

9. Examine the virt-customize.log file that resulted from the previous command, to check that the packages installed correctly using the rt.sh script .

   (undercloud) [stack@undercloud-0 ~]$ cat virt-customize.log | grep Verifying
   
     Verifying  : kernel-3.10.0-957.el7.x86_64                                 1/1
     Verifying  : 10:qemu-kvm-tools-rhev-2.12.0-18.el7_6.1.x86_64              1/8
     Verifying  : tuned-profiles-realtime-2.10.0-6.el7_6.3.noarch              2/8
     Verifying  : linux-firmware-20180911-69.git85c5d90.el7.noarch             3/8
     Verifying  : tuned-profiles-nfv-host-2.10.0-6.el7_6.3.noarch              4/8
     Verifying  : kernel-rt-kvm-3.10.0-957.10.1.rt56.921.el7.x86_64            5/8
     Verifying  : tuna-0.13-6.el7.noarch                                       6/8
     Verifying  : kernel-rt-3.10.0-957.10.1.rt56.921.el7.x86_64                7/8
     Verifying  : rt-setup-2.0-6.el7.x86_64                                    8/8

10. Relabel SELinux:

    (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 --selinux-relabel

11. Extract vmlinuz and initrd:

    (undercloud) [stack@undercloud-0 ~]$ mkdir image
    (undercloud) [stack@undercloud-0 ~]$ guestmount -a overcloud-realtime-compute.qcow2 -i --ro image
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/vmlinuz-3.10.0-862.rt56.804.el7.x86_64 ./overcloud-realtime-compute.vmlinuz
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/initramfs-3.10.0-862.rt56.804.el7.x86_64.img ./overcloud-realtime-compute.initrd
    (undercloud) [stack@undercloud-0 ~]$ guestunmount image

    Note

    The software version in the vmlinuz and initramfs filenames vary with the kernel version.

12. Upload the image:

    (undercloud) [stack@undercloud-0 ~]$ openstack overcloud image upload --update-existing --os-image-name overcloud-realtime-compute.qcow2

1. Create an RT-KVM flavor on the overcloud:

   # openstack flavor create  r1.small 99 4096 20 4
   # openstack flavor set --property hw:cpu_policy=dedicated 99
   # openstack flavor set --property hw:cpu_realtime=yes 99
   # openstack flavor set --property hw:mem_page_size=1GB 99
   # openstack flavor set --property hw:cpu_realtime_mask="^0-1" 99
   # openstack flavor set --property hw:cpu_emulator_threads=isolate 99

2. Launch an RT-KVM instance:

   # openstack server create  --image <rhel> --flavor r1.small --nic net-id=<dpdk-net> test-rt

3. To verify that the instance uses the assigned emulator threads, run the following command:

   # virsh dumpxml <instance-id> | grep vcpu -A1
   <vcpu placement='static'>4</vcpu>
   <cputune>
     <vcpupin vcpu='0' cpuset='1'/>
     <vcpupin vcpu='1' cpuset='3'/>
     <vcpupin vcpu='2' cpuset='5'/>
     <vcpupin vcpu='3' cpuset='7'/>
     <emulatorpin cpuset='0-1'/>
     <vcpusched vcpus='2-3' scheduler='fifo'
     priority='1'/>
   </cputune>

1. Add the NeutronPhysicalDevMappings parameter in the parameter_defaults section to link between the logical network name and the physical interface.

   parameter_defaults:
     NeutronPhysicalDevMappings:
       - sriov2:p5p2

2. Add the new property, trusted, to the SR-IOV parameters.

   parameter_defaults:
     NeutronPhysicalDevMappings:
       - sriov2:p5p2
     NovaPCIPassthrough:
       - devname: "p5p2"
         physical_network: "sriov2"
         trusted: "true"

   Note

   You must include double quotation marks around the value "true".

   Important

   Complete the following step in trusted environments, as it allows trusted port binding by non-administrative accounts.

3. Modify permissions to allow users to create and update port bindings.

   parameter_defaults:
     NeutronApiPolicies: {
       operator_create_binding_profile: { key: 'create_port:binding:profile', value: 'rule:admin_or_network_owner'},
       operator_get_binding_profile: { key: 'get_port:binding:profile', value: 'rule:admin_or_network_owner'},
       operator_update_binding_profile: { key: 'update_port:binding:profile', value: 'rule:admin_or_network_owner'}
     }

1. Create a network of type vlan.

   openstack network create trusted_vf_network  --provider-network-type vlan \
    --provider-segment 111 --provider-physical-network sriov2 \
    --external --disable-port-security

2. Create a subnet.

   openstack subnet create --network trusted_vf_network \
     --ip-version 4 --subnet-range 192.168.111.0/24 --no-dhcp \
    subnet-trusted_vf_network

3. Create a port. Set the vnic-type option to direct, and the binding-profile option to true.

   openstack port create --network sriov111 \
   --vnic-type direct --binding-profile trusted=true \
   sriov111_port_trusted

4. Create an instance, and bind it to the previously-created trusted port.

   openstack server create --image rhel --flavor dpdk  --network internal --port trusted_vf_network_port_trusted --config-drive True --wait rhel-dpdk-sriov_trusted

Verify the trusted VF configuration on the hypervisor

1. On the compute node that you created the instance, run the following command:

1. Verify that the trust status of the VF is trust on. The example output contains details of an environment that contains two ports. Note that vf 6 contains the text trust on.