Chapter 9. Tuning a Red Hat OpenStack Platform environment

1. Deploy an overcloud with NovaComputeCpuSharedSet defined for a given role. The value of NovaComputeCpuSharedSet applies to the cpu_shared_set parameter in the nova.conf file for hosts within that role.

   parameter_defaults:
       ComputeOvsDpdkParameters:
           OvsDpdkCoreList: “0-1,16-17”
           NovaComputeCpuSharedSet: “0-1,16-17”

2. Create a flavor to build instances with emulator threads separated into a shared pool.

   openstack flavor create --ram <size_mb> --disk <size_gb> --vcpus <vcpus> <flavor>

3. Add the hw:emulator_threads_policy extra specification, and set the value to share. Instances created with this flavor use the vCPUs defined in the cpu_share_set parameter in the nova.conf file.

   openstack flavor set <flavor> --property hw:emulator_threads_policy=share

1. Identify the host for a given instance and the name of the instance.

   openstack server show <instance_id>

2. Use SSH to log in to the identified host as heat-admin.

   ssh heat-admin@compute-1
   [compute-1]$ sudo virsh dumpxml instance-00001 | grep `'emulatorpin cpuset'`

1. Install the libguestfs-tools package on the undercloud to get the virt-customize tool:

   (undercloud) [stack@undercloud-0 ~]$ sudo dnf install libguestfs-tools

   Important

   If you install the libguestfs-tools package on the undercloud, disable iscsid.socket to avoid port conflicts with the tripleo_iscsid service on the undercloud:

   $ sudo systemctl disable --now iscsid.socket

2. Extract the images:

   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/overcloud-full.tar
   (undercloud) [stack@undercloud-0 ~]$ tar -xf /usr/share/rhosp-director-images/ironic-python-agent.tar

3. Copy the default image:

   (undercloud) [stack@undercloud-0 ~]$ cp overcloud-full.qcow2 overcloud-realtime-compute.qcow2

4. Register your image to enable Red Hat repositories relevant to your customizations. Replace [username] and [password] with valid credentials in the following example.

   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'subscription-manager register --username=[username] --password=[password]'

   Note

   Remove credentials from the history file anytime they are used on the command prompt. You can delete individual lines in history using the history -d command followed by the line number.

5. Obtain a list of pool IDs from your account’s subscriptions, and attach the appropriate pool ID to your image.

   sudo subscription-manager list --all --available | less
   ...
   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'subscription-manager attach --pool [pool-ID]'

6. Add repositories necessary for Red Hat OpenStack Platform with NFV.

   virt-customize -a overcloud-realtime-compute.qcow2 --run-command \
   'sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms \
   --enable=rhel-8-for-x86_64-appstream-rpms \
   --enable=rhel-8-for-x86_64-highavailability-rpms \
   --enable=ansible-2.8-for-rhel-8-x86_64-rpms \
   --enable=openstack-15-for-rhel-8-x86_64-rpms \
   --enable=rhel-8-for-x86_64-nfv-rpms \
   --enable=advanced-virt-for-rhel-8-x86_64-rpms \
   --enable=fast-datapath-for-rhel-8-x86_64-rpms'

7. Create a script to configure real-time capabilities on the image.

   (undercloud) [stack@undercloud-0 ~]$ cat <<'EOF' > rt.sh
     #!/bin/bash
   
     set -eux
   
     dnf -v -y --setopt=protected_packages= erase kernel.$(uname -m)
     dnf -v -y install kernel-rt kernel-rt-kvm tuned-profiles-nfv-host
     EOF

8. Run the script to configure the RT image:

   (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 -v --run rt.sh 2>&1 | tee virt-customize.log

   Note

   You might see the following error in the rt.sh script output: grubby fatal error: unable to find a suitable template. You can safely ignore this error.

9. To verify that the packages installed correctly, examine the virt-customize.log file that you created with the previous command.

   (undercloud) [stack@undercloud-0 ~]$ cat virt-customize.log | grep Verifying
   
     Verifying  : kernel-3.10.0-957.el7.x86_64                                 1/1
     Verifying  : 10:qemu-kvm-tools-rhev-2.12.0-18.el7_6.1.x86_64              1/8
     Verifying  : tuned-profiles-realtime-2.10.0-6.el7_6.3.noarch              2/8
     Verifying  : linux-firmware-20180911-69.git85c5d90.el7.noarch             3/8
     Verifying  : tuned-profiles-nfv-host-2.10.0-6.el7_6.3.noarch              4/8
     Verifying  : kernel-rt-kvm-3.10.0-957.10.1.rt56.921.el7.x86_64            5/8
     Verifying  : tuna-0.13-6.el7.noarch                                       6/8
     Verifying  : kernel-rt-3.10.0-957.10.1.rt56.921.el7.x86_64                7/8
     Verifying  : rt-setup-2.0-6.el7.x86_64                                    8/8

10. Relabel SELinux:

    (undercloud) [stack@undercloud-0 ~]$ virt-customize -a overcloud-realtime-compute.qcow2 --selinux-relabel

11. Extract vmlinuz and initrd:

    (undercloud) [stack@undercloud-0 ~]$ mkdir image
    (undercloud) [stack@undercloud-0 ~]$ guestmount -a overcloud-realtime-compute.qcow2 -i --ro image
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/vmlinuz-3.10.0-862.rt56.804.el7.x86_64 ./overcloud-realtime-compute.vmlinuz
    (undercloud) [stack@undercloud-0 ~]$ cp image/boot/initramfs-3.10.0-862.rt56.804.el7.x86_64.img ./overcloud-realtime-compute.initrd
    (undercloud) [stack@undercloud-0 ~]$ guestunmount image

    Note

    The software version in the vmlinuz and initramfs filenames vary with the kernel version.

12. Upload the image:

    (undercloud) [stack@undercloud-0 ~]$ openstack overcloud image upload --update-existing --os-image-name overcloud-realtime-compute.qcow2

1. Create an RT-KVM flavor on the overcloud:

   # openstack flavor create  r1.small 99 4096 20 4
   # openstack flavor set --property hw:cpu_policy=dedicated 99
   # openstack flavor set --property hw:cpu_realtime=yes 99
   # openstack flavor set --property hw:mem_page_size=1GB 99
   # openstack flavor set --property hw:cpu_realtime_mask="^0-1" 99
   # openstack flavor set --property hw:cpu_emulator_threads=isolate 99

2. Launch an RT-KVM instance:

   # openstack server create  --image <rhel> --flavor r1.small --nic net-id=<dpdk-net> test-rt

3. Optional: Verify that the instance uses the assigned emulator threads:

   # virsh dumpxml <instance-id> | grep vcpu -A1
   <vcpu placement='static'>4</vcpu>
   <cputune>
     <vcpupin vcpu='0' cpuset='1'/>
     <vcpupin vcpu='1' cpuset='3'/>
     <vcpupin vcpu='2' cpuset='5'/>
     <vcpupin vcpu='3' cpuset='7'/>
     <emulatorpin cpuset='0-1'/>
     <vcpusched vcpus='2-3' scheduler='fifo'
     priority='1'/>
   </cputune>

1. Add the NeutronPhysicalDevMappings parameter under the parameter_defaults section to make the link between the logical network name and the physical interface.

   parameter_defaults:
     NeutronPhysicalDevMappings: "sriov2:p5p2"

2. Add the new property "trusted" to the existing parameters related to SR-IOV.

   parameter_defaults:
     NeutronPhysicalDevMappings: "sriov2:p5p2"
     NeutronSriovNumVFs: ["p5p2:8"]
     NovaPCIPassthrough:
       - devname: "p5p2"
         physical_network: "sriov2"
         trusted: "true"

   Note

   You must include quotation marks around the value "true".

   Important

   Complete the following step only in trusted environments. This step enables non-administrative accounts the ability to bind trusted ports.

3. Modify permissions to allow users the capability of creating and updating port bindings.

   parameter_defaults:
     NeutronApiPolicies: {
       operator_create_binding_profile: { key: 'create_port:binding:profile', value: 'rule:admin_or_network_owner'},
       operator_get_binding_profile: { key: 'get_port:binding:profile', value: 'rule:admin_or_network_owner'},
       operator_update_binding_profile: { key: 'update_port:binding:profile', value: 'rule:admin_or_network_owner'}
     }

1. Create a network of type vlan.

   openstack network create trusted_vf_network  --provider-network-type vlan \
    --provider-segment 111 --provider-physical-network sriov2 \
    --external --disable-port-security

2. Create a subnet.

   openstack subnet create --network trusted_vf_network \
     --ip-version 4 --subnet-range 192.168.111.0/24 --no-dhcp \
    subnet-trusted_vf_network

3. Create a port, setting the vnic-type option to direct, and the binding-profile option to true.

   openstack port create --network sriov111 \
   --vnic-type direct --binding-profile trusted=true \
   sriov111_port_trusted

4. Create an instance binding it to the previously created trusted port.

   openstack server create --image rhel --flavor dpdk  --network internal --port trusted_vf_network_port_trusted --config-drive True --wait rhel-dpdk-sriov_trusted

Verify the trusted virtual function configuration on the hypervisor

1. On the compute node that hosts the newly created instance, run the following command:

1. View the output of the ip link command and verify that the trust status of the virtual function is trust on. The example output contains details of an environment that contains two ports. Note that vf 6 contains the text trust on.