17.6. Optimizing virtual machine CPU performance

Procédure

1. Adjust the maximum number of vCPUs that can be attached to a VM, which takes effect on the VM’s next boot.

   For example, to increase the maximum vCPU count for the testguest VM to 8:

   # virsh setvcpus testguest 8 --maximum --config

   Note that the maximum may be limited by the CPU topology, host hardware, the hypervisor, and other factors.

2. Adjust the current number of vCPUs attached to a VM, up to the maximum configured in the previous step. For example:

   • To increase the number of vCPUs attached to the running testguest VM to 4:

     # virsh setvcpus testguest 4 --live

     This increases the VM’s performance and host load footprint of testguest until the VM’s next boot.

   • To permanently decrease the number of vCPUs attached to the testguest VM to 1:

     # virsh setvcpus testguest 1 --config

     This decreases the VM’s performance and host load footprint of testguest after the VM’s next boot. However, if needed, additional vCPUs can be hot plugged to the VM to temporarily increase its performance.

Procédure

1. In the Virtual Machines interface, click the VM whose information you want to see.

   A new page opens with an Overview section with basic information about the selected VM and a Console section to access the VM’s graphical interface.

2. Click edit next to the number of vCPUs in the Overview pane.

   The vCPU details dialog appears.

   

1. Configure the virtual CPUs for the selected VM.

   • vCPU Count - The number of vCPUs currently in use.

     Note

     The vCPU count cannot be greater than the vCPU Maximum.

   • vCPU Maximum - The maximum number of virtual CPUs that can be configured for the VM. If this value is higher than the vCPU Count, additional vCPUs can be attached to the VM.
   • Sockets - The number of sockets to expose to the VM.
   • Cores per socket - The number of cores for each socket to expose to the VM.

   • Threads per core - The number of threads for each core to expose to the VM.

     Note that the Sockets, Cores per socket, and Threads per core options adjust the CPU topology of the VM. This may be beneficial for vCPU performance and may impact the functionality of certain software in the guest OS. If a different setting is not required by your deployment, keep the default values.

2. Cliquez sur Appliquer.

   The virtual CPUs for the VM are configured.

   Note

   Changes to virtual CPU settings only take effect after the VM is restarted.

1. Pin specific vCPU threads to a specific host CPU or range of CPUs. This is also possible on non-NUMA hosts and VMs, and is recommended as a safe method of vCPU performance improvement.

   For example, the following commands pin vCPU threads 0 to 5 of the testguest6 VM to host CPUs 1, 3, 5, 7, 9, and 11, respectively:

   # virsh vcpupin testguest6 0 1
   # virsh vcpupin testguest6 1 3
   # virsh vcpupin testguest6 2 5
   # virsh vcpupin testguest6 3 7
   # virsh vcpupin testguest6 4 9
   # virsh vcpupin testguest6 5 11

   Afterwards, you can verify whether this was successful:

   # virsh vcpupin testguest6
   VCPU   CPU Affinity
   ----------------------
   0      1
   1      3
   2      5
   3      7
   4      9
   5      11

2. After pinning vCPU threads, you can also pin QEMU process threads associated with a specified VM to a specific host CPU or range of CPUs. For example, the following commands pin the QEMU process thread of testguest6 to CPUs 13 and 15, and verify this was successful:

   # virsh emulatorpin testguest6 13,15
   # virsh emulatorpin testguest6
   emulator: CPU Affinity
   ----------------------------------
          *: 13,15

3. Finally, you can also specify which host NUMA nodes will be assigned specifically to a certain VM. This can improve the host memory usage by the VM’s vCPU. For example, the following commands set testguest6 to use host NUMA nodes 3 to 5, and verify this was successful:

   # virsh numatune testguest6 --nodeset 3-5
   # virsh numatune testguest6

Solution

1. Obtain the information on the host topology:

   # virsh capabilities

   The output should include a section that looks similar to the following:

   <topology>
     <cells num="2">
       <cell id="0">
         <memory unit="KiB">15624346</memory>
         <pages unit="KiB" size="4">3906086</pages>
         <pages unit="KiB" size="2048">0</pages>
         <pages unit="KiB" size="1048576">0</pages>
         <distances>
           <sibling id="0" value="10" />
           <sibling id="1" value="21" />
         </distances>
         <cpus num="6">
           <cpu id="0" socket_id="0" core_id="0" siblings="0,3" />
           <cpu id="1" socket_id="0" core_id="1" siblings="1,4" />
           <cpu id="2" socket_id="0" core_id="2" siblings="2,5" />
           <cpu id="3" socket_id="0" core_id="0" siblings="0,3" />
           <cpu id="4" socket_id="0" core_id="1" siblings="1,4" />
           <cpu id="5" socket_id="0" core_id="2" siblings="2,5" />
         </cpus>
       </cell>
       <cell id="1">
         <memory unit="KiB">15624346</memory>
         <pages unit="KiB" size="4">3906086</pages>
         <pages unit="KiB" size="2048">0</pages>
         <pages unit="KiB" size="1048576">0</pages>
         <distances>
           <sibling id="0" value="21" />
           <sibling id="1" value="10" />
         </distances>
         <cpus num="6">
           <cpu id="6" socket_id="1" core_id="3" siblings="6,9" />
           <cpu id="7" socket_id="1" core_id="4" siblings="7,10" />
           <cpu id="8" socket_id="1" core_id="5" siblings="8,11" />
           <cpu id="9" socket_id="1" core_id="3" siblings="6,9" />
           <cpu id="10" socket_id="1" core_id="4" siblings="7,10" />
           <cpu id="11" socket_id="1" core_id="5" siblings="8,11" />
         </cpus>
       </cell>
     </cells>
   </topology>

2. Optional: Test the performance of the VM using the applicable tools and utilities.

3. Set up and mount 1 GiB huge pages on the host:

   1. Add the following line to the host’s kernel command line:

      default_hugepagesz=1G hugepagesz=1G

   2. Create the /etc/systemd/system/hugetlb-gigantic-pages.service file with the following content:

      [Unit]
      Description=HugeTLB Gigantic Pages Reservation
      DefaultDependencies=no
      Before=dev-hugepages.mount
      ConditionPathExists=/sys/devices/system/node
      ConditionKernelCommandLine=hugepagesz=1G
      
      [Service]
      Type=oneshot
      RemainAfterExit=yes
      ExecStart=/etc/systemd/hugetlb-reserve-pages.sh
      
      [Install]
      WantedBy=sysinit.target

   3. Create the /etc/systemd/hugetlb-reserve-pages.sh file with the following content:

      #!/bin/sh
      
      nodes_path=/sys/devices/system/node/
      if [ ! -d $nodes_path ]; then
      	echo "ERROR: $nodes_path does not exist"
      	exit 1
      fi
      
      reserve_pages()
      {
      	echo $1 > $nodes_path/$2/hugepages/hugepages-1048576kB/nr_hugepages
      }
      
      reserve_pages 4 node1
      reserve_pages 4 node2

      This reserves four 1GiB huge pages from node1 and four 1GiB huge pages from node2.

   4. Make the script created in the previous step executable:

      # chmod +x /etc/systemd/hugetlb-reserve-pages.sh

   5. Enable huge page reservation on boot:

      # systemctl enable hugetlb-gigantic-pages

4. Use the virsh edit command to edit the XML configuration of the VM you wish to optimize, in this example super-VM:

   # virsh edit super-vm

5. Adjust the XML configuration of the VM in the following way:

   1. Set the VM to use 8 static vCPUs. Use the <vcpu/> element to do this.

   2. Pin each of the vCPU threads to the corresponding host CPU threads that it mirrors in the topology. To do so, use the <vcpupin/> elements in the <cputune> section.

      Note that, as shown by the virsh capabilities utility above, host CPU threads are not ordered sequentially in their respective cores. In addition, the vCPU threads should be pinned to the highest available set of host cores on the same NUMA node. For a table illustration, see the Sample topology section below.

      The XML configuration for steps a. and b. can look similar to:

      <cputune>
        <vcpupin vcpu='0' cpuset='1'/>
        <vcpupin vcpu='1' cpuset='4'/>
        <vcpupin vcpu='2' cpuset='2'/>
        <vcpupin vcpu='3' cpuset='5'/>
        <vcpupin vcpu='4' cpuset='7'/>
        <vcpupin vcpu='5' cpuset='10'/>
        <vcpupin vcpu='6' cpuset='8'/>
        <vcpupin vcpu='7' cpuset='11'/>
        <emulatorpin cpuset='6,9'/>
      </cputune>

   3. Set the VM to use 1 GiB huge pages:

      <memoryBacking>
        <hugepages>
          <page size='1' unit='GiB'/>
        </hugepages>
      </memoryBacking>

   4. Configure the VM’s NUMA nodes to use memory from the corresponding NUMA nodes on the host. To do so, use the <memnode/> elements in the <numatune/> section:

      <numatune>
        <memory mode="preferred" nodeset="1"/>
        <memnode cellid="0" mode="strict" nodeset="0"/>
        <memnode cellid="1" mode="strict" nodeset="1"/>
      </numatune>

   5. Ensure the CPU mode is set to host-passthrough, and that the CPU uses cache in passthrough mode:

      <cpu mode="host-passthrough">
        <topology sockets="2" cores="2" threads="2"/>
        <cache mode="passthrough"/>

Vérification

1. Confirm that the resulting XML configuration of the VM includes a section similar to the following:

   [...]
     <memoryBacking>
       <hugepages>
         <page size='1' unit='GiB'/>
       </hugepages>
     </memoryBacking>
     <vcpu placement='static'>8</vcpu>
     <cputune>
       <vcpupin vcpu='0' cpuset='1'/>
       <vcpupin vcpu='1' cpuset='4'/>
       <vcpupin vcpu='2' cpuset='2'/>
       <vcpupin vcpu='3' cpuset='5'/>
       <vcpupin vcpu='4' cpuset='7'/>
       <vcpupin vcpu='5' cpuset='10'/>
       <vcpupin vcpu='6' cpuset='8'/>
       <vcpupin vcpu='7' cpuset='11'/>
       <emulatorpin cpuset='6,9'/>
     </cputune>
     <numatune>
       <memory mode="preferred" nodeset="1"/>
       <memnode cellid="0" mode="strict" nodeset="0"/>
       <memnode cellid="1" mode="strict" nodeset="1"/>
     </numatune>
     <cpu mode="host-passthrough">
       <topology sockets="2" cores="2" threads="2"/>
       <cache mode="passthrough"/>
       <numa>
         <cell id="0" cpus="0-3" memory="2" unit="GiB">
           <distances>
             <sibling id="0" value="10"/>
             <sibling id="1" value="21"/>
           </distances>
         </cell>
         <cell id="1" cpus="4-7" memory="2" unit="GiB">
           <distances>
             <sibling id="0" value="21"/>
             <sibling id="1" value="10"/>
           </distances>
         </cell>
       </numa>
     </cpu>
   </domain>

2. Optional: Test the performance of the VM using the applicable tools and utilities to evaluate the impact of the VM’s optimization.

1. Install the ksmtuned service:

   # yum install ksmtuned

2. Start the service:

   • To enable KSM for a single session, use the systemctl utility to start the ksm and ksmtuned services.

     # systemctl start ksm
     # systemctl start ksmtuned

   • To enable KSM persistently, use the systemctl utility to enable the ksm and ksmtuned services.

     # systemctl enable ksm
     Created symlink /etc/systemd/system/multi-user.target.wants/ksm.service  /usr/lib/systemd/system/ksm.service
     
     # systemctl enable ksmtuned
     Created symlink /etc/systemd/system/multi-user.target.wants/ksmtuned.service  /usr/lib/systemd/system/ksmtuned.service