Red Hat SummitChapter 20. Performance
The following chapter contains the most notable changes to performance between RHEL 8 and RHEL 9.
20.1. Notable changes to performance
Performance Co-Pilot rebased to version 6.0
Starting in RHEL 9.2, Performance Co-Pilot (PCP) has been updated to version 6.0. Notable improvements include:
Version 3 PCP archive support:
This includes support for instance domain change-deltas, Y2038-safe timestamps, nanosecond-precision timestamps, arbitrary timezones support, and 64-bit file offsets used throughout for larger (beyond 2GB) individual volumes.
This feature is currently opt-in via the
PCP_ARCHIVE_VERSIONsetting in the/etc/pcp.conffile.Version 2 archives remain the default.
Only OpenSSL is used throughout PCP. Mozilla NSS/NSPR use has been dropped:
This impacts
libpcp,PMAPIclients andPMCDuse of encryption. These elements are now configured and used consistently withpmproxyHTTPS support andredis-server, which were both already using OpenSSL.New nanosecond precision timestamp
PMAPIcalls forPCPlibrary interfaces that make use of timestamps.These are all optional, and full backward compatibility is preserved for existing tools.
The following tools and services have been updated:
pcp2elasticsearch- Implemented authentication support.
pcp-dstat-
Implemented support for the
top-alikeplugins. pcp-htop- Updated to the latest stable upstream release.
pmseries-
Added
sum,avg,stdev,nth_percentile,max_inst,max_sample,min_instandmin_samplefunctions. pmdabpf- Added CO-RE (Compile Once - Run Everywhere) modules and support for AMD64, Intel 64-bit, 64-bit ARM, and IBM Power Systems.
pmdabpftrace-
Moved example autostart scripts to the
/usr/sharedirectory. pmdadenki- Added support for multiple active batteries.
pmdalinux-
Updates for the latest
/proc/net/netstatchanges. pmdaopenvswitch- Added additional interface and coverage statistics.
pmproxy- Request parameters can now be sent in the request body.
pmieconf-
Added several
pmierules for Open vSwitch metrics. pmlogger_farm- Added a default configuration file for farm loggers.
pmlogger_daily_report- Some major efficiency improvements.
The sadf(1) tool now generates PCP archives from native sadc(1) archives
Starting in RHEL 9, the sadf(1) tool provided by the sysstat package can generate PCP archives from native sadc(1) archives.
By default, when the - flag is used with sadc(1), sadc(1) writes the data to the standard system activity daily data file. This file is named saDD and is located in the /var/log/sa directory by default. Conversely, when no input datafile is specified, the sadf(1) tool uses the standard system activity daily data file to generate archives. Pass numbers as an argument to tell sadf(1) to generate archives from data recorded that specified number of days in the past.
To generate a PCP archive from an
sadc(1)archive recorded 2 days ago, run:Copy to Clipboard Copied! Toggle word wrap Toggle overflow sadf -l -O pcparchive=/tmp/recording -2
# sadf -l -O pcparchive=/tmp/recording -2To show a list of metrics in the PCP archive generated from an
sadc(1)archive archive, run:Copy to Clipboard Copied! Toggle word wrap Toggle overflow pminfo --archive /tmp/recording
$ pminfo --archive /tmp/recording Disk.dev.avactive Disk.dev.read Disk.dev.write Disk.dev.blkread [...]To show the timespace of the archive and hostname of the PCP archive generated from an
sadc(1)archiveCopy to Clipboard Copied! Toggle word wrap Toggle overflow pmdumplog --label /tmp/recording
$ pmdumplog --label /tmp/recording Log Label (Log Format Version 2) Performance metrics from host shard commencing Tue Jul 20 00:10:30.642477 2021 ending Wed Jul 21 00:10:30.222176 2021You can then analyze the PCP archive generated from an
sadc(1)archive by using PCP commands, for example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow pmchart --archive /tmp/recording
$ pmchart --archive /tmp/recording
New PCP PMDA - pmdabpf
RHEL 9 is distributed with the pcp-pmda-bpf package, which provides the pmdabpf Performance Co-Pilot (PCP) Performance Metric Domain Agent (PMDA).
The pmdabpf PMDA extracts live performance data from eBPF programs utilizing BPF CO-RE (Compile Once - Run Everywhere), that is libbpf and BTF.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}