Chapter 13. Recording and analyzing performance profiles with perf
The perf
tool allows you to record performance data and analyze it at a later time.
13.1. The purpose of perf record Copy linkLink copied to clipboard!
The perf record
command samples performance data and stores it in a perf.data
file. You can then read and visualize data with other perf commands. perf.data
is generated in the current directory and can be accessed at a later time, possibly on a different machine. You can profile the entire system by running perf record -a
. This records performance data until you press Ctrl+C or for the duration of a specific command. Similar to that, you can profile a single application/process, either by running perf record ./your_app
or by attaching to an already running one by using perf record -p PID
.
13.2. Recording a performance profile Copy linkLink copied to clipboard!
You can use perf record
to sample and record performance data. The level of data captured depends on the access level:
-
Without root access:
perf record
collects data only in the user space. -
With root access:
perf record
collects data in both the user and kernel space.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf. - You have root access if you want to collect data for user and kernel space.
Procedure
Do one of the following:
To sample and record the performance data for the user space:
perf record command
$ perf record command
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To sample and record the performance data for kernel space (use root or sudo access):
perf record command
# perf record command
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace command with the actual command you want to profile. If you do not specify a command, then
perf record
samples data until you manually stop it by pressing Ctrl+C.
13.3. Recording a performance profile in per-CPU mode Copy linkLink copied to clipboard!
You can use perf record
in per-CPU mode to sample and record performance data in both user-space and the kernel-space simultaneously across all threads on a monitored CPU. By default, per-CPU mode monitors all online CPUs.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf.
Procedure
Sample and record the performance data:
perf record -a command
# perf record -a command
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace command with the command you want to sample data during. If you do not specify a command, then
perf record
samples data until you manually stop it by pressing Ctrl+C.
13.4. Capturing call graph data with perf record Copy linkLink copied to clipboard!
You can configure the perf record
tool so that it records which function calls other functions in the performance profile. This helps to identify a bottleneck if several processes are calling the same function.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf.
Procedure
Sample and record performance data with the
--call-graph
option:perf record --call-graph method command
$ perf record --call-graph method command
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace command with the command you want to sample data during. If you do not specify a command, then
perf record
samples data until you manually stop it by pressing Ctrl+C. Replace method with one of the following unwinding methods:
- fp
-
Uses the frame pointer method. Depending on compiler optimization, such as with binaries built with the GCC option
--fomit-frame-pointer
, this may not be able to unwind the stack. - dwarf
- Uses DWARF Call Frame Information to unwind the stack.
- lbr
- Uses the last branch record hardware on Intel processors.
-
Replace command with the command you want to sample data during. If you do not specify a command, then
13.5. Analyzing perf.data with perf report Copy linkLink copied to clipboard!
You can use perf report
to display and analyze data from the perf.data
file.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf. -
There is a
perf.data
file in the current directory. -
If the
perf.data
file was created with root access, you need to run perf report with root access too.
Procedure
Display the contents of the perf.data file for further analysis:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For more information, see the
perf-record(1)
man page on your system.
13.6. Attaching perf record to a running process Copy linkLink copied to clipboard!
You can attach perf
record to a running process. This instructs perf
record to only sample and record performance data in the specified processes.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf.
Procedure
Attach
perf record
to a running process:perf record -p ID1,ID2 sleep seconds
$ perf record -p ID1,ID2 sleep seconds
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This command samples and records performance data of the processes with the process ID’s
ID1
andID2
for a time period ofseconds
seconds as dictated by using the sleep command. You can also configureperf
to record events in specific threads:perf record -t ID1,ID2 sleep seconds
$ perf record -t ID1,ID2 sleep seconds
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteWhen using the
-t
flag and stipulating thread ID’s,perf
disables inheritance by default. You can enable inheritance by adding the--inherit
option.
13.7. Interpretation of perf report output Copy linkLink copied to clipboard!
The table displayed by running the perf report
command sorts the data into the following several columns:
- Overhead
- Indicates what percentage of overall samples were collected in that particular function.
- Command
- Tells you which process the samples were collected from.
- Shared Object
- Displays the name of the ELF image where the samples come from (the name [kernel.kallsyms] is used when the samples come from the kernel).
- Symbol
- Displays the function name or symbol. In default mode, the functions are sorted in descending order with those with the highest overhead displayed first.
13.8. Generating a perf.data file readable on a different device Copy linkLink copied to clipboard!
You can use the perf
tool to record performance data into a perf.data
file to be analyzed on a different device.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf. -
The kernel
debuginfo
package is installed. For more information, see Getting debuginfo packages for an application or library using GDB.
Procedure
Capture performance data you are interested in investigating further:
perf record -a --call-graph fp sleep <seconds>
# perf record -a --call-graph fp sleep <seconds>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This example generates a
perf.data
over the entire system for a period of seconds seconds as dictated by the use of the sleep command. It would also capture call graph data using the frame pointer method.Generate an archive file containing debug symbols of the recorded data:
perf archive
# perf archive
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Verify that the archive file has been generated in your current active directory:
ls perf.data*
# ls perf.data*
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The output will display every file in your current directory that begins with
perf.data
. The archive file will be named either:perf.data.tar.gz
orperf.data.tar.bz2
.
13.9. Analyzing a perf.data file from a different device Copy linkLink copied to clipboard!
You can use the perf
tool to analyze a perf.data
file that was generated on a different device.
Prerequisites
-
You have the
perf
user space tool installed. For more information, see Installing perf. -
A
perf.data
file and associated archive file generated on a different device are present on the current device being used.
Procedure
-
Copy both the
perf.data
file and the archive file into your current active directory. Extract the archive file into
~/.debug
:mkdir -p ~/.debug tar xf perf.data.tar.bz2 -C ~/.debug
# mkdir -p ~/.debug # tar xf perf.data.tar.bz2 -C ~/.debug
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe archive file might also be named
perf.data.tar.gz
.Open the
perf.data
file for further analysis:perf report
# perf report
Copy to Clipboard Copied! Toggle word wrap Toggle overflow