Monitoring Satellite performance

Prerequisites

• Ensure you have the minimum of 20 GB space available in the /var/log/pcp directory.

  With the default PCP data retention settings, data storage is estimated to use between 100 MB and 500 MB of disk space per day, but may use up to several gigabytes over time. For more information, see Chapter 4, Metrics data retention.

Procedure

• Print a summary of the active PCP configuration:

  # pcp

  Copy to Clipboard Toggle word wrap

  Example output of the pcp command:

  Performance Co-Pilot configuration on satellite.example.com:
  
   platform: Linux satellite.example.com 4.18.0-372.32.1.el8_6.x86_64 #1 SMP Fri Oct 7 12:35:10 EDT 2022 x86_64
   hardware: 16 cpus, 2 disks, 1 node, 31895MB RAM
   timezone: UTC
   services: pmcd pmproxy
       pmcd: Version 5.3.7-17, 13 agents, 4 clients
       pmda: root pmcd proc pmproxy xfs redis linux apache mmv kvm
             postgresql jbd2 openmetrics
   pmlogger: primary logger: /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10
       pmie: primary engine: /var/log/pcp/pmie/satellite.example.com/pmie.log

  Copy to Clipboard Toggle word wrap

  In this example, both the Performance Metrics Collector Daemon (pmcd) and Performance Metrics Proxy Daemon (pmproxy) services are running. It also confirms the PMDA that are collecting metrics. Finally, it lists the active log file, in which pmlogger is currently storing metrics.

Procedure

• To view data storage statistics, enter the following command on your Satellite Server:

  # less /var/log/pcp/pmlogger/satellite.example.com/pmlogger.log

  Copy to Clipboard Toggle word wrap

• To list all metrics available through PCP, enter the following command:

  # pminfo

  Copy to Clipboard Toggle word wrap

• To list all Satellite metrics and their descriptions, enter the following command:

  # foreman-rake telemetry:metrics

  Copy to Clipboard Toggle word wrap

• To list the archived metrics, enter the following command:

  # less /var/log/pcp/pmlogger/satellite.example.com/pmlogger.log

  Copy to Clipboard Toggle word wrap

• The pmlogger daemon archives data as received, according to its configuration. To retrieve the name of the active log file, enter the following command:

  # pcp | grep logger

  Copy to Clipboard Toggle word wrap

  The output includes the file name of the active log file, for example:

   pmlogger: primary logger: /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10

  Copy to Clipboard Toggle word wrap

Procedures

• To print current values of a particular metric, enter the following command:

  # pmval -f 1 disk.partitions.write

  Copy to Clipboard Toggle word wrap

  In this example, metric instances on writes to disk partitions are displayed. PCP converts the number of writes to disk partitions from a counter value to a rate value. The -f 1 argument specifies to abbreviate the values to one decimal place.

  Example output:

  metric:    disk.partitions.write
  host:      satellite.example.com
  semantics: cumulative counter (converting to rate)
  units:     count (converting to count / sec)
  samples:   all
  
                   vda1                  vda2                   sr0
                    0.0                  12.0                   0.0
                    0.0                   1.0                   0.0
                    0.0                   1.0                   0.0
                    0.0                   2.0                   0.0

  Copy to Clipboard Toggle word wrap

• To print system performance summary every 2 seconds, enter the following command:

  # pmstat -t 2sec

  Copy to Clipboard Toggle word wrap

Examples

• To list all metrics that were enabled when the archive file was created, enter the following command:

  # pminfo --archive archive_file

  Copy to Clipboard Toggle word wrap

• To view the host and time period covered by an archive file, enter the following command:

  # pmdumplog -l archive_file

  Copy to Clipboard Toggle word wrap

• To list disk writes for each partition, over the time period covered by the archive file:

  # pmval --archive /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10 \
    -f 1 disk.partitions.write

  Copy to Clipboard Toggle word wrap

• To list disk write operations per partition, with a 2-second interval, over the time period between 14:00 and 14:15:

  # pmval --archive /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10 \
    -d -t 2sec \
    -f 3 disk.partitions.write \
    -S @14:00 -T @14:15

  Copy to Clipboard Toggle word wrap

• To list average values of all performance metrics, including the time and value of the minimum/maximum, over the time period between 14:00 and 14:30, and format the values as a table:

  # pmlogsummary /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10 \
    -HlfiImM \
    -S @14:00 \
    -T @14:30 \
    disk.partitions.write \
    mem.freemem

  Copy to Clipboard Toggle word wrap

• To display system metrics stored in an archive, starting from 14:00, in an interactive manner similar to the top tool:

  # pcp --archive /var/log/pcp/pmlogger/satellite.example.com/20230831.00.10 \
    -S @14:00 \
    atop

  Copy to Clipboard Toggle word wrap

Procedure

• Open the following URL in a browser: http://satellite.example.com:3000