此内容没有您所选择的语言版本。

Chapter 17. Monitoring Red Hat Gluster Storage Gluster Workload


Monitoring storage volumes is helpful when conducting a capacity planning or performance tuning activity on a Red Hat Gluster Storage volume. You can monitor the Red Hat Gluster Storage volumes with different parameters and use those system outputs to identify and troubleshoot issues.
You can use the volume top and volume profile commands to view vital performance information and identify bottlenecks on each brick of a volume.
You can also perform a statedump of the brick processes and NFS server process of a volume, and also view volume status and volume information.

Note

If you restart the server process, the existing profile and top information will be reset.

17.1. Profiling volumes

17.1.1. Server-side volume profiling using volume profile

The volume profile command provides an interface to get the per-brick or NFS server I/O information for each file operation of a volume. This information helps in identifying the bottlenecks in the storage system.
This section describes how to use the volume profile command.

17.1.1.1. Start Profiling

To view the file operation information of each brick, start the profiling command:
# gluster volume profile VOLNAME start
For example, to start profiling on test-volume:
# gluster volume profile test-volume start
Starting volume profile on test has been successful
Copy to Clipboard Toggle word wrap

Important

Running profile command can affect system performance while the profile information is being collected. Red Hat recommends that profiling should only be used for debugging.
When profiling is started on the volume, the following additional options are displayed when using the volume info command:
diagnostics.count-fop-hits: on
diagnostics.latency-measurement: on
Copy to Clipboard Toggle word wrap

17.1.1.2. Displaying the I/O Information

To view the I/O information of the bricks on a volume, use the following command:
# gluster volume profile VOLNAME info
For example, to view the I/O information of test-volume:
# gluster v profile glustervol info
Brick: rhsqaci-vm33.lab.eng.blr.redhat.com:/bricks/brick0/1
-----------------------------------------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             11     RELEASE
      0.00       0.00 us       0.00 us       0.00 us            238  RELEASEDIR
      0.35     380.05 us     380.05 us     380.05 us              1    SETXATTR
      0.40     107.73 us       5.50 us     413.31 us              4     OPENDIR
      0.62     167.65 us      91.33 us     339.28 us              4      STATFS
      0.86     187.42 us      28.50 us     534.96 us              5    GETXATTR
      2.16     106.54 us      32.16 us     383.58 us             22     ENTRYLK
      2.17     106.97 us      39.01 us     251.65 us             22       FLUSH
      2.92     263.57 us     189.06 us     495.05 us             12     SETATTR
      3.22     124.60 us      43.08 us     311.69 us             28     INODELK
      3.41     616.76 us     319.27 us    1028.72 us              6     READDIR
     10.11     997.03 us     413.73 us    3507.02 us             11      CREATE
     73.79     256.58 us      50.02 us     924.61 us            312      LOOKUP

    Duration: 46537 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Interval 1 Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             11     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              4  RELEASEDIR
      0.35     380.05 us     380.05 us     380.05 us              1    SETXATTR
      0.40     107.73 us       5.50 us     413.31 us              4     OPENDIR
      0.62     167.65 us      91.33 us     339.28 us              4      STATFS
      0.86     187.42 us      28.50 us     534.96 us              5    GETXATTR
      2.16     106.54 us      32.16 us     383.58 us             22     ENTRYLK
      2.17     106.97 us      39.01 us     251.65 us             22       FLUSH
      2.92     263.57 us     189.06 us     495.05 us             12     SETATTR
      3.22     124.60 us      43.08 us     311.69 us             28     INODELK
      3.41     616.76 us     319.27 us    1028.72 us              6     READDIR
     10.11     997.03 us     413.73 us    3507.02 us             11      CREATE
     73.79     256.58 us      50.02 us     924.61 us            312      LOOKUP

    Duration: 347 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Brick: rhsqaci-vm33.lab.eng.blr.redhat.com:/bricks/brick1/1
-----------------------------------------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             12     RELEASE
      0.00       0.00 us       0.00 us       0.00 us            238  RELEASEDIR
      0.14     146.24 us     146.24 us     146.24 us              1        OPEN
      0.26     266.64 us     266.64 us     266.64 us              1    SETXATTR
      0.26      67.88 us       2.50 us     243.52 us              4     OPENDIR
      0.42     108.83 us      81.87 us     139.11 us              4      STATFS
      0.98     201.26 us      82.36 us     306.38 us              5    GETXATTR
      2.49     116.34 us      23.53 us     304.10 us             22     ENTRYLK
      2.75     236.13 us     124.73 us     358.80 us             12     SETATTR
      3.12     114.82 us      44.34 us     550.01 us             28     INODELK
      3.17     142.00 us      23.16 us     388.56 us             23       FLUSH
      4.37     748.73 us     324.70 us    1115.96 us              6     READDIR
      6.57     614.44 us     364.94 us     807.17 us             11      CREATE
     75.46     248.88 us      66.43 us     599.31 us            312      LOOKUP

    Duration: 46537 seconds
   Data Read: 0 bytes
Data Written: 0 bytes
Copy to Clipboard Toggle word wrap
To view the I/O information of the NFS server on a specified volume, use the following command:
# gluster volume profile VOLNAME info nfs
For example, to view the I/O information of the NFS server on test-volume:
# gluster volume profile test-volume info nfs
NFS Server : localhost
----------------------
Cumulative Stats:
Block Size:              32768b+               65536b+
No. of Reads:                    0                     0
No. of Writes:                 1000                  1000
%-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
---------   -----------   -----------   -----------   ------------        ----
0.01     410.33 us     194.00 us     641.00 us              3      STATFS
0.60     465.44 us     346.00 us     867.00 us            147       FSTAT
1.63     187.21 us      67.00 us    6081.00 us           1000     SETATTR
1.94     221.40 us      58.00 us   55399.00 us           1002      ACCESS
2.55     301.39 us      52.00 us   75922.00 us            968        STAT
2.85     326.18 us      88.00 us   66184.00 us           1000    TRUNCATE
4.47     511.89 us      60.00 us  101282.00 us           1000       FLUSH
5.02    3907.40 us    1723.00 us   19508.00 us            147    READDIRP
25.42    2876.37 us     101.00 us  843209.00 us           1012      LOOKUP
55.52    3179.16 us     124.00 us  121158.00 us           2000       WRITE

Duration: 7074 seconds
Data Read: 0 bytes
Data Written: 102400000 bytes

Interval 1 Stats:
Block Size:              32768b+               65536b+
No. of Reads:                    0                     0
No. of Writes:                 1000                  1000
%-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
---------   -----------   -----------   -----------   ------------        ----
0.01     410.33 us     194.00 us     641.00 us              3      STATFS
0.60     465.44 us     346.00 us     867.00 us            147       FSTAT
1.63     187.21 us      67.00 us    6081.00 us           1000     SETATTR
1.94     221.40 us      58.00 us   55399.00 us           1002      ACCESS
2.55     301.39 us      52.00 us   75922.00 us            968        STAT
2.85     326.18 us      88.00 us   66184.00 us           1000    TRUNCATE
4.47     511.89 us      60.00 us  101282.00 us           1000       FLUSH
5.02    3907.40 us    1723.00 us   19508.00 us            147    READDIRP
25.41    2878.07 us     101.00 us  843209.00 us           1011      LOOKUP
55.53    3179.16 us     124.00 us  121158.00 us           2000       WRITE

Duration: 330 seconds
Data Read: 0 bytes
Data Written: 102400000 bytes
Copy to Clipboard Toggle word wrap

17.1.1.3. Stop Profiling

To stop profiling on a volume, use the following command:
# gluster volume profile VOLNAME stop
For example, to stop profiling on test-volume:
# gluster volume profile test-volume stop
Stopping volume profile on test has been successful
Copy to Clipboard Toggle word wrap

17.1.2. Client-side volume profiling (FUSE only)

Red Hat Gluster Storage lets you profile how your mount point is being accessed, so that you can investigate latency issues even when you cannot instrument the application accessing your storage.
The io-stats translator records statistics of all file system activity on a Red Hat Gluster Storage volume that travels through a FUSE mount point. It collects information on files opened from the FUSE mount path, the read and write throughput for these files, the number of blocks read and written, and the latency observed for different file operations.
Run the following command to output all recorded statistics for the specified mount point to the specified output file.
# setfattr -n trusted.io-stats-dump -v output_file_id mount_point
Copy to Clipboard Toggle word wrap
This generates a number of files in the /var/run/gluster directory. The output_file_id is not the whole file name, but is used as part of the name of the generated files.
返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务,以及可以信赖的内容,帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情,请参阅红帽博客.

關於紅帽

我们提供强化的解决方案,使企业能够更轻松地跨平台和环境(从核心数据中心到网络边缘)工作。

Theme

© 2025 Red Hat