Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

第 4 章 Useful SystemTap Scripts

This chapter enumerates several SystemTap scripts you can use to monitor and investigate different subsystems. All of these scripts are available in the /usr/share/systemtap/testsuite/systemtap.examples/ directory once you install the systemtap-testsuite package.

4.1. Network
复制链接

The following sections showcase scripts that trace network-related functions and build a profile of network activity.

4.1.1. Network Profiling
复制链接

This section describes how to profile network activity. 例 4.1 “nettop.stp” provides a glimpse into how much network traffic each process is generating on a machine.

例 4.1. nettop.stp

#! /usr/bin/env stap

global ifxmit, ifrecv
global ifmerged

probe netdev.transmit
{
  ifxmit[pid(), dev_name, execname(), uid()] <<< length
}

probe netdev.receive
{
  ifrecv[pid(), dev_name, execname(), uid()] <<< length
}

function print_activity()
{
  printf("%5s %5s %-7s %7s %7s %7s %7s %-15s\n",
         "PID", "UID", "DEV", "XMIT_PK", "RECV_PK",
         "XMIT_KB", "RECV_KB", "COMMAND")

  foreach ([pid, dev, exec, uid] in ifrecv) {
	  ifmerged[pid, dev, exec, uid] += @count(ifrecv[pid,dev,exec,uid]);
  }
  foreach ([pid, dev, exec, uid] in ifxmit) {
	  ifmerged[pid, dev, exec, uid] += @count(ifxmit[pid,dev,exec,uid]);
  }
  foreach ([pid, dev, exec, uid] in ifmerged-) {
    n_xmit = @count(ifxmit[pid, dev, exec, uid])
    n_recv = @count(ifrecv[pid, dev, exec, uid])
    printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n",
           pid, uid, dev, n_xmit, n_recv,
           n_xmit ? @sum(ifxmit[pid, dev, exec, uid])/1024 : 0,
           n_recv ? @sum(ifrecv[pid, dev, exec, uid])/1024 : 0,
           exec)
  }

  print("\n")

  delete ifxmit
  delete ifrecv
  delete ifmerged
}

probe timer.ms(5000), end, error
{
  print_activity()
}
Copy to Clipboard Toggle word wrap
Note that the print_activity() function uses the following expressions: 
n_xmit ? @sum(ifxmit[pid, dev, exec, uid])/1024 : 0
n_recv ? @sum(ifrecv[pid, dev, exec, uid])/1024 : 0
Copy to Clipboard Toggle word wrap
These expressions are if or else conditionals. The second statement is simply a more concise way of writing the following pseudo code: 
if n_recv != 0 then
  @sum(ifrecv[pid, dev, exec, uid])/1024
else
  0
Copy to Clipboard Toggle word wrap
例 4.1 “nettop.stp” tracks which processes are generating network traffic on the system, and provides the following information about each process:
  • PID — the ID of the listed process.
  • UID — user ID. A user ID of 0 refers to the root user.
  • DEV — which ethernet device the process used to send or receive data (for example, eth0, eth1)
  • XMIT_PK — number of packets transmitted by the process
  • RECV_PK — number of packets received by the process
  • XMIT_KB — amount of data sent by the process, in kilobytes
  • RECV_KB — amount of data received by the service, in kilobytes
例 4.1 “nettop.stp” provides network profile sampling every 5 seconds. You can change this setting by editing probe timer.ms(5000) accordingly. 例 4.2 “例 4.1 “nettop.stp” Sample Output” contains an excerpt of the output from 例 4.1 “nettop.stp” over a 20-second period:

例 4.2. 例 4.1 “nettop.stp” Sample Output

[...]
  PID   UID DEV     XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND        
    0     0 eth0          0       5       0       0 swapper        
11178     0 eth0          2       0       0       0 synergyc       
  PID   UID DEV     XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND        
 2886     4 eth0         79       0       5       0 cups-polld     
11362     0 eth0          0      61       0       5 firefox        
    0     0 eth0          3      32       0       3 swapper        
 2886     4 lo            4       4       0       0 cups-polld     
11178     0 eth0          3       0       0       0 synergyc       
  PID   UID DEV     XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND        
    0     0 eth0          0       6       0       0 swapper        
 2886     4 lo            2       2       0       0 cups-polld     
11178     0 eth0          3       0       0       0 synergyc       
 3611     0 eth0          0       1       0       0 Xorg           
  PID   UID DEV     XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND        
    0     0 eth0          3      42       0       2 swapper        
11178     0 eth0         43       1       3       0 synergyc       
11362     0 eth0          0       7       0       0 firefox        
 3897     0 eth0          0       1       0       0 multiload-apple
[...]
Copy to Clipboard Toggle word wrap

4.1.2. Tracing Functions Called in Network Socket Code
复制链接

This section describes how to trace functions called from the kernel's net/socket.c file. This task helps you identify, in finer detail, how each process interacts with the network at the kernel level.

例 4.3. socket-trace.stp

#!/usr/bin/stap

probe kernel.function("*@net/socket.c").call {
  printf ("%s -> %s\n", thread_indent(1), probefunc())
}
probe kernel.function("*@net/socket.c").return {
  printf ("%s <- %s\n", thread_indent(-1), probefunc())
}
Copy to Clipboard Toggle word wrap
例 4.3 “socket-trace.stp” is identical to 例 3.6 “thread_indent.stp”, which was earlier used in SystemTap Functions to illustrate how thread_indent() works.

例 4.4. 例 4.3 “socket-trace.stp” Sample Output

[...]
0 Xorg(3611): -> sock_poll
3 Xorg(3611): <- sock_poll
0 Xorg(3611): -> sock_poll
3 Xorg(3611): <- sock_poll
0 gnome-terminal(11106): -> sock_poll
5 gnome-terminal(11106): <- sock_poll
0 scim-bridge(3883): -> sock_poll
3 scim-bridge(3883): <- sock_poll
0 scim-bridge(3883): -> sys_socketcall
4 scim-bridge(3883):  -> sys_recv
8 scim-bridge(3883):   -> sys_recvfrom
12 scim-bridge(3883):-> sock_from_file
16 scim-bridge(3883):<- sock_from_file
20 scim-bridge(3883):-> sock_recvmsg
24 scim-bridge(3883):<- sock_recvmsg
28 scim-bridge(3883):   <- sys_recvfrom
31 scim-bridge(3883):  <- sys_recv
35 scim-bridge(3883): <- sys_socketcall
[...]
Copy to Clipboard Toggle word wrap
例 4.4 “例 4.3 “socket-trace.stp” Sample Output” contains a 3-second excerpt of the output for 例 4.3 “socket-trace.stp”. For more information about the output of this script as provided by thread_indent(), see SystemTap Functions 例 3.6 “thread_indent.stp”.

4.1.3. Monitoring Incoming TCP Connections
复制链接

This section illustrates how to monitor incoming TCP connections. This task is useful in identifying any unauthorized, suspicious, or otherwise unwanted network access requests in real time.

例 4.5. tcp_connections.stp

#! /usr/bin/env stap

probe begin {
  printf("%6s %16s %6s %6s %16s\n",
         "UID", "CMD", "PID", "PORT", "IP_SOURCE")
}

probe kernel.function("tcp_accept").return?,
      kernel.function("inet_csk_accept").return? {
  sock = $return
  if (sock != 0)
    printf("%6d %16s %6d %6d %16s\n", uid(), execname(), pid(),
           inet_get_local_port(sock), inet_get_ip_source(sock))
}
Copy to Clipboard Toggle word wrap
While 例 4.5 “tcp_connections.stp” is running, it will print out the following information about any incoming TCP connections accepted by the system in real time:
  • Current UID
  • CMD - the command accepting the connection
  • PID of the command
  • Port used by the connection
  • IP address from which the TCP connection originated

例 4.6. 例 4.5 “tcp_connections.stp” Sample Output

UID            CMD    PID   PORT        IP_SOURCE
0             sshd   3165     22      10.64.0.227
0             sshd   3165     22      10.64.0.227
Copy to Clipboard Toggle word wrap

4.1.4. Monitoring Network Packets Drops in Kernel
复制链接

The network stack in Linux can discard packets for various reasons. Some Linux kernels include a tracepoint, kernel.trace("kfree_skb"), which easily tracks where packets are discarded. 例 4.7 “dropwatch.stp” uses kernel.trace("kfree_skb") to trace packet discards; the script summarizes which locations discard packets every five-second interval.

例 4.7. dropwatch.stp

#!/usr/bin/stap

############################################################
# Dropwatch.stp
# Author: Neil Horman <nhorman@redhat.com>
# An example script to mimic the behavior of the dropwatch utility
# http://fedorahosted.org/dropwatch
############################################################

# Array to hold the list of drop points we find
global locations

# Note when we turn the monitor on and off
probe begin { printf("Monitoring for dropped packets\n") }
probe end { printf("Stopping dropped packet monitor\n") }

# increment a drop counter for every location we drop at
probe kernel.trace("kfree_skb") { locations[$location] <<< 1 }

# Every 5 seconds report our drop locations
probe timer.sec(5)
{
	printf("\n")
	foreach (l in locations-) {
		printf("%d packets dropped at location %p\n",
			   @count(locations[l]), l)
	}
	delete locations
}
Copy to Clipboard Toggle word wrap
The kernel.trace("kfree_skb") traces which places in the kernel drop network packets. The kernel.trace("kfree_skb") has two arguments: a pointer to the buffer being freed ($skb) and the location in kernel code the buffer is being freed ($location).
Running the dropwatch.stp script 15 seconds would result in output similar in 例 4.8 “例 4.7 “dropwatch.stp” Sample Output”. The output lists the number of misses for tracepoint address and the actual address.

例 4.8. 例 4.7 “dropwatch.stp” Sample Output

Monitoring for dropped packets
51 packets dropped at location 0xffffffff8024cd0f
2 packets dropped at location 0xffffffff8044b472
51 packets dropped at location 0xffffffff8024cd0f
1 packets dropped at location 0xffffffff8044b472
97 packets dropped at location 0xffffffff8024cd0f
1 packets dropped at location 0xffffffff8044b472
Stopping dropped packet monitor
Copy to Clipboard Toggle word wrap
To make the location of packet drops more meaningful, see the /boot/System.map-$(uname -r) file. This file lists the starting addresses for each function, allowing you to map the addresses in the output of 例 4.8 “例 4.7 “dropwatch.stp” Sample Output” to a specific function name. Given the following snippet of the /boot/System.map-$(uname -r) file, the address 0xffffffff8024cd0f maps to the function unix_stream_recvmsg and the address 0xffffffff8044b472 maps to the function arp_rcv: 
[...]
ffffffff8024c5cd T unlock_new_inode
ffffffff8024c5da t unix_stream_sendmsg
ffffffff8024c920 t unix_stream_recvmsg
ffffffff8024cea1 t udp_v4_lookup_longway
[...]
ffffffff8044addc t arp_process
ffffffff8044b360 t arp_rcv
ffffffff8044b487 t parp_redo
ffffffff8044b48c t arp_solicit
[...]
Copy to Clipboard Toggle word wrap
返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat