Chapter 12. Network Observability CLI
12.1. Installing the Network Observability CLI
The Network Observability CLI (oc netobserv
) is deployed separately from the Network Observability Operator. The CLI is available as an OpenShift CLI (oc
) plugin. It provides a lightweight way to quickly debug and troubleshoot with network observability.
12.1.1. About the Network Observability CLI
You can quickly debug and troubleshoot networking issues by using the Network Observability CLI (oc netobserv
). The Network Observability CLI is a flow and packet visualization tool that relies on eBPF agents to stream collected data to an ephemeral collector pod. It requires no persistent storage during the capture. After the run, the output is transferred to your local machine. This enables quick, live insight into packets and flow data without installing the Network Observability Operator.
CLI capture is meant to run only for short durations, such as 8-10 minutes. If it runs for too long, it can be difficult to delete the running process.
12.1.2. Installing the Network Observability CLI
Installing the Network Observability CLI (oc netobserv
) is a separate procedure from the Network Observability Operator installation. This means that, even if you have the Operator installed from OperatorHub, you need to install the CLI separately.
You can optionally use Krew to install the netobserv
CLI plugin. For more information, see "Installing a CLI plugin with Krew".
Prerequisites
-
You must install the OpenShift CLI (
oc
). - You must have a macOS or Linux operating system.
Procedure
-
Download the
oc netobserv
CLI tar file that corresponds with your architecture. Unpack the archive. For example, for the
amd64
archive, run the following command:$ tar xvf netobserv-cli-linux-amd64.tar.gz
Make the file executable:
$ chmod +x ./oc-netobserv
Move the extracted
netobserv-cli
binary to a directory that is on yourPATH
, such as/usr/local/bin/
:$ sudo mv ./oc-netobserv /usr/local/bin/
Verification
Verify that
oc netobserv
is available:$ oc netobserv version
Example output
Netobserv CLI version <version>
Additional resources
12.2. Using the Network Observability CLI
You can visualize and filter the flows and packets data directly in the terminal to see specific usage, such as identifying who is using a specific port. The Network Observability CLI collects flows as JSON and database files or packets as a PCAP file, which you can use with third-party tools.
12.2.1. Capturing flows
You can capture flows and filter on any resource or zone in the data to solve use cases, such as displaying Round-Trip Time (RTT) between two zones. Table visualization in the CLI provides viewing and flow search capabilities.
Prerequisites
-
Install the OpenShift CLI (
oc
). -
Install the Network Observability CLI (
oc netobserv
) plugin.
Procedure
Capture flows with filters enabled by running the following command:
$ oc netobserv flows --enable_filter=true --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051
Add filters to the
live table filter
prompt in the terminal to further refine the incoming flows. For example:live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
- Use the PageUp and PageDown keys to toggle between None, Resource, Zone, Host, Owner and all of the above.
-
To stop capturing, press Ctrl+C. The data that was captured is written to two separate files in an
./output
directory located in the same path used to install the CLI. View the captured data in the
./output/flow/<capture_date_time>.json
JSON file, which contains JSON arrays of the captured data.Example JSON file
{ "AgentIP": "10.0.1.76", "Bytes": 561, "DnsErrno": 0, "Dscp": 20, "DstAddr": "f904:ece9:ba63:6ac7:8018:1e5:7130:0", "DstMac": "0A:58:0A:80:00:37", "DstPort": 9999, "Duplicate": false, "Etype": 2048, "Flags": 16, "FlowDirection": 0, "IfDirection": 0, "Interface": "ens5", "K8S_FlowLayer": "infra", "Packets": 1, "Proto": 6, "SrcAddr": "3e06:6c10:6440:2:a80:37:b756:270f", "SrcMac": "0A:58:0A:80:00:01", "SrcPort": 46934, "TimeFlowEndMs": 1709741962111, "TimeFlowRttNs": 121000, "TimeFlowStartMs": 1709741962111, "TimeReceived": 1709741964 }
You can use SQLite to inspect the
./output/flow/<capture_date_time>.db
database file. For example:Open the file by running the following command:
$ sqlite3 ./output/flow/<capture_date_time>.db
Query the data by running a SQLite
SELECT
statement, for example:sqlite> SELECT DnsLatencyMs, DnsFlagsResponseCode, DnsId, DstAddr, DstPort, Interface, Proto, SrcAddr, SrcPort, Bytes, Packets FROM flow WHERE DnsLatencyMs >10 LIMIT 10;
Example output
12|NoError|58747|10.128.0.63|57856||17|172.30.0.10|53|284|1 11|NoError|20486|10.128.0.52|56575||17|169.254.169.254|53|225|1 11|NoError|59544|10.128.0.103|51089||17|172.30.0.10|53|307|1 13|NoError|32519|10.128.0.52|55241||17|169.254.169.254|53|254|1 12|NoError|32519|10.0.0.3|55241||17|169.254.169.254|53|254|1 15|NoError|57673|10.128.0.19|59051||17|172.30.0.10|53|313|1 13|NoError|35652|10.0.0.3|46532||17|169.254.169.254|53|183|1 32|NoError|37326|10.0.0.3|52718||17|169.254.169.254|53|169|1 14|NoError|14530|10.0.0.3|58203||17|169.254.169.254|53|246|1 15|NoError|40548|10.0.0.3|45933||17|169.254.169.254|53|174|1
12.2.2. Capturing packets
You can capture packets using the Network Observability CLI.
Prerequisites
-
Install the OpenShift CLI (
oc
). -
Install the Network Observability CLI (
oc netobserv
) plugin.
Procedure
Run the packet capture with filters enabled:
$ oc netobserv packets --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051
Add filters to the
live table filter
prompt in the terminal to refine the incoming packets. An example filter is as follows:live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
- Use the PageUp and PageDown keys to toggle between None, Resource, Zone, Host, Owner and all of the above.
- To stop capturing, press Ctrl+C.
View the captured data, which is written to a single file in an
./output/pcap
directory located in the same path that was used to install the CLI:-
The
./output/pcap/<capture_date_time>.pcap
file can be opened with Wireshark.
-
The
12.2.3. Cleaning the Network Observability CLI
You can manually clean the CLI workload by running oc netobserv cleanup
. This command removes all the CLI components from your cluster.
When you end a capture, this command is run automatically by the client. You might be required to manually run it if you experience connectivity issues.
Procedure
Run the following command:
$ oc netobserv cleanup
Additional resources
12.3. Network Observability CLI (oc netobserv) reference
The Network Observability CLI (oc netobserv
) has most features and filtering options that are available for the Network Observability Operator. You can pass command line arguments to enable features or filtering options.
12.3.1. Network Observability CLI usage
You can use the Network Observability CLI (oc netobserv
) to pass command line arguments to capture flow data and packet data for further analysis, enable Network Observability Operator features, or pass configuration options to the eBPF agent and flowlogs-pipeline
.
12.3.1.1. Syntax
The basic syntax for oc netobserv
commands is as follows:
oc netobserv
syntax
$ oc netobserv [<command>] [<feature_option>] [<command_options>] 1
12.3.1.2. Basic commands
Command | Description |
---|---|
flows | Capture flows information. For subcommands, see the "Flows capture options" table. |
packets | Capture packets data. For subcommands, see the "Packets capture options" table. |
cleanup | Remove the Network Observability CLI components. |
version | Print the software version. |
help | Show help. |
12.3.1.3. Flows capture options
Flows capture has mandatory commands as well as additional options, such as enabling extra features about packet drops, DNS latencies, Round-trip time, and filtering.
oc netobserv flows
syntax
$ oc netobserv flows [<feature_option>] [<command_options>]
Option | Description | Default |
---|---|---|
--enable_pktdrop | enable packet drop | false |
--enable_dns | enable DNS tracking | false |
--enable_rtt | enable RTT tracking | false |
--enable_network_events | enable Network events monitoring | false |
--enable_filter | enable flow filter | false |
--log-level | components logs | info |
--max-time | maximum capture time | 5m |
--max-bytes | maximum capture bytes | 50000000 = 50MB |
--copy | copy the output files locally | prompt |
--direction | filter direction | n/a |
--cidr | filter CIDR | 0.0.0.0/0 |
--protocol | filter protocol | n/a |
--sport | filter source port | n/a |
--dport | filter destination port | n/a |
--port | filter port | n/a |
--sport_range | filter source port range | n/a |
--dport_range | filter destination port range | n/a |
--port_range | filter port range | n/a |
--sports | filter on either of two source ports | n/a |
--dports | filter on either of two destination ports | n/a |
--ports | filter on either of two ports | n/a |
--tcp_flags | filter TCP flags | n/a |
--action | filter action | Accept |
--icmp_type | filter ICMP type | n/a |
--icmp_code | filter ICMP code | n/a |
--peer_ip | filter peer IP | n/a |
--interfaces | interfaces to monitor | n/a |
Example running flows capture on TCP protocol and port 49051 with PacketDrop and RTT features enabled:
$ oc netobserv flows --enable_pktdrop=true --enable_rtt=true --enable_filter=true --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051
12.3.1.4. Packets capture options
You can filter on port and protocol for packet capture data.
oc netobserv packets
syntax
$ oc netobserv packets [<option>]
Option | Description | Default |
---|---|---|
--log-level | components logs | info |
--max-time | maximum capture time | 5m |
--max-bytes | maximum capture bytes | 50000000 = 50MB |
--copy | copy the output files locally | prompt |
--direction | filter direction | n/a |
--cidr | filter CIDR | 0.0.0.0/0 |
--protocol | filter protocol | n/a |
--sport | filter source port | n/a |
--dport | filter destination port | n/a |
--port | filter port | n/a |
--sport_range | filter source port range | n/a |
--dport_range | filter destination port range | n/a |
--port_range | filter port range | n/a |
--sports | filter on either of two source ports | n/a |
--dports | filter on either of two destination ports | n/a |
--ports | filter on either of two ports | n/a |
--tcp_flags | filter TCP flags | n/a |
--action | filter action | Accept |
--icmp_type | filter ICMP type | n/a |
--icmp_code | filter ICMP code | n/a |
--peer_ip | filter peer IP | n/a |
Example running packets capture on TCP protocol and port 49051:
$ oc netobserv packets --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051