SupportConsoleDevelopersStart a trialContact

Chapter 12. Network Observability CLI

12.1. Installing the Network Observability CLI

The Network Observability CLI (oc netobserv) is deployed separately from the Network Observability Operator. The CLI is available as an OpenShift CLI (oc) plugin. It provides a lightweight way to quickly debug and troubleshoot with network observability.

12.1.1. About the Network Observability CLI

You can quickly debug and troubleshoot networking issues by using the Network Observability CLI (oc netobserv). The Network Observability CLI is a flow and packet visualization tool that relies on eBPF agents to stream collected data to an ephemeral collector pod. It requires no persistent storage during the capture. After the run, the output is transferred to your local machine. This enables quick, live insight into packets and flow data without installing the Network Observability Operator.

Important

CLI capture is meant to run only for short durations, such as 8-10 minutes. If it runs for too long, it can be difficult to delete the running process.

12.1.2. Installing the Network Observability CLI

Installing the Network Observability CLI (oc netobserv) is a separate procedure from the Network Observability Operator installation. This means that, even if you have the Operator installed from OperatorHub, you need to install the CLI separately.

Note

You can optionally use Krew to install the netobserv CLI plugin. For more information, see "Installing a CLI plugin with Krew".

Prerequisites

  • You must install the OpenShift CLI (oc).
  • You must have a macOS or Linux operating system.

Procedure

  1. Download the oc netobserv CLI tar file that corresponds with your architecture.

  2. Unpack the archive. For example, for the amd64 archive, run the following command: 

    $ tar xvf netobserv-cli-linux-amd64.tar.gz

  3. Make the file executable: 

    $ chmod +x ./oc-netobserv

  4. Move the extracted netobserv-cli binary to a directory that is on your PATH, such as /usr/local/bin/: 

    $ sudo mv ./oc-netobserv /usr/local/bin/

Verification

  • Verify that oc netobserv is available: 

    $ oc netobserv version

    Example output

    Netobserv CLI version <version>

Additional resources

12.2. Using the Network Observability CLI

You can visualize and filter the flows and packets data directly in the terminal to see specific usage, such as identifying who is using a specific port. The Network Observability CLI collects flows as JSON and database files or packets as a PCAP file, which you can use with third-party tools.

12.2.1. Capturing flows

You can capture flows and filter on any resource or zone in the data to solve use cases, such as displaying Round-Trip Time (RTT) between two zones. Table visualization in the CLI provides viewing and flow search capabilities.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Install the Network Observability CLI (oc netobserv) plugin.

Procedure

  1. Capture flows with filters enabled by running the following command: 

    $ oc netobserv flows --enable_filter=true --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051

  2. Add filters to the live table filter prompt in the terminal to further refine the incoming flows. For example: 

    live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
  3. Use the PageUp and PageDown keys to toggle between None, Resource, Zone, Host, Owner and all of the above.
  4. To stop capturing, press Ctrl+C. The data that was captured is written to two separate files in an ./output directory located in the same path used to install the CLI.

  5. View the captured data in the ./output/flow/<capture_date_time>.json JSON file, which contains JSON arrays of the captured data.

    Example JSON file

    {
  "AgentIP": "10.0.1.76",
  "Bytes": 561,
  "DnsErrno": 0,
  "Dscp": 20,
  "DstAddr": "f904:ece9:ba63:6ac7:8018:1e5:7130:0",
  "DstMac": "0A:58:0A:80:00:37",
  "DstPort": 9999,
  "Duplicate": false,
  "Etype": 2048,
  "Flags": 16,
  "FlowDirection": 0,
  "IfDirection": 0,
  "Interface": "ens5",
  "K8S_FlowLayer": "infra",
  "Packets": 1,
  "Proto": 6,
  "SrcAddr": "3e06:6c10:6440:2:a80:37:b756:270f",
  "SrcMac": "0A:58:0A:80:00:01",
  "SrcPort": 46934,
  "TimeFlowEndMs": 1709741962111,
  "TimeFlowRttNs": 121000,
  "TimeFlowStartMs": 1709741962111,
  "TimeReceived": 1709741964
}

  6. You can use SQLite to inspect the ./output/flow/<capture_date_time>.db database file. For example:

    1. Open the file by running the following command: 

      $ sqlite3 ./output/flow/<capture_date_time>.db

    2. Query the data by running a SQLite SELECT statement, for example: 

      sqlite> SELECT DnsLatencyMs, DnsFlagsResponseCode, DnsId, DstAddr, DstPort, Interface, Proto, SrcAddr, SrcPort, Bytes, Packets FROM flow WHERE DnsLatencyMs >10 LIMIT 10;

      Example output

      12|NoError|58747|10.128.0.63|57856||17|172.30.0.10|53|284|1
11|NoError|20486|10.128.0.52|56575||17|169.254.169.254|53|225|1
11|NoError|59544|10.128.0.103|51089||17|172.30.0.10|53|307|1
13|NoError|32519|10.128.0.52|55241||17|169.254.169.254|53|254|1
12|NoError|32519|10.0.0.3|55241||17|169.254.169.254|53|254|1
15|NoError|57673|10.128.0.19|59051||17|172.30.0.10|53|313|1
13|NoError|35652|10.0.0.3|46532||17|169.254.169.254|53|183|1
32|NoError|37326|10.0.0.3|52718||17|169.254.169.254|53|169|1
14|NoError|14530|10.0.0.3|58203||17|169.254.169.254|53|246|1
15|NoError|40548|10.0.0.3|45933||17|169.254.169.254|53|174|1

12.2.2. Capturing packets

You can capture packets using the Network Observability CLI.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Install the Network Observability CLI (oc netobserv) plugin.

Procedure

  1. Run the packet capture with filters enabled: 

    $ oc netobserv packets --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051

  2. Add filters to the live table filter prompt in the terminal to refine the incoming packets. An example filter is as follows: 

    live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
  3. Use the PageUp and PageDown keys to toggle between None, Resource, Zone, Host, Owner and all of the above.
  4. To stop capturing, press Ctrl+C.

  5. View the captured data, which is written to a single file in an ./output/pcap directory located in the same path that was used to install the CLI:

    1. The ./output/pcap/<capture_date_time>.pcap file can be opened with Wireshark.

12.2.3. Cleaning the Network Observability CLI

You can manually clean the CLI workload by running oc netobserv cleanup. This command removes all the CLI components from your cluster.

When you end a capture, this command is run automatically by the client. You might be required to manually run it if you experience connectivity issues.

Procedure

  • Run the following command: 

    $ oc netobserv cleanup

Additional resources

12.3. Network Observability CLI (oc netobserv) reference

The Network Observability CLI (oc netobserv) has most features and filtering options that are available for the Network Observability Operator. You can pass command line arguments to enable features or filtering options.

12.3.1. Network Observability CLI usage

You can use the Network Observability CLI (oc netobserv) to pass command line arguments to capture flow data and packet data for further analysis, enable Network Observability Operator features, or pass configuration options to the eBPF agent and flowlogs-pipeline.

12.3.1.1. Syntax

The basic syntax for oc netobserv commands is as follows:

oc netobserv syntax

$ oc netobserv [<command>] [<feature_option>] [<command_options>] 1

1 1
Feature options can only be used with the oc netobserv flows command. They cannot be used with the oc netobserv packets command.

12.3.1.2. Basic commands

Table 12.1. Basic commands
CommandDescription

flows

Capture flows information. For subcommands, see the "Flows capture options" table.

packets

Capture packets data. For subcommands, see the "Packets capture options" table.

cleanup

Remove the Network Observability CLI components.

version

Print the software version.

help

Show help.

12.3.1.3. Flows capture options

Flows capture has mandatory commands as well as additional options, such as enabling extra features about packet drops, DNS latencies, Round-trip time, and filtering.

oc netobserv flows syntax

$ oc netobserv flows [<feature_option>] [<command_options>]

OptionDescriptionDefault

--enable_pktdrop

enable packet drop

false

--enable_dns

enable DNS tracking

false

--enable_rtt

enable RTT tracking

false

--enable_network_events

enable Network events monitoring

false

--enable_filter

enable flow filter

false

--log-level

components logs

info

--max-time

maximum capture time

5m

--max-bytes

maximum capture bytes

50000000 = 50MB

--copy

copy the output files locally

prompt

--direction

filter direction

n/a

--cidr

filter CIDR

0.0.0.0/0

--protocol

filter protocol

n/a

--sport

filter source port

n/a

--dport

filter destination port

n/a

--port

filter port

n/a

--sport_range

filter source port range

n/a

--dport_range

filter destination port range

n/a

--port_range

filter port range

n/a

--sports

filter on either of two source ports

n/a

--dports

filter on either of two destination ports

n/a

--ports

filter on either of two ports

n/a

--tcp_flags

filter TCP flags

n/a

--action

filter action

Accept

--icmp_type

filter ICMP type

n/a

--icmp_code

filter ICMP code

n/a

--peer_ip

filter peer IP

n/a

--interfaces

interfaces to monitor

n/a

Example running flows capture on TCP protocol and port 49051 with PacketDrop and RTT features enabled:

$ oc netobserv flows --enable_pktdrop=true  --enable_rtt=true --enable_filter=true --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051

12.3.1.4. Packets capture options

You can filter on port and protocol for packet capture data.

oc netobserv packets syntax

$ oc netobserv packets [<option>]

OptionDescriptionDefault

--log-level

components logs

info

--max-time

maximum capture time

5m

--max-bytes

maximum capture bytes

50000000 = 50MB

--copy

copy the output files locally

prompt

--direction

filter direction

n/a

--cidr

filter CIDR

0.0.0.0/0

--protocol

filter protocol

n/a

--sport

filter source port

n/a

--dport

filter destination port

n/a

--port

filter port

n/a

--sport_range

filter source port range

n/a

--dport_range

filter destination port range

n/a

--port_range

filter port range

n/a

--sports

filter on either of two source ports

n/a

--dports

filter on either of two destination ports

n/a

--ports

filter on either of two ports

n/a

--tcp_flags

filter TCP flags

n/a

--action

filter action

Accept

--icmp_type

filter ICMP type

n/a

--icmp_code

filter ICMP code

n/a

--peer_ip

filter peer IP

n/a

Example running packets capture on TCP protocol and port 49051:

$ oc netobserv packets --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.