This documentation is for a release that is no longer maintained
See documentation for the latest supported version 3 or the latest supported version 4.Chapter 6. Troubleshooting the Red Hat build of OpenTelemetry
The OpenTelemetry Collector offers multiple ways to measure its health as well as investigate data ingestion issues.
6.1. Getting the OpenTelemetry Collector logs Copy linkLink copied to clipboard!
You can get the logs for the OpenTelemetry Collector as follows.
Procedure
Set the relevant log level in the
OpenTelemetryCollector
custom resource (CR):config: | service: telemetry: logs: level: debug
config: | service: telemetry: logs: level: debug
1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Collector’s log level. Supported values include
info
,warn
,error
, ordebug
. Defaults toinfo
.
-
Use the
oc logs
command or the web console to retrieve the logs.
6.2. Exposing the metrics Copy linkLink copied to clipboard!
The OpenTelemetry Collector exposes the metrics about the data volumes it has processed. The following metrics are for spans, although similar metrics are exposed for metrics and logs signals:
otelcol_receiver_accepted_spans
- The number of spans successfully pushed into the pipeline.
otelcol_receiver_refused_spans
- The number of spans that could not be pushed into the pipeline.
otelcol_exporter_sent_spans
- The number of spans successfully sent to the destination.
otelcol_exporter_enqueue_failed_spans
- The number of spans failed to be added to the sending queue.
The operator creates a <cr_name>-collector-monitoring
telemetry service that you can use to scrape the metrics endpoint.
Procedure
Enable the telemetry service by adding the following lines in the
OpenTelemetryCollector
custom resource:config: | service: telemetry: metrics: address: ":8888"
config: | service: telemetry: metrics: address: ":8888"
1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The address at which the internal collector metrics are exposed. Defaults to
:8888
.
Retrieve the metrics by running the following command, which uses the port-forwarding Collector pod:
oc port-forward <collector_pod>
$ oc port-forward <collector_pod>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Access the metrics endpoint at
http://localhost:8888/metrics
.
6.3. Debug exporter Copy linkLink copied to clipboard!
You can configure the debug exporter to export the collected data to the standard output.
Procedure
Configure the
OpenTelemetryCollector
custom resource as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Use the
oc logs
command or the web console to export the logs to the standard output.