Red Hat Summit Red Hat SummitSupportConsoleDéveloppeursCommencer un essaiContact

Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 2. Observing MCP gateway connections

You can use observability in MCP gateway to debug things such as silent failures, track destructive annotations, track latencies, and make an audit trail.

The MCP gateway uses Red Hat build of OpenTelemetry throughout all components to give you consistent logging, distributed tracing, and metrics. By using observability, you can achieve the following goals:

  • Discover which component generated an error, such as Envoy, an MCP gateway component, or a backend MCP server.
  • Understand how requests flow through the system components.
  • See which tools were called and which MCP servers executed those tools.
  • Understand where to find relevant logs for each component.
  • Examine metrics about tool call patterns, success rates, and error rates.
  • Examine distributed tracing across the request path.

Prerequisites

  • You installed MCP gateway.
  • You installed Connectivity Link.
  • You configured a Gateway object.
  • You are logged into a running OpenShift Container Platform cluster with an admin role.
  • You installed Red Hat build of OpenTelemetry.
  • You configured the Red Hat OpenShift Distributed Tracing Platform for traces.
  • You installed the OpenShift Logging Operator (Loki).
  • You configured an S3-compatible persistent storage volume for Loki to store long-term MCP tool call logs.

Procedure

  1. Use the following example YAML to create a collector in your MCP gateway deployment namespace:

    Example OpenTelemetryCollector custom resource (CR)

    apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  name: <mcp_otel_collector>
  namespace: <mcp_gateway_system>
spec:
  mode: deployment
  config:
    receivers:
      otlp:
        protocols:
          http:
            endpoint: 0.0.0.0:4318
          grpc:
            endpoint: 0.0.0.0:4317
    exporters:
      otlp/tempo:
        endpoint: tempo-gateway-http.openshift-tracing.svc.cluster.local:4317
        tls:
          insecure: true
      debug:
        verbosity: basic
    service:
      pipelines:
        traces:
          receivers: [otlp]
          exporters: [otlp/tempo, debug]
        logs:
          receivers: [otlp]
          exporters: [debug]

    • Replace the value of metadata.name: with the name you want to assign to this collector.
    • Replace the value of metadata.namespace: with the namespace of your MCP gateway deployment.
    • The spec.config.exporters: field value points to distributed tracing.
    • The setting, spec.config.exporters.otlp/tempo.tls.insecure: true is for internal cluster communication without TLS.

  2. Apply the following environment variables to your OpenShift Container Platform cluster by running the following command: 

    $ oc set env deployment/mcp-gateway \
  OTEL_EXPORTER_OTLP_ENDPOINT="http://<mcp_otel_collector>-collector.<mcp_gateway_system>.svc.cluster.local:4318" \
  OTEL_EXPORTER_OTLP_INSECURE="true" \
  OTEL_SERVICE_NAME="mcp-gateway"
    • Replace <mcp_otel_collector> with the name you used in your OpenTelemetryCollector CR. When Red Hat build of OpenTelemetry creates a collector, the service name is [metadata.name]-collector.
    • Replace <mcp_gateway_system> with the namespace of your MCP gateway deployment.

  3. Optional. If you want to send traces to one collector and logs to a different one, set the following additional environment variables: 

    $ oc set env deployment/mcp-gateway \
  OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://trace-collector.tracing-namespace.svc:4317" \
  OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="http://log-collector.logging-namespace.svc:4317"
    • Set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT to override the endpoint for traces.
    • Set OTEL_EXPORTER_OTLP_LOGS_ENDPOINT to override the endpoint for logs.
  4. If you are using Red Hat OpenShift Distributed Tracing Platform, select your stack name as the data source in the OpenShift Web Console Observe > Traces dashboard. For example, tempo-mcp.

  5. After you select the stack name, you can consult an attribute in your OpenShift Web Console OpenShift Observe > Traces dashboard to find where errors have occurred or to gather information for trend identification.

    • You can also filter by service.name="mcp-gateway".
  6. When using Loki, use the Observe > Logs view and toggle the Structured view to filter by mcp.session.id.
  7. When using Loki, use the Trace timeline view, look for the Log icon next to spans to jump directly to the relevant logs.

Troubleshooting

  • If you do not see any traces, check if the NetworkPolicy CR in your namespace allows traffic from the MCP gateway to the Red Hat build of OpenTelemetry collector service.

2.1.1. MCP gateway router spans for observability
Copier lien

When enabled, the MCP router, ext_proc, emits trace spans for every request and can export structured logs through Red Hat build of OpenTelemetry.

With traces, you can see one continuous timeline of traffic events across different pods and services. This can help you identify bottlenecks and conduct root-cause analysis.

Spans show one part of the journey. The attributes attached to those spans give you the contextual metadata that you need to have searchable and meaningful traces. The attributes are simple key: value pairs attached to each span.

The MCP gateway uses the following router spans:

Expand
Table 2.1. MCP router spans
SpanWhenDescription

mcp-router.process

Every ext_proc stream

Root span. Starts when request headers arrive, ends after response headers are processed.

mcp-router.route-decision

The request body is parsed

Routing decision: tool-call versus pass-through to broker.

mcp-router.broker-passthrough

Non-tool-call requests

Pass-through to broker: initialize, tools/list, notifications. No child spans means the broker is not instrumented.

mcp-router.tool-call

tools/call requests

Full tool call handling including session and server resolution.

mcp-router.broker.get-server-info

Inside tool-call

Call out to broker to resolve which backend server owns the tool.

mcp-router.session-cache.get

Inside tool-call

Call out to session cache to look up an existing backend session.

mcp-router.session-init

Cache miss during tool-call

Hairpin initialize request through the gateway to the backend MCP server.

mcp-router.session-cache.store

After session-init

Call out to session cache to store the new backend session.

The attributes in the following tables use OpenTelemetry MCP Semantic Conventions.

Expand
Table 2.2. MCP router root span, mcp-router.process, attributes
AttributeSourceDescription

http.method

:method header

HTTP method, POST

http.path

:path header

Request path, /mcp

http.request_id

x-request-id header

Envoy request ID

mcp.method.name

JSON-RPC method field

MCP method, initialize, tools/call, tools/list, and so on

gen_ai.tool.name

JSON-RPC params.name

Tool name, only for tools/call

jsonrpc.request.id

JSON-RPC id field

JSON-RPC request ID

jsonrpc.protocol.version

JSON-RPC jsonrpc field

Always "2.0"

gen_ai.operation.name

JSON-RPC method field

Same as mcp.method.name

mcp.session.id

mcp-session-id header

Gateway session ID

client.address

x-forwarded-for header

Client IP address

http.status_code

:status response header

Response status code

Expand
Table 2.3. MCP route decision span, mcp-router.route-decision, attributes
AttributeDescription

mcp.method.name

MCP method

mcp.route

Routing decision: tool-call, broker, or elicitation-response

Expand
Table 2.4. MCP tool call span, mcp-router.tool-call, attributes
AttributeDescription

gen_ai.tool.name

Tool name from the request

mcp.session.id

Gateway session ID

mcp.server

Resolved backend server name

mcp.server.hostname

Resolved backend server hostname

Expand
Table 2.5. MCP error attributes
AttributeDescription

error.type

Error classification, such as tool_not_found, missing_tool_name, invalid_session, session_cache_error, session_init_error, marshal_error, path_parse_error

error_source

Component that generated the error, such as`ext-proc`

http.status_code

HTTP status code returned

When there is an error, spans include these attributes.

Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Legal Notice

Theme

© 2026 Red HatRetour au début