转发遥测数据
将追踪、日志和指标导出到可观察性后端和云平台
摘要
第 1 章 转发遥测数据 复制链接链接已复制到粘贴板!
您可以使用 OpenTelemetry Collector 来转发您的遥测数据。
1.1. 将 trace 转发到 TempoStack 实例 复制链接链接已复制到粘贴板!
要配置转发追踪到 TempoStack 实例,您可以部署和配置 OpenTelemetry Collector。您可以使用指定的处理器、接收器和导出器在部署模式中部署 OpenTelemetry Collector。有关其他模式,请参阅附加资源中的 OpenTelemetry Collector 文档链接。
先决条件
- 已安装红帽构建的 OpenTelemetry Operator。
- 已安装 Tempo Operator。
- 在集群中部署了 TempoStack 实例。
流程
为 OpenTelemetry Collector 创建服务帐户。
ServiceAccount 示例
apiVersion: v1 kind: ServiceAccount metadata: name: otel-collector-deployment为服务帐户创建集群角色。
ClusterRole 示例
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: otel-collector rules: - apiGroups: [""] resources: ["pods", "namespaces",] verbs: ["get", "watch", "list"]1 - apiGroups: ["apps"] resources: ["replicasets"] verbs: ["get", "watch", "list"]2 - apiGroups: ["config.openshift.io"] resources: ["infrastructures", "infrastructures/status"] verbs: ["get", "watch", "list"]3 将集群角色绑定到服务帐户。
ClusterRoleBinding 示例
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: otel-collector subjects: - kind: ServiceAccount name: otel-collector-deployment namespace: otel-collector-example roleRef: kind: ClusterRole name: otel-collector apiGroup: rbac.authorization.k8s.io创建 YAML 文件以定义
OpenTelemetryCollector自定义资源(CR)。OpenTelemetryCollector 示例
apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: otel spec: mode: deployment serviceAccount: otel-collector-deployment config: receivers: jaeger: protocols: grpc: {} thrift_binary: {} thrift_compact: {} thrift_http: {} opencensus: {} otlp: protocols: grpc: {} http: {} zipkin: {} processors: batch: {} k8sattributes: {} memory_limiter: check_interval: 1s limit_percentage: 50 spike_limit_percentage: 30 resourcedetection: detectors: [openshift] exporters: otlp: endpoint: "tempo-simplest-distributor:4317"1 tls: insecure: true service: pipelines: traces: receivers: [jaeger, opencensus, otlp, zipkin]2 processors: [memory_limiter, k8sattributes, resourcedetection, batch] exporters: [otlp]
您可以将 telemetrygen 部署为测试:
apiVersion: batch/v1
kind: Job
metadata:
name: telemetrygen
spec:
template:
spec:
containers:
- name: telemetrygen
image: ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:latest
args:
- traces
- --otlp-endpoint=otel-collector:4317
- --otlp-insecure
- --duration=30s
- --workers=1
restartPolicy: Never
backoffLimit: 4
1.2. 将日志转发到 LokiStack 实例 复制链接链接已复制到粘贴板!
您可以使用 openshift-logging 租户模式部署 OpenTelemetry Collector 将日志转发到 LokiStack 实例。
先决条件
- 已安装红帽构建的 OpenTelemetry Operator。
- 已安装 Loki Operator。
-
在集群中部署了受支持的
LokiStack实例。有关支持的LokiStack配置的更多信息,请参阅 Logging。
流程
为 OpenTelemetry Collector 创建服务帐户。
ServiceAccount对象示例apiVersion: v1 kind: ServiceAccount metadata: name: otel-collector-deployment namespace: openshift-logging创建一个集群角色,为 Collector 的服务帐户授予将日志推送到
LokiStack应用程序租户的权限。ClusterRole对象示例apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: otel-collector-logs-writer rules: - apiGroups: ["loki.grafana.com"] resourceNames: ["logs"] resources: ["application"] verbs: ["create"] - apiGroups: [""] resources: ["pods", "namespaces", "nodes"] verbs: ["get", "watch", "list"] - apiGroups: ["apps"] resources: ["replicasets"] verbs: ["get", "list", "watch"] - apiGroups: ["extensions"] resources: ["replicasets"] verbs: ["get", "list", "watch"]将集群角色绑定到服务帐户。
ClusterRoleBinding对象示例apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: otel-collector-logs-writer roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: otel-collector-logs-writer subjects: - kind: ServiceAccount name: otel-collector-deployment namespace: openshift-logging创建
OpenTelemetryCollector自定义资源 (CR) 对象。OpenTelemetryCollectorCR 对象示例apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: otel namespace: openshift-logging spec: serviceAccount: otel-collector-deployment config: extensions: bearertokenauth: filename: "/var/run/secrets/kubernetes.io/serviceaccount/token" receivers: otlp: protocols: grpc: {} http: {} processors: k8sattributes: {} resource: attributes:1 - key: kubernetes.namespace_name from_attribute: k8s.namespace.name action: upsert - key: kubernetes.pod_name from_attribute: k8s.pod.name action: upsert - key: kubernetes.container_name from_attribute: k8s.container.name action: upsert - key: log_type value: application action: upsert transform: log_statements: - context: log statements: - set(attributes["level"], ConvertCase(severity_text, "lower")) exporters: otlphttp: endpoint: https://logging-loki-gateway-http.openshift-logging.svc.cluster.local:8080/api/logs/v1/application/otlp encoding: json tls: ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" auth: authenticator: bearertokenauth debug: verbosity: detailed service: extensions: [bearertokenauth]2 pipelines: logs: receivers: [otlp] processors: [k8sattributes, transform, resource] exporters: [otlphttp]3 logs/test: receivers: [otlp] processors: [] exporters: [debug]
您可以将 telemetrygen 部署为测试:
apiVersion: batch/v1
kind: Job
metadata:
name: telemetrygen
spec:
template:
spec:
containers:
- name: telemetrygen
image: ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:v0.106.1
args:
- logs
- --otlp-endpoint=otel-collector.openshift-logging.svc.cluster.local:4317
- --otlp-insecure
- --duration=180s
- --workers=1
- --logs=10
- --otlp-attributes=k8s.container.name="telemetrygen"
restartPolicy: Never
backoffLimit: 4
1.3. 将遥测数据转发到第三方系统 复制链接链接已复制到粘贴板!
OpenTelemetry Collector 通过 OpenTelemetry 协议(OTLP)使用 OTLP 导出数据,通过 gRPC 或 HTTP 传输实现。如果您需要将遥测数据转发到第三方系统,且不支持红帽构建的 OpenTelemetry 中的 OTLP 或其他支持的协议,则您可以通过 OTLP 接收遥测数据,并使用自定义导出器将其导出到第三方系统。
红帽不支持自定义部署。
先决条件
- 您已开发了自己的不支持的自定义导出器,该导出器可以通过 OTLP 将遥测数据导出到第三方系统。
流程
通过 OperatorHub 或手动部署自定义收集器:
- 如果您的第三方系统支持它,请使用 OperatorHub 部署自定义 Collector。
使用配置映射、部署和服务手动部署自定义 Collector。
自定义 Collector 部署示例
apiVersion: v1 kind: ConfigMap metadata: name: custom-otel-collector-config data: otel-collector-config.yaml: | receivers: otlp: protocols: grpc: exporters: debug: {} prometheus: service: pipelines: traces: receivers: [otlp] exporters: [debug]1 --- apiVersion: apps/v1 kind: Deployment metadata: name: custom-otel-collector-deployment spec: replicas: 1 selector: matchLabels: component: otel-collector template: metadata: labels: component: otel-collector spec: containers: - name: opentelemetry-collector image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:latest2 command: - "/otelcol-contrib" - "--config=/conf/otel-collector-config.yaml" ports: - name: otlp containerPort: 4317 protocol: TCP volumeMounts: - name: otel-collector-config-vol mountPath: /conf readOnly: true volumes: - name: otel-collector-config-vol configMap: name: custom-otel-collector-config --- apiVersion: v1 kind: Service metadata: name: custom-otel-collector-service3 labels: component: otel-collector spec: type: ClusterIP ports: - name: otlp-grpc port: 4317 targetPort: 4317 selector: component: otel-collector
1.4. 将遥测数据转发到 AWS 复制链接链接已复制到粘贴板!
要将遥测数据转发到 AWS,请使用带有以下导出器的 OpenTelemetry Collector :AWS CloudWatch Logs Exporter 用于日志,AWS EMF Exporter 用于指标,使用 AWS X-Ray Exporter 用于 trace。
1.5. 将遥测数据转发到 Google Cloud 复制链接链接已复制到粘贴板!
要将遥测数据转发到 Google Cloud Operations Suite,请将 OpenTelemetry Collector 与 Google Cloud Exporter 搭配使用。导出器将指标发送到 Google Cloud Monitoring,日志发送到 Google Cloud Logging,并追踪到 Google Cloud Trace。