第 9 章 从多个集群收集可观察性数据

流程

1. 在 OpenTelemetry Collector 实例中挂载以下证书，跳过已挂载的证书。

   1. 使用 cert-manager Operator for Red Hat OpenShift 生成这些证书的签发者。

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: selfsigned-issuer
      spec:
        selfSigned: {}

   2. 一个自签名证书。

      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: ca
      spec:
        isCA: true
        commonName: ca
        subject:
          organizations:
            - Organization # <your_organization_name>
          organizationalUnits:
            - Widgets
        secretName: ca-secret
        privateKey:
          algorithm: ECDSA
          size: 256
        issuerRef:
          name: selfsigned-issuer
          kind: Issuer
          group: cert-manager.io

   3. 一个 CA 签发者。

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: test-ca-issuer
      spec:
        ca:
          secretName: ca-secret

   4. 客户端和服务器证书。

      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: server
      spec:
        secretName: server-tls
        isCA: false
        usages:
          - server auth
          - client auth
        dnsNames:
        - "otel.observability.svc.cluster.local" 1
        issuerRef:
          name: ca-issuer
      ---
      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: client
      spec:
        secretName: client-tls
        isCA: false
        usages:
          - server auth
          - client auth
        dnsNames:
        - "otel.observability.svc.cluster.local" 2
        issuerRef:
          name: ca-issuer

      1

      在 server OpenTelemetry Collector 实例中映射到 solver 的确切 DNS 名称列表。

      2

      在客户端 OpenTelemetry Collector 实例中映射到 solver 的确切 DNS 名称列表。

2. 为 OpenTelemetry Collector 实例创建服务帐户。

   ServiceAccount 示例

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: otel-collector-deployment

3. 为服务帐户创建集群角色。

   ClusterRole 示例

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: otel-collector
   rules:
     1
     2
   - apiGroups: ["", "config.openshift.io"]
     resources: ["pods", "namespaces", "infrastructures", "infrastructures/status"]
     verbs: ["get", "watch", "list"]

   1

   k8sattributesprocessor 需要 pod 和命名空间资源的权限。

   2

   resourcedetectionprocessor 需要基础架构和状态的权限。

4. 将集群角色绑定到服务帐户。

   ClusterRoleBinding 示例

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: otel-collector
   subjects:
   - kind: ServiceAccount
     name: otel-collector-deployment
     namespace: otel-collector-<example>
   roleRef:
     kind: ClusterRole
     name: otel-collector
     apiGroup: rbac.authorization.k8s.io

5. 创建 YAML 文件，在边缘集群中定义 OpenTelemetryCollector 自定义资源 (CR)。

   边缘集群的 OpenTelemetryCollector 自定义资源示例

   apiVersion: opentelemetry.io/v1alpha1
   kind: OpenTelemetryCollector
   metadata:
     name: otel
     namespace: otel-collector-<example>
   spec:
     mode: daemonset
     serviceAccount: otel-collector-deployment
     config: |
       receivers:
         jaeger:
           protocols:
             grpc: {}
             thrift_binary: {}
             thrift_compact: {}
             thrift_http: {}
         opencensus:
         otlp:
           protocols:
             grpc: {}
             http: {}
         zipkin: {}
       processors:
         batch: {}
         k8sattributes: {}
         memory_limiter:
           check_interval: 1s
           limit_percentage: 50
           spike_limit_percentage: 30
         resourcedetection:
           detectors: [openshift]
       exporters:
         otlphttp:
           endpoint: https://observability-cluster.com:443 1
           tls:
             insecure: false
             cert_file: /certs/server.crt
             key_file: /certs/server.key
             ca_file: /certs/ca.crt
       service:
         pipelines:
           traces:
             receivers: [jaeger, opencensus, otlp, zipkin]
             processors: [memory_limiter, k8sattributes, resourcedetection, batch]
             exporters: [otlp]
     volumes:
       - name: otel-certs
         secret:
           name: otel-certs
     volumeMounts:
       - name: otel-certs
         mountPath: /certs

   1

   Collector exporter 配置为导出 OTLP HTTP，并指向来自中央集群的 OpenTelemetry Collector。

6. 创建 YAML 文件，在中央集群中定义 OpenTelemetryCollector 自定义资源 (CR)。

   Central 集群的 OpenTelemetryCollector 自定义资源示例

   apiVersion: opentelemetry.io/v1alpha1
   kind: OpenTelemetryCollector
   metadata:
     name: otlp-receiver
     namespace: observability
   spec:
     mode: "deployment"
     ingress:
       type: route
       route:
         termination: "passthrough"
     config: |
       receivers:
         otlp:
           protocols:
             http:
               tls: 1
                 cert_file: /certs/server.crt
                 key_file: /certs/server.key
                 client_ca_file: /certs/ca.crt
       exporters:
         logging: {}
         otlp:
           endpoint: "tempo-<simplest>-distributor:4317" 2
           tls:
             insecure: true
       service:
         pipelines:
           traces:
             receivers: [otlp]
             processors: []
             exporters: [otlp]
     volumes:
       - name: otel-certs
         secret:
           name: otel-certs
     volumeMounts:
       - name: otel-certs
         mountPath: /certs

   1

   Collector 接收器需要第一步中列出的证书。

   2

   Collector exporter 配置为导出 OTLP 并指向 Tempo 经销商端点，本例中为 "tempo-simplest-distributor:4317" 并已创建。