3장. 분산 추적 플랫폼(Tempo)
3.1. 설치
분산 추적 플랫폼(Tempo)을 설치하려면 다음 단계를 수행해야 합니다.
- Tempo Operator 설치합니다.
- 지원되는 오브젝트 저장소를 설정하고 오브젝트 저장소 인증 정보에 대한 보안을 생성합니다.
- 권한 및 테넌트 구성.
사용 사례에 따라 선택한 배포를 설치합니다.
-
microservices-mode
TempoStack
인스턴스 -
모놀리식 모드
TempoMonolithic
인스턴스
-
microservices-mode
3.1.1. Tempo Operator 설치
웹 콘솔 또는 명령줄을 사용하여 Tempo Operator를 설치할 수 있습니다.
3.1.1.1. 웹 콘솔을 사용하여 Tempo Operator 설치
웹 콘솔의 관리자 보기에서 Tempo Operator를 설치할 수 있습니다.
사전 요구 사항
-
cluster-admin
역할을 사용하여 클러스터 관리자로 OpenShift Container Platform 웹 콘솔에 로그인되어 있습니다. -
Red Hat OpenShift Dedicated의 경우
dedicated-admin
역할의 계정을 사용하여 로그인해야 합니다. 지원되는 공급자가 필요한 오브젝트 스토리지 설정 완료: Red Hat OpenShift Data Foundation,MinIO,Amazon S3,Azure Blob Storage,Google Cloud Storage. 자세한 내용은 "오브젝트 스토리지 설정"을 참조하십시오.
주의개체 스토리지는 필수이며 분산 추적 플랫폼(Tempo)에 포함되지 않습니다.
- 중요
3.1.1.2.
- 작은 정보
oc login --username=<your_username>
$ oc login --username=<your_username>
Copy to Clipboard Copied!
- 주의
oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: labels: kubernetes.io/metadata.name: openshift-tempo-operator openshift.io/cluster-monitoring: "true" name: openshift-tempo-operator EOF
$ oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: labels: kubernetes.io/metadata.name: openshift-tempo-operator openshift.io/cluster-monitoring: "true" name: openshift-tempo-operator EOF
Copy to Clipboard Copied! oc apply -f - << EOF apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: openshift-tempo-operator namespace: openshift-tempo-operator spec: upgradeStrategy: Default EOF
$ oc apply -f - << EOF apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: openshift-tempo-operator namespace: openshift-tempo-operator spec: upgradeStrategy: Default EOF
Copy to Clipboard Copied! oc apply -f - << EOF apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: tempo-product namespace: openshift-tempo-operator spec: channel: stable installPlanApproval: Automatic name: tempo-product source: redhat-operators sourceNamespace: openshift-marketplace EOF
$ oc apply -f - << EOF apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: tempo-product namespace: openshift-tempo-operator spec: channel: stable installPlanApproval: Automatic name: tempo-product source: redhat-operators sourceNamespace: openshift-marketplace EOF
Copy to Clipboard Copied!
oc get csv -n openshift-tempo-operator
$ oc get csv -n openshift-tempo-operator
Copy to Clipboard Copied!
3.1.2.
|
|
|
|
|
|
|
|
|
|
|
|
|
3.1.2.1.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::${<aws_account_id>}:oidc-provider/${<oidc_provider>}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_PROVIDER}:sub": [ "system:serviceaccount:${<openshift_project_for_tempostack>}:tempo-${<tempostack_cr_name>}" "system:serviceaccount:${<openshift_project_for_tempostack>}:tempo-${<tempostack_cr_name>}-query-frontend" ] } } } ] }
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::${<aws_account_id>}:oidc-provider/${<oidc_provider>}"
1 }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_PROVIDER}:sub": [ "system:serviceaccount:${<openshift_project_for_tempostack>}:tempo-${<tempostack_cr_name>}"
2 "system:serviceaccount:${<openshift_project_for_tempostack>}:tempo-${<tempostack_cr_name>}-query-frontend" ] } } } ] }
Copy to Clipboard Copied! aws iam create-role \ --role-name "tempo-s3-access" \ --assume-role-policy-document "file:///tmp/trust.json" \ --query Role.Arn \ --output text
$ aws iam create-role \ --role-name "tempo-s3-access" \ --assume-role-policy-document "file:///tmp/trust.json" \ --query Role.Arn \ --output text
Copy to Clipboard Copied! aws iam attach-role-policy \ --role-name "tempo-s3-access" \ --policy-arn "arn:aws:iam::aws:policy/AmazonS3FullAccess"
$ aws iam attach-role-policy \ --role-name "tempo-s3-access" \ --policy-arn "arn:aws:iam::aws:policy/AmazonS3FullAccess"
Copy to Clipboard Copied! apiVersion: v1 kind: Secret metadata: name: minio-test stringData: bucket: <s3_bucket_name> region: <s3_region> role_arn: <s3_role_arn> type: Opaque
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: bucket: <s3_bucket_name> region: <s3_region> role_arn: <s3_role_arn> type: Opaque
Copy to Clipboard Copied!
3.1.2.2.
ibmcloud resource service-key-create <tempo_bucket> Writer \ --instance-name <tempo_bucket> --parameters '{"HMAC":true}'
$ ibmcloud resource service-key-create <tempo_bucket> Writer \ --instance-name <tempo_bucket> --parameters '{"HMAC":true}'
Copy to Clipboard Copied! oc -n <namespace> create secret generic <ibm_cos_secret> \ --from-literal=bucket="<tempo_bucket>" \ --from-literal=endpoint="<ibm_bucket_endpoint>" \ --from-literal=access_key_id="<ibm_bucket_access_key>" \ --from-literal=access_key_secret="<ibm_bucket_secret_key>"
$ oc -n <namespace> create secret generic <ibm_cos_secret> \ --from-literal=bucket="<tempo_bucket>" \ --from-literal=endpoint="<ibm_bucket_endpoint>" \ --from-literal=access_key_id="<ibm_bucket_access_key>" \ --from-literal=access_key_secret="<ibm_bucket_secret_key>"
Copy to Clipboard Copied! apiVersion: v1 kind: Secret metadata: name: <ibm_cos_secret> stringData: bucket: <tempo_bucket> endpoint: <ibm_bucket_endpoint> access_key_id: <ibm_bucket_access_key> access_key_secret: <ibm_bucket_secret_key> type: Opaque
apiVersion: v1 kind: Secret metadata: name: <ibm_cos_secret> stringData: bucket: <tempo_bucket> endpoint: <ibm_bucket_endpoint> access_key_id: <ibm_bucket_access_key> access_key_secret: <ibm_bucket_secret_key> type: Opaque
Copy to Clipboard Copied! apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack # ... spec: # ... storage: secret: name: <ibm_cos_secret> type: s3 # ...
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack # ... spec: # ... storage: secret: name: <ibm_cos_secret>
1 type: s3 # ...
Copy to Clipboard Copied!
3.1.3.
|
|
|
| |
|
|
|
3.1.3.1.
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack metadata: name: redmetrics spec: # ... tenants: mode: openshift authentication: - tenantName: dev tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa" # ...
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack metadata: name: redmetrics spec: # ... tenants: mode: openshift authentication: - tenantName: dev
1 tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
2 # ...
Copy to Clipboard Copied! apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: tempostack-traces-reader rules: - apiGroups: - 'tempo.grafana.com' resources: - dev - prod resourceNames: - traces verbs: - 'get'
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: tempostack-traces-reader rules: - apiGroups: - 'tempo.grafana.com' resources:
1 - dev - prod resourceNames: - traces verbs: - 'get'
2 Copy to Clipboard Copied! apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tempostack-traces-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: tempostack-traces-reader subjects: - kind: Group apiGroup: rbac.authorization.k8s.io name: system:authenticated
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tempostack-traces-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: tempostack-traces-reader subjects: - kind: Group apiGroup: rbac.authorization.k8s.io name: system:authenticated
1 Copy to Clipboard Copied!
3.1.3.2.
apiVersion: v1 kind: ServiceAccount metadata: name: otel-collector namespace: <project_of_opentelemetry_collector_instance>
apiVersion: v1 kind: ServiceAccount metadata: name: otel-collector namespace: <project_of_opentelemetry_collector_instance>
Copy to Clipboard Copied! apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: tempostack-traces-write rules: - apiGroups: - 'tempo.grafana.com' resources: - dev resourceNames: - traces verbs: - 'create'
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: tempostack-traces-write rules: - apiGroups: - 'tempo.grafana.com' resources:
1 - dev resourceNames: - traces verbs: - 'create'
2 Copy to Clipboard Copied! apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tempostack-traces roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: tempostack-traces-write subjects: - kind: ServiceAccount name: otel-collector namespace: otel
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tempostack-traces roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: tempostack-traces-write subjects: - kind: ServiceAccount name: otel-collector
1 namespace: otel
Copy to Clipboard Copied! apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: cluster-collector namespace: <project_of_tempostack_instance> spec: mode: deployment serviceAccount: otel-collector config: | extensions: bearertokenauth: filename: "/var/run/secrets/kubernetes.io/serviceaccount/token" exporters: otlp/dev: endpoint: sample-gateway.tempo.svc.cluster.local:8090 tls: insecure: false ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" auth: authenticator: bearertokenauth headers: X-Scope-OrgID: "dev" otlphttp/dev: endpoint: https://sample-gateway.<project_of_tempostack_instance>.svc.cluster.local:8080/api/traces/v1/dev tls: insecure: false ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" auth: authenticator: bearertokenauth headers: X-Scope-OrgID: "dev" service: extensions: [bearertokenauth] pipelines: traces: exporters: [otlp/dev] # ...
apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: cluster-collector namespace: <project_of_tempostack_instance> spec: mode: deployment serviceAccount: otel-collector
1 config: | extensions: bearertokenauth:
2 filename: "/var/run/secrets/kubernetes.io/serviceaccount/token"
3 exporters: otlp/dev:
4 endpoint: sample-gateway.tempo.svc.cluster.local:8090 tls: insecure: false ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt"
5 auth: authenticator: bearertokenauth headers: X-Scope-OrgID: "dev"
6 otlphttp/dev:
7 endpoint: https://sample-gateway.<project_of_tempostack_instance>.svc.cluster.local:8080/api/traces/v1/dev tls: insecure: false ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" auth: authenticator: bearertokenauth headers: X-Scope-OrgID: "dev" service: extensions: [bearertokenauth] pipelines: traces: exporters: [otlp/dev]
8 # ...
Copy to Clipboard Copied!
3.1.4.
3.1.4.1.
- 주의
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
Copy to Clipboard Copied! - 참고
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack metadata: name: simplest namespace: <project_of_tempostack_instance> spec: storage: secret: name: <secret_name> type: <secret_provider> storageSize: <value>Gi resources: total: limits: memory: 2Gi cpu: 2000m tenants: mode: openshift authentication: - tenantName: dev tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa" - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb" template: gateway: enabled: true queryFrontend: jaegerQuery: enabled: true
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack
1 metadata: name: simplest namespace: <project_of_tempostack_instance>
2 spec: storage:
3 secret:
4 name: <secret_name>
5 type: <secret_provider>
6 storageSize: <value>Gi
7 resources: total: limits: memory: 2Gi cpu: 2000m tenants: mode: openshift
8 authentication:
9 - tenantName: dev
10 tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
11 - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb" template: gateway: enabled: true
12 queryFrontend: jaegerQuery: enabled: true
13 Copy to Clipboard Copied!
- 참고
3.1.4.2.
- 작은 정보
oc login --username=<your_username>
$ oc login --username=<your_username>
Copy to Clipboard Copied!
- 주의
oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: name: <project_of_tempostack_instance> EOF
$ oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: name: <project_of_tempostack_instance> EOF
Copy to Clipboard Copied! oc apply -f - << EOF <object_storage_secret> EOF
$ oc apply -f - << EOF <object_storage_secret> EOF
Copy to Clipboard Copied! apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
Copy to Clipboard Copied! - 참고
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack metadata: name: simplest namespace: <project_of_tempostack_instance> spec: storage: secret: name: <secret_name> type: <secret_provider> storageSize: <value>Gi resources: total: limits: memory: 2Gi cpu: 2000m tenants: mode: openshift authentication: - tenantName: dev tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa" - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb" template: gateway: enabled: true queryFrontend: jaegerQuery: enabled: true
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack
1 metadata: name: simplest namespace: <project_of_tempostack_instance>
2 spec: storage:
3 secret:
4 name: <secret_name>
5 type: <secret_provider>
6 storageSize: <value>Gi
7 resources: total: limits: memory: 2Gi cpu: 2000m tenants: mode: openshift
8 authentication:
9 - tenantName: dev
10 tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
11 - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb" template: gateway: enabled: true
12 queryFrontend: jaegerQuery: enabled: true
13 Copy to Clipboard Copied! oc apply -f - << EOF <tempostack_cr> EOF
$ oc apply -f - << EOF <tempostack_cr> EOF
Copy to Clipboard Copied!
oc get tempostacks.tempo.grafana.com simplest -o yaml
$ oc get tempostacks.tempo.grafana.com simplest -o yaml
Copy to Clipboard Copied! oc get pods
$ oc get pods
Copy to Clipboard Copied! oc get route
$ oc get route
Copy to Clipboard Copied! - 참고
3.1.5.
3.1.5.1.
- 중요
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
Copy to Clipboard Copied! - 참고
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoMonolithic metadata: name: <metadata_name> namespace: <project_of_tempomonolithic_instance> spec: storage: traces: backend: <supported_storage_type> size: <value>Gi s3: secret: <secret_name> tls: enabled: true caName: <ca_certificate_configmap_name> jaegerui: enabled: true route: enabled: true resources: total: limits: memory: <value>Gi cpu: <value>m multitenancy: enabled: true mode: openshift authentication: - tenantName: dev tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa" - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb"
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoMonolithic
1 metadata: name: <metadata_name> namespace: <project_of_tempomonolithic_instance>
2 spec: storage:
3 traces: backend: <supported_storage_type>
4 size: <value>Gi
5 s3:
6 secret: <secret_name>
7 tls:
8 enabled: true caName: <ca_certificate_configmap_name>
9 jaegerui: enabled: true
10 route: enabled: true
11 resources:
12 total: limits: memory: <value>Gi cpu: <value>m multitenancy: enabled: true mode: openshift authentication:
13 - tenantName: dev
14 tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
15 - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb"
Copy to Clipboard Copied!
- 참고
3.1.5.2.
- 작은 정보
oc login --username=<your_username>
$ oc login --username=<your_username>
Copy to Clipboard Copied!
oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: name: <project_of_tempomonolithic_instance> EOF
$ oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: name: <project_of_tempomonolithic_instance> EOF
Copy to Clipboard Copied! - 중요
oc apply -f - << EOF <object_storage_secret> EOF
$ oc apply -f - << EOF <object_storage_secret> EOF
Copy to Clipboard Copied! apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
apiVersion: v1 kind: Secret metadata: name: minio-test stringData: endpoint: http://minio.minio.svc:9000 bucket: tempo access_key_id: tempo access_key_secret: <secret> type: Opaque
Copy to Clipboard Copied! - 작은 정보
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoMonolithic metadata: name: <metadata_name> namespace: <project_of_tempomonolithic_instance> spec: storage: traces: backend: <supported_storage_type> size: <value>Gi s3: secret: <secret_name> tls: enabled: true caName: <ca_certificate_configmap_name> jaegerui: enabled: true route: enabled: true resources: total: limits: memory: <value>Gi cpu: <value>m multitenancy: enabled: true mode: openshift authentication: - tenantName: dev tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa" - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb"
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoMonolithic
1 metadata: name: <metadata_name> namespace: <project_of_tempomonolithic_instance>
2 spec: storage:
3 traces: backend: <supported_storage_type>
4 size: <value>Gi
5 s3:
6 secret: <secret_name>
7 tls:
8 enabled: true caName: <ca_certificate_configmap_name>
9 jaegerui: enabled: true
10 route: enabled: true
11 resources:
12 total: limits: memory: <value>Gi cpu: <value>m multitenancy: enabled: true mode: openshift authentication:
13 - tenantName: dev
14 tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
15 - tenantName: prod tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb"
Copy to Clipboard Copied! oc apply -f - << EOF <tempomonolithic_cr> EOF
$ oc apply -f - << EOF <tempomonolithic_cr> EOF
Copy to Clipboard Copied!
oc get tempomonolithic.tempo.grafana.com <metadata_name_of_tempomonolithic_cr> -o yaml
$ oc get tempomonolithic.tempo.grafana.com <metadata_name_of_tempomonolithic_cr> -o yaml
Copy to Clipboard Copied! oc get pods
$ oc get pods
Copy to Clipboard Copied! oc get route
$ oc get route
Copy to Clipboard Copied!