3.3. Camel K Operator のアラート

OpenShift モニタリングスタックからの AlertManager インスタンスが Camel K Operator によって公開されるメトリクスに基づいてアラートをトリガーできるように、PrometheusRule リソースを作成することができます。

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
 name: camel-k-operator
spec:
 groups:
   - name: camel-k-operator
     rules:
       - alert: CamelKReconciliationDuration
         expr: |
           (
           1 - sum(rate(camel_k_reconciliation_duration_seconds_bucket{le="0.5"}[5m])) by (job)
           /
           sum(rate(camel_k_reconciliation_duration_seconds_count[5m])) by (job)
           )
           * 100
           > 10
         for: 1m
         labels:
           severity: warning
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the reconciliation requests
             for {{ $labels.job }} have their duration above 0.5s.
       - alert: CamelKReconciliationFailure
         expr: |
           sum(rate(camel_k_reconciliation_duration_seconds_count{result="Errored"}[5m])) by (job)
           /
           sum(rate(camel_k_reconciliation_duration_seconds_count[5m])) by (job)
           * 100
           > 1
         for: 10m
         labels:
           severity: warning
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the reconciliation requests
             for {{ $labels.job }} have failed.
       - alert: CamelKSuccessBuildDuration2m
         expr: |
           (
           1 - sum(rate(camel_k_build_duration_seconds_bucket{le="120",result="Succeeded"}[5m])) by (job)
           /
           sum(rate(camel_k_build_duration_seconds_count{result="Succeeded"}[5m])) by (job)
           )
           * 100
           > 10
         for: 1m
         labels:
           severity: warning
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the successful builds
             for {{ $labels.job }} have their duration above 2m.
       - alert: CamelKSuccessBuildDuration5m
         expr: |
           (
           1 - sum(rate(camel_k_build_duration_seconds_bucket{le="300",result="Succeeded"}[5m])) by (job)
           /
           sum(rate(camel_k_build_duration_seconds_count{result="Succeeded"}[5m])) by (job)
           )
           * 100
           > 1
         for: 1m
         labels:
           severity: critical
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the successful builds
             for {{ $labels.job }} have their duration above 5m.
       - alert: CamelKBuildFailure
         expr: |
           sum(rate(camel_k_build_duration_seconds_count{result="Failed"}[5m])) by (job)
           /
           sum(rate(camel_k_build_duration_seconds_count[5m])) by (job)
           * 100
           > 1
         for: 10m
         labels:
           severity: warning
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the builds for {{ $labels.job }} have failed.
       - alert: CamelKBuildError
         expr: |
           sum(rate(camel_k_build_duration_seconds_count{result="Error"}[5m])) by (job)
           /
           sum(rate(camel_k_build_duration_seconds_count[5m])) by (job)
           * 100
           > 1
         for: 10m
         labels:
           severity: critical
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the builds for {{ $labels.job }} have errored.
       - alert: CamelKBuildQueueDuration1m
         expr: |
           (
           1 - sum(rate(camel_k_build_queue_duration_seconds_bucket{le="60"}[5m])) by (job)
           /
           sum(rate(camel_k_build_queue_duration_seconds_count[5m])) by (job)
           )
           * 100
           > 1
         for: 1m
         labels:
           severity: warning
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the builds for {{ $labels.job }}
             have been queued for more than 1m.
       - alert: CamelKBuildQueueDuration5m
         expr: |
           (
           1 - sum(rate(camel_k_build_queue_duration_seconds_bucket{le="300"}[5m])) by (job)
           /
           sum(rate(camel_k_build_queue_duration_seconds_count[5m])) by (job)
           )
           * 100
           > 1
         for: 1m
         labels:
           severity: critical
         annotations:
           message: |
             {{ printf "%0.0f" $value }}% of the builds for {{ $labels.job }}
             have been queued for more than 5m.

アラートについての詳細は、OpenShift のドキュメントの「Creating alerting rules」を参照してください。