Chapter 6. Monitoring the Dev Workspace operator
This chapter describes how to configure an example monitoring stack to process metrics exposed by the Dev Workspace operator. You must enable the Dev Workspace operator to follow the instructions in this chapter. See https://access.redhat.com/documentation/en-us/red_hat_codeready_workspaces/2.15/html-single/installation_guide/index#enabling-dev-workspace-operator.adoc.
6.1. Collecting Dev Workspace operator metrics with Prometheus
This section describes how to use the Prometheus to collect, store, and query metrics about the Dev Workspace operator.
Prerequisites
-
The
devworkspace-controller-metrics
service is exposing metrics on port8443
. -
The
devworkspace-webhookserver
service is exposing metrics on port9443
. By default, the service exposes metrics on port9443
. -
Prometheus 2.26.0 or later is running. The Prometheus console is running on port
9090
with a corresponding service and route. See First steps with Prometheus.
Procedure
Create a
ClusterRoleBinding
to bind theServiceAccount
associated with Prometheus to the devworkspace-controller-metrics-readerClusterRole
. Without theClusterRoleBinding
, you cannot access Dev Workspace metrics because they are protected with role-based access control (RBAC).Example 6.1. ClusterRole example
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: devworkspace-controller-metrics-reader rules: - nonResourceURLs: - /metrics verbs: - get
Example 6.2. ClusterRoleBinding example
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: devworkspace-controller-metrics-binding subjects: - kind: ServiceAccount name: <ServiceAccount name associated with the Prometheus Pod> namespace: <Prometheus namespace> roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: devworkspace-controller-metrics-reader
Configure Prometheus to scrape metrics from the
8443
port exposed by thedevworkspace-controller-metrics
service, and9443
port exposed by thedevworkspace-webhookserver
service.Example 6.3. Prometheus configuration example
apiVersion: v1 kind: ConfigMap metadata: name: prometheus-config data: prometheus.yml: |- global: scrape_interval: 5s 1 evaluation_interval: 5s 2 scrape_configs: 3 - job_name: 'DevWorkspace' authorization: type: Bearer credentials_file: '/var/run/secrets/kubernetes.io/serviceaccount/token' tls_config: insecure_skip_verify: true static_configs: - targets: ['devworkspace-controller-metrics:8443'] 4 - job_name: 'DevWorkspace webhooks' authorization: type: Bearer credentials_file: '/var/run/secrets/kubernetes.io/serviceaccount/token' tls_config: insecure_skip_verify: true static_configs: - targets: ['devworkspace-webhookserver:9443'] 5
- 1
- Rate at which a target is scraped.
- 2
- Rate at which recording and alerting rules are re-checked.
- 3
- Resources that Prometheus monitors. In the default configuration, two jobs (
DevWorkspace
andDevWorkspace webhooks
), scrape the time series data exposed by thedevworkspace-controller-metrics
anddevworkspace-webhookserver
services. - 4
- Scrape metrics from the
8443
port. - 5
- Scrape metrics from the
9443
port.
Verification steps
Use the Prometheus console to view targets and metrics.
For more information, see Using the expression browser.
Additional resources
6.2. Dev Workspace-specific metrics
This section describes the Dev Workspace-specific metrics exposed by the devworkspace-controller-metrics
service.
Name | Type | Description | Labels |
---|---|---|---|
| Counter | Number of Dev Workspace starting events. |
|
| Counter |
Number of Dev Workspaces successfully entering the |
|
| Counter | Number of failed Dev Workspaces. |
|
| Histogram | Total time taken to start a Dev Workspace, in seconds. |
|
Name | Description | Values |
---|---|---|
|
The |
|
|
The |
|
| The workspace startup failure reason. |
|
Name | Description |
---|---|
| Startup failure due to an invalid devfile used to create a Dev Workspace. |
|
Startup failure due to the following errors: |
| Unknown failure reason. |