Chapter 8. Managing metrics

Procedure

1. Create a YAML file for the service configuration. In this example, it is called prometheus-example-app.yaml.

2. Add the following deployment and service configuration details to the file:

   apiVersion: v1
   kind: Namespace
   metadata:
     name: ns1
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     labels:
       app: prometheus-example-app
     name: prometheus-example-app
     namespace: ns1
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: prometheus-example-app
     template:
       metadata:
         labels:
           app: prometheus-example-app
       spec:
         containers:
         - image: ghcr.io/rhobs/prometheus-example-app:0.4.2
           imagePullPolicy: IfNotPresent
           name: prometheus-example-app
   ---
   apiVersion: v1
   kind: Service
   metadata:
     labels:
       app: prometheus-example-app
     name: prometheus-example-app
     namespace: ns1
   spec:
     ports:
     - port: 8080
       protocol: TCP
       targetPort: 8080
       name: web
     selector:
       app: prometheus-example-app
     type: ClusterIP

   Copy to Clipboard Toggle word wrap

   This configuration deploys a service named prometheus-example-app in the user-defined ns1 project. This service exposes the custom version metric.

3. Apply the configuration to the cluster:

   $ oc apply -f prometheus-example-app.yaml

   Copy to Clipboard Toggle word wrap

   It takes some time to deploy the service.

4. You can check that the pod is running:

   $ oc -n ns1 get pod

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                      READY     STATUS    RESTARTS   AGE
   prometheus-example-app-7857545cb7-sbgwq   1/1       Running   0          81m

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a new YAML configuration file named example-app-service-monitor.yaml.

2. Add a ServiceMonitor resource to the YAML file. The following example creates a service monitor named prometheus-example-monitor to scrape metrics exposed by the prometheus-example-app service in the ns1 namespace:

   apiVersion: monitoring.coreos.com/v1
   kind: ServiceMonitor
   metadata:
     name: prometheus-example-monitor
     namespace: ns1 

   1

   
   spec:
     endpoints:
     - interval: 30s
       port: web 

   2

   
       scheme: http
     selector: 

   3

   
       matchLabels:
         app: prometheus-example-app

   Copy to Clipboard Toggle word wrap

   1

   Specify a user-defined namespace where your service runs.

   2

   Specify endpoint ports to be scraped by Prometheus.

   3

   Configure a selector to match your service based on its metadata labels.

   Note

   A ServiceMonitor resource in a user-defined namespace can only discover services in the same namespace. That is, the namespaceSelector field of the ServiceMonitor resource is always ignored.

3. Apply the configuration to the cluster:

   $ oc apply -f example-app-service-monitor.yaml

   Copy to Clipboard Toggle word wrap

   It takes some time to deploy the ServiceMonitor resource.

4. Verify that the ServiceMonitor resource is running:

   $ oc -n <namespace> get servicemonitor

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                         AGE
   prometheus-example-monitor   81m

   Copy to Clipboard Toggle word wrap

Procedure

1. From the Administrator perspective in the OpenShift Container Platform web console, select Observe Metrics.

2. To add one or more queries, do any of the following:

   Expand

   Option	Description
   Create a custom query.	Add your Prometheus Query Language (PromQL) query to the Expression field. As you type a PromQL expression, autocomplete suggestions appear in a drop-down list. These suggestions include functions, metrics, labels, and time tokens. You can use the keyboard arrows to select one of these suggested items and then press Enter to add the item to your expression. You can also move your mouse pointer over a suggested item to view a brief description of that item.
   Add multiple queries.	Select Add query.
   Duplicate an existing query.	Select the Options menu next to the query, then choose Duplicate query.
   Disable a query from being run.	Select the Options menu next to the query and choose Disable query.

   Show more

3. To run queries that you created, select Run queries. The metrics from the queries are visualized on the plot. If a query is invalid, the UI shows an error message.

   Note

   Queries that operate on large amounts of data might time out or overload the browser when drawing time series graphs. To avoid this, select Hide graph and calibrate your query using only the metrics table. Then, after finding a feasible query, enable the plot to draw the graphs.

   Note

   By default, the query table shows an expanded view that lists every metric and its current value. You can select ˅ to minimize the expanded view for a query.

4. Optional: The page URL now contains the queries you ran. To use this set of queries again in the future, save this URL.

5. Explore the visualized metrics. Initially, all metrics from all enabled queries are shown on the plot. You can select which metrics are shown by doing any of the following:

   Expand

   Option	Description
   Hide all metrics from a query.	Click the Options menu for the query and click Hide all series.
   Hide a specific metric.	Go to the query table and click the colored square near the metric name.
   Zoom into the plot and change the time range.	Either: Visually select the time range by clicking and dragging on the plot horizontally. Use the menu in the left upper corner to select the time range.
   Reset the time range.	Select Reset zoom.
   Display outputs for all queries at a specific point in time.	Hold the mouse cursor on the plot at that point. The query outputs will appear in a pop-up box.
   Hide the plot.	Select Hide graph.

   Show more

Procedure

1. From the Developer perspective in the OpenShift Container Platform web console, select Observe Metrics.
2. Select the project that you want to view metrics for in the Project: list.

3. Select a query from the Select query list, or create a custom PromQL query based on the selected query by selecting Show PromQL. The metrics from the queries are visualized on the plot.

   Note

   In the Developer perspective, you can only run one query at a time.

4. Explore the visualized metrics by doing any of the following:

   Expand

   Option	Description
   Zoom into the plot and change the time range.	Either: Visually select the time range by clicking and dragging on the plot horizontally. Use the menu in the left upper corner to select the time range.
   Reset the time range.	Select Reset zoom.
   Display outputs for all queries at a specific point in time.	Hold the mouse cursor on the plot at that point. The query outputs appear in a pop-up box.

   Show more