Chapter 2. Monitoring

Procedure

1. In the Administrator perspective of the web console, go to Observe Dashboards.
2. From the Dashboard drop-down list, select the desired GitOps dashboard: GitOps (Overview), GitOps / Components, GitOps / gRPC Services or GitOps / Rollouts.
3. Optional: Choose a specific namespace, cluster, and interval from the Namespace, Cluster, and Interval drop-down lists.
4. Review the metrics displayed in the dashboard to monitor your GitOps instance health and performance.

Procedure

1. In the Developer perspective of the web console, select the namespace where your application is deployed, and navigate to Observe Metrics.
2. From the Select query drop-down list, select Custom query.

3. To check the health status of your application, enter the Prometheus Query Language (PromQL) query similar to the following example in the Expression field:

   Example:

Procedure

1. Log in to the OpenShift Container Platform web console.
2. In the Administrator perspective of the web console, click Operators Installed Operators.
3. From the Project list, select the project where the user-defined Argo CD instance is installed.
4. Select Red Hat OpenShift GitOps from the installed Operators list and go to the Argo CD tab.
5. Click your user-defined Argo CD instance.

6. Configure the ArgoCD CR of your user-defined Argo CD instance to disable the automatic scraping of metrics:

   1. Click the YAML tab and edit the YAML file of the ArgoCD CR.

   2. In the ArgoCD CR, set the spec.monitoring.disableMetrics field value to true:

      Example ArgoCD CR:

      apiVersion: argoproj.io/v1beta1
      kind: ArgoCD
      metadata:
       name: example
       namespace: spring-petclinic
      spec:
       monitoring:
         disableMetrics: true

      where:

      metadata.name

      Specifies the name of the user-defined Argo CD instance.

      metadata.namespace

      Specifies the namespace where you want to run the user-defined Argo CD instance.

      Tip

      Alternatively, use the oc CLI to disable automatic metric scraping:

      $ oc patch argocd example -n spring-petclinic --type='json' -p='[{"op": "replace", "path": "/spec/monitoring/disableMetrics", "value": true}]'

      Example output:

      argocd.argoproj.io/example patched

7. Verify that the Operator adds the openshift.io/cluster-monitoring=false label to your defined namespace:

   1. Go to Administration Namespaces.

      The Namespaces page displays the created namespaces.

   2. Click your defined namespace, go to the YAML tab, and verify that under the metadata.labels section, the openshift.io/cluster-monitoring=false label is added by the Operator.

8. Verify that the Operator deletes the following resources from your defined namespace:

   1. Go to Home Search.

   2. From the Resources list, select PrometheusRule, Role, RoleBinding, and ServiceMonitors.

      The Search page displays the selected resources.

   3. In the Search page, verify that under the PrometheusRule section, the gitops-operator-argocd-alerts prometheus rule is removed.
   4. Under the Roles section, from the Filter list, select Namespace Roles.
   5. Verify that the <argocd_namespace>-read role is removed.
   6. Under the RoleBindings section, from the Filter list, select Namespace RoleBindings.
   7. Verify that the <argocd_namespace>-prometheus-k8s-read-binding role binding is removed.
   8. Verify that under the ServiceMonitors section, the <argocd_name>, <argocd_name>-repo-server, and <argocd_name>-server service monitors are removed.

Procedure

1. In the Administrator perspective of the web console, go to Observe Metrics.

2. In the Expression field, enter the name of the metric you want to query. See the metrics table for available options.

   • active_argocd_instances_total
   • active_argocd_instances_by_phase
   • active_argocd_instance_reconciliation_count
   • controller_runtime_reconcile_time_seconds_per_instance_bucket
   • controller_runtime_reconcile_time_seconds_per_instance_count
   • controller_runtime_reconcile_time_seconds_per_instance_sum

3. Optional: Filter the metric by its properties. For example, filter the active_argocd_instances_by_phase metric by the Available phase:

   Example:

   active_argocd_instances_by_phase{phase="Available"}

4. Optional: Click Add query to enter multiple queries.
5. Click Run queries to enable and observe the GitOps Operator metrics.
6. Review the metrics visualization to monitor Operator performance.

Procedure

1. In the OpenShift Container Platform web console, navigate to the Developer perspective and click Environments. The Environments page shows the list of applications along with their Environment status.
2. Hover over the icons under the Environment status column to see the synchronization status of all the environments.
3. Click the application name from the list to view the details of a specific application.

4. In the Application environments page, view the Resources section under the Overview tab to check the resource health status:

   • A broken heart icon indicates that resource issues have degraded the application’s performance.
   • A yellow yield sign icon indicates that resource issues have delayed the health status data.
5. Optional: In the Application environments page, click the Deployment History tab to view the deployment history. The page displays details such as Last deployment, Description (commit message), Environment, Author, and Revision.

Procedure

1. Set the .spec.monitoring.enabled field value to true on a given Argo CD instance:

   Example Argo CD custom resource:

   apiVersion: argoproj.io/v1beta1
   kind: ArgoCD
   metadata:
     name: example-argocd
     labels:
       example: repo
   spec:
    # ...
     monitoring:
       enabled: true
    # ...

2. Verify whether an alert rule is included in the PrometheusRule created by the Operator:

   Example alert rule:

   apiVersion: monitoring.coreos.com/v1
   kind: PrometheusRule
   metadata:
     name: argocd-component-status-alert
     namespace: openshift-gitops
   spec:
     groups:
       - name: ArgoCDComponentStatus
         rules:
           # ...
           - alert: ApplicationSetControllerNotReady
             annotations:
               message: >-
                 ApplicationSet controller deployment for the Argo CD instance in
                 namespace "default" is not running
             expr: >-
               kube_statefulset_status_replicas{
                 statefulset="openshift-gitops-application-controller",
                 namespace="openshift-gitops"
               } !=
               kube_statefulset_status_replicas_ready{
                 statefulset="openshift-gitops-application-controller",
                 namespace="openshift-gitops"
               }
             for: 1m
             labels:
               severity: critical

   where:

   spec.groups.name

   Specifies the name of the alert rule group.

   spec.groups.rules.alert

   Specifies the name of the alert that triggers when workloads created by the Argo CD instance are not running as expected.

   spec.groups.rules.annotations.message

   Specifies a description of the alert condition.

   spec.groups.rules.expr

   Specifies the Prometheus expression that evaluates the alert condition.

   spec.groups.rules.for

   Specifies the duration for which the condition must be true before the alert fires.

   spec.groups.rules.labels.severity

   Specifies the severity level of the alert.