Chapter 2. Monitoring

Procedure

1. In the Administrator perspective of the web console, go to Observe Dashboards.
2. From the Dashboard drop-down list, select the desired GitOps dashboard: GitOps (Overview), GitOps / Components, or GitOps / gRPC Services.
3. Optional: Choose a specific namespace, cluster, and interval from the Namespace, Cluster, and Interval drop-down lists.
4. View the desired GitOps metrics in the GitOps dashboard.

Procedure

1. In the Developer perspective of the web console, select the namespace where your Argo CD application is installed, and navigate to Observe Metrics.
2. From the Select query drop-down list, select Custom query.

3. To check the health status of your Argo CD application, enter the Prometheus Query Language (PromQL) query similar to the following example in the Expression field:

   Example

   sum(argocd_app_info{dest_namespace=~"<your_defined_namespace>",health_status!=""}) by (health_status) 1

   1

   Replace the <your_defined_namespace> variable with the actual name of your defined namespace, for example openshift-gitops.

Procedure

1. Log in to the OpenShift Container Platform web console.
2. In the Administrator perspective of the web console, click Operators Installed Operators.
3. From the Project list, select the project where the user-defined Argo CD instance is installed.
4. Select Red Hat OpenShift GitOps from the installed Operators list and go to the Argo CD tab.
5. Click your user-defined Argo CD instance.

6. Configure the ArgoCD CR of your user-defined Argo CD instance to disable the automatic scraping of metrics:

   1. Click the YAML tab and edit the YAML file of the ArgoCD CR.

   2. In the ArgoCD CR, set the spec.monitoring.disableMetrics field value to true:

      Example ArgoCD CR

      apiVersion: argoproj.io/v1beta1
      kind: ArgoCD
      metadata:
       name: example  1
       namespace: spring-petclinic 2
      spec:
       monitoring:
         disableMetrics: true

      1

      The name of the user-defined Argo CD instance.

      2

      The namespace where you want to run the user-defined Argo CD instance.

      Tip

      Alternatively, use the following command to disable the automatic scraping of metrics in the Red Hat OpenShift GitOps argocd CLI:

      Example command

      $ oc patch argocd example -n spring-petclinic --type='json' -p='[{"op": "replace", "path": "/spec/monitoring/disableMetrics", "value": true}]'

      Example output

      argocd.argoproj.io/example patched

7. Verify that the Operator adds the openshift.io/cluster-monitoring=false label to your defined namespace:

   1. Go to Administration Namespaces.

      The Namespaces page displays the created namespaces.

   2. Click your defined namespace, go to the YAML tab, and verify that under the metadata.labels section, the openshift.io/cluster-monitoring=false label is added by the Operator.

8. Verify that the Operator deletes the following resources from your defined namespace:

   1. Go to Home Search.

   2. From the Resources list, select PrometheusRule, Role, RoleBinding, and ServiceMonitors.

      The Search page displays the selected resources.

   3. In the Search page, verify that under the PrometheusRule section, the gitops-operator-argocd-alerts prometheus rule is removed.
   4. Under the Roles section, from the Filter list, select Namespace Roles.
   5. Verify that the <argocd_namespace>-read role is removed.
   6. Under the RoleBindings section, from the Filter list, select Namespace RoleBindings.
   7. Verify that the <argocd_namespace>-prometheus-k8s-read-binding role binding is removed.
   8. Verify that under the ServiceMonitors section, the <argocd_name>, <argocd_name>-repo-server, and <argocd_name>-server service monitors are removed.

Procedure

1. In the Administrator perspective of the web console, go to Observe Metrics.

2. Enter the metric in the Expression field. You can choose from the following metrics:

   • active_argocd_instances_total
   • active_argocd_instances_by_phase
   • active_argocd_instance_reconciliation_count
   • controller_runtime_reconcile_time_seconds_per_instance_bucket
   • controller_runtime_reconcile_time_seconds_per_instance_count
   • controller_runtime_reconcile_time_seconds_per_instance_sum

3. (Optional): Filter the metric by its properties. For example, filter the active_argocd_instances_by_phase metric by the Available phase:

   Example

   active_argocd_instances_by_phase{phase="Available"}

4. (Optional): Click Add query to enter multiple queries.
5. Click Run queries to enable and observe the GitOps Operator metrics.

Procedure

1. Click Environments under the Developer perspective. The Environments page shows the list of applications along with their Environment status.
2. Hover over the icons under the Environment status column to see the synchronization status of all the environments.
3. Click the application name from the list to view the details of a specific application.

4. In the Application environments page, if the Resources section under the Overview tab displays icons, hover over the icons to get status details.

   • A broken heart indicates that resource issues have degraded the application’s performance.
   • A yellow yield sign indicates that resource issues have delayed data about the application’s health.
5. To view the deployment history of an application, click the Deployment History tab. The page includes details such as the Last deployment, Description (commit message), Environment, Author, and Revision.

Procedure

1. Set the .spec.monitoring.enabled field value to true on a given Argo CD instance:

   Example Argo CD custom resource

   apiVersion: argoproj.io/v1beta1
   kind: ArgoCD
   metadata:
     name: example-argocd
     labels:
       example: repo
   spec:
    # ...
     monitoring:
       enabled: true
    # ...

2. Verify whether an alert rule is included in the PrometheusRule created by the Operator:

   Example alert rule

   apiVersion: monitoring.coreos.com/v1
   kind: PrometheusRule
   metadata:
     name: argocd-component-status-alert
     namespace: openshift-gitops
   spec:
     groups:
       - name: ArgoCDComponentStatus
         rules:
          # ...
           - alert: ApplicationSetControllerNotReady 1
             annotations:
               message: >-
                 applicationSet controller deployment for Argo CD instance in
                 namespace "default" is not running
             expr: >-
               kube_statefulset_status_replicas{statefulset="openshift-gitops-application-controller statefulset",
               namespace="openshift-gitops"} !=
               kube_statefulset_status_replicas_ready{statefulset="openshift-gitops-application-controller statefulset",
               namespace="openshift-gitops"}
             for: 1m
             labels:
               severity: critical

   1

   Alert rule in the PrometheusRule that checks whether the workloads created by the Argo CD instances are running as expected.