Chapter 3. Monitoring 3scale

Procedure

1. Configure 3scale to enable monitoring by setting the spec.monitoring.enabled parameter of the 3scale deployment YAML to true. For example:

   1. Create an APIManager custom resource named 3scale-monitoring.yml to enable monitoring:

      apiVersion: apps.3scale.net/v1alpha1
      kind: APIManager
      metadata:
        name: apimanager1
      spec:
        wildcardDomain: example.com
        monitoring:
          enabled: true
          enablePrometheusRules: false 1

      1

      You can optionally disable PrometheusRules, which is otherwise enabled by default.

   2. Log in to your OpenShift cluster. You must log in as a user with an edit cluster role in the OpenShift project of the 3scale, for example, cluster-admin.

      oc login

   3. Switch to your 3scale project.

      oc project <project_name>

   4. Deploy the custom resource:

      $ oc apply -f 3scale-monitoring.yml

Procedure

1. Deploy the Prometheus custom resource as follows depending on whether you want to monitor all resources in the cluster or only 3scale resources:

   • To monitor all resources in the cluster, set the spec.podMonitorSelector attribute to {} and set the spec.ruleSelector attribute to {}. For example, apply the following custom resource:

     apiVersion: monitoring.coreos.com/v1
     kind: Prometheus
     metadata:
       name: example
     spec:
       podMonitorSelector: {}
       ruleSelector: {}

   • If you deployed 3scale and the Prometheus operator in the same OpenShift project, and assuming the value of APP_LABEL is set to the default 3scale-api-management, monitor 3scale resources using the following steps:

     1. Set the spec.podMonitorSelector attribute to:

         podMonitorSelector:
          matchExpressions:
          - key: app
              operator: In
              values:
              - 3scale-api-management

     2. Set the spec.ruleSelector attribute to:

           matchExpressions:
           - key: app
             operator: In
             values:
             - 3scale-api-management

        For example, apply the following custom resource:

        apiVersion: monitoring.coreos.com/v1
        kind: Prometheus
        metadata:
          name: example
        spec:
         podMonitorSelector:
          matchExpressions:
          - key: app
              operator: In
              values:
              - 3scale-api-management
         ruleSelector:
           matchExpressions:
           - key: app
             operator: In
             values:
             - 3scale-api-management

   • If you deployed 3scale and the Prometheus operator in different OpenShift projects, monitor 3scale resources using the following steps:

     1. Label the OpenShift project where 3scale is deployed with MYLABELKEY=MYLABELVALUE

     2. Use a podMonitorNamespaceSelector filter to select the 3scale pods. For example, apply the following custom resource:

        apiVersion: monitoring.coreos.com/v1
        kind: Prometheus
        metadata:
          name: example
        spec:
         podMonitorSelector: {}
         ruleSelector: {}
         podMonitorNamespaceSelector:
           matchExpressions:
           - key: MYLABELKEY
             operator: In
             values:
             - MYLABELVALUE

2. To ensure that dashboards and alerts work as expected, you must incorporate Kubernetes metrics, that is, kube-state-metrics, by performing one of the following:

   • Federate the Prometheus instance with the cluster default Prometheus instance.
   • Configure your own scraping jobs to get metrics from kubelet, etcd and others.

Procedure

1. Make sure Grafana services are configured to monitor the GrafanaDashboards resources by overwriting the app=3scale-api-management label. For example, apply the following custom resource:

   apiVersion: integreatly.org/v1alpha1
   kind: Grafana
   metadata:
     name: grafana
   spec:
     dashboardLabelSelector:
     - matchExpressions:
       - key: app
         operator: In
         values:
         - 3scale-api-management

   Grafana Dashboards created by the 3scale operator are labeled as follows:

   app: 3scale-api-management
   monitoring-key: middleware

2. If the Grafana operator is installed in a different namespace than 3scale, configure it to monitor resources outside the namespace using the --namespaces or --scan-all operator flags. See the Grafana documentation for more information about the operator flags.

3. Create a GrafanaDataSource custom resource of type prometheus to make the Prometheus data available in Grafana. For example:

   apiVersion: integreatly.org/v1alpha1
   kind: GrafanaDataSource
   metadata:
     name: prometheus
   spec:
     name: middleware
     datasources:
       - name: Prometheus
         type: prometheus
         access: proxy
         url: http://prometheus-operated:9090
         isDefault: true
         version: 1
         editable: true
         jsonData:
           timeInterval: "5s"

   where http://prometheus-operated:9090 is the Prometheus route.

4. Make sure permissions are set correctly as described in the Grafana documentation.

Procedure

1. Log into the Grafana console.

2. Check that you can view metrics for the following:

   • Kubernetes resources at pod and namespace level where 3scale is installed
   • APIcast Staging
   • APIcast Production
   • Backend worker
   • Backend listener
   • System
   • Zync