Chapter 3. Monitoring and logging Red Hat Developer Hub on Amazon Web Services (AWS)
You can configure Red Hat Developer Hub to use Amazon CloudWatch for real-time monitoring and Amazon Prometheus for comprehensive logging. This is convenient when hosting Developer Hub on Amazon Web Services (AWS) infrastructure.
3.1. Monitoring with Amazon Prometheus Copy linkLink copied to clipboard!
You can configure Red Hat Developer Hub to use Amazon Prometheus for comprehensive logging. Amazon Prometheus extracts data from pods that have specific pod annotations.
3.1.1. Prerequisites Copy linkLink copied to clipboard!
- You configured Prometheus for your Elastic Kubernetes Service (EKS) clusters.
- You created an Amazon managed service for the Prometheus workspace.
- You configured Prometheus to import the Developer Hub metrics.
- You ingested Prometheus metrics into the created workspace.
3.1.2. Configuring annotations for monitoring with Amazon Prometheus by using the Red Hat Developer Hub Operator Copy linkLink copied to clipboard!
To enable logging to Amazon Prometheus, you can configure the required pod annotations by using the Red Hat Developer Hub Operator.
Procedure
As an administrator of the Red Hat Developer Hub Operator, edit the default configuration to add Prometheus annotations as follows:
Update OPERATOR_NS accordingly OPERATOR_NS=rhdh-operator kubectl edit configmap backstage-default-config -n "${OPERATOR_NS}"# Update OPERATOR_NS accordingly $ OPERATOR_NS=rhdh-operator $ kubectl edit configmap backstage-default-config -n "${OPERATOR_NS}"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Find the
deployment.yamlkey in the config map and add the annotations to thespec.template.metadata.annotationsfield as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Save your changes.
Verification
To verify if the scraping works:
Use
kubectlto port-forward the Prometheus console to your local machine as follows:kubectl --namespace=prometheus port-forward deploy/prometheus-server 9090
$ kubectl --namespace=prometheus port-forward deploy/prometheus-server 9090Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Open your web browser and navigate to
http://localhost:9090to access the Prometheus console. -
Monitor relevant metrics, such as
process_cpu_user_seconds_total.
3.1.3. Configuring annotations for monitoring with Amazon Prometheus by using the Red Hat Developer Hub Helm chart Copy linkLink copied to clipboard!
To enable logging to Amazon Prometheus, you can configure the required pod annotations by using the Red Hat Developer Hub Helm chart.
Procedure
To annotate the backstage pod for monitoring, update your
values.yamlfile as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
To verify if the scraping works:
Use
kubectlto port-forward the Prometheus console to your local machine as follows:kubectl --namespace=prometheus port-forward deploy/prometheus-server 9090
kubectl --namespace=prometheus port-forward deploy/prometheus-server 9090Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Open your web browser and navigate to
http://localhost:9090to access the Prometheus console. -
Monitor relevant metrics, such as
process_cpu_user_seconds_total.
3.2. Logging with Amazon CloudWatch Copy linkLink copied to clipboard!
Logging within the Red Hat Developer Hub relies on the Winston library.
3.2.1. Configuring the application log level by using the Red Hat Developer Hub Operator Copy linkLink copied to clipboard!
You can configure the application log level by using the Red Hat Developer Hub Operator.
Procedure
Modify the logging level by including the environment variable
LOG_LEVELin your custom resource as follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2.2. Configuring the application log level by using the Red Hat Developer Hub Helm chart Copy linkLink copied to clipboard!
You can configure the application log level by using the Red Hat Developer Hub Helm chart.
Procedure
Modify the logging level by adding the environment variable
LOG_LEVELto your Helm chartvalues.yamlfile:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2.3. Retrieving logs from Amazon CloudWatch Copy linkLink copied to clipboard!
Prerequisites
- CloudWatch Container Insights is used to capture logs and metrics for Amazon Elastic Kubernetes Service. For more information, see Logging for Amazon Elastic Kubernetes Service documentation.
- To capture the logs and metrics, install the Amazon CloudWatch Observability EKS add-on in your cluster. Following the setup of Container Insights, you can access container logs using Logs Insights or Live Tail views.
CloudWatch names the log group where all container logs are consolidated in the following manner:
/aws/containerinsights/<cluster_name>/application
/aws/containerinsights/<cluster_name>/applicationCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Procedure
To retrieve logs from the Developer Hub instance, run a query such as:
fields @timestamp, @message, kubernetes.container_name | filter kubernetes.container_name in ["install-dynamic-plugins", "backstage-backend"]
fields @timestamp, @message, kubernetes.container_name | filter kubernetes.container_name in ["install-dynamic-plugins", "backstage-backend"]Copy to Clipboard Copied! Toggle word wrap Toggle overflow