Chapter 3. Collecting Kubernetes events
Collect Kubernetes events and forward them through the logging pipeline for storage and analysis. Events provide valuable insight into cluster activity, resource state changes, and system operations.
3.1. About event routing Copy linkLink copied to clipboard!
The Event Router is a pod that watches OpenShift Container Platform events and makes them available to logging. The Event Router collects events from all projects and writes them to STDOUT. The collector gathers those events and forwards them to your configured log storage.
You must manually deploy the Event Router.
3.2. Deploying and configuring the Event Router Copy linkLink copied to clipboard!
Deploy the Event Router into your cluster. Red Hat recommends deploying the Event Router to the openshift-logging project to ensure it collects events from across the cluster.
The Event Router image is not included in the Red Hat OpenShift Logging Operator. You must download it separately.
The following Template object creates the service account, cluster role, and cluster role binding required for the Event Router. The template also configures and deploys the Event Router pod. You can either use this template without making changes or edit the template to change the deployment object CPU and memory requests.
Prerequisites
- You need proper permissions to create service accounts and update cluster role bindings. For example, you can run the following template with a user that has the cluster-admin role.
- You have installed the Red Hat OpenShift Logging Operator.
Procedure
Create a template for the Event Router:
apiVersion: template.openshift.io/v1 kind: Template metadata: name: eventrouter-template annotations: description: "A pod forwarding kubernetes events to OpenShift Logging stack." tags: "events,EFK,logging,cluster-logging" objects: - kind: ServiceAccount apiVersion: v1 metadata: name: eventrouter namespace: ${NAMESPACE} - kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: event-reader rules: - apiGroups: [""] resources: ["events"] verbs: ["get", "watch", "list"] - kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: event-reader-binding subjects: - kind: ServiceAccount name: eventrouter namespace: ${NAMESPACE} roleRef: kind: ClusterRole name: event-reader - kind: ConfigMap apiVersion: v1 metadata: name: eventrouter namespace: ${NAMESPACE} data: config.json: |- { "sink": "stdout" } - kind: Deployment apiVersion: apps/v1 metadata: name: eventrouter namespace: ${NAMESPACE} labels: component: "eventrouter" logging-infra: "eventrouter" provider: "openshift" spec: selector: matchLabels: component: "eventrouter" logging-infra: "eventrouter" provider: "openshift" replicas: 1 template: metadata: labels: component: "eventrouter" logging-infra: "eventrouter" provider: "openshift" name: eventrouter spec: serviceAccount: eventrouter containers: - name: kube-eventrouter image: ${IMAGE} imagePullPolicy: IfNotPresent resources: requests: cpu: ${CPU} memory: ${MEMORY} volumeMounts: - name: config-volume mountPath: /etc/eventrouter securityContext: allowPrivilegeEscalation: false capabilities: drop: ["ALL"] securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: config-volume configMap: name: eventrouter parameters: - name: IMAGE displayName: Image value: "registry.redhat.io/openshift-logging/eventrouter-rhel9:v0.4" - name: CPU displayName: CPU value: "100m" - name: MEMORY displayName: Memory value: "128Mi" - name: NAMESPACE displayName: Namespace value: "openshift-logging"ServiceAccount-
Creates a Service Account in the
openshift-loggingproject for the Event Router. ClusterRole- Creates a ClusterRole to monitor for events in the cluster.
ClusterRoleBinding- Creates a ClusterRoleBinding to bind the ClusterRole to the service account.
ConfigMap-
Creates a config map in the
openshift-loggingproject to generate the requiredconfig.jsonfile. Deployment-
Creates a deployment in the
openshift-loggingproject to generate and configure the Event Router pod. IMAGE-
Specifies the image, identified by a tag such as
v0.4. CPU-
Specifies the minimum amount of CPU to allocate to the Event Router pod. Defaults to
100m. MEMORY-
Specifies the minimum amount of memory to allocate to the Event Router pod. Defaults to
128Mi. NAMESPACE-
Specifies the project to install objects in. Defaults to
openshift-logging. Red Hat recommends using the default value to ensure cluster-wide event collection.
Use the following command to process and apply the template:
$ oc process -f <templatefile> | oc apply -n openshift-logging -f -For example:
$ oc process -f eventrouter.yaml | oc apply -n openshift-logging -f -Example output
serviceaccount/eventrouter created clusterrole.rbac.authorization.k8s.io/event-reader created clusterrolebinding.rbac.authorization.k8s.io/event-reader-binding created configmap/eventrouter created deployment.apps/eventrouter createdValidate that the Event Router installed in the
openshift-loggingproject:View the new Event Router pod:
$ oc get pods --selector component=eventrouter -o name -n openshift-loggingExample output
pod/cluster-logging-eventrouter-d649f97c8-qvv8rView the events collected by the Event Router:
$ oc logs <cluster_logging_eventrouter_pod> -n openshift-loggingFor example:
$ oc logs cluster-logging-eventrouter-d649f97c8-qvv8r -n openshift-loggingExample output
{"verb":"ADDED","event":{"metadata":{"name":"openshift-service-catalog-controller-manager-remover.1632d931e88fcd8f","namespace":"openshift-service-catalog-removed","selfLink":"/api/v1/namespaces/openshift-service-catalog-removed/events/openshift-service-catalog-controller-manager-remover.1632d931e88fcd8f","uid":"787d7b26-3d2f-4017-b0b0-420db4ae62c0","resourceVersion":"21399","creationTimestamp":"2020-09-08T15:40:26Z"},"involvedObject":{"kind":"Job","namespace":"openshift-service-catalog-removed","name":"openshift-service-catalog-controller-manager-remover","uid":"fac9f479-4ad5-4a57-8adc-cb25d3d9cf8f","apiVersion":"batch/v1","resourceVersion":"21280"},"reason":"Completed","message":"Job completed","source":{"component":"job-controller"},"firstTimestamp":"2020-09-08T15:40:26Z","lastTimestamp":"2020-09-08T15:40:26Z","count":1,"type":"Normal"}}