Chapter 4. Viewing audit logs
Audit provides a security-relevant chronological set of records documenting the sequence of activities that have affected the system by individual users, administrators, or other components of the system.
4.1. About the API audit log
Audit works at the API server level, logging all requests coming to the server. Each audit log contains the following information:
Field | Description |
---|---|
| The audit level at which the event was generated. |
| A unique audit ID, generated for each request. |
| The stage of the request handling when this event instance was generated. |
| The request URI as sent by the client to a server. |
| The Kubernetes verb associated with the request. For non-resource requests, this is the lowercase HTTP method. |
| The authenticated user information. |
| Optional. The impersonated user information, if the request is impersonating another user. |
| Optional. The source IPs, from where the request originated and any intermediate proxies. |
| Optional. The user agent string reported by the client. Note that the user agent is provided by the client, and must not be trusted. |
|
Optional. The object reference this request is targeted at. This does not apply for |
|
Optional. The response status, populated even when the |
|
Optional. The API object from the request, in JSON format. The |
|
Optional. The API object returned in the response, in JSON format. The |
| The time that the request reached the API server. |
| The time that the request reached the current audit stage. |
|
Optional. An unstructured key value map stored with an audit event that may be set by plug-ins invoked in the request serving chain, including authentication, authorization and admission plug-ins. Note that these annotations are for the audit event, and do not correspond to the |
Example output for the Kubernetes API server:
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"ad209ce1-fec7-4130-8192-c4cc63f1d8cd","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-kube-controller-manager/configmaps/cert-recovery-controller-lock?timeout=35s","verb":"update","user":{"username":"system:serviceaccount:openshift-kube-controller-manager:localhost-recovery-client","uid":"dd4997e3-d565-4e37-80f8-7fc122ccd785","groups":["system:serviceaccounts","system:serviceaccounts:openshift-kube-controller-manager","system:authenticated"]},"sourceIPs":["::1"],"userAgent":"cluster-kube-controller-manager-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"configmaps","namespace":"openshift-kube-controller-manager","name":"cert-recovery-controller-lock","uid":"5c57190b-6993-425d-8101-8337e48c7548","apiVersion":"v1","resourceVersion":"574307"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-04-02T08:27:20.200962Z","stageTimestamp":"2020-04-02T08:27:20.206710Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:operator:kube-controller-manager-recovery\" of ClusterRole \"cluster-admin\" to ServiceAccount \"localhost-recovery-client/openshift-kube-controller-manager\""}}
4.2. Viewing the audit log
You can view logs for the OpenShift Container Platform API server or the Kubernetes API server for each master node.
Procedure
To view the audit log:
View the OpenShift Container Platform API server logs
If necessary, get the node name of the log you want to view:
$ oc adm node-logs --role=master --path=openshift-apiserver/ ip-10-0-140-97.ec2.internal audit-2019-04-09T00-12-19.834.log ip-10-0-140-97.ec2.internal audit-2019-04-09T11-13-00.469.log ip-10-0-140-97.ec2.internal audit.log ip-10-0-153-35.ec2.internal audit-2019-04-09T00-11-49.835.log ip-10-0-153-35.ec2.internal audit-2019-04-09T11-08-30.469.log ip-10-0-153-35.ec2.internal audit.log ip-10-0-170-165.ec2.internal audit-2019-04-09T00-13-00.128.log ip-10-0-170-165.ec2.internal audit-2019-04-09T11-10-04.082.log ip-10-0-170-165.ec2.internal audit.log
View the OpenShift Container Platform API server log for a specific master node and timestamp or view all the logs for that master:
$ oc adm node-logs <node-name> --path=openshift-apiserver/<log-name>
For example:
$ oc adm node-logs ip-10-0-140-97.ec2.internal --path=openshift-apiserver/audit-2019-04-08T13-09-01.227.log $ oc adm node-logs ip-10-0-140-97.ec2.internal --path=openshift-apiserver/audit.log
The output appears similar to the following:
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"ad209ce1-fec7-4130-8192-c4cc63f1d8cd","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-kube-controller-manager/configmaps/cert-recovery-controller-lock?timeout=35s","verb":"update","user":{"username":"system:serviceaccount:openshift-kube-controller-manager:localhost-recovery-client","uid":"dd4997e3-d565-4e37-80f8-7fc122ccd785","groups":["system:serviceaccounts","system:serviceaccounts:openshift-kube-controller-manager","system:authenticated"]},"sourceIPs":["::1"],"userAgent":"cluster-kube-controller-manager-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"configmaps","namespace":"openshift-kube-controller-manager","name":"cert-recovery-controller-lock","uid":"5c57190b-6993-425d-8101-8337e48c7548","apiVersion":"v1","resourceVersion":"574307"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-04-02T08:27:20.200962Z","stageTimestamp":"2020-04-02T08:27:20.206710Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:operator:kube-controller-manager-recovery\" of ClusterRole \"cluster-admin\" to ServiceAccount \"localhost-recovery-client/openshift-kube-controller-manager\""}}
View the Kubernetes API server logs:
If necessary, get the node name of the log you want to view:
$ oc adm node-logs --role=master --path=kube-apiserver/ ip-10-0-140-97.ec2.internal audit-2019-04-09T14-07-27.129.log ip-10-0-140-97.ec2.internal audit-2019-04-09T19-18-32.542.log ip-10-0-140-97.ec2.internal audit.log ip-10-0-153-35.ec2.internal audit-2019-04-09T19-24-22.620.log ip-10-0-153-35.ec2.internal audit-2019-04-09T19-51-30.905.log ip-10-0-153-35.ec2.internal audit.log ip-10-0-170-165.ec2.internal audit-2019-04-09T18-37-07.511.log ip-10-0-170-165.ec2.internal audit-2019-04-09T19-21-14.371.log ip-10-0-170-165.ec2.internal audit.log
View the Kubernetes API server log for a specific master node and timestamp or view all the logs for that master:
$ oc adm node-logs <node-name> --path=kube-apiserver/<log-name>
For example:
$ oc adm node-logs ip-10-0-140-97.ec2.internal --path=kube-apiserver/audit-2019-04-09T14-07-27.129.log $ oc adm node-logs ip-10-0-170-165.ec2.internal --path=kube-apiserver/audit.log
The output appears similar to the following:
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"ad209ce1-fec7-4130-8192-c4cc63f1d8cd","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-kube-controller-manager/configmaps/cert-recovery-controller-lock?timeout=35s","verb":"update","user":{"username":"system:serviceaccount:openshift-kube-controller-manager:localhost-recovery-client","uid":"dd4997e3-d565-4e37-80f8-7fc122ccd785","groups":["system:serviceaccounts","system:serviceaccounts:openshift-kube-controller-manager","system:authenticated"]},"sourceIPs":["::1"],"userAgent":"cluster-kube-controller-manager-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"configmaps","namespace":"openshift-kube-controller-manager","name":"cert-recovery-controller-lock","uid":"5c57190b-6993-425d-8101-8337e48c7548","apiVersion":"v1","resourceVersion":"574307"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-04-02T08:27:20.200962Z","stageTimestamp":"2020-04-02T08:27:20.206710Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:operator:kube-controller-manager-recovery\" of ClusterRole \"cluster-admin\" to ServiceAccount \"localhost-recovery-client/openshift-kube-controller-manager\""}}