Questo contenuto non è disponibile nella lingua selezionata.

Chapter 7. Security Profiles Operator


7.1. Security Profiles Operator overview

OpenShift Container Platform Security Profiles Operator (SPO) provides a way to define secure computing (seccomp) profiles and SELinux profiles as custom resources, synchronizing profiles to every node in a given namespace. For the latest updates, see the release notes.

The SPO can distribute custom resources to each node while a reconciliation loop ensures that the profiles stay up-to-date. See Understanding the Security Profiles Operator.

The SPO manages SELinux policies and seccomp profiles for namespaced workloads. For more information, see Enabling the Security Profiles Operator.

You can create seccomp and SELinux profiles, bind policies to pods, record workloads, and synchronize all worker nodes in a namespace.

Use Advanced Security Profile Operator tasks to enable the log enricher, configure webhooks and metrics, or restrict profiles to a single namespace.

Use SPO Advanced Audit Logging to access logs in RHCOS containers for container-level security audit features.

Troubleshoot the Security Profiles Operator as needed, or engage Red Hat support.

You can Uninstall the Security Profiles Operator by removing the profiles before removing the Operator.

7.2. Security Profiles Operator release notes

The Security Profiles Operator provides a way to define secure computing (seccomp) and SELinux profiles as custom resources, synchronizing profiles to every node in a given namespace.

These release notes track the development of the Security Profiles Operator in OpenShift Container Platform.

For an overview of the Security Profiles Operator, see Security Profiles Operator Overview.

7.2.1. Security Profiles Operator 0.10.0

The following advisory is available for the Security Profiles Operator 0.10.0: RHSA-2026:2852 - OpenShift Security Profiles Operator update

7.2.1.1. Bug fixes

  • In some instances when Security Profiles Operator (SPO) 0.9.0 was used with OpenShift Container Platform version 4.20 and above, SPO would create the profilerecording resource but the workload would fail. Failure of the workload prevented the creation of the needed container for running the Operator. With the 0.10.0 release of SPO. the profilerecording resource is reliably created, therefore the needed container for running the Operator is reliably created. CMP-3537.
  • For version 0.9.0 of Security Profiles Operator (SPO), the spod pods would fail to run with the error message fsmount:fscontext:proc/: could not get mount id: operation not permitted. With the release of version 0.10.0, the spod pods run reliably. CMP-4007.
  • In releases of SPO 0.9.0 and earlier, there was a bug in syntax of the selinux usage. With this release of SPO, the change is from <policyName>_.process to <policyName>.process. The new syntax omits the _. Examples in the documentation now show this updated usage. CMP-4104

7.2.1.2. New features and enhancements

  • With the release of SPO v0.10.0, the Operator now supports Red Hat Enterprise Linux CoreOS (RHCOS) 10 containers. CMP-4033
  • In this release of the Security Profiles Operator, the Advanced Audit Logging Framework is available as a General Availability (GA) feature. The Advanced Audit Logging Framework uses the Audit JSON Log Enricher to capture and log terminal-based command activity in Red Hat Enterprise Linux CoreOS (RHCOS) containers, including oc rsh, oc exec, and oc debug commands.

7.2.2. Security Profiles Operator 0.9.0

The following advisory is available for the Security Profiles Operator 0.9.0: RHBA-2025:15655 - OpenShift Security Profiles Operator update

This update manages security profiles as cluster-wide resources rather than namespace resources. To update Security Profiles Operator to a version later than 0.8.6 requires manual migration. For migration instructions, see Security Profiles Operator 0.9.0 Update Migration Guide.

7.2.2.1. Bug fixes

  • Before this update, the spod pods could fail to start and enter into a CrashLoopBackOff state due to an error in parsing the semanage configuration file. This issue is caused by a change to the RHEL 9 image naming convention beginning in OpenShift Container Platform 4.19. (OCPBUGS-55829)
  • Before this update, the Security Profiles Operator would fail to apply a RawSelinuxProfile to newly added nodes due to a reconciler type mismatch error. With this update, the operator now correctly handles RawSelinuxProfile objects and policies are applied to all nodes as expected. (OCPBUGS-33718)

7.2.3. Security Profiles Operator 0.8.6

The following advisory is available for the Security Profiles Operator 0.8.6:

This update includes upgraded dependencies in underlying base images.

7.2.4. Security Profiles Operator 0.8.5

The following advisory is available for the Security Profiles Operator 0.8.5:

7.2.4.1. Bug fixes

  • When attempting to install the Security Profile Operator from the web console, the option to enable Operator-recommended cluster monitoring was unavailable for the namespace. With this update, you can now enabled Operator-recommend cluster monitoring in the namespace. (OCPBUGS-37794)
  • Previously, the Security Profiles Operator would intermittently be not visible in the OperatorHub, which caused limited access to install the Operator via the web console. With this update, the Security Profiles Operator is present in the OperatorHub.

7.2.5. Security Profiles Operator 0.8.4

The following advisory is available for the Security Profiles Operator 0.8.4:

This update addresses CVEs in underlying dependencies.

7.2.5.1. New features and enhancements

7.2.6. Security Profiles Operator 0.8.2

The following advisory is available for the Security Profiles Operator 0.8.2:

7.2.6.1. Bug fixes

  • Previously, SELinuxProfile objects did not inherit custom attributes from the same namespace. With this update, the issue has now been resolved and SELinuxProfile object attributes are inherited from the same namespace as expected. (OCPBUGS-17164)
  • Previously, RawSELinuxProfiles would hang during the creation process and would not reach an Installed state. With this update, the issue has been resolved and RawSELinuxProfiles are created successfully. (OCPBUGS-19744)
  • Previously, patching the enableLogEnricher to true would cause the seccompProfile log-enricher-trace pods to be stuck in a Pending state. With this update, log-enricher-trace pods reach an Installed state as expected. (OCPBUGS-22182)
  • Previously, the Security Profiles Operator generated high cardinality metrics, causing Prometheus pods using high amounts of memory. With this update, the following metrics will no longer apply in the Security Profiles Operator namespace:

    • rest_client_request_duration_seconds
    • rest_client_request_size_bytes
    • rest_client_response_size_bytes

      (OCPBUGS-22406)

7.2.7. Security Profiles Operator 0.8.0

The following advisory is available for the Security Profiles Operator 0.8.0:

7.2.7.1. Bug fixes

  • Previously, while trying to install Security Profiles Operator in a disconnected cluster, the secure hashes provided were incorrect due to a SHA relabeling issue. With this update, the SHAs provided work consistently with disconnected environments. (OCPBUGS-14404)

7.2.8. Security Profiles Operator 0.7.1

The following advisory is available for the Security Profiles Operator 0.7.1:

7.2.8.1. New features and enhancements

  • Security Profiles Operator (SPO) now automatically selects the appropriate selinuxd image for RHEL 8- and 9-based Red Hat Enterprise Linux CoreOS (RHCOS) systems.

    Important

    Users that mirror images for disconnected environments must mirror both selinuxd images provided by the Security Profiles Operator.

  • You can now enable memory optimization inside of an spod daemon. For more information, see Enabling memory optimization in the spod daemon.

    Note

    SPO memory optimization is not enabled by default.

  • The daemon resource requirements are now configurable. For more information, see Customizing daemon resource requirements.
  • The priority class name is now configurable in the spod configuration. For more information, see Setting a custom priority class name for the spod daemon pod.

7.2.8.2. Deprecated and removed features

  • The default nginx-1.19.1 seccomp profile is now removed from the Security Profiles Operator deployment.

7.2.8.3. Bug fixes

  • Previously, a Security Profiles Operator (SPO) SELinux policy did not inherit low-level policy definitions from the container template. If you selected another template, such as net_container, the policy would not work because it required low-level policy definitions that only existed in the container template. This issue occurred when the SPO SELinux policy attempted to translate SELinux policies from the SPO custom format to the Common Intermediate Language (CIL) format. With this update, the container template appends to any SELinux policies that require translation from SPO to CIL. Additionally, the SPO SELinux policy can inherit low-level policy definitions from any supported policy template. (OCPBUGS-12879)

7.2.8.4. Known issue

  • When uninstalling the Security Profiles Operator, the MutatingWebhookConfiguration object is not deleted and must be manually removed. As a workaround, delete the MutatingWebhookConfiguration object after uninstalling the Security Profiles Operator. These steps are defined in Uninstalling the Security Profiles Operator. (OCPBUGS-4687)

7.2.9. Security Profiles Operator 0.5.2

The following advisory is available for the Security Profiles Operator 0.5.2:

This update addresses a CVE in an underlying dependency.

7.2.9.1. Known issue

  • When uninstalling the Security Profiles Operator, the MutatingWebhookConfiguration object is not deleted and must be manually removed. As a workaround, delete the MutatingWebhookConfiguration object after uninstalling the Security Profiles Operator. These steps are defined in Uninstalling the Security Profiles Operator. (OCPBUGS-4687)

7.2.10. Security Profiles Operator 0.5.0

The following advisory is available for the Security Profiles Operator 0.5.0:

7.2.10.1. Known issue

  • When uninstalling the Security Profiles Operator, the MutatingWebhookConfiguration object is not deleted and must be manually removed. As a workaround, delete the MutatingWebhookConfiguration object after uninstalling the Security Profiles Operator. These steps are defined in Uninstalling the Security Profiles Operator. (OCPBUGS-4687)

7.3. Security Profiles Operator support

7.3.1. Security Profiles Operator lifecycle

The Security Profiles Operator is a "Rolling Stream" Operator, meaning updates are available asynchronously of OpenShift Container Platform releases. For more information, see OpenShift Operator Life Cycles on the Red Hat Customer Portal.

7.3.2. Getting support

If you experience difficulty with a procedure described in this documentation, or with OpenShift Container Platform in general, visit the Red Hat Customer Portal.

From the Customer Portal, you can:

  • Search or browse through the Red Hat Knowledgebase of articles and solutions relating to Red Hat products.
  • Submit a support case to Red Hat Support.
  • Access other product documentation.

To identify issues with your cluster, you can use Red Hat Lightspeed in OpenShift Cluster Manager. Red Hat Lightspeed provides details about issues and, if available, information on how to solve a problem.

If you have a suggestion for improving this documentation or have found an error, submit a Jira issue for the most relevant documentation component. Please provide specific details, such as the section name and OpenShift Container Platform version.

7.4. Understanding the Security Profiles Operator

OpenShift Container Platform administrators can use the Security Profiles Operator to define increased security measures in clusters.

Important

The Security Profiles Operator supports only Red Hat Enterprise Linux CoreOS (RHCOS) worker nodes. Red Hat Enterprise Linux (RHEL) nodes are not supported.

7.4.1. About Security Profiles

Security profiles can increase security at the container level in your cluster.

Seccomp security profiles list the syscalls a process can make. Permissions are broader than SELinux, enabling users to restrict operations system-wide, such as write.

SELinux security profiles provide a label-based system that restricts the access and usage of processes, applications, or files in a system. All files in an environment have labels that define permissions. SELinux profiles can define access within a given structure, such as directories.

7.5. Enabling the Security Profiles Operator

Before you can use the Security Profiles Operator, you must ensure the Operator is deployed in the cluster.

Important

All cluster nodes must have the same release version in order for this Operator to function properly. As an example, for nodes running RHCOS, all nodes must have the same RHCOS version.

Important

The Security Profiles Operator supports only Red Hat Enterprise Linux CoreOS (RHCOS) worker nodes. Red Hat Enterprise Linux (RHEL) nodes are not supported.

Important

The Security Profiles Operator supports x86_64 and ppc64le architecture.

7.5.1. Installing the Security Profiles Operator

Prerequisites

  • You must have access to the web console as a user with cluster-admin privileges.

Procedure

  1. In the OpenShift Container Platform web console, navigate to Ecosystem Software Catalog.
  2. Search for the Security Profiles Operator, then click Install.
  3. Keep the default selection of Installation mode and namespace to ensure that the Operator will be installed to the openshift-security-profiles namespace.
  4. Click Install.

Verification

To confirm that the installation is successful:

  1. Navigate to the Ecosystem Installed Operators page.
  2. Check that the Security Profiles Operator is installed in the openshift-security-profiles namespace and its status is Succeeded.

If the Operator is not installed successfully:

  1. Navigate to the Ecosystem Installed Operators page and inspect the Status column for any errors or failures.
  2. Navigate to the Workloads Pods page and check the logs in any pods in the openshift-security-profiles project that are reporting issues.

7.5.2. Installing the Security Profiles Operator using the CLI

Prerequisites

  • You must have cluster-admin privileges.

Procedure

  1. Define a Namespace object:

    Example namespace-object.yaml

    apiVersion: v1
    kind: Namespace
    metadata:
        name: openshift-security-profiles
    labels:
      openshift.io/cluster-monitoring: "true"
    Copy to Clipboard Toggle word wrap

  2. Create the Namespace object:

    $ oc create -f namespace-object.yaml
    Copy to Clipboard Toggle word wrap
  3. Define an OperatorGroup object:

    Example operator-group-object.yaml

    apiVersion: operators.coreos.com/v1
    kind: OperatorGroup
    metadata:
      name: security-profiles-operator
      namespace: openshift-security-profiles
    Copy to Clipboard Toggle word wrap

  4. Create the OperatorGroup object:

    $ oc create -f operator-group-object.yaml
    Copy to Clipboard Toggle word wrap
  5. Define a Subscription object:

    Example subscription-object.yaml

    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: security-profiles-operator-sub
      namespace: openshift-security-profiles
    spec:
      channel: release-alpha-rhel-8
      installPlanApproval: Automatic
      name: security-profiles-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
    Copy to Clipboard Toggle word wrap

  6. Create the Subscription object:

    $ oc create -f subscription-object.yaml
    Copy to Clipboard Toggle word wrap
Note

If you are setting the global scheduler feature and enable defaultNodeSelector, you must create the namespace manually and update the annotations of the openshift-security-profiles namespace, or the namespace where the Security Profiles Operator was installed, with openshift.io/node-selector: “”. This removes the default node selector and prevents deployment failures.

Verification

  1. Verify the installation succeeded by inspecting the following CSV file:

    $ oc get csv -n openshift-security-profiles
    Copy to Clipboard Toggle word wrap
  2. Verify that the Security Profiles Operator is operational by running the following command:

    $ oc get deploy -n openshift-security-profiles
    Copy to Clipboard Toggle word wrap

7.5.3. Configuring logging verbosity

The Security Profiles Operator supports the default logging verbosity of 0 and an enhanced verbosity of 1.

Procedure

  • To enable enhanced logging verbosity, patch the spod configuration and adjust the value by running the following command:

    $ oc -n openshift-security-profiles patch spod \
        spod --type=merge -p '{"spec":{"verbosity":1}}'
    Copy to Clipboard Toggle word wrap

    Example output

    securityprofilesoperatordaemon.security-profiles-operator.x-k8s.io/spod patched
    Copy to Clipboard Toggle word wrap

7.6. Managing seccomp profiles

Create and manage seccomp profiles and bind them to workloads.

Important

The Security Profiles Operator supports only Red Hat Enterprise Linux CoreOS (RHCOS) worker nodes. Red Hat Enterprise Linux (RHEL) nodes are not supported.

7.6.1. Creating seccomp profiles

Use the SeccompProfile object to create profiles.

SeccompProfile objects can restrict syscalls within a container, limiting the access of your application.

Procedure

  1. Create a project by running the following command:

    $ oc new-project my-namespace
    Copy to Clipboard Toggle word wrap
  2. Create the SeccompProfile object:

    apiVersion: security-profiles-operator.x-k8s.io/v1beta1
    kind: SeccompProfile
    metadata:
      name: profile1
    spec:
      defaultAction: SCMP_ACT_LOG
    Copy to Clipboard Toggle word wrap

The seccomp profile will be saved in /var/lib/kubelet/seccomp/operator/<namespace>/<name>.json.

An init container creates the root directory of the Security Profiles Operator to run the Operator without root group or user ID privileges. A symbolic link is created from the rootless profile storage /var/lib/openshift-security-profiles to the default seccomp root path inside of the kubelet root /var/lib/kubelet/seccomp/operator.

7.6.2. Applying seccomp profiles to a pod

Create a pod to apply one of the created profiles.

Procedure

  1. Create a pod object that defines a securityContext:

    apiVersion: v1
    kind: Pod
    metadata:
      name: test-pod
    spec:
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: Localhost
          localhostProfile: operator/profile1.json
      containers:
        - name: test-container
          image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
    Copy to Clipboard Toggle word wrap
  2. View the profile path of the seccompProfile.localhostProfile attribute by running the following command:

    $ oc get seccompprofile profile1 --output wide
    Copy to Clipboard Toggle word wrap

    Example output

    NAME       STATUS     AGE   SECCOMPPROFILE.LOCALHOSTPROFILE
    profile1   Installed  14s   operator/profile1.json
    Copy to Clipboard Toggle word wrap

  3. View the path to the localhost profile by running the following command:

    $ oc get sp profile1 --output=jsonpath='{.status.localhostProfile}'
    Copy to Clipboard Toggle word wrap

    Example output

    operator/profile1.json
    Copy to Clipboard Toggle word wrap

  4. Apply the localhostProfile output to the patch file:

    spec:
      template:
        spec:
          securityContext:
            seccompProfile:
              type: Localhost
              localhostProfile: operator/profile1.json
    Copy to Clipboard Toggle word wrap
  5. Apply the profile to any other workload, such as a Deployment object, by running the following command:

    $ oc -n my-namespace patch deployment myapp --patch-file patch.yaml --type=merge
    Copy to Clipboard Toggle word wrap

    Example output

    deployment.apps/myapp patched
    Copy to Clipboard Toggle word wrap

Verification

  • Confirm the profile was applied correctly by running the following command:

    $ oc -n my-namespace get deployment myapp --output=jsonpath='{.spec.template.spec.securityContext}' | jq .
    Copy to Clipboard Toggle word wrap

    Example output

    {
      "seccompProfile": {
        "localhostProfile": "operator/profile1.json",
        "type": "localhost"
      }
    }
    Copy to Clipboard Toggle word wrap

7.6.2.1. Binding workloads to profiles with ProfileBindings

You can use the ProfileBinding resource to bind a security profile to the SecurityContext of a container.

Procedure

  1. To bind a pod that uses a quay.io/security-profiles-operator/test-nginx-unprivileged:1.21 image to the example SeccompProfile profile, create a ProfileBinding object in the same namespace with the pod and the SeccompProfile objects:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileBinding
    metadata:
      namespace: my-namespace
      name: nginx-binding
    spec:
      profileRef:
        kind: SeccompProfile 
    1
    
        name: profile 
    2
    
      image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21 
    3
    Copy to Clipboard Toggle word wrap
    1
    The kind: variable refers to the kind of the profile.
    2
    The name: variable refers to the name of the profile.
    3
    You can enable a default security profile by using a wildcard in the image attribute: image: "*"
    Important

    Using the image: "*" wildcard attribute binds all new pods with a default security profile in a given namespace.

  2. Label the namespace with enable-binding=true by running the following command:

    $ oc label ns my-namespace spo.x-k8s.io/enable-binding=true
    Copy to Clipboard Toggle word wrap
  3. Define a pod named test-pod.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: test-pod
    spec:
      containers:
      - name: test-container
        image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
    Copy to Clipboard Toggle word wrap
  4. Create the pod:

    $ oc create -f test-pod.yaml
    Copy to Clipboard Toggle word wrap
    Note

    If the pod already exists, you must re-create the pod for the binding to work properly.

Verification

  • Confirm the pod inherits the ProfileBinding by running the following command:

    $ oc get pod test-pod -o jsonpath='{.spec.containers[*].securityContext.seccompProfile}'
    Copy to Clipboard Toggle word wrap

    Example output

    {"localhostProfile":"operator/profile.json","type":"Localhost"}
    Copy to Clipboard Toggle word wrap

7.6.3. Recording profiles from workloads

The Security Profiles Operator can record system calls with ProfileRecording objects, making it easier to create baseline profiles for applications.

When using the log enricher for recording seccomp profiles, verify the log enricher feature is enabled. See Additional resources for more information.

Note

A container with privileged: true security context restraints prevents log-based recording. Privileged containers are not subject to seccomp policies, and log-based recording makes use of a special seccomp profile to record events.

Procedure

  1. Create a project by running the following command:

    $ oc new-project my-namespace
    Copy to Clipboard Toggle word wrap
  2. Label the namespace with enable-recording=true by running the following command:

    $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true
    Copy to Clipboard Toggle word wrap
  3. Create a ProfileRecording object containing a recorder: logs variable:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileRecording
    metadata:
      namespace: my-namespace
      name: test-recording
    spec:
      kind: SeccompProfile
      recorder: logs
      podSelector:
        matchLabels:
          app: my-app
    Copy to Clipboard Toggle word wrap
  4. Create a workload to record:

    apiVersion: v1
    kind: Pod
    metadata:
      namespace: my-namespace
      name: my-pod
      labels:
        app: my-app
    spec:
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: nginx
          image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
          ports:
            - containerPort: 8080
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
        - name: redis
          image: quay.io/security-profiles-operator/redis:6.2.1
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
    Copy to Clipboard Toggle word wrap
  5. Confirm the pod is in a Running state by entering the following command:

    $ oc -n my-namespace get pods
    Copy to Clipboard Toggle word wrap

    Example output

    NAME     READY   STATUS    RESTARTS   AGE
    my-pod   2/2     Running   0          18s
    Copy to Clipboard Toggle word wrap

  6. Confirm the enricher indicates that it receives audit logs for those containers:

    $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher
    Copy to Clipboard Toggle word wrap

    Example output

    I0523 14:19:08.747313  430694 enricher.go:445] log-enricher "msg"="audit" "container"="redis" "executable"="/usr/local/bin/redis-server" "namespace"="my-namespace" "node"="xiyuan-23-5g2q9-worker-eastus2-6rpgf" "pid"=656802 "pod"="my-pod" "syscallID"=0 "syscallName"="read" "timestamp"="1684851548.745:207179" "type"="seccomp"
    Copy to Clipboard Toggle word wrap

Verification

  1. Remove the pod:

    $ oc -n my-namespace delete pod my-pod
    Copy to Clipboard Toggle word wrap
  2. Confirm the Security Profiles Operator reconciles the two seccomp profiles:

    $ oc get seccompprofiles -lspo.x-k8s.io/recording-id=test-recording
    Copy to Clipboard Toggle word wrap

    Example output for seccompprofile

    NAME                   STATUS      AGE
    test-recording-nginx   Installed   2m48s
    test-recording-redis   Installed   2m48s
    Copy to Clipboard Toggle word wrap

7.6.3.1. Merging per-container profile instances

By default, each container instance records into a separate profile. The Security Profiles Operator can merge the per-container profiles into a single profile. Merging profiles is useful when deploying applications using ReplicaSet or Deployment objects.

Procedure

  1. Edit a ProfileRecording object to include a mergeStrategy: containers variable:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileRecording
    metadata:
      # The name of the Recording is the same as the resulting SeccompProfile CRD
      # after reconciliation.
      name: test-recording
      namespace: my-namespace
    spec:
      kind: SeccompProfile
      recorder: logs
      mergeStrategy: containers
      podSelector:
        matchLabels:
          app: sp-record
    Copy to Clipboard Toggle word wrap
  2. Label the namespace by running the following command:

    $ oc label ns my-namespace security.openshift.io/scc.podSecurityLabelSync=false pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/audit=privileged pod-security.kubernetes.io/warn=privileged --overwrite=true
    Copy to Clipboard Toggle word wrap
  3. Create the workload with the following YAML:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-deploy
      namespace: my-namespace
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: sp-record
      template:
        metadata:
          labels:
            app: sp-record
        spec:
          serviceAccountName: spo-record-sa
          containers:
          - name: nginx-record
            image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
            ports:
            - containerPort: 8080
    Copy to Clipboard Toggle word wrap
  4. To record the individual profiles, delete the deployment by running the following command:

    $ oc delete deployment nginx-deploy -n my-namespace
    Copy to Clipboard Toggle word wrap
  5. To merge the profiles, delete the profile recording by running the following command:

    $ oc delete profilerecording test-recording -n my-namespace
    Copy to Clipboard Toggle word wrap
  6. To start the merge operation and generate the results profile, run the following command:

    $ oc get seccompprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace
    Copy to Clipboard Toggle word wrap

    Example output for seccompprofiles

    NAME                          STATUS       AGE
    test-recording-nginx-record   Installed    55s
    Copy to Clipboard Toggle word wrap

  7. To view the permissions used by any of the containers, run the following command:

    $ oc get seccompprofiles test-recording-nginx-record -o yaml
    Copy to Clipboard Toggle word wrap

7.7. Managing SELinux profiles

Create and manage SELinux profiles and bind them to workloads.

Important

The Security Profiles Operator supports only Red Hat Enterprise Linux CoreOS (RHCOS) worker nodes. Red Hat Enterprise Linux (RHEL) nodes are not supported.

7.7.1. Creating SELinux profiles

Use the SelinuxProfile object to create profiles.

The SelinuxProfile object has several features that allow for better security hardening and readability:

  • Restricts the profiles to inherit from to the current namespace or a system-wide profile. Because there are typically many profiles installed on the system, but only a subset should be used by cluster workloads, the inheritable system profiles are listed in the spod instance in spec.selinuxOptions.allowedSystemProfiles.
  • Performs basic validation of the permissions, classes and labels.
  • Adds a new keyword @self that describes the process using the policy. This allows reusing a policy between workloads and namespaces easily, as the usage of the policy is based on the name and namespace.
  • Adds features for better security hardening and readability compared to writing a profile directly in the SELinux CIL language.

Procedure

  1. Create a project by running the following command:

    $ oc new-project nginx-deploy
    Copy to Clipboard Toggle word wrap
  2. Create a policy that can be used with a non-privileged workload by creating the following SelinuxProfile object:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha2
    kind: SelinuxProfile
    metadata:
      name: nginx-secure
    spec:
      allow:
        '@self':
          tcp_socket:
          - listen
        http_cache_port_t:
          tcp_socket:
          - name_bind
        node_t:
          tcp_socket:
          - node_bind
      inherit:
      - kind: System
        name: container
    Copy to Clipboard Toggle word wrap
  3. Wait for selinuxd to install the policy by running the following command:

    $ oc wait --for=condition=ready selinuxprofile nginx-secure
    Copy to Clipboard Toggle word wrap

    Example output

    selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure condition met
    Copy to Clipboard Toggle word wrap

    The policies are placed into an emptyDir in the container owned by the Security Profiles Operator. The policies are saved in Common Intermediate Language (CIL) format in /etc/selinux.d/<name>_<namespace>.cil.

  4. Access the pod by running the following command:

    $ oc -n openshift-security-profiles rsh -c selinuxd ds/spod
    Copy to Clipboard Toggle word wrap

Verification

  1. View the file contents with cat by running the following command:

    $ cat /etc/selinux.d/nginx-secure.cil
    Copy to Clipboard Toggle word wrap

    Example output

    (block nginx-secure
    (blockinherit container)
    (allow process nginx-secure.process ( tcp_socket ( listen )))
    (allow process http_cache_port_t ( tcp_socket ( name_bind )))
    (allow process node_t ( tcp_socket ( node_bind )))
    )
    Copy to Clipboard Toggle word wrap

  2. Verify that a policy has been installed by running the following command:

    $ semodule -l | grep nginx-secure
    Copy to Clipboard Toggle word wrap

    Example output

    nginx-secure
    Copy to Clipboard Toggle word wrap

7.7.2. Applying SELinux profiles to a pod

Create a pod to apply one of the created profiles.

For SELinux profiles, the namespace must be labelled to allow privileged workloads.

Procedure

  1. Apply the scc.podSecurityLabelSync=false label to the nginx-deploy namespace by running the following command:

    $ oc label ns nginx-deploy security.openshift.io/scc.podSecurityLabelSync=false
    Copy to Clipboard Toggle word wrap
  2. Apply the privileged label to the nginx-deploy namespace by running the following command:

    $ oc label ns nginx-deploy --overwrite=true pod-security.kubernetes.io/enforce=privileged
    Copy to Clipboard Toggle word wrap
  3. Obtain the SELinux profile usage string by running the following command:

    $ oc get selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure -ojsonpath='{.status.usage}'
    Copy to Clipboard Toggle word wrap

    Example output

    nginx-secure.process
    Copy to Clipboard Toggle word wrap

  4. Apply the output string in the workload manifest in the .spec.containers[].securityContext.seLinuxOptions attribute:

    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx-secure
      namespace: nginx-deploy
    spec:
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - image: nginxinc/nginx-unprivileged:1.21
          name: nginx
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
            seLinuxOptions:
              # NOTE: This uses an appropriate SELinux type
              type: nginx-secure.process
    Copy to Clipboard Toggle word wrap
    Important

    The SELinux type must exist before creating the workload.

7.7.2.1. Applying SELinux log policies

To log policy violations or AVC denials, set the SElinuxProfile profile to permissive.

Important

This procedure defines logging policies. It does not set enforcement policies.

Procedure

  • Add permissive: true to an SElinuxProfile:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha2
    kind: SelinuxProfile
    metadata:
      name: nginx-secure
    spec:
      permissive: true
    Copy to Clipboard Toggle word wrap

7.7.2.2. Binding workloads to profiles with ProfileBindings

You can use the ProfileBinding resource to bind a security profile to the SecurityContext of a container.

Procedure

  1. To bind a pod that uses a quay.io/security-profiles-operator/test-nginx-unprivileged:1.21 image to the example SelinuxProfile profile, create a ProfileBinding object in the same namespace with the pod and the SelinuxProfile objects:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileBinding
    metadata:
      namespace: my-namespace
      name: nginx-binding
    spec:
      profileRef:
        kind: SelinuxProfile 
    1
    
        name: profile 
    2
    
      image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21 
    3
    Copy to Clipboard Toggle word wrap
    1
    The kind: variable refers to the kind of the profile.
    2
    The name: variable refers to the name of the profile.
    3
    You can enable a default security profile by using a wildcard in the image attribute: image: "*"
    Important

    Using the image: "*" wildcard attribute binds all new pods with a default security profile in a given namespace.

  2. Label the namespace with enable-binding=true by running the following command:

    $ oc label ns my-namespace spo.x-k8s.io/enable-binding=true
    Copy to Clipboard Toggle word wrap
  3. Define a pod named test-pod.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: test-pod
    spec:
      containers:
      - name: test-container
        image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
    Copy to Clipboard Toggle word wrap
  4. Create the pod:

    $ oc create -f test-pod.yaml
    Copy to Clipboard Toggle word wrap
    Note

    If the pod already exists, you must re-create the pod for the binding to work properly.

Verification

  • Confirm the pod inherits the ProfileBinding by running the following command:

    $ oc get pod test-pod -o jsonpath='{.spec.containers[*].securityContext.seLinuxOptions.type}'
    Copy to Clipboard Toggle word wrap

    Example output

    profile.process
    Copy to Clipboard Toggle word wrap

7.7.2.3. Replicating controllers and SecurityContextConstraints

When you deploy SELinux policies for replicating controllers, such as deployments or daemon sets, note that the Pod objects spawned by the controllers are not running with the identity of the user who creates the workload. Unless a ServiceAccount is selected, the pods might revert to using a restricted SecurityContextConstraints (SCC) which does not allow use of custom security policies.

Procedure

  1. Create a project by running the following command:

    $ oc new-project nginx-secure
    Copy to Clipboard Toggle word wrap
  2. Create the following RoleBinding object to allow SELinux policies to be used in the nginx-secure namespace:

    kind: RoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: spo-nginx
      namespace: nginx-secure
    subjects:
    - kind: ServiceAccount
      name: spo-deploy-test
    roleRef:
      kind: Role
      name: spo-nginx
      apiGroup: rbac.authorization.k8s.io
    Copy to Clipboard Toggle word wrap
  3. Create the Role object:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      creationTimestamp: null
      name: spo-nginx
      namespace: nginx-secure
    rules:
    - apiGroups:
      - security.openshift.io
      resources:
      - securitycontextconstraints
      resourceNames:
      - privileged
      verbs:
      - use
    Copy to Clipboard Toggle word wrap
  4. Create the ServiceAccount object:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      creationTimestamp: null
      name: spo-deploy-test
      namespace: nginx-secure
    Copy to Clipboard Toggle word wrap
  5. Create the Deployment object:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: selinux-test
      namespace: nginx-secure
      metadata:
        labels:
          app: selinux-test
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: selinux-test
      template:
        metadata:
          labels:
            app: selinux-test
        spec:
          serviceAccountName: spo-deploy-test
          securityContext:
            seLinuxOptions:
              type: nginx-secure.process 
    1
    
          containers:
          - name: nginx-unpriv
            image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
            ports:
            - containerPort: 8080
    Copy to Clipboard Toggle word wrap
    1
    The .seLinuxOptions.type must exist before the Deployment is created.
    Note

    The SELinux type is not specified in the workload and is handled by the SCC. When the pods are created by the deployment and the ReplicaSet, the pods will run with the appropriate profile.

Ensure that your SCC is usable by only the correct service account. Refer to Additional resources for more information.

7.7.3. Recording profiles from workloads

The Security Profiles Operator can record system calls with ProfileRecording objects, making it easier to create baseline profiles for applications.

When using the log enricher for recording SELinux profiles, verify the log enricher feature is enabled. See Additional resources for more information.

Note

A container with privileged: true security context restraints prevents log-based recording. Privileged containers are not subject to SELinux policies, and log-based recording makes use of a special SELinux profile to record events.

Procedure

  1. Create a project by running the following command:

    $ oc new-project my-namespace
    Copy to Clipboard Toggle word wrap
  2. Label the namespace with enable-recording=true by running the following command:

    $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true
    Copy to Clipboard Toggle word wrap
  3. Create a ProfileRecording object containing a recorder: logs variable:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileRecording
    metadata:
      namespace: my-namespace
      name: test-recording
    spec:
      kind: SelinuxProfile
      recorder: logs
      podSelector:
        matchLabels:
          app: my-app
    Copy to Clipboard Toggle word wrap
  4. Create a workload to record:

    apiVersion: v1
    kind: Pod
    metadata:
      namespace: my-namespace
      name: my-pod
      labels:
        app: my-app
    spec:
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: nginx
          image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
          ports:
            - containerPort: 8080
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
        - name: redis
          image: quay.io/security-profiles-operator/redis:6.2.1
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
    Copy to Clipboard Toggle word wrap
  5. Confirm the pod is in a Running state by entering the following command:

    $ oc -n my-namespace get pods
    Copy to Clipboard Toggle word wrap

    Example output

    NAME     READY   STATUS    RESTARTS   AGE
    my-pod   2/2     Running   0          18s
    Copy to Clipboard Toggle word wrap

  6. Confirm the enricher indicates that it receives audit logs for those containers:

    $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher
    Copy to Clipboard Toggle word wrap

    Example output

    I0517 13:55:36.383187  348295 enricher.go:376] log-enricher "msg"="audit" "container"="redis" "namespace"="my-namespace" "node"="ip-10-0-189-53.us-east-2.compute.internal" "perm"="name_bind" "pod"="my-pod" "profile"="test-recording_redis_6kmrb_1684331729" "scontext"="system_u:system_r:selinuxrecording.process:s0:c4,c27" "tclass"="tcp_socket" "tcontext"="system_u:object_r:redis_port_t:s0" "timestamp"="1684331735.105:273965" "type"="selinux"
    Copy to Clipboard Toggle word wrap

Verification

  1. Remove the pod:

    $ oc -n my-namespace delete pod my-pod
    Copy to Clipboard Toggle word wrap
  2. Confirm the Security Profiles Operator reconciles the two SELinux profiles:

    $ oc get selinuxprofiles -lspo.x-k8s.io/recording-id=test-recording
    Copy to Clipboard Toggle word wrap

    Example output for selinuxprofile

    NAME                   USAGE                                 STATE
    test-recording-nginx   test-recording-nginx.process   Installed
    test-recording-redis   test-recording-redis.process   Installed
    Copy to Clipboard Toggle word wrap

7.7.3.1. Merging per-container profile instances

By default, each container instance records into a separate profile. The Security Profiles Operator can merge the per-container profiles into a single profile. Merging profiles is useful when deploying applications using ReplicaSet or Deployment objects.

Procedure

  1. Edit a ProfileRecording object to include a mergeStrategy: containers variable:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileRecording
    metadata:
      # The name of the Recording is the same as the resulting SelinuxProfile CRD
      # after reconciliation.
      name: test-recording
      namespace: my-namespace
    spec:
      kind: SelinuxProfile
      recorder: logs
      mergeStrategy: containers
      podSelector:
        matchLabels:
          app: sp-record
    Copy to Clipboard Toggle word wrap
  2. Label the namespace by running the following command:

    $ oc label ns my-namespace security.openshift.io/scc.podSecurityLabelSync=false pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/audit=privileged pod-security.kubernetes.io/warn=privileged --overwrite=true
    Copy to Clipboard Toggle word wrap
  3. Create the workload with the following YAML:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-deploy
      namespace: my-namespace
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: sp-record
      template:
        metadata:
          labels:
            app: sp-record
        spec:
          serviceAccountName: spo-record-sa
          containers:
          - name: nginx-record
            image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
            ports:
            - containerPort: 8080
    Copy to Clipboard Toggle word wrap
  4. To record the individual profiles, delete the deployment by running the following command:

    $ oc delete deployment nginx-deploy -n my-namespace
    Copy to Clipboard Toggle word wrap
  5. To merge the profiles, delete the profile recording by running the following command:

    $ oc delete profilerecording test-recording -n my-namespace
    Copy to Clipboard Toggle word wrap
  6. To start the merge operation and generate the results profile, run the following command:

    $ oc get selinuxprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace
    Copy to Clipboard Toggle word wrap

    Example output for selinuxprofiles

    NAME                          USAGE                            STATE
    test-recording-nginx-record   test-recording-nginx-record.process   Installed
    Copy to Clipboard Toggle word wrap

  7. To view the permissions used by any of the containers, run the following command:

    $ oc get selinuxprofiles test-recording-nginx-record -o yaml
    Copy to Clipboard Toggle word wrap

7.7.3.2. About seLinuxContext: RunAsAny

Recording of SELinux policies is implemented with a webhook that injects a special SELinux type to the pods being recorded. The SELinux type makes the pod run in permissive mode, logging all the AVC denials into audit.log. By default, a workload is not allowed to run with a custom SELinux policy, but uses an auto-generated type.

To record a workload, the workload must use a service account that has permissions to use an SCC that allows the webhook to inject the permissive SELinux type. The privileged SCC contains seLinuxContext: RunAsAny.

In addition, the namespace must be labeled with pod-security.kubernetes.io/enforce: privileged if your cluster enables the Pod Security Admission because only the privileged Pod Security Standard allows using a custom SELinux policy.

7.8. Advanced Security Profiles Operator tasks

Use advanced tasks to enable metrics, configure webhooks, or restrict syscalls.

7.8.1. Restrict the allowed syscalls in seccomp profiles

The Security Profiles Operator does not restrict syscalls in seccomp profiles by default. You can define the list of allowed syscalls in the spod configuration.

Procedure

  • To define the list of allowedSyscalls, adjust the spec parameter by running the following command:

    $ oc -n openshift-security-profiles patch spod spod --type merge \
        -p '{"spec":{"allowedSyscalls": ["exit", "exit_group", "futex", "nanosleep"]}}'
    Copy to Clipboard Toggle word wrap
Important

The Operator will install only the seccomp profiles, which have a subset of syscalls defined into the allowed list. All profiles not complying with this ruleset are rejected.

When the list of allowed syscalls is modified in the spod configuration, the Operator will identify the already installed profiles which are non-compliant and remove them automatically.

7.8.2. Base syscalls for a container runtime

You can use the baseProfileName attribute to establish the minimum required syscalls for a given runtime to start a container.

Procedure

  • Edit the SeccompProfile kind object and add baseProfileName: runc-v1.0.0 to the spec field:

    apiVersion: security-profiles-operator.x-k8s.io/v1beta1
    kind: SeccompProfile
    metadata:
      name: example-name
    spec:
      defaultAction: SCMP_ACT_ERRNO
      baseProfileName: runc-v1.0.0
      syscalls:
        - action: SCMP_ACT_ALLOW
          names:
            - exit_group
    Copy to Clipboard Toggle word wrap

7.8.3. Enabling memory optimization in the spod daemon

The controller running inside of spod daemon process watches all pods available in the cluster when profile recording is enabled. This can lead to very high memory usage in large clusters, resulting in the spod daemon running out of memory or crashing.

To prevent crashes, the spod daemon can be configured to only load the pods labeled for profile recording into the cache memory.

Note

SPO memory optimization is not enabled by default.

Procedure

  1. Enable memory optimization by running the following command:

    $ oc -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableMemoryOptimization":true}}'
    Copy to Clipboard Toggle word wrap
  2. To record a security profile for a pod, the pod must be labeled with spo.x-k8s.io/enable-recording: "true":

    apiVersion: v1
    kind: Pod
    metadata:
      name: my-recording-pod
      labels:
        spo.x-k8s.io/enable-recording: "true"
    # ...
    Copy to Clipboard Toggle word wrap

7.8.4. Customizing daemon resource requirements

The default resource requirements of the daemon container can be adjusted by using the field daemonResourceRequirements from the spod configuration.

Procedure

  • To specify the memory and cpu requests and limits of the daemon container, run the following command:

    $ oc -n openshift-security-profiles patch spod spod --type merge -p \
        '{"spec":{"daemonResourceRequirements": { \
        "requests": {"memory": "256Mi", "cpu": "250m"}, \
        "limits": {"memory": "512Mi", "cpu": "500m"}}}}'
    Copy to Clipboard Toggle word wrap

7.8.5. Setting a custom priority class name for the spod daemon pod

The default priority class name of the spod daemon pod is set to system-node-critical. A custom priority class name can be configured in the spod configuration by setting a value in the priorityClassName field.

Procedure

  • Configure the priority class name by running the following command:

    $ oc -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"priorityClassName":"my-priority-class"}}'
    Copy to Clipboard Toggle word wrap

    Example output

    securityprofilesoperatordaemon.openshift-security-profiles.x-k8s.io/spod patched
    Copy to Clipboard Toggle word wrap

7.8.6. Using metrics

The openshift-security-profiles namespace provides metrics endpoints, which are secured by the kube-rbac-proxy container. All metrics are exposed by the metrics service within the openshift-security-profiles namespace.

The Security Profiles Operator includes a cluster role and corresponding binding spo-metrics-client to retrieve the metrics from within the cluster. There are two metrics paths available:

  • metrics.openshift-security-profiles/metrics: for controller runtime metrics
  • metrics.openshift-security-profiles/metrics-spod: for the Operator daemon metrics

Procedure

  1. To view the status of the metrics service, run the following command:

    $ oc get svc/metrics -n openshift-security-profiles
    Copy to Clipboard Toggle word wrap

    Example output

    NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
    metrics   ClusterIP   10.0.0.228   <none>        443/TCP   43s
    Copy to Clipboard Toggle word wrap

  2. To retrieve the metrics, query the service endpoint using the default ServiceAccount token in the openshift-security-profiles namespace by running the following command:

    $ oc run --rm -i --restart=Never --image=registry.fedoraproject.org/fedora-minimal:latest \
        -n openshift-security-profiles metrics-test -- bash -c \
        'curl -ks -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://metrics.openshift-security-profiles/metrics-spod'
    Copy to Clipboard Toggle word wrap

    Example output

    # HELP security_profiles_operator_seccomp_profile_total Counter about seccomp profile operations.
    # TYPE security_profiles_operator_seccomp_profile_total counter
    security_profiles_operator_seccomp_profile_total{operation="delete"} 1
    security_profiles_operator_seccomp_profile_total{operation="update"} 2
    Copy to Clipboard Toggle word wrap

  3. To retrieve metrics from a different namespace, link the ServiceAccount to the spo-metrics-client ClusterRoleBinding by running the following command:

    $ oc get clusterrolebinding spo-metrics-client -o wide
    Copy to Clipboard Toggle word wrap

    Example output

    NAME                 ROLE                             AGE   USERS   GROUPS   SERVICEACCOUNTS
    spo-metrics-client   ClusterRole/spo-metrics-client   35m                    openshift-security-profiles/default
    Copy to Clipboard Toggle word wrap

7.8.6.1. controller-runtime metrics

The controller-runtime metrics and the DaemonSet endpoint metrics-spod provide a set of default metrics. Additional metrics are provided by the daemon, which are always prefixed with security_profiles_operator_.

Expand
Table 7.1. Available controller-runtime metrics
Metric keyPossible labelsTypePurpose

seccomp_profile_total

operation={delete,update}

Counter

Amount of seccomp profile operations.

seccomp_profile_audit_total

node, namespace, pod, container, executable, syscall

Counter

Amount of seccomp profile audit operations. Requires the log enricher to be enabled.

seccomp_profile_bpf_total

node, mount_namespace, profile

Counter

Amount of seccomp profile bpf operations. Requires the bpf recorder to be enabled.

seccomp_profile_error_total

reason={
SeccompNotSupportedOnNode,
InvalidSeccompProfile,
CannotSaveSeccompProfile,
CannotRemoveSeccompProfile,
CannotUpdateSeccompProfile,
CannotUpdateNodeStatus
}

Counter

Amount of seccomp profile errors.

selinux_profile_total

operation={delete,update}

Counter

Amount of SELinux profile operations.

selinux_profile_audit_total

node, namespace, pod, container, executable, scontext,tcontext

Counter

Amount of SELinux profile audit operations. Requires the log enricher to be enabled.

selinux_profile_error_total

reason={
CannotSaveSelinuxPolicy,
CannotUpdatePolicyStatus,
CannotRemoveSelinuxPolicy,
CannotContactSelinuxd,
CannotWritePolicyFile,
CannotGetPolicyStatus
}

Counter

Amount of SELinux profile errors.

7.8.7. Using the log enricher

The Security Profiles Operator contains a log enrichment feature, which is disabled by default. The log enricher container runs with privileged permissions to read the audit logs from the local node. The log enricher runs within the host PID namespace, hostPID.

Important

The log enricher must have permissions to read the host processes.

Procedure

  1. Patch the spod configuration to enable the log enricher by running the following command:

    $ oc -n openshift-security-profiles patch spod spod \
        --type=merge -p '{"spec":{"enableLogEnricher":true}}'
    Copy to Clipboard Toggle word wrap

    Example output

    securityprofilesoperatordaemon.security-profiles-operator.x-k8s.io/spod patched
    Copy to Clipboard Toggle word wrap

    Note

    The Security Profiles Operator will re-deploy the spod daemon set automatically.

  2. View the audit logs by running the following command:

    $ oc -n openshift-security-profiles logs -f ds/spod log-enricher
    Copy to Clipboard Toggle word wrap

    Example output

    I0623 12:51:04.257814 1854764 deleg.go:130] setup "msg"="starting component: log-enricher"  "buildDate"="1980-01-01T00:00:00Z" "compiler"="gc" "gitCommit"="unknown" "gitTreeState"="clean" "goVersion"="go1.16.2" "platform"="linux/amd64" "version"="0.4.0-dev"
    I0623 12:51:04.257890 1854764 enricher.go:44] log-enricher "msg"="Starting log-enricher on node: 127.0.0.1"
    I0623 12:51:04.257898 1854764 enricher.go:46] log-enricher "msg"="Connecting to local GRPC server"
    I0623 12:51:04.258061 1854764 enricher.go:69] log-enricher "msg"="Reading from file /var/log/audit/audit.log"
    2021/06/23 12:51:04 Seeked /var/log/audit/audit.log - &{Offset:0 Whence:2}
    Copy to Clipboard Toggle word wrap

7.8.7.1. Using the log enricher to trace an application

You can use the Security Profiles Operator log enricher to trace an application.

Procedure

  1. To trace an application, create a SeccompProfile logging profile:

    apiVersion: security-profiles-operator.x-k8s.io/v1beta1
    kind: SeccompProfile
    metadata:
      name: log
    spec:
      defaultAction: SCMP_ACT_LOG
    Copy to Clipboard Toggle word wrap
  2. Create a pod object to use the profile:

    apiVersion: v1
    kind: Pod
    metadata:
      name: log-pod
      namespace: default
    spec:
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: Localhost
          localhostProfile: operator/log.json
      containers:
      - name: log-container
        image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ALL]
    Copy to Clipboard Toggle word wrap
  3. Examine the log enricher output by running the following command:

    $ oc -n openshift-security-profiles logs -f ds/spod log-enricher
    Copy to Clipboard Toggle word wrap

    Example 7.1. Example output

    …
    I0623 12:59:11.479869 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=3 "syscallName"="close" "timestamp"="1624453150.205:1061" "type"="seccomp"
    I0623 12:59:11.487323 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=157 "syscallName"="prctl" "timestamp"="1624453150.205:1062" "type"="seccomp"
    I0623 12:59:11.492157 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=157 "syscallName"="prctl" "timestamp"="1624453150.205:1063" "type"="seccomp"
    …
    I0623 12:59:20.258523 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=12 "syscallName"="brk" "timestamp"="1624453150.235:2873" "type"="seccomp"
    I0623 12:59:20.263349 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=21 "syscallName"="access" "timestamp"="1624453150.235:2874" "type"="seccomp"
    I0623 12:59:20.354091 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=257 "syscallName"="openat" "timestamp"="1624453150.235:2875" "type"="seccomp"
    I0623 12:59:20.358844 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=5 "syscallName"="fstat" "timestamp"="1624453150.235:2876" "type"="seccomp"
    I0623 12:59:20.363510 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=9 "syscallName"="mmap" "timestamp"="1624453150.235:2877" "type"="seccomp"
    I0623 12:59:20.454127 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=3 "syscallName"="close" "timestamp"="1624453150.235:2878" "type"="seccomp"
    I0623 12:59:20.458654 1854764 enricher.go:111] log-enricher "msg"="audit"  "container"="log-container" "executable"="/usr/sbin/nginx" "namespace"="default" "node"="127.0.0.1" "pid"=1905792 "pod"="log-pod" "syscallID"=257 "syscallName"="openat" "timestamp"="1624453150.235:2879" "type"="seccomp"
    …
    Copy to Clipboard Toggle word wrap

7.8.8. Configuring webhooks

Profile binding and profile recording objects can use webhooks. Profile binding and recording object configurations are MutatingWebhookConfiguration CRs, managed by the Security Profiles Operator.

To change the webhook configuration, the spod CR exposes a webhookOptions field that allows modification of the failurePolicy, namespaceSelector, and objectSelector variables. This allows you to set the webhooks to "soft-fail" or restrict them to a subset of a namespaces so that even if the webhooks failed, other namespaces or resources are not affected.

Procedure

  1. Set the recording.spo.io webhook configuration to record only pods labeled with spo-record=true by creating the following patch file:

    spec:
      webhookOptions:
        - name: recording.spo.io
          objectSelector:
            matchExpressions:
              - key: spo-record
                operator: In
                values:
                  - "true"
    Copy to Clipboard Toggle word wrap
  2. Patch the spod/spod instance by running the following command:

    $ oc -n openshift-security-profiles patch spod \
        spod -p $(cat /tmp/spod-wh.patch) --type=merge
    Copy to Clipboard Toggle word wrap
  3. To view the resulting MutatingWebhookConfiguration object, run the following command:

    $ oc get MutatingWebhookConfiguration \
        spo-mutating-webhook-configuration -oyaml
    Copy to Clipboard Toggle word wrap

7.9. Advanced Audit Logging Framework

The Advanced Audit Logging Framework provided in OpenShift Container Platform Security Profiles Operator (SPO) 0.10.0 provides logging of container activities in an Red Hat Enterprise Linux CoreOS (RHCOS) container back to the hosting cluster. With the Advanced Audit Logging Framework, you can correlate an OpenShift Container Platform user with their direct actions on the node during oc exec, oc rsh, and oc debug sessions. The advanced audit logging results in detailed logs in a JSON Lines format.

7.9.1. Benefits of Advanced Audit Logging Framework

The kubectl exec, oc exec, oc rsh and oc debug commands do not pass user authentication details into the exec session on the container, making it hard to correlate Kubernetes user actions caused by actions on the host. The audit logger in SPO addresses this with mutating webhooks that inject the request UID from the Kubernetes API server as an environment variable into the session. Every request to the API server including the request to start a new exec session has a request UID. This request UID is then logged by the Advanced Audit Logging Framework. The request ID is used to correlate the activity with the API server audit logs, providing an audit trail within the node.

With the addition of the Advanced Audit Logging Framework, SPO now has two use cases:

  • Pod auditing
  • Node auditing

The use of privileged seccompProfile configuration is required only for the case of node auditing.

Note

The Security Profiles Operator supports only Red Hat Enterprise Linux CoreOS (RHCOS) worker nodes appropriate to the version of OpenShift Container Platform in use.

Red Hat Enterprise Linux (RHEL) nodes are not supported.

7.9.2. Performance considerations

It is important to consider the performance cost of using seccomp profiles for extensive logging. SPO is designed to minimize this impact by primarily logging only process creations and handling them asynchronously. This approach helps prevent logging from becoming a bottleneck on your nodes.

The Advanced Audit Logging feature uses eBPF as a supplemental data source. While it is possible for eBPF to be used as a primary data source for this type of logging, that functionality is not currently a configurable feature within the Operator. For most use cases, the default asynchronous, process-creation-focused logging approach provides an excellent balance between security visibility and cluster performance.

7.9.3. Prerequisites for use of the Advanced Audit Logging Framework

Before enabling the Advanced Audit Logging Framework, ensure the following requirements are met.

Security Profiles Operator version 0.10.0 or later is installed. Those instructions can be found at Installing the Security Profiles Operator. The Advanced Audit Logging Framework requires Security Profiles Operator version 0.10.0 or later.

For node debugging sessions:

  • To audit oc debug node sessions, CRI-O version 1.33 or later is required. It is available in OpenShift Container Platform 4.20 or later.
  • The --privileged-seccomp-profile flag must be configured in CRI-O to apply seccompProfiles to privileged containers.
  • The supported Linux used with the Advanced Audit Logging Framework is Red Hat Enterprise Linux CoreOS (RHCOS) running in a container in OpenShift Container Platform 4.20 or later.

If you are using the CRI-O runtime, you must configure it to allow seccompProfile to be applied to privileged containers. Add the following flag to your CRI-O runtime configuration: --privileged-seccomp-profile=/var/lib/kubelet/seccomp/operator/profile1.json. This is explained in more detail in the Advanced Audit Logging installation and enablement steps. The --privileged-seccomp-profile flag is available starting with OCP 4.20 and later.

If you are using any version of SPO before 0.9.0, you must perform a migration procedure to install versions 0.9.0 or 0.10.0. The migration procedure converts SPO to operate on cluster-scoped resources.

First-time installation of SPO version 0.10.0 does not require migration. Also, if you are currently on SPO 0.9.0, you do not require migration and can directly upgrade to SPO 0.10.0.

Important

Do not attempt to upgrade directly from SPO versions before 0.9.0 to either 0.9.0 or 0.10.0 if you are currently running SPO. You must perform the migration procedure to convert SPO for operation at the cluster level.

This change allows Advanced Audit Logging of events inside the worker node.

This capability is not provided for control nodes since SPO does not operate on etcd nodes.

7.9.4. The Audit JSON log enricher

The SPO Advanced Audit Logging Framework is enabled by the Audit JSON log enricher. Similar to the log enricher feature, the Audit JSON log enricher watches the auditd (/var/log/audit/audit.log) or the syslog (/var/log/syslog) daemons and generates a new audit log in JSON lines format.

Each JSON line includes the following:

  • Timestamp: When the activity happened, shown in a standard ISO format
  • Executable Name: The name of the program that was run (bash, ls).
  • Linux Command Line Arguments (cmdline): The extra instructions given when the program was started (ls -l /home).
  • User and Group IDs (UID/GID): The identification numbers of the system user who ran the program.
  • System Calls (syscalls): A list of system calls (syscalls) that the process made

This log format and configuration is similar to how Kubernetes records audit logs. This is useful for:

  • Seeing what users and automated processes are doing inside a pod.
  • Tracking when someone uses commands such as kubectl exec to enter a running container and run commands or scripts.
  • Monitoring activities in debug containers where users might run various tools.

7.9.5. Kubernetes API log compared to Advanced Audit Logging output

To understand the value of Advanced Audit Logging, compare the Kubernetes API Server Audit Log to the Advanced Audit Logging output:

Kubernetes API Server Audit Log

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1",
  "level": "Metadata",
  "auditID": "4d434cd4-xxxx-xxxx-xxxx-2d9aa46292ce",
  "stage": "ResponseComplete",
  "requestURI": "/api/v1/namespaces/test-namespace/pods/test-pod/exec?command=sh&command=-c&command=touch+/tmp/testfile.txt&container=nginx",
  "verb": "create",
  "user": {
    "username": "kube:admin",
    "groups": ["system:cluster-admins", "system:authenticated"]
  },
  "sourceIPs": ["xxx.xxx.xxx.xxx"],
  "userAgent": "oc/4.19.0 (linux/amd64)",
  "objectRef": {
    "resource": "pods",
    "namespace": "test-namespace",
    "name": "test-pod",
    "subresource": "exec"
  },
  "responseStatus": {
    "code": 101
  },
  "requestReceivedTimestamp": "2026-02-16T14:01:06.056518Z",
  "annotations": {
    "authorization.k8s.io/decision": "allow",
    "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding...",
    "execmetadata.spo.io/SPO_EXEC_REQUEST_UID": "aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9"
  }
}
Copy to Clipboard Toggle word wrap

The correlation key is the SPO_EXEC_REQUEST_UID on the last line in the above file.

Advanced Audit Logging Output

{
  "auditID": "d586679d-xxxx-xxxx-xxxx-9dc8ab273065",
  "cmdLine": "touch /tmp/testfile.txt ",		// Linux command with arguments
  "executable": "/bin/dash",
  "gid": 0,
  "node": {
    "name": "worker-1"
  },
  "pid": 144968,
  "requestUID": "aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9", // Correlation key
  "resource": {
    "container": "nginx",
    "namespace": "test-namespace",
    "pod": "test-pod"
  },
  "syscalls": ["execve"],
  "timestamp": "2026-02-16T14:01:07.000Z",
  "uid": 0,
  "version": "spo/v1_alpha"
}
Copy to Clipboard Toggle word wrap

7.9.6. Enabling Advanced Audit Logging

To enable Advanced Audit Logging, configure the Audit JSON log enricher and specify a set of filters to only log user activity.

Procedure

  1. Enable the JSON enricher by running the following command:

    # kubectl -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableJsonEnricher":true}}'
    Copy to Clipboard Toggle word wrap

    Monitor the SPOD pods for correct restart and wait for all SPOD pods to show Running before proceeding.

  2. Check the pods for application of the change with the following command:

    $ oc get pods -n openshift-security-profiles -l name=spod -w
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding.

    Note

    Each configuration change triggers a restart of the SPOD pods. After applying a patch, wait for all SPOD pods to return to the Running state before continuing.

    The Audit JSON log enricher uses eBPF as a supplemental data source. While processing auditd logs from /var/log/audit/audit.log, the enricher attempts to fetch ephemeral data from /proc/<pid> directories. Due to a race condition, these files might be deleted before they can be read. To ensure data completeness, the enricher falls back to fetching the necessary information from eBPF whenever it is not found in /proc/<pid>.

7.9.7. Audit JSON Log Enricher configuration

The Audit JSON Log Enricher requires some configuration to set interval, configure the destination of the log data and set the audit log file path. This procedure demonstrates how to set up and fine tune your Advanced Audit Logging Framework audit logs.

Note

Each configuration change triggers a restart of the SPOD pods. After applying a patch, wait for all SPOD pods to return to the Running state before continuing.

Setting the audit log interval determines how often audit logs are created using the auditLogIntervalSeconds option.

Procedure

  1. Configure the audit log interval to 30 seconds by using the following command:

    # kubectl -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableJsonEnricher":true,"verbosity":0,"jsonEnricherOptions":{"auditLogIntervalSeconds":30}}}'
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding. By default, audit logs go to your standard output in JSON lines format. You can send them to a file instead.

    Configure the security profiles operator to store the log file on the node. Update the security-profiles-operator-profile ConfigMap with two keys. This example yaml uses both keys to set up a host path volume at /tmp/logs.

  2. Save this JSON in a file, such as patch-volume-source.json and check it using the following command:

    $ cat patch-volume-source.json
    Copy to Clipboard Toggle word wrap
    {
      "data": {
        "json-enricher-log-volume-mount-path": "/tmp/logs",
        "json-enricher-log-volume-source.json": "{\"hostPath\": {\"path\": \"/tmp/logs\",\"type\": \"DirectoryOrCreate\"}}"
      }
    }
    Copy to Clipboard Toggle word wrap
    • json-enricher-log-volume-source.json: Defines the type of volume (for example, a host path and empty directory) where logs will be stored. This must be a JSON string representing a corev1.VolumeSource object.
    • json-enricher-log-volume-mount-path: Specifies the directory path where the log file will be generated.
  3. Update the config map by saving this JSON in this file called patch-volume-source.json and then update the config map with the following command:

    # kubectl patch configmap security-profiles-operator-profile -n openshift-security-profiles --patch-file patch-volume-source.json
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding.

  4. To set the audit log file path, configure the JSON log enricher with the full path to your audit log file by including the filename, using this command:

    # kubectl -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableJsonEnricher":true,"verbosity":0,"jsonEnricherOptions":{"auditLogPath":"/tmp/logs/audit1.log"}}}'
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding

7.9.8. Audit log file fine-tuning and rotation

For audit logging to a file, you can manage file size and how long each file is kept. These options are similar to Kubernetes API server log settings.

Procedure

  1. You configure these by patching the JSON log enricher options:

    # kubectl -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableJsonEnricher":true,"verbosity":0,"jsonEnricherOptions":{"auditLogPath":"/tmp/logs/audit1.log","auditLogMaxSize":500,"auditLogMaxBackups":2,"auditLogMaxAge":10}}}'
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding.

    • auditLogMaxSize: The maximum size (in megabytes) a log file can reach before it’s rotated (a new file is started).
    • auditLogMaxBackups: The maximum number of older, rotated log files to keep. Set to 0 for no limit.
    • auditLogMaxAge: The maximum number of days to keep old log files.
  2. Increase the logging level for the JSON log enricher container to help with debugging. A value of 0 sets minimal logs. A value of 1 sets more detailed logs. You can choose either of these two levels and enable either level with the following command:

    # kubectl -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableJsonEnricher":true, "verbosity": 1}}'
    Copy to Clipboard Toggle word wrap

    Wait until all SPOD pods show Running before proceeding.

7.9.9. Advanced Audit Logs for a specific pod

To enable single pod log activity, create a SeccompProfile to log specific syscalls such as execve and clone a ProfileBinding to automatically apply this profile to pods in a target namespace. The SeccompProfile applies to the cluster and the ProfileBinding applies to the workloads in that namespace.

With CRI-O versions 1.33 and newer, starting with OpenShift Container Platform 4.20, a feature was introduced to allow SeccompProfiles for privileged containers. You can apply the SecompProfile we created to the CRI-O configuration. The key is to configure the CRI-O runtime to use the profile by passing an additional flag.

To do this, you must add the flag –privileged-seccomp-profile to the CRI-O runtime configuration. This step ensures that even privileged debugging pods are subject to the SeccompProfile logging, maintaining complete audit coverage.

You must bind the profile to a namespace in order to apply it to a workload. This will automatically apply the profile to new pods in the default namespace.

Procedure

  1. Create a file such as profile1.yaml with the following content:

    apiVersion: security-profiles-operator.x-k8s.io/v1beta1
    kind: SeccompProfile
    metadata:
      name: profile1
      namespace: openshift-security-profiles
    spec:
      defaultAction: SCMP_ACT_ALLOW
      syscalls:
      - action: SCMP_ACT_LOG
        names:
          - execve
          - clone
          - getpid
    Copy to Clipboard Toggle word wrap

    This profile allows all normal actions (defaultAction: SCMP_ACT_ALLOW). It specifically tells the system to log when a process tries to run a new program (execve), create a new process (clone), or get its own process ID (getpid). These actions often indicate user interaction within a pod.

  2. Use`kubectl apply` to apply this SeccompProfile profile to your cluster as in the following command:

    # kubectl apply -f profile1.yaml
    Copy to Clipboard Toggle word wrap
    Note

    SPO must use the privileged SeccompProfile

  3. Create a file named image_sec_comp.yaml containing the following yaml:

    apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
    kind: ProfileBinding
    metadata:
      namespace: default
      name: all-pod-binding
    spec:
      profileRef:
        kind: SeccompProfile
        name: profile1
      image: "*"
    Copy to Clipboard Toggle word wrap
  4. Apply the binding using the following command:

    # kubectl apply -f image_sec_comp.yaml
    Copy to Clipboard Toggle word wrap
  5. Label the namespace to activate it using the following command:

    # kubectl label ns default spo.x-k8s.io/enable-binding=true
    Copy to Clipboard Toggle word wrap
  6. Create a pod using the profile:

    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
      labels:
        app: my-app
    spec:
      securityContext:
        SeccompProfile:
          type: Localhost
          localhostProfile: operator/profile1.json
      containers:
        - name: nginx
          image: quay.io/security-profiles-operator/test-nginx:1.19.1
    Copy to Clipboard Toggle word wrap
    • type: Localhost means you are using a profile you have defined in the cluster.
    • localhostProfile: operator/profile1.json tells the pod to use the profile1 you created. The operator/ part indicates where the Security Profiles Operator stores these profiles.
  7. Apply the pod definition to create the pod by running the following command:

    # kubectl apply -f my-pod.yaml
    Copy to Clipboard Toggle word wrap
  8. Exec/rsh into the pod and run the following command:

    # kubectl exec -it my-nginx-pod -- /bin/sh
    Copy to Clipboard Toggle word wrap
  9. Create an empty file by running this command:

    # touch /tmp/audittest/demo-file
    Copy to Clipboard Toggle word wrap
    • To monitor the advanced audit log tail, use the following command:

      # kubectl -n openshift-security-profiles logs --since=1m --selector name=spod -c json-enricher --max-log-requests 6 -f
      Copy to Clipboard Toggle word wrap
    • Alternatively, to monitor or inspect the advanced audit log file, you need to identify the node on which the pod is scheduled:

      # kubectl get pod my-pod -o wide
      Copy to Clipboard Toggle word wrap

      The audit log file specified in the auditLogPath is written to the node’s file system where the pod is running. To monitor or inspect the audit logs, you must access the node directly and check the file at the specified path such as /tmp/logs/audit1.log.

  10. SSH to a node by using the following command:

    $ sudo ssh core@<node-name>
    Copy to Clipboard Toggle word wrap
  11. View the audit log by using the following command:

    $ cat /tmp/logs/audit1.log
    Copy to Clipboard Toggle word wrap

    Example output

    {
    "auditID": "a1b2c3d4-e5f6-7890-abcd-111111111111",
    "cmdLine": "mkdir /tmp/audittest ",
    "executable": "/bin/bash",
    "gid": 0,
    "node": {"name": "worker-1"},
    "pid": 27184,
    "requestUID": "f011c4a3-b20e-44ed-bb91-23e03ae31b3e",
    "resource": {
    "container": "nginx",
    "namespace": "default",
    "pod": "my-pod"
    },
    "syscalls": ["getpid", "execve"],
    "timestamp": "2026-02-16T06:34:53.000Z",
    "uid": 0,
    "version": "spo/v1_alpha"
    }
    {
    "auditID": "a1b2c3d4-e5f6-7890-abcd-222222222222",
    "cmdLine": "touch /tmp/audittest/demo-file ",
    "executable": "/bin/bash",
    "gid": 0,
    "node": {"name": "worker-1"},
    "pid": 27274,
    "requestUID": "f011c4a3-b20e-44ed-bb91-23e03ae31b3e",
    "resource": {
    "container": "nginx",
    "namespace": "default",
    "pod": "my-pod"
    },
    "syscalls": ["getpid", "execve"],
    "timestamp": "2026-02-16T06:35:02.000Z",
    "uid": 0,
    "version": "spo/v1_alpha"
    }
    Copy to Clipboard Toggle word wrap

    By following above steps, you can enable and monitor audit logs in JSON lines format for your Kubernetes pods, giving you better visibility into their activities.

7.9.10. Monitoring the audit logs

There are two ways to monitor the advanced audit logs generated by the json-enricher container. By following these steps you can enable and monitor audit logs in JSON lines format for your Kubernetes pods, giving you better visibility into their activities.

Procedure

  1. To monitor the advanced audit log tail use the following command:

    # kubectl -n openshift-security-profiles logs --since=1m --selector name=spod -c json-enricher --max-log-requests 6 -f
    Copy to Clipboard Toggle word wrap
  2. To monitor the advanced audit log file, it is specified in the auditLogPath and written to the node’s file system where the pod is running. To monitor or inspect the audit logs, you must access the node directly and check the file at the specified path such as /tmp/logs/audit1.log.
  3. Identify the node on which the pod is scheduled using the following command:

    # kubectl get pod my-pod -o wide
    Copy to Clipboard Toggle word wrap
  4. SSH to a node with the following command:

    $ sudo ssh core@<node-name>
    Copy to Clipboard Toggle word wrap
  5. View the audit log with the following command:

    $ cat /tmp/logs/audit1.log
    Copy to Clipboard Toggle word wrap

    The example output should look like this:

    {
    "auditID": "a1b2c3d4-e5f6-7890-abcd-111111111111",
    "cmdLine": "mkdir /tmp/audittest ",
    "executable": "/bin/bash",
    "gid": 0,
    "node": {"name": "worker-1"},
    "pid": 27184,
    "requestUID": "f011c4a3-b20e-44ed-bb91-23e03ae31b3e",
    "resource": {
    "container": "nginx",
    "namespace": "default",
    "pod": "my-pod"
    },
    "syscalls": ["getpid", "execve"],
    "timestamp": "2026-02-16T06:34:53.000Z",
    "uid": 0,
    "version": "spo/v1_alpha"
    }
    {
    "auditID": "a1b2c3d4-e5f6-7890-abcd-222222222222",
    "cmdLine": "touch /tmp/audittest/demo-file ",
    "executable": "/bin/bash",
    "gid": 0,
    "node": {"name": "worker-1"},
    "pid": 27274,
    "requestUID": "f011c4a3-b20e-44ed-bb91-23e03ae31b3e",
    "resource": {
    "container": "nginx",
    "namespace": "default",
    "pod": "my-pod"
    },
    "syscalls": ["getpid", "execve"],
    "timestamp": "2026-02-16T06:35:02.000Z",
    "uid": 0,
    "version": "spo/v1_alpha"
    }
    Copy to Clipboard Toggle word wrap

7.9.11. Auditing node debugging sessions

To audit kubectl debug sessions, you must enable privileged seccomp profiles in CRI-O.

If you are using the CRI-O runtime, you must configure it to allow seccomp profiles on privileged containers by adding the --privileged-seccomp-profile=/var/lib/kubelet/seccomp/operator/profile1.json flag to your CRI-O runtime configuration.

Note

The --privileged-seccomp-profile flag is available starting with OpenShift Container Platform 4.20 or later and CRI-O version 1.33 or later.

Procedure

  1. SSH into the target node using the following command:

    # ssh core@<node_ip_address>
    Copy to Clipboard Toggle word wrap
  2. Check to see if the files are there:

    # ls /var/lib/kubelet/seccomp/operator/
    Copy to Clipboard Toggle word wrap

    Example output

    # kubelet-config.json  profile1.json
    Copy to Clipboard Toggle word wrap
  3. Stop the kubelet with the following commands:

    # systemctl stop kubelet
    Copy to Clipboard Toggle word wrap
  4. Stop CRI-O with the command:

    # systemctl stop crio
    Copy to Clipboard Toggle word wrap
  5. Set the CRI-O options with the following command:

    # echo "CRIO_CONFIG_OPTIONS --privileged-seccomp-profile=/var/lib/kubelet/seccomp/operator/profile1.json" > /etc/sysconfig/crio
    Copy to Clipboard Toggle word wrap
  6. Now restart the cubelet with this command:

    # systemctl start kubelet
    Copy to Clipboard Toggle word wrap
  7. Restart CRI-O with this command:

    # systemctl start crio
    Copy to Clipboard Toggle word wrap
  8. To audit kubectl debug sessions, run the following command:

    # kubectl debug node/<node_name> -it --image=ubuntu -- bash
    Copy to Clipboard Toggle word wrap

    Create the file on the container for the debugging information.

  9. chroot to the host with this command:

    # chroot /host
    Copy to Clipboard Toggle word wrap
  10. Go into the /tmp directory with this command:

    # cd /tmp
    Copy to Clipboard Toggle word wrap
  11. Create an empty file named demonodedebug with this command:

    # touch demonodedebug
    Copy to Clipboard Toggle word wrap
  12. Exit the node with the command:

    # exit
    Copy to Clipboard Toggle word wrap
  13. To monitor the logs, SSH to a node with this command:

    $ sudo ssh core@<node_name>
    Copy to Clipboard Toggle word wrap
  14. View the audit log using this command:

    $ cat /tmp/logs/audit1.log
    Copy to Clipboard Toggle word wrap

    Example output:

    {
      "auditID": "edce381a-998d-4f3f-99d8-d0c4d0c8a613",
      "cmdLine": "touch demonodedebug",
      "executable": "/usr/bin/bash",
      "gid": 0,
      "node": {"name": "worker-1"},
      "pid": 99086,
      "resource": {
        "container": "container-00",
        "namespace": "openshift-security-profiles",
        "pod": "worker-1-debug"
      },
      "syscalls": ["execve", "getpid"],
      "timestamp": "2026-02-12T13:50:11.000Z",
      "uid": 0,
      "version": "spo/v1_alpha"
    }
    Copy to Clipboard Toggle word wrap

7.9.12. Correlate with Kubernetes audit logs

Use the requestUID from the Security Profiles Operator (SPO) log to find the corresponding API server log entry, confirming who initiated the session.

Procedure

  1. Start the pod by running the following command:

    $ oc exec my-pod -c nginx -- sh -c "touch /tmp/testfile.txt"
    Copy to Clipboard Toggle word wrap
  2. Identify the node where the pod is running:

    $ NODE=$(oc get pod my-pod -o jsonpath='{.spec.nodeName}')
    Copy to Clipboard Toggle word wrap
  3. Access the node and check the JSON enriched audit log using the following commands:

    # oc debug node/$NODE
    # chroot /host
    # grep "testfile" /tmp/logs/audit1.log | jq .
    Copy to Clipboard Toggle word wrap

7.9.13. Audit JSON Log Enricher Output

The Audit JSON Log Enricher captures two entries for this exec session, the SPO-EXEC_REQUEST_UID injection and the command run on the pod. They are connected by looking for the common UID value.

  1. The first listing is the container runtime wrapper.

    {
      "auditID": "062e2bd2-xxxx-xxxx-xxxx-57fb39d65a99",
      "cmdLine": "env SPO_EXEC_REQUEST_UID=aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9 sh -c touch /tmp/testfile.txt ",
      "executable": "/usr/bin/crun",
      "pid": 144966,
      "requestUID": "aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9",
      "resource": {
        "container": "nginx",
        "namespace": "default",
        "pod": "my-pod"
      },
      "node": {
        "name": "worker-1"
      },
      "syscalls": ["execve", "getpid", "clone"],
      "timestamp": "2026-02-16T14:01:07.000Z"
    }
    Copy to Clipboard Toggle word wrap
  2. The second listing is the actual command executed.

    {
      "auditID": "d586679d-xxxx-xxxx-xxxx-9dc8ab273065",
      "cmdLine": "touch /tmp/testfile.txt ",
      "executable": "/bin/dash",
      "pid": 144968,
      "requestUID": "aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9", // Correlation key
      "resource": {
        "container": "nginx",
        "namespace": "default",
        "pod": "my-pod"
      },
      "node": {
        "name": "worker-1"
      },
      "syscalls": ["execve"],
      "timestamp": "2026-02-16T14:01:07.000Z"
    }
    Copy to Clipboard Toggle word wrap
  3. You can search the Kubernetes API audit log by using the requestUID with the following command:

    $ oc adm node-logs --role=master --path=kube-apiserver/audit.log | grep request_UID
    Copy to Clipboard Toggle word wrap

7.9.14. Kubernetes API audit log output

The output of the Kubernetes API audit log is YAML. It contains the SPO_EXEC_REQUEST_UID field which provides the correlation key with which you can search the Advanced Audit Logging output.

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1",
  "level": "Metadata",
  "auditID": "4d434cd4-xxxx-xxxx-xxxx-2d9aa46292ce",
  "stage": "ResponseComplete",
  "requestURI": "/api/v1/namespaces/test-namespace/pods/test-pod/exec?command=sh&command=-c&command=touch+/tmp/testfile.txt&container=nginx",
  "verb": "create",
  "user": {
    "username": "kube:admin",
    "groups": ["system:cluster-admins", "system:authenticated"]
  },
  "sourceIPs": ["xxx.xxx.xxx.xxx"],
  "userAgent": "oc/4.19.0 (linux/amd64)",
  "objectRef": {
    "resource": "pods",
    "namespace": "default",
    "name": "my-pod",
    "subresource": "exec"
  },
  "responseStatus": {
    "code": 101
  },
  "requestReceivedTimestamp": "2026-02-16T14:01:06.056518Z",
  "annotations": {
    "authorization.k8s.io/decision": "allow",
    "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding...",
    "execmetadata.spo.io/SPO_EXEC_REQUEST_UID": "aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9"
}
Copy to Clipboard Toggle word wrap

The final field in this example, SPO_EXEC_REQUEST_UID is the correlation key.

7.9.15. Correlation key

The connection of the two correlation keys enables administrators to establish a complete audit trail. Who executed a command, from Kubernetes API audit log kube:admin at IP xxx.xxx.xxx.xxx and what the command did at the system level, can be found in the SPO JSON Enricher log touch /tmp/testfile.txt.

The requestUID field in Audit JSON Enricher logs matches the annotations.execmetadata.spo.io/SPO_EXEC_REQUEST_UID annotation in the Kubernetes API audit log.

aec3e0e1-xxxx-xxxx-xxxx-a7c58241f1a9
Copy to Clipboard Toggle word wrap

7.9.16. Correlating with API Server Audit Log

By default, when you use the kubectl exec command to access a pod or container, Kubernetes does not pass the user’s authentication details into that session’s environment. This means the Audit JSON log enricher cannot provide audit information for exec commands. The UID or GID shown, maps to the system user. In most cases this would be the root user.

To address this, the Audit JSON log enricher relies on mutating webhooks (execmetadata.spo.io and nodedebuggingpod.spo.io). The webhook injects the exec requestUID as an environment variable into the exec session. When the administrator enables audit logging on the API server, the webhooks add the SPO_EXEC_REQUEST_UID audit annotation. The API server audit log contains this information. This request ID is also available in the JSON lines produced by the Audit JSON log enricher, specifically within the requestUID field.

By default, these webhooks are enabled for all namespaces with the Audit JSON log enricher enabled. To reduce the scope of this webhook you can disable it for certain namespaces.

Procedure

  1. Edit the spod security profile by running the following command:

    $ oc edit spod spod -n openshift-security-profiles
    Copy to Clipboard Toggle word wrap
  2. Add webhookOptions to the spec. Locate the spec section and add the following webhookOptions block to instruct the webhook to apply to a specific namespace.

    spec:
      webhookOptions:
        - name: execmetadata.spo.io # or nodedebuggingpod.spo.io
          namespaceSelector:
          #...add rules
    Copy to Clipboard Toggle word wrap

    After saving your changes, the Operator reconfigures the mutating webhook, allowing request details to be passed into oc exec sessions cluster-wide.

7.9.17. Using the mutating webhook

This webhook injects the environment variable SPO_EXEC_REQUEST_UID into your exec request. If a container in your pod already defines an environment variable with this exact name, the webhook injected value will override it for this exec session.

When you use kubectl debug node/<node_name>, the nodedebuggingpod.spo.io webhook automatically injects the SPO_EXEC_REQUEST_UID environment variable into the debug pod.

7.9.17.1. The Debug Pod

This webhook primarily identifies kubectl debug pods by the label app.kubernetes.io/managed-by: kubectl-debug, which is added by the kubectl client. Because this label might vary across different Kubernetes client implementations, such as how oc debug in OpenShift uses debug.openshift.io/managed-by: oc-debug, you might need to configure additional webhookOptions to ensure the webhook catches all relevant debug pods.

For example, to add oc debug pods, use the following yaml:

# ... (rest of your spod configuration)
spec:
  webhookOptions:
    - name: nodedebuggingpodmetada.spo.io
      objectSelector:
        matchLabels: # Use matchLabels for exact matching
          debug.openshift.io/managed-by: "oc-debug"
# ... (other webhook rule details such as rules, clientConfig, etc.)
Copy to Clipboard Toggle word wrap

7.9.18. Disabling Advanced Audit Logging

You can disable advanced audit logging and revert all configurations by deleting the test pod, the seccompProfile, the JSON Log Enricher and resetting all spod pod options.

Procedure

  1. Delete the test pod with the following command:

    oc delete pod my-pod
    Copy to Clipboard Toggle word wrap
  2. Delete the seccompProfile using this command:

    oc delete seccompprofile profile1 -n openshift-security-profiles
    Copy to Clipboard Toggle word wrap
  3. Disable the JSON Log Enricher and reset all options:

    oc patch spod spod -n openshift-security-profiles --type merge -p '{ "spec": { "enableJsonEnricher": false, "jsonEnricherOptions": { "auditLogPath": "", "auditLogMaxSize": 0, "auditLogMaxBackups": 0, "auditLogMaxAge": 0, "auditLogIntervalSeconds": 0 } }}'
    Copy to Clipboard Toggle word wrap
  4. Wait for spod pods to restart. Run the following command to check:

    oc get pods -n openshift-security-profiles -l name=spod -w
    Copy to Clipboard Toggle word wrap

    Wait until all spod pods show Running.

  5. Revert the ConfigMap volume patch with the following command:

    oc patch configmap security-profiles-operator-profile -n openshift-security-profiles --type merge -p '{"data":{"patch-volume-source.json":""}}'
    Copy to Clipboard Toggle word wrap
  6. Verify that the configuration has been successfully updated:

    oc get spod spod -n openshift-security-profiles -o jsonpath='{.spec.enableJsonEnricher}'
    Copy to Clipboard Toggle word wrap

    Expected output:

    false
    Copy to Clipboard Toggle word wrap

7.9.19. Additional resources

7.10. Troubleshooting the Security Profiles Operator

Troubleshoot the Security Profiles Operator to diagnose a problem or provide information in a bug report.

7.10.1. Inspecting seccomp profiles

Corrupted seccomp profiles can disrupt your workloads. Ensure that the user cannot abuse the system by not allowing other workloads to map any part of the path /var/lib/kubelet/seccomp/operator.

Procedure

  1. Confirm that the profile is reconciled by running the following command:

    $ oc -n openshift-security-profiles logs openshift-security-profiles-<id>
    Copy to Clipboard Toggle word wrap

    Example 7.2. Example output

    I1019 19:34:14.942464       1 main.go:90] setup "msg"="starting openshift-security-profiles"  "buildDate"="2020-10-19T19:31:24Z" "compiler"="gc" "gitCommit"="a3ef0e1ea6405092268c18f240b62015c247dd9d" "gitTreeState"="dirty" "goVersion"="go1.15.1" "platform"="linux/amd64" "version"="0.2.0-dev"
    I1019 19:34:15.348389       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"=":8080"
    I1019 19:34:15.349076       1 main.go:126] setup "msg"="starting manager"
    I1019 19:34:15.349449       1 internal.go:391] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
    I1019 19:34:15.350201       1 controller.go:142] controller "msg"="Starting EventSource" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"defaultAction":""}}}
    I1019 19:34:15.450674       1 controller.go:149] controller "msg"="Starting Controller" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile"
    I1019 19:34:15.450757       1 controller.go:176] controller "msg"="Starting workers" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "worker count"=1
    I1019 19:34:15.453102       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="nginx-1.19.1" "name"="nginx-1.19.1" "resource version"="728"
    I1019 19:34:15.453618       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="openshift-security-profiles" "name"="openshift-security-profiles" "resource version"="729"
    Copy to Clipboard Toggle word wrap
  2. Confirm that the seccomp profiles are saved into the correct path by running the following command:

    $ oc exec -t -n openshift-security-profiles openshift-security-profiles-<id> \
        -- ls /var/lib/kubelet/seccomp/operator/my-namespace/my-workload
    Copy to Clipboard Toggle word wrap

    Example output

    profile-block.json
    profile-complain.json
    Copy to Clipboard Toggle word wrap

7.11. Uninstalling the Security Profiles Operator

You can remove the Security Profiles Operator from your cluster by using the OpenShift Container Platform web console.

7.11.1. Uninstall the Security Profiles Operator using the web console

To remove the Security Profiles Operator, you must first delete the seccomp and SELinux profiles. After the profiles are removed, you can then remove the Operator and its namespace by deleting the openshift-security-profiles project.

Prerequisites

  • You have access to the web console as a user with cluster-admin privileges.
  • The Security Profiles Operator is installed.

Procedure

To remove the Security Profiles Operator by using the OpenShift Container Platform web console:

  1. Navigate to the Ecosystem Installed Operators page.
  2. Delete all seccomp profiles, SELinux profiles, and webhook configurations.
  3. Switch to the Administration Ecosystem Installed Operators page.
  4. Click the Options menu kebab on the Security Profiles Operator entry and select Uninstall Operator.
  5. Switch to the Home Projects page.
  6. Search for security profiles.
  7. Click the Options menu kebab next to the openshift-security-profiles project, and select Delete Project.

    1. Confirm the deletion by typing openshift-security-profiles in the dialog box, and click Delete.
  8. Delete the MutatingWebhookConfiguration object by running the following command:

    $ oc delete MutatingWebhookConfiguration spo-mutating-webhook-configuration
    Copy to Clipboard Toggle word wrap
Red Hat logoGithubredditYoutubeTwitter

Formazione

Prova, acquista e vendi

Community

Informazioni sulla documentazione di Red Hat

Aiutiamo gli utenti Red Hat a innovarsi e raggiungere i propri obiettivi con i nostri prodotti e servizi grazie a contenuti di cui possono fidarsi. Esplora i nostri ultimi aggiornamenti.

Rendiamo l’open source più inclusivo

Red Hat si impegna a sostituire il linguaggio problematico nel codice, nella documentazione e nelle proprietà web. Per maggiori dettagli, visita il Blog di Red Hat.

Informazioni su Red Hat

Forniamo soluzioni consolidate che rendono più semplice per le aziende lavorare su piattaforme e ambienti diversi, dal datacenter centrale all'edge della rete.

Theme

© 2026 Red Hat
Torna in cima