Chapter 7. Security Profiles Operator

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators OperatorHub.
2. Search for the Security Profiles Operator, then click Install.
3. Keep the default selection of Installation mode and namespace to ensure that the Operator will be installed to the openshift-security-profiles namespace.
4. Click Install.

1. Navigate to the Operators Installed Operators page.
2. Check that the Security Profiles Operator is installed in the openshift-security-profiles namespace and its status is Succeeded.

1. Navigate to the Operators Installed Operators page and inspect the Status column for any errors or failures.
2. Navigate to the Workloads Pods page and check the logs in any pods in the openshift-security-profiles project that are reporting issues.

Procedure

1. Define a Namespace object:

   Example namespace-object.yaml

   apiVersion: v1
   kind: Namespace
   metadata:
       name: openshift-security-profiles
   labels:
     openshift.io/cluster-monitoring: "true"

2. Create the Namespace object:

   $ oc create -f namespace-object.yaml

3. Define an OperatorGroup object:

   Example operator-group-object.yaml

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: security-profiles-operator
     namespace: openshift-security-profiles

4. Create the OperatorGroup object:

   $ oc create -f operator-group-object.yaml

5. Define a Subscription object:

   Example subscription-object.yaml

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: security-profiles-operator-sub
     namespace: openshift-security-profiles
   spec:
     channel: release-alpha-rhel-8
     installPlanApproval: Automatic
     name: security-profiles-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace

6. Create the Subscription object:

   $ oc create -f subscription-object.yaml

Verification

1. Verify the installation succeeded by inspecting the following CSV file:

   $ oc get csv -n openshift-security-profiles

2. Verify that the Security Profiles Operator is operational by running the following command:

   $ oc get deploy -n openshift-security-profiles

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Create the SeccompProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1beta1
   kind: SeccompProfile
   metadata:
     namespace: my-namespace
     name: profile1
   spec:
     defaultAction: SCMP_ACT_LOG

Procedure

1. Create a pod object that defines a securityContext:

   apiVersion: v1
   kind: Pod
   metadata:
     name: test-pod
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: Localhost
         localhostProfile: operator/my-namespace/profile1.json
     containers:
       - name: test-container
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

2. View the profile path of the seccompProfile.localhostProfile attribute by running the following command:

   $ oc -n my-namespace get seccompprofile profile1 --output wide

   Example output

   NAME       STATUS     AGE   SECCOMPPROFILE.LOCALHOSTPROFILE
   profile1   Installed  14s   operator/my-namespace/profile1.json

3. View the path to the localhost profile by running the following command:

   $ oc get sp profile1 --output=jsonpath='{.status.localhostProfile}'

   Example output

   operator/my-namespace/profile1.json

4. Apply the localhostProfile output to the patch file:

   spec:
     template:
       spec:
         securityContext:
           seccompProfile:
             type: Localhost
             localhostProfile: operator/my-namespace/profile1.json

5. Apply the profile to any other workload, such as a Deployment object, by running the following command:

   $ oc -n my-namespace patch deployment myapp --patch-file patch.yaml --type=merge

   Example output

   deployment.apps/myapp patched

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SeccompProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Example output

   I0523 14:19:08.747313  430694 enricher.go:445] log-enricher "msg"="audit" "container"="redis" "executable"="/usr/local/bin/redis-server" "namespace"="my-namespace" "node"="xiyuan-23-5g2q9-worker-eastus2-6rpgf" "pid"=656802 "pod"="my-pod" "syscallID"=0 "syscallName"="read" "timestamp"="1684851548.745:207179" "type"="seccomp"

Verification

1. Remove the pod:

   $ oc -n my-namepace delete pod my-pod

2. Confirm the Security Profiles Operator reconciles the two seccomp profiles:

   $ oc get seccompprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace

   Example output for seccompprofile

   NAME                   STATUS      AGE
   test-recording-nginx   Installed   2m48s
   test-recording-redis   Installed   2m48s

Procedure

1. Create a project by running the following command:

   $ oc new-project nginx-deploy

2. Create a policy that can be used with a non-privileged workload by creating the following SelinuxProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha2
   kind: SelinuxProfile
   metadata:
     name: nginx-secure
     namespace: nginx-deploy
   spec:
     allow:
       '@self':
         tcp_socket:
         - listen
       http_cache_port_t:
         tcp_socket:
         - name_bind
       node_t:
         tcp_socket:
         - node_bind
     inherit:
     - kind: System
       name: container

3. Wait for selinuxd to install the policy by running the following command:

   $ oc wait --for=condition=ready -n nginx-deploy selinuxprofile nginx-secure

   Example output

   selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure condition met

   The policies are placed into an emptyDir in the container owned by the Security Profiles Operator. The policies are saved in Common Intermediate Language (CIL) format in /etc/selinux.d/<name>_<namespace>.cil.

4. Access the pod by running the following command:

   $ oc -n openshift-security-profiles rsh -c selinuxd ds/spod

Verification

1. View the file contents with cat by running the following command:

   $ cat /etc/selinux.d/nginx-secure_nginx-deploy.cil

   Example output

   (block nginx-secure_nginx-deploy
   (blockinherit container)
   (allow process nginx-secure_nginx-deploy.process ( tcp_socket ( listen )))
   (allow process http_cache_port_t ( tcp_socket ( name_bind )))
   (allow process node_t ( tcp_socket ( node_bind )))
   )

2. Verify that a policy has been installed by running the following command:

   $ semodule -l | grep nginx-secure

   Example output

   nginx-secure_nginx-deploy

Procedure

1. Apply the scc.podSecurityLabelSync=false label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy security.openshift.io/scc.podSecurityLabelSync=false

2. Apply the privileged label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy --overwrite=true pod-security.kubernetes.io/enforce=privileged

3. Obtain the SELinux profile usage string by running the following command:

   $ oc get selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure -n nginx-deploy -ojsonpath='{.status.usage}'

   Example output

   nginx-secure_nginx-deploy.process

4. Apply the output string in the workload manifest in the .spec.containers[].securityContext.seLinuxOptions attribute:

   apiVersion: v1
   kind: Pod
   metadata:
     name: nginx-secure
     namespace: nginx-deploy
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - image: nginxinc/nginx-unprivileged:1.21
         name: nginx
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
           seLinuxOptions:
             # NOTE: This uses an appropriate SELinux type
             type: nginx-secure_nginx-deploy.process

   Important

   The SELinux type must exist before creating the workload.

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SelinuxProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Example output

   I0517 13:55:36.383187  348295 enricher.go:376] log-enricher "msg"="audit" "container"="redis" "namespace"="my-namespace" "node"="ip-10-0-189-53.us-east-2.compute.internal" "perm"="name_bind" "pod"="my-pod" "profile"="test-recording_redis_6kmrb_1684331729" "scontext"="system_u:system_r:selinuxrecording.process:s0:c4,c27" "tclass"="tcp_socket" "tcontext"="system_u:object_r:redis_port_t:s0" "timestamp"="1684331735.105:273965" "type"="selinux"

Verification

1. Remove the pod:

   $ oc -n my-namepace delete pod my-pod

2. Confirm the Security Profiles Operator reconciles the two SELinux profiles:

   $ oc get selinuxprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace

   Example output for selinuxprofile

   NAME                   USAGE                                       STATE
   test-recording-nginx   test-recording-nginx_my-namespace.process   Installed
   test-recording-redis   test-recording-redis_my-namespace.process   Installed

Procedure

1. Enable memory optimization by running the following command:

   $ oc -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableMemoryOptimization":true}}'

2. To record a security profile for a pod, the pod must be labeled with spo.x-k8s.io/enable-recording: "true":

   apiVersion: v1
   kind: Pod
   metadata:
     name: my-recording-pod
     labels:
       spo.x-k8s.io/enable-recording: "true"
   # ...

Procedure

1. To view the status of the metrics service, run the following command:

   $ oc get svc/metrics -n openshift-security-profiles

   Example output

   NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
   metrics   ClusterIP   10.0.0.228   <none>        443/TCP   43s

2. To retrieve the metrics, query the service endpoint using the default ServiceAccount token in the openshift-security-profiles namespace by running the following command:

   $ oc run --rm -i --restart=Never --image=registry.fedoraproject.org/fedora-minimal:latest \
       -n openshift-security-profiles metrics-test -- bash -c \
       'curl -ks -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://metrics.openshift-security-profiles/metrics-spod'

   Example output

   # HELP security_profiles_operator_seccomp_profile_total Counter about seccomp profile operations.
   # TYPE security_profiles_operator_seccomp_profile_total counter
   security_profiles_operator_seccomp_profile_total{operation="delete"} 1
   security_profiles_operator_seccomp_profile_total{operation="update"} 2

3. To retrieve metrics from a different namespace, link the ServiceAccount to the spo-metrics-client ClusterRoleBinding by running the following command:

   $ oc get clusterrolebinding spo-metrics-client -o wide

   Example output

   NAME                 ROLE                             AGE   USERS   GROUPS   SERVICEACCOUNTS
   spo-metrics-client   ClusterRole/spo-metrics-client   35m                    openshift-security-profiles/default

Procedure

1. Patch the spod configuration to enable the log enricher by running the following command:

   $ oc -n openshift-security-profiles patch spod spod \
       --type=merge -p '{"spec":{"enableLogEnricher":true}}'

   Example output

   securityprofilesoperatordaemon.security-profiles-operator.x-k8s.io/spod patched

   Note

   The Security Profiles Operator will re-deploy the spod daemon set automatically.

2. View the audit logs by running the following command:

   $ oc -n openshift-security-profiles logs -f ds/spod log-enricher

   Example output

   I0623 12:51:04.257814 1854764 deleg.go:130] setup "msg"="starting component: log-enricher"  "buildDate"="1980-01-01T00:00:00Z" "compiler"="gc" "gitCommit"="unknown" "gitTreeState"="clean" "goVersion"="go1.16.2" "platform"="linux/amd64" "version"="0.4.0-dev"
   I0623 12:51:04.257890 1854764 enricher.go:44] log-enricher "msg"="Starting log-enricher on node: 127.0.0.1"
   I0623 12:51:04.257898 1854764 enricher.go:46] log-enricher "msg"="Connecting to local GRPC server"
   I0623 12:51:04.258061 1854764 enricher.go:69] log-enricher "msg"="Reading from file /var/log/audit/audit.log"
   2021/06/23 12:51:04 Seeked /var/log/audit/audit.log - &{Offset:0 Whence:2}

Procedure

1. Set the recording.spo.io webhook configuration to record only pods labeled with spo-record=true by creating the following patch file:

   spec:
     webhookOptions:
       - name: recording.spo.io
         objectSelector:
           matchExpressions:
             - key: spo-record
               operator: In
               values:
                 - "true"

2. Patch the spod/spod instance by running the following command:

   $ oc -n openshift-security-profiles patch spod \
       spod -p $(cat /tmp/spod-wh.patch) --type=merge

3. To view the resulting MutatingWebhookConfiguration object, run the following command:

   $ oc get MutatingWebhookConfiguration \
       spo-mutating-webhook-configuration -oyaml

Procedure

1. Confirm that the profile is reconciled by running the following command:

   $ oc -n openshift-security-profiles logs openshift-security-profiles-<id>

   Example 7.2. Example output

   I1019 19:34:14.942464       1 main.go:90] setup "msg"="starting openshift-security-profiles"  "buildDate"="2020-10-19T19:31:24Z" "compiler"="gc" "gitCommit"="a3ef0e1ea6405092268c18f240b62015c247dd9d" "gitTreeState"="dirty" "goVersion"="go1.15.1" "platform"="linux/amd64" "version"="0.2.0-dev"
   I1019 19:34:15.348389       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"=":8080"
   I1019 19:34:15.349076       1 main.go:126] setup "msg"="starting manager"
   I1019 19:34:15.349449       1 internal.go:391] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
   I1019 19:34:15.350201       1 controller.go:142] controller "msg"="Starting EventSource" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"defaultAction":""}}}
   I1019 19:34:15.450674       1 controller.go:149] controller "msg"="Starting Controller" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile"
   I1019 19:34:15.450757       1 controller.go:176] controller "msg"="Starting workers" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "worker count"=1
   I1019 19:34:15.453102       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="nginx-1.19.1" "name"="nginx-1.19.1" "resource version"="728"
   I1019 19:34:15.453618       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="openshift-security-profiles" "name"="openshift-security-profiles" "resource version"="729"

2. Confirm that the seccomp profiles are saved into the correct path by running the following command:

   $ oc exec -t -n openshift-security-profiles openshift-security-profiles-<id> \
       -- ls /var/lib/kubelet/seccomp/operator/my-namespace/my-workload

   Example output

   profile-block.json
   profile-complain.json

1. Navigate to the Operators Installed Operators page.
2. Delete all seccomp profiles, SELinux profiles, and webhook configurations.
3. Switch to the Administration Operators Installed Operators page.
4. Click the Options menu on the Security Profiles Operator entry and select Uninstall Operator.
5. Switch to the Home Projects page.
6. Search for security profiles.

7. Click the Options menu next to the openshift-security-profiles project, and select Delete Project.

   1. Confirm the deletion by typing openshift-security-profiles in the dialog box, and click Delete.

8. Delete the MutatingWebhookConfiguration object by running the following command:

   $ oc delete MutatingWebhookConfiguration spo-mutating-webhook-configuration