Chapter 7. Security Profiles Operator

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators OperatorHub.
2. Search for the Security Profiles Operator, then click Install.
3. Keep the default selection of Installation mode and namespace to ensure that the Operator will be installed to the openshift-security-profiles namespace.
4. Click Install.

1. Navigate to the Operators Installed Operators page.
2. Check that the Security Profiles Operator is installed in the openshift-security-profiles namespace and its status is Succeeded.

1. Navigate to the Operators Installed Operators page and inspect the Status column for any errors or failures.
2. Navigate to the Workloads Pods page and check the logs in any pods in the openshift-security-profiles project that are reporting issues.

Procedure

1. Define a Namespace object:

   Example namespace-object.yaml

   apiVersion: v1
   kind: Namespace
   metadata:
       name: openshift-security-profiles
   labels:
     openshift.io/cluster-monitoring: "true"

   Copy to Clipboard Toggle word wrap

2. Create the Namespace object:

   $ oc create -f namespace-object.yaml

   Copy to Clipboard Toggle word wrap

3. Define an OperatorGroup object:

   Example operator-group-object.yaml

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: security-profiles-operator
     namespace: openshift-security-profiles

   Copy to Clipboard Toggle word wrap

4. Create the OperatorGroup object:

   $ oc create -f operator-group-object.yaml

   Copy to Clipboard Toggle word wrap

5. Define a Subscription object:

   Example subscription-object.yaml

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: security-profiles-operator-sub
     namespace: openshift-security-profiles
   spec:
     channel: release-alpha-rhel-8
     installPlanApproval: Automatic
     name: security-profiles-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace

   Copy to Clipboard Toggle word wrap

6. Create the Subscription object:

   $ oc create -f subscription-object.yaml

   Copy to Clipboard Toggle word wrap

Verification

1. Verify the installation succeeded by inspecting the following CSV file:

   $ oc get csv -n openshift-security-profiles

   Copy to Clipboard Toggle word wrap

2. Verify that the Security Profiles Operator is operational by running the following command:

   $ oc get deploy -n openshift-security-profiles

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

   Copy to Clipboard Toggle word wrap

2. Create the SeccompProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1beta1
   kind: SeccompProfile
   metadata:
     name: profile1
   spec:
     defaultAction: SCMP_ACT_LOG

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a pod object that defines a securityContext:

   apiVersion: v1
   kind: Pod
   metadata:
     name: test-pod
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: Localhost
         localhostProfile: operator/profile1.json
     containers:
       - name: test-container
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

   Copy to Clipboard Toggle word wrap

2. View the profile path of the seccompProfile.localhostProfile attribute by running the following command:

   $ oc get seccompprofile profile1 --output wide

   Copy to Clipboard Toggle word wrap

   Example output

   NAME       STATUS     AGE   SECCOMPPROFILE.LOCALHOSTPROFILE
   profile1   Installed  14s   operator/profile1.json

   Copy to Clipboard Toggle word wrap

3. View the path to the localhost profile by running the following command:

   $ oc get sp profile1 --output=jsonpath='{.status.localhostProfile}'

   Copy to Clipboard Toggle word wrap

   Example output

   operator/profile1.json

   Copy to Clipboard Toggle word wrap

4. Apply the localhostProfile output to the patch file:

   spec:
     template:
       spec:
         securityContext:
           seccompProfile:
             type: Localhost
             localhostProfile: operator/profile1.json

   Copy to Clipboard Toggle word wrap

5. Apply the profile to any other workload, such as a Deployment object, by running the following command:

   $ oc -n my-namespace patch deployment myapp --patch-file patch.yaml --type=merge

   Copy to Clipboard Toggle word wrap

   Example output

   deployment.apps/myapp patched

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

   Copy to Clipboard Toggle word wrap

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

   Copy to Clipboard Toggle word wrap

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SeccompProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

   Copy to Clipboard Toggle word wrap

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

   Copy to Clipboard Toggle word wrap

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Copy to Clipboard Toggle word wrap

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

   Copy to Clipboard Toggle word wrap

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Copy to Clipboard Toggle word wrap

   Example output

   I0523 14:19:08.747313  430694 enricher.go:445] log-enricher "msg"="audit" "container"="redis" "executable"="/usr/local/bin/redis-server" "namespace"="my-namespace" "node"="xiyuan-23-5g2q9-worker-eastus2-6rpgf" "pid"=656802 "pod"="my-pod" "syscallID"=0 "syscallName"="read" "timestamp"="1684851548.745:207179" "type"="seccomp"

   Copy to Clipboard Toggle word wrap

Verification

1. Remove the pod:

   $ oc -n my-namespace delete pod my-pod

   Copy to Clipboard Toggle word wrap

2. Confirm the Security Profiles Operator reconciles the two seccomp profiles:

   $ oc get seccompprofiles -lspo.x-k8s.io/recording-id=test-recording

   Copy to Clipboard Toggle word wrap

   Example output for seccompprofile

   NAME                   STATUS      AGE
   test-recording-nginx   Installed   2m48s
   test-recording-redis   Installed   2m48s

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a project by running the following command:

   $ oc new-project nginx-deploy

   Copy to Clipboard Toggle word wrap

2. Create a policy that can be used with a non-privileged workload by creating the following SelinuxProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha2
   kind: SelinuxProfile
   metadata:
     name: nginx-secure
   spec:
     allow:
       '@self':
         tcp_socket:
         - listen
       http_cache_port_t:
         tcp_socket:
         - name_bind
       node_t:
         tcp_socket:
         - node_bind
     inherit:
     - kind: System
       name: container

   Copy to Clipboard Toggle word wrap

3. Wait for selinuxd to install the policy by running the following command:

   $ oc wait --for=condition=ready selinuxprofile nginx-secure

   Copy to Clipboard Toggle word wrap

   Example output

   selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure condition met

   Copy to Clipboard Toggle word wrap

   The policies are placed into an emptyDir in the container owned by the Security Profiles Operator. The policies are saved in Common Intermediate Language (CIL) format in /etc/selinux.d/<name>_<namespace>.cil.

4. Access the pod by running the following command:

   $ oc -n openshift-security-profiles rsh -c selinuxd ds/spod

   Copy to Clipboard Toggle word wrap

Verification

1. View the file contents with cat by running the following command:

   $ cat /etc/selinux.d/nginx-secure_.cil

   Copy to Clipboard Toggle word wrap

   Example output

   (block nginx-secure_
   (blockinherit container)
   (allow process nginx-secure_.process ( tcp_socket ( listen )))
   (allow process http_cache_port_t ( tcp_socket ( name_bind )))
   (allow process node_t ( tcp_socket ( node_bind )))
   )

   Copy to Clipboard Toggle word wrap

2. Verify that a policy has been installed by running the following command:

   $ semodule -l | grep nginx-secure

   Copy to Clipboard Toggle word wrap

   Example output

   nginx-secure_

   Copy to Clipboard Toggle word wrap

Procedure

1. Apply the scc.podSecurityLabelSync=false label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy security.openshift.io/scc.podSecurityLabelSync=false

   Copy to Clipboard Toggle word wrap

2. Apply the privileged label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy --overwrite=true pod-security.kubernetes.io/enforce=privileged

   Copy to Clipboard Toggle word wrap

3. Obtain the SELinux profile usage string by running the following command:

   $ oc get selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure -ojsonpath='{.status.usage}'

   Copy to Clipboard Toggle word wrap

   Example output

   nginx-secure_.process

   Copy to Clipboard Toggle word wrap

4. Apply the output string in the workload manifest in the .spec.containers[].securityContext.seLinuxOptions attribute:

   apiVersion: v1
   kind: Pod
   metadata:
     name: nginx-secure
     namespace: nginx-deploy
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - image: nginxinc/nginx-unprivileged:1.21
         name: nginx
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
           seLinuxOptions:
             # NOTE: This uses an appropriate SELinux type
             type: nginx-secure_.process

   Copy to Clipboard Toggle word wrap
   Important

   The SELinux type must exist before creating the workload.

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

   Copy to Clipboard Toggle word wrap

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

   Copy to Clipboard Toggle word wrap

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SelinuxProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

   Copy to Clipboard Toggle word wrap

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     securityContext:
       runAsNonRoot: true
       seccompProfile:
         type: RuntimeDefault
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1
         securityContext:
           allowPrivilegeEscalation: false
           capabilities:
             drop: [ALL]

   Copy to Clipboard Toggle word wrap

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Copy to Clipboard Toggle word wrap

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

   Copy to Clipboard Toggle word wrap

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Copy to Clipboard Toggle word wrap

   Example output

   I0517 13:55:36.383187  348295 enricher.go:376] log-enricher "msg"="audit" "container"="redis" "namespace"="my-namespace" "node"="ip-10-0-189-53.us-east-2.compute.internal" "perm"="name_bind" "pod"="my-pod" "profile"="test-recording_redis_6kmrb_1684331729" "scontext"="system_u:system_r:selinuxrecording.process:s0:c4,c27" "tclass"="tcp_socket" "tcontext"="system_u:object_r:redis_port_t:s0" "timestamp"="1684331735.105:273965" "type"="selinux"

   Copy to Clipboard Toggle word wrap

Verification

1. Remove the pod:

   $ oc -n my-namespace delete pod my-pod

   Copy to Clipboard Toggle word wrap

2. Confirm the Security Profiles Operator reconciles the two SELinux profiles:

   $ oc get selinuxprofiles -lspo.x-k8s.io/recording-id=test-recording

   Copy to Clipboard Toggle word wrap

   Example output for selinuxprofile

   NAME                   USAGE                                       STATE
   test-recording-nginx   test-recording-nginx_.process   Installed
   test-recording-redis   test-recording-redis_.process   Installed

   Copy to Clipboard Toggle word wrap

Procedure

1. Enable memory optimization by running the following command:

   $ oc -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableMemoryOptimization":true}}'

   Copy to Clipboard Toggle word wrap

2. To record a security profile for a pod, the pod must be labeled with spo.x-k8s.io/enable-recording: "true":

   apiVersion: v1
   kind: Pod
   metadata:
     name: my-recording-pod
     labels:
       spo.x-k8s.io/enable-recording: "true"
   # ...

   Copy to Clipboard Toggle word wrap

Procedure

1. To view the status of the metrics service, run the following command:

   $ oc get svc/metrics -n openshift-security-profiles

   Copy to Clipboard Toggle word wrap

   Example output

   NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
   metrics   ClusterIP   10.0.0.228   <none>        443/TCP   43s

   Copy to Clipboard Toggle word wrap

2. To retrieve the metrics, query the service endpoint using the default ServiceAccount token in the openshift-security-profiles namespace by running the following command:

   $ oc run --rm -i --restart=Never --image=registry.fedoraproject.org/fedora-minimal:latest \
       -n openshift-security-profiles metrics-test -- bash -c \
       'curl -ks -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://metrics.openshift-security-profiles/metrics-spod'

   Copy to Clipboard Toggle word wrap

   Example output

   # HELP security_profiles_operator_seccomp_profile_total Counter about seccomp profile operations.
   # TYPE security_profiles_operator_seccomp_profile_total counter
   security_profiles_operator_seccomp_profile_total{operation="delete"} 1
   security_profiles_operator_seccomp_profile_total{operation="update"} 2

   Copy to Clipboard Toggle word wrap

3. To retrieve metrics from a different namespace, link the ServiceAccount to the spo-metrics-client ClusterRoleBinding by running the following command:

   $ oc get clusterrolebinding spo-metrics-client -o wide

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                 ROLE                             AGE   USERS   GROUPS   SERVICEACCOUNTS
   spo-metrics-client   ClusterRole/spo-metrics-client   35m                    openshift-security-profiles/default

   Copy to Clipboard Toggle word wrap

Procedure

1. Patch the spod configuration to enable the log enricher by running the following command:

   $ oc -n openshift-security-profiles patch spod spod \
       --type=merge -p '{"spec":{"enableLogEnricher":true}}'

   Copy to Clipboard Toggle word wrap

   Example output

   securityprofilesoperatordaemon.security-profiles-operator.x-k8s.io/spod patched

   Copy to Clipboard Toggle word wrap

   Note

   The Security Profiles Operator will re-deploy the spod daemon set automatically.

2. View the audit logs by running the following command:

   $ oc -n openshift-security-profiles logs -f ds/spod log-enricher

   Copy to Clipboard Toggle word wrap

   Example output

   I0623 12:51:04.257814 1854764 deleg.go:130] setup "msg"="starting component: log-enricher"  "buildDate"="1980-01-01T00:00:00Z" "compiler"="gc" "gitCommit"="unknown" "gitTreeState"="clean" "goVersion"="go1.16.2" "platform"="linux/amd64" "version"="0.4.0-dev"
   I0623 12:51:04.257890 1854764 enricher.go:44] log-enricher "msg"="Starting log-enricher on node: 127.0.0.1"
   I0623 12:51:04.257898 1854764 enricher.go:46] log-enricher "msg"="Connecting to local GRPC server"
   I0623 12:51:04.258061 1854764 enricher.go:69] log-enricher "msg"="Reading from file /var/log/audit/audit.log"
   2021/06/23 12:51:04 Seeked /var/log/audit/audit.log - &{Offset:0 Whence:2}

   Copy to Clipboard Toggle word wrap

Procedure

1. Set the recording.spo.io webhook configuration to record only pods labeled with spo-record=true by creating the following patch file:

   spec:
     webhookOptions:
       - name: recording.spo.io
         objectSelector:
           matchExpressions:
             - key: spo-record
               operator: In
               values:
                 - "true"

   Copy to Clipboard Toggle word wrap

2. Patch the spod/spod instance by running the following command:

   $ oc -n openshift-security-profiles patch spod \
       spod -p $(cat /tmp/spod-wh.patch) --type=merge

   Copy to Clipboard Toggle word wrap

3. To view the resulting MutatingWebhookConfiguration object, run the following command:

   $ oc get MutatingWebhookConfiguration \
       spo-mutating-webhook-configuration -oyaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Confirm that the profile is reconciled by running the following command:

   $ oc -n openshift-security-profiles logs openshift-security-profiles-<id>

   Copy to Clipboard Toggle word wrap

   Example 7.2. Example output

   I1019 19:34:14.942464       1 main.go:90] setup "msg"="starting openshift-security-profiles"  "buildDate"="2020-10-19T19:31:24Z" "compiler"="gc" "gitCommit"="a3ef0e1ea6405092268c18f240b62015c247dd9d" "gitTreeState"="dirty" "goVersion"="go1.15.1" "platform"="linux/amd64" "version"="0.2.0-dev"
   I1019 19:34:15.348389       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"=":8080"
   I1019 19:34:15.349076       1 main.go:126] setup "msg"="starting manager"
   I1019 19:34:15.349449       1 internal.go:391] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
   I1019 19:34:15.350201       1 controller.go:142] controller "msg"="Starting EventSource" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"defaultAction":""}}}
   I1019 19:34:15.450674       1 controller.go:149] controller "msg"="Starting Controller" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile"
   I1019 19:34:15.450757       1 controller.go:176] controller "msg"="Starting workers" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "worker count"=1
   I1019 19:34:15.453102       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="nginx-1.19.1" "name"="nginx-1.19.1" "resource version"="728"
   I1019 19:34:15.453618       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="openshift-security-profiles" "name"="openshift-security-profiles" "resource version"="729"

   Copy to Clipboard Toggle word wrap

2. Confirm that the seccomp profiles are saved into the correct path by running the following command:

   $ oc exec -t -n openshift-security-profiles openshift-security-profiles-<id> \
       -- ls /var/lib/kubelet/seccomp/operator/my-namespace/my-workload

   Copy to Clipboard Toggle word wrap

   Example output

   profile-block.json
   profile-complain.json

   Copy to Clipboard Toggle word wrap

1. Navigate to the Operators Installed Operators page.
2. Delete all seccomp profiles, SELinux profiles, and webhook configurations.
3. Switch to the Administration Operators Installed Operators page.
4. Click the Options menu on the Security Profiles Operator entry and select Uninstall Operator.
5. Switch to the Home Projects page.
6. Search for security profiles.

7. Click the Options menu next to the openshift-security-profiles project, and select Delete Project.

   1. Confirm the deletion by typing openshift-security-profiles in the dialog box, and click Delete.

8. Delete the MutatingWebhookConfiguration object by running the following command:

   $ oc delete MutatingWebhookConfiguration spo-mutating-webhook-configuration

   Copy to Clipboard Toggle word wrap