Chapter 6. Building of container images using Buildah as a non-root user


Running OpenShift Pipelines as the root user on a container can expose the container processes and the host to other potentially malicious resources. You can reduce this type of exposure by running the workload as a specific non-root user in the container.

In most cases, you can run Buildah without root privileges by creating a custom task for building the image and configuring user namespaces in this task.

If your image does not build successfully using this configuration, you can use custom service account (SA) and security context constraint (SCC) definitions; however, if you use this option, you must enable the Buildah step to raise its privileges (allowPrivilegeEscalation: true).

6.1. Running Buildah as a non-root user by configuring user namespaces

Configuring user namespaces is the simplest way to run Buildah in a task as a non-root user. However, some images might not build using this option.

Prerequisites

  • You have installed the oc command-line utility.

Procedure

  1. To create a copy of the buildah task, which is provided in the openshift-pipelines namespace, and to change the name of the copy to buildah-as-user, enter the following command:

    $ oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f -
  2. Edit the copied buildah task by entering the following command:

    $ oc edit task buildah-as-user

    In the new task, create annotations and stepTemplate sections, as shown in the following example:

    Example additions to the buildah-as-user task

    apiVersion: tekton.dev/v1
    kind: Task
    metadata:
      annotations:
        io.kubernetes.cri-o.userns-mode: 'auto:size=65536;map-to-root=true'
        io.openshift.builder: 'true'
      name: assemble-containerimage
      namespace: pipeline-namespace
    spec:
      description: This task builds an image.
    #  ...
      stepTemplate:
        env:
          - name: HOME
            value: /tekton/home
        image: $(params.builder-image)
        imagePullPolicy: IfNotPresent
        name: ''
        resources:
          limits:
            cpu: '1'
            memory: 4Gi
          requests:
            cpu: 100m
            memory: 2Gi
        securityContext:
          capabilities:
            add:
              - SETFCAP
          runAsNonRoot: true
          runAsUser: 1000 1
        workingDir: $(workspaces.working-directory.path)
    #  ...

    1
    The runAsUser: setting is not strictly necessary, because podTemplate is used.
  3. Use the new buildah-as-user task to build the image in your pipeline.

6.2. Running Buildah as a non-root user by defining a custom SA and SCC

To run builds of container images using Buildah as a non-root user, you can perform the following steps:

  • Define custom service account (SA) and security context constraint (SCC).
  • Configure Buildah to use the build user with id 1000.
  • Start a task run with a custom config map, or integrate it with a pipeline run.

6.2.1. Configuring custom service account and security context constraint

The default pipeline SA allows using a user id outside of the namespace range. To reduce dependency on the default SA, you can define a custom SA and SCC with necessary cluster role and role bindings for the build user with user id 1000.

Important

At this time, enabling the allowPrivilegeEscalation setting is required for Buildah to run successfully in the container. With this setting, Buildah can leverage SETUID and SETGID capabilities when running as a non-root user.

Procedure

  • Create a custom SA and SCC with necessary cluster role and role bindings.

    Example: Custom SA and SCC for used id 1000

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: pipelines-sa-userid-1000 1
    ---
    kind: SecurityContextConstraints
    metadata:
      annotations:
      name: pipelines-scc-userid-1000 2
    allowHostDirVolumePlugin: false
    allowHostIPC: false
    allowHostNetwork: false
    allowHostPID: false
    allowHostPorts: false
    allowPrivilegeEscalation: true 3
    allowPrivilegedContainer: false
    allowedCapabilities: null
    apiVersion: security.openshift.io/v1
    defaultAddCapabilities: null
    fsGroup:
      type: MustRunAs
    groups:
    - system:cluster-admins
    priority: 10
    readOnlyRootFilesystem: false
    requiredDropCapabilities:
    - MKNOD
    - KILL
    runAsUser: 4
      type: MustRunAs
      uid: 1000
    seLinuxContext:
      type: MustRunAs
    supplementalGroups:
      type: RunAsAny
    users: []
    volumes:
    - configMap
    - downwardAPI
    - emptyDir
    - persistentVolumeClaim
    - projected
    - secret
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: pipelines-scc-userid-1000-clusterrole 5
    rules:
    - apiGroups:
      - security.openshift.io
      resourceNames:
      - pipelines-scc-userid-1000
      resources:
      - securitycontextconstraints
      verbs:
      - use
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: pipelines-scc-userid-1000-rolebinding 6
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: pipelines-scc-userid-1000-clusterrole
    subjects:
    - kind: ServiceAccount
      name: pipelines-sa-userid-1000

1
Define a custom SA.
2
Define a custom SCC created based on restricted privileges, with modified runAsUser field.
3
At this time, enabling the allowPrivilegeEscalation setting is required for Buildah to run successfully in the container. With this setting, Buildah can leverage SETUID and SETGID capabilities when running as a non-root user.
4
Restrict any pod that gets attached with the custom SCC through the custom SA to run as user id 1000.
5
Define a cluster role that uses the custom SCC.
6
Bind the cluster role that uses the custom SCC to the custom SA.

6.2.2. Configuring Buildah to use build user

You can define a Buildah task to use the build user with user id 1000.

Procedure

  1. Create a copy of the buildah task, which is provided in the openshift-pipelines namespace; change the name of the copy to buildah-as-user.

    $ oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f -
  2. Edit the copied buildah task.

    $ oc edit task buildah-as-user

    Example: Modified Buildah task with build user

    apiVersion: tekton.dev/v1
    kind: Task
    metadata:
      name: buildah-as-user
    spec:
      description: >-
        Buildah task builds source into a container image and
        then pushes it to a container registry.
        Buildah Task builds source into a container image using Project Atomic's
        Buildah build tool.It uses Buildah's support for building from Dockerfiles,
        using its buildah bud command.This command executes the directives in the
        Dockerfile to assemble a container image, then pushes that image to a
        container registry.
      params:
      - name: IMAGE
        description: Reference of the image buildah will produce.
      - name: BUILDER_IMAGE
        description: The location of the buildah builder image.
        default: registry.redhat.io/rhel8/buildah@sha256:99cae35f40c7ec050fed3765b2b27e0b8bbea2aa2da7c16408e2ca13c60ff8ee
      - name: STORAGE_DRIVER
        description: Set buildah storage driver
        default: vfs
      - name: DOCKERFILE
        description: Path to the Dockerfile to build.
        default: ./Dockerfile
      - name: CONTEXT
        description: Path to the directory to use as context.
        default: .
      - name: TLSVERIFY
        description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
        default: "true"
      - name: FORMAT
        description: The format of the built container, oci or docker
        default: "oci"
      - name: BUILD_EXTRA_ARGS
        description: Extra parameters passed for the build command when building images.
        default: ""
      - description: Extra parameters passed for the push command when pushing images.
        name: PUSH_EXTRA_ARGS
        type: string
        default: ""
      - description: Skip pushing the built image
        name: SKIP_PUSH
        type: string
        default: "false"
      results:
      - description: Digest of the image just built.
        name: IMAGE_DIGEST
        type: string
      workspaces:
      - name: source
      steps:
      - name: build
        securityContext:
          runAsUser: 1000 1
        image: $(params.BUILDER_IMAGE)
        workingDir: $(workspaces.source.path)
        script: |
          echo "Running as USER ID `id`" 2
          buildah --storage-driver=$(params.STORAGE_DRIVER) bud \
            $(params.BUILD_EXTRA_ARGS) --format=$(params.FORMAT) \
            --tls-verify=$(params.TLSVERIFY) --no-cache \
            -f $(params.DOCKERFILE) -t $(params.IMAGE) $(params.CONTEXT)
          [[ "$(params.SKIP_PUSH)" == "true" ]] && echo "Push skipped" && exit 0
          buildah --storage-driver=$(params.STORAGE_DRIVER) push \
            $(params.PUSH_EXTRA_ARGS) --tls-verify=$(params.TLSVERIFY) \
            --digestfile $(workspaces.source.path)/image-digest $(params.IMAGE) \
            docker://$(params.IMAGE)
          cat $(workspaces.source.path)/image-digest | tee /tekton/results/IMAGE_DIGEST
        volumeMounts:
        - name: varlibcontainers
          mountPath: /home/build/.local/share/containers 3
      volumes:
      - name: varlibcontainers
        emptyDir: {}

    1
    Run the container explicitly as the user id 1000, which corresponds to the build user in the Buildah image.
    2
    Display the user id to confirm that the process is running as user id 1000.
    3
    You can change the path for the volume mount as necessary.

6.2.3. Starting a task run with custom config map, or a pipeline run

After defining the custom Buildah task, you can create a TaskRun object that builds an image as a build user with user id 1000. In addition, you can integrate the TaskRun object as part of a PipelineRun object.

Procedure

  1. Create a TaskRun object with a custom ConfigMap and Dockerfile objects.

    Example: A task run that runs Buildah as user id 1000

    apiVersion: v1
    data:
      Dockerfile: |
        ARG BASE_IMG=registry.access.redhat.com/ubi9/ubi
        FROM $BASE_IMG AS buildah-runner
        RUN dnf -y update && \
            dnf -y install git && \
            dnf clean all
        CMD git
    kind: ConfigMap
    metadata:
      name: dockerfile 1
    ---
    apiVersion: tekton.dev/v1
    kind: TaskRun
    metadata:
      name: buildah-as-user-1000
    spec:
      taskRunTemplate:
        serviceAccountName: pipelines-sa-userid-1000 2
      params:
      - name: IMAGE
        value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
      taskRef:
        kind: Task
        name: buildah-as-user
      workspaces:
      - configMap:
          name: dockerfile 3
        name: source

    1
    Use a config map because the focus is on the task run, without any prior task that fetches some sources with a Dockerfile.
    2
    The name of the service account that you created.
    3
    Mount a config map as the source workspace for the buildah-as-user task.
  2. (Optional) Create a pipeline and a corresponding pipeline run.

    Example: A pipeline and corresponding pipeline run

    apiVersion: tekton.dev/v1
    kind: Pipeline
    metadata:
      name: pipeline-buildah-as-user-1000
    spec:
      params:
      - name: IMAGE
      - name: URL
      workspaces:
      - name: shared-workspace
      - name: sslcertdir
        optional: true
      tasks:
      - name: fetch-repository 1
        taskRef:
          resolver: cluster
          params:
          - name: kind
            value: task
          - name: name
            value: git-clone
          - name: namespace
            value: openshift-pipelines
        workspaces:
        - name: output
          workspace: shared-workspace
        params:
        - name: URL
          value: $(params.URL)
        - name: SUBDIRECTORY
          value: ""
        - name: DELETE_EXISTING
          value: "true"
      - name: buildah
        taskRef:
          name: buildah-as-user 2
        runAfter:
        - fetch-repository
        workspaces:
        - name: source
          workspace: shared-workspace
        - name: sslcertdir
          workspace: sslcertdir
        params:
        - name: IMAGE
          value: $(params.IMAGE)
    ---
    apiVersion: tekton.dev/v1
    kind: PipelineRun
    metadata:
      name: pipelinerun-buildah-as-user-1000
    spec:
      taskRunSpecs:
        - pipelineTaskName: buildah
          taskServiceAccountName: pipelines-sa-userid-1000 3
      params:
      - name: URL
        value: https://github.com/openshift/pipelines-vote-api
      - name: IMAGE
        value: image-registry.openshift-image-registry.svc:5000/test/buildahuser
      pipelineRef:
        name: pipeline-buildah-as-user-1000
      workspaces:
      - name: shared-workspace 4
        volumeClaimTemplate:
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 100Mi

    1
    Use the git-clone task to fetch the source containing a Dockerfile and build it using the modified Buildah task.
    2
    Refer to the modified Buildah task.
    3
    Use the service account that you created for the Buildah task.
    4
    Share data between the git-clone task and the modified Buildah task using a persistent volume claim (PVC) created automatically by the controller.
  3. Start the task run or the pipeline run.

6.3. Limitations of unprivileged builds

The process for unprivileged builds works with most Dockerfile objects. However, there are some known limitations might cause a build to fail:

  • Using the --mount=type=cache option might fail due to lack of necessay permissions issues. For more information, see this article.
  • Using the --mount=type=secret option fails because mounting resources requires additionnal capabilities that are not provided by the custom SCC.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.