Chapter 3. Use fuse-overlayfs for containers
Use the fuse-overlayfs storage driver for Podman and Buildah in OpenShift Dev Spaces workspaces.
3.1. The fuse-overlayfs storage driver for Podman and Buildah Copy linkLink copied to clipboard!
By default, newly created workspaces that do not specify a devfile use the Universal Developer Image (UDI). The UDI contains common development tools and dependencies commonly used by developers.
Podman and Buildah are included in the UDI, allowing developers to build and push container images from their workspace.
By default, Podman and Buildah in the UDI are configured to use the vfs storage driver. For more efficient image management, use the fuse-overlayfs storage driver which supports copy-on-write in rootless environments.
You must meet the following requirements to use fuse-overlayfs in a workspace:
-
For OpenShift versions older than 4.15, the administrator has enabled
/dev/fuseaccess on the cluster. -
The workspace has the necessary annotations for using the
/dev/fusedevice. -
The
storage.conffile in the workspace container is configured to use fuse-overlayfs.
3.2. Access /dev/fuse in workspace containers Copy linkLink copied to clipboard!
Enable access to /dev/fuse in workspace containers to use fuse-overlayfs as a storage driver for Podman.
Prerequisites
-
For OpenShift versions older than 4.15: you have administrator-enabled access to
/dev/fuse. See Configuring fuse-overlayfs. - You have identified a workspace to use with fuse-overlayfs.
Procedure
Use the
pod-overridesattribute to add the required annotations defined in Configuring fuse-overlayfs to the workspace. Thepod-overridesattribute allows merging certain fields in the workspace pod’sspec.For OpenShift versions older than 4.15:
$ oc patch devworkspace <DevWorkspace_name> \ --patch '{"spec":{"template":{"attributes":{"pod-overrides":{"metadata":{"annotations":{"io.kubernetes.cri-o.Devices":"/dev/fuse","io.openshift.podman-fuse":""}}}}}}}' \ --type=mergeFor OpenShift version 4.15 and later:
$ oc patch devworkspace <DevWorkspace_name> \ --patch '{"spec":{"template":{"attributes":{"pod-overrides":{"metadata":{"annotations":{"io.kubernetes.cri-o.Devices":"/dev/fuse"}}}}}}}' \ --type=merge
Verification
Start the workspace and verify that
/dev/fuseis available in the workspace container:$ stat /dev/fuse
3.3. Enable fuse-overlayfs with a ConfigMap Copy linkLink copied to clipboard!
Enable fuse-overlayfs as the storage driver for Podman and Buildah by mounting a storage.conf ConfigMap into all workspaces in your project.
Here are the default contents of the /home/user/.config/containers/storage.conf file in the UDI container:
# storage.conf
[storage]
driver = "vfs"
To use fuse-overlayfs, storage.conf can be set to the following:
# storage.conf
[storage]
driver = "overlay"
[storage.options.overlay]
mount_program="/usr/bin/fuse-overlayfs"
where:
mount_program-
The absolute path to the
fuse-overlayfsbinary. The/usr/bin/fuse-overlayfspath is the default for the UDI.
You can do this manually after starting a workspace. Another option is to build a new image based on the UDI with changes to storage.conf and use the new image for workspaces.
Otherwise, you can update the /home/user/.config/containers/storage.conf for all workspaces in your project by creating a ConfigMap that mounts the updated file. See Section 7.7, “Mount ConfigMaps”.
Prerequisites
-
For OpenShift versions older than 4.15, the administrator has enabled access to
/dev/fuse. See Configuring fuse-overlayfs. You have a workspace with the required annotations as described in Section 3.2, “Access /dev/fuse in workspace containers”.
NoteConfigMaps mounted by following this guide apply to all workspaces and set the storage driver to fuse-overlayfs globally. Verify that your workspaces contain the required annotations to use fuse-overlayfs as described in Section 3.2, “Access /dev/fuse in workspace containers”.
Procedure
Create a ConfigMap that mounts a
/home/user/.config/containers/storage.conffile:kind: ConfigMap apiVersion: v1 metadata: name: fuse-overlay labels: controller.devfile.io/mount-to-devworkspace: 'true' controller.devfile.io/watch-configmap: 'true' annotations: controller.devfile.io/mount-as: subpath controller.devfile.io/mount-path: /home/user/.config/containers data: storage.conf: | [storage] driver = "overlay" [storage.options.overlay] mount_program = "/usr/bin/fuse-overlayfs"Apply the ConfigMap to your project:
WarningApplying this ConfigMap causes all running workspaces in the project to restart.
$ oc apply -f fuse-overlay.yaml -n <your_namespace>- Start or restart your workspace.
Verification
Verify that the storage driver is
overlay:$ podman info | grep overlayExample output:
graphDriverName: overlay overlay.mount_program: Executable: /usr/bin/fuse-overlayfs Package: fuse-overlayfs-1.12-1.module+el8.9.0+20326+387084d0.x86_64 fuse-overlayfs: version 1.12 Backing Filesystem: overlayfs
The following error might occur for existing workspaces:
ERRO[0000] User-selected graph driver "overlay" overwritten by graph driver "vfs" from database - delete libpod local files ("/home/user/.local/share/containers/storage") to resolve. May prevent use of images created by other tools
In this case, delete the libpod local files shown in the error message.
3.4. Containers with kubedock Copy linkLink copied to clipboard!
Kubedock is a minimal container engine implementation that gives you a Podman-/docker-like experience inside an OpenShift Dev Spaces workspace.
Kubedock is especially useful when dealing with ad-hoc, ephemeral, and testing containers, such as in the use cases listed below:
- Executing application tests which rely on the Testcontainers framework.
- Using Quarkus Dev Services.
- Running a container stored in remote container registry, for local development purposes
The image you want to use with kubedock must be compliant with OpenShift Container Platform image creation guidelines. Otherwise, running the image with kubedock results in a failure even if the same image runs locally without issues.
3.4.1. Supported commands Copy linkLink copied to clipboard!
After enabling the kubedock environment variable, kubedock runs the following podman commands:
-
podman run -
podman ps -
podman exec -
podman cp -
podman logs -
podman inspect -
podman kill -
podman rm -
podman wait -
podman stop -
podman start
Other commands such as podman build are started by the local Podman.
Using podman commands with kubedock has the following limitations:
-
The
podman build -t <image> . && podman run <image>command fails. Usepodman build -t <image> . && podman push <image> && podman run <image>instead. -
The
podman generate kubecommand is not supported. -
--envoption causes thepodman runcommand to fail.
3.5. Enable kubedock in a workspace Copy linkLink copied to clipboard!
Enable kubedock in an OpenShift Dev Spaces workspace by adding environment variables to the devfile.
Prerequisites
- You have an image compliant with OpenShift Container Platform guidelines.
Procedure
-
Add
KUBEDOCK_ENABLED=trueenvironment variable to the devfile. Optional: Use the
KUBEDOCK_PARAMSvariable to specify additional kubedock parameters. The list of parameters is available in the kubedock server source. Alternatively, you can use the following command to view the available options:# kubedock server --helpConfigure the Podman or docker API to point to kubedock by setting
CONTAINER_HOST=tcp://127.0.0.1:2475orDOCKER_HOST=tcp://127.0.0.1:2475in the devfile.ImportantConfigure Podman to point to local Podman when building containers, and to kubedock when running containers.
The following example devfile enables kubedock with Testcontainers support:
schemaVersion: 2.2.0 metadata: name: kubedock-sample-devfile components: - name: tools container: image: quay.io/devfile/universal-developer-image:latest memoryLimit: 8Gi memoryRequest: 1Gi cpuLimit: "2" cpuRequest: 200m env: - name: KUBEDOCK_PARAMS value: "--reverse-proxy --kubeconfig /home/user/.kube/config --initimage quay.io/agiertli/kubedock:0.13.0" - name: USE_JAVA17 value: "true" - value: /home/jboss/.m2 name: MAVEN_CONFIG - value: -Xmx4G -Xss128M -XX:MetaspaceSize=1G -XX:MaxMetaspaceSize=2G name: MAVEN_OPTS - name: KUBEDOCK_ENABLED value: 'true' - name: DOCKER_HOST value: 'tcp://127.0.0.1:2475' - name: TESTCONTAINERS_RYUK_DISABLED value: 'true' - name: TESTCONTAINERS_CHECKS_DISABLE value: 'true' endpoints: - exposure: none name: kubedock protocol: tcp targetPort: 2475 - exposure: public name: http-booster protocol: http targetPort: 8080 attributes: discoverable: true urlRewriteSupported: true - exposure: internal name: debug protocol: http targetPort: 5005 volumeMounts: - name: m2 path: /home/user/.m2 - name: m2 volume: size: 10G
3.6. Use Kubedock in a workspace Copy linkLink copied to clipboard!
Use Kubedock to run containers in your workspace when Docker-in-Docker or privileged containers are not available.
Prerequisites
- You have a running OpenShift Dev Spaces workspace.
-
You have the Kubedock command-line interface (CLI) (
kubedock) available in your workspace. It is included in the Universal Developer Image (UDI).
Procedure
Set the
DOCKER_HOSTenvironment variable to point to the Kubedock server:export DOCKER_HOST=tcp://127.0.0.1:2475Start the Kubedock server in the background:
kubedock server --port-forward &Use standard Docker commands that Kubedock handles:
docker run --rm hello-world
Verification
- The container runs successfully and outputs its message.