Chapter 22. Creating and restoring container checkpoints
Checkpoint/Restore In Userspace (CRIU) is a software that enables you to set a checkpoint on a running container or an individual application and store its state to disk. You can use data saved to restore the container after a reboot at the same point in time it was checkpointed.
The kernel does not support pre-copy checkpointing on AArch64.
22.1. Creating and restoring a container checkpoint locally
This example is based on a Python based web server which returns a single integer which is incremented after each request.
Prerequisites
-
The
container-tools
module is installed.
Procedure
Create a Python based server:
# cat counter.py #!/usr/bin/python3 import http.server counter = 0 class handler(http.server.BaseHTTPRequestHandler): def do_GET(s): global counter s.send_response(200) s.send_header('Content-type', 'text/html') s.end_headers() s.wfile.write(b'%d\n' % counter) counter += 1 server = http.server.HTTPServer(('', 8088), handler) server.serve_forever()
Create a container with the following definition:
# cat Containerfile FROM registry.access.redhat.com/ubi8/ubi COPY counter.py /home/counter.py RUN useradd -ms /bin/bash counter RUN yum -y install python3 && chmod 755 /home/counter.py USER counter ENTRYPOINT /home/counter.py
The container is based on the Universal Base Image (UBI 8) and uses a Python based server.
Build the container:
# podman build . --tag counter
Files
counter.py
andContainerfile
are the input for the container build process (podman build
). The built image is stored locally and tagged with the tagcounter
.Start the container as root:
# podman run --name criu-test --detach counter
To list all running containers, enter:
# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e4f82fd84d48 localhost/counter:latest 5 seconds ago Up 4 seconds ago criu-test
Display IP address of the container:
# podman inspect criu-test --format "{{.NetworkSettings.IPAddress}}" 10.88.0.247
Send requests to the container:
# curl 10.88.0.247:8088 0 # curl 10.88.0.247:8088 1
Create a checkpoint for the container:
# podman container checkpoint criu-test
- Reboot the system.
Restore the container:
# podman container restore --keep criu-test
Send requests to the container:
# curl 10.88.0.247:8080 2 # curl 10.88.0.247:8080 3 # curl 10.88.0.247:8080 4
The result now does not start at
0
again, but continues at the previous value.
This way you can easily save the complete container state through a reboot.
Additional resources
22.2. Reducing startup time using container restore
You can use container migration to reduce startup time of containers which require a certain time to initialize. Using a checkpoint, you can restore the container multiple times on the same host or on different hosts. This example is based on the container from the Creating and restoring a container checkpoint locally.
Prerequisites
-
The
container-tools
module is installed.
Procedure
Create a checkpoint of the container, and export the checkpoint image to a
tar.gz
file:# podman container checkpoint criu-test --export /tmp/chkpt.tar.gz
Restore the container from the
tar.gz
file:# podman container restore --import /tmp/chkpt.tar.gz --name counter1 # podman container restore --import /tmp/chkpt.tar.gz --name counter2 # podman container restore --import /tmp/chkpt.tar.gz --name counter3
The
--name
(-n
) option specifies a new name for containers restored from the exported checkpoint.Display ID and name of each container:
# podman ps -a --format "{{.ID}} {{.Names}}" a8b2e50d463c counter3 faabc5c27362 counter2 2ce648af11e5 counter1
Display IP address of each container:
#️ podman inspect counter1 --format "{{.NetworkSettings.IPAddress}}" 10.88.0.248 #️ podman inspect counter2 --format "{{.NetworkSettings.IPAddress}}" 10.88.0.249 #️ podman inspect counter3 --format "{{.NetworkSettings.IPAddress}}" 10.88.0.250
Send requests to each container:
#️ curl 10.88.0.248:8080 4 #️ curl 10.88.0.249:8080 4 #️ curl 10.88.0.250:8080 4
Note, that the result is
4
in all cases, because you are working with different containers restored from the same checkpoint.
Using this approach, you can quickly start up stateful replicas of the initially checkpointed container.
Additional resources
22.3. Migrating containers among systems
You can migrate the running containers from one system to another, without losing the state of the applications running in the container. This example is based on the container from the Creating and restoring a container checkpoint locally section tagged with counter
.
Migrating containers among systems with the podman container checkpoint
and podman container restore
commands is supported only when the configurations of the systems match completely, as shown below:
- Podman version
- OCI runtime (runc/crun)
- Network stack (CNI/Netavark)
- Cgroups version
- kernel version
- CPU features
You can migrate to a CPU with more features, but not to a CPU which does not have a certain feature that you are using. The low-level tool doing the checkpointing (CRIU) has the possibility to check for CPU feature compatibility: https://criu.org/Cpuinfo.
Prerequisites
-
The
container-tools
module is installed. The following steps are not necessary if the container is pushed to a registry as Podman will automatically download the container from a registry if it is not available locally. This example does not use a registry, you have to export previously built and tagged container (see Creating and restoring a container checkpoint locally).
Export previously built container:
# podman save --output counter.tar counter
Copy exported container image to the destination system (
other_host
):# scp counter.tar other_host:
Import exported container on the destination system:
# ssh other_host podman load --input counter.tar
Now the destination system of this container migration has the same container image stored in its local container storage.
Procedure
Start the container as root:
# podman run --name criu-test --detach counter
Display IP address of the container:
# podman inspect criu-test --format "{{.NetworkSettings.IPAddress}}" 10.88.0.247
Send requests to the container:
# curl 10.88.0.247:8080 0 # curl 10.88.0.247:8080 1
Create a checkpoint of the container, and export the checkpoint image to a
tar.gz
file:# podman container checkpoint criu-test --export /tmp/chkpt.tar.gz
Copy the checkpoint archive to the destination host:
# scp /tmp/chkpt.tar.gz other_host:/tmp/
Restore the checkpoint on the destination host (
other_host
):# podman container restore --import /tmp/chkpt.tar.gz
Send a request to the container on the destination host (
other_host
):# *curl 10.88.0.247:8080* 2
As a result, the stateful container has been migrated from one system to another without losing its state.
Additional resources