Chapter 21. Monitoring containers
Use Podman commands to manage a Podman environment. With that, you can determine the health of the container, by displaying system and pod information, and monitoring Podman events.
21.1. Using a health check on a container Copy linkLink copied to clipboard!
You can use the health check to determine the health or readiness of the process running inside the container.
If the health check succeeds, the container is marked as "healthy"; otherwise, it is "unhealthy". You can compare a health check with running the podman exec command and examining the exit code. The zero exit value means that the container is "healthy".
Health checks can be set when building an image using the HEALTHCHECK instruction in the Containerfile or when creating the container on the command line. You can display the health-check status of a container using the podman inspect or podman ps commands.
A health check consists of six basic components:
- Command
- Retries
- Interval
- Start-period
- Timeout
- Container recovery
The description of health check components follows:
- Command (
--health-cmdoption) - Podman executes the command inside the target container and waits for the exit code.
The other five components are related to the scheduling of the health check and they are optional.
- Retries (
--health-retriesoption) - Defines the number of consecutive failed health checks that need to occur before the container is marked as "unhealthy". A successful health check resets the retry counter.
- Interval (
--health-intervaloption) - Describes the time between running the health check command. Note that small intervals cause your system to spend a lot of time running health checks. The large intervals cause struggles with catching time outs.
- Start-period (
--health-start-periodoption) - Describes the time between when the container starts and when you want to ignore health check failures.
- Timeout (
--health-timeoutoption) - Describes the period of time the health check must complete before being considered unsuccessful.
The values of the Retries, Interval, and Start-period components are time durations, for example “30s” or “1h15m”. Valid time units are "ns," "us," or "µs", "ms," "s," "m," and "h".
- Container recovery (
--health-on-failureoption) Determines which actions to perform when the status of a container is unhealthy. When the application fails, Podman restarts it automatically to provide robustness. The
--health-on-failureoption supports four actions:-
none: Take no action, this is the default action. -
kill: Kill the container. -
restart: Restart the container. stop: Stop the container.NoteThe
--health-on-failureoption is available in Podman version 4.2 and later.
-
Do not combine the restart action with the --restart option. When running inside of a systemd unit, consider using the kill or stop action instead, to make use of systemd restart policy.
Health checks run inside the container. Health checks only make sense if you know what the health state of the service is and can differentiate between a successful and unsuccessful health check.
21.2. Performing a health check using the command line Copy linkLink copied to clipboard!
You can set a health check when creating the container on the command line.
Prerequisites
-
The
container-toolsmodule is installed.
Procedure
Define a health check:
podman run -dt --name=hc-container -p 8080:8080 --health-cmd='curl http://localhost:8080 || exit 1' --health-interval=0 registry.access.redhat.com/ubi8/httpd-24
$ podman run -dt --name=hc-container -p 8080:8080 --health-cmd='curl http://localhost:8080 || exit 1' --health-interval=0 registry.access.redhat.com/ubi8/httpd-24Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
The
--health-cmdoption sets a health check command for the container. -
The
--health-interval=0option with 0 value indicates that you want to run the health check manually.
-
The
Check the health status of the
hc-containercontainer:Using the
podman inspectcommand:podman inspect --format='{{json .State.Health.Status}}' hc-container$ podman inspect --format='{{json .State.Health.Status}}' hc-container healthyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Using the
podman pscommand:podman ps
$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a680c6919fe localhost/hc-container:latest /usr/bin/run-http... 2 minutes ago Up 2 minutes (healthy) hc-containerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Using the
podman healthcheck runcommand:podman healthcheck run hc-container
$ podman healthcheck run hc-container healthyCopy to Clipboard Copied! Toggle word wrap Toggle overflow
21.3. Performing a health check using a Containerfile Copy linkLink copied to clipboard!
You can set a health check by using the HEALTHCHECK instruction in the Containerfile.
Prerequisites
-
The
container-toolsmodule is installed.
Procedure
Create a
Containerfile:cat Containerfile FROM registry.access.redhat.com/ubi8/httpd-24 EXPOSE 8080 HEALTHCHECK CMD curl http://localhost:8080 || exit 1
$ cat Containerfile FROM registry.access.redhat.com/ubi8/httpd-24 EXPOSE 8080 HEALTHCHECK CMD curl http://localhost:8080 || exit 1Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe
HEALTHCHECKinstruction is supported only for thedockerimage format. For theociimage format, the instruction is ignored.Build the container and add an image name:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the container:
podman run -dt --name=hc-container localhost/hc-container
$ podman run -dt --name=hc-container localhost/hc-containerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the health status of the
hc-containercontainer:Using the
podman inspectcommand:podman inspect --format='{{json .State.Health.Status}}' hc-container$ podman inspect --format='{{json .State.Health.Status}}' hc-container healthyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Using the
podman pscommand:podman ps
$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a680c6919fe localhost/hc-container:latest /usr/bin/run-http... 2 minutes ago Up 2 minutes (healthy) hc-containerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Using the
podman healthcheck runcommand:podman healthcheck run hc-container
$ podman healthcheck run hc-container healthyCopy to Clipboard Copied! Toggle word wrap Toggle overflow
21.4. Displaying Podman system information Copy linkLink copied to clipboard!
The podman system command enables you to manage the Podman systems by displaying system information.
Prerequisites
-
The
container-toolsmodule is installed.
Procedure
Display Podman system information:
To show Podman disk usage, enter:
podman system df
$ podman system df TYPE TOTAL ACTIVE SIZE RECLAIMABLE Images 3 2 1.085GB 233.4MB (0%) Containers 2 0 28.17kB 28.17kB (100%) Local Volumes 3 0 0B 0B (0%)Copy to Clipboard Copied! Toggle word wrap Toggle overflow To show detailed information about space usage, enter:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To display information about the host, current storage stats, and build of Podman, enter:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To remove all unused containers, images and volume data, enter:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
The
podman system prunecommand removes all unused containers (both dangling and unreferenced), pods and optionally, volumes from local storage. -
Use the
--alloption to delete all unused images. Unused images are dangling images and any image that does not have any containers based on it. -
Use the
--volumeoption to prune volumes. By default, volumes are not removed to prevent important data from being deleted if there is currently no container using the volume.
-
The
21.5. Podman event types Copy linkLink copied to clipboard!
You can monitor events that occur in Podman. Several event types exist and each event type reports different statuses.
The container event type reports the following statuses:
- attach
- checkpoint
- cleanup
- commit
- create
- exec
- export
- import
- init
- kill
- mount
- pause
- prune
- remove
- restart
- restore
- start
- stop
- sync
- unmount
- unpause
The pod event type reports the following statuses:
- create
- kill
- pause
- remove
- start
- stop
- unpause
The image event type reports the following statuses:
- prune
- push
- pull
- save
- remove
- tag
- untag
The system type reports the following statuses:
- refresh
- renumber
The volume type reports the following statuses:
- create
- prune
- remove
21.6. Monitoring Podman events Copy linkLink copied to clipboard!
You can monitor and print events that occur in Podman using the podman events command. Each event will include a timestamp, a type, a status, name, if applicable, and image, if applicable.
Prerequisites
-
The
container-toolsmodule is installed.
Procedure
Run the
myubicontainer:podman run -q --rm --name=myubi registry.access.redhat.com/ubi8/ubi:latest
$ podman run -q --rm --name=myubi registry.access.redhat.com/ubi8/ubi:latestCopy to Clipboard Copied! Toggle word wrap Toggle overflow Display the Podman events:
To display all Podman events, enter:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
--stream=falseoption ensures that thepodman eventscommand exits when reading the last known event.You can see several events that happened when you enter the
podman runcommand:-
container createwhen creating a new container. -
image pullwhen pulling an image if the container image is not present in the local storage. -
container initwhen initializing the container in the runtime and setting a network. -
container startwhen starting the container. -
container attachwhen attaching to the terminal of a container. That is because the container runs in the foreground. -
container diedis emitted when the container exits. -
container removebecause the--rmflag was used to remove the container after it exits.
-
You can also use the
journalctlcommand to display Podman events:journalctl --user -r SYSLOG_IDENTIFIER=podman
$ journalctl --user -r SYSLOG_IDENTIFIER=podman Mar 08 14:27:20 fedora podman[129324]: 2023-03-08 14:27:20.913786892 +0100 CET m=+0.066920979 container remove ... Mar 08 14:27:20 fedora podman[129289]: 2023-03-08 14:27:20.696167362 +0100 CET m=+0.079089208 container create d4748226a2bcd271b1bc4b9f88b54e8271c13ffea9b30529968291c62d72f>Copy to Clipboard Copied! Toggle word wrap Toggle overflow To show only Podman create events, enter:
podman events --filter event=create
$ podman events --filter event=create 2023-03-08 14:27:20.696167362 +0100 CET container create d4748226a2bcd271b1bc4b9f88b54e8271c13ffea9b30529968291c62d72fe09 (image=registry.access.redhat.com/ubi8/ubi:latest, name=myubi,...)Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can also use the
journalctlcommand to display Podman create events:journalctl --user -r PODMAN_EVENT=create
$ journalctl --user -r PODMAN_EVENT=create Mar 08 14:27:20 fedora podman[129289]: 2023-03-08 14:27:20.696167362 +0100 CET m=+0.079089208 container create d4748226a2bcd271b1bc4b9f88b54e8271c13ffea9b30529968291c62d72f>Copy to Clipboard Copied! Toggle word wrap Toggle overflow
21.7. Using Podman events for auditing Copy linkLink copied to clipboard!
Previously, the events had to be connected to an event to interpret them correctly. For example, the container-create event had to be linked with an image-pull event to know which image had been used. The container-create event also did not include all data, for example, the security settings, volumes, mounts, and so on.
Beginning with Podman v4.4, you can gather all relevant information about a container directly from a single event and journald entry. The data is in JSON format, the same as from the podman container inspect command and includes all configuration and security settings of a container. You can configure Podman to attach the container-inspect data for auditing purposes.
Prerequisites
-
The
container-toolsmodule is installed.
Procedure
Modify the
~/.config/containers/containers.conffile and add theevents_container_create_inspect_data=trueoption to the[engine]section:cat ~/.config/containers/containers.conf [engine] events_container_create_inspect_data=true
$ cat ~/.config/containers/containers.conf [engine] events_container_create_inspect_data=trueCopy to Clipboard Copied! Toggle word wrap Toggle overflow For the system-wide configuration, modify the
/etc/containers/containers.confor/usr/share/container/containers.conffile.Create the container:
podman create registry.access.redhat.com/ubi8/ubi:latest
$ podman create registry.access.redhat.com/ubi8/ubi:latest 19524fe3c145df32d4f0c9af83e7964e4fb79fc4c397c514192d9d7620a36cd3Copy to Clipboard Copied! Toggle word wrap Toggle overflow Display the Podman events:
Using the
podman eventscommand:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
The
--format "{{.ContainerInspectData}}"option displays the inspect data. -
The
jq ".Config.CreateCommand"transforms the JSON data into a more readable format and displays the parameters for thepodman createcommand.
-
The
Using the
journalctlcommand:Copy to Clipboard Copied! Toggle word wrap Toggle overflow The output data for the
podman eventsandjournalctlcommands are the same.