disk space below ~/.local/share/containers/storage/overlay
After stop of a container massive data remains below ~/.local/share/containers/storage/overlay Steps to reproduce: cups@srap57dxr1:~> podman image list REPOSITORY TAG IMAGE ID CREATED SIZE localhost/sles15-sp6-v73-sp1 latest caeabd52aa4a 46 hours ago 26 GB localhost/sles15-sp6-v73 latest 690f90eaed0f 46 hours ago 20.2 GB registry.suse.com/bci/bci-base 15.6 075e3ce8c342 11 days ago 125 MB cups@srap57dxr1:~> podman run --tz='Europe/Berlin' -d -p 2022:22 sles15-sp6-v73 cups@srap57dxr1:~> podman container list CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 43e7d4f12f27 localhost/sles15-sp6-v73:latest 10 minutes ago Up 10 minutes 0.0.0.0:2022->22/tcp gracious_sinoussi This start creates two new dirs below /home/cups/.local/share/containers/storage/overlay: srap57dxr1:/home/cups/.local/share/containers/storage/overlay # ls -ltr | tail -2 drwxr-xr-x 2 cups root 17408 Nov 13 11:14 l drwx------ 6 cups root 1024 Nov 13 11:14 fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba srap57dxr1:/home/cups/.local/share/containers/storage/overlay # du -sh fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba 5.4G fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba Now I kill the container with: cups@srap57dxr1:~> podman kill 43e7d4f12f27 43e7d4f12f27 The dirs remain: srap57dxr1:/home/cups/.local/share/containers/storage/overlay # ls -ltr | tail -2 drwxr-xr-x 2 cups root 17408 Nov 13 11:14 l drwx------ 5 cups root 1024 Nov 13 11:27 fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba and occupy ~5.4G space: srap57dxr1:/home/cups/.local/share/containers/storage/overlay # du -sh fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba 5.4G fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba and this on any start/stop of the container. I did a closer look what is there below such a directory. We use such containers to test the update of our server software to the next release. Such update brings ~1 GByte gzip'ed packages and files and below such a directory is exactly what was updated, binary identical with what was stored as new versions of the files, for example a new delivered file '/opt/lib/sisis/etc/pos.rc' is there as fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba/diff/opt/lib/sisis/etc/pos.rc like many other files: # find fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba | grep /opt/lib/sisis | wc -l 2494 # find fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba | grep /usr/local/sisis-pap | wc -l 27879 Questions: 1) Why this data remains after stop of container? 2) Can I just remove this directory with 'rm -r ....'? 3) I did a test and renamed such directory to xxxx.away fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba.away No error message showed up on next start. Any hits on this? Thanks -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub Annalena Baerbock: "We are fighting a war against Russia ..." (25.1.2023) I, Matthias, I am not at war with Russia. Я не воюю с Россией. Ich bin nicht im Krieg mit Russland.
I'm not a professional, just a user, but maybe I can help. Am 2024-11-14 08:02, schrieb Matthias Apitz:
1) Why this data remains after stop of container?
Well, you killed the container and didn't kill it gracefully. Podman doesn't have a central daemon, so who's going to clean it up? In the same way, firewall and network rules that you haven't mentioned will remain - no one will clean them up either.
2) Can I just remove this directory with 'rm -r ....'?
Normally yes, but why don't you use `podman container cleanup`?
3) I did a test and renamed such directory to xxxx.away
fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba.away No error message showed up on next start. Any hits on this?
I believe this is assigned dynamically and controlled via BoltDB or SQLite. Otherwise start podman with `--log-level debug` HTH Tobias
El día jueves, noviembre 14, 2024 a las 11:39:24 +0100, Tobias Wendorff escribió:
I'm not a professional, just a user, but maybe I can help.
Thanks in any case for reading and your hints.
Am 2024-11-14 08:02, schrieb Matthias Apitz:
1) Why this data remains after stop of container?
Well, you killed the container and didn't kill it gracefully. Podman doesn't have a central daemon, so who's going to clean it up? In the same way, firewall and network rules that you haven't mentioned will remain - no one will clean them up either.
I kill the container because 'stop' does not work either: cups@srap57dxr1:~> podman stop zealous_blackburn WARN[0010] StopSignal SIGQUIT failed to stop container zealous_blackburn in 10 seconds, resorting to SIGKILL zealous_blackburn
2) Can I just remove this directory with 'rm -r ....'?
Normally yes, but why don't you use `podman container cleanup`?
I used this like: cups@srap57dxr1:~> podman container cleanup zealous_blackburn zealous_blackburn but the data below .local/share/containers/storage/overlay remain.
3) I did a test and renamed such directory to xxxx.away fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba.away No error message showed up on next start. Any hits on this?
I believe this is assigned dynamically and controlled via BoltDB or SQLite. Otherwise start podman with `--log-level debug`
Will try this. Thanks matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub Annalena Baerbock: "We are fighting a war against Russia ..." (25.1.2023) I, Matthias, I am not at war with Russia. Я не воюю с Россией. Ich bin nicht im Krieg mit Russland.
If you add `--rm` to the run command does the behavior change ? On Thu, Nov 14, 2024 at 1:03 AM Matthias Apitz <guru@unixarea.de> wrote:
After stop of a container massive data remains below ~/.local/share/containers/storage/overlay
Steps to reproduce:
cups@srap57dxr1:~> podman image list REPOSITORY TAG IMAGE ID CREATED SIZE localhost/sles15-sp6-v73-sp1 latest caeabd52aa4a 46 hours ago 26 GB localhost/sles15-sp6-v73 latest 690f90eaed0f 46 hours ago 20.2 GB registry.suse.com/bci/bci-base 15.6 075e3ce8c342 11 days ago 125 MB
cups@srap57dxr1:~> podman run --tz='Europe/Berlin' -d -p 2022:22 sles15-sp6-v73
cups@srap57dxr1:~> podman container list
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 43e7d4f12f27 localhost/sles15-sp6-v73:latest 10 minutes ago Up 10 minutes 0.0.0.0:2022->22/tcp gracious_sinoussi
This start creates two new dirs below /home/cups/.local/share/containers/storage/overlay:
srap57dxr1:/home/cups/.local/share/containers/storage/overlay # ls -ltr | tail -2 drwxr-xr-x 2 cups root 17408 Nov 13 11:14 l drwx------ 6 cups root 1024 Nov 13 11:14 fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba
srap57dxr1:/home/cups/.local/share/containers/storage/overlay # du -sh fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba 5.4G fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba
Now I kill the container with:
cups@srap57dxr1:~> podman kill 43e7d4f12f27 43e7d4f12f27
The dirs remain:
srap57dxr1:/home/cups/.local/share/containers/storage/overlay # ls -ltr | tail -2 drwxr-xr-x 2 cups root 17408 Nov 13 11:14 l drwx------ 5 cups root 1024 Nov 13 11:27 fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba
and occupy ~5.4G space:
srap57dxr1:/home/cups/.local/share/containers/storage/overlay # du -sh fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba 5.4G fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba
and this on any start/stop of the container.
I did a closer look what is there below such a directory. We use such containers to test the update of our server software to the next release. Such update brings ~1 GByte gzip'ed packages and files and below such a directory is exactly what was updated, binary identical with what was stored as new versions of the files, for example a new delivered file '/opt/lib/sisis/etc/pos.rc' is there as
fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba/diff/opt/lib/sisis/etc/pos.rc like many other files:
# find fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba | grep /opt/lib/sisis | wc -l 2494
# find fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba | grep /usr/local/sisis-pap | wc -l 27879
Questions:
1) Why this data remains after stop of container?
2) Can I just remove this directory with 'rm -r ....'?
3) I did a test and renamed such directory to xxxx.away fb646bf8e1eb6f157c8b2430897df11da78a9e7c0e38858959066d5c40a2adba.away No error message showed up on next start. Any hits on this?
Thanks
-- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub
Annalena Baerbock: "We are fighting a war against Russia ..." (25.1.2023)
I, Matthias, I am not at war with Russia. Я не воюю с Россией. Ich bin nicht im Krieg mit Russland. _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
-- Brent Baude Container Runtimes, Podman Architect https://github.com/containers/podman
El día jueves, noviembre 14, 2024 a las 07:47:47 -0600, Brent Baude escribió:
If you add `--rm` to the run command does the behavior change ?
Brent Baude Container Runtimes, Podman Architect https://github.com/containers/podman
This really did it. While the container was running: # ls -ltr | tail -3 drwx------ 6 cups root 1024 Nov 14 17:24 7829f67c27d816793cd0159afd140224afeb5fbb3082599144365e4820d75087 drwxr-xr-x 2 cups root 17408 Nov 15 07:28 l drwx------ 6 cups root 1024 Nov 15 07:28 b9973358275422ff0afa9c0dc1d853c91daaeddf1a73773be3e3c1f5030f0052 cups@srap57dxr1:~> podman stop c069d9415ce9 WARN[0010] StopSignal SIGQUIT failed to stop container gifted_murdock in 10 seconds, resorting to SIGKILL c069d9415ce9 # ls -ltr | tail -3 drwx------ 5 cups root 1024 Nov 14 17:24 1b753bb8fe11b0c8aa4526ecff2ffcc8d9e701de2c5bf2e7711bdbf87663c9a7 drwx------ 6 cups root 1024 Nov 14 17:24 7829f67c27d816793cd0159afd140224afeb5fbb3082599144365e4820d75087 drwxr-xr-x 2 cups root 17408 Nov 15 07:50 l i.e. the dir below b9973358275422ff0afa9c0dc1d853c91daaeddf1a73773be3e3c1f5030f0052 was removed. The man page for podman-run explains: ... --rm Automatically remove the container and any anonymous unnamed volume associated with the container when it exits. The default is false. Lesion learned: One must study the complete man page. Thanks! matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub Annalena Baerbock: "We are fighting a war against Russia ..." (25.1.2023) I, Matthias, I am not at war with Russia. Я не воюю с Россией. Ich bin nicht im Krieg mit Russland.
participants (3)
-
Brent Baude -
Matthias Apitz -
Tobias Wendorff