HELP! recover files from a deleted container
Help! Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate! This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`. As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant. grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it. A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore. I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer. I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome. I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected. Thanks, Alvin
Hi Alvin, I am really sorry about the data loss. Unfortunately, there is no magic `podman recover` feature that would bring the data back. The behavior of `network rm` is documented but I sympathize that it's not helpful in your situation. I feel bad that I cannot help you much. Good luck, Valentin On Mon, Sep 4, 2023 at 12:28 PM Alvin Thompson <alvin@thompsonlogic.com> wrote:
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
I was afraid of that. Perhaps some level of protection from accidental data loss while in interactive mode (maybe just unlinking files and later deleting them when space is needed) is a decent feature suggestion. Thanks, Alvin
On Sep 4, 2023, at 7:40 AM, Valentin Rothberg <vrothberg@redhat.com> wrote:
Hi Alvin,
I am really sorry about the data loss.
Unfortunately, there is no magic `podman recover` feature that would bring the data back. The behavior of `network rm` is documented but I sympathize that it's not helpful in your situation.
I feel bad that I cannot help you much.
Good luck, Valentin
On Mon, Sep 4, 2023 at 12:28 PM Alvin Thompson <alvin@thompsonlogic.com <mailto:alvin@thompsonlogic.com>> wrote:
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io <mailto:podman@lists.podman.io> To unsubscribe send an email to podman-leave@lists.podman.io <mailto:podman-leave@lists.podman.io>
1. Immediately stop using the system: Cease all activities and avoid any further operations on the affected system. This minimizes the risk of overwriting the data you want to recover. 2. Turn it off as soon as possible. Maybe unplug the power supply to turn it off immeditely. 3. Don't boot from the disk again. Remove it if necessaray. 4. Boot into a data-recovery DVD or put it on another system and mount it read-only. The more you do on the hard drive, the more likely it is that the data will be overwritten. The data is then virtually unrecoverable. Normally, however, you can recover deleted data. They were not intentionally overwritten (shredded). Am 04.09.2023 um 12:26 schrieb Alvin Thompson:
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Hi and thanks for the suggestions, Since this is Podman for Windows which uses a WSL instance, I’m hopeful that not starting Podman or messing within the WSL instance will preserve the data if necessary. WSL stores the EXT4 filesystem in a vhdx image which hopefully is isolated from Windows enough. If I’m wrong about this please let me know. This is a work computer with rather strict controls so what I can do with it is limited. I did make a copy of the WSL disk image so that’s something. Unfortunately, I may have already overwritten the data because in a panic the first thing I did was try to copy any folder I could find with the name “container”. I was hoping the files would be unlinked and cleaned up later if space were needed. Perhaps that’s a feature suggestion. I’ll see if I can grab another Intel computer, install VirtualBox on it, attach a copy of the image, and boot a recovery DVD with that. Thanks, Alvin
On Sep 4, 2023, at 8:15 AM, Tobias Wendorff <tobias.wendorff@tu-dortmund.de> wrote:
1. Immediately stop using the system: Cease all activities and avoid any further operations on the affected system. This minimizes the risk of overwriting the data you want to recover.
2. Turn it off as soon as possible. Maybe unplug the power supply to turn it off immeditely.
3. Don't boot from the disk again. Remove it if necessaray.
4. Boot into a data-recovery DVD or put it on another system and mount it read-only.
The more you do on the hard drive, the more likely it is that the data will be overwritten. The data is then virtually unrecoverable. Normally, however, you can recover deleted data. They were not intentionally overwritten (shredded).
Am 04.09.2023 um 12:26 schrieb Alvin Thompson:
Help! Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate! This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`. As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant. grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it. A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore. I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer. I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome. I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected. Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
If you have the image, and you know what the data you're looking for looks like (i.e. it's text you can search for), try just loading the image in a hex editor and searching for it. On Mon, Sep 04, 2023 at 09:05:37AM -0400, Alvin Thompson wrote:
Hi and thanks for the suggestions,
Since this is Podman for Windows which uses a WSL instance, I’m hopeful that not starting Podman or messing within the WSL instance will preserve the data if necessary. WSL stores the EXT4 filesystem in a vhdx image which hopefully is isolated from Windows enough. If I’m wrong about this please let me know.
This is a work computer with rather strict controls so what I can do with it is limited. I did make a copy of the WSL disk image so that’s something. Unfortunately, I may have already overwritten the data because in a panic the first thing I did was try to copy any folder I could find with the name “container”. I was hoping the files would be unlinked and cleaned up later if space were needed. Perhaps that’s a feature suggestion.
I’ll see if I can grab another Intel computer, install VirtualBox on it, attach a copy of the image, and boot a recovery DVD with that.
Thanks, Alvin
On Sep 4, 2023, at 8:15 AM, Tobias Wendorff <tobias.wendorff@tu-dortmund.de> wrote:
1. Immediately stop using the system: Cease all activities and avoid any further operations on the affected system. This minimizes the risk of overwriting the data you want to recover.
2. Turn it off as soon as possible. Maybe unplug the power supply to turn it off immeditely.
3. Don't boot from the disk again. Remove it if necessaray.
4. Boot into a data-recovery DVD or put it on another system and mount it read-only.
The more you do on the hard drive, the more likely it is that the data will be overwritten. The data is then virtually unrecoverable. Normally, however, you can recover deleted data. They were not intentionally overwritten (shredded).
Am 04.09.2023 um 12:26 schrieb Alvin Thompson:
Help! Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate! This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`. As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant. grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it. A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore. I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer. I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome. I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected. Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Thanks. I have a call to my company’s IT department hoping they made a recent backup (not likely), but if that doesn’t work I’ll try that. Thanks, Alvin
On Sep 4, 2023, at 2:32 PM, Robin Lee Powell <rlpowell@digitalkingdom.org> wrote:
If you have the image, and you know what the data you're looking for looks like (i.e. it's text you can search for), try just loading the image in a hex editor and searching for it.
On Mon, Sep 04, 2023 at 09:05:37AM -0400, Alvin Thompson wrote:
Hi and thanks for the suggestions,
Since this is Podman for Windows which uses a WSL instance, I’m hopeful that not starting Podman or messing within the WSL instance will preserve the data if necessary. WSL stores the EXT4 filesystem in a vhdx image which hopefully is isolated from Windows enough. If I’m wrong about this please let me know.
This is a work computer with rather strict controls so what I can do with it is limited. I did make a copy of the WSL disk image so that’s something. Unfortunately, I may have already overwritten the data because in a panic the first thing I did was try to copy any folder I could find with the name “container”. I was hoping the files would be unlinked and cleaned up later if space were needed. Perhaps that’s a feature suggestion.
I’ll see if I can grab another Intel computer, install VirtualBox on it, attach a copy of the image, and boot a recovery DVD with that.
Thanks, Alvin
On Sep 4, 2023, at 8:15 AM, Tobias Wendorff <tobias.wendorff@tu-dortmund.de> wrote:
1. Immediately stop using the system: Cease all activities and avoid any further operations on the affected system. This minimizes the risk of overwriting the data you want to recover.
2. Turn it off as soon as possible. Maybe unplug the power supply to turn it off immeditely.
3. Don't boot from the disk again. Remove it if necessaray.
4. Boot into a data-recovery DVD or put it on another system and mount it read-only.
The more you do on the hard drive, the more likely it is that the data will be overwritten. The data is then virtually unrecoverable. Normally, however, you can recover deleted data. They were not intentionally overwritten (shredded).
Am 04.09.2023 um 12:26 schrieb Alvin Thompson:
Help! Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate! This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`. As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant. grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it. A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore. I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer. I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome. I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected. Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Hey, Alvin. Sorry to read about that. But anyways I'm glad that you've lost only your personal work, not a production one. *> One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!* It was painful but good experience to remember that the main idea of containers consists in stateless approach. In general, there is no guarantees that your container and its data will be available for a long time, but there is a guarantee that it can be quickly started up again, scaled to a number of instances. So, if you work under container environment, you have to store your data outside, whether it's a database, or a file storage. С уважением, А. И. пн, 4 сент. 2023 г., 13:27 Alvin Thompson <alvin@thompsonlogic.com>:
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Yep, when I’m working on my Mac I do that. Working from Windows is a little more problematic since it doesn’t have unix-like file permissions. I’m not sure if Podman works around that somehow—does it? I guess I could have tried to map the volume to somewhere in the WSL install (if possible). Live and learn. -Alvin
On Sep 4, 2023, at 12:05 PM, Александр Илюшкин <ailjushkin@gmail.com> wrote:
Hey, Alvin.
Sorry to read about that. But anyways I'm glad that you've lost only your personal work, not a production one.
One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
It was painful but good experience to remember that the main idea of containers consists in stateless approach. In general, there is no guarantees that your container and its data will be available for a long time, but there is a guarantee that it can be quickly started up again, scaled to a number of instances.
So, if you work under container environment, you have to store your data outside, whether it's a database, or a file storage.
С уважением, А. И.
пн, 4 сент. 2023 г., 13:27 Alvin Thompson <alvin@thompsonlogic.com <mailto:alvin@thompsonlogic.com>>:
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks, Alvin _______________________________________________ Podman mailing list -- podman@lists.podman.io <mailto:podman@lists.podman.io> To unsubscribe send an email to podman-leave@lists.podman.io <mailto:podman-leave@lists.podman.io>
participants (5)
-
Alvin Thompson -
Robin Lee Powell -
Tobias Wendorff -
Valentin Rothberg -
Александр Илюшкин