Idmapped volumes in rootful container with userns=auto
by Andre Nathan
Hello
I'm trying to mount a volume in a rootful container using --userns=auto.
This is an NFS volme mounted in the host and then onto the container.
The container should have write access to the volume, so I'm using the
ID mapping options to map the volume owner's UID and GID to 0 in user
namespace and the mount parameters.
These are the NFS exports:
/srv 10.20.3.140/32(rw,fsid=0,no_subtree_check,sync)
/srv/home 10.20.3.140/32(rw,sync,nohide,no_subtree_check)
The NFS server's /home is bind-mounted on /srv/home. The client mounts
the NFS volume on its own /srv:
$ mount | grep nfs
10.20.3.137:/ on /srv type nfs4
(rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.20.3.140,local_lock=none,addr=10.20.3.137)
# ls -l /srv/home
total 4
drwxr-x--- 5 local local 4096 Sep 19 17:36 local
The "local" user's UID and GID (this user was created on both machines
[NFS server and client] with the same UID and GID):
# grep local /etc/passwd
local:x:1000:1000:local:/home/local:/bin/bash
The command I'm using to start the container:
# podman run -d --rm --name foo \
--userns=auto:uidmapping=1000:0:1,gidmapping=1000:0:1 \
-v '/srv/home/local:/home/local:idmap=uids=1000-0-1;gids=1000-0-1' \
quay.io/libpod/alpine_nginx
This results in the following error:
Error: OCI runtime error: crun: mount_setattr `/home/local`: Invalid
argument
Checking the man page for mount_setattr, there are a bunch of reasons
for it to fail with EINVAL, including "MOUNT_ATTR_IDMAP was specified in
attr_clr", so my question is, was this supposed to work?
If it wasn't, is there a way to achieve a writable NFS volume combined
with --userns=auto in a rootful container?
I'm using Ubuntu 22.04 with the Kubic packages:
# podman version
Client: Podman Engine
Version: 4.6.2
API Version: 4.6.2
Go Version: go1.18.1
Built: Thu Jan 1 00:00:00 1970
OS/Arch: linux/amd64
# podman info
host:
arch: amd64
buildahVersion: 1.31.2
cgroupControllers:
- cpuset
- cpu
- io
- memory
- hugetlb
- pids
- rdma
- misc
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon_100:2.1.2~0_amd64
path: /usr/libexec/podman/conmon
version: 'conmon version 2.1.2, commit: '
cpuUtilization:
idlePercent: 99.99
systemPercent: 0.01
userPercent: 0.01
cpus: 48
databaseBackend: boltdb
distribution:
codename: jammy
distribution: ubuntu
version: "22.04"
eventLogger: journald
freeLocks: 2048
hostname: local2
idMappings:
gidmap: null
uidmap: null
kernel: 5.15.0-52-generic
linkmode: dynamic
logDriver: journald
memFree: 263899713536
memTotal: 270361567232
networkBackend: netavark
networkBackendInfo:
backend: netavark
dns:
package: aardvark-dns_1.6.0-0ubuntu22.04+obs34.25_amd64
path: /usr/libexec/podman/aardvark-dns
version: aardvark-dns 1.6.0
package: netavark_1.3.0-0ubuntu22.04+obs22.9_amd64
path: /usr/libexec/podman/netavark
version: netavark 1.8.0-dev
ociRuntime:
name: crun
package: crun_101:1.9-0ubuntu22.04+obs59.11_amd64
path: /usr/bin/crun
version: |-
crun version 1.9
commit: a538ac4ea1ff319bcfe2bf81cb5c6f687e2dc9d3
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
os: linux
pasta:
executable: ""
package: ""
version: ""
remoteSocket:
path: /run/podman/podman.sock
security:
apparmorEnabled: false
capabilities:
CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: false
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 0
swapTotal: 0
uptime: 137h 41m 19.00s (Approximately 5.71 days)
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
- ipvlan
volume:
- local
registries:
search:
- docker.io
- quay.io
store:
configFile: /etc/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions:
overlay.mountopt: nodev,metacopy=on
graphRoot: /var/lib/containers/storage
graphRootAllocated: 8340897792
graphRootUsed: 6129999872
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "true"
imageCopyTmpDir: /var/tmp
imageStore:
number: 1
runRoot: /run/containers/storage
transientStore: false
volumePath: /var/lib/containers/storage/volumes
version:
APIVersion: 4.6.2
Built: 0
BuiltTime: Thu Jan 1 00:00:00 1970
GitCommit: ""
GoVersion: go1.18.1
Os: linux
OsArch: linux/amd64
Version: 4.6.2
Thanks in advance,
Andre
2 years, 1 month
Announcement: New Podman pre-release - v4.7.0-rc1
by Lokesh Mandvekar
Happy late Friday all,
Podman v4.7.0-rc1 is now available.
v4.7.0-rc1 brings two shiny new commands:
1. `podman farm` to "farm" out builds to machines running Podman for
different architectures.
2. `podman compose` as a thin wrapper around an external compose provider
such as docker-compose or podman-compose.
along with a host of enhancements to `podman kube`, support for more
Quadlet fields, and lots of bugfixes.
Check out the release page for a full list of features, changes and
bugfixes.
https://github.com/containers/podman/releases/tag/v4.7.0-rc1
Enjoy!
--
Lokesh
Libera, GitLab, GitHub, Fedora: lsm5
Matrix: @lsm5:lsm5.ems.host
GPG: 9E33DD8704CC03E2DEB84D9A1C1EDD7CC7C3A0DD
https://keybase.io/lsm5
2 years, 1 month
Rootless podman mount capabilities
by Lewis Gaul
Hi,
I'm trying to understand something about how capabilities in rootless
podman work.
How does rootless podman have the capability to set up container mounts
(such as cgroup mounts) given a privileged container itself doesn't? Does
podman deliberately drop caps, or somehow get elevated privileges to do
this?
This is the process tree podman sets up (where bash is the container
entrypoint here):
systemd(1)---conmon(1327421)---bash(1327432)
I'm assuming it's conmon that sets up the container's mounts (via runc in
this case), which is a process running as my user (rootless). How is it
that conmon has the capabilities required (SYS_ADMIN?) to create the
container's cgroup and sysfs mounts but within the container itself this is
not possible?
Thanks for any insight!
Lewis
2 years, 1 month
Why do use podman machine on Mac?
by Mehdi Haghgoo
The container experience with podman machine on Windows and mac is not optimal because the containers are slow. Mac is a Linux-based OS. So, why can't we create native containers on it as we do on Linux?
That applies to WSL. It's kind of Linux. Why cannot we create native Linux containers on it without resorting to Podman machine and podman clients?
2 years, 1 month
Re: Why do use podman machine on Mac?
by Jason Greene
That’s right Darren. The short answer is “containers are Linux”. The
longer answer is you need two key elements: a Kernel name-spacing facility,
and second a user-land ecosystem. On the former MacOS doesn’t have
something comparable. The closest is App Sandbox, but that is primarily a
syscall/ permission filtering mechanism (something also important for
containers, but only part of the problem). On the latter, this is what we
know of as containers, the user-land is the OS without the kernel, and as
such is very much tied to the kernel implementation. Darwin libraries and
apps use a different ABI and syscall interface than Linux, so basically
every container you use today would be incompatible even if there was
kernel support.
This is the situation with Windows containers (not to be confused with
using Podman on Windows or other container envs that run Linux containers).
Windows containers have kernel support and a win32 user-land, so to produce
them you have to port everything in the image to be native windows code.
They are useful but not anywhere near as predominant, since a big part of
the value is the portability a Linux container brings.
In contrast, WSL2 does run a Linux kernel in a shared VM, and internally it
is using Linux namespaces itself, with all distributions of a user sharing
the same Kernel. They ultimately abandoned the previous syscall emulation
approach since the NT kernel and Linux are just too different and the
behaviors don’t map cleanly. We leverage WSL2 for the Podman on Windows
implementation.
On Sep 7, 2023, at 11:51 AM, Darren Dupre <darren.dupre(a)gmail.com> wrote:
OS X has a Mach kernel and a BSD user land. Don’t think Apple is interested
in adding containers framework to that. I guess someone could write a
kernel extension but it’s a niche use case.
WSL 2 uses a real Linux kernel doesn’t it? But it’s still a VM inside of
Hyper V so I don’t think the performance is any different. WSL 1 is a posix
emulation with binary compatibility but no container support under the
hood.
Correct me if I'm wrong about that.
- Darren
On Thu, Sep 7, 2023 at 11:19 Mehdi Haghgoo via Podman <
podman(a)lists.podman.io> wrote:
> The container experience with podman machine on Windows and mac is not
> optimal because the containers are slow.
> Mac is a Linux-based OS. So, why can't we create native containers on it
> as we do on Linux?
>
> That applies to WSL. It's kind of Linux. Why cannot we create native Linux
> containers on it without resorting to Podman machine and podman clients?
>
>
> _______________________________________________
> Podman mailing list -- podman(a)lists.podman.io
> To unsubscribe send an email to podman-leave(a)lists.podman.io
>
_______________________________________________
Podman mailing list -- podman(a)lists.podman.io
To unsubscribe send an email to podman-leave(a)lists.podman.io
2 years, 1 month
Fwd: Re: What to use instead of RemapUsers/RemapUid/RemapGid in Quadlet now?
by jklaiho＠iki.fi
Hi,
(Apologies, my last e-mail I think I accidentally sent to Erik, not the list; missing bits quoted below)
OK, I did not realize that the Remap* commands used --userns behind the scenes, I thought they used --uidmap and --gidmap.
That brings me back to my original need, which I'll reiterate:
"When using a regular host user to run a rootless container, map the container user as the host user."
That's really basic, right? Surely works without PodmanArgs already.
My only addition to that is: "If the container user is root (or sudoed to root), allow it to drop privileges so it can install apt packages."
Still sounds really basic, and where I work, it's really common. If there's problems, we exec into a deployed container and install some troubleshooting tools with apt that the minimalistic image doesn't contain by default. Based on previous discussions, the apt privilege drop needs one extra subordinate UID/GID to be available.
While I'm currently using the Remap* options to achieve these, I'm not married to them – they're just the first thing that I got to work.
I've used PodmanArgs before, but in each case only as a stopgap measure while waiting for an option to appear in Quadlet.
So, hopefully my final question in this thread: is this "map user and allow dropping privileges" scenario doable with the Quadlet UserNS option, or do I need to open an issue?
- JK
> Begin forwarded message:
>
> From: Erik Sjölund <erik.sjolund(a)gmail.com (mailto:erik.sjolund@gmail.com)>
> Date: Tuesday, Sep 05, 2023 at 10:37 AM
> To: Me <jklaiho(a)iki.fi (mailto:jklaiho@iki.fi)>
> Subject: [Podman] Re: What to use instead of RemapUsers/RemapUid/RemapGid in Quadlet now?
>
> Hi,
>
> > Yeah, I'm fairly sure that would work, but am I to understand from your reply
> > that it is not possible to achieve with the UserNS quadlet option the things that the Remap* quadlet options did?
> Correct. It's not possible.
>
> If we just look at the use of --uidmap there is an enormous amount of
> possibilities of how to map those.
> Each container UID can be mapped to an arbitrary "intermediate UID"
> but the mappings must be unique. By that I mean that different
> container UIDs must not be mapped to the same intermediate UID. Also
> different intermediate UIDs must not be mapped to the same container
> UID. The number of possible ways of providing --uidmap corresponds to
> a factorial in mathematics.
>
> See how n! grows with n:
> https://en.wikipedia.org/wiki/Factorial
>
> The argument --userns=keep-id does not carry so much information. For instance
> --userns=keep-id:uid=$uid,gid=$gid
> can only be provided one time and has two numbers.
>
> Another option to consider is
> --userns=auto
> but that option is quite different from --uidmap.
> It provides different functionality.
> The situation is a bit similar to static IP addresses vs dynamic IP
> addresses in DHCP.
>
> I see there is some discussion in
> https://github.com/containers/podman/issues/17984
> and
> https://github.com/containers/podman/pull/17961
>
> Erik
>
>
>
>
> On Mon, Sep 4, 2023 at 9:46 PM jklaiho(a)iki.fi <jklaiho(a)iki.fi> wrote:
> >
> > Hi Erik,
> >
> > Yeah, I'm fairly sure that would work, but am I to understand from your reply that it is not possible to achieve with the UserNS quadlet option the things that the Remap* quadlet options did?
> >
> > If so, why on earth did the Remap* options go away if a) they're still fully supported 'podman run' command-line options, and b) they serve a legit use case that UserNS doesn't fill?
> >
> >
> > - JK
> >
> >
> >
> >
> > On Monday, Sep 04, 2023 at 10:36 PM, Erik Sjölund <erik.sjolund(a)gmail.com> wrote:
> > I think the directive "PodmanArgs" can be used to set podman arguments
> > that don't have any
> > container option counterpart.
> > See man page:
> > https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html
> >
> > Maybe something like this could work?
> > PodmanArgs=--uidmap 0:0:1 --uidmap 100:1:1 --gidmap 0:0:1 --gidmap 65534:1:1
> > (untested)
> >
> > Erik
2 years, 1 month
HELP! recover files from a deleted container
by Alvin Thompson
Help!
Is there any way to recover files from a deleted container? Long story short, I found the behavior of `podman network rm -f` unexpected, and it wound up deleting most of my containers. One in particular had a month of work in it (I was using it as a development environment), and it turns out only part of it was backed up. I’m desperate!
This is Podman for Windows, so most of the files on the “host” are in the WSL environment. I can get into that no problem with `wsl -d podman-machine-default`.
As an added wrinkle, my default connection was `podman-machine-default-root`, but I was was not running Podman rootful. I’m not sure this is particularly relevant.
grep-ing for strings which are unique to the development environment shows one hit in Windows, in %HOME%/.local/containers/podman/machine/wsl/wsldist/podman-machine-default/ext4.vhdx - which I assume is the file system for the WSL layer itself. I made a copy of it.
A grep within WSL itself doesn’t show so any hits, so it’s possible the files were deleted as far as WSL is concerned. I tried searching for an EXT4 undelete tool, but the only one I found (extundelete) is from 10+ years ago and doesn’t appear to work anymore.
I haven’t stopped WSL (I’m using /tmp as a staging area) or restarted the computer.
I’m at wit’s end. I really don’t know where to begin or look to recover these files, which I really, really need. Any recovery suggestions (no matter how tedious) would be welcome.
I know it’s too late to change now, but man, the behavior of `podman network remove` is unexpected.
Thanks,
Alvin
2 years, 1 month
What to use instead of RemapUsers/RemapUid/RemapGid in Quadlet now?
by jklaiho＠iki.fi
I'm running a bunch of rootless Podman containers. I noticed that the RemapUsers, RemapUid and RemapGid options (and possibly others that I haven't used) disappeared from the documentation of podman-systemd.unit in 4.5.0.
I barely and partially understood what the options did in the 4.4.0 days when we started using them, but got them working through trial and error.
Here's what we have across the board right now in our Quadlet generators. They still work in 4.5.0, but I'm assuming they'll go away eventually:
RemapUsers=manual
RemapUid=0:0:1
RemapUid=100:1:1
RemapGid=0:0:1
RemapGid=65534:1:1
With the 0:0:1 options, the root user/group inside the containers are mapped to the regular (non-root) host user/group. We need this, since the container bind mounts volumes from the host and must appear to the host as the regular user while doing so.
The 100:1:1 and 65534:1:1 options have to do with the special _apt user in Debian-based containers; apt drops privileges to that user in some circumstances. I couldn't tell you why remapping those are needed, but not having them caused problems when installing packages inside the containers.
What Quadlet options in Podman >=4.5.0 would be equivalent to the above legacy options?
2 years, 1 month
