Daniel Walsh <dwalsh@redhat.com> writes:

On 8/29/22 02:22, jklaiho@iki.fi wrote:

...

I've had quite a lot of success with running rootless Podman containers in a Ubuntu 22.04 Vagrant box. They're able to connect to services running on the host, and by using the --uidmap parameter, I've been able to make the container user write to bound volumes from the host with the privileges of the non-root host user that is running the service.

One last hurdle remains: I have a container running as a systemd user service as a non-root user, but internally the container runs as root. I'm using --uidmap 0:0:1 so that when the container's root user writes to bound host volumes, on the host they appear to have been created by the non-root service user.

What surprised me is that when this UID mapping is in place, the root user seems to lose root privileges inside the container. I was trying to install redis-tools to debug a Redis connection issue inside the running container, and ran 'apt update' as the container root user. This failed with errors:

E: setgroups 65534 failed - setgroups (22: Invalid argument) E: setegid 65534 failed - setegid (22: Invalid argument) E: seteuid 100 failed - seteuid (22: Invalid argument) rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied Reading package lists... Done W: chown to _apt:root of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (22: Invalid argument) W: chown to _apt:root of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (22: Invalid argument) E: setgroups 65534 failed - setgroups (22: Invalid argument) E: setegid 65534 failed - setegid (22: Invalid argument) E: seteuid 100 failed - seteuid (22: Invalid argument) E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (22: Invalid argument) E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (22: Invalid argument) E: Method http has died unexpectedly! E: Sub-process http returned an error code (112)

If I run the container without the --uidmap parameter, this command starts working again, but naturally I lose the user mapping I described above.

Honestly, I'm probably able to rebuild the image that the container uses in such a way that its application runs as a non-root user (and then I'll just use e.g. --uidmap 1000:0:1, which I've found to work elsewhere), but I'm clearly missing something about the UID mapping functionality with an in-container root user, because I don't understand what about it is causing these errors. Any ideas? _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io

You can not use a --uidmap 0:0:1 mapping in a rootless container, since  you don't have access to the real root user within your username.  Since the real root is not mapped to your default user namespace you see it as a mapping of 65534, as I understand what is going on.

I think the problem is that the container doesn't have enough IDs available for performing operations like seteuid/setegid/setgroups. For running this container there is need to allocate more IDs, you could just cherry-pick the IDs you need: podman run --uidmap 0:0:1 --uidmap 100:1:1 --uidmap 65534:2:1 ... You may need to add other IDs, or adapt the range size.

I introduced the terminology "intermediate UID" in the man page as I thought it would make it easier to explain how --uidmap works for rootless Podman. Here is how it works: A user normally has multiple UIDs: the UID of the regular user the lowest subordinate UID the second lowest subordinate UID ... the nth lowest subordinate UID You could order a set of UIDs in many ways but the order I listed the UIDs above is special. In that order the positional indexes can be used as the "from_uid" value for the "podman run" command-line option --uidmap=container_uid:from_uid:amount (for rootless Podman). Here I write the positional index to the left of each UID: 0 : the UID of the regular user 1 : the lowest subordinate UID 2 : the second lowest subordinate UID ... n - 1 : the nth lowest subordinate UID For example: --uidmap=0:1:1 means map from_uid=1 to container_uid=0 (with amount=1), in other words, map the lowest subordinate UID to the container UID 0. If you have set of UIDs in a specific order, you could describe that as a mapping between numbers (the UIDs and the positional indexes), hence the use of the phrase "first mapping step". Instead of the invented terminology "intermediate UID", you could also call it "positional index" or something like that. Currently the "podman run" man page has this text: ----------- rootful user: container_uid:host_uid:amount rootless user: container_uid:intermediate_uid:amount ---------- Maybe it could be changed to: ----------- rootful user: container_uid:host_uid:amount rootless user: container_uid:positional_index:amount ---------- But then the reader needs to understand the concept of a range of positional indexes. On the other hand, the terminology "intermediate UID" might mislead the reader in believing that an "intermediate UID" can be used in other ways on the computer. The terminology "intermediate UID" is just used in a few Podman man pages. I don't think the phrase exists elsewhere (at least not in this sense). On Wed, Aug 31, 2022 at 4:19 PM <jklaiho@iki.fi> wrote:

I got this to work. It's clear that I fundamentally don't understand UID mapping and user namespaces well enough, even after reading a bunch of man pages and articles about them. Still, after a lot of further experimentation, trial and error, I succeeded. I'll post my findings here in case someone else struggles with this (and, judging by Google results for the error messages in my original post, I'm not alone).

TL;DR here are the parameters I had to add:

--uidmap 0:0:1 --uidmap 100:1:1 --gidmap 0:0:1 --gidmap 65534:1:1

Inside the container, the corresponding /proc entries match:

root@c4a7043b2e10:/# cat /proc/self/uid_map 0 0 1 100 1 1 root@c4a7043b2e10:/# cat /proc/self/gid_map 0 0 1 65534 1 1

After this, 'apt update' ran fine. The mapping of the container's root to the host's non-root user when writing files to bind mounted volumes was also preserved.

More details—in particular about the "magic numbers" 100 and 65534—follow for anyone interested, or in need of applying this to their situation.

- - - - -

Here's what's on the host side in both /etc/subuid and /etc/subgid, encompassing all the five non-root users on the host (names obscured):

vagrant:100000:65536 user2:165536:65536 <-- this one is the container-running service user user3:231072:65536 user4:296608:65536 user5:362144:65536

(The corresponding host UIDs and GIDs for the listed users are 1000–1004, though that's not relevant for this case.)

Inside the container, the user is just root with UID/GID 0. There, both /etc/subuid and /etc/subgid are blank.

During research for writing this, I realized where the UID 100 and GID 65534 in the error message are coming from. Inside the container's /etc/passwd, there's this:

_apt:x:100:65534::/nonexistent:/usr/sbin/nologin

UIDs 100-999 are "dynamically allocated system users and groups" according to https://www.debian.org/doc/debian-policy/ch-opersys.html#uid-and-gid-classes. Fortunately, in this case, the _apt user always seems to be dynamically allocated the UID 100, so that I can just hardcode that into my --uidmap invocation. GID 65534 is "nogroup", the group equivalent of the "nobody" user with the corresponding UID.

_apt is an unprivileged user that owns some of the files/dirs that the 'apt' command uses when dealing with packages. When 'apt' runs as root, it drops privileges to become this user as a security measure; see https://askubuntu.com/a/810213/1265622

To summarize my goals:

1. When the container's root user writes into volumes bind mounted from the host, it should have the privileges of user2, who is running the container. When it creates new files into those volumes, they should appear on the host side with the same owner/group/mode as they would have if user2 had just written them to the bind mount source directly on the host.

2. Inside the container, the root user should have full root privileges to do what it wants – within the limits of what's even possible to do in rootless containers. (And, of course, limited to the privileges of user2 whenever touching something on the host.) In this case, it needs to be able to drop privileges and become the _apt user within the container.

The first goal was doable with just --uidmap 0:0:1. Interestingly, when I added --uidmap 100:1:1 and --gidmap 65534:1:1 for the second goal, the container refused to start, complaining that UID mapping is used but GID 0 is not mapped. This is why --gidmap 0:0:1 was added as the last piece of the puzzle. I have no idea why it wasn't initially required for the first goal.

- - - - -

After all this, I still do not at all understand what the second number ("intermediate UID/GID") in rootless mode for the --uidmap and --gidmap parameters even is. I've read and re-read https://docs.podman.io/en/latest/markdown/podman-run.1.html#uidmap-container... a dozen times to no avail, as well as 'man user_namespaces', and I almost feel like I know less than when I started. At this point I guess I'd need to understand something about the Linux kernel for any explanation to make sense.

Still, I hope this helps someone else! _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io