On Fri, Jan 28, 2022 at 11:30 AM Nalin Dahyabhai <nalin(a)redhat.com> wrote:
On Thu, Jan 27, 2022 at 12:17:52PM -0500, Peter Portante wrote:
> We are struggling to understand why we can run rootless containers on
RHEL
> 8.5.
>
> Why can't I do the following (as described at [1]) as a non-root user:
>
> [pportant@intlab-006 ~]$ podman run --rm --name=myubi -it
>
registry.access.redhat.com/ubi8/ubi /bin/bash
> [pportant@intlab-006 ~]$ echo $?
> 0
>
> Shouldn't that start an interactive shell in the container?
>
> When I run as root I see:
>
> [root@intlab-006 ~]# podman run --rm --name=myubi -it
>
registry.access.redhat.com/ubi8/ubi /bin/bash
> [root@intlab-006 ~]# echo $?
> 127
>
> While on another RHEL 8.5 host it works just fine:
>
> [pportant@intlabproxy-002 ~]$ podman run --rm --name=myubi -it
>
registry.access.redhat.com/ubi8/ubi /bin/bash
> [root@a9ef24a2578b /]#
>
> Any help would be appreciated.
What differences do you see between the two situations when you pass a
--log-level=info, or --log-level=debug, to podman?
*[root@intlab-006 ~]# podman --log-level=info run --rm --name=myubi -it
registry.access.redhat.com/ubi8/ubi
<
http://registry.access.redhat.com/ubi8/ubi> /bin/bash*INFO[0000] podman
filtering at log level info
INFO[0000] Not using native diff for overlay, this may cause degraded
performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR
enabled
INFO[0000] Found CNI network podman (type=bridge) at
/etc/cni/net.d/87-podman-bridge.conflist
INFO[0000] Setting parallel job count to 97
INFO[0000] Got pod network &{Name:myubi Namespace:myubi
ID:c198a57f8fb8eebb2c8f391341fbb8bf0c02b84be2ee5b8b648e675adf07fb72
NetNS:/run/netns/cni-e0647a42-5e73-d803-a59b-b6d7102a61d3
Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC:
PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}
INFO[0000] Adding pod myubi_myubi to CNI network "podman" (type=bridge)
INFO[0000] Running conmon under slice machine.slice and unitName
libpod-conmon-c198a57f8fb8eebb2c8f391341fbb8bf0c02b84be2ee5b8b648e675adf07fb72.scope
INFO[0000] Got Conmon PID as 527872
*/bin/bash: error while loading shared libraries: libtinfo.so.6: cannot
change memory protections*INFO[0000] Got pod network &{Name:myubi
Namespace:myubi
ID:c198a57f8fb8eebb2c8f391341fbb8bf0c02b84be2ee5b8b648e675adf07fb72
NetNS:/run/netns/cni-e0647a42-5e73-d803-a59b-b6d7102a61d3
Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC:
PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}
INFO[0000] Deleting pod myubi_myubi from CNI network "podman" (type=bridge)
*[root@intlab-006 ~]# podman --log-level=debug run --rm --name=myubi -it
registry.access.redhat.com/ubi8/ubi
<
http://registry.access.redhat.com/ubi8/ubi> /bin/bash*INFO[0000] podman
filtering at log level debug
DEBU[0000] Called run.PersistentPreRunE(podman --log-level=debug run --rm
--name=myubi -it
registry.access.redhat.com/ubi8/ubi /bin/bash)
DEBU[0000] Merged system config "/usr/share/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at
/var/lib/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /var/lib/containers/storage
DEBU[0000] Using run root /run/containers/storage
DEBU[0000] Using static dir /var/lib/containers/storage/libpod
DEBU[0000] Using tmp dir /run/libpod
DEBU[0000] Using volume path /var/lib/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is being used
DEBU[0000] cached value indicated that native-diff is not being used
INFO[0000] Not using native diff for overlay, this may cause degraded
performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR
enabled
DEBU[0000] backingFs=extfs, projectQuotaSupported=false,
useNativeDiff=false, usingMetacopy=true
DEBU[0000] Initializing event backend file
DEBU[0000] configured OCI runtime kata initialization failed: no valid
executable found for OCI runtime kata: invalid argument
DEBU[0000] configured OCI runtime runsc initialization failed: no valid
executable found for OCI runtime runsc: invalid argument
DEBU[0000] Using OCI runtime "/usr/bin/runc"
INFO[0000] Found CNI network podman (type=bridge) at
/etc/cni/net.d/87-podman-bridge.conflist
DEBU[0000] Default CNI network name podman is unchangeable
INFO[0000] Setting parallel job count to 97
DEBU[0000] Pulling image
registry.access.redhat.com/ubi8/ubi (policy:
missing)
DEBU[0000] Looking up image "registry.access.redhat.com/ubi8/ubi" in local
containers storage
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi" ...
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi:latest" ...
DEBU[0000] parsed reference into "[overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
([overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36)
DEBU[0000] Looking up image "registry.access.redhat.com/ubi8/ubi:latest" in
local containers storage
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi:latest" ...
DEBU[0000] parsed reference into "[overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi:latest" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi:latest" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
([overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36)
DEBU[0000] Looking up image "registry.access.redhat.com/ubi8/ubi" in local
containers storage
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi" ...
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi:latest" ...
DEBU[0000] parsed reference into "[overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
([overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36)
DEBU[0000] Inspecting image
fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Looking up image "registry.access.redhat.com/ubi8/ubi" in local
containers storage
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi" ...
DEBU[0000] Trying "registry.access.redhat.com/ubi8/ubi:latest" ...
DEBU[0000] parsed reference into "[overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
DEBU[0000] Found image "registry.access.redhat.com/ubi8/ubi" as "
registry.access.redhat.com/ubi8/ubi:latest" in local containers storage
([overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36)
DEBU[0000] Inspecting image
fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] Inspecting image
fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36
DEBU[0000] using systemd mode: false
DEBU[0000] setting container name myubi
DEBU[0000] No hostname set; container's hostname will default to runtime
default
DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json"
DEBU[0000] Allocated lock 0 for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] parsed reference into "[overlay@
/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] exporting opaque data as blob
"sha256:fca12da1dc30ed8e7d03afb84b287fc695673fff9c04bfcb2ff404b558670a36"
DEBU[0000] created container
"278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b"
DEBU[0000] container
"278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b" has work
directory
"/var/lib/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata"
DEBU[0000] container
"278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b" has run
directory
"/run/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata"
DEBU[0000] Handling terminal attach
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is being used
DEBU[0000] backingFs=extfs, projectQuotaSupported=false,
useNativeDiff=false, usingMetacopy=true
DEBU[0000] cached value indicated that volatile is being used
DEBU[0000] overlay:
mount_data=nodev,metacopy=on,lowerdir=/var/lib/containers/storage/overlay/l/O3PSOUOEW3QSVICI3R4JKCLUZM:/var/lib/containers/storage/overlay/l/D3BH2BNFY3ROCUAPMECXAE73X6,upperdir=/var/lib/containers/storage/overlay/b98d9c77348a5c73f50158fddb496fec6e73dd4b5258df1137bddf62095606c1/diff,workdir=/var/lib/containers/storage/overlay/b98d9c77348a5c73f50158fddb496fec6e73dd4b5258df1137bddf62095606c1/work,volatile,context="system_u:object_r:container_file_t:s0:c767,c1001"
DEBU[0000] mounted container
"278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b" at
"/var/lib/containers/storage/overlay/b98d9c77348a5c73f50158fddb496fec6e73dd4b5258df1137bddf62095606c1/merged"
DEBU[0000] Created root filesystem for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b at
/var/lib/containers/storage/overlay/b98d9c77348a5c73f50158fddb496fec6e73dd4b5258df1137bddf62095606c1/merged
DEBU[0000] Made network namespace at
/run/netns/cni-61dfd753-babd-927f-78f8-6bafafbb78a5 for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
INFO[0000] Got pod network &{Name:myubi Namespace:myubi
ID:278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
NetNS:/run/netns/cni-61dfd753-babd-927f-78f8-6bafafbb78a5
Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC:
PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}
INFO[0000] Adding pod myubi_myubi to CNI network "podman" (type=bridge)
DEBU[0000] [0] CNI result: &{0.4.0 [{Name:cni-podman0 Mac:2a:5c:33:fc:7e:44
Sandbox:} {Name:veth9766c440 Mac:d2:62:4b:e5:c2:29 Sandbox:} {Name:eth0
Mac:2a:6e:79:27:dc:9c
Sandbox:/run/netns/cni-61dfd753-babd-927f-78f8-6bafafbb78a5}] [{Version:4
Interface:0xc0001ff3a8 Address:{IP:10.88.0.9 Mask:ffff0000}
Gateway:10.88.0.1}] [{Dst:{IP:0.0.0.0 Mask:00000000} GW:<nil>}] {[] [] []}}
DEBU[0000] Workdir "/" resolved to host path
"/var/lib/containers/storage/overlay/b98d9c77348a5c73f50158fddb496fec6e73dd4b5258df1137bddf62095606c1/merged"
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode
subscription
DEBU[0000] Setting CGroups for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b to
machine.slice:libpod:278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] reading hooks from /usr/share/containers/oci/hooks.d
DEBU[0000] Created OCI spec for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b at
/var/lib/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata/config.json
DEBU[0000] /usr/bin/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/bin/conmon
args="[--api-version 1 -c
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b -u
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b -r
/usr/bin/runc -b
/var/lib/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata
-p
/run/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata/pidfile
-n myubi --exit-dir /run/libpod/exits --full-attach -s -l
k8s-file:/var/lib/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata/ctr.log
--log-level debug --syslog -t --conmon-pidfile
/run/containers/storage/overlay-containers/278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b/userdata/conmon.pid
--exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg
/var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg
/run/containers/storage --exit-command-arg --log-level --exit-command-arg
debug --exit-command-arg --cgroup-manager --exit-command-arg systemd
--exit-command-arg --tmpdir --exit-command-arg /run/libpod
--exit-command-arg --runtime --exit-command-arg runc --exit-command-arg
--storage-driver --exit-command-arg overlay --exit-command-arg
--storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on
--exit-command-arg --events-backend --exit-command-arg file
--exit-command-arg --syslog --exit-command-arg container --exit-command-arg
cleanup --exit-command-arg --rm --exit-command-arg
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b]"
INFO[0000] Running conmon under slice machine.slice and unitName
libpod-conmon-278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b.scope
DEBU[0000] Received: 528374
INFO[0000] Got Conmon PID as 528361
DEBU[0000] Created container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b in OCI
runtime
DEBU[0000] Attaching to container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] Received a resize event: {Width:194 Height:151}
DEBU[0000] Starting container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b with
command [/bin/bash]
DEBU[0000] Started container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
*/bin/bash: error while loading shared libraries: libtinfo.so.6: cannot
change memory protections*DEBU[0000] Enabling signal proxying
DEBU[0000] Checking if container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b should
restart
DEBU[0000] Removing container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] Removing all exec sessions for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] Cleaning up container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] Tearing down network namespace at
/run/netns/cni-61dfd753-babd-927f-78f8-6bafafbb78a5 for container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
INFO[0000] Got pod network &{Name:myubi Namespace:myubi
ID:278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
NetNS:/run/netns/cni-61dfd753-babd-927f-78f8-6bafafbb78a5
Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC:
PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}
INFO[0000] Deleting pod myubi_myubi from CNI network "podman" (type=bridge)
DEBU[0000] Successfully cleaned up container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b
DEBU[0000] unmounted container
"278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b"
DEBU[0000] Container
278fce491f48c3ed0496881a5adf544719e9f713f57189074c9214eb494fc46b storage is
already unmounted, skipping...
DEBU[0000] Called run.PersistentPostRunE(podman --log-level=debug run --rm
--name=myubi -it
registry.access.redhat.com/ubi8/ubi /bin/bash)
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is being used
DEBU[0000] backingFs=extfs, projectQuotaSupported=false,
useNativeDiff=false, usingMetacopy=true
Nalin