Search results for "2023" - Podman List Archives

[Podman] Speeding up podman by using cache

by Ganeshar, Puvi

Hello Podman team, I am about explore this option so just wanted to check with you all before as I might be wasting my time. I am in Platform Engineering team at DirecTV, and we run Go and Java pipelines on Jenkins using Amazon EKS as the workers. So, the process is that when a Jenkins build runs, it asks the EKS for a worker (Kubernetes pod) and the cluster would spawn one and the new pod would communicate back to the Jenkins controller. We use the Jenkins Kubernetes pod template to configure the communication. We are currently running the latest LTS of podman, v5.2.2, however still using cgroups-v1 for now, planning to migrate early 2025 by upgrading the cluster to use Amazon Linux 2023 which uses cgroups-v2 as default. Here’s the podman configuration details that we use: host: arch: arm64 buildahVersion: 1.37.2 cgroupControllers: - cpuset - cpu - cpuacct - blkio - memory - devices - freezer - net_cls - perf_event - net_prio - hugetlb - pids cgroupManager: cgroupfs cgroupVersion: v1 conmon: package: conmon-2.1.12-1.el9.aarch64 path: /usr/bin/conmon version: 'conmon version 2.1.12, commit: f174c390e4760883511ab6b5c146dcb244aeb647' cpuUtilization: idlePercent: 99.22 systemPercent: 0.37 userPercent: 0.41 cpus: 16 databaseBackend: sqlite distribution: distribution: centos version: "9" eventLogger: file freeLocks: 2048 hostname: podmanv5-arm idMappings: gidmap: null uidmap: null kernel: 5.10.225-213.878.amzn2.aarch64 linkmode: dynamic logDriver: k8s-file memFree: 8531066880 memTotal: 33023348736 networkBackend: netavark networkBackendInfo: backend: netavark dns: package: aardvark-dns-1.12.1-1.el9.aarch64 path: /usr/libexec/podman/aardvark-dns version: aardvark-dns 1.12.1 package: netavark-1.12.2-1.el9.aarch64 path: /usr/libexec/podman/netavark version: netavark 1.12.2 ociRuntime: name: crun package: crun-1.16.1-1.el9.aarch64 path: /usr/bin/crun version: |- crun version 1.16.1 commit: afa829ca0122bd5e1d67f1f38e6cc348027e3c32 rundir: /run/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL os: linux pasta: executable: /usr/bin/pasta package: passt-0^20240806.gee36266-2.el9.aarch64 version: | pasta 0^20240806.gee36266-2.el9.aarch64-pasta Copyright Red Hat GNU General Public License, version 2 or later https://www.gnu.org/licenses/old-licenses/gpl-2.0.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. remoteSocket: exists: false path: /run/podman/podman.sock rootlessNetworkCmd: pasta security: apparmorEnabled: false capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: false seccompEnabled: true seccompProfilePath: /usr/share/containers/seccomp.json selinuxEnabled: false serviceIsRemote: false slirp4netns: executable: /usr/bin/slirp4netns package: slirp4netns-1.3.1-1.el9.aarch64 version: |- slirp4netns version 1.3.1 commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236 libslirp: 4.4.0 SLIRP_CONFIG_VERSION_MAX: 3 libseccomp: 2.5.2 swapFree: 0 swapTotal: 0 uptime: 144h 6m 15.00s (Approximately 6.00 days) variant: v8 plugins: authorization: null log: - k8s-file - none - passthrough - journald network: - bridge - macvlan - ipvlan volume: - local registries: search: - registry.access.redhat.com - registry.redhat.io - docker.io store: configFile: /etc/containers/storage.conf containerStore: number: 0 paused: 0 running: 0 stopped: 0 graphDriverName: overlay graphOptions: overlay.mountopt: nodev,metacopy=on graphRoot: /var/lib/containers/storage graphRootAllocated: 107352141824 graphRootUsed: 23986397184 graphStatus: Backing Filesystem: xfs Native Overlay Diff: "false" Supports d_type: "true" Supports shifting: "true" Supports volatile: "true" Using metacopy: "false" imageCopyTmpDir: /var/tmp imageStore: number: 1 runRoot: /run/containers/storage transientStore: false volumePath: /var/lib/containers/storage/volumes version: APIVersion: 5.2.2 Built: 1724331496 BuiltTime: Thu Aug 22 12:58:16 2024 GitCommit: "" GoVersion: go1.22.5 (Red Hat 1.22.5-2.el9) Os: linux OsArch: linux/arm64 Version: 5.2.2 We migrated to podman when Kubernetes deprecated docker and have been using podman for the last two years or so. Its working well, however since we run over 500 builds a day, I am trying to explore whether I can speed up the podman build process by using image caching. I wanted to see if I use an NFS file system (Amazon FSX) as the storage for podman (overlay-fs) would it improve podman performance by the builds completing much faster as of the already downloaded images on the NFS. Currently, podman in each pod on the EKS cluster would download all the required images every time so not taking advantage of the cached images. These are my concerns: 1. Any race conditions, a podman processes colliding with each other during read and write. 2. Performance of I/O operations as NFS communication will be over the network. Have any of you tried this method before? If so, can you share any pitfalls that you’ve faced? Any comments / advice would be beneficial as I need to weigh up pros and cons before spending time on this. Also, if it causes outage due to storage failures it would block all our developers; so, I will have to design this in a way where we can recover quickly. Thanks very much in advance and have a great day. Puvi Ganeshar | @pg925u Principal, Platform Engineer CICD - Pipeline Express | Toronto [Image]

10 months, 3 weeks

[Podman] Re: Need some help for a rather strange usecase

by Boris Behrens

That was easy. Thanks. Am Di., 1. Aug. 2023 um 22:59 Uhr schrieb Daniel Walsh <dwalsh(a)redhat.com>: > On 7/31/23 09:40, Boris Behrens wrote: > Try > --cap-add net-admin > > And you should be allowed to modify the IPs inside of the container. > > Hi, > > sorry if this question is bad. You are allowed to flame me for this :) > > I would like to create a container which is basically connected to two > separate VLANs and does some sort of bridging between them. > I also would like to be able to assign IP addresses from inside the > container, because I would like to assign IP addresses via keepalived. > > The reason, why I would like to do it that way is to reduce cross traffic > between hosts. > > I have three hosts, that are attached to a public VLAN. All three hosts > got a public IP address, which needs to be assigned to another host in case > of something is going wrong. > HAProxy picks up the request on the public VLAN and forwards it to the > underlying backend, which is in the same container. This backend talks to a > storage cluster via the private VLAN. > > The container host is a ubuntu 20.04 with Podman 3.4.2 > > What I did until now: > - create two additional networks [1] > - create a container [2] > > But now I have the problem that I am not allowed to add an IP address from > inside the container [3] > > I also don't know if I have a conceptual error in the whole thing, because > it is a strange thing to use containers as a VM replacement. > But currently I just don't know better. > > Hope someone can help me. > > --- > A more in depth description of what I try to solve: > I have a ceph cluster that serves s3 traffic via radosgw. > radosgw talks to all the physical disks in the cluster directly. So it > does the discribution of objects. > To do TLS termination, some basic HTTP header manipulation, and other HTTP > related stuff a HAProxy is sitting in front of the radosgw. > > I don't want to have a public IP address directly on a storage host, > because misconfiguration happen and this is something I want to avoid. > > So I thought I could spin up a container on some storage server, map the > public VLAN and the private storage VLAN into the container and combine > HAProxy and radosgw into one unit. > > Now there is the problem with public availablity. I want to used DNS load > balancing for the HAproxy. So every HAProxy gets it's own public IP > address. But when one ore more HAproxy instance fail (there are so many > things that can go south) I would like to add the IP address to another > container. > > Here comes keepalived, which does VRRP from inside the containers and when > some container stop anouncing it is available, another host spins up the IP > address, and starts to serve it. > > And because I am struggling with even those simple tasks, I don't want to > even try k8s/k3s. Also I think k8s/k3s have a lot of cross traffic between > the instances, which might kill the performance really hard. > --- > > [1] > $ podman network create --disable-dns --driver=macvlan -o parent=bond0.50 > --subnet 10.64.1.0/24 public > $ podman network create --disable-dns --driver=macvlan -o parent=bond0.43 > --subnet 10.64.2.0/24 management > > [2] > $ podman run --detach --hostname=frontend-`hostname` --name > frontend-`hostname -s` \ > > --mount=type=bind,source=/opt/frontend/etc/haproxy,destination=/etc/haproxy,ro > \ > > --mount=type=bind,source=/opt/frontend/etc/ssl/frontend,destination=/etc/ssl/frontend,ro > \ > --network=podman,public,management \ > -it ubuntu:20.04 /bin/bash > > [3] > root@frontend-0cc47a6df14e:/# ip addr add 192.168.0.1/24 dev eth2 > RTNETLINK answers: Operation not permitted > > Best wishes > Boris > > _______________________________________________ > Podman mailing list -- podman(a)lists.podman.io > To unsubscribe send an email to podman-leave(a)lists.podman.io > > > _______________________________________________ > Podman mailing list -- podman(a)lists.podman.io > To unsubscribe send an email to podman-leave(a)lists.podman.io > -- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groÃƒ¼en Saal.

2 years, 1 month

[Podman] Re: Improving our CI-driven, rootless, systemd user service-based container deployment with Quadlet?

by Valentin Rothberg

Thanks for reaching out! On Tue, Feb 14, 2023 at 4:54 PM <jklaiho(a)iki.fi> wrote: > > Hi all. Late last year, we managed to deploy one of our web applications as a rootless Podman container and a systemd user service on a Ubuntu 22.04 VPS, using Podman 4.2.0, also building a full GitLab CI pipeline around it. > > Now Podman 4.4.0 is out, featuring Quadlet. There's no mention of the release on podman.io, let alone any docs for the Podman-integrated version of Quadlet that I can find. (Any ETA on these? I only realized it was out thanks to this mailing list.) > > Below I'll describe our deployment workflow and show an annotated version of the systemd service we use. What I'd like to know is: is there a way of using Quadlet for this use case? Would it be an improvement over the current setup in terms of using systemd/Podman "optimally"? You can find the docs of Quadlet here: https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html Note that we only support systemd units generated by `podman-generate-systemd` and Quadlet. Any other customized units are not supported. There are too many things that can go wrong. > It would especially be nice if Quadlet could give us a working Type=notify unit file, to eliminate the need for PID files. I was previously unable to get it to work (I no longer remember why), and had to use Type=forking instead. Yes, Quadlet supports that. `podman-generate-systemd --new` supports that, too. > - - - - - > > Container deployment is done with an Ansible playbook, run by GitLab CI. It connects to the VPS, updates the image from our private registry, builds an entirely new container out of it with a unique suffix (appname-<imagetag>_<githash>) and templates out a unit file with a matching name (the Jinja2 template is based on 'podman generate systemd' output). The old systemd service is stopped and disabled, and the new one started and enabled. > > We do this because we want deploys to be "atomic", minimizing downtime. Building a new container instead of updating the old one lets us quickly revert to the previous version if the new container is faulty somehow. (Old containers and their unit files are eventually removed with a cron job.) > > Here's the unit file, with some annotations about our changes: > > [Unit] > Description=AppName > Wants=network-online.target > After=network-online.target > RequiresMountsFor=%t/containers > > [Service] > # No Environment=PODMAN_SYSTEMD_UNIT=%n clause, because we don't use podman-auto-update > > Restart=on-failure > TimeoutStopSec=10 > > # Don't try to start unless the container exists that this service is named after. > ExecCondition=/bin/bash -c "/usr/bin/podman container inspect %N -f '{{.State.Status}}' 2> /dev/null" > > # Symlink the PID file to a more convenient location (not strictly necessary, > # looks nicer here in PIDFile and ExecStopPost). > ExecStartPre=/bin/bash -c "ln -sf \ > $XDG_RUNTIME_DIR/containers/vfs-containers/$(podman container inspect %N -f '{{.Id}}')/userdata/conmon.pid \ > $XDG_RUNTIME_DIR/.%N_conmon.pid" > > # Type=notify would be nicer > Type=forking > PIDFile=%t/.%N_conmon.pid > > ExecStart=/usr/bin/podman start %N > > # This pattern of running 'podman stop' in both ExecStop and ExecStopPost > # is from podman-generate-systemd, but I never understood the reasoning for it. > ExecStop=/usr/bin/podman stop --ignore -t 10 %N > ExecStopPost=/usr/bin/podman stop --ignore -t 10 %N > > # Clean up the PID file symlink > ExecStopPost=/bin/rm -f $XDG_RUNTIME_DIR/.%N_conmon.pid > > [Install] > # Saves us from having to deal with the full unit name with the image tag and the > # git hash; the symlink to this name is replaced to point to the new unit file > # during the Ansible deployment. > Alias=appname.service > WantedBy=default.target > > - - - - - > > This is pretty robust for our purposes, but my systemd and overall Podman knowledge is limited, so I don't know what I could be doing better. Quadlet has a rather different philosophy overall than what we're used to, but can it be leveraged in this workflow, for CI-driven replacements of rootless containers running as systemd user services? Yes, Quadlet should integrate well into this workflow. Kind regards, Valentin > _______________________________________________ > Podman mailing list -- podman(a)lists.podman.io > To unsubscribe send an email to podman-leave(a)lists.podman.io

2 years, 7 months

[Podman] Re: fs.mqueue.msg_max rootless problem

by Михаил Иванов

Hallo, thanks for advice! But sorry, for me it did not work: podman create --name ctest --pod test --ipc private --cap-add=SYS_PTRACE --init --replace test-image container=99425540b8e3544409e4086cf1a44b04cf9f402f1d7505f807324dce71eb2373 podman init test test podman inspect -f '{{.State.Pid}}' test pid=2157674 sudo nsenter --target 2157674 --user --ipc sysctl fs.mqueue.msg_max=64 sysctl: permission denied on key "fs.mqueue.msg_max" Anyway, even if it would work, this method would not be appropriate in my case, since eventually my containers should be run from quadlet (which in turn uses podman kube play). Shell is used only during development. Best regards, On 29.11.2023 18:10, Lewis Gaul wrote: > Hi, > > I think this is the same thing I raised in > https://github.com/containers/podman/discussions/19737? > > This seems to be a kernel limitation - I'm not sure where the mqueue > limits come from when creating a new IPC namespace, but it doesn't > inherit the limits from the parent namespace and the root user within > the user namespace does not have permissions to modify the limits. > This was supposedly fixed in a recent kernel version although I > haven't tested it. > > The workaround I'm currently using (requiring sudo permissions) is > along the lines of: > podman create --ipc private --name ctr_foo ... > podman init ctr_foo > ctr_pid=$(podman inspect -f '{{.State.Pid}}' ctr_foo) > sudo nsenter --target $ctr_pid --user --ipc sysctl fs.mqueue.msg_max=64 > podman start ctr_foo > > Obviously this isn't ideal, and I'd be open to alternatives... > > Regards, > Lewis > > On Mon, 27 Nov 2023 at 12:23, Daniel Walsh <dwalsh(a)redhat.com> wrote: > > On 11/27/23 02:04, Михаил Иванов wrote: >> Hallo, >> >> For me rootful works: >> >> island:container [master]> cat /proc/sys/fs/mqueue/msg_max >> 256 > > $ podman run alpine ls -ld /proc/sys/fs/mqueue/msg_max > -rw-r--r-- 1 nobody nobody 0 Nov 27 12:10 > /proc/sys/fs/mqueue/msg_max > > Since it is owned by real root, a rootless user can not write to > it. I guess we could ague this is a bug with the kernel. > mqeueu/msg_max should be owned by root of the user namespace as > opposed to real root. > >> ## Rootful: >> island:container [master]> sudo podman run --sysctl=fs.mqueue.msg_max=64 --rm centos cat /proc/sys/fs/mqueue/msg_max >> 64 >> >> ## Rootless: >> island:container [master]> podman run --sysctl=fs.mqueue.msg_max=64 --rm centos cat /proc/sys/fs/mqueue/msg_max >> Error: crun: open `/proc/sys/fs/mqueue/msg_max`: Permission denied: OCI permission denied >> >> ## What rootless gets by default (changed as compared to host setting!): >> island:container [master]> podman run --rm centos cat /proc/sys/fs/mqueue/msg_max >> 10 >> >> Rgrds, >> On 25.11.2023 20:17, Daniel Walsh wrote: >>> On 11/25/23 10:44, Михаил Иванов wrote: >>>> Hallo, >>>> Is it possible to get podman to propagate current host fs.mqueue.msg_max >>>> value to rootless container? I can do that if I specify --ipc host when >>>> running the container, but this also exposes other ipc stuff from host >>>> to container, including shared memory, which I do not want. >>>> >>>> If I specify --sysctl fs.mqueue.msg_size=64 to podman it gives me >>>> "OCI permission denied" error, even when my host setting (256) is greater >>>> than requested value. >>>> Thanks, >>>> -- >>>> Micvhael Ivanov >>>> >>>> _______________________________________________ >>>> Podman mailing list --podman(a)lists.podman.io >>>> To unsubscribe send an email topodman-leave(a)lists.podman.io >>> >>> The way you attempted is correct. Might not be allowed for >>> rootless containers. >>> >>> I attempted this in a rootful container and it blows up for me. >>> >>> >>> podman run --sysctl fs.mqueue.msg_size=64 alpine echo hi >>> Error: crun: open `/proc/sys/fs/mqueue/msg_size`: No such file >>> or directory: OCI runtime attempted to invoke a command that was >>> not found >>> >>> >>> _______________________________________________ >>> Podman mailing list --podman(a)lists.podman.io >>> To unsubscribe send an email topodman-leave(a)lists.podman.io > > > _______________________________________________ > Podman mailing list -- podman(a)lists.podman.io > To unsubscribe send an email to podman-leave(a)lists.podman.io >

1 year, 9 months

[Podman] Re: Follow-up: Rootless storage usage

by Михаил Иванов

I did this already several times. It does not. I also checked ~/.config/containers, there were only two items there: empty directory podman and registries.conf with following contents: ## [[registry]] ## prefix = "test" ## location = "docker.test.local" unqualified-search-registries = [ 'docker.test.local', 'docker.io' ] To be on the very safe side I repeated the test, I ran podman system reset and then cleaned up whatever was remaining in ~/.config/containers and ~/.local/share/containers manually. Then I just podman run alpine. vfs was used again: island:kit [master]> ls -la ~/.local/share/containers/storage/ итого 48 drwx------ 10 ivans users 4096 янв 25 20:41 ./ drwxr-xr-x 5 ivans users 4096 янв 25 20:41 ../ -rw-r--r-- 1 ivans users 8 янв 25 20:41 defaultNetworkBackend drwx------ 2 ivans users 4096 янв 25 20:41 libpod/ drwx------ 2 ivans users 4096 янв 25 20:41 mounts/ drwx------ 2 ivans users 4096 янв 25 20:41 networks/ -rw-r--r-- 1 ivans users 64 янв 25 20:41 storage.lock drwx------ 2 ivans users 4096 янв 25 20:41 tmp/ -rw-r--r-- 1 ivans users 0 янв 25 20:41 userns.lock drwx------ 3 ivans users 4096 янв 25 20:41 vfs/ drwx------ 3 ivans users 4096 янв 25 20:41 vfs-containers/ drwx------ 3 ivans users 4096 янв 25 20:41 vfs-images/ drwx------ 2 ivans users 4096 янв 25 20:41 vfs-layers/ My system has following fuse-related packages: island:kit [master]> dpkg -l | grep fuse ii fuse-overlayfs 1.9-1 amd64 implementation of overlay+shiftfs in FUSE for rootless containers ii fuse3 3.13.0-2 amd64 Filesystem in Userspace (3.x version) ii libfuse2:amd64 2.9.9-6 amd64 Filesystem in Userspace (library) ii libfuse3-3:amd64 3.13.0-2 amd64 Filesystem in Userspace (library) (3.x version) Rgrds, On 25.01.2023 17:55, Daniel Walsh wrote: > Is there any config files in ~/.config/containers? > > podman system reset > > Should remove everything, and from then on Podman should use rootless > overlay. > > On 1/25/23 09:52, Михаил Иванов wrote: >> Is native overlay available in rootless mode? >> When I run podman as root there's no problem, overlayfs is picked up >> as default in debian. VFS is selected as default only in rootless mode. >> Rgrds, >> On 25.01.2023 14:03, Giuseppe Scrivano wrote: >>> Reinhard Tartler<siretart(a)gmail.com> writes: >>> >>>> On Tue, Jan 24, 2023 at 2:08 PM Daniel Walsh<dwalsh(a)redhat.com> wrote: >>>> >>>> On 1/24/23 03:47, Reinhard Tartler wrote: >>>> >>>> Dan, >>>> >>>> In Debian, I've chosen to just go with the upstream defaults: >>>> https://github.com/containers/storage/blob/8428fad6d0d3c4cded8fd7702af36a... >>>> >>>> This file is installed verbatim to /usr/share/containers/storage.conf. >>>> >>>> Is there a better choice? Does Fedora/Redhat provide a default storage.conf from somewhere else? >>>> >>>> Thanks, >>>> -rt >>>> >>>> That should be fine. Fedora goes with that default as well. Does debian support rootless overlay by default? >>>> >>>> If not then it would fail over to VFS if fuse-overlayfs is not installed. >>>> >>>> I'm a bit confused about what you mean with that. >>>> >>>> In Debian releases that ship podman 4.x we have at least Linux kernel 6.0. The fuse-overlayfs package is installed by default, but users may opt to not >>>> install it by configuring apt to not install "Recommends" by default. >>>> >>>> What else is required for rootless overlay? >>>> >>>> Also, if I follow this conversation, then it appears that the default storage.conf requires modification in line 118 (to uncomment the mount_program >>>> option) in order to actually use fuse-overlayfs. I would have expected podman to use fuse-overlayfs if it happens to be installed, and fallback to direct >>>> mount if not. I read Michail's email thread that this appears to be not the case and he had to spend a lot of effort figuring out how to install an >>>> appropriate configuration file. Maybe I'm missing something, but I wonder what we can do to improve the user experience? >>> what issue do you see if you use native overlay? >>> >>> Podman prefers native overlay if it is available, since it is faster. >>> If not, it tries fuse-overlays and if it is not available, it falls back >>> to vfs. >>> >>> Could you try from a fresh storage though? If fuse-overlayfs was >>> already used, then Podman will continue using it even if native overlay >>> is available, since the storage metadata is slightly different. >>> >>> Thanks, >>> Giuseppe >>> _______________________________________________ >>> Podman mailing list --podman(a)lists.podman.io >>> To unsubscribe send an email topodman-leave(a)lists.podman.io > >

2 years, 7 months

[Podman] Re: Rootless podman mount capabilities

by Lewis Gaul

Hi Giuseppe, Thanks, some useful points there. However, my question was more specifically around how "special" mounts get created in containers, given it's not possible for the container process itself to create them. A concrete example below using rootless podman... > podman run --rm -it --name ubuntu --privileged ubuntu:20.04 root@b2069e97cd13:/# findmnt -R /sys/fs/cgroup/freezer TARGET SOURCE FSTYPE OPTIONS /sys/fs/cgroup/freezer cgroup cgroup rw,nosuid,nodev,noexec,relatime,seclabel,freezer root@b2069e97cd13:/# umount /sys/fs/cgroup/freezer root@b2069e97cd13:/# mount -t cgroup cgroup /sys/fs/cgroup/freezer -o rw,nosuid,nodev,noexec,relatime,seclabel,freezer mount: /sys/fs/cgroup/freezer: permission denied. This shows that cgroup mounts are present in the container, and yet the container does not have permission to create the mount. However, I've realised these are perhaps just bind mounts from the host mount namespace? I can simulate this as follows: > podman run --rm -it --name ubuntu --privileged -v /sys/fs/cgroup:/tmp/host/cgroup:ro ubuntu:20.04 root@495f11acdd5b:/# findmnt -R /tmp/host/cgroup/freezer/ TARGET SOURCE FSTYPE OPTIONS /tmp/host/cgroup/freezer cgroup cgroup rw,nosuid,nodev,noexec,relatime,seclabel,freezer root@495f11acdd5b:/# umount /sys/fs/cgroup/freezer root@495f11acdd5b:/# mount --bind /tmp/host/cgroup/freezer /sys/fs/cgroup/freezer root@495f11acdd5b:/# findmnt -R /sys/fs/cgroup/freezer/ TARGET SOURCE FSTYPE OPTIONS /sys/fs/cgroup/freezer cgroup cgroup rw,nosuid,nodev,noexec,relatime,seclabel,freezer One further thing I'm unclear on is as follows. It seems when a new mount namespace is created that the mount list is copied from the parent process, but some of the container cgroup mounts are bind mounts at some point in the hierarchy rather than being the same as the host mounts. Perhaps the container runtime first unmounts /sys/fs/cgroup in the container mount namespace before creating these bind mounts? root@495f11acdd5b:/# findmnt /sys/fs/cgroup/devices TARGET SOURCE FSTYPE OPTIONS /sys/fs/cgroup/devices cgroup[/user.slice] cgroup rw,nosuid,nodev,noexec,relatime,seclabel,devices root@495f11acdd5b:/# findmnt /tmp/host/cgroup/devices TARGET SOURCE FSTYPE OPTIONS /tmp/host/cgroup/devices cgroup cgroup rw,nosuid,nodev,noexec,relatime,seclabel,devices Thanks, Lewis On Thu, 14 Sept 2023 at 12:46, Giuseppe Scrivano <gscrivan(a)redhat.com> wrote: > Lewis Gaul <lewis.gaul(a)gmail.com> writes: > > > Hi, > > > > I'm trying to understand something about how capabilities in rootless > podman work. > > > > How does rootless podman have the capability to set up container mounts > (such as cgroup mounts) given a privileged container itself doesn't? Does > > podman deliberately drop caps, or somehow get elevated privileges to do > this? > > > > This is the process tree podman sets up (where bash is the container > entrypoint here): > > systemd(1)---conmon(1327421)---bash(1327432) > > > > I'm assuming it's conmon that sets up the container's mounts (via runc > in this case), which is a process running as my user (rootless). How is it > that > > conmon has the capabilities required (SYS_ADMIN?) to create the > container's cgroup and sysfs mounts but within the container itself this is > not > > possible? > > > > Thanks for any insight! > > a rootless container is able to perform "privileged" operations by using a > user namespace, and in that user namespace it gains the capabilities > required to perform mounts. > > Be aware that in a user namespace, the root user is still limited to > what it can do, as the kernel differentiates between the root user on > the host (known as the initial user namespace) and any other user > namespace. > > The user namespace is a special namespace, that alters how other > namespaces work since each namespace is "owned" by a user namespace. > > So a user namespace alone is not enough to perform mounts, the user must > also create a new mount namespace. The combination user namespace+mount > namespace is what "podman unshare" creates. > > For example: > > $ podman unshare > $ id > uid=0(root) gid=0(root) groups=0(root),65534(nobody) > context=unconfined_u:unconfined_r:container_runtime_t:s0-s0:c0.c1023 > $ mkdir /tmp/test > $ mount -t tmpfs tmpfs /tmp/test > $ exit > > You can try manually: > > $ unshare -r bash ## creates a user namespace and maps your user to root > $ mkdir /tmp/test; mount -t tmpfs tmpfs /tmp/test > mkdir: cannot create directory ‘/tmp/test’: File exists > mount: /tmp/test: permission denied. > dmesg(1) may have more information after failed mount system call. > > The failure happens because the user namespace does not own the mount > namespace as it is owned by the initial user namespace. > > So in order to perform a mount, you must create a mount namespace: > > $ unshare -m bash ## the new mount namespace is owned by the current > ## user namespace > $ mount -t tmpfs tmpfs /tmp/test > > In the rootless container case, the container mounts are performed by > the OCI runtime that runs in the user+mount namespace created by > Podman. > > Regards, > Giuseppe > >

2 years

[Podman] Re: RunRoot & mistaken IDs

by Daniel Walsh

On 1/29/24 02:35, lejeczek via Podman wrote: > > > On 28/03/2023 21:00, Chris Evich wrote: >> On 3/28/23 09:06, lejeczek via Podman wrote: >>> I think it might have something to do with the fact that I changed >>> UID for the user >> >> The files under /run/user/$UID are typically managed by >> systemd-logind. I've noticed sometimes there's a delay between >> logging out and the files being cleaned up. Try logging out for a >> minute or three and see if that fixes it. >> >> Also, if you have lingering enabled for the user, it may take a >> restart of particular the user.slice. >> >> Lastly, I'm not certain, but you (as root) may be able to `systemctl >> reload systemd-logind`. That's a total guess though. >> >> > Those parts seem very clunky - at least in up-to-date Centos 9 stream > - I have removed a user and re/created that user in IdM and.. > even after full & healthy OS reboot, containers/podman insist: > > -> $ podman container ls -a > WARN[0000] RunRoot is pointing to a path (/run/user/2001/containers) > which is not writable. Most likely podman will fail. > Error: default OCI runtime "crun" not found: invalid argument > > -> $ id > uid=1107400004(podmania) gid=1107400004(podmania) > groups=1107400004(podmania) > context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 > > Where/what does it persist/insist on that old, non-existent UID - > would anybody know? > > many thanks, L. > _______________________________________________ > Podman mailing list -- podman(a)lists.podman.io > To unsubscribe send an email to podman-leave(a)lists.podman.io Do you have XDG_RUNTIME_DIR pointing at it?

1 year, 7 months

[Podman] Re: Should I run podman-based systemd services as root?

by Mark Raynsford

On 2023-07-28T09:13:14 -0400 Daniel Walsh <dwalsh(a)redhat.com> wrote: > > The issue with running all of your containers as a non root users, is if > every container runs as a non-root user, then the containers would be > allowed to attack the user account and every other container, if they > were to escape confinement (SELinux). Hello! I read back what I wrote and realized it was a bit ambiguous. To be clear: I'm running each container as a separate non-root user; one user ID per container (not one user ID shared between all containers). > Running containers with the least privs possible is always the goal, > but it really is up to the application. This is where I'm still not entirely clear: Is running a container as root with SELinux and with flags such as --unprivileged=false really more powerful than as a regular user with the same kinds of flags? I haven't heard of anyone escaping SELinux confinement, although I'm guessing it has probably been done. I'd assume a kernel-level exploit would probably be required and, in that case, running under a different UID wouldn't help matters. I've tried setting up machines with all of the containers running as root, and it's certainly a lot less of an administrative burden. I run my own registry so I'm not _too_ concerned about hostile images making it into production containers. I feel like there's a huge hole in the documentation around this subject, and it's really weird that noone appears to be talking about it. Fedora Server runs all containers as root if configured via Cockpit, so presumably someone at least considered the issue? -- Mark Raynsford | https://www.io7m.com

2 years, 1 month

[Podman] Re: --dns=ipaddr - no effect of it

by lejeczek

On 30/05/2023 14:00, Daniel Walsh wrote: > On 5/29/23 05:59, lejeczek via Podman wrote: >> Hi guys. >> >> --dns=none renders what expected but with an actual >> server, say: --dns=10.3.1.200 >> resolve.conf seems to be the one of the host's, as if >> --dns did not happen. >> Can anybody else say that is the case? Am I missing >> something? >> I'm on Centos 9 stream with all bits up-to-date. >> many thanks, L. >> >> _______________________________________________ >> Podman mailing list --podman(a)lists.podman.io >> To unsubscribe send an email topodman-leave(a)lists.podman.io > > Here is what I am getting? > > ``` > # podman run --dns=10.3.1.200 alpine cat /etc/resolv.conf > nameserver 10.3.1.200 > # podman run --dns=none alpine cat /etc/resolv.conf > cat: can't open '/etc/resolv.conf': No such file or directory > ``` > > Rootless > > ``` > $ podman run --dns=10.3.1.200 alpine cat /etc/resolv.conf > nameserver 10.3.1.200 > $ podman run --dns=none alpine cat /etc/resolv.conf > cat: can't open '/etc/resolv.conf': No such file or directory > ``` > I'm trying, for production setup, Centos 9 (perhaps all official, available versions?) podman run -dt --network=off-host --ip=${_IP} --dns=10.3.1.200 --hostname ${_H}.${_DOM} --name ${_NAME} localhost/centos9-mine ... [root@centos-whale /]# cat /etc/resolv.conf search mine.priv mszczonow.vectranet.pl nameserver 10.3.1.254 nameserver 89.228.4.126 nameserver 31.11.173.2 nameserver 10.1.1.254 options timeout:1 that 'resolv.conf' is an exactl copy off the host, this: podman run -dt --network=off-host --ip=${_IP} --dns=none --hostname ${_H}.${_DOM} --name ${_NAME} localhost/centos9-mine [root@centos-whale /]# cat /etc/resolv.conf # Generated by NetworkManager nameserver 192.168.122.1 Perhaps the issue(s) is with centos? centos9-mine is done off the 'quay.io/centos/centos' with only a couple add rpm packages.

2 years, 3 months

[Podman] systemctl status during podman build

by etc＠balosh.net

Hi, question that I don't understand, I'd be grateful for explaining or reference to what I should read to get it. Why during Podman build command `systemctl enable httpd` is working but `systemctl status httpd` is not working? Dockerfile not working: ``` FROM registry.access.redhat.com/ubi8/ubi-init RUN yum -y install httpd; yum clean all; RUN systemctl enable httpd; RUN systemctl status httpd; ``` output of `podman build .`: ``` STEP 2/4: RUN yum -y install httpd; yum clean all; STEP 1/4: FROM registry.access.redhat.com/ubi8/ubi-init Build output: --> 02f6efde590f --> Using cache 02f6efde590f9fec989c04a01a661d2650b462aeb8e61ad3c0e00aae1b16b1ef --> Using cache 4f85f566fdee4fd8f5e8058dbf39c5ec9be95a4879d4d9a8c7a77f5b9cadf8a7 STEP 3/4: RUN systemctl enable httpd; STEP 4/4: RUN systemctl status httpd; --> 4f85f566fdee System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down ``` But! If I exec into the container when he is running both of them are working. Working Dockerfile: ``` FROM registry.access.redhat.com/ubi8/ubi-init RUN yum -y install httpd; yum clean all; RUN systemctl enable httpd; ``` command: `podman build . -t x ; podman run -d --name x x ; podman exec -ti x bash -c "systemctl status httpd"` runs with success ``` STEP 1/3: FROM registry.access.redhat.com/ubi8/ubi-init STEP 2/3: RUN yum -y install httpd; yum clean all; --> Using cache 02f6efde590f9fec989c04a01a661d2650b462aeb8e61ad3c0e00aae1b16b1ef --> 02f6efde590f STEP 3/3: RUN systemctl enable httpd; --> Using cache 4f85f566fdee4fd8f5e8058dbf39c5ec9be95a4879d4d9a8c7a77f5b9cadf8a7 COMMIT x --> 4f85f566fdee Successfully tagged localhost/x:latest 4f85f566fdee4fd8f5e8058dbf39c5ec9be95a4879d4d9a8c7a77f5b9cadf8a7 214ee56866fc0e7d71b6d152749bdcb65d4e5aadb95dafcebb5661ee20770619 [root@214ee56866fc /]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2023-09-19 20:07:22 UTC; 6s ago Docs: man:httpd.service(8) Main PID: 30 (httpd) Status: "Started, listening on: port 80" Tasks: 213 (limit: 1638) Memory: 22.3M CGroup: /system.slice/httpd.service ├─30 /usr/sbin/httpd -DFOREGROUND ├─34 /usr/sbin/httpd -DFOREGROUND ├─35 /usr/sbin/httpd -DFOREGROUND ├─36 /usr/sbin/httpd -DFOREGROUND └─37 /usr/sbin/httpd -DFOREGROUND versions: podman version 4.6.2 on macos ventura 13.5.2

1 year, 12 months

Search results for query "2023"