mqueue msg_max in rootless container
by Michael Ivanov
Hallo!
I'm trying to run my application in podman rootless container and I stumble
on following problem: my program needs /proc/sys/fs/mqueue/msg_max to be at
least 256, but in running container this value is just 10. When I try to
specify this parameter while running the image (--sysctl 'fs.mqueue.msg_max=256')
I get the following error:
Error: open /proc/sys/fs/mqueue/msg_max: Permission denied: OCI permission denied
and container is not created.
My host where container is being run has this parameter set to 256. How can I
expose current host setting for msg_max to my container?
Best regards,
--
\ / | |
(OvO) | Михаил Иванов |
(^^^) | |
\^/ | E-mail: ivans(a)isle.spb.ru |
^ ^ | |
12 months
podman container storage backup
by Michael Ivanov
Greetings,
I make periodic backups of my laptop where I use some podman containers.
To perform a backup I just invoke rsync to copy my /home/xxxx/.local/share/containers
directory to nfs mounted filesystem.
Containers are running, but quiescent, no real activity occurs.
Is this a correct way to back up or is there anything special about
container directory to be taken into account? As far as I understand
some hash-named subdirectories are shared between different containers
and images using special kind of mounts, can this lead to duplicate
copies r inconsistencies?
Underlying fs is btrfs.
Thanks,
--
\ / | |
(OvO) | Михаил Иванов |
(^^^) | |
\^/ | E-mail: ivans(a)isle.spb.ru |
^ ^ | |
2 years, 1 month
Upgrade Recommendations from 1.4.2 to 3.3.1
by Christopher.Miller＠gd-ms.com
Hello, currently running Podman 1.4.2 on RHEL 8.1 and want to upgrade 3.3.1 so we can take advantage of the newer features (like networking).
I am running containers currently in the 1.4.2 environment. Is there anything to think of or do before doing an upgrade, or a best practice?
Thanks
Chris Miller
Altron INC.
703-814-7647
Christopher.miller(a)altroninc.com
Christopher.Miller(a)gd-ms.com<mailto:Christopher.Miller@gd-ms.com>
2 years, 9 months
journald log driver not working
by Ranbir
Hello Everyone,
I'm testing out using journald for logging from one of my containers. I
think I configured it correctly, but I don't see anything in the path I
passed to it.
Here's the create command I used:
CreateCommand": [
"podman",
"run",
"-d",
"--name",
"postgrey",
"--pod",
"mailman",
"--volume",
"postgrey:/var/spool/postfix/postgrey:Z",
"--log-driver",
"journald",
"--log-opt",
"path=/var/log/containers/postgrey.log",
"postgrey"
],
/var/log/containers is empty:
# ls -l /var/log/containers/
total 0
Docker's docs on retrieving the logs with journalctl don't show me
anything:
journalctl CONTAINER_NAME=postgrey
-- Logs begin at Wed 2022-01-19 17:04:20 EST, end at Tue 2022-01-25
18:40:27 EST. --
-- No entries --
Incidentally, I used podman to crate the systemd unit files for the pod
and dumped them into /etc/systemd/system. I'm starting, stopping, etc
the pod using systemctl.
Using journald for logging means the container's logs get written to
the host's journal, right? If that's true, what am I doing wrong?
--
Ranbir
2 years, 9 months
Problem adding a routing rule for containers before they start
by Daniel Gray
Hi,
My server has multiple network interfaces.
- bond0.3 <- default route via 192.168.3.1
- bond0.7 <- lower priority interface via 192.168.7.1
In this case I want my containers to be using bond0.7.
I add a separate routing table and prepare default route:
echo "7 CONTAINERS" >> /etc/iproute2/rt_tables
sudo ip route add default via 192.168.7.1 table CONTAINERS
sudo ip route add 192.168.7.1 dev bond0.7 table CONTAINERS
sudo ip rule add from 10.89.0.0/24 lookup CONTAINERS
For this I am using rootful containers, though I plan to adapt these to
be rootless when I get them working.
sudo podman network create -d bridge net1
sudo podman run -dt --name test --network net1 --cap-add NET_RAW --rm busybox
sudo podman exec -it test ash
/ # wget -q -O - ifconfig.me
Nothing.........
/ # ip route get 1.1.1.1
1.1.1.1 via 10.89.0.1 dev eth0 src 10.89.0.2
/ # ip route
default via 10.89.0.1 dev eth0
10.89.0.0/24 dev eth0 scope link src 10.89.0.2
/ # traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 46 byte packets
1 128bedbab28b (10.89.0.2) 3034.227 ms !H 3039.811 ms !H 3039.875 ms !H
Now if I delete the rule:
sudo ip rule del from 10.89.0.0/24 lookup CONTAINERS
and do something like a traceroute (which takes the default route out):
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 46 byte packets
1 host.containers.internal (10.89.0.1) 0.006 ms 0.005 ms 0.004 ms
2 192.168.3.1 (192.168.3.1) 0.562 ms 0.815 ms 0.766 ms
...
then add the rule:
sudo ip rule add from 10.89.0.0/24 lookup CONTAINERS
/ # traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 46 byte packets
1 host.containers.internal (10.89.0.1) 0.008 ms 0.005 ms 0.004 ms
2 192.168.7.1 (192.168.7.1) 0.533 ms 0.558 ms 0.440 ms
it works.
The issue is I want to add the routing rules when my machine boots up.
I was able to reproduce the same problem with Docker.
--
Daniel Gray 0x41911F722B0F9AE3
https://mastodon.social/@dngray
2 years, 10 months
Thanks for blog post: How to use Podman inside of Kubernetes
by Craig Rodrigues
Hi,
Many thanks to Urvashi Mohnani and Dan Walsh for this blog post:
https://www.redhat.com/sysadmin/podman-inside-kubernetes
I worked at a recent gig, where I replaced a very fragile
"Docker-in-Docker" + /var/run/docker.setup setup with podman. This setup
was used to build about
50 different containers, inside containers running in a Kubernetes cluster.
The July blog post was very straightforward and easy to follow.
The resulting setup with podman was solid, and didn't have the fragility of
requiring
mounting /var/run/docker.sock inside containers.
Thanks!
--
Craig
2 years, 10 months
Can't maintain connection to container's listening port
by Ranbir
Hello,
I have a rootless container running postgrey on a Rocky Linux 8 server.
Besides the fact I had problems getting the container running rootless,
which I overcame, the new issue is that connections to the exposed port
are established and then immediately dropped. I can't figure out why
this is happening.
Here's postgrey listening inside the container:
[containers@bigsecret ~]$ podman exec -ti postgrey ss -tln
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.1:10023 0.0.0.0:*
I can connect to the port inside of the container and the connection
stays up until I cancel it:
[containers@bigsecret ~]$podman exec -ti postgrey telnet localhost 10023
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
If I try to telnet to the port from the container host using the host's
routable IP or from different server, I get a "Connection closed by
foreign host." message immediately after the connection is established.
I have systemd enabled in the container. I can control the postgrey
daemon with systemd and systemd doesn't report any errors when I check
the daemon's status.
I don't see any selinux denials. I tried turning off enforcement anyway
and saw no change. I did see language errors being logged by postgrey,
so I installed the missing RPMs in the running container (I'm just
testing things out with this container), which got rid of those errors.
But, that didn't change the connection weirdness.
Any ideas what the problem could be? The pod and container definitions
are below.
pod
{
"Id": "a9292128fc778c6287e80ff71d5e2ee1320b3395dc48a7e31af1db77cc7f695a",
"Name": "smtp",
"Created": "2021-11-25T12:58:55.447833371-05:00",
"CreateCommand": [
"podman",
"pod",
"create",
"--name",
"smtp",
"--publish",
"1.2.3.4:10023:10023",
"--publish",
"1.2.3.4:1587:587",
"--publish",
"1.2.3.4:1783:783",
"--publish",
"1.2.3.4:1025:25"
],
"State": "Running",
"Hostname": "smtp",
"CreateCgroup": true,
"CgroupParent": "user.slice",
"CgroupPath": "user.slice/user-libpod_pod_a9292128fc778c6287e80ff71d5e2ee1320b3395dc48a7e31af1db77cc7f695a.slice",
"CreateInfra": true,
"InfraContainerID": "a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4",
"InfraConfig": {
"PortBindings": {
"10023/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "10023"
}
],
"25/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1025"
}
],
"587/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1587"
}
],
"783/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1783"
}
]
},
"HostNetwork": false,
"StaticIP": "",
"StaticMAC": "",
"NoManageResolvConf": false,
"DNSServer": null,
"DNSSearch": null,
"DNSOption": null,
"NoManageHosts": false,
"HostAdd": null,
"Networks": null,
"NetworkOptions": null
},
"SharedNamespaces": [
"ipc",
"net",
"uts"
],
"NumContainers": 2,
"Containers": [
{
"Id": "a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4",
"Name": "a9292128fc77-infra",
"State": "running"
},
{
"Id": "f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57",
"Name": "postgrey",
"State": "running"
}
]
}
container
[
{
"Id": "f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57",
"Created": "2021-12-05T00:18:28.942285862-05:00",
"Path": "/usr/sbin/init",
"Args": [
"/usr/sbin/init"
],
"State": {
"OciVersion": "1.0.2-dev",
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 6047,
"ConmonPid": 6031,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-12-22T14:32:26.339653403-05:00",
"FinishedAt": "2021-12-22T14:27:28.403171029-05:00",
"Healthcheck": {
"Status": "",
"FailingStreak": 0,
"Log": null
}
},
"Image": "9aefd5346e1f34b16a096b52575cc249b14a9a56664c6e1f2113ad3ef449c025",
"ImageName": "localhost/postgrey-v0.0.3:latest",
"Rootfs": "",
"Pod": "a9292128fc778c6287e80ff71d5e2ee1320b3395dc48a7e31af1db77cc7f695a",
"ResolvConfPath": "/tmp/podman-run-1000/containers/overlay-containers/a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4/userdata/resolv.conf",
"HostnamePath": "/tmp/podman-run-1000/containers/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata/hostname",
"HostsPath": "/tmp/podman-run-1000/containers/overlay-containers/a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4/userdata/hosts",
"StaticDir": "/srv/containers/storage/1000/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata",
"OCIConfigPath": "/srv/containers/storage/1000/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata/config.json",
"OCIRuntime": "runc",
"ConmonPidFile": "/tmp/podman-run-1000/containers/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata/conmon.pid",
"PidFile": "/tmp/podman-run-1000/containers/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata/pidfile",
"Name": "postgrey",
"RestartCount": 0,
"Driver": "overlay",
"MountLabel": "system_u:object_r:container_file_t:s0:c654,c974",
"ProcessLabel": "system_u:system_r:container_init_t:s0:c654,c974",
"AppArmorProfile": "",
"EffectiveCaps": [
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_NET_RAW",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYS_CHROOT"
],
"BoundingCaps": [
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_NET_RAW",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYS_CHROOT"
],
"ExecIDs": [],
"GraphDriver": {
"Name": "overlay",
"Data": {
"LowerDir": "/srv/containers/storage/1000/overlay/2a356f237c2fe380f476133e0939553512ac8167ff7cbb2338d9405090528f7e/diff:/srv/containers/storage/1000/overlay/04f97fe38f3ca40a0d4a7ee7f6da4276ab30746e05c360975bd2e3569afde128/diff:/srv/containers/storage/1000/overlay/4d50441def2b07f8fcd48aad187815089621ddccf2384180db0c28c5272889f8/diff:/srv/containers/storage/1000/overlay/7933807b1a3f6ecbc852d38f269984065dfb57d49ddf40fdea70dfe66a6c6b14/diff:/srv/containers/storage/1000/overlay/1855256707116c0c229fec2d3a60bce4a11fdfc8b0bffa9663c84e69ec326160/diff",
"MergedDir": "/srv/containers/storage/1000/overlay/113bb9169c33b29659143e14363c6a8fc07a7cd6a8ffc72697337a83200db18e/merged",
"UpperDir": "/srv/containers/storage/1000/overlay/113bb9169c33b29659143e14363c6a8fc07a7cd6a8ffc72697337a83200db18e/diff",
"WorkDir": "/srv/containers/storage/1000/overlay/113bb9169c33b29659143e14363c6a8fc07a7cd6a8ffc72697337a83200db18e/work"
}
},
"Mounts": [
{
"Type": "volume",
"Name": "postgrey",
"Source": "/srv/containers/storage/1000/volumes/postgrey/_data",
"Destination": "/var/spool/postfix/postgrey",
"Driver": "local",
"Mode": "Z",
"Options": [
"nosuid",
"nodev",
"rbind"
],
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "5e82bc179344af8710114ca61f84dbfe7a8866c8aac5fab6bcef70e6cba6df76",
"Source": "/srv/containers/storage/1000/volumes/5e82bc179344af8710114ca61f84dbfe7a8866c8aac5fab6bcef70e6cba6df76/_data",
"Destination": "/sys/fs/cgroup",
"Driver": "local",
"Mode": "",
"Options": [
"nodev",
"exec",
"nosuid",
"rbind"
],
"RW": true,
"Propagation": "rprivate"
}
],
"Dependencies": [
"a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4"
],
"NetworkSettings": {
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"Bridge": "",
"SandboxID": "",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"10023/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "10023"
}
],
"25/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1025"
}
],
"587/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1587"
}
],
"783/tcp": [
{
"HostIp": "1.2.3.4",
"HostPort": "1783"
}
]
},
"SandboxKey": "/run/user/1000/netns/cni-a2c22e7a-f19f-8320-fe77-9d44a822154d"
},
"ExitCommand": [
"/usr/bin/podman",
"--root",
"/srv/containers/storage/1000",
"--runroot",
"/tmp/podman-run-1000/containers",
"--log-level",
"warning",
"--cgroup-manager",
"systemd",
"--tmpdir",
"/tmp/run-1000/libpod/tmp",
"--runtime",
"runc",
"--storage-driver",
"overlay",
"--storage-opt",
"overlay.mount_program=/usr/bin/fuse-overlayfs",
"--events-backend",
"file",
"container",
"cleanup",
"f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57"
],
"Namespace": "",
"IsInfra": false,
"Config": {
"Hostname": "smtp",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"TERM=xterm",
"container=docker",
"HOME=/root",
"HOSTNAME=smtp"
],
"Cmd": [
"/usr/sbin/init"
],
"Image": "localhost/postgrey-v0.0.3:latest",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": "",
"OnBuild": null,
"Labels": {
"io.buildah.version": "1.21.3",
"org.label-schema.build-date": "20210620",
"org.label-schema.license": "BSD-3-Clause",
"org.label-schema.name": "Rocky Linux Base Image",
"org.label-schema.schema-version": "1.0",
"org.label-schema.vendor": "Rocky Enterprise Software Foundation",
"org.opencontainers.image.created": "2021-06-20 00:00:00+01:00",
"org.opencontainers.image.licenses": "BSD-3-Clause",
"org.opencontainers.image.title": "Rocky Linux Base Image",
"org.opencontainers.image.vendor": "Rocky Enterprise Software Foundation"
},
"Annotations": {
"io.container.manager": "libpod",
"io.kubernetes.cri-o.ContainerType": "container",
"io.kubernetes.cri-o.Created": "2021-12-05T00:18:28.942285862-05:00",
"io.kubernetes.cri-o.SandboxID": "smtp",
"io.kubernetes.cri-o.TTY": "true",
"io.podman.annotations.autoremove": "FALSE",
"io.podman.annotations.init": "FALSE",
"io.podman.annotations.privileged": "FALSE",
"io.podman.annotations.publish-all": "FALSE",
"org.opencontainers.image.stopSignal": "37"
},
"StopSignal": 37,
"CreateCommand": [
"podman",
"run",
"-d",
"-t",
"--name",
"postgrey",
"--pod",
"smtp",
"--volume",
"postgrey:/var/spool/postfix/postgrey:Z",
"postgrey-v0.0.3"
],
"SystemdMode": true,
"Umask": "0022",
"Timeout": 0,
"StopTimeout": 10
},
"HostConfig": {
"Binds": [
"postgrey:/var/spool/postfix/postgrey:Z,rw,rprivate,nosuid,nodev,rbind",
"5e82bc179344af8710114ca61f84dbfe7a8866c8aac5fab6bcef70e6cba6df76:/sys/fs/cgroup:rprivate,rw,nodev,exec,nosuid,rbind"
],
"CgroupManager": "systemd",
"CgroupMode": "private",
"ContainerIDFile": "",
"LogConfig": {
"Type": "k8s-file",
"Config": null,
"Path": "/srv/containers/storage/1000/overlay-containers/f32c676da8eb38f3e45bb8670e0d8330707fa3dfc216238e4f73bbe638d85a57/userdata/ctr.log",
"Tag": "",
"Size": "0B"
},
"NetworkMode": "container:a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4",
"PortBindings": {},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": [],
"CapDrop": [
"CAP_AUDIT_WRITE",
"CAP_MKNOD"
],
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": [],
"GroupAdd": [],
"IpcMode": "container:a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4",
"Cgroup": "",
"Cgroups": "default",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "private",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [],
"Tmpfs": {},
"UTSMode": "container:a75ed34c8117daaff8be1e9060c07478b6894d4d06a93c963142d8b3de95b0a4",
"UsernsMode": "",
"ShmSize": 65536000,
"Runtime": "oci",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "user.slice/user-libpod_pod_a9292128fc778c6287e80ff71d5e2ee1320b3395dc48a7e31af1db77cc7f695a.slice",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": 0,
"OomKillDisable": false,
"PidsLimit": 0,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"CgroupConf": null
}
}
]
--
Ranbir
2 years, 10 months
