Macvlan
by Rusty Sapper
I’m having trouble setting up a macvlan. I created my config under
/etc/cni/net.d and can successfully start my continater with the macvlan
network.
However, when I exec into the container, it has no network access to
anything except itself. I apologize, I'm new to podman. I need to be able
to ssh into other hosts(jenkins nodes) to run tests. I am unable to use
the host network due to other requirements.
Any help would be appreciated. here is my macvlan config:
{
"cniVersion": "0.4.0",
"name": "testmac1",
"plugins": [
{
"type": "macvlan",
"master": "eno1np0",
"ipam": {
"type": "host-local",
"routes": [
{
"dst": "0.0.0.0/0"
}
],
"ranges": [
[
{
"subnet": "10.89.0.0/24",
"gateway": "10.89.0.1"
}
]
]
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
~
2 hours, 50 minutes
OCI permission denied
by lejeczek
Hi guys.
Here are errors from my unsuccessful attempt to create a
rootless container, which to novices such as myself are
quite cryptic:
-> $ podman run -d --restart=always
--pod=jat-${HOSTNAME%%.*} --security-opt label=disable
--volume /srv/containers/podmania/jat-redis:/data --name
redis docker.io/library/redis
ERRO[0000] error starting some container dependencies
ERRO[0000] "container_linux.go:370: starting container
process caused: process_linux.go:459: container init caused:
rootfs_linux.go:59: mounting \"sysfs\" to rootfs at \"/sys\"
caused: operation not permitted: OCI permission denied"
Error: error starting some containers: internal libpod error
I'm trying to drop the container into a pod which is
rootless too. The error persist also with/in 'Permissive'
selinux.
Somebody could help decrypt & troubleshoot this?
I'm on Centos Steam with:
podman-3.0.0-0.33rc2.module_el8.4.0+673+eabfc99d.x86_64
many thanks, L.
2 days, 10 hours
Re: OCI permission denied
by Scott McCarty
We believe it's a kernel bug. We're working on fixing it asap. In fact,
we're also working on figuring out how it got through CI/CD.
Could you try running it as root to verify it works?
3 days, 10 hours
all rootless - container in a pod - stderr=\"Link not found\\n\""
by lejeczek
Hi guys.
I'm trying to set a rootless container in a pod but that
fails with:
-> $ podman run -d --restart always --pod
jat-${HOSTNAME%%.*} --name redis docker.io/bitnami/redis
ERRO[0059] error starting some container dependencies
ERRO[0059] "command rootless-cni-infra [alloc
6a1c6d6201bd72a083a0882a1e2cfb52e6b06251d98f4895c352921f5d781ba5
host_local jatymy-c8kubernode3 10.1.1.245 ] in container
6ad3d7cac619f5c98cb2f09d6e5bd0396a77958e7e8f61e69eeab569fbfe34ab
failed with status 1, stdout=\"\", stderr=\"Link not
found\\n\""
Error: error starting some containers: internal libpod error
-> $ cat /etc/cni/net.d/host_local.conflist
{
"cniVersion": "0.4.0",
"name": "host_local",
"plugins": [
{
"type": "macvlan",
"master": "ens10",
"ipam": {
"type": "host-local",
"ranges": [
[
{
"subnet": "10.1.1.0/24",
"rangeStart": "10.1.1.2",
"rangeEnd": "10.1.1.254",
"gateway": "10.1.1.254"
}
]
],
"routes": [
{"dst": "0.0.0.0/0"}
]
}
},
{
"type": "tuning",
"capabilities": {
"mac": true
}
}
]
}
There are no other pods nor containers, neither rootfull or
rootless, in the system which is Centos Stream with
podman-3.0.0-0.33rc2.module_el8.4.0+673+eabfc99d.x86_64
Any suggestions on what the problem might be are greatly
appreciated.
many thanks, L.
3 days, 10 hours
Pushing image tagged with multiple tags to remote repo with the same digest
by Marcin Zajączkowski
Hi. I might miss something obvious - I don't have deep knowledge about
containers, however, trying to migrate my docker-in-docker image
building CI job to buildah, I've encountered the following problem.
My image should be tagged with its real version and in addition with a
special marker tag (such as "devel" or "stable" or "latest"). It seems
to be a common practice with images. While in docker, I could just call:
> docker build -t "$CI_REGISTRY_IMAGE:$VERSION" -t "$CI_REGISTRY_IMAGE:$MARKER_TAG"
> docker push $CI_REGISTRY_IMAGE"
with buildah, it is more problematic as I cannot use multiple tags on
push (--all-tags switch has been also abandoned).
I tried:
> buildah bud -t "$CI_REGISTRY_IMAGE:$VERSION" -t "$CI_REGISTRY_IMAGE:$MARKET_TAG"
> buildah push "$CI_REGISTRY_IMAGE:$VERSION" "$CI_REGISTRY_IMAGE:$VERSION"
> buildah push "$CI_REGISTRY_IMAGE:$MARKET_TAG" "$CI_REGISTRY_IMAGE:$MARKET_TAG"
but while "buildah images --digests" shows the same digest for those two
tags, in the remote repository (GitLab CI container repository) these
tags have two different digests (also both differ from the local one). I
suspect there could be something changed in the manifest.
Q. How can I have one image (with one digests) tagged multiple times in
a container repository using buildah?
buildah version 1.18.0 (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev) on
Fedora 33.
Marcin
P.S. I've seen https://github.com/containers/buildah/issues/2034, but I
don't know if it is related.
4 days
OCI runtime error when starting hub.docker.com/r/rocker images as non-root - how to troubleshoot?
by boardbill_unpretended＠simplelogin.co
Dear Podman list,
I am just starting to dip my toes into running containers in Podman 2.2.1 on RHEL 8.3 (Linux kernel 4.18) and would appreciate guidance on how to diagnose and solve an error I get when running as non-root.
My goal is to do R language development in containers, and I used `podman pull` to get these R images from the popular Rocker project (https://www.rocker-project.org/):
https://hub.docker.com/r/rocker/r-base
https://hub.docker.com/r/rocker/tidyverse
https://hub.docker.com/r/rocker/rstudio
The `r-base` image is just the R interpreter and it starts fine where I can just attach it to a terminal to run R commands.
However, the other images (`tidyverse` and `rstudio`) are built on top of `r-base` and when I run them as non-root (e.g. `podman run -ti rocker/tidyverse`) they give this error:
```
Error: OCI runtime error: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr:
```
I have no idea how to interpret and understand this error message. The only clue I have is that these two images run without errors if I append `sudo` to my `podman run` command. I also checked the Dockerfiles (such as this one: https://github.com/rocker-org/rocker-versioned/blob/master/tidyverse/3.6....) and it's not clear what might be giving the error.
Oh, and the `rocker/rstudio` image allows you to specify ports i.e. `-p 8787:8787` but even if I leave out this argument, I still get the error above when running `podman run` on this image. So I don't think it's an issue with opening or mapping ports (and Podman allows this as non-root, right?).
How should I begin to diagnose the root of this problem and start fixing it?
To be clear, my goal is to run the images listed above ideally as a non-root user. Thank you!
4 days, 6 hours
Permissions on top level of mounted volume in rootless container
by Peter Upfold
Hello,
I'm struggling a little with the permissions set on the top level
directory of a volume that is mounted in a rootless container.
My Containerfile:
https://gist.github.com/PeterUpfold/2f63ad5341ffd9079bc2683a5bb2744c
The top level directory of the volume mount,
/var/www/html/websites/windows, ends up with root:nobody and 0755
permissions inside the container.
I've seen similar issues on this list: Daniel Walsh's suggestion of
`--annotation run.oci.keep_original_groups=1` seems to work beautifully
to change the ownership of the volume folder in the container to be
windowsnoob:windowsnoob, as I would want it, _if_ I'm doing `podman run`.
However, I'm trying to create a pod as follows. Is it possible to have
this permissions configuration work in this scenario?
podman pod create -n windowsnoob -p 8081
podman build -t windowsnoob-fpm .
podman create --name windowsnoob-fpm --pod windowsnoob -v
/var/www/html/websites/windows:/var/www/html/websites/windows:Z,noexec,nodev,rw
windowsnoob-fpm
podman pod start windowsnoob
At the moment, doing this and checking the permissions on the
/var/www/html/websites/windows volume in the created container (via
`podman exec -it [container] bash`) still shows the following:
drwxr-xr-x. 2 root nobody 28 Feb 14 09:45 windows
(Note that I can write to a subfolder already owned by
windowsnoob:windowsnoob _inside_ the volume just fine — I don't believe
this is an SELinux issue, or a permissions issue on anywhere except the
top level of the volume mount!)
Thank you for any insight you might be able to provide!
Peter Upfold
1 week
Podman Community Meeting Agenda - Tuesday March 2, 2021 11:00 a.m. Eastern (UTC-5)
by Tom Sweeney
Hi All,
I've just posted the Agenda for the next Podman Community Meeting which
is coming up in a little less than two weeks on Tuesday March 6, 2021 at
11:00 a.m. Eastern (UTC-5). The agenda is on the podman.io site at:
https://podman.io/community/meeting/agenda/. We've a number of demos
slated and the agenda is packed full! Also on the page I've a link to
the WorldTimeBuddy site at the top. It is a very handy website that
makes translating our meeting time to you locale uber easy. Just add
your city name to the top left text box, then move the slider to 11:00
a.m. eastern to see what time it will be in your part of the world.
If you can't make it, we will be recording the meeting and will post it
to the podman.io site a few days later.
Hope to see a bunch of you there!
t
1 week
flooded with - Couldn't stat device /dev/char/10:200: No such file or directory
by lejeczek
Hi guys.
I'd like to ask around about some error messages journal
gets full of, namely:
...
SELinux: mount invalid. Same superblock, different security
settings for (dev mqueue, type mqueue)
Couldn't stat device /dev/char/10:200: No such file or directory
Couldn't stat device /dev/char/10:200: No such file or directory
Started libcontainer container
391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.
SELinux: mount invalid. Same superblock, different security
settings for (dev mqueue, type mqueue)
Couldn't stat device /dev/char/10:200: No such file or directory
libpod-238a14cc41bdb2826850c00907c249e43b0b3333c0a344f99920adddff5c38e3.scope:
Succeeded.
libpod-238a14cc41bdb2826850c00907c249e43b0b3333c0a344f99920adddff5c38e3.scope:
Consumed 168ms CPU time
libpod-391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.scope:
Succeeded.
libpod-391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.scope:
Consumed 150ms CPU time
Couldn't stat device /dev/char/10:200: No such file or directory
Started libcontainer container
391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.
SELinux: mount invalid. Same superblock, different security
settings for (dev mqueue, type mqueue)
Couldn't stat device /dev/char/10:200: No such file or directory
Couldn't stat device /dev/char/10:200: No such file or directory
Started libcontainer container
238a14cc41bdb2826850c00907c249e43b0b3333c0a344f99920adddff5c38e3.
SELinux: mount invalid. Same superblock, different security
settings for (dev mqueue, type mqueue)
Couldn't stat device /dev/char/10:200: No such file or directory
libpod-391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.scope:
Succeeded.
libpod-391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.scope:
Consumed 153ms CPU time
Couldn't stat device /dev/char/10:200: No such file or directory
Started libcontainer container
391b1013c06ea5abe461d9474ec3b8f2c8e902e9d4b0e0cbf5ea8b8b0394541f.
SELinux: mount invalid. Same superblock, different security
settings for (dev mqueue, type mqueue)
Couldn't stat device /dev/char/10:200: No such file or directory
libpod-238a14cc41bdb2826850c00907c249e43b0b3333c0a344f99920adddff5c38e3.scope:
Succeeded.
libpod-238a14cc41bdb2826850c00907c249e43b0b3333c0a344f99920adddff5c38e3.scope:
Consumed 160ms CPU time
...
Two questions really.
a) how critical those are?
b) how to fix the problem? Rather obvious one.
many thanks, L
2 weeks, 1 day
DIY networking for rootless containers/pods
by Rudolf Vesely
Hi Podman Developers and Users,
Thank you very much for Podman and related tools. It's a fantastic project.
I'm trying to convert my current container host VPS into a number of rootless pods and I'm thinking about the pods networking. Some pods will need to be able to communicate with each other (for example HAProxy has to be able to connect both WordPress and Nextcloud) and some don't (WordPress and Nextcloud don't need to talk to each other). From security least privilege principle: pods that don't need to communicate shouldn't be allowed to.
The obvious solution is to use default settings slirp4netns and listen (publish port) on 127.0.0.1 or maybe on a dedicated private IP created by "ip link add name something type dummy". That means that for example WordPress will listen on 8080 and Nextcloud on 8081 (more info in Brent Baude's article https://www.redhat.com/sysadmin/container-networking-podman).
As Dan Walsh often mentions in his Podman presentations one of the best things about Podman is that it's not just one tool - it's Podman/libpod, Buildah, Skopeo, CRI-O, RunC and they all do one thing and do it well which enables me to try some DIY networking.
DIY:
==========================================================
### create bridge using "ip"
$ sudo ip link add name bridge1 type bridge
$ sudo ip link set dev bridge1 up
$ sudo ip address add 10.11.22.1/24 dev bridge1
### or by "systemd-networkd"
$ sudo systemctl --now enable systemd-networkd
$ cat << EOF | sudo tee /etc/systemd/network/bridge1.netdev
[NetDev]
Name=bridge1
Kind=bridge
EOF
$ cat << EOF | sudo tee /etc/systemd/network/bridge1.network
[Match]
Name=bridge1
[Network]
Address=10.11.22.1/24
EOF
### run rootless container
$ sudo mkdir /test-www
$ echo "Hello, World!" | sudo tee /test-www/index.html
$ cont_id=$(podman run --net=none -d --volume=/test-www:/usr/share/nginx/html docker://docker.io/library/nginx:latest)
$ [[ ${cont_id} =~ ^[0-9a-z]{64}$ ]] &&
printf '%s\n' "OK: \"${cont_id}\""
> OK: "5811ac2e25dec942fd22c2e83657d103bbce199aa7775d7f4d10bf5c53af4778"
$ net_ns_name="cont-${cont_id}"
$ cont_pc_id=$(podman inspect -f '{{.State.Pid}}' "${cont_id}")
$ [[ ! -d /var/run/netns ]] &&
sudo mkdir -v /var/run/netns
$ sudo ln -sfTv "/proc/${cont_pc_id}/ns/net" "/var/run/netns/${net_ns_name}"
> '/var/run/netns/cont-5811ac2e25dec942fd22c2e83657d103bbce199aa7775d7f4d10bf5c53af4778' -> '/proc/1217/ns/net'
$ ip netns list
> cont-5811ac2e25dec942fd22c2e83657d103bbce199aa7775d7f4d10bf5c53af4778
$ sudo ip link add veth300 type veth peer name veth300p
$ sudo ip link set dev veth300 master bridge1
$ sudo ip link set veth300p netns "${net_ns_name}"
$ sudo ip -netns "${net_ns_name}" link set veth300p name eth0 # optional: rename peer in namespace
$ sudo ip link set dev veth300 up
$ sudo ip -netns "${net_ns_name}" link set dev eth0 up
$ sudo ip -netns "${net_ns_name}" address add 10.11.22.50/24 dev eth0
$ sudo ip -netns "${net_ns_name}" route add default via 10.11.22.1
### to make it work, the host has to have routing enabled
$ sudo sysctl -w net.ipv4.ip_forward=1
### and iptables/nftables configured
$ sudo nft add table ip nat
$ sudo nft add chain ip nat nat-prerouting "{ type nat hook prerouting priority -100; policy accept; }"
$ sudo nft add chain ip nat nat-postrouting "{ type nat hook postrouting priority 100; policy accept; }"
$ sudo nft add rule ip nat nat-prerouting iifname "eth0" tcp dport { 80, 8080, 8081 } counter dnat 10.11.22.50
$ sudo nft add rule ip nat nat-postrouting oifname "eth0" counter masquerade
### and to test that the container can go out
$ podman exec -it "${cont_id}" curl https://1.1.1.1/
> <a lot of html>
### and to access the container (the web server)
$ curl http://<container host public IP>/
> Hello, World!
==========================================================
For those that don't want to read the code:
1. create bridge
2. run container without slirp4netns (--net=none) => that means it has only localhost
3. create a network namespace for the container process
4. create virtual ethernet pair (VETH), move one interface into the new bridge and the second into the new network namespace
5. make it work by assigning IP addresses, default route in the new namespace, enabling routing on the host and NAT on the host firewall
Note: At this moment this is not possible for pods since pods in the current stable version of Podman don't support --net=none. But that will change in 3.0: https://github.com/containers/podman/issues/9165, https://github.com/mheon/libpod/commit/6bd3a6bcabda682243f531bacf3659b95d..., https://github.com/containers/podman/releases/tag/v3.0.0-rc3.
Thank you Matthew Heon!
The benefits I get by doing this:
1. Rootless containers, no need to run rootfull for this.
2. Easy to firewall - for example interfaces in one bridge can connect interfaces in another bridge but not in the opposite way
3. Easy to understand and visualize
4. Can be integrated with VLANs, Open vSwitch VXLANs and anything that uses bridges (QEMU VMs...)
Could you please tell me is this a good idea?
Thank you.
Kind regards,
Rudolf Vesely
2 weeks, 2 days
