12:16 p.m.
We have a cluster of machines where /home is a remote gluster mount. Running podman
rootless nicely solves the problem of accessing the remote filesystem with user
credentials. Since remote filesystems do not currently support namespaces, podman is run
with --root, --runroot, and --tmpdir set to be /tmp/$USER. All works well on the first
client machine, but an image pulled successfully on one machine will fail to pull on a
second. For example, on the second machine:
$ podman run --rm -it ubuntu
Trying to pull docker.io/library/ubuntu...Getting image source signatures
Copying blob c58094023a2e done
Copying blob 079b6d2a1e53 done
Copying blob 11048ebae908 done
Copying blob 22e816666fd6 done
Copying config cf0f3ca922 done
Writing manifest to image destination
Storing signatures
ERRO[0168] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: lchown
/etc/gshadow: operation not permitted
ERRO[0200] Error pulling image ref //ubuntu:latest: Error committing the finished image:
error adding layer with blob
"sha256:22e816666fd6516bccd19765947232debc14a5baf2418b2202fd67b3807b6b91":
ApplyLayer exit status 1 stdout: stderr: lchown /etc/gshadow: operation not permitted
Failed
Trying to pull registry.fedoraproject.org/ubuntu...ERRO[0200] Error pulling image ref
//registry.fedoraproject.org/ubuntu:latest: Error initializing source
docker://registry.fedoraproject.org/ubuntu:latest: Error reading manifest latest in
registry.fedoraproject.org/ubuntu: manifest unknown: manifest unknown
Failed
Trying to pull quay.io/ubuntu...ERRO[0201] Error pulling image ref
//quay.io/ubuntu:latest: Error initializing source docker://quay.io/ubuntu:latest: Error
reading manifest latest in quay.io/ubuntu: error parsing HTTP 404 response body: invalid
character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC
\"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not
Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not
found on the server. If you entered the URL manually please check your spelling and try
again.</p>\n"
Failed
Trying to pull registry.centos.org/ubuntu...ERRO[0201] Error pulling image ref
//registry.centos.org/ubuntu:latest: Error initializing source
docker://registry.centos.org/ubuntu:latest: Error reading manifest latest in
registry.centos.org/ubuntu: manifest unknown: manifest unknown
Failed
Error: unable to pull ubuntu: 4 errors occurred:
* Error committing the finished image: error adding layer with blob
"sha256:22e816666fd6516bccd19765947232debc14a5baf2418b2202fd67b3807b6b91":
ApplyLayer exit status 1 stdout: stderr: lchown /etc/gshadow: operation not permitted
* Error initializing source docker://registry.fedoraproject.org/ubuntu:latest: Error
reading manifest latest in registry.fedoraproject.org/ubuntu: manifest unknown: manifest
unknown
* Error initializing source docker://quay.io/ubuntu:latest: Error reading manifest latest
in quay.io/ubuntu: error parsing HTTP 404 response body: invalid character '<'
looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2
Final//EN\">\n<title>404 Not Found</title>\n<h1>Not
Found</h1>\n<p>The requested URL was not found on the server. If you entered
the URL manually please check your spelling and try again.</p>\n"
* Error initializing source docker://registry.centos.org/ubuntu:latest: Error reading
manifest latest in registry.centos.org/ubuntu: manifest unknown: manifest unknown
Our guess is that this is happening because blob-info-cache-v1.boltdb is in the shared
/home filesystem.
Is there a suggested approach to running rootless podman on multiple machines with a
shared /home directory?
Thanks,
Eddie