5:23 p.m.
On Wed, Nov 11, 2020 at 10:27:49AM +0100, Alexander Wellbrock via Podman wrote:
Hey there, since buildx is not a thing yet to build multi-arch images
on one host I'd like to make use of the multi-arch feature of registries like
docker.io and quay.io.
I'm using the podman manifest command for this purpose but struggle to get it to work
properly. I hope someone can point me in the right direction.
I've get it to work (kind-of) by building two images on two different hosts
architecture wise and then used podman manifest create to create a multi-arch manifest of
both images. I then used podman manifest push to get it onto quay.io. This works and
I'm able to use it, but unfortunately there is a catch right now.
If I'm doing this multiple times, it complains that since a manifest for
example-image:latest is already present. So it only work if the manifest does not already
exist. I can't update it with newer images. Only if I remove the manifest :latest
manually beforehand it obviously works again, but I'd like to avoid that. Apart from
that it makes sense I suppose, because I'd rather like to update the manifest with new
images then to override the old manifest, right?
I then tried to use podman manifest add, remove etc. Podman then complains about manifest
version not supported. E.g. I tried to update the actual images per architecture and then
add the new ones to the manifest which I couldn't get to work. I'm highly confused
about the podman manifest command man page.
Someone asked a similar question on IRC a few days ago, and I didn't
have a good answer then, so had to poke at it a bit.
Anyway, the sequence of steps for creating a list using images that have
previously been pushed to a registry looks something like this:
on a box of each $arch: podman build -t docker://registry/repo:$arch-tag
on one box: podman manifest create $listname
repeat for each $arch on that one box: podman manifest add $listname
docker://registry/repo:$arch-tag
finally, on that one box: podman manifest push [--all] $listname
docker://registry/repo:$single-tag
Now, if you build a new image and want to replace it in the list, you
should be able to start with the list you already have, or you can do
this to start to build a new, updated version of the list:
podman manifest create updated-list
podman manifest add --all updated-list docker://registry/repo:$single-tag
Note: "podman manifest add" (and "buildah manifest add") looks like
it
just looks at a locally-stored image if it finds one with the same name
as the reference being passed on the command line. That locally-stored
image isn't going to include all of the other entries in the copy of the
list that the registry has, and that can cause some problems. I think
we'll need to fix that.
If it works as intended (or you started with the list you already had),
you should have a copy of the list that's up at the registry, and
"podman manifest inspect" should show you several entries in that list.
The spec doesn't say anything about what to do when there are multiple
entries in a list for the same platform, so we'll want to remove the
entry that the list has for the current machine before we try to add our
newly-built image to the list as its replacement.
arch=$(go env GOARCH)
instance=$(podman manifest inspect updated-list | jq -r \
'.manifests|map(select(.platform.architecture=="'$arch'"))[].digest')
podman manifest remove updated-list $instance
Now you can go ahead and add your new image to the list and push the
updated list to the registry.
If the new image has already been pushed to the registry:
podman manifest add updated-list docker://registry/repo:$arch-tag
podman manifest push updated-list docker://registry/repo:$single-tag
If not:
podman manifest add updated-list local-$arch-image
podman manifest push --all updated-list docker://registry/repo:$single-tag
The main difference between these two cases is that the --all flag tells
podman to push the images that the list references, in addition to the
list itself. That can take a while, since for every image, even though
the pushing logic checks if a part of an image has already been uploaded
to the registry before trying to upload it, it's still a lot of parts,
and that can require a lot of round trips to the registry. But if the
image wasn't pushed first, the registry will probably reject the new
list because it references things that the registry doesn't have, so
it's required.
I've got a couple of ideas of things we can do to help simplify this
process, so that it requires less typing at the CLI, but none of that's
implemented yet.
If you run into trouble, please follow up with the specific errors
you're getting, and we'll see where they point us.
Cheers,
Nalin