[Podman] Re: Multi-arch with podman manifests

Laurent Meunier <laurent(a)deltalima.net&gt; hat am 17.11.2020 06:56 geschrieben: On 16/11/2020 23:23, Nalin Dahyabhai wrote: > On Wed, Nov 11, 2020 at 10:27:49AM +0100, Alexander Wellbrock via Podman wrote: >> Hey there, since buildx is not a thing yet to build multi-arch images on one host I'd like to make use of the multi-arch feature of registries like docker.io and quay.io. >> >> I'm using the podman manifest command for this purpose but struggle to get it to work properly. I hope someone can point me in the right direction. >> >> I've get it to work (kind-of) by building two images on two different hosts architecture wise and then used podman manifest create to create a multi-arch manifest of both images. I then used podman manifest push to get it onto quay.io. This works and I'm able to use it, but unfortunately there is a catch right now. >> >> If I'm doing this multiple times, it complains that since a manifest for example-image:latest is already present. So it only work if the manifest does not already exist. I can't update it with newer images. Only if I remove the manifest :latest manually beforehand it obviously works again, but I'd like to avoid that. Apart from that it makes sense I suppose, because I'd rather like to update the manifest with new images then to override the old manifest, right? >> >> I then tried to use podman manifest add, remove etc. Podman then complains about manifest version not supported. E.g. I tried to update the actual images per architecture and then add the new ones to the manifest which I couldn't get to work. I'm highly confused about the podman manifest command man page. > > Someone asked a similar question on IRC a few days ago, and I didn't > have a good answer then, so had to poke at it a bit. > > Anyway, the sequence of steps for creating a list using images that have > previously been pushed to a registry looks something like this: > > on a box of each $arch: podman build -t docker://registry/repo:$arch-tag > on one box: podman manifest create $listname > repeat for each $arch on that one box: podman manifest add $listname docker://registry/repo:$arch-tag > finally, on that one box: podman manifest push [--all] $listname docker://registry/repo:$single-tag > > Now, if you build a new image and want to replace it in the list, you > should be able to start with the list you already have, or you can do > this to start to build a new, updated version of the list: > > podman manifest create updated-list > podman manifest add --all updated-list docker://registry/repo:$single-tag > > Note: "podman manifest add" (and "buildah manifest add") looks like it > just looks at a locally-stored image if it finds one with the same name > as the reference being passed on the command line. That locally-stored > image isn't going to include all of the other entries in the copy of the > list that the registry has, and that can cause some problems. I think > we'll need to fix that. > > If it works as intended (or you started with the list you already had), > you should have a copy of the list that's up at the registry, and > "podman manifest inspect" should show you several entries in that list. > > The spec doesn't say anything about what to do when there are multiple > entries in a list for the same platform, so we'll want to remove the > entry that the list has for the current machine before we try to add our > newly-built image to the list as its replacement. > > arch=$(go env GOARCH) > instance=$(podman manifest inspect updated-list | jq -r \ > '.manifests|map(select(.platform.architecture=="'$arch'"))[].digest') > podman manifest remove updated-list $instance > > Now you can go ahead and add your new image to the list and push the > updated list to the registry. > > If the new image has already been pushed to the registry: > > podman manifest add updated-list docker://registry/repo:$arch-tag > podman manifest push updated-list docker://registry/repo:$single-tag > > If not: > > podman manifest add updated-list local-$arch-image > podman manifest push --all updated-list docker://registry/repo:$single-tag > > The main difference between these two cases is that the --all flag tells > podman to push the images that the list references, in addition to the > list itself. That can take a while, since for every image, even though > the pushing logic checks if a part of an image has already been uploaded > to the registry before trying to upload it, it's still a lot of parts, > and that can require a lot of round trips to the registry. But if the > image wasn't pushed first, the registry will probably reject the new > list because it references things that the registry doesn't have, so > it's required. > > I've got a couple of ideas of things we can do to help simplify this > process, so that it requires less typing at the CLI, but none of that's > implemented yet. > > If you run into trouble, please follow up with the specific errors > you're getting, and we'll see where they point us. > > Cheers, > > Nalin Hello, I'm also building multi-arch images with buildah and podman, and I just would like to add a few remarks. I use "buildah manifest" instead of "podman manifest". Is there some behavior differences between these two commands? If you build your image on an ARM architecture, you may need to add the --variant option to the "podman manifest add" command: # example for ARMv7 (like the Raspberry Pi 3) buildah manifest add --variant 7 image_name:$single-tag docker.io/$USERNAME/image_name:$arch-tag When pushing the manifest to Docker Hub, I had to add the "--format v2s2" to my "buildah manifest push" command. It seems that Docker Hub does not support the OCI format and I need to use the v2s2 format (not 100% sure of this): buildah manifest push --all --format v2s2 image_name:$single-tag docker://docker.io/$USERNAME/image_name:$single-tag I globally use the same steps as Alexander to push multi-arch images to a registry: - build the image on each architecture (amd64 and armv7 in my case) - push each image on docker hub with an architecture specific tag - create and push a manifest as described above Here is a link to the detailed steps I use to push multi-arch images to Docker Hub: https://github.com/lmeunier/oci-image-ejabberd#login-to-the-docker-hub-re... Best regards, Laurent _______________________________________________ Podman mailing list -- podman(a)lists.podman.io To unsubscribe send an email to podman-leave(a)lists.podman.io