Merging published container manifests (multi-arch)

[unkempt igloo]

Is there a dagger native way to merge manifests of published images? This is mainly for publishing multi arch images. Emulation is super slow so we build on specific arch machines and push separately and use docker buildx imagetools create to merge the manifests.

Dagger publish automatically handles this when I pass in multiple architectures but that also requires me to run on a single machine. Is there a better way to handle this with dagger? If I publish individually, it just overwrites that's already there.

[dusk hare]

I personally don't know of one but I'm wondering if you could split this in 3 steps.

1 & 2 run un parallel and build + push each corresponding arch image.

3 runs after 1 & 2, pulls each respective image with `container.From("image") and re-publish in a single operation

[unkempt igloo]

oh interesting, I can try that. Thank you @dusk hare ! The only downside is, you are pulling down an image that you already pushed, where as, in the buildx method, it handles the merge remotely. But the engine I use should technically still have most of the artifacts cached so it could be a nonfactor.

[dusk hare]

ofc merging the manifests is way more efficient but there's no Dagger op for that currently

[dusk hare]

well. I assume in your case it's separate engines right? One for each arch?

[unkempt igloo]

yes one for each. But the merging will happen in either one of them. There's no 3rd engine, if that makes sense

[dusk hare]

yep, totally

I think it's worth trying for sure

if the images are not big it should be ok

[unkempt igloo]

Thank you for the quick response! I will try it and get back to you. Worst case, we can save the state (manifest names) and directly use docker buildx within dagger.

[dusk hare]

I guess the downside of docker buildx is the fact that it requires Docker isn't it?

[unkempt igloo]

Right, it's easy enough to get an image with docker CLI. I believe creds are passed through via dagger

[dusk hare]

yeah but you still need an docker engine running somewhere.. if you already have that then it's no big deal just needing a dockerservice just for this seems 😬

[unkempt igloo]

Right. I'd probably need docker:dind and then install the CLI in it.

[dusk hare]

maybe umoci (https://github.com/opencontainers/umoci) allows the same without the need of the Docker engine 🤔

AFAIK it's a pretty active project

[unkempt igloo]

From what I'm reading, umoci alone isn't enough. I'd probably need skopeo too. However, it's probably easier to use docker manifest

[dusk hare]

hmm seems like oras has everything https://oras.land/docs/commands/oras_manifest_index_create

looks quite straighforward to use

[unkempt igloo]

nice! that seems like exactly what I need let me try it out

[storm vine]

That's not true actually, dagger should work exactly how you want in this scenario.

If all you do is two container.froms and then combine them in a multiplatform publish, dagger will never pull the images and instead just push a multiplatform manifest

(this requires obviously that the source of the images and where you are publishing to are all the same registry)

I had to double check this still worked since it originates from work many years ago now, but yeah it still works like that today:

sipsma@dagger_dev:~/repo/github.com/sipsma/dagger$ ./hack/with-dev dagger -M
Dagger interactive shell. Type ".help" for more information. Press Ctrl+D to exit.

● ✔ a=$(container | from docker.io/eriksipsma/dagger-test:alpineamd64@sha256:dd3ca5ce8c43485b954341e3fcb895809eaf79da34cddec6f52363f37b6c6d69) 0.0s

● ✔ b=$(container | from docker.io/eriksipsma/dagger-test:alpinearm64@sha256:06b8983e0909519c7d7235417a6c04b3273dd2e652f733bd48248e4e47dc66ca) 0.0s

▶ ✔ container | publish eriksipsma/dagger-test:alpine --platform-variants $a,$b 5.8s
docker.io/eriksipsma/dagger-test:alpine@sha256:789a65cbd70d604ef76791117c6aa7801601bc341aeeb45165ea0cef8d9eec4c   

• after running the above I manually checked the engine's snapshot/content dirs and confirmed no layers were pulled

(cc @dusk hare, fyi)