Avoid re-pulling images | Dagger | Page 1

half carbon Jun 13, 2023, 10:59 AM

#

Hi! First time dagger user here.
I was wondering, is there away to have dagger re-use an already pulled image?

Thanks!

crimson rampart Jun 13, 2023, 1:53 PM

#

👋 no, it's not possible since the Dagger engine and the Docker engine do not share their metadata. That's why the image needs to be pulled withing the Dagger engine once you run your pipelines

half carbon Jun 13, 2023, 2:26 PM

#

thanks @crimson rampart , what's the lifecycle of the image? the context of the client connection? If I have the engine container running, won't it persist the image once it's pulled?

crimson rampart Jun 13, 2023, 4:01 PM

#

no, the image is being stored in the engine cache. The engine will always persist the image as long as you're using the same Dagger SDK version

#

since we generally prune the cache for each new dagger release .

half carbon Jun 13, 2023, 4:35 PM

#

So I'm seeing something that doesn't add up (unless I just don't understand).
I have a pipeline, using the Go SDK, that involved just 1 container.
Each time I run my go binary, the image gets re-pulled, the dagger engine container is up for 4 hours

#

So either it's not beigng cached, or I'm doing something wrong

crimson rampart Jun 13, 2023, 4:39 PM

#

@half carbon can you please share an example on how you're creating your pipeline?

sinful bolt Jun 13, 2023, 4:50 PM

#

An image will not be pulled again if the manifest for the tag doesn't change.

It may look like a pull since by default it checks that nothing changed for that image tag. If you were to use the sha256 digest instead of the image tag, Dagger will not verify the tag manifest.

I re-ran a cached pipeline which uses sha256 digests instead of the image tag: https://github.com/dagger/dagger/issues/5135#issuecomment-1560559569

#

When we address the issue that I linked to, Dagger should not pull images for tags that have already been pulled.

crimson rampart Jun 14, 2023, 3:35 PM

#

@half carbon do you still have issues with this?

half carbon Jun 14, 2023, 3:48 PM

#

hi guys, thanks, I've been down the rabbit hole 🙂

#

As far as I understand this is expected since I'm using the image tags and not the digest?

#

@crimson rampart I watched your video to day on the Knative approach with Dagger, can I hijack this thread for a followup question?

crimson rampart Jun 14, 2023, 4:28 PM

#

@half carbon might be better if you open a new help thread so it's easier for users to find it and we can close this one 🙏

#Avoid re-pulling images