May `dockerBuild` caching work differently from native Dagger APIs caching?

[cedar sun]

I'll copy-paste my message from #docker here.

---

Maybe that's a weird question, but does dockerBuild cache intermediate instructions results the same way as if they would have been executed via the Dagger API? This was my assumption.

I am facing this issue (have been facing from the very beginning with Dagger): I am keeping most of my base build steps in Dockerfile and hand it to Dagger to perform some fine-grained dynamic steps after the main bulk has been finished.

And apparently it hasn't been cached for this entire time (like 8 moths lol).

I am experiencing full rebuilds of the dockerBuild step quite frequently (even when doing innocent changes to other projects in my monorepo). I think it is cached across dagger call invocations within the same GitHub Actions run, but not across runs (commits).

@solomon could you helm me out here, please? I wasn't able to find any docs on this.

---

related issue? https://github.com/dagger/dagger/issues/6170

[austere echo]

cc @gilded idol @lofty estuary for more in-depth 👀 .

@cedar sun by any chance have you tried porting the Dockerfile or part of it to Dagger? Mostly double checking if the issue is just with Dockerifles or might be affecting some other things in the engine

[cedar sun]

I haven’t tried porting it. The very first step that does a simple two-file COPY is not cached already.

Would you (or somebody else) mind having a call with me? This would greatly help us.

I’ll have to double check but I’m pretty sure the dockerBuild step is running on top of a dagger.Directory assembled from an empty one.

Is it possible that creating an empty Directory invalidates the cache? I wouldn’t expect it to happen, but…

But actually, our Depot dashboard has 0% cache hit for Dagger, so perhaps the issue is in the integration? We have other Dagger functions which don’t use dockerBuild

[austere echo]

yes sure, we can have a call. Sharing a minimal repro could also help since as you saw in the issue you linked, we do still have some improvements to do in the Dockerfile compat layer regarding disk usage

[cedar sun]

Alright, I’m currently breaking down our setup into something more modular, and I’ll prepare a MRE in the following days. I’ll be on vacation on Tuesday and Wednesday. Perhaps we could have a call today first?

[austere echo]

have you verified if this invalidation also occurs locally? or does it only happen on depot?

[austere echo]

yes, we can have a quick call. What TZ are you on? Mostly asking since it might be nice if someone else from the team is also around to check this out

[cedar sun]

It seems to be working locally. I’m in GMT+2, but I’ll be available for a call in 2.5 hours

[austere echo]

@cedar sun just made a quick simple check for Dockerfiles locally and caching, COPY specifically, seems to be working as intended

We'll be around for a call in about so 2.5hs so feel free to ping us

[cedar sun]

let's talk now? https://meet.google.com/bnn-iieu-ssy