burnt atlas
Mostly for security reasons, we are unable to adopt your cloud cache for now.
On another help question i was asking about pre-warming the engine.
Reading the discord i see a lot of comments about how export cache / buildkit layer cache has too many issues to be a really good solution.
Are you guys able to sort of enumerate them here? Would help save me some time investigating.
Im aware of the lack of cache volume support - wondering what else i might run into.
Thanks!
fervent holly
@burnt atlas this thread has a good summary about that https://discord.com/channels/707636530424053791/1169459664472248581
burnt atlas
Sounds like
Anything else would you say?
fervent holly
cc @brisk wharf not sure if we're missing anything else here
brisk wharf
The above are the major points 👍
Only other I'd add is that dagger cloud cache includes base image layers, git sources (i.e. Directory objects created by our Git apis), and http sources (i.e. File objects created by our HTTP api), but upstream buildkit remote cache currently doesn't.
For some use cases, this can make a difference since the cache service is often closer+faster than the places those image/git/http sources were originally pulled from. But this is relatively minor compared to the above points.
burnt atlas
Awesome thank you guys
wind echo
The above are the major points 👍
Only other I'd add is that dagger cloud cache...
out of curiosity - was discussing this with @fervent holly, i don't actually know where the code that does this is 🤔
is there something explicit in buildkit that means sources aren't exported, or is the functionality in our cache export just completely different?
brisk wharf
out of curiosity - was discussing this with <@336241811179962368>, i don't actua...
It's that buildkit explicitly doesn't do it right now. When I talked to Tonis about it most recently he said he wasn't sure if discluding images was the right call, but that's the behavior currently (unless anything changed recently). I believe the behavior is enforced by buildkit somewhere around here: https://github.com/sipsma/buildkit/blob/e08224fa5567ad24f66b5ed6ead04c5c318ec86e/solver/exporter.go#L59-L59
But I can't remember exactly where and that code is extremely dense to sift through
Dagger's cache export doesn't go through those code paths at all, our export code is here: https://github.com/sipsma/dagger/blob/fb3a9d78cb38e03135f4c24f2bf4414543d007e3/engine/cache/manager.go#L141-L141
Mainly due to the fact that we want engine-wide cache exports that run in the background independent of a single client session, but the fact that we can choose to include images/git sources was a nice side effect
wind echo
agh i was expecting it might be something like that lol - i was hoping for a nice little "oh just switch this option off/on"
brisk wharf
*agh* i was expecting it might be something like that lol - i was hoping for a n...
I'm pretty sure that could most likely be added to buildkit upstream; it just isn't relevant to us anymore since the codepaths have diverged entirely
wind echo
Ah, no it was more just relevant in me trying to understand caching differences between the remote buildkit exporters and the cloud caching 😄
oh actually there is a toggle in buildkit for this: https://github.com/moby/buildkit/blob/master/solver/exporter.go#L100-L103
if ExportRoots is set, then the Sources get exported