#Remote Caching?

1 messages · Page 1 of 1 (latest)

prisma oxide
#

Hello,

Is there any way to use a remote cache for Dagger?

I'm using Dagger with Depot, which mostly works, except when new machines need to be spun up to handle load and their cache isn't saved so I start getting a ton of cache misses and really slow CI builds.

Feels like a remote cache using a docker registry or similar would work really well.

vivid dew
#

LMK if that helps 🙏

prisma oxide
#

so not right now, but maybe soon?

vivid dew
prisma oxide
#

maybe - could be neat to try
is this something you guys are building into dagger cloud?

vivid dew
#

@prisma oxide besides inconsistent caching do you have any other compliments or observartions about the depot integration? any feedback helps us to provide a better overdall Dagger experience

tame gulch
#

@prisma oxide there is an "escape hatch" to access the underlying buildkit cache export, which in theory supports export to registry, s3, etc. But unless you already have run buildkit export in production and are familiar with the pitfalls, we don't recommend following this route. We're working on giving you better options, but as @vivid dew mentioned it's work in progress.

prisma oxide
#

our setup is probably a bit nonstandard, where we're running all tasks for our monorepo in a single Promise.all block with dagger call ci
we rely on the cache entirely then to not run bits of the CI if no inputs have changed
downside is our checks are super heavy, so we start getting timeouts and failures if running more than 2 in parallel on the same dagger machine in depot
so we have that set as the max concurrency

#

so when we have multiple machines running and most of them don't end up saving the cache, this ends up leading to pretty expensive cache misses for us

tame gulch
#

specifically this part 👆

#

and this part 👆

prisma oxide
#

yup, relying on the build cache is how we have it optimized right now
with the main issue being that depot only saves the cache of the primary builder, not any of the extras spun up to handle load

tame gulch
#

Also copying @solar charm to share how we configure our own CI (also a big GHA config with a giant monorepo and big dagger pipelines..)

#

@prisma oxide beyond the bigger improvements coming in Dagger (referenced in that thread I linked to), seems like maybe you could split your big dagger call into several smaller ones, to take advantage of native GHA scale-out?

prisma oxide
#

depot uses remote dagger engines for this stuff, so the actual gha runners aren't doing much

#

but it's somewhat a 'business requirement' that it's all in one ci job for now

#

or at least that they all need to run on every commit, can't use github actions path filters and similar to entirely skip chunks of the work

prisma oxide
#

following up on this - is there a way to do something like this in dagger like i can in with docker buildx?

 --cache-from type=registry,ref=myrepo/buildcache \
  --cache-to type=registry,ref=myrepo/buildcache,mode=max \

or anything with a similar effect? still running into the need for a reliable cache

vivid dew
# prisma oxide following up on this - is there a way to do something like this in dagger like i...

yes, buildkit cache exporters are available in dagger. Here's an example on how you can set them. Here's an example of one of our tests using this:

https://github.com/dagger/dagger/blob/main/core/integration/remotecache_test.go#L355

GitHub
dagger/core/integration/remotecache_test.go at main · dagger/dagger

An open-source runtime for composable workflows. Great for AI agents and CI/CD. - dagger/dagger

noble kernel
vivid dew
# noble kernel <@336241811179962368> any news on that?

hey Dominik! that's still work in progress. Have one question for you: Do you have a strong requirement of running the pipelines in your own infra? We have a already something available in tech preview to help with this but it currently requires running your work in a compute that we manage.

hollow hamlet
mystic pollen
tame gulch
#

Hi @hollow hamlet @mystic pollen we'll be in touch soon 🙂 We're working on something which we're very excited to show, soon

noble kernel
tame gulch
noble kernel
tranquil wren
#

Sorry for jumping in here, colleague from @noble kernel . Is there any update on that topic?
We'd be fine to have a dagger cloud subscription, but beside compliance, our build depend on different local dependencies which we can not (easily) make accessible for external access.