#So how can people realistically use

1 messages · Page 1 of 1 (latest)

feral ore
leaden quarry
#

There's also different strategies depending on how you're running GHA runners. If you have self-hosted runners (in k8s or otherwise) there are options. For managed runners, we had the experimental distributed cache which aimed to solve this. The issue to keep an eye on for a generalized solution is https://github.com/dagger/dagger/issues/8004

GitHub
Storage Drivers · Issue #8004 · dagger/dagger

This has been discussed on various internal trackers and work has been started, converting to a public issue now. Goals Primary: Make the cache storage of the Dagger Engine pluggable and configurab...

upbeat plume
ashen ruin
upbeat plume
ashen ruin
#

Hard drive ?

#

Dump dagger volume to disk.

#

Point github cache at that directory

lethal plaza
# ashen ruin Dump dagger volume to disk.

the dagger volume is already in the disk at /var/lib/dagger but this approach has several issues as it's not usually safe to share the same underlying dagger or docker volume across multiple builds

#

as it's not efficient (it uses a bunch of space) and data can become correputed quite easily

#

that's why you don't see any Docker docs suggesting to do the same in GHA

#

the reality is that there's no current silver bullet solution for ephemeral container caching. We had an attempt with our experimental distributed caching serivce and we're collecting that feedback to hopefully come back with a better solution 🙏

ashen ruin
#

So not docker caching but exporting the directory contents onto the github disk.

Then GHA cache of a .zip file

You don't need to bring docker into here.

So if the contents of that directory can be zipped up, then you can store a zip for next time, and then mount it in and unpack it .. on the next CI

#

Like ->export()

#

Without needing to call export()

#

@craggy crown

craggy crown
#

Maybe not?

ashen ruin
#

Creating zip exports of volume dirs

And then mounting and unpacking them ok the next run.

It would work. I think.

Has this been tried?

craggy crown
# craggy crown Maybe not?

Kinda failing to remember all of the stuff around that, but skimming it seems like this was for a similar thing

spice scaffold
#

Also why we worked with Depot.dev @ashen ruin

ashen ruin
#

Is it free?

spice scaffold
#

Same price as normal accelerated GHA from Depot

ashen ruin
#

Okay, so it's going to cost teams $200/month to use Dagger + Depot (for caching). Got it 👍

What about open source projects who want to use Dagger, instead of GitHub or GitLab? Are we not targetting those projects? (their build times would be very very slow if they switched)

#

@lethal plaza why wouldn't this work?

Creating zip exports of volume dirs

And then mounting and unpacking them on the next run. 

It would work. I think.  Has this been tried?
spice scaffold
# ashen ruin Okay, so it's going to cost teams $200/month to use Dagger + Depot (for caching)...

Some people have small needs and spend zero, but there are lots of folks who spend many hundreds, thousands, and multiples of that on CI including GH Actions. Someone needs to pay for the compute. Many of our bigger users on GH employ self-hosted GH actions runners, often running on a manged Kubernetes like EKS. You better believe they get an AWS bill, but they can get a lot of caching from the cluster nodes through the basic /var/lib/dagger mechanism mentioned above.

ashen ruin
#

Sure. I get that.

#

The little man just wants to bring Dagger into their company, from an OSS perspective, they probably have free github as well (it comes with github actions) .. so those people are no longer users anymore, since they can't cache anything .. build time will go up and Dagger won't get adopted by the team ..

Just one scenario that I foresee

spice scaffold
#

This has been a hot topic for a while (plenty in Discord history), for sure, and one we'd love to see a solution for.

ashen ruin
# 
Creating zip exports of volume dirs

And then mounting and unpacking them on the next run. 

It would work. I think.  Has this been tried?
#

Poor mans caching. I think it would work. Why wouldn't it?

lethal plaza
#

@ashen ruin this should/could work with the following caveats:

  1. It's not currently possible to reliably export / import the cache volumes from the engine. As @craggy crown mentioned, more info about that here #1344341021055647845 message

  2. Cache volumes are only a small part of the overall pipeline caching. Sure, they generally speed up the wall time of a pipeline but it really depends on what the pipeline is doing to get the most out of it. @upbeat plume kind of convinced me that an "ideal" world, cache volumes shouldn't exist as all the steps in a DAG should be content addressable, including pulling whatever asserts or artifacts that any vertex in the DAG needs.

So, to summarize, even if https://github.com/dagger/dagger/pull/7646 gets merged, it still won't be close to a "definitive" solution towards ephemeral runner caching

ashen ruin
#

Hey @lethal plaza

We don't need any pipeline caching, just being able to export/import key things to/from the GHA cache.

For example node_modules, or the php_composer cache.

This stuff takes a lot of time, and team's won't bring Dagger OSS into the mix if they can't cache anything, without paying money to third party services.

upbeat plume
#

(which is why we haven't solved it once and for all - yet!)

queen linden
#

👋 I'm hijacking a bit this thread - we're on a different context, with self-hosted runner (using ARC). Is the approach described in this document still valid ?
https://dagger.io/blog/argo-cd-kubernetes
Are you aware of other infra setups solving the cache problem in the context of hosted GHA runners ?

On-Demand Dagger Engines with Argo CD, EKS, and Karpenter - Dagger

Powerful, programmable CI/CD engine that runs your pipelines in
containers — pre-push on your local machine and/or post-push in CI

lethal plaza
ashen ruin
#

Oh, the joys!

│ ! fatal: unable to access 'https://github.com/levlaz/daggerverse/': Could not resolve host: github.com
#

I'm gonna re-install dagger, 15 minutes before the community call 😅

EDIT: Working nicely, found a few bugs in the Uninstall/Install docs.

#

Who can send me the github URL to work on the Install/Uninstall page?