#Understanding cache options for GitHub Actions

1 messages · Page 1 of 1 (latest)

paper vine
#

I'm trying to figure out the best way for us to cache our GitHub actions jobs to save time on dagger prereq steps. Via dagger cloud I'm seeing a recent run of our e2e test suite taking 4m25s, but we spend 5m25s in Dagger on steps that aren't even running dagger call ...

2m44s are spent just starting up and connecting to the engine and another 2m41s are spent loading module and dependencies.

For comparison sake, running dagger call with the same test locally takes just 5m21s with the connect phase taking 0.4s and load modules at 3.3s.

It's concerning that dagger is taking this long to get into the dagger call stage and is literally taking longer than the time it takes for the test suite to run.

I've looked around Discord and the interwebs and it appears there's a few options here:

  • Use _EXPERIMENTAL_DAGGER_CACHE_CONFIG with GHA actions cache which apparently is unreliable and I'm not really sure how to use it. Right now I've forked dagger-for-github and added a cache-config var to it and am trying to go down this path.
  • Use https://depot.dev/
  • Run a self-hosted runner that's running dagger and therefore will just end up acting like a locally ran dagger

Are there any other recommendations around this that you all can share? I see https://github.com/dagger/dagger/issues/8004 is in the works now, but are there any options for solving this today outside of the ones above, or any recommendations with setting up any of the above options?

Depot

Accelerated builds for Docker images and GitHub Actions workflows.

GitHub

This has been discussed on various internal trackers and work has been started, converting to a public issue now. Goals Primary: Make the cache storage of the Dagger Engine pluggable and configurab...