actions/cache

[alpine ice]

Looking at the directions for running Dagger in GitHub: https://docs.dagger.io/145912/ci#github-actions it's not clear how the builds are being cached in GitHub Actions' cache (i.e. actions/cache)?

I tried the following (since /var/lib/dagger is referenced in other Engine setups):

      - name: Setup Dagger BuildKit cache
        id: cache-dagger-buildkit
        uses: actions/cache@v3
        with:
          path: /var/lib/dagger
          key: ${{ runner.os }}-dagger-buildkit
        timeout-minutes: 2

but GitHub said the /var/lib/dagger directory was empty? And re-running my GitHub Actions job shows that containers are being re-built instead of being served from cache somewhere...?

[wet smelt]

Hi Ari, the Github Actions cache cannot handle Dagger’s caching data. You can use Dagger Cloud which includes a distributed caching service to solve exactly this problem, See https://docs.dagger.io/cloud#distributed-caching

[alpine ice]

Hi @wet smelt , I responded to you privately as well.
Is there no way to achieve this entirely within GitHub Actions' cache?

[sweet perch]

@alpine ice similarly to any container building tools (Docker, podman, lima, img, kaniko, etc) they don't play very nice with GHA action's cache since it's mostly designed a distriubted FS instead of being a container image optimized cache. Having said that, buildkit (the low-level DAG engine) which powers Dagger, has some "experimental" GHA caching providers (https://github.com/moby/buildkit/blob/master/README.md#github-actions-cache-experimental). We tried to leverage this initially but given how inefficient and limited they are, that's what made us build our own distributed caching solution to meet both our and ours users needs.

[alpine ice]

@sweet perch even so, is there a way to configure it?
Buildkit also has experimental support for treating s3 as a cache; using a Cloudflare R2 bucket would be great for that purpose as well with 10 GB free per month; as someone building a side project, I'm really looking for cheap options here, not ideal ones

[sweet perch]

yes, there's a way to pass those flags to Dagger. You can search in this Discord server for _EXPERIMENTAL_DAGGER_CACHE_CONFIG as it has been brought up before

keep in mind that we personally don't recommend and/or support using this feature at all since besides being experimental, it has some weird bugs which are unrelated to Dagger.

you can find and read more about this in this Discord and the buildkit issue tracker

[alpine ice]

@sweet perch thanks, perhaps encountering those bugs for myself will help me appreciate the journey you went through better. It's certainly not obvious why it's not supported.

[sweet perch]

I recall @wet smelt going into this a bit more in detail in one of our community calls when we announced Dagger Cloud and the reason for building our own distributed caching solution but can't seem to find the recording now. There's another important aspect @alpine ice which is that buildkit cache exporters don't handle cache volumes but Dagger's solution does. Just brining this up in case you're relying on them in your pipelines

[wet smelt]

Buildkit cache export is slower, caches less things, and performance gets worse as you scale