#disable / change cache garbage collection behavior

1 messages · Page 1 of 1 (latest)

trail rover
#

Hey,

We're using dagger on a long-running hosted GitHub actions runner to build a large go project (lint,test,build pipeline).

We're using mounted cache for /root/.cache/<cachedirs>.

The build is quite heavy.
The first build takes ~20 minutes.
After we have cached Go packages, build cache, golangci cache etc, the build takes ~ 1 min.

Once every several builds it seems that the cache gets invalidated and the build takes ~20 minutes again.

I suspect that it's related to Buildkit's garbage collection mechanism.

I'm seeing the following in dagger engine logs:
gc cleaned up 1389721294 bytes

Does builtkit actually gc my mounted cache volumes?
How does dagger interact with buildkit? what driver does it use, and will it respect a custom buildkit configuration?

Is it possible to modify buildkit configuration to keep the cache for an extended period?

trail rover
#

disable / change cache garbage collection behavior

dusty vapor
#

Does builtkit actually gc my mounted cache volumes?

yes, that's part of buildkit's gc behavior.

How does dagger interact with buildkit? what driver does it use, and will it respect a custom buildkit configuration?
Is it possible to modify buildkit configuration to keep the cache for an extended period?

yes, there's an engine.toml you can provision in your Dagger engine which can be used to set buildit's params. More info about that here: https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-operator_manual.md

trail rover
#

thanks @dusty vapor

viscid oriole
#

@dusty vapor any plans to let users of Dagger customize those params via the sdk or CLI?

dusty vapor