#Pruning local cache

1 messages · Page 1 of 1 (latest)

plain ridge
#

Dagger cache often fully fills up my machine's hard disk. I think the gc kicking in at 75% of total disk space might be too high of a threshold when doing development on a local machine.

Does anyone have a handy one-liner that would allow me to purge the local Dagger cache?

sturdy owl
#

Hey mike!! What I usually do is I delete the container running the dagger engine. The next dagger call will start it up when no engine is running. Typically with a docker rm -f. Maybe there are better ways so I'll lurk here to see if better options show up 👀

acoustic depot
#

Erik added some APIs recently to the engine that enable this. There's no current "nice" UX around it but you can call it like this:

echo '{daggerEngine{localCache{prune}}}' | dagger query

green matrix
#

Yeah, to be clear we are planning on adding nice UX frontends in the CLI. The API is just in early stages and being iterated on atm, you won't have to run raw gql queries forever 🙂

plain ridge
#

Thanks @acoustic depot, that was nice enough, as long as I can easily c/p it 😄

autumn shell
#

Are there any queries for getting the current usage?

acoustic depot
autumn shell
#

Is it possible to prune build results vs host directories or image pulls?

acoustic depot
autumn shell
#

Is it possible to configure the TTL for cache entries in the buildkit engine? We are seeing 60G of layer cache, much of which we know will not be used again

#

( I assume these are the layer caches that are in buildkit and would be the same between dagger/docker, not the cache's we manually create for go mods, etc...)

acoustic depot
autumn shell
#

do you have a link for where/how to set buildkit config with dagger generally?

acoustic depot
autumn shell
#

thank you @acoustic depot

#

one of the big cache entries is a Go binary we build and then write out to disk, which we know we'll never need within the layer caching

acoustic depot
autumn shell
acoustic depot
#

?

autumn shell
#

Mostly leaving a link so that others have a quick reference when trying to understand the behavior

for example, the filter selects what types to prune, rather than "filtering" out those types

also, no pruning happens until the bytes threshold is met, I would prefer an "or" here, certainly would like to have more fine-grained here.

Maybe Dagger could add a query that allows me to prune specific entries by id? This would allow me to write custom logic

#

exposing this "delete cache entry by id" in the SDK would be great too

acoustic depot
autumn shell
green matrix
#

The ids are completely random, what I want is for users to be able to go from a dagger object ID to corresponding cache entry and prune that

autumn shell
#

Interesting, particularly having something like GCS acting as a distrubuted cache for the devs

#

The problem we are hitting is that their disks are filling up, BuildKit being a significant contributor. We have a binary we build that creates around 6G in cache entries, notable

  • .5G git source
  • 1G git clone
  • 1G go mods
  • 1G go pkg compiler cache
  • .5G source context
  • .4G binary build
  • iirc another .4G for the binary copy before export
#

(the resulting binary is almost 400Mb, mainly test files embedded into the Go binary)