A lot of 'remove snapshot' when running in jenkins kubernetes worker | Dagger | Page 1

wraith girder Aug 2, 2023, 7:05 AM

#

Hi, i'm trying to find out why when running in jenkins kubernetes, some cache miss happens even though i'm running the exact same pipline. This does not happens when using docker-compose.

When i set the dagger-engine log to debug and compare the two environment, i see a lot more of "remove snapshot" lines in jenkins compared to the one in docker compose. is that normal?

📎 logs

drowsy atlas Aug 2, 2023, 1:31 PM

#

snapshotter cleanup is probably buildkit's GC kicking in. Check out the GC settings here: https://github.com/moby/buildkit/blob/319739513bf0c75c0031d892a0c3e6cf3b1dd755/docs/buildkitd.toml.md

wraith girder Aug 2, 2023, 1:46 PM

#

Hi @drowsy atlas , thanks again for the pointer, disabling it using --oci-worker-gc=false seems to improve the cache. But i still see some cache miss with longer pipeline. I'll check again.

drowsy atlas Aug 3, 2023, 2:09 AM

#

You mean in your local machine?

wraith girder Aug 3, 2023, 2:32 AM

#

nvm, it seems to completely remove all the cache when i run it in the same container of dagger-engine

#

thanks

drowsy atlas Aug 3, 2023, 2:51 AM

#

@wraith girder are you still strulling with this?

#

can we help somehow?

wraith girder Aug 3, 2023, 3:01 AM

#

Hi @drowsy atlas , i'm still trying several things. the pipeline is a bit long so i need some time to verify it.

will ping you if i need more helps. Thank you very much.

drowsy atlas Aug 3, 2023, 3:01 AM

#

I guess the best thing is trying to replicate it in a small pipeline

#A lot of 'remove snapshot' when running in jenkins kubernetes worker