buildkit cache config inpython sdk | Dagger | Page 1

wet sphinx Jul 16, 2023, 11:52 AM

#

Hi, how do you specify buildkit caching config in python sdk? I would like to run my dagger pipeline inside kubernetes pod started by jenkins, thus I need external cache storage

honest warren Jul 16, 2023, 3:27 PM

#

check: https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-operator_manual.md#execution-requirements. Particularly point 2

GitHub

dagger/core/docs/d7yxc-operator_manual.md at main · dagger/dagger

A programmable CI/CD engine that runs your pipelines in containers - dagger/dagger

#

if you provision your own engine, you can assign a volume to /var/lib/dagger to increase the buildkit cache storage

wet sphinx Jul 16, 2023, 11:16 PM

#

I see, can this volume be used by multiple dagger runner at the same time?

honest warren Jul 17, 2023, 12:12 AM

#

wet sphinx I see, can this volume be used by multiple dagger runner at the same time?

No, it can't. Each engine needs its own volume, that's a buildkit design limitation. If you're looking for a highly scalable distributed caching solution, we're developing that as part of a commercial Dagger product. We're extending invitations to early adopters looking for feedback if you're interested. Cc @tepid hatch

wet sphinx Jul 17, 2023, 12:41 AM

#

Got it, thanks once again

wet sphinx Jul 17, 2023, 1:44 PM

#

Hi @honest warren , if i were to connect to dagger engine trhough tcp port, how should i run the dagger engine and specify the tcp port it should use?

#

honest warren Jul 17, 2023, 1:48 PM

#

docker run --privileged --rm registry.dagger.io/engine:v0.6.3 --help
^ that shows all the flags the engine accepts. What you're looking for is --addr

#buildkit cache config inpython sdk