We've been moving our CI into a GKE cluster piece-by-piece, and I'm currently trying to get dagger to work in that context. There's lots of options for deploying buildkit and it's pretty tricky -- I was wondering if the team has recommendations for running in this context.
#Best practices for running Dagger in a Kubernetes cluster (for CI)?
half cape
brazen drum
We started packaging Dagger in production-grade, supported editions for different infrastructure environments, including Kubernetes. We'll make those bits available as part of Dagger Cloud (so that they can be hooked up to an overall "fleet management" strategy: inventory of runners, telemetry tokens, auto-update etc)
We started deploying early versions of this with some of our early customers
dry perch
Yeah, thats interesting. I was already thinking of experimenting with running buildkit in Kubernetes and have dagger use it in some way. But I didn't get to it yet. I am not sure if its still possible to tell dagger to use a self hosted buildkit deamon.
narrow crater
Yeah, thats interesting. I was already thinking of experimenting with running bu...
It isn't. It has to be the dagger engine daemon.
half cape
Read up on this in other threads a little bit and found https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-operator_manual.md#connection-interface -- this'll definitely work for me, but I haven't figured out how to set the engine container's TCP port yet. Is there an environment variable?
half cape
found it! For anybody else who's searching, you want to pass --addr tcp://0.0.0.0:someport to the container, e.g. docker run --privileged ghcr.io/dagger/engine:v0.5.0 --addr tcp://0.0.0.0:8372
Next question: how do I authenticate with a private registry? When I ran the daemon locally it inherited my credentials from my docker environment, but that doesn't seem to work when running this way, even if there's a docker client with credentials set up on the machine that's running the dagger SDK... 🤔
twin sparrow
You could use withRegistryAuth: https://pkg.go.dev/dagger.io/dagger#Container.WithRegistryAuth
half cape
@twin sparrow Is there a way to do this with a credential helper rather than a password?
twin sparrow
You mean using the docker credentials even in this case?