kaniko support

[west pond]

Our enterprise k8s gitlab runners don't support dind and privilege escalation. The recommendation for our teams is to use kaniko. Is there any way that the SDKs could do their magic using kaniko instead of docker directly?

[burnt bloom]

Hi, Kaniko doesn’t have the capability to run containers (in fact that’s its distinctive feature) so unfortunately Dagger can’t use it as a runtime.

However there is a way to run dagger in kubernetes without dind or escalating the gitlab runner’s privileges. You can run the engine as a kubernetes daemonset and configure the dagger CLI to connect to it. This configuration is in production in a few places already.

[west pond]

interesting any ptrs where I can read more?

[burnt bloom]

here is another discussion about it: #kubernetes message

[crystal bronze]

happy to answers any questions about that @west pond. I've been involved in that work

[west pond]

Thanks @crystal bronze I think I'm still trying to wrap my head around how this all is supposed to work. My mental model is something like a hosted service deployed in k8s that my dagger scripts and API client can connect to. When requests are made to build our output execution results I assume we are connecting to this server and transmitting information back and forth, so the k8s service is effectively performing our builds for us. And I assume this would be the case both for local development and cicd env on gitlab etc?

[crystal bronze]

yes, that's correct. For local development it really depends if you need / want / can use this remote service or give developers the ability to run their build pipelines entirely from their machine by using docker or lima to provision the engine locally

In our case we only have our gitlab self-hosted pipeliens talk to this kubernets service and developers just use their local dagger engine to perform builds

[west pond]

that makes sense. I'm not sure I have the time / energy to look at setting all this up now but once the merge request is merged and docs are updatd to explaing this new deployment model I'll probably revist setting it up. Really love the local development workflow and excited to use dagger on future projects once this capability is fully cooked. Appreciate the ptrs and wishing you all the best. This really is a game changer for CICD.

[crystal bronze]

That's great to hear Matt! We're working hard to get this out the door at any moment so stay tuned!. cc @molten gyro

[sick canopy]

@crystal bronze do you have any guides / tutorials handy as to how this works! replacing Kaniko seems incredibly ideal

[crystal bronze]

@sick canopy what parts are you particularly looking forward to implement?

do you already have a kubernetes cluster that runs your builds?

[sick canopy]

Can I email you?

[crystal bronze]

no chance to discuss it here in the open?

if there's confidential info involved, I'd suggest to ping @granite dew directly