#Using Dagger with Kaniko?

1 messages · Page 1 of 1 (latest)

jaunty flicker
#

Re-posting original message from @mental steppe :

Anyone using dagger with kaniko? Wonder if there's a way to get dagger running (and building docker containers) on kubernetes without doing privileged docker socket mount.

mental steppe
#

Thank you @jaunty flicker and from the looks of it. no one's using kaniko 😭

jaunty flicker
#

Well it’s buildkit under the hood. Anything that works for buildkit on kubernetes will work for dagger as well

#

I don’t use kaniko either but it seems like it could work well with dagger. What kind of integration did you have in mind?

mental steppe
#

Really just a kaniko.#Build or anything that doesn't need the docker daemon to be running

jaunty flicker
#

Dagger itself has builtin dockerfile support, no docker engine required

#

kaniko.#Build should be possible too, can kaniko run inside a container?

mental steppe
#

yes, i think that's how it's supposed to be run

#

is it possible to use buildkit without docker daemon ? since i saw your message, i looked it up in dagger docs and sawthere's a BUILDKIT_HOST settings

jaunty flicker
#

Are you looking to replace docker with another container runtime? Or not use any container runtime at all?

The former should not be a problem. The latter is not possible as buildkit needs a way to run containers

mental steppe
#

This was my goal:

kubernetes without doing privileged docker socket mount.

which AFAIK only kaniko's is easily available right now.

jaunty flicker
#

I see. Do you know how kaniko runs containers from within its own pod? The answer is probably to configure your buildkit in the same way. It's all the same OCI runtimes under the hood, so anything kaniko does should also be available to buildkit and therefore to dagger.

Sorry that I don't know the exact answer off hand. But (Erik Sipsma, currently on vacation) , @humble tulip or @leaden heath might know more. Also perhaps @normal elbow @hallow solstice ? 😇

#

Hum, I might actually be mistaken on the reliance on OCI runtime, from the kaniko website:

kaniko does not actually create nested containers, so it does not require seccomp and AppArmor to be disabled.

kaniko doesn't use runc so it doesn't require the use of kernel namespacing techniques.

#

It appears that kaniko has a custom command executor, which simulates a containerized environment in userland. Unclear what the simulation technique is unclear, or what its performance or compatibility tradeoffs might be.

#

This reinforces my feeling that implementing a kaniko.#Build as a regular action, run by dagger in a regular container, should work fine

#

As for the tradeoffs of userland execution: my guess is it's either A) fast, B) fully compatible with docker builds; but probably not both.

#

My best guess is that it just runs as root inside a special container, and just takes it over (FROM alpine will just unpack the alpine image in rootfs)

normal elbow
#

@jaunty flicker it does not create containers yes

#

kaniko makes quite some assumptions over what control it has in the containers, but it might work relatively well in dagger I feel

jaunty flicker
#

To summarize:

  1. Since Kaniko works just fine inside a container, you should be able to run it as part of a Dagger pipeline
  2. However there is not much to gain by this, since Dagger already can build docker images natively. The main benefit of Kaniko is not requiring access to a container runtime - however Dagger uses buildkit which does require access to a container runtime, so using kaniko in a Dagger pipeline will not help in that regard.
  3. The Dagger Engine cannot run pipelines without access to a container runtime. Therefore running it on a Kubernetes cluster requires one of the usual techniques: privileged access, cgroup nesting etc.
  4. There might be more advanced ways to run Dagger on Kubernetes in the future, leveraging the capabilities of buildkit. But those require more design discussions first.

Hope this helps