vito Erik Sipsma Do you know if it s

[gray pecan]

I don't think it's possible to connect Dagger directly to Buildkit - there's a lot of modification of upstream BuildKit, so it doesn't really work in the same kind of way at all.

On the other hand, I have a bit of context for how rootless works (I looked into https://github.com/dagger/dagger/issues/151 with @vernal fulcrum a while ago). I think wrapping the dagger engine in rootlesskit would work in a similar way to how rootless buildkit works for that.

Getting the cgroup error is a buildkit thing somewhere (can't remember off the top of my head), but you shouldn't get it if the engine gets executed with rootlesskit. There's some stuff around that in buildkit: https://github.com/moby/buildkit/blob/master/docs/rootless.md

To wrap with rootlesskit, you'd probably want to modify https://github.com/dagger/dagger/blob/a320703365749a37923dc84a17f54d675e012604/internal/mage/util/engine.go#L193-L229, to something like https://github.com/moby/buildkit/blob/bbe48e778f9df07eabc7fc05023c8e97e3c5c5ce/Dockerfile#L302-L320 in buildkit.

^ that's all a guess, I'm new to dagger architecture so not 100% sure.

tl;dr rootless is exceptionally tricky to get right, and ends up depending on fun features that are heavily kernel-dependent. even in buildkit, there's a long list of incompatibilities - https://github.com/moby/buildkit/issues?q=is%3Aissue+label%3Aarea%2Frootless+

[jade oyster]

Totally agree, that's exactly the purpose of my experimentation: explain why rootless is too tricky to be supported by dagger for now
You can find more context on this issue too: https://github.com/dagger/dagger/issues/5763

To wrap with rootlesskit, you'd probably want to modify https://github.com/dagger/dagger/blob/a320703365749a37923dc84a17f54d675e012604/internal/mage/util/engine.go#L193-L229, to something like https://github.com/moby/buildkit/blob/bbe48e778f9df07eabc7fc05023c8e97e3c5c5ce/Dockerfile#L302-L320 in buildkit.

That's a good idea! I'll try that and see how it reacts!

[gray pecan]

yeah, we should really try and steer clear of recommending it to users (but might still let them if they really want to)
from experience, it produces a ton of difficult-to-debug issues, for a relatively small subset of powerusers

[jade oyster]

Something different is that the entrypoint become rootlesskit, buildkitd (https://github.com/moby/buildkit/blob/bbe48e778f9df07eabc7fc05023c8e97e3c5c5ce/Dockerfile#L320), however in our case we are running ./cmd/engine, we are never running the buildkit daemon itself :/
I'm not sure I can adapt it to our case to run dagger engine rootless

[gray pecan]

I think it should just be to make the entrypoint rootlesskit, ./emd/engine? rootlesskit just runs everything after it, like sudo or ltrace or similar