dagger in dagger, but same engine

[tough tusk]

Hello, i would like to be able to dynamically extend my pipeline by running dagger client inside dagger. But i would like to be able to connect the dagger client in dagger to the same engine so that it can share cache with the main pipeline. Is this scenario possible? If so, how should i configure the dagger client inside dagger to talk to the same engine?
Thanks

[gusty pollen]

hi! yes, it's possible. You just need to use the experimentalPrivilegedNesting argument in the withExec (https://docs.dagger.io/api/reference/#Container-withExec) call and that will share the same engine that's currently running your pipeline

[tough tusk]

Wow cool.

Just wondering how does this works actually? I mean withexec is used to not only execute dagger client right, it can be used to execute anything.

Does it mean this argument only useful if executing dagger client inside dagger?

Or is it just a way to natively inject correct address to _EXPERIMENTAL_DAGGER_RUNNER_HOST env variable?

[tough tusk]

Also, why would it give full access to the host?

the command being executed WILL BE GRANTED FULL ACCESS TO YOUR HOST FILESYSTEM

[gusty pollen]

This basically configures the environment that the withExec uses so whatever process you run there can access the dagger engine

[gusty pollen]

You can still run whatever you want there, but if you run something that expects the Dagger engine to be present, like a pipeline, it will just work

[gusty pollen]

You don't need to do this because the withExec with privileged nesting does it automatically for you

[tough tusk]

Hey @gusty pollen , what about this?

Does it means that when we access Client.Host in dagger client in dagger it will refer to the host that run the main dagger client instead of the container that run the dagger client inside dagger?

[gusty pollen]

Because since anything that runs there has access to the host Dagger engine, someone could just do client.Host.Directory() and read anything from the host filesystem.

[tough tusk]

btw why do i get this error?
runc run failed: unable to start container process: error during container init: error mounting "/run/buildkit/buildkitd.sock" to rootfs at "/.runner.sock": stat /run/buildkit/buildkitd.sock: no such file or directory

note that in the main pipeline, i used _EXPERIMENTAL_DAGGER_RUNNER_HOST since the container that run dagger client and the dagger engine is in different pod

this are the python code

src.directory(".dagger")
        .docker_build()
        .with_mounted_directory(".", src)
        .with_exec([], experimental_privileged_nesting=True)

src is the result of cloning git repo which is also done inside dagger

[tough tusk]

but since it is dagger inside dagger, doesn't client.Host refer to the container that was run inside dagger, not the actual host?

[gusty pollen]

Yes, you're correct. I got confused. The security concerns come from the fact that you have access to the host buildkit socket that can be abused to access the host filesystem using buildkit specific APIs

Like running a privileged container from example

[gusty pollen]

Are you connecting to the engine via TCP?

[tough tusk]

Yes

[gusty pollen]

In that case you also need to add the --addr unix:///run/buildkit/buildkitd.sock flag as well for nesting to work

--addr can be specified multiple times

[tough tusk]

Oh i see. Let me try

[tough tusk]

Thanks a lot @gusty pollen , works like a charm.

[tough tusk]

hei @gusty pollen , sorry for another question
if we disable/remove insecure-entitlements = ["security.insecure"], will this prevent any kind of priviledge container started, hence reduce the possibility of script escaping to the host?

[gusty pollen]

yes, this is correct. Having said that, it won't fully guarantee a secure environment since having access to the buildkit socket which runs as a root could still be the an attack vector for some bad intended actors.

[tough tusk]

Got it.

Just another questuon. Is it possible for any program that have access to the buildkit socket to somehow export the buildkit cache?
Thus exposing things like secret that was used inside buildkit?

[gusty pollen]

secrets are not stored in the cache so you're safe in that sense. When using the nestedExec option, since it's a new buildkit session, whatever runs there won't have access to the secrets you're using in your parent buildkit session

since secrets are scoped per session