Running a Custom Dagger Engine with a UNIX Socket & Dagger Function Core Execution

[faint phoenix]

1. In the documentation’s “Connection Interface” section, there’s a mention of using a UNIX socket. How do I run a custom Dagger Engine with this parameter? Could you provide an example?
   2. Do Dagger Function Core tasks run as runc containers inside the Dagger Engine, or do they execute locally without containers?

[noble otter]

I'm guessing you mean https://docs.dagger.io/configuration/custom-runner/#connection-interface for part 1

you can set _EXPERIMENTAL_DAGGER_RUNNER_HOST=unix:///var/run/buildkit/buildkitd.sock for example, if there's a dagger socket at /var/run/buildkit/buildkitd.sock

e.g., this is the case in the dagger engine container itself:

$ docker exec -it dagger-engine-v0.15.1 sh -c "env | grep EXPERIMENTAL"
_EXPERIMENTAL_DAGGER_RUNNER_HOST=unix:///var/run/buildkit/buildkitd.sock

generally this option only makes sense with dagger-in-dagger setups

[faint phoenix]

Thanks, Got it.

[noble otter]

for 2, it's a bit tricky, but generally, most of the functions execute without launching a runc container

[faint phoenix]

I guess it’s the same for TCP? Only in Dagger Engine, right?

[faint phoenix]

Ok, just wanted to confirm. Thanks

[noble otter]

yeah, that can also be useful for example if you want to run the dagger engine as a kubernetes sidecar for example

then you can connect the cli to the sidecar using the tcp connection

[faint phoenix]

This is exactly what I’m looking for: running Kubernetes with different Git runners and either setting up a central Dagger service or running it alongside a Git runner using a UNIX socket. I believe this is the standard configuration, but I can’t find any documentation that describes it.

[faint phoenix]

@noble otter
Could you provide an example of how to run this with TCP, please? If I understand correctly, I need to configure it in engine.toml, specifically in the [grpc] section with something like address = [“tcp://0.0.0.0:1234”]. Will the [grpc.tls] subsection also for cert and key?

And just to confirm, is the engine based on—or does it utilize—BuildKit, potentially sharing a similar configuration file structure?

[gilded vessel]

@faint phoenix you can set the TCP listener by adding the --addr when running the engine container.

i.e: docker run --rm --privileged -v /var/lib/dagger --name dagger-engine-v0.15.1 registry.dagger.io/engine:v0.15.1 --addr tcp://0.0.0.0:1234 --addr unix:///run/buildkit/buildkitd.sock

^ that will start the engine listening both on a TCP and UNIX socket

you can follow the same pattern if you're running the engine in k8s also. Just modify the engine pod argument's and that should be it

[faint phoenix]

Thank you!