Run dagger modules from custom dagger application?

[round coral]

Hi
Not sure if this was asked before

Is it possible to run dagger modules from inside dagger custom applications? https://docs.dagger.io/extending/custom-applications/go/

My use case is that I want to provide a dagger module that has specific logic for building certain applications, where developers can utilize the power of dagger modules functionality to enable TUI and have the ability to debug

But at the same time use the same code inside a custom dagger application so I can utilize multiple clients to connect to multiple hosts and architectures using https://pkg.go.dev/dagger.io/dagger#WithRunnerHost

For example, I have code that builds for arm64 and x86_64, now I know dagger supports cross builds using builtkit but its very slow, and I have access to an arm64 host and an x86_64 host, so I would like to have the custom application to create two clients, one for arm64 and one for x86_64, run the dagger module to prevent duplicate code, and get the result from it

Is such thing possible?

[lost scaffold]

yes it's possible. This feature is called "client generators". The feature has been implemented here:
https://github.com/dagger/dagger/pull/9531

^ the description in that PR has a simple example on how to use it

[round coral]

This is

Incredible

Thank you!

[round coral]

in continuation to the example above

is it possible to "transfer" content between clients?

For example

dagX86 is an x86_64 client and dagArm64 is an arm64 client

here i'm copying fileX86 from dagX86 to dagArm64

fileX86 := dagX86.Container().From("ubuntu:22.04").WithNewFile("test", "testfromx86").File("test")

fileArm64Transfer := dagArm64.Container().From("ubuntu:22.04").WithFile("testx86", fileX86).File("testx86")

fmt.Println(fileArm64Transfer.Contents(ctx))

I tried to run the example above and I did get testfromx86 without any errors

Is this a supported behaviour?

[round coral]

it seems like both clients are connected to the same engine, as the second connect is what apparently sets for the entire session? is there a way to confirm this?

[lost scaffold]

Hi, sharing objects across engines is not currently possible, sorry.

There's been discussions about the ability to handle artifacts as a universal asset that can be retrieved anywhere. Haven't fully fleshed out the details yet

[round coral]

Thanks again
Eagerly waiting for this feature

[strong wigeon]

@round coral did you get the dagger client install command working - im trying to use this feature as well and the command has changed a little since the MR

[woven junco]

Happy to help if I can, what are you stuck on?

[strong wigeon]

i was able to get it running with the go sdk - but after a little digging it looks like maybe this feature isnt supported for python. is that correct

[lost scaffold]

yep, not currently supported for python. @unique birch might have an update about this since he's currently lurking the python realms

[strong wigeon]

i can always force myself to learn golang if i really need this 🙂

[lost scaffold]

SGTM. The python one is particularly tricky since it works a bit different than the other SDKs. Let's wait for Yves to see what he says about it 🙏

[unique birch]

yes, for now it's not supported in python. We are currently doing some changes on the python codegen tool, maybe that could be a great opportunity to also add the client generation (cc @timid bolt )

[strong wigeon]

cool for the short term ill use go - but definitely interested if this feature makes it into the python support

[round coral]

For reference I used

dagger client install go ./dagger-module
dagger client update go

where ./dagger-module was copy-paste of the original modules i wanted, i think you can replace directly with a git url