How could I make the best use of toolchains while keeping the flexibility ?

[marsh lava]

I've started migrated some our microservices builds to use toolchains but I find myself in a bit of an odd situation and would like to understand if there's a way around it.

I had a microservice module use my golang module which could be explained like that

- New()
  - Initialize the golang module etc

- Build() *dagger.Container
  - call golang.Build
  - return the container produced

- Test() (string,error)
  - call golang.Test()
  - return (string, error)

- Serve() *dagger.Service
  - call Build()
  - Add Env Variables to container
  - return container.AsService()

---

Now that i'm moving to toolchain i can get rid of Build() and Test() and have the setup the customizations according to my needs
Which leaves Serve(), this wont/can't move to a toolchain because specific to the microservice.
I find myself in a situation where I cant get the *dagger.Container from Build() because it's out of my module now (whole point of using toolchain).
Sure, I could get a instance of the golang module and use it but it feels like it would be defeating the purpose of toolchains (and i would repeat myself injecting customizations via my module's code)

What would be an elegant way of resolving this ? Is there a way to get the configured toolchain directly in my module ?

[dense jewel]

@marsh lava toolchains are just like dependencies with the exception that they also enable checks. So if you have a module with actual code that has toolchains, you should be able to all the toolchains funtions in the same way as if they were dependencies

so in your module's Serve function, you should be able to call dag.ToolchainName().Build() to get the container from the toolchain directly

It's also possible to make the Serve function part of the "microservice" toolchain and allow it to be customized via toolchain "customizations" (https://docs.dagger.io/core-concepts/toolchains/#advanced-customizing-in-daggerjson). It's not as flexible as being able to write the function code, but maybe via customizations you can find a good enough trade-off so teams don't have to maintain a module themselves

[marsh lava]

*so in your module's Serve function, you should be able to call dag.ToolchainName().Build() to get the container from the toolchain directly *
would this have the customizations coming from the dagger.json ?

[marsh lava]

it would be amazing to be able to have the toolchain module injected ( via constructor ) pre-configured (with what's in the dagger.json customizations)

[nocturne vine]

@marsh lava the way we do it in our own repo, is to move everything in toolchains (no more inline code in the project .dagger), and some toolchains are general while others are very specialized or even project-specific.

Then from there, as marcos said, toolchains can import functions from other toolchains

[marsh lava]

what would be the benefit of having a project-specific toolchain ? I mean if it's too specific it cant really be re-used anywhere else

[nocturne vine]

mostly not having to answer "it depends" when asked "where does the code for dagger functions go?" and not having to explain both the .dagger system & toolchain system to new users.

But you're not the first to ask this, we recently had the same discussion with @past panther and @tawdry fiber

[dense jewel]

no it won't since you're not calling the toolchain function directly but wrapping it with your Serve wrapper. You can define user-defaults if you want to go this way and pass them over when you make your programatic toolchain function call

[marsh lava]

i need to have think about that. most of all i want our devs to just be able to run the code locally via dagger and get a instance of the app running wihout worrying about sdk, dependencies, services etc (docker-compose up like simplicity) .
I think i can live with instantiating my go module in Serve() and make the best use of the user defaults as mentioned by @dense jewel

[nocturne vine]

I actually don't know if customizations are applied if you call toolchain functions directly from the main module. it might work (accidentally). once we settle on a model we can intentionally make it work one way or the other

[marsh lava]

it's not a bug it's a feature 🙂 . Let me try if it works

[dense jewel]

I was almost sure they don't work in the same way user-defaults also don't work. It's easy to check though 🙌

[nocturne vine]

customizations definitely won't be applied if you import toolchain A from toolchain B directly. Mmm that is giving me pause, need to think about that

[marsh lava]

it actually works

[dense jewel]

woot! 🙌

[marsh lava]

calling dag.Golang().Build() from my microservice reads the customizations from my dagger.json

ah that's awesome, and totally intended i'm sure 😄

[marsh lava]

is there any reason ou guys would lock that feature down so it's not possible to use toolchains/customizations that way ?

[nocturne vine]

actually it's the other way around now I'm motivated to make sure this works also in the everything-is-a-toolchain model 🙂

we just need to make sure the whole system is consistent, with a good balance of simplicity and control

one improvement we want to make is introduce a simple workspace configuration file, to unify all the different configuration knobs for your dagger project that aren't in module code (because of the sandboxing)

so: customizations, user defaults, etc

[marsh lava]

that would be awesome. why not use the dagger.json ?

[nocturne vine]

that is an option, but the json format is quite user hostile..

[marsh lava]

what's the goto these days ? YAML ? TOML ?

[nocturne vine]

we're considering toml (recommended by @past panther ) because it's simpler and more constrained than yaml, which limits the temptation to distort it into pseudo-code 🙂

[nocturne vine]

Yes just to confirm that is the priority: whatever design leads to the least friction for the team, will be our favorite design 😁

@marsh lava how strong of a negative reaction do you have to the idea of a project getting all its dagger functions from toolchains, with some toolchains being universally reusable, some only useful to a single project, and a whole spectrum in between?

[marsh lava]

I would need to see the implementation of that ( if you have a dagger example i'm keen to have a look). To me if a module is so specific that it cant be reused then it shouldn't be a toolchain. I think of a toolchain as any other code library. Following the DRY principles as much as possible.