Using Dagger for multi-repo project

[willow dagger]

Hello!

I am currently evaluating Dagger and was wondering if there was a recommended way to setup Dagger for a multi-repo project. I think it would be nice to colocate the Dagger CI code with each repo. However, I am worried that it might get complex doing it this way unless Dagger has some mechanism that would make this nice?

If that is not possible, my current thought would just be to create a Dagger CI specific repo and then compiling that for use in CI in the various repo pipelines. Would this be a reasonable/possible approach? I also see that Dagger utilizes Magefiles I have personally never used them before, but from what I see in the Dagger repo it looks like a nice way to work with Dagger. Would this second approach work well with Mage?

As a side note, I am really liking Dagger so far!

Thank you!

[onyx cloud]

Hey there! Are you mostly a single language shop/company? Or do you have different teams that use these pipelines for different languages?

Depending on this and how your DevOps team currently operates you could follow either approach or a combination of both

Do your dev teams currently own the process of building their applications via Makefiles, Dockerfiles? Or do your currently gatekeeps that for compliance reasons?

If you don't have any strong preference, I personally find it quite convenient for some Dagger code to live within the application repository so I can clone the repo and run those Dagger pipelines without needing anything else.

[willow dagger]

Thank you for the reply!

We are a multi-language shop, python and typescript mostly, but some others sprinkled around here and there. For Dagger pipelines specifically we are thinking about using Go. Although our org is fairly large the team I am working on is very small and autonomous. We own most of the process including CI and deployment. At the moment, we work with just Dockerfile and scripts to work with Compose when we need integration tests.

Awesome, that is something that I think would be preferable! I would assume that we can then import Dagger code from the other repos as modules for reuse for integration test scenarios?

[onyx cloud]

Yes, exactly. You can have a core common Dagger repository where all the app modules pull dependencies from so you can centralize some things that you need to have better control of.

Having said that, we've heard from some other users that generally developers like having their CI pipelines written in their core language so they can improve or fix them when needed. In that context, there's a soon to be released feature that allows wrapping different modules that can be written in multiple languages and orchestrating them together as a single pipeline. You can read more about it here: https://docs.dagger.io/zenith/

I'd suggest keeping moving towards the way you're heading but if you get some pushback from the typescript or python devs about pipelines being in Go, this feature will be quite handy to overcome that

[willow dagger]

Awesome! Thank you very much for help. Looking forward to implementing the initial dagger setup for my team!