#What's the state of replacing mage/makefile with dagger soley for simple automation?

1 messages · Page 1 of 1 (latest)

hidden coral
#

I have a new project coming up next week.

I am evaluating a few task running options for automation in CI and local.
Some of this is local tooling though, and not all containerized, like lint/dotnet build/run etc.

I'd like to eliminate Mage reliance if I can to have a single dagger binary call for the tasks, but not sure where it stands right now to replace the simplier automation capabilities. In addition a big part of this with mage is a "remote" module import of tasks defined and discoverable. I need to be able to centralize these and then run renovate to just update when I do a new module release of tasks.

Any updates on where this stands?
The use case has to be really solid as not everyone installs docker, so simple local tasks being fast and easy is ideal. I can give more ideas on requirements I'd look for if it helps, but wanted to get the conversation started as I haven't kept up with what's going on there recently.

clever pawn
#

Do you mean you are trying to replace some of your own Mage tooling with Dagger? And that some of that work involves tasks that run outside of containers?

hidden coral
#

yep

clever pawn
#

I assume you ideally want this to take advantage of the DAG engine?

if this is the desire, it seems unlikely to happen as Dagger is using BuildKit under the hood, which requires a container runtime.

Also consider most operations are performed in the BuildKit engine, away from the host.

#

For this kind of thing, I'm still relying on other workflow engines, and call out to Dagger for the Dagger parts. We've been considering adding Dagger tasks to our DAG system (hof/flow) which itself is built on cue/flow, which coincidentally Dagger was built on before v0.3

#

If there is enough interest, we could put the time in to make a CUE SDK for dagger and wire it up into our flow system

But we don't want the pain of integrating 2 DAG engines that work in opposite directions (like the Dagger team experienced a while back), so it will look similar to the Dagger CUE SDK, but will likely be different this time around

(https://github.com/dagger/dagger/tree/main/sdk/cue seems like the content is now gone on GH and docs)

feral shale
#

as @clever pawn correcly mentioned it's not possible currently in Dagger given it's strong dependency in Docker / Buikdkit and the OCI runtime. Having said that, I don't fully understand what's the main limiting factor that you have with mage. Similarly to Makefiles, you can still define a DAG in Mage plus the ability to run things in parallel.

I'd like to eliminate Mage reliance if I can to have a single dagger binary call for the tasks

You could also have a single go binary as an entrypoint to your tasks. Mage doesn't necessarily requires the mage CLI to run.

#

In addition a big part of this with mage is a "remote" module import of tasks defined and discoverable. I need to be able to centralize these and then run renovate to just update when I do a new module release of tasks.

You can also do this in mage as well.. You can use go to include dependencies from other repositores that when updated via renovate, can automatically bring new entrypoints to your task runner

hidden coral
#

Well, I'm fine with mage even though there's things I'd like improved (better flags etc).
The main thing is I was hoping to eliminate the dependency on mage entirely in a pilot.

That's totally fine though, sounds like not the end game/current state now.
The cue stuff is really cool and I appreciate it being shared. I'm not sure i'll move away from Go/Mage for Cue right now as it last swing I took it just ended up back running Go commands :-p

#

And yes I do this already with mage (the remote tasks).
Just was checking in on dagger progress to see where it overlapped now with task discovery/self contained. Sounds like for my needs I'll probably stick with mage for a while longer.

feral shale
little bramble
#

Yeah Zenith will give you your mage replacement. But it will run in containers

#

Doesn’t have to be docker necessarily

#

Containerizing is the only sane way for reproducible& environments