#Makefile vs Dagger

1 messages · Page 1 of 1 (latest)

half linden
#

Similarities between Dagger and Makefiles:

  • They both describe a DAG
  • They both execute in parallel (make -j)
  • They both cache aggressively (Make's file modification timestamps being the analogue for Dagger's content-addressed caching)
  • They both are typically used for scripting commands.

Differences:

  • Dagger works on every dev platform (no GNU vs BSD make with Linux/Mac).
  • Dagger builds are self-contained; don't need to know to install tools X, Y, and Z ahead of time, and which versions.
  • Make's .PHONY targets aren't cached. In Dagger, everything is cached.
  • Dagger lets you write in your preferred language instead of a DSL. No need for templates or other convoluted schemes. Just write code.
  • Dagger allows you to abstract away commands behind local functions - or, soon (#daggernauts), GraphQL APIs that can be shared and used from any language.

Just some quick thoughts, others can probably come up with more.

unreal comet
#

I dont work for Dagger. I would also add that Dagger allows isolation (no polluted builds). Which is pretty huge for C/C++, and in principle vastly improved resource constraints on builds via CGroups.

#

I wrote up a pretty exhaustive discussion of the differences in an issue on the dagger repo a while back.

paper pasture
#

Thank you! The response I get for this

Dagger builds are self-contained; don't need to know to install tools X, Y, and Z ahead of time, and which versions.

and what @unreal comet mentioned (isolation) is that you can still run stuff in containers with Make.

unreal comet
#

Sure, you can run containers with Make, but since Make handles invalidation with files this gets messy quick involving “stamp files” having written make files that starting and stopping services for tests, building multiple container images, etc. Make isn’t reallly the tool for that job IMHO

#

Make still does appear in my “tool belt” post dagger because theyre are few things with the simplicity for simple cases as Make, but for “production” things with complex dependencies where I want to write functions for similar tasks and ensure that no “cowboy compiled” things seep in, or I am already am building a complex multi stage container that is where dagger is a most useful for me right now

half linden
#

Agreed - and to be clear, I don't think we would argue that Dagger is a drop-in upgrade for Makefiles. I still really like them and use them in side projects/etc., as long as they're able to stay simple

kind dome
#

Personally I like to describe Dagger as “transforming your Makefile and artisanal scripts into an API”

paper pasture
#

Can you elaborate that a bit @kind dome ? What's the API part? Are you referring to zenith improvements?

kind dome
# paper pasture Can you elaborate that a bit <@488409085998530571> ? What's the API part? Are yo...

Zenith will definitely make it more obvious. But if you squint a little, you can already see it today. A Makefile, like a custom shell script, is made to be invoked directly by an end user, but not to be instrumented by code. In other words it has a UI but no API. As soon as you try composing Makefiles/scripts together, to create higher-level logic, that's when things get very painful very quickly

#

Today (pre-Zenith) you can solve this problem with Dagger, but it requires creating your own library of language-specific packages.

#

With Zenith, the full potential of Dagger finally comes together, because from day one, you will be able to replace your loose collection of scripts and Makefiles, with a loose collection of Dagger functions. Then package those functions into a Dagger module. And then, the magic happens: Dagger can expose any module as a language-agnostic HTTP API, to be queried and instrumented at your team's leisure

#

One of the very first use cases we should try with Zenith, is the following flow:

  1. Start from an existing project's Makefile
  2. Write a Dagger function for each of the Makefile's rule, making sure it properly encapsulates dependencies, parameters etc. Now all the implicit API surrounding that Makefile is made explicit. Note that this does not require rewriting the Makefile: just packaging it as-is
  3. Package the result as a Dagger module, start cleaning up downstream dependencies on that Makefile. Everywhere there is code calling make <foo> - perhaps a shell script, another Makefile, or a YAML CI configuration, replace that with a cleaner API call
  4. Look for the next ugly script-to-script dependency that should be code; rinse and repeat until everything flows through a nice clean API, and you can cleanly model all inter-team and inter-component dependencies

Bonus: you can keep your Makefiles. (although perhaps you will decide along the way to refactor it away - depends on the team's preference)

paper pasture
#

That makes sense! I think when that's possible, some solid examples will go a long way in getting the Dagger story out there. How powerful and flexible it can be.