putting dagger.json in the root of the repo

[hexed whale]

afaik there's an open question around whether root: should be renamed or rethought but no resolution yet, so this seems good to think about at the same time

putting dagger.json in the root of the repo

[analog grove]

https://github.com/dagger/dagger/issues/6237

I took the liberty of opening an issue

[analog grove]

IMO there should not be a root field at all

The root of a module should always be the directory which has a dagger.json . No other good answer IMO.

(or maybe I misunderstand what root is supposed to do)

[tacit grotto]

root currently handles inverse cases of source, e.g. you have multiple modules in a monorepo, but they need some files from parent dirs in order to build such as a go.mod in parent dir or a go module pointed to be a go mod replace in go.mod.

I don't really like root either: https://github.com/dagger/dagger/issues/5862#issuecomment-1762507181

But it handles cases source doesn't. But source also handles cases root doesn't 🙂 So there's probably some grand unified theory of path specification to be decided on here

It's essentially the same problem as the difference between the context dir and the dockerfile itself when using docker build. Except for us:

1. we point to your module's source code rather than a dockerfile
2. the context dir is relevant in determining whether your module source code can build in the first place (whereas in dockerfiles it just gives the dockerfile access to more stuff)

[analog grove]

you have multiple modules in a monorepo, but they need some files from parent dirs in order to build such as a go.mod in parent dir or a go module pointed to be a go mod replace in go.mod.

I didn't understand that part. My naive reaction is "why share files across modules in the first place?", seems like a niche requirement not worth over-complicating everything, but I'm probably missing something

EDIT: share

[hexed whale]

An example where that's useful is if all your modules are written in Go and you want them all to share the same go.mod in the repo root - they're not duplicating files in that case, they're deduplicating them

[analog grove]

Right, I used the wrong wording. I guess I meant "share"

[hexed whale]

Ah got it. Yeah that's fair to some extent, you could end up with a needlessly large go.mod containing the union of all dependencies for a wide array of modules, instead of having individual tightly-scoped ones.

Which could get tricky at publishing time (imagining you're versioning the modules independently)

[analog grove]

I honestly don't think it's worth it

On the one hand: a slight inconvenience at worst
On the other hand: compromising a core principle of how modules work (1 module = 1 directory, always)

[hexed whale]

Probably worth going over these to understand each use case: https://github.com/search?type=code&q=path%3Adagger.json+"root"%3A

One other case where it's used is when your module depends on a sibling module in the same repo; afaik you have to do root: .. before you can have ../foo as a dependency

e.g. https://github.com/vito/daggerverse/blob/5a53a93aaa08e2caf02c249fe045ee0ae613c85e/testcontainers/dagger.json#L6

[analog grove]

https://github.com/search?type=code&q=path%3A%2F^dagger.json%24%2F+"root"%3A

Slight regexp correction 🙂

[hexed whale]

oh that's only finding ones in the root of the repo

[analog grove]

oops

[hexed whale]

https://github.com/search?type=code&q=path%3A%2F\bdagger.json%24%2F+"root"%3A

[analog grove]

https://github.com/search?type=code&q=path%3A%2F\%2Fdagger.json%24%2F+"root"%3A

?

Yay we did it

[hexed whale]

https://tenor.com/uefx3JT0Irh.gif

[analog grove]

Hey @fading mortar why do you use root: ".." in your modules?

[tacit grotto]

I’m not sure it’s just an inconvenience, there are going to be monorepo cases where users strongly want/need a single common go.mod for all code, including dagger modules. If the equivalent of “root” functionality doesn’t exist then they can only have one dagger module in their monorepo

And also the other case i mentioned where you have a go mod replace that points to parent paths

[analog grove]

There's also a safety issue with allowing root to point to a parent directory. It could be used by a malicious module to access directories it's not supposed to.

[tacit grotto]

Remote modules that point to a root are scoped to the git checkout, can’t refer to the callers local fs

[analog grove]

Sure and maybe that's sufficient, but it's one more moving part to worry about, and one more rule for devs to learn

[tacit grotto]

It’s not something we had to go out of our way to do, it works like that because a git source in buildkit is inherently rooted at the git repository

Like I don’t think any of these are common cases but they aren’t rare enough to just say user’s can’t use dagger if they hit these rare situations that call for this

[analog grove]

Doesn't root also control what Host().Directory() will return inside the module?

[analog grove]

Yeah if the penalty is really "you can't use Dagger" then I agree that would be a terrible outcome. I wasn't sure if that's how badly this is needed, as opposed to a more subjective "my language community prefers it this way"

[fading mortar]

I think it's in kpenfound/greetings-api/ci/dagger.json just for the convenience of calling the module from the ci directory if you want. I don't think it has any effect on calling from the repo root (dagger call -m ./ci)

[analog grove]

Personally I have a repo full of Go modules, and it never occured to me once that I may want to share the same go.mod across them...

[analog grove]

But wouldn't that work exactly the same without the root field?

[analog grove]

I haven't needed a go mod replace though, so not sure how stuck I would be if I needed that

[fading mortar]

it might now, it's a pretty old dagger.json

[tacit grotto]

Right, but for people with highly opinionated monorepo setups where they control all dependencies across all their go code from one go.mod, telling them to abandon their opinions and setup just to use dagger seems likely to just result in them not using dagger.

Also worth keeping in mind the unknown of how package managers in every language in every SDK we could hypothetically want to support work. Which is to say, who even knows what requirements will come up.

I do think there's alternatives to "root" that give the same functionality.

One possibility that would integrate with source is if a single dagger.json could be used to configure multiple modules (in separate source dirs). Then you could have a shared dagger.json and go.mod in the root of your monorepo but still have multiple dagger modules in different source subdirs

Or as a slight variation, allow users to use different names besides dagger.json, which would let them put multiple dagger module configuration files in the root of the repo

[analog grove]

maybe a lightweight variation too, where a parent dagger.json can inject an overlay into its sub-modules, without hijacking the filesystem as the authoritative catalog of those sub-modules

[analog grove]

in other words, the answer to "what modules exist in this directory tree" would always be "whatever directories have a dagger.json", instead of "well it depends on what's in the parent dagger.json, and recursively in all dagger.json below it"

But yeah I like all those options 👆 better than root because it "inverts" the control, outer->inner instead of inner->outer