Nested Modules / Context Arguments

[sly wave]

My understanding was that if I have this in a Dagger module:

        /**
         * Run Bun install for the entire monorepo
         */
        @func()
        bunInstall(
                @argument({
                        defaultPath: ".",
                        ignore: [
                                "*",
                                "!package.json",
                                "!bunfig.toml",
                                "!bun.lockb",
                                "!**/package.json",
                                "**/node_modules/**/package.json",
                        ],
                })
                directory: Directory
        ): Container {
                return dag.nodeJs().withBun(directory).withExec(["bun", "install"]);
        }

Calling this module from anywhere in my nested monorepo without arguments would run with . being relative to the module itself

I even sure I've tried this and it worked

However, it doesn't seem to be working with 0.13?

[narrow lynx]

this was changed in https://github.com/dagger/dagger/pull/8391

@bitter narwhal @bleak pilot can we get a release note next time 🙂

sorry, that sounded snippy, that's not what i meant

[bitter narwhal]

That wasn't necessary because it's an adjustment to a PR that's in this release.

[sly wave]

I don't think that's the problem I'm facing?

[narrow lynx]

context directories were added in v0.12.6

[sly wave]

You're right, module source is now where dagger.json is

OK, I think I can fix this

[bleak pilot]

Amazing!

[bleak pilot]

Oh yeah, I forget sorry 😦

[bitter narwhal]

Oh, right! For some reason I thought it wasn't released yet so I didn't flag the missing change log.

[sly wave]

THIS IS AMAZING

    bunInstall(
        @argument({
            defaultPath: ".",
            ignore: [
                "*",
                "!package.json",
                "!bunfig.toml",
                "!bun.lockb",
                "!**/package.json",
                "**/node_modules/**/package.json",
                "**/dagger/**",
            ],
        })

[bitter narwhal]

Don't you need ** instead of just *?

[sly wave]

Nope, ** is for finding files or directories that are nested within

• is everything

I can now have a function in the root of my mono repo that only mounts project package.jsons, runs a bun recursive install and cache that node_modules for all future per project runs

and it's working nicely

not as fast as I'd like, but one step at a time 😄

[bitter narwhal]

What's the slowest part? Is it initialize?

[sly wave]

Hmm, perhaps my internet today

I'll share some traces

No cache: 2m https://dagger.cloud/RawkodeAcademy/traces/6769ed91afb18e4ba06cd673c0ecb633

With cache: 11s

https://dagger.cloud/RawkodeAcademy/traces/824066fa1f65ae233f85d714c5c68a19

2m is .sync ?

Which I assume is the actual bun install ?

[bitter narwhal]

Seems like the slowest part for you is bun install. 1m20s.

[sly wave]

Yeah

Let me try without dagger

[bitter narwhal]

Try with a clean bun cache.

2069 packages installed 😲

[sly wave]

2177 packages installed [7.81s]

Locally

after running rm -rf ~/.bun

Hmm, maybe that's not my bun cache

[bitter narwhal]

In uv (Python), it uses hardlinks locally (between the cache and project), but in a container it's different file systems so it has to copy the files instead. Do you want to try in a docker container (with a cache volume for the bun cache)?

[sly wave]

find . -name "node_modules" -exec rm -rf {} \; && bun install --no-cache

36s

@bitter narwhal I'll try that now, good idea

Fresh install is still 2 mins, but got subsequent installs down to 6s with using a cacheVolume for global cache

[stark bone]

dumb question: where does that exec bun install even come from? i don't see a withExec() for it anywhere in the trace 🤔

[sly wave]

It's part of the Void.sync?

I tried putting the global bun cache inside the node_modules cacheVolume to see if that could force hardlinks

but no improvement

[stark bone]

that's what forces it to run, but I don't see what actually installs the exec - normally there's a corresponding withExec(["bun", "install"]) somewhere

[sly wave]

    @func()
    bunInstall(
        @argument({
            defaultPath: ".",
            ignore: [
                "*",
                "!package.json",
                "!bunfig.toml",
                "!bun.lockb",
                "!**/package.json",
                "**/node_modules/**/package.json",
                "**/dagger/**",
            ],
        })
        directory: Directory
    ): Container {
        return dag.nodeJs().withBun(directory).withExec(["bun", "install"]);
    }

[stark bone]

hmm ok that's what I would have assumed, but it's missing from here for some reason

(sorry, this is a tangent, I'm just doing trace UI work, so paying extra attention to things like that)

[bitter narwhal]

I found it under .sync.

[stark bone]

that's the exec, not the withExec that installed it 🙂 - that one's nowhere to be found, mysteriously

[bitter narwhal]

Ah, I see what you mean now 🙂

[sly wave]

This one seems broken and I can't Ctrl-C