Dagger Cloud | Dagger | Page 1

tardy sedge Oct 22, 2025, 2:07 AM

#

Actually @quiet maple do you mind taking this? I sent out the change here: https://github.com/dagger/dagger/pull/11266

But it looks like there's a couple failures related to the various package managers that can be config'd https://dagger.cloud/dagger/traces/9c6a47d30202d29304695f01dbbc1aa9?listen=49fc26064eb5092e&listen=231f5e7f70ece9c9&listen=a956fa1a87f3ef68&listen=f6972a2b5e2c06e2&listen=18bd7ad2e7ee6b52&listen=89579cc80a13c837&listen=fd455d5a04a12f20&listen=d739107a63d1585d&listen=4fb77a3eb1e7e550

I don't want to get overly distracted from finishing up function caching stuff

quiet maple Oct 22, 2025, 2:08 AM

#

Hey sure, I'm also doing couple of things on the TS container so it's the right time

#

I have a branch almost ready to apply Yves' optimization with moduleTypeDef

tardy sedge Oct 22, 2025, 2:09 AM

#

There's two fixes in the PR:

don't reinstall npm if it's already present at the given version in the container
put the withSetupPackageManager step before mounting in the module source code, that way we can re-use those expensive steps across modules rather than re-run them for every different module source

tardy sedge Oct 22, 2025, 2:11 AM

#

quiet maple Hey sure, I'm also doing couple of things on the TS container so it's the right ...

Thanks! The fix there seems extremely worth it, but looking at the traces of typescript SDK steps I think there's probably a lot of other worthwhile optimizations

quiet maple Oct 22, 2025, 2:11 AM

#

We already setupPackageManager before mounted the source code 😮

tardy sedge Oct 22, 2025, 2:11 AM

#

quiet maple We already setupPackageManager before mounted the source code 😮

oh really? I might have misunderstood then, I just saw that it was after withGeneratedSDK

#

maybe that's different

quiet maple Oct 22, 2025, 2:12 AM

#

    return runtimeBaseContainer(cfg, t.SDKSourceDir).
        withConfiguredRuntimeEnvironment().
        withGeneratedSDK(introspectionJSON). <-- Generate client (because we need it)
        withSetupPackageManager(). <-- Setup
        withInstalledDependencies().
        withUserSourceCode(). <--- Here
        withEntrypoint().
        Container(), nil

We can't put the setupPackageManager before withGeneretedSDK because some dependency will be missing in case the SDK library is local and not bundle

#

And also because for a stupid reason, the way we install yarn using corepack will automatically also run an install that we cannot disable (because that's stupid)

At some point I think I might just drop yarn v2+ support so we don't have to deal with that anymore

tardy sedge Oct 22, 2025, 2:15 AM

#

I see, let me see if my PR passes if I get rid of that part and just leave in the npm install npm optimization

quiet maple Oct 22, 2025, 2:16 AM

#

The error you see here: https://dagger.cloud/dagger/traces/9c6a47d30202d29304695f01dbbc1aa9?listen=18bd7ad2e7ee6b52&listen=eeb855db53985ea1&listen=5073b2888022f206&listen=d0f900bba334aa3e&listen=25fdaccf7e97cc5b&listen=9364a1f745ca75a8&listen=5cd2c2d5df37de87&listen=a956fa1a87f3ef68#a956fa1a87f3ef68:L17

Is because of that, the local lib is installed after the setup so it created an issue

Dagger Cloud

Browse and visualize Dagger traces.

#

But ideally, I want to deprecate the local lib support, it creates so many problem for absolutely no gain

#

(And that way we could in theory setup the package manager at the beggining)

tardy sedge Oct 22, 2025, 2:18 AM

#

Makes sense, yeah whatever we can do to put the expensive steps before any steps that might invalidate their cache would be good

quiet maple Oct 22, 2025, 2:19 AM

#

Yeah, I don't think many users would complain if I just deprecate that, I could fail with an explicit warning to explain how to migrate to the new way

tardy sedge Oct 22, 2025, 2:23 AM

#

In the meantime, there actually is cache invalidation happening because we are in the previous step mounting to a path like /src/sdk/typescript/dev/node/sdk, which is gonna be different for each module. So modules can't share the cache for that step.

quiet maple Oct 22, 2025, 2:23 AM

#

Hmm

tardy sedge Oct 22, 2025, 2:24 AM

#

idk if it's possible to at least make that directory the same for each module or if it really has to be different

#

if not and it requires what you were just describing, that's ok, at least the npm install only takes 0.8s instead of 40s now

#

(for the default npm case)

quiet maple Oct 22, 2025, 2:24 AM

#

The SDK dir content will be different for each module (since they may have dependencies) so not sure it will cache hit even with the same path, so I'm more into moving up the setup by deprecating localDir

#

And deprecating yarn v2+ support

#

Green and approved

#Dagger Cloud