daggernauts

[sharp zealot]

cache volumes are always namespaced by module now. So that shouldn't be an issue

the only way for 2 modules to share a cache volume is by one module explicitly passing a cache volune as argument to another

[latent trellis]

I just realized the term "module" is a bit overloaded in this context.

In my case, it's two instances of the same module and I want them to use different cache volumes.

[sharp zealot]

Ah I see! Well in that case that's different, yes all instances of a module share the same cache volume namespace. That can indeed cause issues for toolchains that don't support a globally shared cache directory, but I believe Go is fine with that?

[latent trellis]

The problem is (or at least used to be with Dagger Cloud, not sure anymore) that when two different pipelines are running in parallel the last one wins and overwrites the cache (they don't get merged)

Is the distributed cache feature still available in Dagger Cloud? Or is it disabled?

[sharp zealot]

yes still available (although we're not taking new signups at the moment, it's still operational for customers who signed up earlier).

But in the Dagger Cloud org setting you can disable sync for specific volumes

but then you don't get persistence for that volume of course..

By far the most powerful solution would be to implement a dagger-aware tool that can download the go cache without cache volumes

[latent trellis]

Which is why I started using unique names for each instance. But the default volume kinda defeats the purpose in that case. Hence my original question.

[sharp zealot]

Then you just run that in your module and you delete the cache volumes altogether - layers only. that is my dream

[sharp zealot]

yeah I guess the pragmatic solution is to have the DX I suggested, and in the case where you use dagger cloud volume sync, pass different cache volumes as arguments for different instances of the module

that way you get a pragmatic deployment without sacrificing a portable clean module API

[latent trellis]

That might work for the module cache, not for the build cache.

Also, in my experience non-traditional tooling is often a pain to use if not well maintained enough and most build systems out there had to go through a really long journey to get to a maturity where the "average" user is confident enough to use it in the first place.

[latent trellis]

and that goes back to my question about modules initialized inside modules: https://github.com/sagikazarmark/daggerverse/blob/main/golangci-lint/main.go#L62

I either add constructor arguments for the dependent module.

The "nice" API would be passing the dependency directly, but that's not possible at the moment.

[restive shore]

The feedback from the TUI when publishing looks ominous and frozen. It's hard to detect what's going on because all of those spans looks frozen. When pulling an image I see the yellow dots and then green checkmarks. Can this be improved?

[sharp zealot]

Fun fact: when refactoring a Go Dagger module, LLMs get confused by dagger.TYPE vs. dag.CONSTRUCTOR()

[willow rune]

Hey is there some dagger command which is similar to docker system prune? Basically delete all cache, volumes, etc. Except restarting/removing the container.

[sharp zealot]

yes check out the engine top-level function:

dagger -c 'engine | .help'

[sharp zealot]

https://github.com/dagger/dagger/issues/10112

[spiral whale]

it could be good idea to version docs, i was using Quick Start docs of the dagger to onboard devs to dagger faster but with new version it replaced all cli commands with shell version only and it became confusing for the devs since we're still using v0.16.x

[willow rune]

Hey,
I have a monorepo with go and typescript services.
I pushed a commit and the change only happened is in a go service, so I only want the CI to trigger go to the appropriate related service.

Can this be done using GitHub Actions and Dagger.io?

This can be done using path filters in GHA, but I'm optimistically aiming for 1 GHA yaml which trigger the pipeline and dagger takes care of the which service has changed.

[sharp zealot]

At the moment there are 2 options:

1. No CI filters, Dagger job gets triggered for all changes BUT with good caching, it doesn't matter because Dagger correctly skips execution for unchanged inputs. This requires good cache configuration in your CI runner

2. Manual CI filters. Basically keep configuring GHA to filter by path. This is not ideal, should be a temporary stopgap only if all else fails.

3. Coming soon: Dagger Cloud will provide "smart events" telling you which Dagger module has actually changed. Then you can hook up your dagger pipelines to that. cc @dense canyon @lime comet @hallow gust

[willow rune]

Just to clarify if I opt for 1. having self-hosted CI runners is sufficient, since dagger will take care of it? considering I have optimised my dagger code to leverage cache.

[sharp zealot]

If you use the official kubernetes daemonset, and your kubernetes nodes are not too ephemeral, then yes you should see good results (although still best effort). We are actively working on major caching improvements

[willow rune]

I am actually using docker in locally and podman in CI. Sticking to basics.

[dense canyon]

@willow rune even though Dagger doesn't support it natively, it shouldn't be difficult to create a function that receives the GHA payload and runs whatever functions you need based of the changes that were introduced in the commit. Not sure if that might be an option for you.

[sharp zealot]

@upbeat herald 👋 where's your experimental Docker SDK?

I'm interested in taking a look for reference

[upbeat herald]

Hey: https://github.com/quartz-technology/daggerverse/tree/main/docker_sdk 😄

[sharp zealot]

@upbeat herald right off the bat: love the trick using constructor + default args to load useful directories in a less verbose way

[upbeat herald]

Yeah! Context directory is such a good feature! Really changed my way of building modules 😄

[sharp zealot]

@upbeat herald this snippet is... something. What does it do?

Is it basically a super complicated way to get the module source directory?

    modulePath, err := modSource.SourceRootSubpath(ctx)
    if err != nil {
        return nil, fmt.Errorf("failed to get module path: %w", err)
    }
    sourceDir := dag.Directory()
    _, err = modSource.ContextDirectory().Directory(modulePath).Entries(ctx)
    if err == nil {
        sourceDir = modSource.ContextDirectory().Directory(modulePath)
    }

[upbeat herald]

The module Source root directory yeah 😄 (location of the dagger.json)

[sharp zealot]

And discarding the error is to gracefully handle cases where the target module has no source dir?

I'm confused: does SourceRootSubpath return the path of the module root, or the path of the module source?

(because it has both source and root in the name...)

[upbeat herald]

It's in case of dagger init, since it might be an empty directory yeah or a non-created directory, meaning that modSource.ContextDirectory().Directory(path) will fails, but there's no straigh forward way to verify it exist so I do that instead

[upbeat herald]

module root path (location of the dagger.json)

[sharp zealot]

Ok so the word source shouldn't be there I guess

[upbeat herald]

SourceRoot

SourceSubpath = source code of the module, sourceRootSubPath, root of the modulesource

Source make sense IMO since a module is built from moduleSource

[sharp zealot]

Cool, let's add container as a prefix to every function of the Container type then

it makes sense, it's a container

😛

OK thank you I will copy-paste your snippet and never look at it again, I think you just saved me 2 hours of misery

(minus 10mn to complain about our API)

[upbeat herald]

Hmm it's because moduleSource has multiple important path, the contextdir, the sourceDir, the rootDir but rootdir != contextdir so it's named sourceRootDir because it's more clear that way that it's not the actual / root but the root of the source haha

contextDir == rootDir in a moduleSource

If user do modSource.RootSubPath, I'm not sure what they would expect to get

[upbeat herald]

Np! What are you building?

[sharp zealot]

All good, the important thing is that thanks to you I will never have to look at that API again

[upbeat herald]

Sorry about that haha! After my iteration on SDK/Codegen/Introspection I'll try to improve this area

[sharp zealot]

Actually sorry, I didn't understand this part.

How can the module's root directory not exist?

[upbeat herald]

dagger init --name=foo --sdk=go ./foo/bar/baz -> ./foo/bar/baz is the root source dir but doen't exist yet

[sharp zealot]

it's the module root dir - please stop saying the "root source dir" it's confusing, because there is a "module source dir" and a "module root dir" - which is already confusing enough without mixing them

OK let me try running that command so I understand...

When I run dagger init --name=foo --sdk=go ./foo/bar/baz it does create ./foo/bar/baz. But you're saying the SDK will be loaded before the directory is created?

[upbeat herald]

It seems yeah, I mean, I also have this logic in the Typescript SDK: https://github.com/dagger/dagger/blob/84e21c755d35f26b2a6c8288840121cc80a8be81/sdk/typescript/runtime/main.go#L882

[sharp zealot]

@upbeat herald @kind carbon I'm making a toy SDK (for dogfooding purposes), want to declare a function that returns a core type (say, directory), what would that look like?

The closest I have is from my "compose-sdk" I made during your workshop @kind carbon, it has a function that returns a string:

objType := dag.TypeDef().
            WithObject("compose").
            WithFunction(
                dag.Function(
                    // Function name
                    "db",
                    // Function return type
dag.TypeDef().WithKind(dagger.StringKind),
                ),
            )

[sharp zealot]

REQUEST: very very simple SDK skeleton, to help developers who want to experiment with the mechanics, but don't want to spend days just getting a skeleton to work. There is a LOT of undocumented arcane knowledge buried in there

[upbeat herald]

It should look like this : https://github.com/quartz-technology/daggerverse/blob/2fe856d7b645e25ad51ee86f20204e973d86dbbe/docker_sdk/src/module/docker/build.go#L125C2-L125C74

For your func that would be:

dag.Function("foo", dag.TypeDef().WithObject("Directory"))

[upbeat herald]

Yeah, we should have a doc for both making a "language SDK" and a "non-language SDK", the structure and what you need to do is very different ^^

[restive shore]

hey, so I'm running source.AsGit().WithAuthToken().Tags(ctx) and realized it won't show the remote tags because it doesn't do a fetch. Would it be possible to add a AsGit().Fetch(branch or all)? The AsGit is very useful but I can't use it because I need to fetch before doing much with the repo. The fetch itself requires a PAT so I have to set up all of that whereas AsGit().WithAuthToken() handles it.

[sharp zealot]

Would be worth an issue. My first instinct is to add source.AsGit().Remote("origin").Tags()

That way there are no side effects, seems less messy

[restive shore]

I'll open an issue. I was also checking out this https://github.com/dagger/dagger/pull/10195 where Jed is making inroads into adding some of this functionality

https://github.com/dagger/dagger/issues/10200

[restive shore]

In shell .refresh is not working for me after I make code changes. Is that bugged?

[sharp zealot]

Are the code changes to the module itself, or to files that are loaded as arguments?

[restive shore]

to the module itself

[sharp zealot]

Ah yeah, it might be a bug then. I personally never use .refresh, so haven't noticed. cc @kind carbon

[restive shore]

Right now, I have to exit and then dagger again. But when I do that I have to re-create the variables I set.

[sharp zealot]

Yeah that's a pain

When I find myself in such a loop, I usually write the script to a file

[restive shore]

I'll probably do that for now. Thank you Solomon!

[restive shore]

I am seeing a strange behavior when using a forked repo as a ref. When I point a Directory arg to git fork, it sets the origin to upstream instead of the fork. Even if I explicitly tell it to use my default branch like github.com/private_org/repo.git#main it points to github.com/upstream_org/repo.git.

but,

If I use a branch other than default, it sets the right remote github.com/private_org/repo.git#custom_branch works.

[void widget]

mmmh thanks for finding that out 🙏 Could you file an issue so that it doesn't get lost ?

[void widget]

Yeah i've been bit by it too -- currently trying to fix it 👀

[restive shore]

Sure thing! https://github.com/dagger/dagger/issues/10208

[latent trellis]

Looks like dagger update doesn't update a module, the source specifies a version. For example, github.com/sagikazarmark/daggerverse/helm@helm/v0.13.0

Running dagger update helm. does nothing.

Either I remove the version manually or run dagger install github.com/sagikazarmark/daggerverse/helm.

Is that intentional?

I could see it working either way, but I find the current behavior a bit unintuitive, especially since there is a "pin" field present in dagger.json.

[mossy bay]

Can dagger translate (compile? dagger-late?) one CI event into a different CI event, such that I could use it to syndicate events?
For instance, Could I syndicate a Github Event into a mirrored GitLab repository?

[wintry prism]

Can dagger translate (compile? dagger-

[restive shore]

This must be an issue with tags because if you specify a branch or a pull like github.com/org/repo@pull/123/merge it does update the pin.

[sharp zealot]

Yes I think it's unfortunately designed that way

The idea I think is that if you installed a module at a specific branch or tag, update means "give me the latest commit for this branch or tag".

[sharp zealot]

it's pretty cool when you installed from a branch. But I agree it's confusing when you installed from a tag

[deft rain]

mm, i seem to remember designing it with the idea that we could/would do this later 🤔 would it make sense to always do this automatically? and have different behavior for branches/tags?
another idea i had was to add --patch/--minor/--major/--dev flags to control exactly what new versions are allowed (which would allow for exact control, bumping from v0.1.0 to v13.4.0 probably isn't super desirable as the default for example)

[subtle plinth]

Is there anyway to expose a dependency module function to a local function without having to wrap it in every downstream dagger module?

Example: I have a module for generating migrations with drizzle-kit, but in every single project I want to use it I'm writing the same boilerplate to wrap it and call it

I know there's the verbose action, such as dagger call -m "../../daggerverse/drizzle-kit", but it would be nice if it was easier

[sharp zealot]

my dream

[subtle plinth]

Mine too. Without it, we're right back to needing mage or Justfiles and it feels like we're going round in circles a little bit.

[sharp zealot]

I don't think mage or justfiles solve this at all.

Dagger doesn't have zero boilerplate (yet) but it gets you pretty close and will keep getting closer.

You can move everything to a shared module and call it from each app if that's your preference. The main feature you lose is contextual directories which makes me sad. But we will eventually find a solution.

I think it's reasonable to write a little bit of entrypoint code for each of your apps. After all, you're writing code for the app.. Makes sense to also write code for building & testing it. Of course the less boilerplate the better.. Dagger as a platform is striving towards that , like your app platform hopefully.

[wintry prism]

Not sure if same thing, but I was imagining getting any module directly from the dag:

dag.m("github.com/foo/bar").myFunc("baz")

[sharp zealot]

you can load modules dynamically, the dagger API supports it. But it won't work the way you're hoping because your code won't have type information for the module. So that snippet is impossible without some sort of codegen like the one provided by dagger install

[subtle plinth]

The problem for me is my Monorepo. I have many services that all need the same migration action, the same deploy action, and the same format and lint actions .

Writing a little bit of code is fine, repeating this N times isn't.

Of course, maybe I'm doing it wrong and I am hoping to find a solution

Perhaps I shouldn't be writing a Dagger module for each service and instead orchestrate it at a different level, perhaps per project

But I feel like Dagger shouldn't care too much?

[sharp zealot]

> I am hoping to find a solution

I just saw your very public post about uninstalling dagger from your all your monorepos, in favor of another tool that "actually works".

So I suppose that was the solution in your case 😁

Sorry it didn't work out.

[subtle plinth]

This post isn't a "Dagger sucks" post, it's just not what I need today.

I share all my changes publicly

Dagger team are smart, they'll do what they need to make it better; but I needed to make things work now.

I am not trying to cause offence and my end goal is to have Dagger power it all, but right now that isn't possible

[sharp zealot]

I don't care what you post. If you want feedback on your posting style I'm happy to give it to you in another setting.

Right now I'm just trying to clarify the status of this support conversation. Should I continue investing time in it? It looked like you were still trying to find the right way to use dagger. But that post makes me think it's too late? Different time investment for me.

[subtle plinth]

Ok. The "actually works" was in poor taste and shouldn't have been posted, reading it back it provides FUD and that wasn't my intention or what I wanted to portray. My apologies.

You are right though, it is too late for right now. I needed the ability to share tasks across many services and another tool makes that possible today.

Thanks for your support

[restive shore]

hey there! What's the best way to get auth set up inside a container for private go modules? Is it upto me to configure it inside the container? Something like this? - https://github.com/dagger/dagger/issues/9858. I was somehow under the impression that for Go there's a better way. This is for PAT based auth. There's the ssh_auth_sock for SSH I believe?

[deft rain]

the thing that was for go was for the internal builder of your code to access the ssh socket - you still can't get it inside a container automatically

[restive shore]

Crying in PAT auth :*(

[sharp zealot]

@dense canyon it's back 😭 😭 😭 😭

(unreliable remote engine when using remote docker + high latency internet, ie. on a plane)

[dense canyon]

Agh... forgot about keep investigating this one. Opening an issue for that

[sharp zealot]

But I remember that you opened a PR with a fix? I just couldn't test it at the time, because I was afk

[dense canyon]

Yes, I opened a PR to minimize the error but it can still happen if the initial 10s timeout is reached

My initial findings led me to some stdio connection handling in buildkit but I didn't go through the rabbit whole

I bumped the timeout from the Dagger side to minimize the occurrence of that happening but seems like I need to go all the way on fixing this

[warped canyon]

Just hit this bug on 0.18.5: https://github.com/dagger/dagger/issues/10298

A required argument to my function is missing from --help in dagger call but it shows up in shell .help <function>

Actually it's worse, I just added a comment. My module doesn't work with dagger call but it works fine in shell

[warped canyon]

I'm also seeing a codegen problem on 0.18.5, opening an issue for that now

[warped canyon]

May be specific to typescript, I'm not positive https://github.com/dagger/dagger/issues/10300

[sharp zealot]

@upbeat herald looking at your magic-sdk / docker-sdk implementation, it looks like you built a whole little framework for yourself, to abstract away the messy details of implementing a dagger runtime. I tried to use it, but it felt tightly coupled to your particular runtime. Do you think it would be possible to decouple it? Or not worth the effort?

[upbeat herald]

I made the implementation as extendable as possible, most of the code except the actual object definition could be abstract easily.
If you can point me to what part of the code you're interested to split, I can do it 😄

[void widget]

I'm interested too 👀

[sharp zealot]

that's the thing, I don't know which part of the code, because I didn't fully understand which part of the code is specific to your implementation, and which part can be reused for other runtimes (like mine)

[sharp zealot]

@south helm just checking, in the latest release it looks like shell variables are not available by name to the CLI agent right?

eg. this doesn't work:

myctr=$(container | from alpine)
> I gave you a container called myctr. Do you see it?

[warped canyon]

this works for me on 18.5

✔ myctr=$(container | from alpine) 0.0s

● please install git in $myctr

[sharp zealot]

yeah because you're relying on $myctr to expand

I don't to rely on that expansion because it's confusing to users

try please install git in myctr

[warped canyon]

looks ok with claude 3.5 https://v3.dagger.cloud/kpenfound/traces/5ce7bed37afc920fdd196c7a9a350c8b#f93246aeb98c3821

[south helm]

also seems to work on gpt4.1 https://v3.dagger.cloud/dagger/traces/430cae73fdac49dccb9d0a22dd2ae954#670881924b5d4865

[upbeat herald]

Do you have a public repo of your implementation? I can have a look to check

[dim falcon]

I have the same issue. I would like to automatically update my Module dependencies with renovate or dependabot.
I tried to update to the "latest" version. E.g. dagger update github.com/sagikazarmark/daggerverse/helm@latest
This did not work.
To run renovate properly I'd need to get the latest version of a module on Daggerverse.
This, because I have to know about the new version before I run dagger update
Is there an API or URL providing this information? (Getting the target of the link "Go to latest version" is no very useful)
Thanks!

[subtle plinth]

Is there a heuristic for when to use with directory vs with mounted directory?

[wintry prism]

Yes! If you don't need the contents of the mounted directory in a final directory or container image then use with-mounted-directory for better performance. The mounted overlay will go away when you publish, export, etc.

If you DO need the contents to be persisted, then you should use with-directory as it will actually copy the contents in.

[subtle plinth]

Ah. Thank you.

So it's just a Docker file bind mount thing. Perfect!

[wintry prism]

Yep, except not bi-directional mount back to the source (#sandbox). Just a mounted snapshot.

[subtle plinth]

Yeah. I actually tried to sync to host. Works with withDirectory and then export which is sufficient 😁

[wintry prism]

nice, that prob deserves a diagram/snippet in docs to show the flow.

If you want to do this, here's how you do it.

[sharp zealot]

@warped canyon did you ever find a way to connect to a service from the function that created it?

I can't remember where that saga ended

I have the same problem now 😭

[warped canyon]

can't remember the context on that... do you remember what it was for?

[sharp zealot]

Something related to an agent or demo you were building... trying to remember

[warped canyon]

not sure. Is it related to self-calls?

[sharp zealot]

can't remember

@chrome pilot @mossy hazel @dense canyon this 👆 is a very common situation... Information lost in the discord & github firehose. I know "RAG" is not trendy anymore, but an agent with basic information retrieval with specialized knowledge (ie. Dagger development) would be super useful on its own

btw information retrieval can be done with tools, doesn't have to be old-school pre-prompt RAG

and information retrieval can be done on artifacts like files and directories, which dagger is very good at

[sharp zealot]

@upbeat herald I just used the client generator for the first time and it feels like magic 🙂

I encountered a few issues:

1. Syntax of client install doesn't match the spec
2. Go generator fails when the dagger module is not a go module, but rather a subdirectory of a go module
3. Generated client doesn't expose dag.Host(), which breaks code that used that feature of dagger.io/dagger

[sharp zealot]

Also:

4. It would be nice to have a separate generated file for each dependency, plus a separate generated file for core. They can all be in the same package & directory. But splitting them that way makes debugging and customization easier.

[sharp zealot]

Actually it's failing to query:

panic: returned error 422: {"data":null,"errors":[{"message":"Cannot query field \"stdio\" on type \"Query\".","locations":[{"line":1,"column":7}],"extensions":{"code":"GRAPHQL_VALIDATION_FAILED"}}]}

(stdio is the name of my dependency)

[sharp zealot]

Generated client doesn't expose dag.Host(), which breaks code that used that feature of dagger.io/dagger

If anyone needs this in Go, I manually extracted the missing Host part, you can drop the file alongside dagger.gen.go:
https://github.com/shykes/x/blob/main/mcp-sdk/cmd/mcp-runtime/internal/dagger/host.gen.go

[sharp zealot]

Does anyone have any experience using the experimental generated clients feature? I can't get mine to work..

I'm wondering if it's because I'm trying to use it in the context of a dagger runtime. Maybe there's something that fundamentally prevents it from working in that context?

[sharp zealot]

It's this error 👆

To repro: dagger -m github.com/shykes/x/mcp-sdk/examples/think-mcp-server functions

[void widget]

It's gonna be this weekend rush 👀 so not yet

[upbeat herald]

Host exposed PR in https://github.com/dagger/dagger/pull/10326 😄
/cc @deft rain @kind carbon @rocky harbor for review

[wintry prism]

I have the same issue. I would like to

[sharp zealot]

I opened an issue for the 80k char truncation: https://github.com/dagger/dagger/issues/10328

cc @south helm @thorn moat @void widget @muted plank

[restive shore]

Hey folks, seeing something strange with the latest build (v0.18.6)

I'm trying to build and push our custom image (no other changes since 0.18.5 which worked fine) and seeing this

failed to resolve git src: failed to resolve git src: select: Get "https://github.com/....-daggerverse/info/refs?service=git-upload-pack": dial tcp 140.82.113.3:443: connect: connection timed out

I haven't see that git-upload-pack before in logs. What is it?

[rocky harbor]

Is it consistent? It looks like a possible ephemeral networking error. git-upload-pack I'm not sure off the top of my head but seems like some internal git protocol endpoint. I know we expose more error details around git ops now after recent changes (though they were present in v0.18.5)

[restive shore]

I tried 2 times. Let me try again

[rocky harbor]

@deft rain is out today (silent ping) but back tomorrow, he is most familiar with the changes made to git implementation recently

[restive shore]

Let me also try 0.18.5 again just to be sure nothing changed on our side

0.18.5 still works so it is a regression in the engine.

[atomic citrus]

I'd love to have that feature too!

[atomic citrus]

For now, i'm doing something like this but if we could use Directory.walk instead, it would much better.

import dagger
from typing import AsyncGenerator
from dagger import QueryError
import pathspec


async def walk_directory(
    dir: dagger.Directory,
    current_path: str = "",
    ignore_spec: pathspec.PathSpec | None = None
) -> AsyncGenerator[str, None]:
    """
    Recursively yield all file paths inside a Dagger directory.
    Yields relative file paths like 'src/main.py' or 'Dockerfile'.
    """
    try:
        entries: list[str]  = []
        if current_path == "":
            entries = await dir.entries()
        else:
            subdir = dir.directory(current_path)
            entries = await subdir.entries()
    except QueryError:
        return

    for entry in entries:
        full_path = f"{current_path}/{entry}" if current_path else entry
        # Skip files or directories matching the ignore_spec
        if ignore_spec and ignore_spec.match_file(full_path):
            continue

        try:
            subdir = dir.directory(full_path)
            _ = await subdir.entries()
            # It's a directory — recurse
            async for path in walk_directory(dir, full_path, ignore_spec):
                yield path
        except QueryError:
            # Not a directory — treat as a file
            yield full_path

[sharp zealot]

You can use Directory.glob() it supports a single include glob pattern. Better than nothing

[dense canyon]

I've traced the buildkit code and changing this (https://github.com/moby/buildkit/blob/master/cache/util/fsutil.go#L116) to make a recursive call enables Entries to be fully recursive and return the whole directory tree. Having said that, not sure what the implicancies of changing that is. cc @rocky harbor @deft rain

[deft rain]

it should be trivial to extend glob to take include/exclude patterns

I think the buildkit call allows those args

[dense canyon]

yep, it's here. https://github.com/dagger/dagger/blob/586956a5dd71574ccd7842a6d39b9af197fa1c33/engine/buildkit/ref.go?plain=1#L607. Should be trivial to implement indeed

[subtle plinth]

Can someone help me explain why this first example workds, but the second doesn't? From the docs, I feel like it should?

Thanks!

      dagger <<.
        . | build | export dist

        ../../dagger/cloudflare | deploy \
          ./dist \
          wrangler.toml \
          env:CLOUDFLARE_PAGES_TOKEN \
          env:CICD_COMMENT_TOKEN \
          --merge-request-id=env:$CI_MERGE_REQUEST_ID

vs.

        dist=$(. | build)
        ../../../dagger/cloudflare | deploy \
          $dist \
          wrangler.toml \
          env:CLOUDFLARE_PAGES_TOKEN \
          env:CICD_COMMENT_TOKEN \
          --merge-request-id=env:$CI_MERGE_REQUEST_ID

[wintry prism]

Can someone help me explain why this

[sonic vessel]

Is it possible to include file/directory from outside module? My use-case came from Earthly that it can reference with relative directory like this:

FROM ../..+task-image-a

[deft rain]

is it within the same git repo?

if so, yes, you can do this using a contextual directory

if not, then no, you need to pass it in as an arg

[sonic vessel]

Oh great to know. Thank you. 🙏

[wintry prism]

@unique nymph 👆
Couple of relevant docs
https://docs.dagger.io/api/custom-types
https://docs.dagger.io/api/interfaces

If you have a simple code example of what you want to do (or detailed description), it could help a lot.

[unique nymph]

i asked chatgpt for some help tidying my dagger module. it hallucinated a wonderful dream (which didn't work) but i feel it could be made to work.

it suggested defining this class (with hallucinated input_field decorator)

---

@object_type
class WPConfig:
@input_field
def admin_user(self) -> str:
return "admin" # default value if not specified

@input_field
def admin_password(self) -> str:
    return "password" # default value if not specified

@input_field
def admin_email(self) -> str:
    return "[e-mail]" # default value if not specified

---

define a function like this

@object_type
class Wordpress:
@function
async def setup_wordpress_stack(self, wp: WPConfig) -> dagger.Service:
pass

---

use it like so:

dagger call setup-wordpress-stack
--wp.admin_user=admin
--wp.admin_password=1234

[unique nymph]

Thanks. They seem to cover different use cases from what I was looking for.

[rustic thicket]

is your module public where we could take a look?

[unique nymph]

i was developing something. and i've changed the code to make it work. its more verbose. what can i do is go through my chatgpt history. and perhaps share a portion of it

[restive shore]

Verbosity of --progress=plain is still very unreadable. I see a lot of duplicate spans, on top of that, if I have a Stdout() and a dagger.File() in the same function it shows me the stdout 4 times. The Stdout was necessary for my test results to show up in the TUI. When I run this in Jenkins, it's virtually unreadable :(. Would it be possible to tune that way down? When I show that output with a comparison of a regular pipeline output, it's a huge turn off.

[thorn moat]

--progress=plain too verbose

[subtle plinth]

How does one debug "Encountered an unknown error while requesting data via graphql" error? I'm seeing nothing of value in my logs 😦

[dense canyon]

Have a trace by any chance? Do the engine logs show anything interesting?

[subtle plinth]

I was passing a container to a function that wanted a directory 🤦‍♂️

I did run into an interesting gotcha though. My Bun function mounts /code/node_modules via a mountedCache.

I was calling bun install and then using directory function on it to get my code with node_modules, but you don't get back the mountedCache.

Is there a way to get everything?

I've removed the mountedCache for now but if I was using a third party module, I'm curious how I'd handle that?

[warped canyon]

to restate the question to make sure I have it right, you want to get a Directory out of a Container that includes a mounted cache volume? AFAIK the only way to do this is to do a copy (like withExec(['cp', '-r', '/mnt/cache', '/code/node_modules'])) but that can be really slow for something like node_modules where there's a huge number of files

I think the best way to handle that with a 3rd party module would probably be to pass around the Container instead of the Directory, that way the cache volume stays in the DAG until you need some artifact out

[subtle plinth]

I've revisited this code and it's tricky to get this right. I am now re-using the container, but it means I can't use independent modules for different steps.

My app uses a Bun module, which runs Bun install.

I then wanted to use a Cloudflare and WunderGraph module to handle deployment and supergraph updates, but these both need the node_modules dir, so I guess my option is to:

• Remove the node_modules cache
  or
• Pass these modules my container and assume I can install wrangler and wgc
  or
• Something else I've not thought of?

[dense canyon]

@subtle plinth have you considered passing the node_volumes dagger.CacheVolume around and mount that where needed?

cv := dag.CacheVolume("node_modules")

bun.Install(cv)
cloudFlare.Deploy(cv)
wunderGraph.Deploy(cv)

^ all those modules can just mount that CV in their respective container and do whatever they need

if you're looking to re-use the same node_modules in all the modules

[subtle plinth]

Oh, good idea

Thank you

Let me see how it feels 🙂

[warped canyon]

One thing to watch out for with that - if multiple containers are using that cache volume at the same time and they have different dependencies, the node_modules can get changed and node apps tend to get mad about that

[subtle plinth]

Is this still being discussed? I like it's direction for solving the Env / Secret chaos that exists ATM

https://github.com/dagger/dagger/discussions/10370

[sharp zealot]

yes absolutely. want to make progress on it next week

[honest hearth]

Is it possible to get a nonbuffered stdout stream from a container?

use case is streaming LLM responses back for rendering

[dense canyon]

not currently from the Dagger SDKs. There's some requirements for things like that here https://github.com/dagger/dagger/issues/6553

[honest hearth]

not currently from the Dagger SDKs.

[sharp zealot]

Update: Blueprint Modules (formerly known as "platform modules" and "inline dependencies") are making progress again. This is to help platform teams daggerize large codebases with many similar components, without getting overrun with boilerplate. Updated issue: https://github.com/dagger/dagger/issues/10464

[sharp zealot]

The PR is ready for review. I included test instructions if you want to try it.

cc @restive shore @sweet nacelle @warped canyon @plucky ermine

[sweet nacelle]

Thank you Solomon!! Will do 🙂

[restive shore]

My colleague is getting this error on a new Macbook. I can't reproduce it on mine. Does this make sense to anyone here?

dagger core version works for them.

[sharp zealot]

Could be related to uploading local files to the engine. Impacted by 1) workdir 2) location of nearest .git in the findup.

[warped canyon]

I was going to suggest dagger -M core version but it seems like the global -M flag doesn't work in core 🤔 is that a bug?

[restive shore]

Hm, I'll confirm but they were not in $HOME. It was a subdirectory which is a git repo.

[deft rain]

dagger core never needs to look for a module (so -M would always be the case if it existed, so it's not)

[deft rain]

what's the pwd

[restive shore]

I'll get that from them.. I'm acting as middleman as they don't have access to discord 🙂

[sharp zealot]

And it's merged 🙂 Module blueprints will ship in the next release (planned for this week)

[restive shore]

Following up on this. That person just got a new laptop and a restart fixed their issue 🤷‍♂️

[sweet nacelle]

This is big news for us Solomon! Thank you so much for getting this across!

[sharp zealot]

FYI @subtle plinth this might interest you, based on your issues with dagger modules boilerplate 🙂

[subtle plinth]

Fantastic feature! I have 20 micro services all using projen to generate the same Dagger module, so glad I can remove that soon 😄

[void widget]

Hi @sweet nacelle,

I’ve been playing with your dagger-deps-example and the GraphQL-ID trick you use to pass a live Secrets-Manager into downstream modules. My rough read is that the workaround exists because Dagger still lacks a first-class way to inject module-level dependencies ?

• If the new Env.withBinding() + generic ID proposal (dagger/dagger#10370) lands, would that fully cover your need—or are there edge-cases it misses?
• Does any part of your approach overlap with the unified private-package auth work (dagger/dagger#10693), or are they orthogonal? (Why do you need this injected secrets manager for, usually ?)
• Finally, if the engine shipped with an opt-in “engine-side auth” layer (credentials bundled into the engine itself), would that eliminate the ID-passing pattern for you?

[sweet nacelle]

Hi @void widget thanks for having a look at that. The main goal here is to implement a pattern that we often see in application which is being able to create a dependency and inject it into the constructor of another type that requires it.

In the dagger setup we intend to roll out at nine, we forsee having 2 major layers in dagger modules:

1. workflow modules
2. component modules

The workflow modules are like CI templates and will be eventually installed by the new blueprint feature to our repos.
They are meant to be a "single point of entry" i.e. most of the time repos will not use any other dagger modules directly.
The component modules are reusable components that the workflow modules will consume to get the job done.

We want to ship CI that can run both locally on the developers machine and remotely in the CI. However we have to also acknowledge that the environments are different. For example when authing locally to AWS we use the ~/.aws/credentials file, but when authing in CI we use the node role IMDS. Our AWS component module can support both, but because workflows are a single point of entry for all repos, we must proxy configuration flags through our workflow module to the AWS module. This is less than ideal as it requires us to add lots of extra flags to the workflow modules. A more elegant solution would be to construct the AWS module outside of the workflow (via dagger shell) and then pass it in. This is the approach we want to take.

The need to do this graphQL hack arises from then wanting to pass the AWS module a level deeper to any component module that is interested in it. For example one of our component module runs post deploy tests. As part of its running, it needs to pull from our private ECR. By passing it the AWS module, it can obtain the ECR credentials to do this.

I took a quick look at Env ( though I am not super familiar with it ) and it looks like an interesting concept. Not sure how it would fit into this picture persee but it could be useful. Feel free to correct me, but my rough understanding is that it is like a "context" object but more of a dagger one. You are able to attach containers, directories and other resources to it, then pass it through to anything that requires it. I am aware that Envs were originally intended for LLMs but now are being reworked for more general use. Something like this could be desirable and may eliminate the need to pass things around by interface.

Regarding your last point about an "engine side auth layer"... yes I believe this would eliminate the need to pass by interface. Right now we are simply trying to easily authenticate component modules to talk to our secrets manager, AWS and GCP.

[sharp zealot]

@sweet nacelle it seems like this issue would be of interest to you: https://github.com/dagger/dagger/issues/8740

[sweet nacelle]

will take a look 🙂 Thanks @sharp zealot

@vivid coyote fyi

[sweet nacelle]

Dropped a comment on that issue thanks @sharp zealot 🙂

I have no idea if what we are doing is bad or not, but I guess the core problem we are trying to solve is auth. Certain modules hold an auth context for things like AWS, GCP and the secrets manager. We dont want to feed loads of parameters constantly throughout the code to give every module that might need these resources access ( and all the combinations of parameters depending on the auth method e.g. CI vs Local ). Passing by interface was the most obvious way to solve it right now. If it were easier to fetch these auth contexts from the engine runtime itself, this would probably change the game for us 🙂

[void widget]

cc @dense canyon -- regarding https://github.com/dagger/dagger/issues/10693. Another validation that users request an engine auth layer. Orienting the proposal towards that, it does make sense 👍

[sweet nacelle]

Hey guys just to let you know. I am on leave for about 6 weeks so if you have any more comms please direct them to @vivid coyote

[dense canyon]

take into account that auth in this context means explicit set of credentials which the pipeline requires so it's able to accomplish its task. I think we should differentiate those kind of secrets that are not directly part of the pipeline and are usually "org/team deployment specific" like image registry tokens, SDK specific package manager tokens, etc. vs other secrets like the ones Chris is mentioning which here which are more module / pipeline specific like AWS, GCP etc.

[restive shore]

@sharp zealot found a bug with blueprints. dagger update always sets/resets dagger.json to direct module path. If I set a PR or branch like module@pull/123/head and run dagger update it resets to module

I have to delete dagger.json and re-run dagger init with the PR refspec to use it

[restive shore]

Can dagger init --blueprint be a no-op if the module is already installed? Use case example: We have a common Jenkins pipeline that will make use of --blueprint and when a consumer of that pipeline has already installed (either blueprint or regular) module in their project, what's the best course of action for me to identify that and not re-init? I would like to support folks that don't use dagger at all in their project too and I don't want to maintain two pipelines for it. If dagger init --blueprint ignored an already installed module I think I can achieve that. Thoughts?

[sharp zealot]

I think at the moment, dagger init will fail on a module that's already initialized... I can look into it but not sure how big of a lift it is (I only pulled off this feature by making a very minimal change to the load-bearing init & dependency management logic)

Side note, I wouldn't recomment running dagger init at runtime in CI, that seems like an anti-pattern to me

[restive shore]

How would I use a blueprint module in CI then?

I guess by asking everyone to install it?

[sharp zealot]

Yes that would be the idea, since it's just one small dagger.json and no boilerplate, seems more realistic

[vivid coyote]

Hey guys, should secret file mounts .WithMountedSecret() be accessible through .File()?

I got stuck today trying to access one this way, thinking it was possible as the comment on File() states "Mounts are included" , but that was not the case here.

I did some testing on the mounts:

#!/usr/bin/env dagger

# --- Directory mounts --- ✔
.core | container | with-mounted-directory /mounted-dir . | directory /mounted-dir | entries

.core | container | with-mounted-directory /mounted-dir . | mounts

# --- File mounts --- ✔
.core | container | with-mounted-file /mounted-file test-file | file /mounted-file | contents

.core | container | with-mounted-file /mounted-file test-file | mounts

# --- Secret mounts ---
# ✔
.core | container | with-mounted-secret /mounted-secret file://test-file | from alpine | with-exec "cat,/mounted-secret" | stdout

# ✘ no mounts listed
.core | container | with-mounted-secret /mounted-secret file://test-file | mounts

# ✘ fails at `file` with "empty reference"
.core | container | with-mounted-secret /mounted-secret file://test-file | file /mounted-secret | contents

[dense canyon]

👋 no, neither Secrets nor CacheVolumes can be retrieved. The "mounts are included" is referring to WithMountedDirectory

[sharp brook]

@warped canyon your proxy module works in series for every withService, right? I could shave a few seconds off my startup time if that were parallelized.

[warped canyon]

I think technically in the module the services would be started in parallel by the engine, however when the services are passed to the module from your module they're each resolved individually I think. If anyone has some ideas on how to avoid this I'm all ears 🙂

afaik this happens because each time with_service(Service) is called the service being passed in is fully resolved. I wonder if a with_services([]Service) would resolve them in parallel in the engine? not sure

[dense canyon]

I just did a quick check and services are already being started and health-checked in parallel

with and without the proxy module cc @sharp brook

[sharp brook]

It looks serial to me

[wintry prism]

[restive shore]

A colleague pinged me on this. Is there a limitation of dagger shell not being able to refer to directories/files outside of the module root?

[dense canyon]

https://github.com/marcosnils/dagger/blob/997d9f7def31513953476de13b165f362d6a40dc/core/services.go?plain=1#L202

it's in parallel. Those WithService calls are the calls to register the services in the engine / the proxy module. It doesn't effectively start them

services are started when the DAG needs to be resolved which is when the StartBindingscall that I linked above happens

[wintry prism]

hmmm works fine for me to do something like

dagger -c 'grep-dir .. foo'

demo ⋈ grep-dir .. foo

[restive shore]

Shell dagger root escape

[warped canyon]

I'm running my slides in a dagger terminal now which is super awesome but for some reason when i exit the terminal dagger hangs 🤔 any ideas?

dagger -m github.com/kpenfound/slides call slides --presentation self_healing_ci.md

https://github.com/kpenfound/slides/blob/main/.dagger/main.go#L25

[restive shore]

Should there be an example here to showcase that you can pass a context to the constructor and also return an error? https://docs.dagger.io/extending/constructors?sdk=go

[sour citrus]

I'm daggerizing a docker compose stack.
The db component of my stack has a healthcheck and my app is waiting for the check to pass.
Is there a way to do that? I'm not seeing a way to easily run a command inside a started service

[dense canyon]

not currently at the Service level. Healtchecks could definitely use an improvement here. The way that I currently do this is by running a healcheck container after the service has been started or adding a healthcheck step after binding the service to a container to verify that everything's ok

healthchecks in general are a bit tricky since in some cases you might not even have the proper tools to perform the healthcheck at all in the service container. So there's been a never-ending back and forth about how and where healthchecks should effectively be performed.

[sour citrus]

In my case it's a healthcheck on a DB service. So it can't be from the outside, it's not an exposed endpoint, and it's tight to the entrypoint doing things.
But yes I agree, the question of tools is a good one (and also a reason why a lot of services don't have health checks, especially when you have something based on some kind of distroless)

[sharp zealot]

another tricky use case is db migrations... hard to do cleanly right now

[dense canyon]

I guess in this case you already have the tools in the service container, correct? Funny enough the RunningService sturct has a way to Exec stuff into it (https://github.com/dagger/dagger/blob/b9a64f6bf2dff6e9c851b9b5a73d569c163f8fa2/core/services.go?plain=1#L69-L71). I don't think we exposed that API yet since that probably requires some high level API design

[spiral whale]

Hey guys, prune cache button from dagger cloud is not working, i'm trying to prune caches from our cluster, how can i do it?

[deft rain]

by not working, do you mean you can't find it? cc @dense canyon i think we removed it

[spiral whale]

no there is button but it returns prune failed error

[dense canyon]

@spiral whale that button is in effect not supported anymore. We've stopped serving the distributed cache service for a while now as we're working on new primitives that will allow us to provide a better service

[spiral whale]

okay debugging some go 1.25 tooling chance issue, i was resetting some caches. When i saw button i thought it's still active

[dense canyon]

I'll remove it in a bit

[spiral whale]

From the Go 1.25 Release Notes:

“The Go distribution will include fewer prebuilt tool binaries. Core toolchain binaries such as the compiler and linker will still be included, but tools not invoked by build or test operations will be built and run by go tool as needed.” 

This means tools like covdata, which weren’t previously invoked by default, are no longer bundled in the distribution. Instead, the go command will compile them on demand using the build cache.

When we combine this release with go build cache we're getting errors similar to go tool covdata: fork/exec /root/.cache/go-build/.../covdata: permission denied. The question here, are we mounting cache volumes with noexec option or is there such a limitation in k8s setup of the dagger engine?

We saw this issue only in our k8s cluster with our DS dagger engine setup

cc: @deft rain @dense canyon I'll translate this to issue as well but it's kind a blocker for us. want to raise it here first

[deft rain]

do you have an example of what you're doing to get this message?

[spiral whale]

you can have a look this trace: https://dagger.cloud/CASTAI/traces/333d805309de0f7aa191c813558576dc

[spiral whale]

for command it's just gotestsum execution with code coverage option

[deft rain]

(just a note, i recommend that the error itself doesn't contain all of stderr, it makes the trace very hard to read)

if you remove the WithMountedCache for that directory, does it work properly?

we don't add the noexec flag as far as i can tell, i wonder if there's something specific about your configuration? are you using the dagger helm chart?

[spiral whale]

yes, when we remove the build cache, it start working again.

[spiral whale]

yes, we're using dagger helm chart and we're only setting some node affinity and tolerations nothing special

[dense canyon]

@spiral whale what about file permissions? If the container which originally wrote those files set them with a specific permission and a new container with different permissions try to access those, you'll get this permission denied message

[spiral whale]

this is standart go container we're using, these files created by go itself and we don't touch permissions. When we run same pipeline locally everything works. It's somewhat related dagger helm installation on our GKE cluster

[dense canyon]

it shouldn't have to be any different in your cluster though

by any chance do you have access to the cluster via kubectl?

one thing that you could do is to connect to that cluster's engine and check what's effectively in the cache volume

those permission denied errors seems a bit strange though..

[restive shore]

When we combine this release with go build cache we're getting errors similar to go tool covdata: fork/exec /root/.cache/go-build/.../covdata: permission denied. The question here, are we mounting cache volumes with noexec option or is there such a limitation in k8s setup of the dagger engine?
Are you running your container as root?

[spiral whale]

yes,

[spiral whale]

will try to do it today, i restarted cache volume, nodes couple of times before, behaviour is same in gke cluster.

[spiral whale]

one more issue, we started to see following log and connection drops in our CI?CD. Do we have a way to adjust this in helm charts

2025-09-11T04:50:39.743072756Z stderr F runtime: program exceeds 10000-thread limit

link: https://github.com/dagger/dagger/issues/11028

[sharp brook]

@dense canyon recently mentioned something about a new caching feature in tech preview. Can anyone say more?

[upbeat herald]

Hey, it's wip but: https://github.com/dagger/dagger/pull/10975 😄

[sharp zealot]

👀 https://github.com/dagger/dagger/issues/11216

as discussed @rocky harbor 🙂 👆

[spiral whale]

i was playing with remote cache exporters locally and i'm getting following error during engine shutdown. How can I resolve this one? as far as i can see timout is hard coded it's not possible to increase it

Error: cleanup failed: "close dagger session": shutdown: do shutdown: Post "http://dagger/shutdown": context deadline exceeded

[sharp zealot]

@thorn moat FYI https://github.com/dagger/dagger/issues/11288 🙏

[sharp zealot]

I don't mind taking it

Just want to make sure you're ok with it

[thorn moat]

Is the 2. Copy matching files from before necessary? Would have thought that excluding them from the incoming changes would cover that

hmm i guess unless it plants tombstones, and those layer over the original

[sharp zealot]

UX preference question. Which feels nicer:

• dagger checks go/lint* helm/test
• dagger checks go.lint* helm/test
• dagger checks "go lint*" "helm test"

cc @warped canyon this is for supporting the toolchain pattern smoothly with checks

Heads up: renaming this channel to make its purpose more clear. It's a place for advanced Dagger users & fans ("daggernauts") to discuss their problems and ideas beyond the basics

[restive shore]

I don't have a ton of context around the new checks feature. Is this something I would/could use locally? What's the difference between running dagger checks and dagger call? From the examples, it looks like it's aimed at testing/linting reports, what would be some other use cases? Not looking for immediate answers as things may be in flux. Just some questions that popped into mind 🙂

[sharp zealot]

It's an opinionated layer on top of functions. It scans the module for functions that match the check interface, and gives you a convenient interface to listing and calling them.

A "check" can be anything that produces a red/green result useful to the end user. So: testing, linting, checking that your generated files are up-to-date, checking that you ran go mod tidy, whatever

Ideally, dagger checks would map 1-1 to Github checks for example

It gives you a simple entrypoint for CI, and for local development: run all the checks (dagger checks), and you know you didn't break anything 🙂

[sharp zealot]

Wrote my first module in Dang, took me 3mn, worked on the first try 🙂 10/10 would use again @thorn moat

(a tiny module, but still)

[thorn moat]

nice! is it pushed somewhere?

[sharp zealot]

yeah in my "more CI improvements" pr

[thorn moat]

ah cool

are you using Zed? any LSP issues? it's getting more and more featureful, added symbol renaming the other day

[sharp zealot]

I had zero lsp support that I could see - but didn't try

[thorn moat]

if you install the zed extension, it should auto-configure it, if dang is in your $PATH

[sharp zealot]

should zed prompt me automatically to install the extension?

[thorn moat]

oh sorry it's not published, it's in the repo - so you'll have to clone https://github.com/vito/dang somewhere and do install dev extension, and select zed-dang/

[sharp zealot]

$ dagger -q checks -l

Name                                Description
scan                                Scan source code and artifacts for security vulnerabilities
check-generated                     Verify that generated code is up to date
ci-in-ci                            "CI in CI": check that Dagger can still run its own CI
lint-helm                           Lint the helm chart
test-helm                           Verify that helm works correctly
sdk/typescript/lint-typescript      CheckTypescriptFormat checks the formatting of the Typescript SDK code
sdk/typescript/lint-docs-snippets   CheckDocsSnippetsFormat checks the formatting of Typescript snippets in the docs
sdk/typescript/lint-go              CheckGoFormat checks the formatting of the typescript runtime, which is written in Go
sdk/typescript/test-node
sdk/typescript/test-bun
sdk/typescript/release-dry-run      Test the publishing process
sdk/elixir/lint                     Lint the Elixir SDK
sdk/elixir/test                     Test the Elixir SDK
sdk/elixir/release-dry-run          Test the publishing process
sdk/rust/check-format
sdk/rust/check-compilation
sdk/rust/test                       Test the Rust SDK
sdk/rust/release-dry-run            Test the publishing process
sdk/php/php-code-sniffer            Lint the PHP code with PHP CodeSniffer (https://github.com/squizlabs/PHP_CodeSniffer)
sdk/php/php-stan                    Analyze the PHP code with PHPStan (https://phpstan.org)
sdk/php/test                        Test the PHP SDK
sdk/php/release-dry-run             Test the publishing process
sdk/java/lint                       Lint the Java SDK
sdk/java/test                       Test the Java SDK
sdk/dotnet/lint
sdk/dotnet/test
sdk/go/test                         Test the Go SDK
sdk/go/release-dry-run              Test the release
sdk/python/lint-python              CheckPythonFormat checks the Python code formatting
sdk/python/lint-go                  CheckGoFormat checks the Go code formatting for the Python runtime
sdk/python/test                     Test the Python SDK
sdk/python/release-dry-run          Test the publishing process
scripts/lint-sh                     ShellCheck scripts files
scripts/lint-powershell             LintPowershell scripts files
scripts/test                        Test install scripts
docs/lint                           Lint the documentation
go/lint                             Lint the Go codebase
go/check-tidy                       CheckTidy checks that go modules have up-to-date go.mod and go.sum
release/dry-run                     DryRun performs a dry run of the release process

[sharp zealot]

https://github.com/dagger/dagger/issues/11307 😇

[restive shore]

What makes a dagger function a "check"?

[sharp zealot]

• no required arguments (after user defaults appear)
• return a CheckStatus

[restive shore]

Curious to see some real world examples. Would there be a case for chaining checks? Or is to meant to be an exit node?

[sharp zealot]

We're in the process of hooking it up in our own CI (see #daggernauts message)

[sharp zealot]

POV you're running your entire CI on a single cloud machine 😛

[sharp zealot]

If anyone is looking for an easy core contribution to cut their teeth: https://github.com/dagger/dagger/issues/11313

[sharp zealot]

<@&946480760016207902> 🙏 🙏 how hard is this? https://github.com/dagger/dagger/issues/11317

[thorn moat]

my guess is not that hard, MVP is just passing a SDK pragma to a bool flag in an existing API (Host.directory), which is a well-beaten path, albeit with extra legwork across the SDKs.

[sharp zealot]

Was wondering if piggybacking on the existing +ignore pragma might be less SDK legwork.. Just need to check if the SDKs enforce the string-array type, or will allow a string value to be passed. In which case I can handle both cases both, in the engine

[thorn moat]

hmm I see the appeal, but I suspect that'll end up being just a different kind of change to make to each SDK (handling different types for +ignore vs. adding another pragma)

[sharp zealot]

if that turns out to be the case, then no point in piggypacking, I agree

Just trying to avoid touching every sdk...

https://tenor.com/view/we-don't-have-that-kind-of-time-dr-aditya-singh-sweet-tooth-we're-on-a-tight-deadline-adeel-akhtar-gif-2172504612198576780

Maybe doug could do it? 🙂

[thorn moat]

maybe - i'll reply on the issue (didn't notice you asked about piggybacking there), so we can just point it at the issue

[thorn moat]

replied with another idea: https://github.com/dagger/dagger/issues/11317#issuecomment-3458882319

[sharp zealot]

I swear I just saw a loading module span time elapsed counter drop down then start increasing again... First time I see that. It jumped back for a half second, maybe twice. Known issue @rocky harbor @thorn moat ?

Trace for reference: https://dagger.cloud/dagger/traces/716dc2f9839da9f19be9e9bb27c1b7a2#5826220a804a3119

[rocky harbor]

I have seen it go negative before and in the TUI do occasionally see it flash and back forth between two different intervals, but was never common enough to reach the threshold of looking into it

[sharp zealot]

@dense canyon I see filesync metrics

But, filesync activity is still completely invisible on default verbosity...

[sharp zealot]

OK will let you know if it ever happens more than 3 times in the same day 🙂

(this was my first time ever)

[dense canyon]

That's honoring the default visibility of the filesync span. We can make it appear with the default verbosity but I'm a bit unsure if it's the best choice given that it'll very likely spam the TUI wirh a bunch of info since it needs to also show all the parent spans above filesync

[sharp zealot]

Yeah my issue is when everything visible on the TUI says ✅ and there's nothing else visible going on... So it looks like the TUI is just hanging... sometimes for a long time. I guess it would be OK for the filesync itself to be hidden, if its parent were visible

@dense canyon this is torture 😂 you're teasing me.

https://tenor.com/view/tell-me-more-summer-nights-grease-john-travolta-olivia-newton-john-gif-26436825

you just gave us the feature and we're already spoiled 😛

[dense canyon]

Lol. Gotta keep those docs under control 😄

50s to upload 12MB? :sus:

There seems to be something odd in the span duration calculation. Will 👀 tomorrow

Having said that... The whole filesync span actually took 2m so it's possible that it took that much time

Did you intentionally throttle your upload or something?

[sharp zealot]

No! It's home internet -> PARC. My bandwidth is good. No idea why it's so slow

[dense canyon]

it's definitely faster here using PARC.. having said that, 13 and 19s to upload the whole thing 😬 . I'd assume it's very likely since it's transferring files one by one and doesn't have an optimization to avoid several roundtrips. @rocky harbor surely knows better

[rocky harbor]

no it does transfers in parallel

[dense canyon]

what makes me think of this is the directories that take the most (.git and sdk) have a bunch of files in them

[rocky harbor]

i think there's a semaphore though

[dense canyon]

I know, I know.. but it still has to do a bunch of roundtrips for each file

and it seems to hurt when there's hundreds (?) of them

[rocky harbor]

yeah it's grpc so there's the tcp roundtripping, plus it also sends in chunks for large files. it's all one grpc stream though

the semaphore is fixed to be 4 though, i feel like it could be bigger

only being able to send 4 chunks in parallel at a time would probably hurt the "lots of little files" case

[dense canyon]

yeah.. for me each roundtrip to the US is ~120/150ms

[dense canyon]

nice, I'll try playing with this tomorrow and see how it changes 🙏

[rocky harbor]

the relevant part is in the client-side code, which is mostly still fsutil, so this is what you'd need to change https://github.com/tonistiigi/fsutil/blob/586307ad452fadb42e28be16d2b995b2eb6755e9/send.go#L69

[sharp zealot]

oooh could it be an east coast vs. west coast latency difference?

a good reminder that our VMs are... where, europe?

[rocky harbor]

I think they are all in northern virginia

CI was hitting timeouts downloading a lot of stuff today, wondering if something is just up network wise w/ the infra

[dense canyon]

the relevant part is in the client-side

[dense canyon]

^ here's the time it took from a VM that I have in Virginia against PARC. Same thing on my local machine takes ~ 10x more

considering that the latency between Argentina and the US is around ~60 times worse than this VM.. doesn't seem that bad

[sharp zealot]

Orphan filesync: https://github.com/dagger/dagger/issues/11351

[sharp zealot]

FYI daggernauts: another massive performance optimization coming: https://github.com/dagger/dagger/pull/11336

https://tenor.com/view/not-left-handed-gif-13945460414114735140

[restive shore]

I am trying to understand toolchains and I want to clarify the difference between blueprint and toolchain. From what I understand in docs, the only difference between the two are,

Given that my toolchain (mymodule) has a function called hello-world

if you install toolchain you use the toolchain namespace to call functions (otherwise they behave the same)

dagger toolchain install path/to/mymodule
dagger call mymodule hello-world

If you init a blueprint, I get to directly use my function as if it's in my module.

dagger init --blueprint path/to/mymodule
dagger call hello-world

Is that correct?

If so, feel like "blueprint" adds adds unnecessary complexity/confusion and probably should be deprecated.

[sharp zealot]

A blueprint is useful when you need to daggerize many projects with the exact same stack, with minimal disruption and boilerplate for the end user.

It's exactly the same as dropping the same boilerplate on X different directories - but without actually polluting the repo with boilerplate

[restive shore]

Oh.. so toolchain adds boilerplate? just like regular dagger init?

[sharp zealot]

(guessing you mean blueprint) No it achieves the same result as boilerplate, without the downside of actual boilerplate

[restive shore]

No I meant toolchain. When i do dagger toolchain install does it add more than a dagger.json to my project?

[sharp zealot]

It's the same as installing a dependency, just a special kind of dependency that is exposed to the end user, instead of your code

Instead of adding the dependency to dependencies in dagger.json, it adds it to toolchains.

[restive shore]

I'm still confused why blueprint and toolchain are different from a consumer standpoint.

[sharp zealot]

Granted, we need some real-world testing to better understand how blueprint and toolchain will be used in practice, and if one emerges as more important than the other. The way they're designed, they are complimentary in how they work, and you can combine them.

[restive shore]

Sorry, I'm going just by docs. Let me play around with it first. That may answer my questions.

[sharp zealot]

It's mostly terminology. Initially "toolchains" were just called "multiple blueprints". But it feels weird to tell a user "you can install multiple blueprints in the same module". It doesn't really fit the meaning of the word "blueprint".

[restive shore]

ah! so my understanding is correct. In that case, personally, I won't even introduce blueprints. I'll just let my users install toolchains. Because, I'm dreading introducing another feature that's almost exactly like the other.

[sharp zealot]

It is possible that toolchains turn out to be versatile enough to make blueprints unnecessary. But too early to tell.

I think it depends on how much you want to centralize vs. decentralize the ownership of Dagger modules.

• Blueprints are most centralized.
• Toolchains is intermediate
• Hand-rolled functions are most decentralized

[warped canyon]

I see the two paths as

1. (blueprints) "i want my users to have exactly this module"
   2 (toolchains) "i want to provide my users multiple tools they can use, in addition to their own customization"

[sharp zealot]

I think blueprints will be a powerful but niche feature, for large platform teams that need full centralized control

Toolchains IMO have the potential to become mainstream, the first thing you learn when setting up dagger. Because it strikes a good balance

[restive shore]

That makes more sense. If i have a module that handles CI/CD completely, that will be a blueprint. But parts of that blueprint (other modules used by that blueprint) can be adopted by users as toolchains if they want

Users could install that CI/CD module as a toolchain too. (that's where the confusion will happen)

So to avoid confusion, I may stick to toolchains only.

[sharp zealot]

Sounds reasonable

Also @restive shore there's another major feature coming up, which combines with toolchains -> checks

[restive shore]

ah! the next piece of the puzzle.

[sharp zealot]

dagger checks -l [FILTER] -> list available checks in the module
dagger checks [FILTER] -> run checks and show their results

dagger checks walks the entire module looking for checks, even top-level functions that return objects. That includes functions implemented by toolchains.

Example:

$ dagger toolchain install go
$ dagger checks -l
go/lint
go/test
go/check-tidy
$ dagger checks
<runs all go checks>

This is how we will standardize CI integration. New check = new CI entrypoint.

[warped canyon]

yeah this part is going to be so nice

[sharp zealot]

SO nice

[warped canyon]

like the biggest thing for making dagger easier to adopt since the Go SDK

[restive shore]

How do checks handle credentials?

[sharp zealot]

User defaults are applied, same as regular functions.

(you can't pass explicit CLI args, so only user defaults are available)

--> .env

(another important piece in the puzzle)

[restive shore]

Does the upcoming toolchain config allow for setting values like env://MY_SECRET?

[sharp zealot]

No

It's sandboxed, so you can only express in there what you could express in a custom module: +default, +defaultPath, +ignore etc

Otherwise it would be unsafe, any remote module could escape the sandbox

btw @restive shore we're looking for the right name for that field in dagger.json.

[restive shore]

Which field?

[sharp zealot]

https://github.com/dagger/dagger/pull/11359

[restive shore]

I saw that.. but I guess it's already merged 😄

[warped canyon]

just merged 😄 here's what a dagger.json would look like to set a value for the message argument of the configurableMessage function https://github.com/dagger/dagger/pull/11359/files#diff-079f50f46cf3ce4cb1186f5cf20f138de9d57c857e8a7bf96997815254d781c9R175-R191

{
  "name": "app",
  "engineVersion": "v0.19.4",
  "toolchains": [
    {
      "name": "hello",
      "source": "../hello",
      "arguments": [
        {
          "function": ["configurableMessage"],
          "name": "message",
          "default": "hola"
        }
      ]
    }
  ]
}

[restive shore]

This allows me to change defaults without touching or writing code. Sweet!!

[sharp zealot]

Yes but only within the sandbox as we discussed just before. It's basically an ultra-lightweight middleware system, so you can "wrap" the toolchain with custom argument values, filters etc. without having to actually eject to the full-blown SDK system (which is particularly bad at middleware/wrapping)

[restive shore]

0.19.5 feels snappy! 🚀 At least in the TUI.

[sharp zealot]

@restive shore it's because of this 👆 🙂

[restive shore]

Right! I've been waiting for it to get released. Fired it up as soon as it was out 🙂

One more question about toolchains. Since toolchains, is there no benefit to doing a direct dagger install?

[sharp zealot]

Toolchains can only be called by the end user. You can't call them from your own module's code. For that you need a regular dependency (currently dagger install but we might alias that to dagger dep install to make it more clear)

[restive shore]

oops, I thought toolchains can be dag. called. Gotcha

[sharp zealot]

We went back and forth on it. But figured it would be easier to explain and understand this way.

[restive shore]

We'll put it in practice and see. I see why you did it that way, but that's also 1 more nuance to explain 🙂 I love all the new features but I am a power user. I can see it being overwhelming to someone new. I do like the direction with toolchains TBH. That may potentially make it easier for newcomers. It's still too early to tell.

[sharp zealot]

The features will continue until morale improves 😛

[restive shore]

are .env defaults not documented? I can't find any docs on it

[sharp zealot]

not yet... 😔

[sharp zealot]

Dang feature request @thorn moat : builtins to decode/encode base64

Also: explicit error handling is not possible (I think). Which is good (I think), but prevents direct translation of code like this:

        if err != nil {
            var execErr *dagger.ExecError
            if errors.As(err, &execErr) {
                if strings.Contains(execErr.Stderr, "invalid reference: "+destTag) {
                    // this is a ref that only exists in the source, and not in the
                    // dest, so no overwriting will occur
                    return nil
                }
            }
            return err
        }

I've never liked this particular pattern of special-casing the errors, for the same reason it can't be translated: it's SDK-specific, and makes errors part of the workflow logic, which graphql really doesn't want us to do

[thorn moat]

interesting, what's the use case? debugging/snooping around?

[thorn moat]

error handling is on the backlog, will think about it!

[sharp zealot]

porting .dagger/sdk.go 😁

[thorn moat]

encodedPAT := base64.URLEncoding.EncodeToString([]byte("pat:" + githubTokenRaw))

giga brain move: shell out to base64

(j/k, will add)

(edit: decided against:) side note: i think I'm gonna have to move where directives are declared in Dang - right now they're post-field, which matches GraphQL, but that doesn't work as well now that field declarations also have bodies:

pub test: Void {
  # ...
} @check # <- weird

[sharp zealot]

I definitely considered it 😁 but worried I'd slow down the monolith

[sharp zealot]

🤔

https://tenor.com/view/but-why-tho-huh-whatt-me-gif-5419603452100877803

[sharp zealot]

@thorn moat @warped canyon I had my agent write a manual for porting Dagger modules from Go to Dang. Then I feed it to the next agent, and ask it to improve the manual based on its own experience, and so on.

In case it's useful 🙂

https://github.com/shykes/dagger/blob/967ec136267467dab7d3ab838250ad7488b480dd/docs/dang/go-to-dang.md

Currently part of https://github.com/dagger/dagger/pull/11373

@thorn moat dang papercuts so far:

• Regular arguments can omit names, but directive arguments can't. It stumped both me and my coding agent.
• Does let mean both "variable inside a body" and "private field or function in a type"? I still am not sure 100%.

[warped canyon]

pretty sweet! I wonder how much it'll trip over the corresponding Go and Dang examples not being exactly the same logic. Most of them are the same but a few have small differences.

Also curious if it would help to have an early section explaining when a Go module can't be converted to Dang

[sharp zealot]

I figured it out, it was a name conflict (my module has the same name as one of its dependencies)... Error is wrong

[sharp zealot]

@thorn moat another dang request: path.Clean 🙏

(also a list of builtins so I know what to request 🙂

[thorn moat]

here's the list so far: https://github.com/vito/dang/blob/main/pkg/dang/stdlib.go
(i have an in-memory TODO to improve LSP completion so you can at least see them on strings/lists/etc)

[sharp zealot]

@thorn moat wait... Does dang have native self-calls?

https://tenor.com/view/cosmos-mind-fractal-space-unlock-gif-4851645

[thorn moat]

no, but i think it could at some point, not 100% sure

[sharp zealot]

Since there's no codegen, my first instinct was: "oh we get this for free?"

[thorn moat]

almost - i think my SDK needs to at least implement the new typedefs API in order to support that. would need to take another swing at it

[sharp zealot]

OK cool! Not a blocker for me

[sharp zealot]

Thinking about it some more... You could inject a custom span on every internal function call, since you control the runtime anyway. It's not technically self-calls, but it's more like a built-in parallel 🙂 Since unlike Go & other runtimes, you know for sure that every function call is traceable and desirable to trace

[sharp zealot]

Just thought of a cool pattern: blueprints for vendoring dependencies 🙂 Example:

• I'm porting a bunch of modules to Dang
• I want all my modules to use the exact same version of the Dang SDK
• Normally this requires manually updating N modules each time. Laborious and error-prone
• Instead, I can "mirror" the Dang SDK module by initializing a local module using the SDK as a blueprint. Then point all my modules at that local mirror. Now I only need to update the blueprint once, and all my modules get updated

--> This works for any dependency, not just SDKs 🙂

TLDR: we accidentally implemented vendoring 🙂

@thorn moat tried to port a Go module that uses parallel... Any plans for that in the stdlib or syntax? 😛

[thorn moat]

@sharp zealot that's been in the "fun things to figure out" pile for a bit but haven't had time to even think about what it should look like, feel free to mock some things up!

[sharp zealot]

First instinct: probably easier to just do it at the core API level. Then dang gets it for free

Could be a pragma on an array argument, or a special array type? That's the function telling the engine "when the caller passes an array with N elements, fan out to N instances of this function, and pass a single value to each instance.

It's a mirror image of parallel since the callee would be controlling its own parallelism, instead of the caller

Maybe too restrictive? But would dodge the issue of callbacks & generics

The other approach is to finish shipping generic objects, and build a generic dag.Parallel([]Object) []Object primitive that relies on laziness?

[thorn moat]

Side thing I wanted to try (haven't processed above but maybe related): having Dang just construct IDs client-side instead of sending a request to the server just to get the ID to pass an object as an arg somewhere. Then instead of N+1 queries, it's just one big query. Curious how many parallelization opportunities we lose in some SDKs. (I think the Go SDK at least parallelizes ID fetches, not sure about other SDKs though, it's a subtle thing to get right)

[sharp zealot]

the exact relationship between query and ID still eludes me 😅

[thorn moat]

in dang or in general?

[sharp zealot]

in general. skill issue 😁

what I mean is that I don't intuitively understand whether a client should be able to construct an ID like it can construct a query

[thorn moat]

oh, yeah, it would be tightly coupled ✨ vertically integrated ✨ for sure. but, it's been designed to support that since the start, at least

[warped canyon]

I think this is still relevant https://github.com/dagger/dagger/issues/5083

[sharp zealot]

I'm really not a fan of doing it at the SDK level. Feels like a worst of both worlds between language-native and dagger-native

[sharp zealot]

@thorn moat just in case: in dang, can a default value reference a function? Or literal only?

[thorn moat]

it can be any expression

[sharp zealot]

https://tenor.com/view/sparkle-eyes-eevvee-amazed-gif-12566593

[thorn moat]

i have no idea how that bubbles up to dagger sdk at the moment

like, if you do base: Container! = container.from("golang") or something

[sharp zealot]

you mean for visualization?

or for actual resolution

[thorn moat]

actual resolution (but both, i have all the questions)

[sharp zealot]

I think for resolution, it traces back to the kinds of questions @upbeat herald and @sour citrus are struggling with, re: how lazy can runtimes be? If a default value in the schema can be the result of a dynamic expression, then that expression must be resolved at registration time, which means you need a full runtime at registration, which means dagger functions requires a complete runtime build + load.

For most runtimes that's disastrous, but Dang might be a special case since it's so lightweight to begin with, and the build/run distinction maybe doesn't hold since there's no codegen?

[sharp zealot]

Update: I just tried this with the dang SDK, and it doesn't work... Because any defaultPath will be resolved in the context of the caller... In the case of the Dang SDK, there is a dangRoot that resolves to the root of the repo with some filters... That completely breaks when loading it as a blueprint.

--> I don't know if the solution is to "fix" the dang module, or our blueprint feature? Or something else?

Once again our entangled filesystem model comes back to bite us

[sharp zealot]

@thorn moat is let foo="bar" allowed inside a if{} statement? nevermind I found the root cause of my parse error: if foo instead of if (foo)

[thorn moat]

ah yep, that was a recent change. there's also case (foo) { ... } now, which replaces pattern-matching.

re: let vs. pub (since you mentioned it was confusing before and I don't remember if I answered) - they both declare a new slot, only pub gives it public visibility (which only really matters within a type), and let makes it private (so it won't be exposed in a Dagger SDK for example, and can't be called from outside). If you omit let or pub and just do x = 1 it'll update the outer slot where x was declared. Kinda similar to Go := vs = semantics

[sharp zealot]

nice thanks

[sharp zealot]

@rocky harbor how do I enable function caching in the monolith?

[rocky harbor]

it's enabled provided none of the dagger.json's have disableDefaultFunctionCaching: true, but I handled all that during v0.19.4, so it should be enabled

I added //+cache="session" in a couple places that needed it, but otherwise everything was cacheable

[restive shore]

I have curl --silent calls in my dagger module but in notty (--progress=report,dots,plain) I am still seeing the output in the logs. is that expected?

[crude gazelle]

I've forwarded this to the #daggernauts as it's a good place to discuss pain points for advanced use cases.

[sharp zealot]

@thorn moat how do I mark a check in dang?

[thorn moat]

it goes just between the field decl and the block: https://github.com/vito/dang/blob/b94a4340f16f4fc60e78019068e5025468b1de9d/.dagger/main.dang#L203

[sharp zealot]

@thorn moat I think maybe I'm pinned to a version of dang that is too old

In the context of our kill-monolith PR, is it OK to just pin on the very latest? Not sure what to do with the misadventure you had that Tom was mentioning this morning

(I didn't understand the issue this morning)

[thorn moat]

it's a bit of a knot - if you pin to the latest, the latest stable engine won't be able to run it, only a dev engine

the issue was that the codegen module was un-pinned, so it strayed forward

[sharp zealot]

OK. That's fine, we're actually going to ping the new CI to use main for now (because we need the latest dagger checks) cc @warped canyon did you expose that knob in the new action?

[thorn moat]

should be fine then 👍

[sharp zealot]

So, I'll just pin everything to today's main

dagger -M -c 'address https://github.com/vito/dang | git-ref | commit'

[warped canyon]

Yeah you can specify a commit for the engine

[sharp zealot]

@warped canyon one box I forgot to add to the checklist: how do we get a .env in there

btw @thorn moat -> https://github.com/vito/dang/issues/7

This would make it easier to manage the pinned version in a single place, via the magic of blueprints 🙂

[thorn moat]

am i getting deja vu https://github.com/vito/dang/issues/5

[sharp zealot]

https://tenor.com/view/ryan-reynolds-goddammit-damn-hitmans-bodyguard-hitmans-bodyguard-gifs-gif-8352668

I checked... I guess I'm blind

AND you already fixed it, lol

[thorn moat]

maybe cause its' closed 😂 - yea

[sharp zealot]

oh right

well ok then

[warped canyon]

Is this the part @dense canyon was thinking about?

[sharp zealot]

I think he was talking about just having the plumbing to get secrets in native CI via 1password

But separately we haven't discussed the best practice for how to use that plumbing

(and one step removed from that, how to prepare for that best practice in GHA)

[warped canyon]

yeah I guess we could assemble a .env in bash from gha secrets but its not great

[sharp zealot]

.env.ci and check into git?

It wont have any actual secret values in there...

[warped canyon]

yeah possibly. What goes in there vs toolchain config in dagger.json?

[sharp zealot]

toolchain config is part of the module logic itself. same across all environment, and sandboxed.

[sharp zealot]

@thorn moat does dang support optional args?

[thorn moat]

yep

[sharp zealot]

(ie. can I just omit an argument internally)

even without going through a dagger function call? ie. local function calling another local function?

[thorn moat]

not sure what you mean by that, but yeah. same semantics as graphql, either nullable type or has default

non-null-with-default translates to an optional/nullable type on caller side, and the default is applied on the callee side

https://github.com/vito/dang/blob/main/tests/test_default_args.dang

[sharp zealot]

It's happening 😄 https://github.com/dagger/dagger/issues/11440

[sharp zealot]

@thorn moat what quote characters does dang support beyond double-quote? single-quote, backtick, other?

[thorn moat]

no backtick or single, just double quote and triple-double-quote

triple-double-quote will automatically word wrap and trim leading whitespace, so it's great for prompts (+ docs)

[sharp zealot]

Was looking for a way to have literal double quotes without escaping them

No raw-strings then right?

[thorn moat]

not yet yeah, will have to figure that out for regexes (unless i make regexes more special like /foo/)

[sour citrus]

Maybe a dumb idea, but could we imagine to integrate the dagger.json inside a *.dang file? Like in some kind of header?
That way we could imagine to have a single file that declares the dependencies and settings. Not a big deal to have two, just wondering if that could be a good or bad idea.

[thorn moat]

Kinda related thread that mentions import: https://discord.com/channels/707636530424053791/1440868206045691904

But in the end I don't think we can avoid dagger.json, we at least need it around to be able to identify where a module exists, and even the things I might want to 'import' in Dang (like GitHub's API) might need to be in dagger.json for secret integration (#1440868206045691904 message)

[sharp zealot]

@thorn moat not a fan of the top-level description= to add a description to the module in dang. A docstring on the type would make more sense to me

[thorn moat]

oh yeah that's cruft

at the time description= was added doc strings were probably not a thing

[sharp zealot]

https://github.com/vito/dang/issues/8

[sour citrus]

On SDKs if I'm right we have the description on the objects, and the description of the module.
Quite often we have modules with one single type. And there's always a main type.
IIRC on dagger shell we are displaying the module's doc and if it's not set the doc of the main object for the module. But it's only for shell, not for dagger functions for instance.

Maybe we can generalise that, saying the module description is the top level doc OR the main module doc.
It can be done at two different places: on the SDK, when registering the types, but it means each SDK need to do it. Or at the engine level, when we inspect a module. That way it's "just" a question of representation.
The advantage of the latest is it will work everywhere, with all SDKs with no change.

[sour citrus]

@thorn moat a dang suggestion, to support ruby-like arrays %w
Instead of

buildCtr.withExec(["go", "test", "-coverpkg=./...", "-coverprofile=.coverage", "-shuffle=on", "./..."])

it could be

buildCtr.withExec(%[go test -coverpkg=./... -coverprofile=.coverage -shuffle=on ./...])

or even something like

buildCtr.withExec(%"go test -coverpkg=./... -coverprofile=.coverage -shuffle=on ./...")

This is not a very important feature, but kind of sugar that would make things even better to write.
Happy to try to implement it if that's something that could make sense.

[thorn moat]

ha yea that's been on my mind. haven't jumped on it yet since i'm weighing the option of custom quoters, like %re{foo/bar/baz} for regexes, %w{1 2 3} for word splitting, %sh{echo 'foo bar'} for shell-aware splitting, etc.

happy to bikeshed that if you want to try implementing it, even though it's a bit more complicated

rough idea is it translates to a call like "raw contents".re and then we just define those String methods, but there may be a more sane way to organize them

[sharp zealot]

Please no

Don't turn dang into perl 🙏

[sour citrus]

as much as I used to like perl (long time ago) that wasn't the goal 😄

that said, we can achieve the almost same thing with something like

"go test -coverpkg=./... -coverprofile=.coverage -shuffle=on ./...".words

that could return the string split by words, doing the same thing but without perl/ruby/black magic 😅

[sharp zealot]

I don't want to meddle too much - I just like that the syntax is pretty easy to guess, and feels like a logical continuity of graphql. If we start going in more esoteric territory... I'm just not going to learn it

[thorn moat]

Yeah there's a reason I haven't pulled the trigger on it yet. Ironically one of the motivations for quoters is to keep the grammar small, since we'll at least need one new string form to support regexes without escaping hell. MVP for that is raw strings, but if we have to add new syntax for that, quoters are tempting because it's still just one new syntax, with a straightforward desugaring rule, that can pay dividends later.

[sharp zealot]

@thorn moat I'm ready for dang query support in the CLI 😛

[thorn moat]

in the REPL? or for flags? (anticipating a 'yes' lol)

[sharp zealot]

https://github.com/dagger/dagger/issues/11487

@thorn moat got weird github spam on that issue

[thorn moat]

https://tenor.com/bWMBz.gif

[sharp zealot]

looks like a phishing attempt on github???

[thorn moat]

lol yea. wild, seems hooked up to github events somehow, was almost immediate

reporting it

[sharp zealot]

And I blocked it

[thorn moat]

root@d84b9c256a7d:~# unzip -l 103c30e3eec8.zip
Archive:  103c30e3eec8.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2025-11-25 21:23   Password - fix.txt
   936846  2025-11-25 21:24   Winver-x86-x32-fix10.18.25.rar
---------                     -------
   936846                     2 files

[crude gazelle]

I don't want to meddle too much - I just like that the syntax is pretty easy to guess, and feels like a logical continuity of graphql. If we start going in more esoteric territory... I'm just not going to learn it

As a community maintainer / end-user: I agree with this sentiment.

I appreciate DSLs, but by definition, I'm only going to be using it in that one Domain. It needs to be easy to learn and easy to remember.

[thorn moat]

Yep, that'll always be the design ethos - see https://github.com/vito/dang#design-philosophy

But, by the same logic of it being a domain specific language, it's worth exploring (which doesn't mean committing to!) ideas that help in that specific domain. With ["cmd", "arg1", "arg2"] being present in practically every Dang script it makes sense to think about a bit, just like Dagger Shell gave special treatment to args so you don't have to do with-exec foo,bar,baz and can type them more naturally.

Whether we commit to these ideas or decide it's worth it in the end is another story. We need to consider second-order effects, too - for example, what happens when I have %w{go test} but then want to have an expr arg. Does %w{} support interpolation? Is it worth that complexity? Is the pain of switching from %w{foo bar} to ["foo", "bar", baz] worth having %w{} in the first place? etc. etc. etc.

I promise this language isn't going to become a kitchen sink. Don't take my excitement for exploring ideas as getting distracted by shiny things and jingling keys. 🙂 It's the early days for a language, it makes sense to experiment a bit and try ideas and throw them out, which has been the process thus far.

[thorn moat]

Personally, I have some comfort in manually typing out ["foo", "bar"] and knowing exactly what's going into argv without worrying about whether %w{sh -c 'foo bar'} parses how I expect (it wouldn't - so I would only trust a quoter that understands shell syntax).

Also, the existing triple-quoted strings work great for sh -c:

withExec(["sh", "-c", """
  entire script
  goes here
  and is parsed nicely without leading whitespace
"""])

But, still, seems like we'll need some sort of new syntax for regexes to avoid escape hell, which is the real reason I'm considering general quoter syntax, vs. Python-style raw strings which are of similar complexity but more limited (can only be a raw String, can't one-shot a Regex literal)

[crude gazelle]

That's completely fair, the %w syntax is unfamiliar to me but I understand if it's necessary.

Out of curiosity, does the end-goal involve the other SDKs being written in Dang?

[sour citrus]

That's something we discussed, yes. To write SDKs in Dang will improve performances. For now we're lacking a few features, but when possible that's the goal. We have already re-written some of the "dev" sdk modules (like the ones for java, go, typescript) in Dang for that.

[crude gazelle]

Sounds like a plan, since every SDK (besides Go) uses the Go SDK as glue already, makes sense to use a lightweight glue instead

[sharp zealot]

👀 👀 👀 https://dagger.cloud/dagger/traces/e6de662c187bc24e252b6a9bdd4e7a30#4aeb6f1b292f6d62

What are those multipliers on each check line in the TUI? 3x, 4x, 5x...

[sharp zealot]

@thorn moat @dense canyon @lime comet 👆

[thorn moat]

a UI trade-off, alternative ideas welcome. it's a multiplier for the number of dots to the right - it scales down when it would have exceeded the screen size

[sharp zealot]

OK my first thought was: re-runs / auto retries?

[thorn moat]

ha, yeah it's not ideal for such a nuanced thing to show up kind of prominently, but the alternative seemed worse since it's misleading (you might think your heaviest check is actually the lightest)

[sharp zealot]

So it's like a zoom level?

(kind of)

[thorn moat]

yeah exactly

[sharp zealot]

couldn't we just shrink the smaller ones, so the heaviest check always takes 100% of available space?

"just" 😛

[thorn moat]

yeah 😛 - i'd like to experiment with that

[sharp zealot]

could be a cool scale effect, like in some extreme cases, one check would absolutely dward the others

YES first successful run of a dynamic check!!

[sharp zealot]

Experiment: split each test in our e2e suite as a separate check. https://dagger.cloud/dagger/traces/e65f1fa8782d7ceff96fef129d348cab?span=be3d68b5aa0a3431

Next step: scale it out 😛

[sharp zealot]

What's with the "xxx% dropped"? dropping telemetry?

[thorn moat]

this was a surprise to me too, apparently it's packet loss? but i don't know where. i guess in the engine? (super cool that it shows up if it's actually relevant, like if there's network instability)

[sharp zealot]

@thorn moat any known bugs that might cause Missing.foo() spans in my module? I'm not doing anything with ctx, getting it straight from the go sdk...

[thorn moat]

yeah calls made against array elements have that bug

[sharp zealot]

Ah that explains it yup

[sharp zealot]

https://github.com/dagger/dagger/issues/11528

[sharp zealot]

@thorn moat Want to try a new mega-trace with engine test splitting? 😛

https://dagger.cloud/dagger/traces/73747a956b9c2c45049111555bb099a3

well that did not go well 😂

 1

  ✘ engineDev:tests:core/integration_TestLegacy 3m48s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⡇⣿⡄ ERROR
  ┇ engineDev:tests:core/integration_TestLegacy:run › Missing.run ›
  ✘ withExec ln -s /.dagger-cli /usr/local/bin/dagger 32.3s ERROR
  ┃ runc run failed: unable to start container process: unable to start init: fork/exec /proc/self/fd/6: resource tempo
  ┃ ly unavailable
  ! exit code: 1
    cleanup failed: "runc delete container": /usr/local/bin/runc did not terminate successfully: exit status 1:
    container does not exist

[sharp zealot]

Nice side effect of your new "checks view" UI @thorn moat : even our existing Github Checks from GHA get the UI improvement when you click on them

[sharp zealot]

If you're looking to make a first contribution to the dagger engine, this is a good one: https://github.com/dagger/dagger/issues/11569

[restive shore]

I added a comment here - https://github.com/dagger/dagger/issues/11188#issuecomment-3697066336. But just FYI, I don't think the user defaults should be printed out when the terminal progress ends.

[sharp zealot]

Yeah agree. Even during progress we're getting too many lines printed in some cases

[restive shore]

I am testing checks. I have the following pseudocode

jobs := parallel.New().WithLimit(3).WithRollupLogs(true).WithRollupSpans(true)

// +check
func Lint(...) error {
  ...some work...
  
  for _, module := range modules {
    jobs = jobs.WithJob(module, func(...stuff...) error {})
  }
  
  return jobs.Run(ctx)
}

When I run dagger check lint, I expected each job to show as a line item. However, I only see lint and everything else is hidden. I only see them when I expand (press Enter) into that span. Is that expected?

[sharp zealot]

👋 there is no stable API (yet) for reporting subchecks in a check's telemetry. There is a hack to achieve it, but again, not stable 😁

look at toolchains/go and how I call parallel

[restive shore]

Understood! I thought I applied the hack with WithRollupLogs and WithRollupSpans. I was going by the dagger/dagger Go toolchain. Let me check again

[sharp zealot]

that's the one. the api is brittle atm.

[restive shore]

it has potential

[sharp zealot]

oh it's right there in your snippet - sorry

[sharp zealot]

🙋 https://github.com/dagger/dagger/discussions/11652

cc <@&946480760016207902> as discussed today!

[sharp zealot]

+ship 🙂 https://github.com/dagger/dagger/discussions/11653

cc <@&946480760016207902> also as discussed today

dagger ship let's go!

[shadow notch]

This is interesting, for a while I've been trying to formulate a somewhat related question.

I'd like to use dagger to replace an entire github workflow.
To use dagger terms our workflow has check jobs and ship jobs.
For PR events all steps execute in parallel, for merges ship jobs wait for check jobs.

This would make it theoretically possible to run both scenarios as a single dagger invocation.

[sharp zealot]

exactly. We actually just unplugged GHA for all checks in our own CI 😇 Just straight git events to Dagger engines on scale-out infra.

Would like to do the same for releases/deployments

[shadow notch]

own CI?

I think this feature would allow for our merge events to work, it's just dagger check && dagger ship.
But for PRs, where we ship regardless of check results, would be nice to have a dagger ship --include-checks

or --include-checks=parallel vs --include-checks=first

[sharp zealot]

Yes. We're onboarding our first early access customers this week if you're interested 😁

The issue with running this on top of legacy CI is that you still need to configure scale-out manually in a separate CI config

[shadow notch]

I only know about scale-out from looking at the code, now I can guess what it's for 🙂

and yes, I am interested

[restive shore]

I noticed that changesets are not documented at all. Do you want me to open an issue for it? It's not a hidden feature is it?

[sharp zealot]

Mmm, I guess they are in the reference, but otherwise yeah - we have a docs bottleneck at the moment. A contribution would be appreciated! 🙏

[restive shore]

@thorn moat , I am not seeing my printf statements in the TUI even with -vvv. Is that a regression? I remember being able to see those with just -v. I have to use --progress=plain to get to those. v0.19.19

[thorn moat]

what SDK? a repro would help too if ya have one

[restive shore]

Go, I'll create a repro

Prints are not displayed in TUI

[restive shore]

With everything new going on recently, I wanted to clarify. Is there work happening on enabling native testing? Like using go test to test dagger? I am getting more folks that are adopting dagger and one of the big gaps is not being able to run test tooling to generate code coverage. It's a requirement for any project at the enterprise level.

For example, we can even generate test coverage for Groovy code written for Jenkins.

[sharp zealot]

Are you talking about testing your non-Dagger codebase? Or testing your Dagger modules?

[restive shore]

Dagger modules. I can still use native tools to test other code.

[sharp zealot]

Sorry for dropping. I believe @upbeat herald had a POC of native testing of your modules

Built on generated clients (which we're starting to use more)

dagger toolchain install vs dagger install 🧵 🍿

[upbeat herald]

Yup: https://github.com/TomChv/dagger-native-test-example/tree/main/go
/cc @restive shore

[restive shore]

Native toolchain testing with generated clients

[restive shore]

How does function caching interact with image layer caching? Does function caching supercede layer caching? Or is layer caching still happening too? It wasn't clear to me from this doc https://docs.dagger.io/extending/function-caching/. Do I still have to do cache busting in certain scenarios?

[restive shore]

When I install a bunch of toolchains, and do dagger toolchain list their description is blank. Where is that description coming from?

[warped canyon]

It should be the module description of each toolchain module, but I think it was broken on 0.19.8 or 0.19.9. Are you seeing missing descriptions on 0.19.10?

[restive shore]

I don't think I have module descriptions. Is that something that goes in he dagger.json of each module? Or is it the package description?

[warped canyon]

Yeah the package description, like https://github.com/dagger/dagger/blob/main/toolchains/engine-dev/main.go#L1

[restive shore]

I'll try that, thank you! One more question. Is there an issue with toolchains inheriting defaultPath? I have a situation like the following

Module A -> sets defaultPath
Module B uses Module A

Toolchain install Module B in a entirely different app. It doesn't seem to inherit Modul A's default path. Does it only inherit the default path for the immediate parent?

If I toolchain install Module A in that app, the default path work fine

[sharp zealot]

I see Kyle typing with more details 🙂 I'll just say - the paint is not dry on this general area of the design

[warped canyon]

Sounds like a case we don't have a test for, but also exactly why we're trying to create a better context API https://discord.com/channels/707636530424053791/1463332258453651569

For now, would you be able to use a defaultPath in B and pass that as an explicit argument to A?

[restive shore]

oops! Trying to sail ahead of the ship I see.. I am liking the toolchain design so far!

[restive shore]

Sounds like I'll need both. I want module A to be usable standalone too. So I'll have to enable that argument in Mobule B as a defaultPath. I was hoping to not expose that arg in module B.

[sharp zealot]

Yeah FYI @restive shore we want to give toolchains a new API for accessing their caller's context directory (their "workspace"), other than +defaultPath - because right now we are overloading the meaning of +defaultPath to mean very different things depending on how you use a module.

If we succeed in designing this API (want to get consensus on a proposal this week), then in the future we would revert +defaultPath to always mean "path in the current module's directory", and never "path in the caller's context directory". This would be a breaking change for current toolchains... But we'd make sure it's a seamless migration

[warped canyon]

Yeah I should be able to support this, I was more proposing a workaround until I can make that change 🙂

[restive shore]

Yeah, even being a power user, it's not easy to understand. A new user will be totally lost

[sharp zealot]

Yeah exactly, we've been layering features organically for a while, we want to take a step back and make sure the end-to-end UX makes intuitive sense

[sharp zealot]

Non-urgent dang question: how easy is it to embed code in my Go program that can evaluate a dang expression?

If hypothetically, I wanted to allow certain values in my config file format to be strings evaluated as dang expressions? 😇

[aliases]

playground="engine-dev.playground"

[sharp zealot]

@rocky harbor @thorn moat I'm geting the same entrypoint/main.go:23:2: package dagger/dang/internal/querybuilder is not in std that you guys were talking about, while developing a module. I just upgraded to 0.19.11.

Am I holding it wrong?

[thorn moat]

try bumping the sdk pin, if it has one?

[rocky harbor]

Yeah need to bump the commit

I guess it was breaking-ish in that sense but in experimental territory

[sharp zealot]

ok I'm on it

[sharp zealot]

👉 https://github.com/dagger/dagger/pull/11810

[sharp zealot]

@sour citrus this should fix it

[sharp zealot]

<@&946480760016207902> after talking with @upbeat herald and @sour citrus a few minutes ago, I am now up-to-speed on the whole "telemetry import" problem. Here is an issue: https://github.com/dagger/dagger/issues/11815

[rocky harbor]

That's all done here https://github.com/dagger/dagger/pull/11809, discussion here #1468421235364073582 message

There's a remaining problem though that I just discovered: #1468421235364073582 message

[sour citrus]

As a way to catch this kind of issue earlier, would it make sense to do something like that? (adding one extra layer on ci:bootstrap)
That way it should ensure a built version of dagger can build itself.

diff --git a/toolchains/ci/bootstrap.dang b/toolchains/ci/bootstrap.dang
index 590b5ea88f..1104225457 100644
--- a/toolchains/ci/bootstrap.dang
+++ b/toolchains/ci/bootstrap.dang
@@ -18,7 +18,10 @@ type CI {
                                                playground().
                                                withMountedDirectory("./dagger", source).
                                                withWorkdir("./dagger").
-                                               withExec(["dagger", "checks", "**:*lint*"]).
+                                               withExec([
+                                                 "dagger", "call", "engine-dev", "playground","with-directory", "--source=.", "--path=.",
+                                                 "terminal", "--cmd","dagger,check,**:*lint*",
+                                               ]).
                                                sync()
                                null
                }

[restive shore]

I am trying to install a toolchain in my CI but keep seeing this
module source "github.com/mydaggerverse/go-pipeline@main" kind must be "local", got "git"

What could it be? I can run the same command from my local machine and it works fine. Here's some more context. This is happening in 0.19.10 and 0.19.11 both

Error: failed to check if module already exists: module source "github.com/mydaggerverse/go-pipeline@main" kind must be "local", got "git"

My command dagger toolchain install github.com/mydaggerverse/go-pipeline@main

[urban granite]

hey all... Trying to create a module. But when I install this module and try to return the main object from it I get the error failed to add object to module "app": failed to validate type def: object "App" function "debug" cannot return external type from dependency module "php" which seems weird. Can't we return objects from modules at all ?

[sharp zealot]

Not at the moment... It's to avoid a dependency hell problem, explained in https://github.com/dagger/dagger/issues/8529

There are possible solutions, but we haven't prioritized them so far.

[urban granite]

I see... thanks for the link. It does create a hard limit on what's possible with dagger... I'll find a workaround at the moment. thank you

[sharp zealot]

Happy to look at code snippets to help find the best design for what you're trying to achieve!

[urban granite]

basically : a Php module to help php developpers to get started with dagger and easy setup pipelines. So a Php object of course. Then I have nested objects to easily add tools that are common such as Pie and Composer. So I created a withPie and withComposer methods on Php. Doing so it felt "logical" at the time to create a module that could be used in dagger shell to progressivelly build something in place of Dockerfile. Instead they must build it themselves and cannot debug intermeidary parts without breaking the online in Php into multiple ones so they can call container()->terminal() for example. I'm finishing a minor issue I have and will publish the draft here to get feedback

on another note for some reason: for a few eeks now, I don't have any error details locally. I always have to go on the dagger.cloud to undertsand why it failed. Would be nice to have full output of errors locally too.

[urban granite]

@sharp zealot : Here is my WIP https://github.com/Neirda24/dagger-php-module/tree/feature/php (started yesterday trully workign on it). I'll be pairing up with @fervent willow to try to submit a live coding at SymfonyCon 2026 of a pipeline from scratch based on a Php module + blueprint

[restive shore]

This feels like a bug. When I have a dagger module in a repo (say a go app), and I install a toolchain on to it and call the toolchain, it still calls the constructor of the module in my repo. That shouldn't happen IMO.

[restive shore]

dagger toolchain update seems to be a no-op 🙁

[restive shore]

I also noticed, when I install a toolchain to a existing module, I can also access it from the dag client. Is that intentional? I was under the impression toolchains don't work that way.

[restive shore]

Another one! Sorry, I'm on a frenzy plugging in toolchains to my pipeline

I have a scenario where I want to install a toolchain in a pipeline (in CI) regardless of whether the app repo set it as a toolchain. Basically, I want to run my toolchain in CI and not the version that the app has installed. To do so, my plan was to dagger toolchain install with a --name of first 7 chars of the git sha. The toolchain installs fine BUT when dagger exposes the function in the CLI, it does the hyphens between letters and numbers. For example, if my name was go-pipeline-e855c03, the function generated is go-pipeline-e-855-c-03. Which it makes it hard to determine the exact way to call it. Hope this makes sense.

[warped canyon]

Yeah that's a tricky one 😂 Is this committed or just done in the CI environment and lost? If its ephemeral, I guess you could use a known safe name instead of the commit

[sharp zealot]

@restive shore FYI we are collecting all feedback on toolchains so far (from ourselves and others), including on these tricky edge cases, and incorporating them into "modules v2" a backwards-compatible overhaul of the module system, that will keep the best parts of toolchains and resolve the awkward parts. We will most likely retire the word "toolchain" in favor of "modules" (basically all modules will gain optional toolchain capability). It will include backwards compat and migration tooling 🙂

So, don't stop what you're doing, but be advised we're working on solving these rough edges.

PR if you're interested: https://github.com/dagger/dagger/pull/11812

[restive shore]

Thank you! I've been following and I am excited for it! 🙂 I'll keep using toolchains and report issues

[sharp zealot]

That PR addresses a big "knot" of interconnected problems... And as a result it drops a big "knot" of interconnected features 🙂 Usually we like to ship small and often, but this one is the exception. Hopefully everything will feel "cleaner" across the platform afterwards.

Among other features:

• Module config (user defaults v2): instead of juggling cryptic .env keys (hard to guess) and the new toolchain "customizations" in dagger.json (annoying that they're separate; no UX for setting them), you just set key-values directly in a little toml file. Will be sweet 🙂 Also when you install a module, all available config keys are added to the config file (commented out) to help you easily figure out what config is available

• Dynamic filtering. Unlike toolchains / +defaultPath, modules can dynamically read files in the their workspace, with arbitrary ignore filters. So for example, a module for go development can dynamically read and parse go.mod, find replace directives, and load only those files from the workspace. etc.

• Happy path for project-specific modules. dagger module init <foo> creates a module in a standardized location in your workspace (.dagger/modules/<foo>) and auto-installs it in the workspace. You can override it, but out of the box it's less decisions to make, and less internal concepts to learn.

• Remote workspace 🙂 dagger --workdir github.com/dagger/dagger@main check -> boom, load a workspace from a remote branch, run your checks on the fly.

• Compose modules and workspace. dagger --workdir ./my/app -m github.com/dagger/jest check -> run the jest module in the context of my app's workspace. Nice separation of concerns. Also works with remote workspaces of course 🙂

[restive shore]

Lot to take in, most of it makes sense, others I'll have to evaluate in practice 🙂

With user defaults v2. Are you taking away the ability to share user defaults? OR can a module define "env" defaults that will be inherited?

[quick wind]

Glancing through that PR... Yep, a lot to figure out. I bounced off toolchains, didn't see what they would give me that I didn't already have (which probably means I'm not the target audience). Happy to see there's some effort into streamlining as well, features are well and good but one of Dagger's key selling points is usability, and complexity can quickly get in the way of that

[sharp zealot]

The implementation of the PR is complex, and the design diff is large from the POV of a power user... But the result is a MUCH SIMPLER user experience. It's the design equivalent of a huge PR with 90% red

[quick wind]

More of user defaults is happy news, I'm using that feature heavily and it has streamlined my iteration cycles considerably

[sharp zealot]

The end result is:

• End users can setup dagger and get real value with zero code
• Power users (typically central platform team) has more control so they can focus on writing and distributing modules, and waste less energy helping end users write boilerplate code, and/or maintain a wrapper tool to "hide the complexity of dagger"
• Concepts are more familiar, less "weird and uniquely dagger" concepts that break your brain

[quick wind]

.dagger/config.toml — workspace config. Lists which modules are loaded when you run dagger call.

Now that is an excellent idea

[sharp zealot]

and how to configure them

[quick wind]

If I'm understanding correctly that solves two minor issues with adoption: distribution (user has to know there's a module at some git location, and to call it with dagger -m) and convenience (user doesn't have to spend as much time running dagger functions dagger call <method> --help)

[sharp zealot]

Exactly. Especially in combination with new higher-level commands like dagger check, dagger generate, and others yet to come. Modules can just extend those commands directly without end-users needing to pass extra arguments, write custom shell scripts, or write boilerplate dagger functions

[sharp zealot]

I think once this series of changes is shipped and stabilized, we should be able to start the countdown to 1.0

[sharp zealot]

https://tenor.com/view/dont-matter-care-peaceful-laid-gif-22992945

[restive shore]

Had a closer look at the PR and have a few questions:

1. With user defaults v2. Are you taking away the ability to share user defaults? OR can a module define "env" defaults that will be inherited?
2. Is there a change in how things are cached? Is function or layer caching changing? Cache volumes?
3. Is it still possible to run dagger modules in a repo that doesn't have any dagger config? Today, from CI, I can dagger init, dagger toolchain install and run CI without any user config. From what I'm gathering, that's no longer possible with workspaces? The end user will have to "initialize" a dagger module (with or without code)?
4. In the workspace world, how does an end user expand with code if they want to? Is it, initialize the repo as a workspace, and dagger install [module] as usual?
5. How does this affect "daggerverse" type dagger monorepos? Would module producers also follow the workspace pattern or is it business as usual?

[sharp zealot]

1. With user defaults v2. Are you taking away the ability to share user defaults? OR can a module define "env" defaults that will be inherited?

What do you mean by "sharing user defaults"? I don't think we'll be taking away any features there, but curious what you mean.

[sharp zealot]

2. Is there a change in how things are cached? Is function or layer caching changing? Cache volumes?

No change.

[sharp zealot]

3. Is it still possible to run dagger modules in a repo that doesn't have any dagger config? Today, from CI, I can dagger init, dagger toolchain install and run CI without any user config. From what I'm gathering, that's no longer possible with workspaces? The end user will have to "initialize" a dagger module (with or without code)?

I don't think there will be any change in that regard. But wait, you re-run dagger init on every CI run? That's kind of unusual if so. Why not init once and commit the result? Anyway, this workflow should get easier actually: dagger -m <module> call ... will be the same as dagger install <module>; dagger call .... In both cases the module will run in the context of your workspace.

[sharp zealot]

4. In the workspace world, how does an end user expand with code if they want to? Is it, initialize the repo as a workspace, and dagger install [module] as usual?

Depends what you mean by "extend with code". Do you mean, write a project-specific module, eg. with project-specific build functions etc?

If so, from anywhere in your workspace you run dagger module init <foo>, and a new module is initialized in .dagger/modules/<foo>. The module is also added to your workspace config so it's ready to be used.

[sharp zealot]

5. How does this affect "daggerverse" type dagger monorepos? Would module producers also follow the workspace pattern or is it business as usual?

Those will continue to work the same. But yes, developers of standalone modules (daggerverse style) can themselves create a workspace to help develop their modules. the workspace can be per-module, or repo-wide, up to you. This makes it more obvious how to develop dagger modules with dagger. Basically it's an evolution of the "dev module pattern"

[restive shore]

Today, we are able to upload a .env file to the module repo and the consumers of that module will get that env by default. They can of course modify that by having their own .env.

[sharp zealot]

That's not supposed to work... Unless you're talking about fetching that repo locally, then running dagger against the local checkout

We intentionally disabled loading of .env from within the module when loading it remotely (I mean from the beginning of the user defaults feature)

[restive shore]

I'm pretty sure it works today. I'm not at a PC but I can confirm.

[sharp zealot]

If it does work, then yes that would break with -m. But if you walk me through your use case, I can make sure there's a clean and pleasant solution

If it does in fact work and you rely on it, I'm sorry- that's a bug that we failed to avoid

[quick wind]

I've just read through https://github.com/dagger/dagger/blob/workspace/docs/design/proposals/01-module-vs-workspace.md, this is a brilliant direction.

"Only constructor arguments can be configured" -> this is a loss of functionality over the current .env files which can provide function arguments as well as constructor arguments. I think in most/(maybe)all of our modules we could shift stuff to the constructor but we're probably in for some overloaded constructor functions to do so.

".env files remain supported for non-constructor function argument defaults, which cannot be expressed in workspace config." -> I don't want to have to use two sources of values, and I don't want to explain this to users either. The workspace config move is a good one, is there a reason why workspace config doesn't allow function args?

Aliasing functions but not allowing workspace args to provide values for those now aliased commands feels like a missed opportunity.

[sharp zealot]

Well you're the one who worried about simplicity 🙂 A config file that only supports constructor arguments, is simpler for the end-user. It's just key-values per module. Supporting arbitrary functions & types made the .env system that much more complicated.

We're going to start more constrained - since it's always possible for module devs to move arguments into their constructor. Then we'll see what breaks. I suspect in practice, it will work fine.

Basically, you get a new interface with your end-users: a super user-friendly config interface. The price to pay is, if you want it exposed to the end user, you have to put it in your constructor. I say it's a good trade, for getting your end users to actually using dagger, and your modules

.env remains supported for backwards compat, but we won't steer new users towards it at all. It will be disabled by default, with a special config key to enable it in your workspace config (for power users who were using it before and need compat)

[quick wind]

Yeah, as I said I think this can be done with the outcome that constructors are going to be unusually large in a small number of cases

I'll take that over "so there's two sources of inputs, depending on where (in the module you didn't write) the value is required" which just won't land

I think I've gotten too used to dagger call <method> <method> and everything, constructor and both methods' inputs are envfiled away

I see a few "coming soon"s in and around this, do you have an estimate for this being released?

[quick wind]

Whilst I'm pondering, questions about the workspaces:

1. If I have a constructor that has args 1..10, args 1-4 are used in both local/ci, args 5..7 are used locally and args 8..10 are used in ci... Do I just duplicate args 1..4 in my workspace config, so workspace "local" has args 1..7 and workspace "ci" has args 1..4 & 8..10?

2. Related to the above, is the default workspace named? Is it clear from config files what you're adding to a default vs a provided workspace? Could the "local" just be default and I otherwise define a "ci" workspace and use the default for local?

[restive shore]

"Only constructor arguments can be configured"
I missed this when I read the proposal. IDK how I feel about this. For example, say I have a sonarScan function. The SONAR_TOKEN is only required for that function. I am sharing a pipeline module with the user that will selectively execute functions from that module. However, if only constructor args can be defaulted, the user will have to provide the SONAR_TOKEN even when they run a lint operation for example.

One way to avoid this would be the make the SONAR_TOKEN optional. But then, we'll be doing checks within the sonarScan function to fail if the token is not present. Do I have a duplicate required param for the function and do a check? What about other specific settings that I'd like the user to be able to default because there are a ton? Passing everything through the constructor seems cumbersome. Maybe I'm thinking too much into this and it'll be better in practice

The price to pay is, if you want it exposed to the end user, you have to put it in your constructor.
Most credentials can't be defaulted though. In my case, there are a lot more settings exposed to the user than not. Majority of functions require some sort of credential and not always the same within the module.

[sharp zealot]

In your case the answer is probably to move sonarScan to its own module

If your constructor becomes a kitchen sink of unrelated parameters, it could be a sign that you should split it up into more decoupled modules. Which modulesv2 makes easier

[restive shore]

It is a separate module. I have bigger pipeline modules that compose an e2e pipeline with that module.

Sticking to the same example, sonar is a thing that is required in every pipeline. Why would I require consumers to manually pull it in every time?

Let's say we did do that. Sonar can't work by itself. It needs artifacts from other steps like testing to do its job. End user will have to do some wiring (now in a config) again if I don't do it for them.

[sharp zealot]

Yes there's a missing composition primitive for scanning I agree.

By "separate module" I mean: a separate top-level module to be installed directly in the project. In the particular case of scanning, that requires filling the composition gap.

[thorn moat]

for folks using Dang - just merged a new REPL, with autocomplete and a new docs browser, :doc: https://asciinema.org/a/kpZm7kn9l6FZeh2d
usage: go install github.com/vito/dang/cmd/dang@main; cd my/module; dang (works for any module, not just Dang modules)

[sour citrus]

quick request: items under :help are not sorted, so each time you run :help it gives them in a different order 🫣
fixed, PEBKAC

[sour citrus]

I don't have the same output, for instance I don't have the loader, and no auto completion. am I missing something? I just tried dang in toolchains/java-sdk-dev
fixed, PEBKAC

[restive shore]

Wow, this is really impressive. Won't this work for modules written in any language? Using dang in the shell

nvm! I can't read

really cool!

[thorn moat]

yep! i've started using it already to debug stuff during dev

[sharp zealot]

@thorn moat we should add that repl to the dagger CLI somehow (or maybe with modules v2, it can just be a module?).

Also we should talk about moving the sdk in the dagger repo maybe - or at least getting the "sdk": "dang" short name

[thorn moat]

tl;dr: completely agree with the end goal, need to investigate how to bridge the gap
longer context: this started as exploring Bubbletea v2, and then led to flipping the table and rolling my own TUI model inspired by pi's TUI, since Bubbletea is just inherently very bad at the REPL use case. Our current implementation is insanely complicated (you may remember me battling with scrollback behavior for days ages ago), so this was a weekend exploration to see if v2 somehow helps (it doesn't)

[restive shore]

Is it intended that we can't set flags like verbose and use it from the CLI because dagger already specifies it? I see ! flag already exists: verbose.. Lots of tools use verbose so it's not easily replaced

[thorn moat]

not intended, but known issue: https://github.com/dagger/dagger/issues/7032 - historically a bit of a pain for obscure technical reasons, could maybe be reconsidered

[restive shore]

That would be awesome! Thank you! Not a critical thing but helps with the DX for sure

[restive shore]

I tried to find some reference, but couldn't is there a // +deprecate pragma to deprecate a module function? I remember seeing something like that before

I see this - https://github.com/dagger/dagger/pull/11071. Hidden feature? 🙂

[sharp zealot]

indeed @void widget shipped this 😁

[restive shore]

I didn't see this in the notes @void widget for that PR but the Go way of deprecating functions with // Deprecated: details also seems to work. The generated code is marked deprecated. This would be a great feature to be documented! 🙂

[void widget]

Thanks Nipuna, I'm glad you find it useful 🙏 I wanted to do another pass on top before fully documenting, but you're right. Adding a todo for this week 🙏

[sharp zealot]

Heads up Daggernauts. It's been several months since we've communicated about what we shipped, beyond this discord. I'm drafting a product Changelog, which we'll publish on our website, newsletter and social.

It's still WIP. Would love your feedback!

https://gist.github.com/shykes/c71882508105862c15ef37a5f40705d8

[restive shore]

lgtm. I have one question. I was under the impression that --progress=logs was more for LLM usage than for CI/terminal output. This comment by vito seems to suggest the same https://github.com/dagger/dagger/pull/11847#issuecomment-3888128540.

Should I be using logs over dots for CI?

[restive shore]

Does dagger vault secret provider handle oidc auth flow? (open browser to login)

[warped canyon]

Not at the moment, but it should be possible to implement as we're just passing through to the native go client https://github.com/dagger/dagger/blob/main/engine/client/secretprovider/vault.go#L114

[restive shore]

Starting to enforce vault and oidc is the only option for auth 🙂 Would be awesome to have that capability ootb. Because afaik, the browser auth flow won't work anywhere else in dagger either right?

[warped canyon]

Yeah makes sense. We've been talking about how to support it for Dagger's native CI and that would apply to the open source client too. The browser flow should probably work since the secret integration is purely client side

[restive shore]

Do you think you would work on it in the near future? If not, I could take a stab at it with a little bit of guidance.

[warped canyon]

Its probably on the medium-term future vs near term, so any help is welcome 🙂 the code currently looks for an environment configured token or app role, and I think all thats needed is to fall back to try oidc if those are not set

[sharp zealot]

@thorn moat let's say hypothetically I gave a Workspace binding to a LLM. Would it work? (trying now 🙂

[thorn moat]

https://tenor.com/RfTn.gif

(claude)

[sharp zealot]

Ironically, passing a Workspace might improve LLM performance compared to passing a pre-fetched Directory, because there are way less functions 🙂

[sharp zealot]

@thorn moat anecdotal feedback: by far the most common issue with my dang code is nullable vs. non-nullable.

Error: cannot use String as String!
  --> /mod/modules/docusaurus/docusaurus.dang:119:7
     |
 117 |     } else {
 118 |       let docusaurus = json.field(["docusaurus"])
 119 |       docusaurus.field(["nginxConfig"]).asString
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 120 |     }
 121 |   }
     |
! failed to call module "docusaurus" to get functions: cannot use String as String!

--> I see variations of this error dozens of times a day

[thorn moat]

that's meant to be a feature - the thought being to handle both possibilities or disambiguate, rather than blowing up with a null panic or something (catch more errors prior to runtime)

so i wonder what should be fixed around the periphery, to find paths out of that state more easily

[sharp zealot]

Yeah not making a judgement on that, just dumping raw experience

[thorn moat]

what's the resolution with this one? foo ?? ""? or maybe raise? (that's a thing now)

[sharp zealot]

I don't know

Because I don't really understand the error 🙂

[thorn moat]

oh, it's saying that value is nullable, but your function signature says non-null, but a special message is def warranted given the frequency

[sharp zealot]

But the inline IDE doc says asString returns String!

So why do I need to add the ?? ""

[thorn moat]

ah it's because the .field is probably nullable

[sharp zealot]

ah but json is a nullable field

[thorn moat]

huh. maybe the LSP is wrong?

oh

[sharp zealot]

So the nullable property is "contagious" kind of?

never thought of that

[thorn moat]

yeah - maybe that should be changed

[sharp zealot]

field of (nullable thing) == nullable (field of thing) ?

[thorn moat]

exactly

[sharp zealot]

https://giphy.com/gifs/confused-brain-confusing-EvaWVaNS55g4tHc12w

[thorn moat]

will have to check if that's load-bearing. it might map to GraphQL semantics in some way

if it's likely to propagate into a type mismatch error anyway, maybe it's for the best that x.foo requires x to be non-null. feels less surprising, at least

could always add a ! postfix operator for non-null assertion/casts with better errors (foo.bar!.baz)

[sharp zealot]

Really felt the need for this today 😭 https://github.com/dagger/dagger/issues/11190

[sharp zealot]

@thorn moat @warped canyon in case you remember - in LLM, can I block individual functions of a single type, or only top-level functions? (trying to give my LLM a workspace, but only allow calling Workspace.file() and not Workspace.directory()

[thorn moat]

you can block any function of any type i believe - it's LLM.withBlockedFunction("TypeName", "funcName") iirc

[sharp zealot]

oooh

[thorn moat]

(and it'll error if you give something unknown)

[sharp zealot]

Yay, IDE is happy (0.20.0 has Workspace)

@thorn moat can I cast a string to JSON (scalar byte-array wrapper type) in dang?

[thorn moat]

"foo" :: JSON should do the trick

[sharp zealot]

Tried... will try again

ARG it was "foo" :: JSON! nullable strikes again

btw auto-completion in object chains is not working in IDE (even though the rest of the LSP works fine)

[thorn moat]

@sharp zealot while we're on nullability, wdyt of changing JSONValue.field to return null if the field is not found, instead of erroring and returning JSONValue!? seems like that would simplify a bunch of cases in your docusaurus module

[sharp zealot]

Yes 1000% , that was on my todolist already- it even already returns a nullable ! we just never actually return null...

[thorn moat]

connecting dots for continuity - i'm seeing the value of null cascading, at least for this style of code, which might be common in Dang's domain: #1476967508354535587 message

in Haskell terms this is sort of like the Maybe monad 🤓 - which I also used heavily back in the day

[thorn moat]

@sharp zealot am I crazy or is json.withContents(toJSON(ws.id) :: JSON!).asString just toJSON(ws.id) with extra steps 😂

[sour citrus]

https://github.com/dagger/dagger/pull/11926

[sharp zealot]

probably, I couldn't figure out a typing error so threw the ai at it. I'm sure it took a little over engineering detour

[sharp zealot]

@thorn moat do you have a gap in our progress outputs for LLMs? There's no frontend made specifically to give the AI maximum context on the trace, but in a readable way

--> no tty
--> span hierarchy presented as structured data, not visually
--> timestamps (to perceive slowness etc)
--> logs not dumped by default, but queriable with special flags on dagger trace?
--> span IDs?

[sharp zealot]

Context:

[warped canyon]

Makes sense. In the same way that spans are collapsed in the TUI to save human context, it would benefit the LLMs the same way. We'd still need a way to search for some pattern in all child spans rather than traversing span by span

[thorn moat]

i wouldn't take much from that, dagger trace is bugged at the moment and doesn't produce any logs every time i try it

i'm also not convinced that an LLM would prefer structured data over visual representation, in my experience the human-readable representation always works better, but maybe that experience is outdated

i still expect --progress=report and --progress=logs to do a pretty good job

not discounting that there may be a better way, just contending with this specific repro, really need an example where it legitimately broke down since I've seen it work just fine locally for a while now

[thorn moat]

just merged https://github.com/vito/dang/pull/37 - try that out! now the completion is powered by tree-sitter, and parse errors use it too so those should be much better

[quick wind]

I see this has been incredibly active since it was linked, is this a 'sooner rather than later' thing? I've been talking about this already and how it can reduce the need for Makefiles / Gitlab components and the idea is well-received.

[sharp zealot]

yes sooner rather than later. It's a major UX change (although with solid backwards compat & migration) and a multiplier on everything else. So the sooner we ship it, the better

[restive shore]

I started working on this and I think I've gotten the impl working. However, I'm having trouble running the integ tests. Is there an easy way to run them on my Mac? When I try to run them, I see

go test -v -run "TestSecretProvider/TestVaultOIDC"     
# github.com/dagger/dagger/engine/buildkit
../../engine/buildkit/executor_spec.go:1271:25: undefined: unix.OpenTree
../../engine/buildkit/executor_spec.go:1271:72: undefined: unix.OPEN_TREE_CLONE
../../engine/buildkit/executor_spec.go:1271:93: undefined: unix.OPEN_TREE_CLOEXEC
../../engine/buildkit/executor_spec.go:1271:116: undefined: unix.AT_RECURSIVE
../../engine/buildkit/executor_spec.go:1305:27: undefined: unix.OpenTree
../../engine/buildkit/executor_spec.go:1305:64: undefined: unix.OPEN_TREE_CLONE
../../engine/buildkit/executor_spec.go:1305:85: undefined: unix.OPEN_TREE_CLOEXEC
../../engine/buildkit/executor_spec.go:1320:20: undefined: unix.Unshare
../../engine/buildkit/executor_spec.go:1320:33: undefined: unix.CLONE_FS
../../engine/buildkit/executor_spec.go:1324:20: undefined: unix.Setns
../../engine/buildkit/executor_spec.go:1324:20: too many errors
FAIL    github.com/dagger/dagger/core/integration [build failed]

[warped canyon]

You can use the engine-dev toolchain 🙂 like this dagger call engine-dev test --pkg ./core/integration --run TestSecretProvider/TestVaultOIDC

I'm currently working on porting the test setup from that toolchain into the tests themselves so that go test would have worked. Close to being done with that!

[restive shore]

Here you go :). I can't add reviewers because of access. https://github.com/dagger/dagger/pull/11929
The hardest part was int testing the OIDC browser flow. I tested this on the enterprise vault as well.

[restive shore]

@warped canyon how can I help get this merged? 🙂

[warped canyon]

I can review and test it this week! We had a team off-site last week so I didn't get a chance!

[restive shore]

I am always amazed at how awesome the TUI is! Yesterday I had to debug something and pressing t anywhere in the TUI to get a terminal came in really handy. One thing I felt missing is a text search. Like a / to search forward/backwards. How hard would that be to implement? Wonder if it's easier in the new TUI? 🙂

[keen dome]

Oh yes, please, a search would be awesome!!! Was thinking the same

[thorn moat]

what do you want to search? span names? logs? both? (i'll go for both, just trying to understand the initial goal)

[restive shore]

Both, yes. If it was output in the terminal, I want to be able to search. Including nested (folded by default) text.

[thorn moat]

https://asciinema.org/a/831665

[restive shore]

Fantastic! Exactly what I was looking for!! ❤️

[thorn moat]

thanks for asking for it lol, now i'm gonna be using this all the time

[quick wind]

That is a fantastic little addition

[restive shore]

Updated the tests they all pass now 🙂 https://github.com/dagger/dagger/pull/11929

[warped canyon]

Merged! Thanks for working on that one!

[sharp zealot]

https://x.com/dagger_io/status/2034346749683454280

Dagger (@dagger_io)

Shipped: GCP Secret Manager is now a native Dagger secret provider.

`gcp://my-secret` works with ADC, service account keys, and Workload Identity on GKE. No wrapper scripts, no shell glue.

That's four cloud secret backends supported natively: AWS, GCP, Vault, 1Password. Which

X · Mar 18, 2026, 7:10 PM

[restive shore]

https://github.com/dagger/dagger/pull/11773 Gets me most of the way there but I still need our certs to be mounted so I still have to maintain a custom image. Any thoughts on auto mounting the system certs based on an env var?

[sharp zealot]

Next chunk of the Modules v2 / workspace branch just dropped, and it's useful on its own: Dagger Lockfile.

--> Calling for pre-reviews 🙂

https://github.com/dagger/dagger/pull/12046

[restive shore]

Love this! Hope it enables a path to using dependency update bots like renovate and dependabot

[sharp zealot]

you mean for updating the code of your modules? or something else?

[restive shore]

Opening PRs to update dependencies. For example, if I have pinned to a alpine:1.2.3 I want renovate to open a PR to update to alpine:1.2.4. OR if I'm importing a dagger module from somewhere which is v1.2.3, open a PR to update that dagger module to the next version when one is available. Today we have to ignore go.mod and go.sum because it breaks things if renovate updates depds off band

[sharp zealot]

Yeah should be as simple as calling dagger lock update and opening a PR with the result

[restive shore]

Does dagger lock update actually update // +default= values too?

[sharp zealot]

Can you give an example default value, so I'm sure I answer what you have in mind?

[restive shore]

This is fresh in my mind so I'll use it. I use trivy for container scans. I have a default set to // +default=v0.69.3. When a new version is available, I'd like dependabot or renovate to tell me to update it. They both have the ability to watch Dockerfile and suggest updates to the FROM line (if a version is provided). Same concept. This isn't probably possible without some sort of plugin so that renovate would understand how to update dagger deps. Today, dagger doesn't have any lockfile (or a single place where all versions are managed). I'm hoping there's some sort of a solution to this in future.

[sharp zealot]

I see. This will be possible in the future using Dagger's lockfile, but not right away. The lockfile system is designed to be extensible, so we're starting with the absolute minimum feature set, and will expand over time

[sharp zealot]

At its core, a lockfile is a persistent record of lookup functions. Dagger has plenty of lookup functions, so over time the lockfile will persist more and more of them.

In the first version, we persist:

• Container.from() (image address -> digest)
• GitRepository.branch() (branch name -> commit ID)
• GitRepository.tag() (tag name -> commit ID)
• GitRepository.ref() (generic ref name -> commit ID)

For your example to be covered by the lockfile, it would first have to be expressed as a lookup function.

Potentially it could be a new Container.fromLatest() operation which takes an image name, and returns the latest semver tag.

Or, you could define your own custom lookup function, and annotate it with +lookup, and it will be automatically persisted in the lockfile as well 🙂

[restive shore]

Understood! That's what I assumed. With all the security breaches recently (cough!.. Trivy), we are under more pressure to use lockfiles and dependency update tools to manage versions.

[sharp zealot]

yeah exactly 👍

Dagger is in a position to help with that. Orchestrating your CI/CD with Dagger will get you a universal lockfile basically

[restive shore]

Yes that's a huge plus for Dagger, honestly

[sharp zealot]

Open question. Modules v2 look pretty aligned with skills. Can we take advantage of this somehow?

[restive shore]

Are you referring to agent skills?

[sharp zealot]

yes

[restive shore]

So, yes! Any help we can get with agents is good.

[wintry prism]

If you make it easy for agents to use Dagger, they will.
I see a lot of projects that have a simple Makefile generated by the agents to do tasks. Guessing because make is on many systems, writing them is well-understood by the agents (even if fiddly for us), and running them is easy for humans.

[sharp zealot]

Would there be any interest in Dagger Cloud being a supported service in the new "Stripe Projects" tool? https://docs.stripe.com/projects#available-providers

[sharp zealot]

Is there a standard way for a client to serialize a client's public fields to JSON? Including its children objects, recursively?

[sharp zealot]

https://x.com/dagger_io/status/2041207631675793609

[sharp zealot]

Fellow daggernauts... if you know about project Theseus, you'll be excited to know: we're aiming to merge it this week 🤞

[restive shore]

I have a couple of dang questions 🙂

1. To get benefits of no code generation, do all the modules in the tree have to be dang? Just having the consuming module be dang is not sufficient I would assume?
2. Is there a skill/agent that you have been using to convert existing modules to dang?

[sharp zealot]

@sour citrus continuing separately: you know you don't actually need Workspace.path() - I'm curious why you use it

(ie. is there an actual use case where it's necessary)

[sour citrus]

I'll check again, but it was when replacing dagger develop by dagger generate, at some point I needed to know where the current directory is relatively to the root of the workspace. But maybe there's a different way to do that.
(IIRC)

[sharp zealot]

Workspace.path() will give the location of the workspace directory relative to the filesystem boundary (eg. the git root)

[sour citrus]

yes, I think that was the need, because I was dealing with multiple workspaces (one per module) so I needed to know where they were.
But based on discussions the other day, maybe it will not be needed anymore

[sharp zealot]

Ah of course!

Makes sense

(and yes I also hope you won't need it - but if anyone needs to deal with multiple workspaces in the repo, Workspace.path() would be very useful)

[sour citrus]

1. Yes, if some modules are not in dang they will require their own codegen. This is going to change in the future as we are removing the codegen from the runtime phase to keep it on the develop phase only. Meaning the generated files will be committed and so a dagger call will only build, not codegen
2. I'm not sure. I think people have some dang ones, but not 100% sure. I did use Claude to write some Dang code and it was good enough at it for my cases, but it wasn't to convert modules.

[restive shore]

It would be great if the dagger team could create a skill (or agent) that's capable of doing conversions between any lang SDK

[sharp zealot]

Calling daggernauts: we need testers for the workspace branch...

dagger -m github.com/dagger/dagger@workspace call engine-dev playground terminal

And in-progress new docs:

dagger -m github.com/dagger/dagger@workspace call docs server up

[quick wind]

Is this the only way to test, or is building a cli/engine container from that branch an option?

[sharp zealot]

you can use all the usual methods to build and test. These just happen to be convenient one liners 🙂

[quick wind]

So instead of terminal I can just export it and run as usual? I'll give it a go tomorrow.

[sharp zealot]

yes

[sharp zealot]

actually @quick wind if specifically you build playground, then that can only be used within dagger. It's a multi-container environment so can't be cleanly exported as a single container image to a registry (if that's what you meant by "export")

basically if you have a method fir building & testing dagger from source: that method will work with that branch too. Just make sure to test CLI & engine together (that's what playground makes easy)

@warped canyon FYI just pushing a docs commit that removes a bunch of FIXMEs in "Adopting Dagger"

[quick wind]

Testing .dagger/config.toml in the playground doesn't seem straightforward

[sharp zealot]

Yeah I just git clone repos, then use vi to edit 🙂

[quick wind]

Yeah that's basically what I'll have to do here

[sharp zealot]

there's also helpers dagger install, dagger config

and dagger migrate

[quick wind]

Is this close to release or is there more rebasing/autotesting to do?

[sharp zealot]

Still some rebasing. Trying to get as much QA and polish done in parallel

This branch got triaged to merge after theseus (full removal of buildkit) -> and that one is in the final stretch.

So rebasing on theseus added some delay - worth it though, we should be on much more solid ground (and much faster) on the other side

Meanwhile we've had a pretty disastrous week trying to cut the final release before the theseus/workspace combo

So most of the team has been busy stabilizing that - adding further delay

[quick wind]

Yeah I've seen a lot of the rebasing / automated testing chatter, have been keeping a close eye on the modules v2 progress. For our use-cases it's a significant step towards further roll-outs in cases where my team isn't both maintainer and user, but only maintainer

[sharp zealot]

Really appreciate that you're so locked in, and really sorry that it's taking so long. We're very anxious to get it out - IMO it's the single biggest blocker to adoption and expansion, you're very representative in that way.

[sour citrus]

Just sharing my workflow, when I'm testing it, I run this playground command: (that I have in a ~/.dagger loaded by my shell, with a lot more dagger aliases 😅 )

playground() {
    dagger call \
     engine-dev playground \
     with-directory --path=./src/current --source=. \
     with-directory --path=./src/dagger --source=https://github.com/dagger/dagger \
     with-directory --path=./src/demo-react-app --source=https://github.com/kpenfound/demo-react-app \
     terminal
}

That way I have the playground with:

• the current branch code (from my host) on /src/current
• the main of dagger/dagger on /src/dagger (so I can enter it and run a dagger migrate
• a demo app on /src/demo-react-app that is simpler than dagger/dagger to quickly test
  That way I don't have to think a lot and always have a read environment to cover a lot of different scenarios

[sharp zealot]

@warped canyon now making a big change to "Developing modules". Trying to imagine the coolest possible docs structure, assuming awesome tooling that we don't yet have 🙂 Then if we like it we can back into actually having the tools

[warped canyon]

Awesome sounds cool!

[sharp zealot]

Teasing thesus 😇 https://x.com/solomonstre/status/2044548400503472189

Solomon Hykes (@solomonstre)

FYI, Dagger is about to move off Buildkit, to a cleanroom reimplementation.

This matters beyond Dagger. Buildkit is load-bearing infrastructure for a huge chunk of CI/CD. It has fundamental limitations that are getting harder to work around, but it's too entrenched and complex

X · Apr 15, 2026, 10:48 PM

[quick wind]

Considered writing a blog post about this? Many of us aren't buildkit experts and there's probably a lot to learn about the limitations, workarounds, and reimplementation outcomes

[sharp zealot]

Oh yes. Perhaps a whole series!

After we ship it of course 🙂

[restive shore]

dagger update does not update toolchain pin. Has anyone else noticed that?

[warped canyon]

yeah its under dagger toolchain update intentionally, but this will go away with modules v2

[restive shore]

Oh, I didn't notice that! What will be the modules v2 way?

[warped canyon]

So in the current version dagger install | update | uninstall works with dependencies and dagger toolchain install | update | uninstall works with toolchains since they are separate configurations. In modules v2, its essentially flipped where toolchains get the top level, even though they won't be called toolchains anymore, just modules installed in a Workspace. This is because your .dagger/config.toml doesn't have dependencies or code, its a Workspace. All of the modules in it are used like toolchains today. And then modules themselves with a dagger.json will have dependencies

[restive shore]

gotcha!

[restive shore]

Is this something we can do today in 0.20.6? https://github.com/dagger/dagger/pull/13032 (Move Helm checks to native Go e2e
). I am very interested in adopting/teststing this pattern as I have a team that was put off by dagger becasue they couldn't write native go tests for dagger stuff.

[sharp zealot]

yes 👍 but note that this test suite is not testing a dagger module

We're gradually switching everything to this pattern - which in turn decreases the need for project-specific modules

@restive shore what's the test scenario exactly?

[restive shore]

What is it testing then? The workspace integration?

My test scenario is testing any dagger module. Workspace or daggerverse (module monorepo)

[sharp zealot]

no it's testing our official helm chart.

Testing dagger modules adds another layer to the problem since the module runtime is involved. We also want to solve that but it's not a requirement for that e2e/helm PR

[restive shore]

I see! I got too excited then 🙂 So this PR is using the dagger SDK to run and test an arbitrary helm chart?

[sharp zealot]

Well it's running one specific chart, but yes could be changed to run any chart. Then runs a bunch of kubectl commands against it

[restive shore]

Gotcha! Sort of like the testcontainers pattern.

[sharp zealot]

Yes exactly like that. It's actually a smaller version of what @warped canyon started doing for our main e2e test suite. I'm just doing it for 2 small and isolated suites, which made them easy to switch.

1. e2e/installers (testing our install scripts)
2. e2e/helm (testing our helm chart)

The main benefit was removing custom module code, demonstrating our theory that we can reduce the need to write custom module code to adopt dagger (ie. the dream of modulesv2)

[sharp zealot]

If we shipped dagger --x-release=VERSION which auto-downloaded & ran a CLI of the given version and handed off the command to that CLI; and --x-release=next ran the command on a dev build; would that be useful?

[restive shore]

It could be! I'd still be limited to testing use cases that don't involve (or need) ca certs. But I realize that's not a limitation for everyone.

[restive shore]

I am getting checksum failed when downloading the arm64 variant of the 0.20.7 CLI. I originally had the same problem with the amd64 version too but now that's resolved. Is this a propagation issue?

[dense canyon]

seems like. Let me prune the CDN entries just to double check Nipuna 🙏

@restive shore should be fixed 🙏

[restive shore]

checking!

looks good! Thank you!

[sharp zealot]

Am I the only one who kind of wishes I could re-generate my Dagger go bindings with go generate? 😇

[restive shore]

Hm, haven't thought about it. What's the benefit? When I use dagger, I'd rather use dagger and not require the go toolchain too.

[sharp zealot]

Well you could still do it via dagger - by virtue of Dagger providing a standard module for Go projects that exposes go generate as a dagger generate function

It's part of a more general evolution towards avoiding stack drift -> embrace native tools, while wrapping them into a higher-level standard interface. Best of both worlds

[restive shore]

I think I understand. You are looking at a pure go project using the tooling they are already familiar with. Probably not the case for a daggerverse module?

[sharp zealot]

same principle that leads to "I want to test my module with go test", "I want to lint my module with a vanilla go linter", etc. Thinking about how far we could take this principle

[restive shore]

Yes, that makes sense!

[warped canyon]

Right I think at least in the case of generated clients where the native toolchain is intended to be the entrypoint anyway it would make sense to let that native toolchain hook into the client generation

[sharp zealot]

@warped canyon OK I have a basic SDK-as-module to try 🙂

looking for volunteers

[warped canyon]

Nice! How do I try it?

[sharp zealot]

on stable: dagger toolchain install github.com/shykes/dagger-go-sdk

on workspace branch: dagger install

might get annoying with toolchain since updates don't work i think?

[warped canyon]

dagger toolchain update should work

should dagger generate effectively do the same as dagger develop with this installed in a go module?

in an emtpy repo, I ran:

dagger init, installed the sdk toolchain, and then

dagger call go-sdk init --name testsdk:
read output: no .dagger directory found

dagger call go-sdk init --name testsdk --path .
failed to stat local path: failed to stat path: failed to get requester session: session for "ks8abq7coh9vfek02jhpv6gtl" not found

[sharp zealot]

Sorry about that. I pushed a bunch of fixes, it should work now

[restive shore]

Btw, on 0.20.8, running this command also generates codegen artifacts locally and even crates a scaffolded example module.

[sharp zealot]

@restive shore I would be interested in your testing also...

1. Install go sdk, to develop modules in go

$ dagger toolchain install github.com/shykes/dagger-go-sdk

2. Initialize a new module in your workspace (happy path)

$ dagger call go-sdk init --name=my-module

3. Re-generate all modules developed with the go sdk

$ dagger generate go-sdk

Or just:

$ dagger generate

4. Manage dependencies (coming soon)

Add a dependency:

$ cd my/module ; dagger call go-sdk mod deps add --source=https://github.com/my/dependency

Remove a dependency:

$ cd my/module ; dagger call go-sdk mod deps rm --name=my-dep

List dependencies:

$ cd my/module ; dagger call go-sdk mod deps list

Update dependencies:

$ cd my/module ; dagger call go-sdk mod deps update

Bump target engine verison:

$ cd my/module ; dagger call go-sdk mod engine-version bump

[restive shore]

oh interesting! Is this what lets me use the go native tooling?

[sharp zealot]

It's a structural change in how SDKs are delivered to module devs. By itself it doesn't add or remove features, it just changes the delivery vehicle for the features. But, it becomes much easier to improve the UX from there, because they will all be exposed in a regular module rather than trapped in the CLI and engine

[restive shore]

Ah, so this is part of removing the codegen at runtime

[sharp zealot]

Orthogonal but yes very related

Also related to the workspace branch - we're refocusing the dagger commands on managing your workspace, NOT your individual module

[restive shore]

I see

[sharp zealot]

so in the workspace branch, dagger install will mean "install a module in my workspace", NOT "add a runtime dependency from my module to this other module".

Instead of moving module runtime dependency management to another command in the same CLI - and cause confusion with 2 different meanings of "install" and "dependency", we want to remove it altogether from the CLI -> and let each SDK control the experience of developing a module

[restive shore]

Cool! I think that makes sense

Manage dependencies (coming soon)
Is this something that's not merged yet?

[sharp zealot]

I'm adding it now 🙂

[restive shore]

ok cool! I'll give it a spin when you add it

[sharp zealot]

almost done😁 I think this is the future

so much easier to develop, decoupled from engine codebase!

[restive shore]

Nice!

[sharp zealot]

@restive shore ready 🙂

[restive shore]

Late night for me, about to sign out. I will test tomorrow 🙂

[quick wind]

Actually quite happy to read this. I've wanted a clearer line between "commands that are for module developers" and "commands for users to use/install modules" for a while. Centering the developer experience on modules just makes sense.

Some of the example commands might be a bit verbose but as a first pass I think this is fine

[sharp zealot]

@warped canyon should be good to try again

[warped canyon]

Awesome I'll give it a go!

[sharp zealot]

I added support for custom init templates (by default no template)

Also disabke gitignore by default, but configure as a flag to init 🙂

So technically it already has more features than the builtin commands

[warped canyon]

Oh that's great!

[sharp zealot]

Also clean commands to manage required engine version explicitly

• go-sdk mod engine required
• go-sdk mod engine require --version=0.20.6
• go-sdk mod engine require-latest
• go-sdk mod engine require-current

[warped canyon]

I must be doing something wrong with my testing flow. I was expecting go-sdk init to actually behave more like develop --sdk go since I must already be in an initialized module with the toolchain installed. Unless I should use -m for init?

[sharp zealot]

what's your command?

Remember it's designed to work in a workspace, so you have to pretend that top-level dagger.json isn't there

[warped canyon]

dagger init, dagger toolchain install github.com/shykes/dagger-go-sdk, then
dagger call go-sdk init --name testsdk or dagger call go-sdk init --name testsdk --path .

got it, i'll just test it with the workspace branch then instead of toolchains

[sharp zealot]

it should work with both, just that by default it creates modules under .dagger/modules/

[warped canyon]

if I dont specify path, I get

✘ withExec module-source generate --local .dagger/modules/testsdk . /before /after (experimentalPrivilegedNesting: true) 0.2s ERROR
failed to mount directory: empty result reference [traceparent:4b2cac3d0863e4400a50cf6f77f489c9-07dfd28c2848b5b6]

and with path=. it fails because there's already a dagger.json there, which makes sense if its expecting a Workspace instead

[sharp zealot]

1. damn I thought I fixed that... on it

2. yeah that's expected.. there's already a module at . in your case

[sharp zealot]

@warped canyon I can't repro the first issue... Can you give me an exact repro? Maybe dagger version is also a factor

[warped canyon]

• dagger init in empty git repo (with v0.20.8)
• dagger toolchain install github.com/shykes/dagger-go-sdk
• dagger call go-sdk init --name testsdk --path ./foo
• failed to mount directory: empty result reference

[sharp zealot]

@thorn moat any thoughts on the code? https://github.com/shykes/dagger-go-sdk/blob/main/go-sdk.dang

Large prod-ready dang modules get pretty unruly, I kind of give up on reading them. Have to trust the LLM, which sometimes makes things worse 😅

The mandatory if/else chains make things pretty verbose, not sure if there's an alternative there

Maybe there are patterns & tricks I don't know about

Also it is very quick to say "F this" and sneak in an inline python or go helper... I caught it doing that several times without asking

[sharp zealot]

Can you try again @warped canyon ?

[thorn moat]

nothing glaring - looks like to some extent there's just a lot of conditional logic involved, but aside from that:

• some strings might be more readable as triple-quoted literals to avoid all the escaping, e.g. search patterns, but they don't support interpolation so not a total win
• the let config = ... could just be one big toJSON({{ ...}}) instead of string-encoding and concat'ing toJSONed values (LLM didn't know about object literals?)
• i see a if (foo) { raise } else if (bar) { raise } else { ... } that could un-nest the final else (i.e. raise early)
• needs a dang fmt
• maybe supporting early returns would help? already have that for raise, just not happy exits

[sharp zealot]

yes early returns would definitely help (assuming they don't cause problems elsewhere)

Also @thorn moat a random question popped in my head: since graphql-native is such a focus, why the mandatory pub - simply allowing some graphql fields to have implementation brackets { ... } would be more natural and idiomatic no? (basically an implicit pub)

type Foo {

   foo: String!

   bar: Boolean!

   baz(name: String!): String! {
      "hello, " + name + "!"
   }
}

Or maybe there's a parsing issue that makes pub necessary.

Anyway it's just a thought that popped in my head as I was drafting graphql schemas and dang implementations, it tripped me up a few times (I would forget the pub)

[thorn moat]

default answer is that opens up to a world of parsing ambiguities - leading keywords help a TON. i can give it a try sometime, i agree the end state would be nice and have had the same temptation before. The keywords are also important for distinguishing local declarations vs. reassigning values in an outer scope (but maybe we could add :=)

[warped canyon]

looks like it was able to generate the changeset this time, but the changeset apply fails with internal error: Directory.diff received different relative paths: "/before" != "/after"

[restive shore]

does the //+cache pragma (function caching) work for private functions?

[sharp zealot]

Depends what you mean by private

[restive shore]

In a module, there may be functions that are used internally that aren't public. Some of them may be valid function caching targets. For example, if I have a function that retrieves an oauth token, I may want to cache the function that actually does the work but not the public function (which may do other things besides the token fetch).

I am in the process of converting some things I do with images into language native constructs. For example, instead of pulling a container image with curl in it, use the go http client.

[sharp zealot]

As long as they are invoked via dagger engine (as opposed to private to the native language runtime, eg. lower-case functions in Go) then yes I believe caching applies equally

[sharp zealot]

there's also dag.Http() that will get you caching, tracing and lockfile support for immutable downloads

[restive shore]

dag.Http() is pretty limited in comparison to Go std lib client. For example I can't do a POST with it. If it is enhanced in the future, I could switch.

It also only supports binary data (Files) I think. Not text

[sharp zealot]

Makes sense. It's only worth considering dag.Http() for downloading artifacts anyway.

[restive shore]

And I am using it for that purpose 🙂

[sharp zealot]

(if you want that HTTP resource to be materialized as part of your supply chain)

Oh I see. Then yeah we should definitely fill the gaps so you can use it

You need a POST to download an artifact? 🤔 Anyway fair enough, we can make the http method configurable

I'm guessing this is all much easier now post-buildkit

[restive shore]

No but some token endpoints use POST to "hide" data

[sharp zealot]

@thorn moat heads up issue incoming in the dang repo

[restive shore]

just to close on this. This means, only functions that are public (upper-case in Go) will work with function caching.

[sharp zealot]

More generally: only actual dagger functions, that are actually called by the dagger engine, can be cached

(less ambiguous that way)

[restive shore]

Understood!

[sharp zealot]

https://github.com/vito/dang/issues/47 🙏

For the longest time I though the LLM was bullshitting me, as an excuse to do its stupid workaround hacks.

But it appears that there really is a dang problem. I execute a helper that takes a WorkspaceID + path, and spits out a ModuleSourceID. But the module can't use the returned ID for some reason 🤷‍♂️

The helper itself is a workaround for another (non-dang) bug, which is that modules cannot safely load certain types of ModuleSources...

[sharp zealot]

Dang rejects ModuleSourceID in loadModul...

[sharp zealot]

@thorn moat another dang question... I'm trying to convert core JSON (scalar json bytes) to String! (to write it to a file...). Trivial to cast in Go, I can't get it to work in Dang though..

[thorn moat]

x :: String! didn't work?

ah guess not, huh could have sworn that was a thing. oh maybe it's only with literals

dang> "{}" :: JSON!
=> {}
dang> "{}" :: JSON! :: String!
parse error: Error: syntax error: unexpected ':: JSON!'
  --> repl:1:6
     |
   1 | "{}" :: JSON! :: String!
            ^^^^^^^^
     |

dang> ("{}" :: JSON!) :: String!
type error: type hint mismatch: expression has type JSON!, but hint expects String!

there's toString(x)

[sharp zealot]

I think I tried both...

[thorn moat]

toString works for me

dang> toString("{}" :: JSON!)
=> {}
dang> :type toString("{}" :: JSON!)
Expression: toString("{}" :: JSON!)
Type: String!
dang> toJSON(toString("{}" :: JSON!))
=> "{}"

(last call just to clear up repl induced formatting ambiguity)

[sharp zealot]

Mmm let me try again

[sharp zealot]

It worked 🙂

@warped canyon fixing your bug report pulled me into a cleanup, the go-sdk implementation was a mess. We hit several niche core API bugs, LLM went crazy with the workarounds, and workarounds for the workarounds...

[thorn moat]

@sharp zealot is there a way to control the image used by dagger/go? in dang for example I need an image with tree-sitter installed for go generate ./...

[sharp zealot]

That's next on my todolist... I also need it, to switch our go generate in dagger/dagger (need protoc and git to generate the engine shims for example)

Would be cool if you could control the container image globally, but also from a custom directive in the Go file 😇

[thorn moat]

as long as there's some way for me to provide it in code and not just an image ref 🙏 - maybe a directive could point to a function to call?