using the test containers pattern

[eager robin]

Starting a thread

cc <@&946480760016207902>

@tough hound you can repro with:

dagger call -i -m github.com/dagger/dagger@pull/11746/head \
 go env \
 with-workdir --path=core/integration \
 with-exec --args=go,test

[tough hound]

Thx!

I'll work on that tomorrow, I'm still on vitest for now but I think I found the light to make it work properly 🤣

[eager robin]

OK. Do you mind taking a quick look now, just to share your first impressions? If you recognize anything

[viral anchor]

If you have any ideas/pointers I can try to make progress today 🙂

[tough hound]

Okay I'm checking

[eager robin]

Look at this one for example: https://github.com/dagger/dagger/pull/11746/changes#diff-5c82460862e1b9131694d8a1c5a6113e0ee1a2707060a1e8ebc2f7e8de1e27a1R150-R153

func EngineDev(opts ...dagger.EngineDevOpts) *dagger.EngineDev {
    client := initClient()
    return client.EngineDev(subnetNumber, opts...)
}

it references subnetNumber but that's not declared anywhere. So it looks like a bug in the generator maybe?

[tough hound]

Or in the schema

[eager robin]

But the schema loads fine otherwise

And I didn't make any change to the part of the schema that references subnetNumber

dagger call -i -m github.com/dagger/dagger/pull/11746/head \
 engine-dev --help

[tough hound]

Yeah looks like there's something wrong with the generation, arguments that are scalar are set even if they should be optional

[eager robin]

OK thanks! We'll look in that direction

[tough hound]

I'm verifying something

Because the codegen for client isn't doing anything special, it's the regular codegen for this part

[eager robin]

Maybe confused by toolchains? I noticed in the generated bindings, everything is generated twice.

dag.EngineDev() -> direc to to toolchain dep
dag.DaggerDev().EngineDev() -> via main module name (names are confusingly similar, but they are 2 different modules: . -> toolchains/engine-dev)

I wonder if that's also why generation is so slow (2mn+ on a warm cache)

[tough hound]

Okay interesting, so I cannot repro the bug if I try to install the toolchain in the module because we don't generate a global client in module code.

So I would look at:
1 - The schema received by the client generator in case of toolchain
2 - The codegen for the global client (the one in dag/dagger.gen.go), I think there's also an issue there that we didn't noticed before

I'll continue on vitest now, but please update that thread if you find anything, I can also do some work tomorrow on it, it's a nice catch

[eager robin]

Okay interesting, so I cannot repro the bug if I try to install the toolchain in the module because we don't generate a global client in module code.

Did not understand that

Ah "global client" as in dag

What do you mean by "install the toolchain in the module"? It's already installed

And obviously the bindings don't fail to load (or github.com/dagger/dagger would never load)

[tough hound]

I created a new module in a directory foo and installed the Php SDK as a toolchain (that is als generated wrong in the client)
I wanted to see if we could see the same issue in a module, but we don't because we already generate a regular client

[tough hound]

yes

[viral anchor]

@eager robin whats the exact command to regenerate that internal/testutil/dagger/dag/dag.gen.go? so I can poke around

[eager robin]

dagger client update

[eager robin]

You'll need to immediately wipe changes to go.sum, go.mod and sdk/go/dagger.gen.go because of the other bug

[viral anchor]

Got through the type errors and on to new ones now 😛

I'll keep going until I've got through them all

[viral anchor]

Here's the fix for the codegen

index 1db49d20b..6263b6899 100644
--- a/cmd/codegen/generator/go/templates/src/dag/dag.gen.go.tmpl
+++ b/cmd/codegen/generator/go/templates/src/dag/dag.gen.go.tmpl
@@ -82,11 +82,11 @@ func Close() error {
                ctx,
                {{- end -}}
                {{- range $arg := $field.Args -}}
-               {{- if not $arg.TypeRef.IsOptional -}}
+               {{- if not (IsArgOptional $arg) -}}
                {{ $arg.Name }},
                {{- end -}}
                {{- end -}}
-               {{- if $field.Args.HasOptionals -}}
+               {{- if HasOptionals $field.Args -}}
                opts...,
                {{- end -}}
        )

It needs a dev build to do the client update after that. This part is the easy part

after that, you have to update all references to core types in core/integration/*_test.go to use the generated client instead of dagger.io/dagger. Some of these are tangled with helpers used outside of core/integration, so those have to be duplicated.

This is where we need the "dependencies only" generated client to be a thing so that we can just use the published client everywhere else. Because its going to be painful

[tough hound]

This is where we need the "dependencies only" generated client to be a thing so that we can just use the published client everywhere else. Because its going to be painful
That's on my list once I'm done with vitest / typescript lib split 😄

[eager robin]

I started patching core/integration/test_main.go is that what you mean?

[eager robin]

what's the deps-only client? never mind i got it.

What's an example of entanglement in our test code? I was going to search/replace dagger.io/dagger to internal/testutil/dagger everywhere in the test code

[viral anchor]

like any utils we might be sharing from tests outside of integration that accept or return core types

it also seems like there's parts of dagger.io/dagger that aren't included in internal/testutil/dagger such as WithLogOutput and WithWorkdir

[eager robin]

Oh yeah, sounds super fun

[eager robin]

That would be a generator bug right?

[viral anchor]

probably yeah. I think it also doesn't expose the Client in any way, which we need. All things we can change

Its also looking like dependencies are not being served, which means dag.EngineDev() is not available. So pretty hard stuck on that. Working on a fix. Could be related to local changes, not certain yet.

[eager robin]

Mmm I'm getting dag.EngineDev()

Are you trying this on my branch?

[eager robin]

👆

[viral anchor]

yeah but I have a bunch of changes locally for compatibility with dagger.io/dagger

[eager robin]

But if you enter that env you can see that internal/testutil/dagger/dag/dag.gen.go does define func EngineDev(...) on line 150

Oh you mean the bindings are generated, but they're getting runtime errors because the engine isn't serving the deps?

[viral anchor]

right the function is generated but during execution it ran into errors with trying to find up go.mods

[eager robin]

more missing include in dagger.json?

[viral anchor]

https://dagger.cloud/kpenfound/traces/cdd2f78026eb87087da6b0bc6a4005ce?span=1350e8408cc407d8

Failed to serve module: failed to resolve dep to source: include/exclude path "../../go.mod" escapes context

[eager robin]

I had to add an include of engine/distconsts to fix a build error at client generation. Maybe there are more mising includes that block at serve time?

[viral anchor]

oh and the updated command i'm using to test is dagger-dev call go env with-workdir --path=. with-exec --args=go,test,./core/integration --experimental-privileged-nesting stdout

[eager robin]

I kind of want a "open interactive shell" button in that web window now 😛

[viral anchor]

I have claude churning on it while I work on this toolchain panic from #general

[eager robin]

Should I open a PR with that?

[viral anchor]

yeah i was going to add it to your branch but it makes sense to get it to main asap. It'll fix all the type issues for optional args

something I'm not clear on ( @tough hound can enlighten me tomorrow) is why the generated client is so different from a module's generated client. Or in other words, why we can't use those same existing utilities to generate a client

[eager robin]

One different I think is bootstrap: the generated client needs to take care of serving its own dependencies when it connects. Whereas inside a dagger function runtime, the dependencies are already served for you by the engine - the client only needs to connect (I think)

[eager robin]

https://github.com/dagger/dagger/pull/11752

[eager robin]

Fix merged

There are still 2 unresolved go clientgen issues that I know of:

1. Overwrite replaced core lib: https://github.com/dagger/dagger/issues/11751
2. Nuke go.mod and go.sum (no issue yet)

[viral anchor]

looks like the main issue is because the includes are outside of the module's source path, and the findUp logic in loadModuleSource can either go up to the module's own dagger.json, or a .git. But we've excluded the .git, so it isn't able to find the includes

"include": [
    "../../engine/distconsts/**/*",
    "../../sdk/typescript/runtime/**/*",
    "../../go.mod",
    "../../go.sum",
    "../../util/parallel"
  ],

so we can either include the .git in the root dagger.json, which doesn't seem good, or make the module loading smarter

[eager robin]

ah, I think I understand. A long time ago we debated whether the SDK should be responsible for careful coexistence with the host codebase. We concluded that no, it was too late to change the design for that, and we were already committed to a design where the SDK gets exclusive jurisdiction over an area of the repo (the module source dir).

Now with generated clients, we no longer have this luxury: careful coexistence is a must. But our SDKs are not equipped for it.

I think the correct answer is probably to not touch the go.mod and go.sum, since they are not exclusive to the dagger module. It's up to the go developer to call it as part of their normal workflow

also, we should switch to returning a Changeset in that interface

ooooh could this become a toolchain hooked into dagger generate?

[viral anchor]

yeah 100% agree it needs to be a Changeset. dagger generate makes sense! Even with the current CLI I'd prefer it to get generated from dagger develop instead of a special command

the generated package should just have its own go.mod/sum rather than fighting with the project?

[eager robin]

oh yeah that's an option

[viral anchor]

That's what we eventually recommended pre-zenith, wasn't it? The dagger code should be its own go module?

[eager robin]

yes - by default, although we support both.

However, that's for a different kind of go code: a user's implementation of dagger functions + associated bindings, mixed together.

In this case, it's a purely generated binding, in a self-contained directory not mixed with anything else. If we make that directory its own go.mod, I guess it does keep "shared custody" of files to a minimum

[tough hound]

The nuke of the go.mod comes from the go mod tidy executed at the end of the generation
In the case of a generated client, it will override the go.mod because we don't get the source files so that's why the go.mod is broken.

I think I should not run go mod tidy in case of a generated client, should it be the user responsability to run go mod tidy?

Or the other way is to do what you say and create a specific go mod for these generated bindings, but then the user has to maintain it, how would it work?

[eager robin]

@tough hound option 1 👆😁

[eager robin]

@tough hound option 2 👆

I think we should start with option 1 - do not run tidy. Then see if it's enough

[viral anchor]

makes sense to me! the other change I'm trying is to move the embedded sdk under the generated client so we dont have to worry about overwriting sdk/go/dagger.gen.go

[eager robin]

it depends if we need to mess with the go.mod for eg. otel versions? @lucid solar ? Is
it feasible to leave go.mod untouched when dropping a generated client in a go project?

would be less intrusive but maybe we need control over which specific version of our own dependencies we need?

[viral anchor]

option 3 is to run go mod tidy but with the context of the project included

[eager robin]

I don't think we have that capability. That requires toolchains v2 with dynamic filtering i think

[viral anchor]

hm ok. I'm confused about that but I believe you 🤣

[lucid solar]

frustratingly the otel log sdk is still pre-1.0 which is the root of most issues, but maybe it's stable enough anyway to go without the explicit pinning (it definitely wasn't at the time)

[eager robin]

Maybe I'm getting it wrong. It seems to me that would need the end user to manually add every part of their go project to an include in their dagger.json, which is a pretty rough user experience - and normally reserved for module development, not your actual codebase. If we add dynamic filtering (the ability for a dagger function to dynamically filter pre-upload) then we could make that automated.

Am I barking up the wrong tree here?

[viral anchor]

oh I see, but does client update really need to be bound by the dagger.json includes?

[tough hound]

Hmm no we could get the project

If we update this: https://github.com/dagger/dagger/blob/d285f0fd6b90cdc3e9025409e589418674e1f6c9/core/sdk/go_sdk.go#L63-L65

We can get every go file of the project (assuming the dagger.json is at the same location of the go.mod)

[eager robin]

Even files referenced in replace directives, go:embed, go:generate pragmas?

[tough hound]

It depends what we put in the patterns

[eager robin]

I don't think that can be solved with patterns... for replace directives you have to parse the go.mod dynamically. For go:embed you have to parse all the go files. For go:generate I don't think there's any way to access any files required by those commands - it will have to be manually added by the end user.

IMO much easier to avoid doing it at all, if possible

@viral anchor @tough hound can we talk live before you sign out Tom?

To make sure we know the task list

[tough hound]

I'm free in 15 minutes

[eager robin]

Ok!

[viral anchor]

I'm going to work on a PR that gives the generated client its own go.mod, and then client install will also install+replace it

[eager robin]

Nice 👍

I'm trying an end-to-end run with the latest fixes applied. See what the next blocker is

[viral anchor]

What did we agree on to stop overwriting the library (sdk/go/dagger.gen.go) in the dagger repo? Bundling the library with the client?

[eager robin]

• Assume the genclient bundles its own go.mod
• When running in stable engine: make sure the bundled go.mod points to the right version of the go lib
• When running in dev engine: vendor the engine's bundled go lib, and replace it in the bundled go.mod
• Stop messing with the parent go.mod altogether (including no more checking for any replace of the go lib, it's an app issue)

[viral anchor]

When running in dev engine

and this is a change to our integration test setup?

[eager robin]

No I don't think so, but forgot to confirm with @tough hound . I think it's a flag that the engine passes to the SDK's generator function. So we don't have to worry about it basically

[tough hound]

It’s not a flag, the Go SDK (module inside the engine, not the lib) just do an API call to get the caller’s engine version and check if it’s a dev version or not using a function

We already do it when generating module’s binding

[eager robin]

@viral anchor back from demo to Jeremy 🙂

FYI I tried an end-to-end run, with engine build from main, and still getting the original error from this thread...

either that, or a very similar one

https://dagger.cloud/dagger/traces/96bb1fdce5d1bb83599cf8d259e4d48c?listen=93235b55023ed137&span=e793afc8c263d368#93235b55023ed137:L1

Oh, I probably need to run dagger develop...

Trying that

Here we go: https://dagger.cloud/dagger/traces/92b7d0e8a4b3c9a805ca75207fb38ca4

1. Build a playground from main, enter it
2. Checkout PR 11746 (new test harness)
3. Run 'dagger develop' -> re-generates client with fixes
4. Build go env, enter it
5. run go mod tidy
6. Run go test

[viral anchor]

Nice! thats with all of the test imports updated to point to the generated client?

(nice that its at least running tests now 🙃 )

[eager robin]

I'm stuck at a new error...

@hybrid cosmos 👋 (we were just discussing it)

/app/core/integration $ go test -v
# github.com/docker/docker/pkg/archive
/go/pkg/mod/github.com/docker/docker@v28.5.2+incompatible/pkg/archive/archive_deprecated.go:103:47: undefined: archive.Compression
/go/pkg/mod/github.com/docker/docker@v28.5.2+incompatible/pkg/archive/archive_deprecated.go:159:43: undefined: archive.Compression
FAIL    github.com/dagger/dagger/core/integration [build failed]

I don't think it runs tests yet @viral anchor

[viral anchor]

curious that this is passing with the old tests

[eager robin]

Seems to fail while building the test binary

[viral anchor]

ah yeah you're right

[eager robin]

it's gotta be because of new dependencies added by the gen client

Maybe it goes away as soon as we drop a go.mod in gen client?

[viral anchor]

is archive.Compression only referenced by the dependency itself? nowhere in our client?

[eager robin]

no idea

[viral anchor]

I didn't run into that on my checkout from yesterday but i don't know if its because I just didn't get that far lol. Running it again now to see

I almost have a branch ready with the go.mod in the client, just fixing existing tests

[hybrid cosmos]

managed to repro

[hybrid cosmos]

sorry, i don't know the root cause, but I can fix the symptom. We shouldn't be using pkg/archive and pkg/chrootarchive, as they've been removed from docker/docker and moved to moby/go-archive. I'm senidng a pr to fix it

[eager robin]

Nice thanks

[hybrid cosmos]

i'm also fixing the copyFile issues

[hybrid cosmos]

I think the root cause is something updating go.mod dependencies higher than it should.

[hybrid cosmos]

Sooo... I'm pretty sure Kyle will have the root cause fix, but in case it's not or you need a stopgap now, take the commit from https://github.com/dagger/dagger/pull/11759 @eager robin

[eager robin]

@viral anchor @tough hound good morning, sorry Tom we didn't sent a task list yesterday.

[hybrid cosmos]

Happy to sync with any of you if i can pick up something

[tough hound]

No problem!

I started my prototype on generated client v2.
I’ll keep going and do a demo when I have something cool to show 🙂

[eager robin]

Nice 🙂 Remind me what's the scope of that?

[tough hound]

• Generate client with its own go.mod
• Don’t generate core bindings but only dependencies and depend on either the published library Or a bundle version local to the generated client go.mod
• Generate dependencies in their own go pkg or at least independant files

• update the project go.mod with a replace to use the generated client pkg

I’m focused on 1-2 and 4 for now
The 3 is mostly a refactor to make things cleaner but not the main target for a 1st implementation

[eager robin]

Generate client with its own go.mod

@tough hound I believe @viral anchor has a PR for that?

Just want to make sure you guys are in sync on who does what

Anyway sounds great, that's exactly what we need. If some parts are ready before others, please ship early and often 🙏 we are stuck on this for dogfooding

Generate dependencies in their own go pkg or at least independant files

Definitely spin out this part in a separate PR, it will take longer - right?

I’m focused on 1-2 and 4 for now

Ok I see -> OK! Good luck

cc @viral anchor in case it overlaps

[viral anchor]

yeah I have it working but it broke basically all the original tests so I'm still working through those

[viral anchor]

@hybrid cosmos here's the command i'm using dagger call go env with-workdir --path=. with-exec --args=go,test,./core/integration --experimental-privileged-nesting stdout

[hybrid cosmos]

internal/testutil/dagger/dag/dag.gen.go:152:26: undefined: subnetNumber
internal/testutil/dagger/dag/dag.gen.go:152:40: too many arguments in call to client.EngineDev
    have (unknown type, []"github.com/dagger/dagger/internal/testutil/dagger".EngineDevOpts...)
    want (..."github.com/dagger/dagger/internal/testutil/dagger".EngineDevOpts)
internal/testutil/dagger/dag/dag.gen.go:721:23: undefined: sourcePath
internal/testutil/dagger/dag/dag.gen.go:721:35: undefined: doctumConfigPath
internal/testutil/dagger/dag/dag.gen.go:721:35: too many arguments in call to client.PhpSDK
    have (unknown type, unknown type, []"github.com/dagger/dagger/internal/testutil/dagger".PhpSDKOpts...)
    want (..."github.com/dagger/dagger/internal/testutil/dagger".PhpSDKOpts)
internal/testutil/dagger/dag/dag.gen.go:726:26: undefined: sourcePath
internal/testutil/dagger/dag/dag.gen.go:726:38: too many arguments in call to client.PythonSDK
    have (unknown type, []"github.com/dagger/dagger/internal/testutil/dagger".PythonSDKOpts...)
    want (..."github.com/dagger/dagger/internal/testutil/dagger".PythonSDKOpts)
internal/testutil/dagger/dag/dag.gen.go:736:24: undefined: sourcePath
internal/testutil/dagger/dag/dag.gen.go:736:36: too many arguments in call to client.RustSDK
    have (unknown type, []"github.com/dagger/dagger/internal/testutil/dagger".RustSDKOpts...)
    want (..."github.com/dagger/dagger/internal/testutil/dagger".RustSDKOpts)

i'm gonna fix these now ^

[viral anchor]

Making progress on my end finally. I don't think we're actually overlapping since mine will be a small patch we can apply to the current client generator and v2 sounds like a longer term change

[viral anchor]

Success! @eager robin @tough hound https://github.com/dagger/dagger/pull/11768

@hybrid cosmos for context, once this is in, you wont have to worry about the main go.mod, go.sum, or sdk/go/dagger.gen.go getting overwritten

[viral anchor]

thanks for the review @tough hound , just 3 questions https://github.com/dagger/dagger/pull/11768

[tough hound]

Thx for the ping! I'll reply 😄

[viral anchor]

Ready for another look when the sun is up on your side of the world! https://github.com/dagger/dagger/pull/11768

[viral anchor]

Still pushing on getting the tests working with the go toolchain. I've made pretty good progress and if I can get any full suites to pass I'll push up my branch 🙂 its based on your PR 11746 @eager robin

[eager robin]

Nice thanks for pushing! A super important piece of the puzzle

[viral anchor]

Probably makes sense to keep it on the side in favor of the Workspace API once we have that underway, but I'll keep on this for now

[eager robin]

You mean in terms of priorities?

[viral anchor]

Yeah exactly

[eager robin]

I guess it depends how much work left to make the switch to testcontainers-ish

[viral anchor]

True. Also if I'm not needed to prototype phase 1 or 2 then I can keep on this too

I think its getting close though. I'm now in the whack-a-mole phase for individual suite's dependencies

[eager robin]

oh! not just global setup for everything?

[viral anchor]

some parts are global like the dev engine and cli build, but the only certain suites need registries and stuff

we could just start with everything global and then refactor but i dont think itll be too bad to just do it right the first time

[eager robin]

Also highlights the strengths of the model

[viral anchor]

exactly

[viral anchor]

making a note here so I dont forget: dagger.WithWorkdir() isn't available in the generated client

example of where we're doing this: c := connect(ctx, t, dagger.WithWorkdir(wd))

[eager robin]

https://github.com/dagger/dagger/issues/11798

https://giphy.com/gifs/jira-ticket-maragifstwo-zp1a2ZOXJLCfSdtrPK

[viral anchor]

@eager robin can we rename this thread to something like 'embedded dagger for integration tests' or something? or testcontainers mode 🤷‍♂️ i've been avoiding using the term testcontainers

[eager robin]

I think it's OK to call it the "test containers pattern"

using the test containers pattern

[viral anchor]

Bit of a status update / current thing I'm stuck on:
I have a handful of suites in core/integration passing in the go toolchain. It relies on a generated client and starting a dev engine in suite_test.go

I have to run the tests with experimental-privileged-nesting so that the tests are able to spin up service containers. Its looking like this has the unintended side-effect that client.Host gets passed through to the outer host. For our tests in particular that ends up breaking a bunch of suites. Maybe I need to carefully manage which clients get the session variables and which dont? cc @lucid solar @daring sonnet

[eager robin]

My knee-jerk reaction is that it's a bug in experimental-privileged-nesting. The goal of nesting is that every client running inside dagger gets attached to the same session it's running on. But it's not to give it elevated privileges within that session

[viral anchor]

btw the Host.Tunnel thing we talked about yesterday isn't working but I think its connected to this same thing

[eager robin]

Even though there's "privileged" in the flag name, that was just a "keep out" sign to make it extra scary until we made it safe to enable by default (which until 30 seconds ago, I thought it was)

[viral anchor]

Right we talked about enabling it by default and if we do that we obviously dont want this behavior with it

I'm trying to think if there's a reason it currently works that way. If that behavior is needed we could move it to an even scarier flag I guess

[viral anchor]

side note, following your lead @eager robin now that I have a few suites working I had claude create a skill based on what it took to migrate suites and debugging, important context, etc, so hopefully that pays off as I continue 🙂

[eager robin]

Ha ha nice. To be clear I was following @daring sonnet 's lead 😛

[viral anchor]

I'm opening a PR for this and we can discuss the unintended consequences there 😂

[daring sonnet]

Yeah I'd have to see exactly what's going on to say for sure. If you run the tests today via ⁨go test⁩ on the host, then you have to wrap in ⁨./hack/with-dev⁩ to hit the dev engine. I guess what might be idea is if the tests spun up a dev engine and then connected to that, same as any test that needs to depend on a service. It's just a little trickier and more mind-melting since it's dagger testing itself...

[viral anchor]

right I think in this case since we're using a generated client based on the dependencies in the dagger.json, we really want the dagger client in the test code to have a proper session. So if we're dagger-in-dagger (as in running go test using the go toolchain), it needs privileged nesting. But we dont want the outer client's filesync

[viral anchor]

I guess before I push this up, is there a case you're aware of where we rely on privileged-nesting to forward the original client's filesync? Basically should I split that out to a separate flag or disable that behavior? (or neither but please dont say this one)

[daring sonnet]

Filesync of nested clients should point to the nested client, not the "original" host. I'm pretty sure it must have that behavior or else very few of our tests would work. But it's possible there's some situation where it breaks?

Basically I have no idea what's happening, so I'd just need to check

[eager robin]

Yeah I'd be very surprised if anyone expected it to work any other way

[viral anchor]

Trying to get a repro to demonstrate and its not doing what I expect, so I'll get back to you on that

[viral anchor]

ok I think I have it working without any changes, so that was probably just a red herring. Moving on to more test suites. Also means my attempts with Host.Tunnel were unrelated but I'll come back to that after I have more/all of the tests working with dagger-in-dagger because the service setup stuff is pretty centralized at the moment so it'll be easy to tweak later

[eager robin]

Love a good false alarm 🙂

[eager robin]

@viral anchor I'm giving a try at fixing integration tests on Mac. Would be a cool way to leverage your hard work 🙂 Run integration tests straight from mac, boom

[viral anchor]

You might be interested in my last commit to main 😄

[eager robin]

Oh no

Did you already do it 😅

nice

[viral anchor]

Just the one thing that's been broken on mac, no integration tests yet until the tunnelling stuff is ready

[eager robin]

mmm, I'm seeing other errors though? Is there something wrong with my setup? I can't build the integration test binary on mac even with your fix

[viral anchor]

Maybe not sure, I was able to after that change 🤷‍♂️

The dangers of running things outside of dagger 😂

[eager robin]

https://giphy.com/gifs/BookCameo-quarantine-bubble-inflatable-QA6Y9EsMxwBfiTgJuO

We're not used to the dangers of the outside world anymore

[viral anchor]

Works on my machine!

[viral anchor]

We're over 70% of suites passing now. Most issues encountered so far are either difference between generated client and sdk, side effects of privileged nesting, or incorrect relative paths

One annoying thing is that my test command looks like this: _EXPERIMENTAL_DAGGER_RUNNER_HOST=docker-container://dagger-engine.dev ./bin/dagger call go env with-workdir --path=. with-exec --args=go,test,./core/integration,-run,'^TestSecret$',-v --experimental-privileged-nesting stdout, and claude messes this up every time it compresses, even though it has it noted in a skill and in a markdown file at the root

[eager robin]

I bundle shell scripts in my skills - then the skill says "use the script"

[viral anchor]

Context left until auto-compact: 1%

Here we go again

[daring sonnet]

<unsolicited ad>
fwiw I don't fear context compaction on codex 5.3 anymore (i very much did on 5.2). I don't know what they did but it remembers important details from like 10 compactions ago now
</unsolicited ad>

[viral anchor]

I should switch to Opus 4.6 to see if its similarly improved

[viral anchor]

Update: we now have all suites passing and updated the test-split module to use the go toolchain instead of engine-dev 🎉

Now I'm going through checks to make sure we're still passing. Some suites were hitting what claude thought were timeouts when run as a whole suite vs filtering parts of the suite and I suspect these are engine panics and not timeouts

Main TODOs

• Review changed test code to make sure its correct
• Split out engine changes into their own PR because right now the tests have to be run with a build from their own branch. The only change here is to wrap nested graphql exec errors with ExecError. Our tests rely on this formatting and its just a bug in how nested execs pass errors
• go lint. There's probably a bunch of linting issues claude can tackle while I eat dinner
• tunnelling. Current state relies on dagger-in-dagger by using Service.Start, but now I can revisit Host.Tunnel

[eager robin]

Amazing. This is a super valuable track all on its own

[viral anchor]

Definitely learned a lot about the rough edges in our UX and pairing with claude on a large change. Probably 1 in 10 context compressions from claude it was like "But wait! If I just move these services to the toolchain, we dont need the generated client at all!" 🙃

[eager robin]

"Claude we talked about this"

https://giphy.com/gifs/groundhogday-groundhog-day-movie-S9crjCfQXC78ST61iv

[viral anchor]

I had it keep track of its own progress tracker markdown which leads with the general goal and how to run tests. It also created its own skill that mainly focuses on how to run tests. But still sometimes that compression it just goes dumb and I have to stop it and do that 👆

it was interesting to see when opus 4.6 would choose to drop itself back to planning mode

[viral anchor]

@eager robin want me to push to your PR or open a new one? Doing a final review locally and then ready to push. Follow up PRs will be

• the Host.Tunnel part so that you could run the integration tests with just go test. Since the existing tests require dagger-in-dagger I think it'll be fine to do this in a follow up rather than adding more on this already huge PR
• moving the registry services just to the suites that need them instead of the test wrapper so that we dont spin them up when we dont need them. Pretty small optimization

[daring sonnet]

I've also accumulated quite a bit of that in ./skills/cache-expert/references/debugging.md, just FYI. Most of it is not even cache specific anymore in case you want to copy it out to a separate thing.

[viral anchor]

nice, it sounds like we can move a lot into a "developing dagger" skill

[eager robin]

Your call. If it's my PR, basically you can take it over completely

[viral anchor]

Just pushed up everything! 🚀 https://github.com/dagger/dagger/pull/11746
Now to make it green 🟢

[viral anchor]

Status update: still a bunch of failing checks. Working through them one at a time. I thought I was fully passing before because of the --scale-out bug 😭 Some of the failures are from the toolchains-in-dependency bug https://discord.com/channels/707636530424053791/1470450881903198259 https://github.com/dagger/dagger/pull/11753. For now I have my fix in that PR locally so I can move forward and hopefully Workspaces will sidestep the issue.

The performance of some of the checks seems worse than before even though we're effectively doing the same thing and I've verified the cache volumes are put in the right places. I'm keeping an eye on this

The dev engine logs are currently at the top level which sometimes makes the trace page unusable (engine logs can be 256mb+). I need to wrap this in a span to drop it down a level I guess.