Synthetic workspace support

[lunar summit]

I checked the VM and there isn’t a repo checked out here yet, just the Discord thread metadata.

I can implement dag.Workspace(myDir), but I need the project checkout or a repo/branch to work from. Once it’s available, I’ll wire it into the existing workspace construction path and add focused tests for synthetic/local-directory workspaces.

💻

Synthetic workspace support

Synthetic workspace support
PR #13217 targets workspace at 1a756c600, milestone 1.0.0-beta.1. Xhigh review fixes landed: experimental Query.workspace(directory:), explicit unsupported errors for synthetic config/module/env/install/check/generator/service APIs, scoped gitignore behavior, generated SDK/docs incl PHP refs. Full remote CI passes; only changelog check is skipped. No open main-targeting synthetic PR was found; unrelated #13212 left open.

[near halo]

checkout dagger/dagger

[lunar summit]

Checking out dagger/dagger now. I’ll inspect the workspace APIs after the clone finishes.

💻 📝

[near halo]

stuck?

[lunar summit]

Not stuck. I finished a first pass in /home/exedev/dagger on branch codex/synthetic-workspace.

Implemented Query.workspace(directory) plus generated Go SDK support for dag.Workspace(dir), and fixed Workspace.findUp for synthetic/rootfs-backed workspaces. go test ./core/schema passes, sdk/go compile smoke passes, and the Go SDK generator completed successfully.

The only snag is local verification of the new integration test: this VM’s available CLI/engine is still using old embedded Go SDK content, so the test module generated inside the test does not know dag.Workspace yet. The source changes are in the worktree.

💻

[near halo]

how disruptive is the change for existing workspace plumbing - contextual injection, cache invalidation, distinguishing between git and local source etc

[lunar summit]

Short version: low disruption for existing workspace plumbing as long as synthetic workspaces stay an explicit dag.Workspace(dir) constructor, not part of current-workspace detection/loading.

What this patch does not touch:

• contextual/magical injection of Workspace args: still comes from currentWorkspace() and the session-loaded workspace
• workspace binding mode / --workspace / inherit-vs-detect paths
• git-vs-local detection and module loading
• existing local host workspace cache behavior

How synthetic workspaces behave in this patch:

• They are rootfs-backed, like remote/git workspaces internally, but with a directory://<digest> address.
• file, directory, and findUp work without a host client ID.
• Cache invalidation is content-addressed through the Directory input. It will not reflect live host changes unless the caller passes a new/changed Directory, which is the right behavior for dag.Workspace(myDir).

Edges / things I’d call out before merging:

• update() stays local-only and should continue to error for synthetic workspaces.
• checks/generators/services still rely on the loaded current workspace modules, so synthetic workspaces are not yet a replacement for workspace module discovery.
• gitignore: true on workspace.directory is effectively host-local behavior today; synthetic/rootfs-backed paths don’t get special host gitignore handling.

So: additive and fairly contained for filesystem-style workspace use. It only becomes more disruptive if we want synthetic workspaces to participate in module loading, source identity, or generator/check/service discovery as first-class workspace roots.

💻

[near halo]

run adversarial review in extra high mode

[lunar summit]

the patch has one hard blocker.

Findings:

1. Blocker: workspace() builds Address with dir.ID().Digest() (core/schema/workspace.go:103-109). ObjectResult.ID() returns a handle-form engine result ID, and call.ID.Digest() panics on handle IDs. So dag.Workspace(dir) can panic before returning. Use a recipe/content digest path, actual Directory.digest, or leave address non-canonical.
2. Address semantics are overclaimed. Even after fixing the panic, directory://<digest> should not be called content-addressed unless it is based on real directory content. Otherwise same content can get different addresses, and the address is not a git/local source identity.
3. findUp still gets the BuildKit engine before deciding host vs rootfs (workspace.go:408-438). Synthetic/rootfs mode only needs DirectoryStatFS; move engine lookup into the host branch.
4. checks, generators, and services will hit workspace has no client ID for synthetic workspaces (workspace.go:472, 567, 678). That should be an explicit unsupported error or real module-loading semantics.
5. workspace.directory(..., gitignore:true) silently does nothing for rootfs/synthetic paths. Pick doc, error, or implementation; silent ignore is the bad option.
6. Add a rootfs guard for non-host workspaces. A workspace with neither HostPath nor Rootfs should fail clearly before resolveRootfs/findUp try to use it.
7. Test gap: the integration test is useful but did not exercise this here because the local CLI generated an old SDK. Add a lower-level schema test for workspace(directory:) that covers constructor, file/dir, findUp, address behavior, and unsupported methods.

Contextual injection, workspace binding, and git-vs-local detection still look mostly untouched. The risk is semantic: this is a Directory-backed filesystem view, not a full loaded workspace/source identity yet.

💻

[near halo]

4. -> seems like a bug no? you seemed to accept the unfinished feature?

5. why?

6. explain (tldr)

7. ok do it

[lunar summit]

Done. And yes: on 4, I undercalled it. If checks/generators/services are visible on Workspace, synthetic workspaces falling through to workspace has no client ID is a bug, not an acceptable unfinished state.

Changes made on codex/synthetic-workspace:

• Fixed the constructor blocker: workspace(directory:) now uses the directory content-preferred digest for directory://..., not dir.ID().Digest().
• Added rootfs guard for non-host workspaces.
• Moved findUp engine/client setup into the host-only branch.
• Made gitignore:true on synthetic/rootfs-backed workspace.directory return an explicit error.
• Made checks, generators, and services return explicit unsupported errors for synthetic/rootfs-backed workspaces.
• Added raw GraphQL integration coverage for workspace(directory:): address, file, directory entries, findUp, gitignore error, and unsupported methods. This avoids depending on generated Go module SDK content.

Verification:

• go test ./core/schema passes.
• go test ./core/integration -run '^$' passes package compile.
• cd sdk/go && go test . -run TestWithWorkspace passes.
• Targeted runtime integration still cannot run in this VM: the available engine schema says Cannot query field "workspace" on type "Query", so it is still not serving the patched schema. That is the same local engine freshness issue as before, but the test is now in the tree for a current-engine CI run.

💻 📝

[near halo]

@lunar summit open a second PR implementing synthetic workspace on top of the workspac branch.

[lunar summit]

Opened the second PR on top of workspace: https://github.com/dagger/dagger/pull/13217

Branch: codex/synthetic-workspace-workspace
Commit: 583955e51 feat: support synthetic workspaces

Passed:

• go test ./core/schema
• go test ./core/integration -run '^$'
• (cd sdk/go && go test . -run TestWithWorkspace)
• go build -o /tmp/dagger-synthetic ./cmd/dagger

Runtime caveat: the targeted synthetic integration test still cannot run to completion in this VM because the engine schema available to the test is stale and rejects Query.workspace with Cannot query field "workspace" on type "Query". I included that note in the PR body.

💻 🛠️ 📝

[near halo]

Deep analysis. What are structural differences between the two patches?

[lunar summit]

h. workspace changed resolveWorkspacePath from a simple string helper into (string, error) with root-escape checks. The port preserves that, so synthetic findUp now sits behind the newer traversal guard. The first patch was structurally simpler but tied to the older non-erroring resolver.

3. Schema registration moved. On main, the patch directly added workspace next to currentWorkspace. On workspace, currentWorkspace is factored into currentWorkspaceField with passthrough telemetry, and Workspace has many more fields. The second patch is less disruptive because it only adds the root constructor and wraps the existing module-backed feature paths.

4. Unsupported-feature handling matters more on workspace. checks, generators, and services are now closer to real workspace config/module plumbing. The guard for synthetic/rootfs-backed workspaces is not just nicer errors; it prevents rootfs-only synthetic workspaces from entering host/client/module assumptions.

5. Tests changed the most. First patch added both a Go module integration test exercising dag.Workspace(dir) and a raw GraphQL test inside the monolithic workspace_test.go. Second PR adds a separate workspace_synthetic_test.go with the raw GraphQL coverage only. The generated SDK API is still present and compile-smoked, but the second PR has weaker end-to-end Go-module coverage until the local/CI engine can expose the new field.

6. SDK changes are effectively identical: dag.Workspace(directory) plus Query.Workspace(directory). On workspace, they just land amid a much larger generated Workspace API.

Net: first patch was simpler but on obsolete workspace structure. Second patch is better aligned with the workspace branch and less invasive to its plumbing, but it intentionally gives up the high-level Go integration test for now; that is the main coverage gap.

💻 🛠️

[near halo]

I don't like raw graphql in the tests. dagger generate -y then you'll have the bindings

[lunar summit]

Done. I amended and force-pushed PR #13217: https://github.com/dagger/dagger/pull/13217

Commit: 60c2a4617 feat: support synthetic workspaces

Changed from the previous version:

• removed raw GraphQL from the synthetic workspace integration test
• test now uses generated Go SDK bindings: c.Workspace(dir), ws.File, ws.Directory, ws.FindUp, and unsupported Checks/Generators/Services calls
• ran dagger generate -y and included the relevant generated SDK/docs output
• updated the PR body and local runtime note

Checks passed:

• dagger generate -y
• go test ./core/schema
• go test ./core/integration -run '^$'
• (cd sdk/go && go test . -run TestWithWorkspace)
• go build -o /tmp/dagger-synthetic ./cmd/dagger

The targeted runtime test still fails in this VM because the test engine schema rejects Query.workspace as unknown. It is now going through generated SDK bindings, so the remaining issue is engine/schema freshness, not raw GraphQL. I left the broad unstaged go.mod/go.sum churn from full generation out of the commit.

💻 🛠️ 📝

[near halo]

close the one targeting main.

run xhigh subagent for adversarial review. goal is to make it ready to merge. no shortcuts! Maintainers will decide if mergeable.

Add the pr to 1.0.0-beta.1 milestone. create milestone if needed

[lunar summit]

Done.

PR #13217 is updated and pushed: https://github.com/dagger/dagger/pull/13217
Commit: 1a756c600 feat: support synthetic workspaces
Milestone: 1.0.0-beta.1 created/assigned.

Xhigh adversarial review found one real semantic gap: synthetic workspaces could look like empty config/modules instead of saying those APIs are unsupported. Fixed that by making config/module/env/install/check/generator/service paths fail explicitly for synthetic/rootfs-backed workspaces. Also marked Query.workspace(directory:) experimental, clarified workspace/address docs, scoped the gitignore: true unsupported error to synthetic workspaces, and included generated SDK/docs output including PHP refs.

Verification:

• local: dagger generate -y, go test ./core/schema, go test ./core/integration -run '^$', (cd sdk/go && go test . -run TestWithWorkspace), go build -o /tmp/dagger-synthetic ./cmd/dagger
• remote: all PR checks pass; only check-for-changelog is skipped

I did not close #13212: it is the only open bot-authored PR targeting main, and it is unrelated (Prototype repo-first Dagger Cloud CLI). I found no open main-targeting synthetic-workspace PR to close.

Local VM note: full generation left broad unstaged go.mod/go.sum churn in the worktree; it is not in the PR.

💻 🛠️ 📝

[near halo]

tldr

[lunar summit]

PR #13217 is ready for maintainer review: pushed at 1a756c600, milestone 1.0.0-beta.1, all remote checks pass except skipped changelog. Xhigh review fix landed: synthetic workspaces now explicitly reject unsupported config/module/env/install/check/generator/service APIs instead of silently looking empty. I did not close any main-targeting PR because the only open one on main is unrelated (#13212).

💻

[near halo]

what are these "unsupported APIs" that you now reject? Use specific examples

[lunar summit]

Concrete examples, all on a synthetic workspace like ws := dag.Workspace(dir):

Supported:

• ws.File("foo.txt")
• ws.Directory("subdir")
• ws.FindUp("dagger.json")

Now explicitly rejected:

• ws.ConfigRead() -> workspace feature "config" is not supported...
• config mutations like ws.ConfigWrite(...), ws.EnvCreate(...), ws.EnvRemove(...), ws.ModuleInit(...) -> workspace feature "config mutations" is not supported...
• ws.ModuleList() -> workspace feature "modules" is not supported...
• module.Settings() from a workspace module -> workspace feature "module settings" is not supported...
• ws.EnvList() -> workspace feature "environments" is not supported...
• ws.Install("github.com/dagger/dagger/modules/wolfi") -> workspace feature "module installation" is not supported...
• ws.Checks().List(), ws.Generators().List(), ws.Services().List() -> workspace feature "checks"/"generators"/"services" is not supported...

Reason: those APIs assume a loaded workspace config/modules/client context. dag.Workspace(dir) is only a Directory-backed filesystem workspace for now.

💻