#github-feed
1 messages ยท Page 22 of 1
Thanks for the suggestion @grouville, I gave it a try under https://github.com/dagger/dagger/pull/12922/changes/0fb4d41a03569731d1eab65ca11241a86940d56f and it seems to be working.
But that seems safer than accidentally reusing a checkout when kept .git state / tags actually changed the result.
erring on the side of safety feels good, especially since this rebase is rather large. Maybe we could come back to it later if we really want to.
PS the above link is to a temporary PR i've been testing with, once I get everything passing I'll try to merge or squash it into the existing lockfile PR.
[dagger/dagger] Pull request opened: #13047 core: include module variants in function cache identity
Summary
Fixes #13046 by making the module implementation-scoped cache identity include the AsModuleVariantDigest computed by moduleSource.asModule. Toolchain constructor customization defaults are part of that variant digest, so changing a default from alpha to beta now gives omitted-argument calls a distinct function cache key.
The change also persists AsModuleVariantDigest in the module payload so imported persisted modules keep the same variant-sensitive identity.
Testin...
cached-execs and use-cached-exec-service also needed an update but the corresponding tests were passing because their failure was silenced when they were marked as flaky in 6fdf164c7f.
This commit removes the mark and updates the golden tests.
Problem
Workspace.path should return the path of the detected workspace. Today, in some cases, it returns the path where workspace detection started.
Repro:
cd PATH/TO/WORKSPACE/subdir
dagger core current-workspace path
# actual: subdir
# expected: . or the detected workspace path
This is inconsistent with:
dagger core current-workspace directory --path=.
which loads the detected workspace, not subdir.
Likely cause: core/workspace/detect.go sets Path from...
Fixes #13051.
Summary
Workspace.path now always reports the detected workspace path. The directory where detection started is stored separately and exposed through Workspace.cwd.
This adds WorkspaceCwd.path, WorkspaceCwd.directory, and WorkspaceCwd.file. Relative paths on those fields resolve from the current working directory within the workspace; absolute paths still resolve from the workspace boundary.
The change also keeps nested uninitialized workspace flows coherent: `w...
Refs #13051.
Summary
Workspace.file and Workspace.directory now resolve relative paths from the workspace directory, not from the nested cwd where workspace detection started.
This is intentionally narrower than fixing Workspace.path on main. The public Workspace.path behavior is left unchanged for now, while Workspace carries an internal filesystem path base for the file and directory APIs.
For remote rootfs-backed workspaces, the internal filesystem path includes the dete...
Let container commands inherit a workspace
Problem
When a command runs inside a Dagger container and opens its own Dagger connection, the inner client detects its workspace from the container filesystem.
That is often the wrong workspace.
Here, "workspace" means the Workspace object selected by the outer caller: its source, config, lock state, and module layout.
For example, ../dagger_e2e-helm/e2e/helm/helm_test.go runs as a Go test suite, and the tests use the Dagger SDK:
Enforcing policies with hooks/middleware in Dagger pipelines is an important enterprise feature โ you want guardrails that apply consistently across all pipelines without requiring each pipeline author to implement them.
The use cases:
- Security: no container images from untrusted registries
- Compliance: all builds must produce a SBOM, all containers must be signed
- Cost: pipelines above a certain complexity require manager approval
- Quality: no deployments to production without pass...
Dagger Attributes as a concept makes sense โ it's the missing metadata layer that lets functions declare capabilities and constraints without baking them into the type system.
A few use cases where attributes would shine:
1. Cache invalidation hints:
// @cache: content-addressable (invalidate only when inputs change)
// @cache: none (always re-run, e.g. for fetching external state)
// @cache: time-based (invalidate after N hours)
func (m *Module) BuildImage(src *Directory) *Contai...
Hi there!
We're currently using Dagger extensively in our company's pipelines. On some of our monolithic repositories that hold our modules, we've committed codegen and also have additional checks to verify that the codegen is in sync for a given PR.
A recent change in Dagger has seemingly made it such that dagger develop will introduce changes to go.mod and go.sum if there are upstream dependency updates. For us, this doesn't feel quite ideal as it can lead to polluting PRs with ...
we've committed codegen and also have additional checks to verify that the codegen is in sync for a given PR
We are working toward this goal, as natively supported: push to have codegen files committed (and used at runtime), migrating from dagger develop to dagger generate and generate functions will automatically come with built-in check. That to say what your doing will be the default way to deal with modules.
With that, maybe we can simply not run the go mod tidy in more ...
We are working toward this goal, as natively supported: push to have codegen files committed (and used at runtime), migrating from dagger develop to dagger generate and generate functions will automatically come with built-in check. That to say what your doing will be the default way to deal with modules.
I'm happy to see that our ideas around codegen make sense for others too ๐
With that, maybe we can simply not run the go mod tidy in more cases. Right now, during a develop the ex...
What is the issue?
Dagger v0.20.4+ fails to load any Go-based Dagger module whose go.mod has a private transitive Go dependency (github.com/org/private/libs).
The failure happens during SDK codegen
Might have been introduced in #11826
Works on v0.20.3
Dagger version
v0.20.4
Steps to reproduce
Create a dagger module using go with a dependency on a private go library
{
"name": "test-module",
"engineVersion": "v0.20.4",
"sdk": {
"source": "go",
"config": {
...
fix test breakage introduced in 13048
Summary
Adds a statically generated __dagger.entrypoint.ts for the TypeScript SDK,
mirroring Go's dagger.gen.go. The file is emitted at the module root by
dagger develop and imports the user's classes directly โ replacing the
runtime AST + reflection dispatch path with a flat switch(parentName) โ switch(fnName) that the engine can jump straight into.
Design doc: design/typescript-module-entrypoint.md.
What changes
- New emitter `sdk/typescript/src/module/i...
Summary
dagger toolchain uninstall fails if the registered toolchain points at a local source path that no longer exists. In that state, uninstall should still be able to remove the toolchain registration instead of trying to resolve/load the missing source first.
Reproduction
From a Dagger repo with a local helm toolchain registered at ./toolchains/helm-dev:
-
Remove the local source directory:
rm -rf ./toolchains/helm-dev -
Try to uninstall the toolch...
Short version
This used to work on v0.20.6. On current main, it does not: a service with a PRIVATE cache mount can resolve to different hostnames through Service.Endpoint and WithServiceBinding.
Theseus made this visible: some checks start a local/dev engine or an SDK runtime, then connect a client to it.
The concrete bug is this: a lazy service graph with a CacheSharingModePrivate cache volume can get a fresh dagql privateNonce on each evaluation. Since service runtime id...
Fixes #13060
Why
PRIVATE cache mounts are supposed to isolate cache contents. They should not make the same lazy graph resolve to different service identities each time it is evaluated.
After the dagql cache migration, service hostnames are derived from the resolved graph. That made the old privateNonce behavior visible: every evaluation of a PRIVATE cache volume injected fresh identity into the graph.
So a service like this could split:
srv := dag.Containe...
Thanks both! Just wanted to also point out that my initial speculation for go mod tidy causing dependencies to be upgraded was incorrect, it's actually a go get -u that runs within the Dagger develop that is causing those. Is there any intention in keeping this behaviour when running develop?
That to say what your doing will be the default way to deal with modules.
Glad to hear that this will become a natively supported pattern though! It will simplify a lot of our existing ...
Summary
-
remove Changie .changes/.next from version computation
-
compute dev versions from latest stable v* git tag with a patch bump
-
split release target-version regeneration into individual @generate functions
-
update release checklist to run plain dagger generate
Verification
- dagger --progress=plain generate -l
- dagger --progress=plain call -m version next-patch-version
- dagger --progress=plain call -m version version
- go test . in version, toolchains/cli-dev, toolchains...
Bumps the engine group with 29 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.19.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
Problem
Workspace-aware modules often need local git facts about the workspace they are operating on: HEAD, commit metadata, semver tags, and whether the worktree has uncommitted changes. Today there is no small workspace-scoped API for that.
The available option is GitRepository, which is a broad, remote-oriented git API. Using it for local workspace state leaks more model than callers need and can bring along extra behavior and perf risk. Some callers work around this by passing sepa...
[!WARNING]
To be merged only when this change will land as the CI engine. Before that we might lose the
generatedcheck.
this was only used to check generated files, but it's now built-in
What are you trying to do?
Run multiple Dagger engines in parallel in Kubernetes using StatefulSet, such as multiple across nodes
Want to have some setting in the Helm chart to override this. For example:
engine:
statefulSet:
replicas: 1
Why is this important to you?
Parallelizing execution by selecting a different statefulset pod...
Summary
Moves the docs-dev reference generation breakout proposals onto a clean branch based on latest main.
This PR contains planning docs for:
- CLI reference generation via an importable Cobra command tree and a reusable Cobra docs generator
- GraphQL reference generation via live engine service introspection and a reusable SpectaQL module
- JSON schema reference generation via source-adjacent
go:generatedeclarations
Notes
This is documentation only. It does not change gene...
Currently argument default values for arguments that are enums do not have the default value loaded in the schema, and therefore also not show in the --help text.
The dagql cache migration exposed a bad interaction between PRIVATE cache mounts and lazy service identity. PRIVATE currently injects a fresh dagql cache key input, so a service can resolve one hostname through Endpoint and another through WithServiceBinding.
This change does not remove or redesign PRIVATE. It is a small internal unblocker for the dev runtime paths that hit the issue today.
I verified this in a temporary worktree by building the engine with ./hack/dev and ru...
Noticed a handful of low-hanging cleanup fruit while catching back up. I'm sure there's more than this but sending out this batch of what I've noticed so far today.
All low-risk, mostly deleting functionally dead code. See each commit for details.
Been working on better support for tests in the Cloud UI - this PR adds the TUI equivalent, pulling logic down into dagui where it'll be shared between them.
TODO:
- [ ] Automatically enter tests UI when tests are detected?
- [ ] Integrate with
dagger check- what happens when multiple checks produce tests?
Along the way:
- Prevent zeroing out
ChildCount; fixes incremental loading for traces with >100k spans. With this zeroing out you could get a trace that only had the top...
Summary
- add
Workspace.gitand the newWorkspaceGitobject for workspace-local git state - add
GitRef.committedAt - expose
head,uncommitted,latestSemverTag, andcurrentSemverTag - explicitly reject git worktrees for the first pass instead of expanding the workspace boundary
- regenerate GraphQL docs and SDK bindings
Fixes #13064.
Tests
git diff --checkGOCACHE=/tmp/go-build go test ./core/workspace- `GOOS=linux GOARCH=amd64 GOCACHE=/tmp/go-build go test -c ...
The change in #13071 removed a comment line, changing the line numbers in go-based dagger runtimes prompting a regeneration.
Nested Dagger commands can run with their own process-level presentation settings even when they inherit OpenTelemetry baggage from a parent command. In particular, TestDaggerUp starts an inner dagger CLI with NO_COLOR=true and DAGGER_PROGRESS=logs so it can parse the tunnel log line that contains the random host port.
telemetry.Init extracts trace context and baggage from the environment. When inherited baggage is present, OpenTelemetry baggage extraction replaces the baggage already on t...
The Vault secret provider helper starts hashicorp/vault:1.18 as a Dagger service and then talks to it on port 8200. That image has OCI EXPOSE metadata for 8200/tcp, but Dagger does not currently promote image EXPOSE metadata into Container.Ports for service healthchecks or service port introspection.
This matches the existing note in core/integration/container_test.go around line 4831: after Import or container.From, ports can show up under image config, but do not actually get set up as D...
We have been investigating rare CI hangs where engine work appeared to stall behind container execution. The strongest captured evidence was not a dagql cache or Directory.Entries deadlock: the goroutine dump showed Dagger blocked inside go-runc's monitor wait for a runc run, while ps output showed the corresponding runc process still alive and its .init child stuck as a zombie.
That zombie process was the key breadcrumb. It pointed the investigation away from Dagger dependency trave...
Bumps the sdk-java group in /sdk/java with 4 updates: com.fasterxml.jackson:jackson-bom, io.smallrye:smallrye-graphql-client-api, io.smallrye:smallrye-graphql-client-implementation-vertx and com.palantir.javapoet:javapoet.
Updates com.fasterxml.jackson:jackson-bom from 2.21.2 to 2.21.3
Commits
374fbd0 [maven-release-plugin] prepare release jackson-bom-2.21.3
7059df7 Prep for 2.21.3 release
2fd60bd Merge b...
Bumps the sdk-typescript group in /sdk/typescript with 9 updates:
| Package | From | To |
|---|---|---|
| @opentelemetry/core | 2.7.0 |
2.7.1 |
| @opentelemetry/exporter-jaeger | 2.7.0 |
2.7.1 |
| @opentelemetry/exporter-trace-otlp-http | 0.215.0 |
0.216.0 |
| [@opentelemetry/sdk-metrics](https://github.com/open-te... |
Bumps the engine group with 28 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.20.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
I don't have the data right now, but i've been monitoring dagger engine v0.20.6 last week and it was OOMing despite having 64GB of RAM.
Need to identify what's leaking and see if a fix can be backported to 0.20.
This simplifies our CI by removing a special case. Specifically, 'go generate' works as expected, its output can always be checked in git, and CI can check for an empty diff when calling it post-push.
The price to pay is that we track 4 small binary objects in git. This pattern is officially recommended by upstream ebpf tooling: per https://ebpf-go.dev/guides/portable-ebpf/ :
We recommend building eBPF C code from within a container with a stable LLVM toolchain, as well as checking all ge...
Problem
The core Git API currently models branches, tags, HEAD, and commit IDs through GitRef. That works well for ref lookup and tree checkout, because a commit ID can be used as a ref-like revision.
But commit metadata does not belong directly on GitRef. A ref is a pointer or name that resolves to a commit; the commit itself is the immutable object that has author, committer, message, parent, and tree metadata.
Today this leaves no clean place to expose commit details such as comm...
Summary
This PR fixes the missing shared result failure mode that could show up after an earlier failed module SDK/type-def generation attempt. The root cause was that execution metadata persisted encoded dagql object/call IDs such as module IDs, function calls, and other loadable graph handles. If the first attempt failed at the wrong point, a later retry could read stale execution metadata from cache and try to reload graph state that was no longer valid, producing errors like `failed...
Summary
- add a new GitCommit core type with commit metadata fields and tree checkout support
- add GitRef.asCommit and make GitRepository.commit(id:) return GitCommit for v0.21.0+ while preserving the legacy GitRef return for older schema views
- regenerate the Go SDK and add focused parser/integration coverage
Fixes #13086
Verification
- GOOS=linux GOARCH=amd64 go test -c ./core -o /tmp/core.test
- GOOS=linux GOARCH=amd64 go test -c ./core/schema -o /tmp/schema.test
- GOOS=linux GOA...
What is the issue?
Description
Since Dagger v0.20.7, @function-decorated methods inherited from parent classes are no longer visible in dagger functions output or dagger call --help. They were working correctly in v0.20.6.
Root Cause
PR #11803 ("feat(sdk/python): AST based analysis to introspect"), backported into v0.20.7, replaced the runtime-based introspection with a static AST parser.
Before (v0.20.6): The SDK used inspect.getmembers(cls) which traverses the Python ...
Summary
- Fixes #13089. Inherited
@functionmethods stopped showing up indagger functions/dagger call --helpwhen the Python SDK switched from runtime introspection to a static AST analyzer in v0.20.7 (#11803). The new analyzer iterated only over a class's ownnode.bodyand never walked base classes for@functionmethods, whileinspect.getmembers()previously discovered them via Python's MRO. - The fix mirrors the existing
_find_inherited_constructorpattern: a new `_find...
[dagger/dagger] Pull request opened: #13091 fix(sdk/python): handle relative imports in AST analyzer
Summary
The AST analyzer used by dagger develop for the Python SDK didn't handle relative imports correctly:
from . import xraisedValueErrorbecauseimportlib.import_modulewas called with an empty module name.from .x import y(and other dotted relative forms) fell through toimportlib, where they either errored or silently resolved to the wrong top-level module.
This meant a Dagger module split across multiple files using ordinary Python relative imports could fail t...
This PR allows us to switch off the new Python module AST analyzer (introduced at commit d3a8152) using an environment variable "DAGGER_PYTHON_SDK_LEGACY". This is a temporary solution to prevent widespread breakage.
Summary
The AST analyzer used by dagger develop for the Python SDK silently mishandled module-level type aliases such as:
Source = Annotated[dagger.Directory, dagger.DefaultPath(".")]
@dagger.object_type
class Foo:
@dagger.function
def build(self, src: Source) -> dagger.Container: ...
Two things went wrong:
- Type resolution:
Sourcewas never registered in the analyzer's namespace, so_resolve_namefell through to its catch-all path and warn-assumed `So...
Running dagger lock update, when the file doesn't exist, should create an empty file rather than return an error.
Why
The Python SDK's static AST analyzer (introduced in #11803) replaced the
runtime typing.get_type_hints() introspection that used to back
dagger develop / dagger functions. The runtime path quietly handled
a lot of Python's surface area โ aliases, MRO walks, type-alias
re-exports, optional wrappings, cross-file imports โ because Python
itself does that work for you. The AST analyzer does it from scratch
against the parse tree, so each Python idiom that the analyzer doesn't
expl...
This reverts commit bcb2193d97ae78e6f404979fe6b278d19d739be7.
Despite the welcome speed gains, it is not ready for general availability due to gaps in feature coverage.
What happened
In v0.20.7 (#11803) we changed how the Python SDK reads your module to register it with the engine. The old way ran your code; the new way reads it as text only. Reading text instead of running code is the right long-term move โ it unlocks a feature called "self calls" โ but the new reader doesn't yet recognise everything Python can express.
That means modules that worked fine in v0.20.6 may now register the wrong types or skip parts of the schema in v0.20.7 and later.
...
Summary
- cancel
withExeccalls when a bound service exits so the service failure is surfaced - propagate bound-service exits through container-backed services
- suppress service-exit propagation for interactive terminals so the user's shell stays open
Tests
dagger call --progress=dots engine-dev test --run 'TestServices/TestServiceDependencyExitShortCircuits' --pkg ./core/integration/- `dagger call --progress=dots engine-dev test --run 'TestModule/TestDaggerTerminal/bound ser...
Bumps the engine group with 30 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.20.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
The Dagger example linked also linked to OpenMeter, now it points to the correct repo.
Also removed OpenMeter from the list of examples as they no longer use Dagger, could alternatively link to an older revision of OpenMeter.
What are you trying to do?
Coming from the Go SDK, I'm really used to the With function on types, because it allows using the builder pattern without breaking it up into different parts.
For example:
container.
WithEnvVar("FOO", "BAR").
With(cacheMounts)
I think this should already be possible to implement using blocks?
Why is this important to you?
Nicer code :)
How are you currently working around this?
I don't :(
Follow-up to #13098 which only partially addressed parallel initialization races.
Bumps the engine group with 30 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.20.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
Fix nested client session resolution
Main goal here is to fix a regression introduced by fix: builtin dang dependency module loading (#13007).
- It resulted in occasional errors like
Error: failed to load module source dependencies: loading workspace: workspace detection: failed to stat path: failed to get requester session: session for "v51anobcgxoitkmvey93jzs6z" not found
The first commit is the direct fix: it separates exact clie...
Hit a failed run in CI due to race detector going off during test: https://dagger.cloud/dagger/checks/github.com/dagger/dagger@a1fb6be1af1460b4470021e69ba126463cd0fcb6?check=test-split:test-base&span=59440d44c1f434db&viewMode=trace
TestUserDefaults/TestSimple runs table cases under the integration suite's parallel middleware. The test built one shared base container in the outer function scope and then one table case reassigned that shared variable from inside its parallel subtest when app...
Summary
Replaces the Go SDK's packages.Load-based source analysis with a pure AST scan and folds type discovery into the same generate-module codegen call that produces the bindings. The standalone generate-typedefs codegen path and the Go SDK's moduleTypes engine entrypoint are both removed.
What changes
- New
cmd/codegen/generator/go/astscanpackage โgo/parser+go/astwalker that extracts a module's declared types and resolves references against the introspection sc...
What does this look like in practice?
What is the issue?
We're seeing some layer-cache misses where we'd expect hits since the buildkit removal. Two parallel checks resolving to an identical withExec() chain each run the the underlying exec. Pre PR-#11856, the same chain was deduped and only ran one exec.
Dagger version
dagger v0.20.6 (but running hack/dev from main)
Steps to reproduce
# src/dedupe/main.py
import dagger
from dagger import dag, check, function, object_type
CACHE_KEY = "dedupe_probe"
...
This was causing log telemetry to be dropped when running Dagger's Vitest integration in dagger check, while working fine in dagger run.
Not sure if there was a good reason for this check - it's valid to not have a schema URL. Maybe it mattered at an earlier revision?
In https://github.com/dagger/dagger/pull/13049#pullrequestreview-4248120719 we realized that new files needed to be added.
That prompts the question: when upgrading dagger versions, but keeping an old version specified in dagger.json, the generated bindings should honor the version in the dagger.json.
Closes #13066.
Tiny fix, but useful imo: StatefulSet.spec.replicas was hardcoded to 1, so the chart could not scale StatefulSet engines even when users passed a replicas value.
Repro before this patch:
helm template repro helm/dagger \
--set engine.kind=StatefulSet \
--set engine.statefulSet.replicas=3
Expected spec.replicas: 3, got spec.replicas: 1 because the template had the value baked in.
This adds engine.statefulSet.replicas with the same default of 1, so ...
Use the session ID as the default DagQL concurrency key so equivalent in-flight calls can be single-flighted across clients in the same session.
Add same-session fallback for secret and socket session resources when the original client attachable is no longer available. This intentionally does not cover host services or mid-stream SSH forwarding retry.
Fix for https://github.com/dagger/dagger/issues/13112, but requires some more validation; deduping inflight requests is good but ...
Background
Module type generation currently has SDKs construct a container that writes a module ID to a file, then the server reads that ID back. That path is being reworked separately, but the current implementation is flaky: the written ID can be handle-based, and if that handle goes stale before the dependency wiring catches up, module type generation can fail intermittently.
This change is a narrower fix for that race while the larger module types rework is still in progress.
Cha...
Summary
- Re-enable
TestConnectOptionin the Go SDK test suite - Replace
io.Pipelog capture with a thread-safe buffer becauseio.Pipewas hanging atConnect - Update the assertions to match the current connection and log output more reliably
What I changed
This updates sdk/go/client_test.go to make TestConnectOption work again.
The previous version of the test was skipped because it was broken and producing empty output. In practice, the io.Pipe approach was hanging at ...
Summary
- Disable Git auto-maintenance and auto-gc for ephemeral changeset merge repositories.
- Add context to temporary
.gitcleanup errors in the changeset merge paths. - Rename engine snapshotter temp mount directories from
buildkit-mounttodagger-mount.
Why
CI has seen flaky unlinkat failures while removing the temporary .git directory used during generated changeset checks. This is still a hypothesis rather than a proven root cause, but the current evidence points at...
Summary
dagger check dagger-dev:generated spends a lot of time in go:generate-dagger-runtimes, but the Go runtime generation work was being forced serially across modules.
This changes GenerateDaggerRuntimes to generate per-module runtime layers in parallel, using the existing Go toolchain limit setting. The default limit is bumped to 10, and generated layers are applied back in module order so the final changeset remains deterministic.
The generated Go bindings were updated for...
Summary
- Replace eager per-operation containerd lease creation with a lazy operation lease scope.
- Ensure the lazy lease at snapshot/content boundaries that actually need containerd lease membership, with existing explicit fallback leases still available outside DAGQL operation scopes.
- Add focused coverage for lazy lease behavior, DAGQL lease acquisition, and snapshot manager create boundaries.
Root Cause
The e-graph-era operation lease provider eagerly called leaseutil.WithLease b...
Summary
- Flatten lazy
Directory.withDirectorychains so evaluating the final directory only materializes the base and each source once, then applies the chain in a single pass. - Reuse that directory chain replay for
Container.withDirectorychains, including mounted directory targets, while conservatively falling back for named owner resolution that requires parent filesystem lookup. - Add integration coverage for layered chained directory/container semantics, mounted container direc...
Bumps the sdk-java group with 7 updates in the /sdk/java directory:
| Package | From | To |
|---|---|---|
| com.fasterxml.jackson:jackson-bom | 2.21.2 |
2.21.3 |
| io.smallrye:smallrye-graphql-client-api | 2.18.0 |
2.18.1 |
| io.smallrye:smallrye-graphql-client-implementation-vertx | 2.18.0 |
2.18.1 |
| io.vertx:vertx-web-client | 4.5.26 |
4.5.27 |
| [com.palantir.javapoet:javapoet](http... |
Bumps the sdk-typescript group with 13 updates in the /sdk/typescript directory:
| Package | From | To |
|---|---|---|
| @opentelemetry/core | 2.7.0 |
2.7.1 |
| @opentelemetry/exporter-jaeger | 2.7.0 |
2.7.1 |
| @opentelemetry/exporter-trace-otlp-http | 0.215.0 |
0.217.0 |
| [@opentelemetry/sdk-metrics](https://git... |
Bumps the sdk-elixir group in /sdk/elixir with 2 updates: ex_doc and jason.
Updates ex_doc from 0.40.1 to 0.40.2
Changelog
Sourced from ex_doc's changelog.
v0.40.2 (2026-05-08)
Bug fixes
Add rel="nofollow" to external links in HTML output
Use blockquote in llms.txt description
Void elements in epub, such wbr, must be terminated by the matching end-tag
Fix content container scrolling in older ve...
Bumps the engine group with 31 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.20.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
Summary
- Adds
dagger module-checks @โ an experimental Cloud-discovery subcommand that fetches the recorded checks for a given commit and renders a colored progress bar + grouped per-check table. - Two flags for non-interactive consumers:
--jsonemits a stable JSON document (matched commit, refs, summary counts, per-check status/duration/traceId/traceUrl).--watchrefetches and redraws every 5 seconds until interrupted; works for both human and JSON output.
- Factors a reus...
Thanks both! Just wanted to also point out that my initial speculation for go mod tidy causing dependencies to be upgraded was incorrect, it's actually a go get -u that runs within the Dagger develop that is causing those. Is there any intention in keeping this behaviour when running develop?
We might be able to get the dagger package without dynamically add it to the module's code if we don't codegen at runtime because we would assume the go.mod of that module is right.
We could just a...
What changed
This fixes the Helm e2e package under the filtered golang:check workspace.
- Include
e2e/helm/dagger.jsonin the Helm test workspace inputs so the runtime module config matches the generated Go client. - Harden
version.Newso it only stores a Git repository when the provided.gitdirectory can actually resolveHEAD; otherwise it uses the existing fallback version path.
Why
TestInstallK3S mounts dag.DaggerCli().Binary() into the Helm/kubectl container. The ...
This PR adds first-class Incus runtime support to Dagger and hardens the image loading/extraction path so it correctly handles modern Linux rootfs layouts.
The user-facing runtime name is now consistently Incus, not LXC. The implementation aligns with the incus CLI/daemon and image+incus://... engine config format, which matches how users actually install and operate the runtime.
This PR also improves the Incus image conversion path to be safer and more compatible with real-worl...
Recently the bitbucket public repo seems to have been cleaned up by Atlassian. We believe this is due to inactivity, as reported by other Reddit members.
No warning emails seem to have been sent on their end. Instead of re-enabling until they wipe it out again, let's commit and re-enable in the future if needed
Problem
Runtime modules are still handled like normal module sources. That makes engine-bundled SDK runtimes harder to expose, list, and cache as first-class sources.
Solution
Add a built-in module catalog and a BUILTIN_SOURCE module kind. Add GraphQL fields to resolve one built-in source or list the catalog. Load bundled runtimes from their embedded rootfs when a manifest digest is available. Keep git-backed fallbacks for other runtimes. Regenerate SDKs and API docs.
Problem
SDK runtime modules need stable names. Today, callers must know where each runtime module lives. That makes module references harder to use and harder to move.
Solution
Add built-in module sources backed by an engine catalog. Dagger can now resolve names like python-runtime and typescript-runtime through the GraphQL API. The change also exposes catalog listing, persists built-in source metadata, resolves built-in dependencies, and regenerates SDK and API references.
NixOS and Nix-built distroless images expose the system CA bundle as a symlink into a read-only /nix/store target, so commonInstaller's append path can't write through it. Add a nixosLike installer that detects NixOS via /etc/NIXOS or ID=nixos/ID_LIKE=nixos, locates the bundle via $SSL_CERT_FILE (falling back to /etc/ssl/certs/ca-certificates.crt), materializes the symlink as a writable regular file for the duration of the exec, and restores the symlink on uninstall.
Also adds a small Cont...
Problem
@tiborvass reported a false-positive cache hit around inline module dependencies and contextual directory defaults. The important shape is: module A calls module B, B loads a host directory through an annotated/defaultPath argument, A returns a value derived from B, and a later session can reuse the cached result for A after the contextual directory used by B changed.
Reproduction
This adds TestCrossSessionInlineDependencyContextualDirChange, a focused Go-only cross-session...
Summary
When Directory.asModuleSource is called from inside a Dagger module, resolving a local module dependency can fail while loading user defaults.
It looks like DIR_SOURCE user-default loading calls Host.findUp(".env") with the module client context instead of the original caller/workspace context, so host cwd/session lookup times out.
Repro
type DirModsourceRepro {
pub direct(ws: Workspace!): ModuleSource! {
ws
.directory("/", include: ["main/**", "dep...
test was broken, should have been using dagger install rather than dagger module install, which was previously changed in 338692fbd54b358b86417fcdd3ca8133b22ac67a
Summary
- Add an integration repro for a remote git tree failing after explicit local cache prune.
- Remove the separate
git-snapshotmetadata fast path from coreRemoteGitRef.Tree. - Rely on the persistable dagql
GitRef.treeresult as the persistent cache authority for remote git trees.
Details
@vito found and reported a failure where a remote git tree could be fetched successfully, pruned, and then fail on the next fetch with a raw snapshot key not found error.
The repro s...
Summary
- Fixes #13059.
- Allow local toolchains to be uninstalled after their source path is deleted.
- Preserve other toolchain resolution errors.
- Add integration coverage for toolchain uninstall behavior.
Testing
dagger call engine-dev tests- Specific test added for this fix:
dagger call -m ./toolchains/engine-dev test --pkg=./core/integration --run='TestToolchain/TestToolchainUninstall' --test-verbose -d dag checks 'go:lint'
Summary
- route ordinary execs with the default
daggerhostname through the existing CNI namespace pool - keep explicit hostnames and insecure execs on the fresh namespace path
- fix
--oci-cni-config-pathso it reads the correct flag value - remove the stale inherited BuildKit bridge-provider mode, including
networkMode=bridge,BUILDKIT_NETWORK_BRIDGE_AUTO, and the unused bridge config fields
Details
Dagger already configures a CNI namespace pool, but normal execs were not us...
Problem
When running Dagger on a headless Linux machine, such as a VPS over SSH, `dagger` may try to open a Dagger Cloud trace URL with `xdg-open`.
If no browser is available, this currently warns with output like:
```text
04:59:45 WRN failed to open URL url=https://dagger.cloud/paulvinueza30/traces/40467b9c069ba80a7977f6c45ae36c73#58fe4c21d79fc856 err="exit status 3" output="/usr/bin/xdg-open: line 881: www-browser: command not found\n/usr/bin/xdg-open: line 881: links2: command...
Change our CI to run Go linting through github.com/dagger/go.
This shifts our own Dagger usage toward the practices we advocate: reusable modules, faster checks, and less project-specific code and configuration.
This does not change Dagger runtime behavior.
What is the issue?
Reported by @danielgafni here: #typesafe message
According to him, this did not happen in v0.20.7.
Dagger version
v0.20.8
Steps to reproduce
@function
async def test_secret_override(self) -> str:
"""Reproduce with_secret_variable override regression in Dagger 0.20.8."""
short = dag.set_secret("short", "AA")
long = dag.set_secret("long", "BBBBBBBBB")
...
This reverts commit 25be7a162ff2ff8541869ab0645b39b238701582.
_expand_alias only walked bare ast.Name nodes via a while loop, so compound annotations like Name | None passed through with the alias unexpanded. The resolver then saw "Name" as an unknown type and assumed it was an object, which the engine rejected with "find mod type for function โฆ arg โฆ type: "Name"".
After the top-level name-expansion loop, recurse into ast.BinOp BitOr nodes and expand aliases in both operands. An identity check ensures a new node is only allocated when something ...
What
Make the Java codegen Maven plugin preserve and report failures from dagger query -s -M instead of suppressing stderr and handing empty stdout to schema parsing.
Why
The current Java SDK CI failure is a NullPointerException that masks the real nested Dagger query error. This diagnostic change should let CI show the underlying query stderr/stdout and exit status.
Validation
dagger -m toolchains/java-sdk-dev call lint --progress=plain- `dagger -m toolchains/java-sdk-...
Summary
ModuleSource.withDependencies fails when the target module source is created with Directory.asModuleSource.
This is not specific to module execution. It reproduces from a normal Dagger client. A nested helper does not avoid the bug if it constructs module sources with Directory.asModuleSource, because the resulting source kind is DIR_SOURCE.
Repro
Requirements: Go and the Dagger CLI.
tmp="$(mktemp -d)"
cd "$tmp"
mkdir -p app lib
cat > app/dagger.json < app/m...
Problem
dagger trace replayed recorded progress through the frontend, but it constructed fresh frontend options instead of using the global CLI options. That meant replay ignored verbosity and debug flags such as -v, -vvvv, and --debug.
Fix
Copy the global frontend options for trace replay and override only NoExit, preserving the existing behavior of keeping the trace UI open.
Test
- `dagger --progress=plain call engine-dev test --pkg ./cmd/dagger --run TestTraceUsesGl...
What is the issue?
As discussed with @marcosnils, I am opening this issue. Here's my original message in #help on the Discord:
Is there native Dagger support for checking out repos using Git LFS with dag.git() and having it do the full pull operation? I noticed one of my builds was failing, and when I peeked inside the container, it appears that some of the files stored in Git LFS had Git metadata in place of the actual file contents. I see some hits on this Discord about how to po...
Add an experimental CLI selection path for testing Dagger builds from a github.com/dagger/dagger git ref.
Users can pass either --x-release= or DAGGER_X_RELEASE= to run a specific unreleased CLI build. Flags take precedence over the environment when both are present. If the ref is not already a full commit SHA, the CLI resolves it through GitHub, prints the resolved SHA, and tells the user how to pin that exact build for repeatable testing.
The selected CLI is downloaded and re-exec...
Problem
A module can receive a Workspace, but it cannot load another Dagger module from that workspace.
For example, a module should be able to say: "load the module at ./foo in this workspace" and get back a ModuleSource.
Today there is no direct core API for that. The only option is to implement a module-side shim for Workspace.moduleSource().
That shim is difficult and unreliable for several reasons:
- Low-level module loading logic must be reimplemented: walking dependenc...
Problem
When the engine pulls a container image, the TUI shows a stream of raw HTTP GET log lines. From a user on Discord:
This is still very confusing. Is it pulling an image? Is Docker having issues? Even a simple progress bar to tell me that it's trying to pull an image would be helpful.
Solution
Replace the raw log output with a proper progress display. From @vito on Discord:
We used to have a slick 2-dimensional progress bar in the TUI (X axis = layers, Y axis = per-laye...
[dagger/dagger] Pull request opened: #13160 fixes issue where user defaults are incorrectly expanded
avoids double expanding envfile values when using a Namespace.
fixes #12855
closes #12014
Remove the private cache nonce so PRIVATE cache volumes keep a stable cache key. As a stop-gap, serialize writes for PRIVATE caches the same way as LOCKED caches so behavior remains correct while avoiding the performance regression from always-unique cache volumes.
Stopgap for #13060
Summary
This PR tightens the Python SDK module analyzer around type annotations, especially type aliases, quoted annotations, and Annotated[...] metadata.
The analyzer runs during the Python SDK module-types registration path. When the engine asks the SDK runtime for a module's type definitions, the runtime discovers the module's Python source files and analyzes them without importing the user module. The analyzer produces internal ModuleMetadata, which the SDK converts into Dag...
Summary
- derive provider-secret handles for zero-length plaintext instead of treating them as missing
- add an integration repro for an empty env-backed Secret used through constructor user defaults
Root Cause
Provider-backed secrets without an explicit cache key derive their session resource handle from plaintext. Empty plaintext previously returned an empty handle, so user defaults like password=env://PASSWORD failed during object decoding when PASSWORD was set to an empty str...
I am trying to optimize my local development workflow and ensure that running a dagger call for the second time hits the cache completely (no code changes have occurred).
However, I am currently experiencing unexpected cache-misses on the second run, and certain steps are executing all over again.
I want to inspect exactly why Dagger/BuildKit is invalidating the cache for these specific steps.
- How can I see which input data, arguments, environment variables, or directory snapsh...
cache invalidation could happen because of several things like actual invalidatoins because of inputs changing, garbage collection, non-deterministic steps and of course Dagger bugs.
Is there any chance you can provide here a snippet that reproduces consistently what you're seeing? This would be the fastest way for us to provide feedback.
On top of that, if you have a https://dagger.cloud (free for single users) trace we could take a look at, that's also super useful.
Overview
This PR introduces a new --fail-on-cache-miss flag for the dagger command that allows users to enforce cache-only execution mode.
Motivation for the --fail-on-cache-miss Argument
The --fail-on-cache-miss flag serves several important purposes:
- Cache Validation and Testing: During development and CI/CD pipelines, developers need to verify that their builds are being properly cached and served from cache. This flag enables strict cache validation by failing the ex...
I am not in a hurry. First I want to solve the issue in general before I dive into solving the particular cache-miss.
With the help of some smart tool, I create a PR which adds a new flag: --fail-on-cache-miss. My goal is to execute dagger call always twice. The second time with --fail-on-cache-miss. For my use case the overhead of the second call is fine, as long as it terminates immediatly when the second call terminate on cache-miss.
PR:
[feat: add --fail-on-cache-miss fla...
Thx for taking the time to send a PR for this. Getting something like this merged is a bit more complex than just looking at the call spans from the telemetry. Dagger has different caching layers and sometimes even when a specific step is cached, it might not appear as so in the telemetry because of different reasons
We're constantly improving this so caching is more robust and observability more consistent. As mentioned above, my advise to proceed here would be to share some traces from...
Thx for taking the time to send a PR for this. Getting something like this merged is a bit more complex than just looking at the call spans from the telemetry. Dagger has different caching layers and sometimes even when a specific step is cached, it might not appear as so in the telemetry because of different reasons
We're constantly improving this so caching is more robust and observability more consistent. As mentioned above, my advise to proceed here would be to share some traces ...
Closes #13040
Summary
Bumps the four replace directives (and matching require lines) in both go.mod and sdk/go/go.mod for the OpenTelemetry log packages from v0.16.0/v0.17.0 to v0.19.0:
go.opentelemetry.io/otel/loggo.opentelemetry.io/otel/sdk/loggo.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpgo.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc
Why
- CVE-2026-39882 ([GHSA-w8rr-5gcm-pp58](https://github.com/advisories/GHSA-w8rr-5gcm...
FAQ: How do RTC bounty payments work?
#723
@Scottcjn
Scottcjn
on Mar 7 ยท 14 comments
Return to top
Scottcjn
on Mar 7
Maintainer
Common questions about the RustChain bounty payment process:
Q: How ...
Hey everyone! I just created a high-speed asynchronous compute core in C++ and Python. Check it out, running tests now! The GitHub view counter is live: https://github.com/nlozkina19-crypto/vector-zero-compute
Thx for taking the time to send a PR for this. Getting something like this merged is a bit more complex than just looking at the call spans from the telemetry. Dagger has different caching layers and sometimes even when a specific step is cached, it might not appear as so in the telemetry because of different reasons
Is that an issue of Dagger or buildKit LLB, that --fail-on-cache-miss can't be easily implemented?
Bumps the sdk-java group with 9 updates in the /sdk/java directory:
| Package | From | To |
|---|---|---|
| com.fasterxml.jackson:jackson-bom | 2.21.2 |
2.21.3 |
| io.smallrye:smallrye-graphql-client-api | 2.18.0 |
2.18.1 |
| io.smallrye:smallrye-graphql-client-implementation-vertx | 2.18.0 |
2.18.1 |
| io.vertx:vertx-web-client | 4.5.26 |
4.5.27 |
| [com.palantir.javapoet:javapoet](http... |
Bumps the sdk-typescript group in /sdk/typescript with 10 updates:
| Package | From | To |
|---|---|---|
| @opentelemetry/exporter-trace-otlp-http | 0.217.0 |
0.218.0 |
| @opentelemetry/sdk-node | 0.217.0 |
0.218.0 |
| @opentelemetry/semantic-conventions | 1.40.0 |
1.41.1 |
| [@types/node](https://github.com/Defini... |
What is the issue?
When a consumer container has any WithServiceBinding(alias, svc) and is then evaluated with WithExec(...).Stdout(ctx) (or .Sync(ctx)), the exec fails immediately at host-alias setup time with:
lookup for hosts file: lookup on 10.87.0.1:53: no such host
lookup ..dagger.local on 10.87.0.1:53: no such host
This happens before the consumer's command runs at all โ the consumer's command can be a harmless echo, never referencing the bound alias, and i...
Disclaimer: SORRY THIS IS AI SLOPPED. I however think, the analysis makes sense and the suggested fix is reasonable.
Summary
In the TypeScript SDK, marking a primitive boolean argument or class field as optional (either with ? or with a default value) breaks module introspection. The introspector tries to resolve "boolean | undefined" as a type reference and fails:
IntrospectionError: could not resolve type reference for boolean | undefined.
at DaggerModule.resolveReferenc...
I wouldn't say it's an issue. It's just that the executor solver and the caching layer has been changed a lot recently and it wasn't so straightforward to implement a way to perform this. Now that Dagger doesn't rely on buildkit anymore, it shouldn't be as complicated.
What this fixes
The Python SDK's static module analyzer derives a module's API from its source code, without importing it. When a module is split across files, it only followed relative imports (from .params import Foo) to find names defined in sibling files.
Modules that import their own siblings absolutely (from my_pkg.params import Foo) โ a common style โ left those names unresolved. A type alias imported that way collapsed to an unresolved custom type instead of its real ...
Summary
Lock the TypeScript SDK Yarn cache volume so concurrent lint checks do not mutate Yarn v1 global cache at the same time.
Root Cause
ci:bootstrap can run typescript-sdk:lint-typescript and typescript-sdk:lint-docs-snippets concurrently. Both construct the same Node dev container and run yarn --cwd sdk/typescript install against the same Yarn cache volume. With the default SHARED cache mode, those installs can overlap and leave partial Yarn v1 cache entries, producing...
Testing only
This draft PR intentionally makes a tiny docs-only change so we can trigger CI and reproduce/cache-capture an intermittent Namespace outer-engine shutdown issue.
We are specifically investigating errors while the Namespace-backed outer engine writes the dagql cache persistence snapshot to disk during engine shutdown. The expected signal is in the outer engine logs near shutdown, around dagql cache close / persistence / clean-shutdown metadata handling.
This is not intended ...
The Java SDK runtime's moduleRuntime function runs mvn versions:set against the SDK parent POM. This command only modifies version strings in the POM XML, but Maven still resolves the full dependency tree (including BOM imports like jackson-bom and junit-bom and others from Maven Central).
The problem is that Maven Central started rate limiting (HTTP 429) and as a result, module loading has become flaky, sometimes completely failing. This seems to have been more common (complete failures o...
Bumps protobufjs from 7.5.4 to 7.6.0.
Release notes
Sourced from protobufjs's releases.
protobufjs: v7.6.0
7.6.0 (2026-05-18)
Features
Support BigInt conversions (7.x) (#2258) (f769242)
protobufjs: v7.5.9
7.5.9 (2026-05-17)
Bug Fixes
Backport bundler-safe optional module lookups (#2254) (0853a62)
protobufjs: v7.5.8
7.5.8 (2026-05-12)
Bug Fixes
Backport parser hardening to 7.x (#2245) (54b593f)
protobufjs: v7.5.7
7.5.7 (2026-05-09)
Bug Fixes...
If a toolchain function is called from a module function, it should forward any applicable toolchain customizations.
Treat schemeless .env values for Secret user defaults as plaintext secrets instead of address-style secret provider URIs.
This routes those values through setSecret so they resolve as Dagger Secrets without exposing the raw value in the user default console output. URI-style values such as env:// and op:// keep the existing provider behavior.
Adds regression coverage for a plaintext Secret default to ensure the displayed default is scrubbed while the secret still resolves correctly.
What changed
This fixes persisted container lazy hit reloads for Container.withNewFile.
withNewFile builds a File and stores the container using the same ContainerWithFileLazy persisted payload shape as withFile, but the persisted lazy decoder only accepted the withFile GraphQL field. After an engine restart, loading an unevaluated persisted withNewFile hit failed with unsupported field "withNewFile".
The decoder now accepts both withFile and withNewFile for that pay...
Summary
Remove stale wording from the cache-expert skill that claimed the reference docs were very old / unavailable. Keep the existing pointer to references/debugging.md for test-running and debugging workflow.
Validation
git diff --check upstream/main...HEAD- confirmed stale wording is absent from
skills/cache-expert/SKILL.md
Summary
This PR adds a focused integration repro for a persisted source-backed cache volume mount failure, then fixes the persisted cache volume decode path.
After a persisted engine restart, CacheVolume reopened its mutable snapshot by snapshot ID without restoring the exec.cachemount record type that fresh cache-volume initialization sets. That record type is required for mutable cache mounts to use the existing shareable mount wrapper. Without it, concurrent execs mounting the sam...
Summary
Draft PR for CI-only testing. The diff is intentionally inert; it exists only to trigger CI so we can investigate intermittent disk-full failures on Namespace-backed outer engines.
The suspected failure mode is not caused by this README marker. We are trying to observe jobs where the outer engine restarts from previously used disk state, then collect failing traces, Namespace instance details, outer-engine logs, pruning logs, and engine metrics around disk usage.
Validation
...
Great, can you give me some hints how to implement a way how to debug cache misses automatically?
Current idea: run dagger twice. The second call should fail if there is a cache miss. But if you have better ideas how to implement it, then please tell me!
Allow to install Wolfi packages in the playground
$ dagger call engine-dev playground --extra-packages openssh
What is the issue?
We're seeing secrets print to the TUI console (withMountedSecret step) since 0.20.8 on main. This seems to be related to dagql caching introduced in PR #11856.
Eg,
9ย ย : ReproCacheIssueDagger.test: String!
10ย : โ Container.from(address: "alpine:latest"): Container!
10ย : โ Container.from DONE [0.1s]
11ย : โ Container.withMountedSecret(
11ย : โ โ path: "/123"
11ย : โ โ source: setSecret(name: "toast", plaintext: "<>"): Secret!
11ย : โ ): Container!
11ย : โ Contain...
Summary
- use Dang binding origins to distinguish local types from imported Dagger core types
- avoid exporting imported bindings as Dang module definitions
- add integration coverage for local
Container/Directoryshadowing and explicitDagger.Container - bump Dang to the version containing the language-side shadowing fixes
Tests
dagger call --progress=dots engine-dev test --pkg ./core/sdk/- `dagger call --progress=dots engine-dev test --run 'TestDang/TestCoreTypeShadowing...
ResultCall.callPB / resultCallArgPB encodes the call protobuf used for the
DagCall telemetry span attribute (core/telemetry.go). It emitted arg values
verbatim and ignored ResultCallArg.IsSensitive, so sensitive arguments such
as setSecret(plaintext:) were sent to telemetry and rendered unredacted in
the CLI --progress output (e.g. as the embedded source: of withMountedSecret).
This regressed in #11856 ("migrate all caching to dagql"): the old call.ID
path redacts sensitive args via r...
Rewrite the fail-multi viztest helper so the two failing child spans are created, executed, and ended sequentially instead of through nested function arguments passed to errors.Join.
The test is meant to verify how multiple failed child spans are rendered, not to depend on evaluation or span-end ordering. Making the two failing effects explicitly ordered keeps the golden output stable while preserving the same joined-error behavior.
Summary
- harden
TestEnvFile/TestSecretFileoutput parsing so combined stderr diagnostics cannot be decoded as scalar output - add a stdout-only host Dagger helper and use it for socket ID scalar output passed between calls
Details
The flaky env-file test captured combined CLI output and decoded the last raw line as base64. The CLI can still emit user-default diagnostics to stderr despite -s, and combined stdout/stderr can leave those diagnostics after a trailing newline or interle...
Summary
- pin
TestSecretProvider/TestVaultTTLtohashicorp/vault:1.18 - reuse that pinned image for the Vault service, copied CLI binary, and nested module Vault clients
Details
TestSecretProvider/TestVaultTTL was pulling the untagged hashicorp/vault image in several places. That tag now resolves to Vault 2.0.1, created on 2026-05-19. In this environment, the new image fails before any Dagger-specific secret-provider behavior is exercised:
sh: vault: Operation not perm...
We've been seeing issues with yarl 1.24.1:
https://dagger.cloud/dagger/traces/033d4d3ec42819ba3e721733671f69d8?listen=aeec109729c3c6ad&listen=ff30275ebbf1b697&test=TestPython/TestVersion/.python-version_takes_precedence&viewMode=tests#aeec109729c3c6ad:L131
This commit is a stopgap
Summary
- The generator scale-out path returned a
ChangesetwhoseDirectoryfields reference objects that only exist on the remote engine, so the local engine cannot materialize the generated files โ it could never work in practice. - Drop
tryRunGeneratorScaleOutand the scale-out branch inRunGenerator;dagger generatealways runs locally. - Scale-out is preserved for
dagger checkand the generate-as-check flow, where only a boolean/error result crosses engines.
Test plan...
This draft PR exists to reproduce the intermittent unknown command failure in TestModule/TestFunctionCacheControl under CI.
It intentionally contains no tree changes; the branch has an empty signed-off commit so the normal CI checks run against the current code.
Plan:
- Watch
test-split:test-modulesfor this PR. - Rerun that Dagger Cloud check up to 30 times.
- Stop if a failure contains the unexpected
unknown commandsignature.
Summary
This is a no-diff draft PR for validating reused cache mounts in CI.
The branch contains only an empty commit so it can produce PR CI runs. After creation, this PR number can be shared so the CI info feature can be selectively enabled for this PR, letting us observe and debug whether cache mounts on CI runners are reused across runs.
Testing
Not run locally; this PR is intended to exercise CI.
Summary
- treat SDK-only
dagger.jsonas legacy compat state for install mutation tests - document compat detection expectations for SDK-only root and child configs
- extend migration coverage so SDK-only config is migrated before install mutates workspace config
Tests
go test ./core/integration -run '^$'- `dagger --progress=logs call engine-dev test --pkg=./core/integration --run='^TestWorkspaceModules/TestWorkspaceModuleInstall/install_initializes_empty_workspace$|^TestWorkspac...
The Python SDK property-based differential test generated a module where a Dagger field was named bool:
@dagger.object_type
class Foo:
bool: str = dagger.field(default="")
@dagger.function
def a(self) -> bool:
...
That source is legal Python syntax, but it is outside the supported Dagger schema grammar.
That source is outside the supported Dagger grammar: generated Dagger members should not shadow the finite set of ...
Summary
- add versions plugin flags to skip dependency and plugin processing when updating the Java SDK parent POM version
- relies on the existing mavenCommand wrapper for --no-transfer-progress
Fixes #13174
Test
- go test ./... (in sdk/java/runtime)
Summary
- fix workspace docs terminology around module settings and Cloud Checks
- expand migration, first-run recovery, CI install, security, configuration, and module author guidance
- pin script install examples and remove stale workspace-era command references
Validation
dagger check -ldagger check markdown-lint:lint docs:references python-sdk:lint-docs-snippets typescript-sdk:lint-docs-snippets
The selected docs checks exited 0. Trace: https://dagger.cloud/dagger/traces/...
Summary
- Persist
SearchResult,SearchSubmatch, andDiffStatvalues so cache-persisted list results round-trip after engine restart. - Reject object-like values without explicit persisted-object encoding instead of silently storing them as
scalar_json. - Add dagql codec tests and disk-persistence integration coverage for
Directory.search, search submatches, andChangeset.diffStats.
Validation
- `dagger --progress=plain call go test --pkgs ./dagql --run 'TestPersistedSelfC...
:robot:
The deferred EndWithCause for the lazy-evaluation resume span fired only when the goroutine returned, which was after close(waitCh). Callers awaiting Evaluate could observe completion, flush, and read exported spans before the span was ended, racing the deferred End. Wrap the eval body in an inner function so the defer fires on its return, before close(waitCh). This also ensures the resume span is ended on the operation-lease error path, where it was previously created but never en...
Summary
Restarted engines can import persisted dagql object results lazily because object payload decode needs a dagql server context. Before this change, those lazy imported results retained snapshot-owner links and leases, but local-cache usage accounting ignored them until a later cache hit materialized the object value.
This makes lazy imported snapshot-owner links participate in local-cache usage and max-used prune before payload decode. The snapshot manager now exposes direct snaps...
Summary
- update generated Go module files for
golang.org/x/sys v0.45.0 - refresh generated
go.mod/go.sumfiles across module, SDK runtime, testdata, and toolchain fixtures
Tests
dagger --progress=plain call generatedgit diff --check
- https://github.com/vito/dang/pull/58
- contains some internal refactors
I think it would be good to assume that the go.mod is correct as is, seems like didn't cause many issues (that I'm aware of) in previous Dagger versions, but I could be wrong :)
Summary
This is a test-only change for TestLocalCachePruneReclaimsStoppedServiceSnapshots.
The underlying engine behavior is fine: stopped service snapshots are being pruned. The previous assertion measured global filesystem free space with df, which is inherently noisy under parallel integration tests and could hide the local cache reclaim signal.
This updates the test to run against an isolated nested dev engine and assert via Engine().LocalCache().EntrySet().DiskSpaceBytes(ctx)...
Summary
This PR collects a few related cache pruning and disk-pressure improvements for the engine and the engine-heavy integration checks.
Commits
core: use available disk space for min-free pruning- Fixes the min-free-space pruning decision to use filesystem available bytes instead of raw free bytes, so reserved blocks do not make the engine overestimate space users can actually consume.
- Updates the cache and server GC plumbing/tests to carry available space through the ...
What are you trying to do?
A user building a Python Dagger module creates a non-root user in a container, for example python with uid/gid 1234, and wants every directory mounted into that container to be owned by that user. Today each call has to pass the owner explicitly:
container.with_directory(path, source, owner="1234:1234")
When this logic is shared across modules, callers either need to plumb uid/gid through their own APIs, which adds boilerplate, or resolve ui...
Bumps the engine group with 31 updates in the / directory:
| Package | From | To |
|---|---|---|
| charm.land/lipgloss/v2 | 2.0.2 |
2.0.3 |
| cloud.google.com/go/secretmanager | 1.16.0 |
1.20.0 |
| github.com/1password/onepassword-sdk-go | 0.3.1 |
0.4.0 |
| github.com/99designs/gqlgen | `0.17.... |
Draft for discussion. Stacked on the
workspacebranch.
What this does
Reworks the Go SDK codegen path so that:
- A module can opt out of codegen at runtime. With
codegen.legacyCodegenAtRuntime=falseindagger.json, the Go SDK trusts
the committeddagger.gen.go+internal/daggerfiles and skips the
in-container codegen pass at runtime (it errors with an actionable message
if the committed files are missing). - **
generate-moduleruns once for Go modules, in...
Summary
- add API-backed Dagger Cloud git source inspection/configuration commands
- add repo-first commands for info, link, unlink, transfer, and autocheck status
- project linked repos from enabled git sources, with autocheck as a read-only repo setting
- add
dagger integration add github,dagger repo enable autocheck, and a post-login setup hint when a local GitHub repo needs Cloud setup
Verification
go test ./internal/cloud ./cmd/dagger -run '^$'- `go build -o /tmp/dagge...
Fixes a few migration issues found while dogfooding module repos.
- remove
dagger workspace initand the matching schema field - seed migrated workspace configs without verbose starter comments
- map the
dangruntime togithub.com/dagger/dang-sdk - report skipped SDK installs in migrated configs and migration reports
Tests:
go test ./cmd/dagger -run 'TestRemovedWorkspaceCommands|TestWorkspaceInitCommandRemoved|TestRootHelpShowsWorkspaceCommandGroup|TestWorkspaceFlagPolicy'\n- `go ...
Summary
- make
dagger migrateinclude planned.dagger/lockwrites in the returned changeset instead of staging lock entries during planning - merge refreshed migrated lock data into the correct per-plan workspace lock, so nested migrations write nested
.dagger/lockfiles - add regression coverage for preview/discard not writing lockfiles and nested pinned refs staying out of the root lock
- update the workspace upgrade docs from
configtosettingsterminology
Tests
- `go test...
Bumps the sdk-elixir group with 3 updates in the /sdk/elixir directory: ex_doc, jason and req.
Updates ex_doc from 0.40.1 to 0.40.3
Changelog
Sourced from ex_doc's changelog.
v0.40.3 (2026-05-21)
Enhancements
Add autolinking for Erlang/OTP 29 native records
v0.40.2 (2026-05-08)
Bug fixes
Add rel="nofollow" to external links in HTML output
Use blockquote...
What are you trying to do?
Check if a file exists in ${HOME}/foo
Why is this important to you?
No response
How are you currently working around this?
container.envVariable("HOME")
Summary
- add
Query.workspace(directory:)for Directory-backed synthetic workspaces - support
file,directory, andfindUpon rootfs-backed workspaces without host client metadata - return explicit errors for unsupported synthetic workspace features and
gitignore: true - add Go SDK
dag.Workspace(dir)plus synthetic workspace coverage
Tests
go test ./core/schemago test ./core/integration -run '^$'\n-(cd sdk/go && go test . -run TestWithWorkspace)\n- `go build -o /...
Bumps the sdk-java group with 10 updates in the /sdk/java directory:
| Package | From | To |
|---|---|---|
| com.fasterxml.jackson:jackson-bom | 2.21.2 |
2.21.3 |
| io.smallrye:smallrye-graphql-client-api | 2.18.0 |
2.18.1 |
| io.smallrye:smallrye-graphql-client-implementation-vertx | 2.18.0 |
2.18.1 |
| io.vertx:vertx-web-client | 4.5.26 |
4.5.27 |
| [com.palantir.javapoet:javapoet](htt... |
Bumps the sdk-typescript group in /sdk/typescript with 7 updates:
| Package | From | To |
|---|---|---|
| @grpc/grpc-js | 1.14.3 |
1.14.4 |
| @types/node | 25.8.0 |
25.9.1 |
| @typescript-eslint/eslint-plugin | 8.59.3 |
8.59.4 |
| [@typescript-eslint/parser](https://github.c... |
Bumps the docs group with 12 updates in the /docs directory:
| Package | From | To |
|---|---|---|
| @docusaurus/core | 3.9.2 |
3.10.1 |
| @docusaurus/plugin-content-docs | 3.9.2 |
3.10.1 |
| @docusaurus/preset-classic | 3.9.2 |
... |
Now that we have dagger query, we donโt need to print the contents of the config every time we re-up it.
- split integration tests into small manageable files
- add
SUITE=llb make integrationsupport (suite can becompute,llb,stdlib,cli,examples) - fixed test for
examples/react
Currently in main dagger creates an actual directory called โ$HOMEโ instead of expanding the value of $HOME.
Also changing the location of the default store to follow the xdg spec. This will cause all deployment to be โforgottenโ, better do it now before anyone is using it seriously.
Just got the issue using input dir:
$ dagger input dir source ../../
This is what we store:
"inputs": [
{
"key": "source",
"value": {
"type": "dir",
"dir": {
"path": "../../"
}
}
}
]
It's not limited to input dir -- dagger plan dir has the same issue, and I suspect dagger new --plan-dir too.
To fix this once for all, I suggest we resolve the absolute path directly inside `dagger....
The API was a bit confusing with code mixing the usage of Deployment and
DeploymentState.
This change marks a clear separation:
- Client is the only piece of code manipulating *Deployment
- CLI commands can manipulate DeploymentState objects and pass them to
Client
// Apply a Kubernetes configuration
#Apply: {
// Kubernetes config to deploy
source: string | dagger.#Artifact
...
if (source & string) != _|_ {
...
},
if (source & dagger.#Artifact) != _|_ {
...
}
}
The ifs are not working anymore
/cc @samalba @TomChv
In preparation of support dagger down.
Also matches the dagger up command.
Previously dagger eval always returned the cue output. This makes sense for cue debugging, but not to get quick information about the current state.
The previous behavior is still available with dagger query -f cue. All cue flags have been preserved.
Running dagger up produces output values, but they are not stored in the deployment state by the client. As a result, dagger query will not include outputs.
- Add support for multiple deployments per path in the Store
- Add a bunch of tests
- Change the Lookup deployment API
- Add disambiguation in the CLI commands
Fixes #231
Signed-off-by: Andrea Luzzardi
Implemented tests for:
- plan dir / git
- input dir / git
- new --plan-git
Restored JSON output for up
Fixed absolute path for dir path
Signed-off-by: Frederick F. Kautz IV
Fixes a build break in test.
--- FAIL: TestInputDir (0.00s)
input_test.go:20:
Error Trace: input_test.go:20
Error: "map[/Users/fkautz/dagger/dagger:/Users/fkautz/dagger/dagger /tmp/source:/tmp/source]" does not contain "."
Test: TestInputDir
currently, integration tests run first then unit tests.
This patch should swap the order.
Bumps github.com/stretchr/testify from 1.5.1 to 1.7.0.
Release notes
Sourced from github.com/stretchr/testify's releases.
Minor improvements and bug fixes
Minor feature improvements and bug fixes
Fixes breaking change with HTTPBodyContains
A breaking change was accidentally released in v1.6.0 which breaks the API for the HTTPBodyContains and HTTPBodyNotContains, this release reverts that change.
v1.6.0
Latest release of testify. This includes many fixe...
It appears multiple deployments can have the same name, which is confusing.
$ dagger up
7:50PM FTL system | multiple deployments match the current directory, select one with `--deployment` deploymentPath=/home/shykes/dagger deployments=[
"dagger",
"dagger-dev",
"dagger-dev"
]
$ ls ~/.dagger/store/
dagger dagger-dev react
Sometimes one wants to embed a Dagger deployment plan directly in an application repository, in the same way one might embed a Dockerfile or docker-compose.yaml.
This is currently possible but not seamless. It requires several redundant steps:
- The deployment plan declares an input of type artifact. For example
source: dagger.#Artifact - Each new deployment must explicitly load the current directory into that input:
dagger input dir source .
With embedding, that second step w...
When running make integration, I saw a message error sops: command not found, but it did not interrupt test execution. It was not clear if it should be treated as an error or a warning.
A deployment with multiple times the same path would appear repeated in
LookupDeploymentByPath.
This change indexes paths as a map rather than a list, thus avoiding the
issue altogether.
Fixes #276
This feels more natural, let me know what you think.
This solves 2 problems:
-
No more worrying about the technical distinction between llb, buildkit API etc. This was listed as a negative for the
dagger.io/llbname, but the pros seemed to outweigh the cons. This could remove the downside without adding new ones. -
It opens the possibility of adding base operations that donโt map to llb/buildkit at all. Thereโs already
#DockerBuildwhich is a higher-level composition of buildki...
Ignore metadata from IntelliJ based IDEs such as Goland
@snow kayak #272 is rebased
Currently the dagger stdlib is embeded in the dagger binary. This is a short-term solution, meant to save time in the early phases of development. But the plan is to change how stdlib is distributed before launch.
The issue is a placeholder for planning purposes.
Various improvements to developer experience, from writing real-world configurations.
When setting an input value or artifact, for example with dagger input text, the valus is not checked for validity. If I provide an invalid input, this will only be apparent when running dagger up, at which point the context of the original change may be lost.
Example using the โhello-worldโ example:
$ cd examples/hello-world
$ dagger new
# NOTE: the key โmessageโ will conflict with the input
$ dagger input text message bla
$ dagger up
12:21AM FTL system | failed to up dep...
Massive hacks and shortcuts, DO NOT MERGE
As soon as a Dagger configuration is moderately complex, calling dagger up will hang after computing everything, and consume high levels of CPU while not printing anything. Eventually it will return. As the configuration grows in size and complexity, so does the duration of the hang.
This can be reproduced with the dev.cue configuration at the root of the dagger repository; or with examples/react as well.
Fixes #263
The current definit of dagger.#Artifact matches anything (_), then it's not possible to differentiate a #Artifact from a string (if (source & string) & _|_ is always true).
This workaround splits both types in 2 separate keys to restore the lib and the example.
In certain configurations, the subdir operation is not applied - the result is as if the operation was not there.
Example:
package main
import (
"dagger.io/dagger"
"dagger.io/dagger/op"
)
#Container: {
command: string
input: dagger.#Artifact | *null
#up: [
op.#FetchContainer & {
ref: "index.docker.io/alpine"
},
op.#Exec & {
arg...
When developing for Dagger, this is a very common pattern:
- Import a package
- Wonder what definitions are available in the package
- Wonder what fields are available in a given definition
It would be useful to have an easy way to look up that information, either from the command line or in the browser, or both.
This would be the equivalent of go doc or https://golang.org/doc
As mentioned by @kelseyhightower :) A very successful feature of Kubernetes is the ability to extend it with custom policies out-of-band from actual configurations. So an admin can inject a policy once, and everything will have to run through it.
There are 2 parallel tracks available to implement this:
- Cue itself is well suited for policy injection. I defer to @mpvl @myitcv @verdverm for details and examples.
- Runtime hooks. This is what Kubernetes does. It could be webhooks; or i...
Summary
TLDR: the developer experience would be nicer if #up supported a declarative form in addition to an imperative form.
- Imperative form: array of low-level operations:
#up: [...op.#Op] - Declarative form: any object with its own nested array:
#up: { #up: [...op.#Op], _ }
This is the same pattern as op.#Load.from for example.
Context
Currently any value can be assigned a processing pipeline via the special definition #up. The value of this field must be ...
dagger.io/docker: support for computed values (export)dagger.io/netlify: simplify by usingdagger.io/dockerpackage
Solve #237 and add an example to run a configuration that deploy a pod in a KinD kubernetes cluster.
- Support Entitlements
- Run configuration in local network
- Unsafe flags to prevent users from unwanted behavior
- Run buildkit container in
--net=host modeand--allow-insecure-entitlement network.host
It's currently not possible to provide Registry credentials for both ops "push-container" and "fetch-container".
Currently it's only possible to do "fetch-git" on publis repositories.
ref is an arg of the op.#PushContainer, but there is currently no way to get the full resolved ref of the pushed image (that includes the sha256).
Here is something that should be possible to do:
source: dagger.#Artifact
pushTarget: "samalba/myimage:latest"
ref: {
string
#up: [
op.#DockerBuild & {
context: source
},
op.Export & {
format: "string"
source: op.#PushContainer & {
ref: pushTarget
}
}
]
}
dagger up fails with โnot executableโ on certain configurations that should be valid.
The error is triggered by the following conditions:
op.#Load- The
fromfield has a default value, which is used in evaluation - The default value is in the โstructโ form
from: { #up: [...] }
Here is a repro configuration that shows 4 different load operations. All 4 should be valid, but one fails.
package main
import (
"dagger.io/dagger/op"
"dagger.io/dag...
Allows a file to be injected into the cue graph.
dagger compute --input-file a.b=foo.txt dir/
Bumps go.mozilla.org/sops/v3 from 3.7.0 to 3.7.1.
Release notes
Sourced from go.mozilla.org/sops/v3's releases.
v3.7.1
Commits
[9cc95d4]: Add release workflow (AJ Bahnken) #843
[dfc7af2]: swap to fork of action-automatic-releases (AJ Bahnken) #843
[fdf4517]: Trim space from age keys (Johan Fleury) #846
[1504dbc]: Run CI tests against master as well (AJ Bahnken) #848
[8a2fbc0]: Initial patch for advisory (AJ Bahnken) #852
[e5bf171]: go.sum fix (AJ Bahnken)...
Setup
In order to reach local kubernetes cluster. I've added the flag --net=host on buildkit container.
:warning: Actually, buildkit has an isolation problem, when we use
--net=hostto run buildkit, it doesn't isolate commands anymore. This bug make this PR possible.
Kubernetes Fix
I also fixed the kubernetes library, there was a problem with source inline :
if sourceInline != _|_ {
op.#WriteFile & {
dest: "/source"
content: source // Must ...
The opposite of dagger up, to bring down a deployment.
#198 added some minimal secrets support in dagger compute, but that's not nearly enough and broken.
Considerations:
dagger input secretmust provide a secure way to supply and store secrets- secrets must be encrypted at rest (e.g. in the state)
- [Thinking out loud] Just in time Deciphering: Secrets must be encrypted in the Cue tree, decrypted by BuildKit when accessed. This is a safety net to avoid leaking secrets when representing the Cue tree. e.g.
mySecret: 42--> `error: i...
This examples adds some useful packages to stdlib/aws and aims to show how to deploy a complete app:
- backend: container deployed to ECS
- frontend: js app deployed to Netlify
We need a way to know the current deployment when working from a local directory. For instance a small * or any visual indication when doing dagger list. Otherwise, there is no way to know a given directory is mapped to a deployment.
It would also be nice to know which path is mapped to which deployment.
Example:
$ dagger list
myapp (~/work/myapp)
test (/tmp/thisisatest)
$ cd ~/work/myapp
$ dagger list
myapp (~/work/myapp) - current
test (/tmp/thisisatest)
Signed-off-by: Frederick F. Kautz IV
Bumps cuelang.org/go from 0.3.0 to 0.3.1.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebas...
Bumps github.com/containerd/console from 1.0.1 to 1.0.2.
Release notes
Sourced from github.com/containerd/console's releases.
v1.0.2
Notable changes:
Fix support for big endian architectures such as s390x (containerd/console#51 , thanks to @โkolyshkin)
Support FreeBSD (containerd/console#50 , thanks to @โsamuelkarp)
Commits
2f1e3d2 Merge pull request #52 from zhsj/fix-import
63603f4 Add missing import in freebsd nocgo stub
413d286 Merge pull re...
It's important to be able to to disable the cache on-demand, one-time.
dagger up --no-cache
- Both FetchContainer and DockerBuild read the image metadata and
convert to LLB (e.g.ENV foo barin Dockerfile shows up in
op.#Exec) - Image metadata is "sticky" between Pipelines (e.g.
op.#Loadwill
re-use the same metadata) - Image metadata is injected back to #PushContainer, so that
DockerBuild+PushContainer and FetchContainer+PushContainer do not lose
any metadata. - Tests for all the above
Fixes #142
While building the jamstack example, I noticed a cue eval 6x slower (from 0.2s to 1.2 on a MBP with M1), which makes dagger up almost unusable.
After investigating, it looks like the culprit is this for loop: https://github.com/dagger/dagger/blob/2d778ea2a7c4b173c674efbec851b6541b9903e3/stdlib/docker/container.cue#L114
If I comment it, it goes back to normal. As a workaround, I'll move react lib back to the initial implementation.
As a user, I have two repos, each with a folder dagger. I am unable to dagger new both of them.
I actually cloned dagger twice (2nd to ensure a fresh copy) and I am unable to new within the 2nd / fresh one
I have dagger new'd a directory. Dagger now reports no deployment matches.
Bumps cuelang.org/go from 0.3.1 to 0.3.2.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebas...
This implements a simple static page deployed to an S3 bucket
My current build system often fails when a remote network / service behaves badly. 502's, host hangs up, services being down... Today, I have to add custom logic at all of these points as we discover them, or we move to replicating these remotes internally.
git clonego mod downloaddocker pullapt update- the list goes on and on...
It would be great if there was retry / expo decay options across the building blocks for these situations. Detecting which are network ...
Converted the CLI tests to using bats to get started
Signed-off-by: Andrea Luzzardi
Signed-off-by: Andrea Luzzardi
When deploying A and B, if A succeeds then B fails the rollout is left as is. This can result in a broken deployment since we now have a new version of A with an old version of B.
Ideally, if a version N of a plan fails, dagger should be re-apply N-1 to bring back A and B to the same version.
/cc @samalba
- add support for
dagger input jsonanddagger input yaml - add file input support for text/yaml/json.
dagger input {text,yaml,json} -f - add stdin support as well:
echo foo | dagger input {text,yaml,json} -f -
Change
The old skip condition was cheaking if the name kind exist.
That was not working because every kubeconfig has the field kind: Config.
Example of a common AWS kubeconfig :
apiVersion: v1
clusters:
- cluster:
- context:
current-context: ...
kind: Config # the field
preferences: {}
users:
I changed the condition to match the field user: kind-kind, which is the default user for all kind kubeconfig.
I also check that cluster is runnin...
This is a proposal that fixes #303
The change allows certain ops to export meta data to the image fs, under /dagger/. We could reuse the same mechanism for fetch-git to write things like /dagger/commit or /dagger/remote, etc...
I was initially not ok to alter the pushed image, but after testing it, I think it's fine as the pushed image does not contain the meta-data. It's only available within the #up pipeline.
Also, I like the fact that it does not break the current `op.#...
Add the kustomize command in the package.
I used the "real" kustomize binary because kubectl -k is outdated.
Something strange I met during the development is that container can mount volumes from nested compute...
It create some errors because the kustomization.yaml was interpreted as yaml by the kubectl apply command.
To dodge this problem, I remove the file after the compute.
I trie...
Bumps github.com/uber/jaeger-client-go from 2.25.0+incompatible to 2.26.0+incompatible.
Release notes
Sourced from github.com/uber/jaeger-client-go's releases.
Release 2.26.0
Delete a baggage item when value is blank (#562) -- evan.kim
Trim baggage key when parsing (#566) -- sicong.huang
feat: extend configuration to support custom randomNumber func (#555) -- NemoO_o
Support JAEGER_TRACEID_128BIT env var (#547) -- Yuri Shkuro
Additional context p...
Add some guidelines on the suggested GitHub workflow, commit messages
and so on.
Signed-off-by: Andrea Luzzardi
Implements the dagger input list command to scan a plan for inputs
walks a plan Value looking for potential inputs
- non-concrete values or values with defaults
- include any @dagger(input) found
- exclude @dagger(computed) and #up
- exclude values which have references
This removes the untyped from: _ fields in operations that reference other operations. Specifically:
- Load from ...
- Copy from ...
- Exec with mounts from ...
- Dockerbuild with context from ...
By embedding the smallest possible #Runnable definition directly in the dagger.io/dagger/op package, we avoid circular imports. It makes sense for op to have its own internal definition of what is runnable. Higher-level packages may use it, but donโt have to.
TL;DR
- Add tests on AWS eks
- Fix kubeconfig type from
aws/ekspackage
Tests made
- Verify that we can execute a
kubectl get pods -Aon the aws cluster
It's not necessary to do more, we already wrote tests for kubernetes.
Fix
There is a problem with the dagger.#Secret type. It wasn't possible to generate the kubeconfig.
Click to see an example
Here is a simple example :
import (
"dagger.io/aws"
"dagger.io/dagger/op"
)
#Def: {
...
- Fixes an error in RDS when the database fails to be created
- Improves error handling with op.#WriteFile when the content key is not set
TL;DR
- Add tests on AWS s3
Tests made
- Verify that we can upload a file on s3 bucket
How to reproduce:
cd ./examples/jamstack
# configure app_config.cue
dagger new
dagger up
dagger query # takes 7 seconds on a MBP with M1 proc
Likely to be introduced with this commit: https://github.com/dagger/dagger/commit/3e2b46bf3aaefdfe552bb8d42513293a286d0481 (will confirm asap)
In previous prototypes we had generated reference documentation directly from the cue code. This allowed the documentation to be up-to-date with little effort. I suggest that we prioritize this before launch, because encountering out-of-date docs results in immense user frustration.
Follow-up from #343
Reverts dagger/dagger#347
Fixes #353
After reverting the commit, my dagger query goes from 7.5 seconds to 0.9 second (MBP with M1)
Each dagger environment (aka deployment) has a simple name that is only unique locally. There should also be a globally unique ID for each environment. This ID should not depend on the simple name.
If the ID is derived from a unique per-environment keypair (eg. ID = hash of the pubkey), then we can use that keypair as a foundation for very promising features: cryptographic access control; content trust via TUF; etc.
Problem
Currently input artifacts (local directories, git repositories, docker repositories) are identified only by how to access them in their respective address spaces. Local dir is identified by its local path; git repo by its git remote and ref; etc.
This poses a problem specifically for local directories, because it is common for a directory to be moved to a different path. Currently dagger cannot differentiate between a directory changing path, and a directory being destroyed. I...
This is a thread to discuss Daggerโs terminology (the choice of specific words to designate specific things), our experience with it, suggestions for changing it, etc.
Daggerโs terminology is still changing rapidly. Product terminology is part of product design, and the only way to find the right design is to try lots of bad ones. So, weโve been trying lots of words to describe the important parts of Dagger. As we use the product, we get a better idea of which words work and which donโt.
...
After a few weeks of feedback and hands-on practice, I am leaning towards the following changes:
- Deployment: a continuously running workflow delivering a specific application in a specific way.
Rename to Environment.
- Deployment Plan: the source code of the deployment, written in Cue.
Rename to Workflow.
Rationale for environment:
- It comes up all the time in natural conversation. No matter what the terminology says, I end up using the word โenvironmentโ to e...
I have observed 2 distinct patterns for how to use Dagger in combination with an application repository:
-
Embedded. Sometimes one wants to embed a Dagger workflow directly in an application repository, in the same way one might embed a Dockerfile or docker-compose.yaml.
-
Standalone. Sometimes one wants to develop a Dagger workflow separately, from any application repository.
How Dagger supports these two patterns, and how to make sure each user follows the correct pattern for...
For #2, what if the encrypted secret is an X509 cert or something else not json compatible? We could base64 encode it before passing it in, though that seems like an extra step that may not be necessary when cue supports bytes naturally.
SGTM
Do you want to go ahead with the renaming or still need some time to see how it feels?
Demonstrates edge cases for auto-detection
Problem
Actually, it's not possible to remove an input from a deployment and that's really frustrating...
Imagine an user who do a typo when adding an input ? He will not delete the deployment and restart all from the beggining.
Proposal
Add a flag unset on dagger input which take the cue path as argument and remove the input
Example
$ dagger input text name "John Doe"
updated deployment...
$ dagger input list
Saved Inputs:
name: {text 0xc00061...
- Rename โdeploymentโ to โenvironmentโ
- Rename โdeployment planโ to โworkflowโ
- Light documentation upgrades along the way
POC: For discussions only
This is a fully functional POC of dagger down working with examples/react
How it works
There's many ways to do this, I went with one that seemed to work, very open to suggestions.
Here's how this approach works (look at stdlib/netlify, stdlib/react and examples/react to see it in action):
- We're back with
#compute - There's a new
#hooksthat contains bothupanddown - On
dagger up, every#computeis executed, along wi...
TL;DR
Fix #291
Problem
Subdir behavior was not the wanted one.
Actually, he just copy the content of the given folder in /.
Fix
Copy the content of the folder from Scratch instead of current state.
Tests
I added tests to match the wanted behavior of subdir.
TL;DR
- Rename all tests with prefix
Test - Trigger some test which works now
- Add more test to
op.#Subdir - Rename some package
testing
Tests Convention
All tests in operations where outdated and doesn't respect the naming convention. I walk through all ops tests to fix name and sometimes tests.
Changes
op.#Load
- Rename Test
- Trigger
op.#Mount
- Rename test
- Add some asserts
- Update tests : #46 is fixed
op.#Copy
- Rename test
...
Implement Docker login as a new op.
Fixes #301
This changes the way inputs are passed to the jamstack example. Using a yaml file instead of cue code.
dagger up should return an error right away if there are missing (mandatory) inputs.
FIXME: missing context. Starting with a placeholder for future discussion.
Buildkit supports fetching artifacts from a tar archive over http(s). Dagger should expose this in its pipeline API.
Implement a searchable plan catalog, as discussed in #378.
Implement a searchable catalog of reusable components, as discussed in #378.
[dagger/dagger] Pull request opened: #381 \[URGENT\] Fix broken store layout from terminology change
This fixes a break in the store layout introduced by the terminology change merged earlier today.
Bumps google.golang.org/grpc from 1.29.1 to 1.37.0.
Release notes
Sourced from google.golang.org/grpc's releases.
Release 1.37.0
API Changes
balancer: Add UpdateAddresses() to balancer.ClientConn interface (#4215)
NOTICE: balancer.SubConn.UpdateAddresses() is now deprecated and will be REMOVED in gRPC-Go 1.39
Behavior Changes
balancer/base: keep address attributes for pickers (#4253)
Special Thanks: @โlongXboy
New Features
xds: add support for c...
Fix a bug where you are locked with inconsistency between your intention and the reality.
How to reproduce
cd example/simple-s3dagger up- Modify the html body
dagger up- Rollback the body to a previously deployed version (Ctrl+Z)
dagger up- Nothing is sent to S3
We cannot reuse the buildkit caching logic, maybe a more efficient way to achieve a proper caching could be to compare a checksum of local object and remote one.
TL;DR
Add tests to AWS ecr stdlib package.
Tests
- Build and push an image
- Pull and verify image
- Pull image in dockerfile and verify image
TL;DR
Resolve #364
Changes
Features
It's now possible to remove input from environement through dagger input unset .
@aluzzardi has already develop the feature in dagger. I've only linked it to CLI.
Tests
All input tests now unset the input after run.
It was the simplest and more effective way to test all types of input with unset command.
This is a proof-of-concept of a Dagger implementation optimized for integration with an existing git flow. In this implementation, all Dagger state is embedded in a git repository.
Usage example:
$ alias dgr=$(pwd)/cmd/dgr
$ dgr list
dev
hello-world
release
$ dgr dir dev source .
$ dgr up dev
$ dgr query dev
$ dgr set hello-world name โโBobโโ
$ dgr query hello-world msg
Stdlib uses cue attributes to mark special types:
- Artifact
- Secret
- Computed value
- Relay (primary node in the DAG)
Attributes are not yet used in the runtime, but can already be used in user configurations in a forward-compatible way.
- Remove .daggerignore
- Polish stdlib as needed
- New stdlib package:
io - ...
Bumps github.com/moby/buildkit from 0.8.2 to 0.8.3.
Release notes
Sourced from github.com/moby/buildkit's releases.
v0.8.3
https://hub.docker.com/r/moby/buildkit
Notable changes
Update containerd with fixes to rootless overlay on kernel 5.11 and push panic #2014
Add retry on 5xx push errors moby/buildkit#2043
Include basename in content checksum for wildcards #2018
Fix missing mounts in execOp cache map #2076
Commits
81c2cbd Merge pull reque...
Currently Dagger does not support inlining pipelines. For example:
This works:
import (
โdagger.io/dagger/opโ
)
// Intermediary pipeline cannot be inlined: it must be visible in a field
sayHello: {
message: {
string
#up: [
op.#FetchContainer & { ref: โalpineโ },
op.#Exec & { args: [โshโ, โ-cโ, โecho helloโ > /messageโ],
op.#Export & { source: โ/messageโ, format: โstringโ },
]
}
}
hello: sayHel...
Sometimes my configuration fetches a git or docker repository without specifying a digest, and I want to add the current digest to avoid unexpected changes in the future. But how do I find out the current digest? Dagger does not provide a command to do this. I have to search for specialized tooling and learn how to use it, just to get that ref.
Since git and docker refs and digests are important concepts in Dagger, it makes sense for the dagger command to make it easy to manage digests.
[dagger/dagger] Issue opened: #397 stdlib coding style: use nouns instead of verbs whenever possible
To make the stdlib more consistent, I propose we implement this simple rule:
- Whenever possible, stdlib packages should use nouns (eg.
#Website,#File) instead of verbs (eg.#DeploySite,#WriteFile). When a verb is used, there should be a comment explaining why itโs better.
Dagger logs should clearly output each individual operation as it executes it. This information is partially available today, but mixed with lots of other information and must be inferred. This makes debugging difficult.
As reported by @TomChv, it appears there's a regression with buildkit 0.8.3.
9:35PM FTL system | failed to up environment: buildkit solve: TestECR.creds.secret: 2 errors in empty disjunction:
TestECR.creds.secret: conflicting values "XXX" and "XXX":
/config/cue.mod/pkg/dagger.io/aws/ecr/ecr.cue:17:12
/config/ecr.cue:32:9
TestECR.creds.secret: conflicting values string and bytes (mismatched types string and bytes):
/config/cue.mod/pkg/dagger.io/aws/aws.cue:63:3
/c...
Reverts dagger/dagger#394
See #399
Sometimes a cue error will not be caught when loading the config, but instead will be logged as a warning during execution, under the prefix โmissing inputโ (even though the error is unrelated).
No repro case yet.
- Improve programmer guide
- Small stdlib fixes along the way
Bumps github.com/uber/jaeger-client-go from 2.27.0+incompatible to 2.28.0+incompatible.
Release notes
Sourced from github.com/uber/jaeger-client-go's releases.
Release 2.28.0
HTTPSamplingStrategyFetcher: Use http client with 10 second timeout (#578) -- Joe Elliott
Changelog
Sourced from github.com/uber/jaeger-client-go's changelog.
Changes by Version
2.28.1 (unreleased)
Nothing yet
2.28.0 (2021-04-30)
HTTPSamplingStrategyFetcher: Use http...
Bumps github.com/moby/buildkit from 0.8.2 to 0.8.3.
Release notes
Sourced from github.com/moby/buildkit's releases.
v0.8.3
https://hub.docker.com/r/moby/buildkit
Notable changes
Update containerd with fixes to rootless overlay on kernel 5.11 and push panic #2014
Add retry on 5xx push errors moby/buildkit#2043
Include basename in content checksum for wildcards #2018
Fix missing mounts in execOp cache map #2076
Commits
81c2cbd Merge pull reque...
Sometimes APIs will change in the stdlib. The bigger the stdlib, the more certain this is. Sometimes the API change is a mistake which needs to be caught and rolled back. Sometimes it is intentional, because the package is still young and needs to continue evolving. In both cases, the experience for end users and developers should be good.
Adds:
- install.sh script for easy install
- tweaks to gorealeaser config for excluding commits from changelog
- install doc
We don't do any kind of de-duplication and instead rely on buildkit to avoid executing pipelines multiple times when referred.
This is slightly confusing in how the information is presented.
For instance, given this config:
ActuallyDoStuff: #up: [some code here]
TestReference: ActuallyDoStuff
TestAnotherReference: TestReference
3 pipelines will show up:
6:43PM INF ActuallyDoStuff | computing
6:43PM INF ActuallyDoStuff | completed duration=700ms
6:43PM INF Test...
Bumps cuelang.org/go from 0.4.0-alpha.2 to 0.4.0-beta.1.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself...
TL;DR
- Check each 24h dagger binary version and notice user if an update is available
- Add a
--checkflag todagger versionto verify that dagger is up to date manually. - Update documentation to explain update steps
Changes
Hooks
I've added a PreRun and PostRun hooks to dagger cli in order to check periodically dagger version.
It will compare the dagger binary version with the latest version.
If the u...
TL ; DR
Add tests for docker example
Tests
The actual docker example build an image from a Dockerfile.
It's a bit complex to test so I wrote a file test.cue that use image created by the example.
In the test, I just copy the file test.cue in the example folder and start test.
Currently on 09b614f3ffe18451dcdff8ddae8b37a91f07e54f
dmp@macArena:~/Projects/Go/src/github.com/blocklayerhq/dagger$ ./cmd/dagger/dagger compute --input-dir=../context .
panic: runtime error: index out of range [1] with length 1
goroutine 1 [running]:
dagger.io/go/cmd/dagger/cmd.glob..func2(0x2bd4f60, 0xc0005a0120, 0x1, 0x2)
/Users/dmp/Projects/Go/src/github.com/blocklayerhq/dagger/cmd/dagger/cmd/compute.go:59 +0x25f6
github.com/spf13/cobra.(*Command).execute(0x2bd4f60, 0xc000...
#DockerBuild.dockerfilePath (apparently: https://github.com/dagger/dagger/blob/62eedd647039f9d237b5be4609f4fe6cb1cda3c8/dagger/pipeline.go#L815) translate to --option filename (for buildkit), hence allows simply to specify an alternative name for the Dockerfile, and not to specify an actual path (that part would translate to --local dockerfile= IIRC and seem to be missing).
Looking at https://github.com/dagger/dagger/blob/c8dbbaa1c6c585c9a44b68b1e55deaeb9709ef9f/tests/ops/dockerbuil...
This PR makes the static dagger binary smaller. The linker flags make it smaller through the removal of the unneeded symbols.
debian: #up: [
op.#DockerBuild & {
dockerfile: """FROM debian
"""
},
]
Should have been a new line before FROM - but then, should not SIGSEGV right?
(seems like a cue bug?)
This requires UX discussion.
The reason why filename was represented as dockerfilePath is not clear to me, and to what extent there is a desire to diverge (or not) from buildkit options and default behavior overall.
Furthermore, all of this requires a lot more tests, as there is a lot of possible combinations.
package test
import (
"this.does.not.exist.and.will.sigsegv/dubo"
)
foo: string
#418 added dagger.io/os (#File, #Dir) which duplicates the functionality of dagger.io/file (#180).
The two need to be merged together
package cake
import (
"dagger.io/dagger/op"
)
mybuild: #up: [
op.#DockerBuild & {
dockerfile: """
FROM debian
"""
platforms: ["linux/amd64", "linux/arm64"]
},
]
6:22PM FTL system | failed to up environment: buildkit solve: task failed: invalid map result
Buildkitd output the same (unhelpful) message:
ERRO[0010] /moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = task failed: invalid map result
T...
Following previous conversation, adding missing options for DockerBuild:
- target, providing the same functionality as
docker build --target - hosts, providing the same functionality as
docker build --add-host
Bumps github.com/hashicorp/go-version from 1.2.1 to 1.3.0.
Release notes
Sourced from github.com/hashicorp/go-version's releases.
v1.3.0
FEATURES:
Add Core function to return a version without prerelease or metadata (#85)
Changelog
Sourced from github.com/hashicorp/go-version's changelog.
1.3.0 (March 31, 2021)
Please note that CHANGELOG.md does not exist in the source code prior to this release.
FEATURES:
Add Core function to return a versi...
Specifically, it should be possible to specify:
- additional hosts (--add-hosts in docker lingo)
- the network the container is attached to
- specify dns servers
If the sops tool is not installed, make integration will skip all tests which require encrypted inputs, with the confusing error message โskipped: foo/bar/inputs.yaml cannot be decryptedโ).
Make installing sops and parallel mandatory (is there any scenario where it should be possible to run tests without these?)
Plus minor changes to test/README:
- fix for borked yarn call
- word about parallel and sops
This simplifies the s3 example to make it easier to understand a simple use case.
It appears that execution time for dagger up grows logarithmically with configuration size. When a configuration reaches medium-to-large size, dagger up can take several minutes to complete.
This issue can be found, in a more moderate form, when running cue eval.
There are many ongoing discussions and investigations related to this issue. If possible, please refer them here, so we can have all information in one place, and discuss how to fix it.
The dagger compute command is deprecated. Itโs not documented, and no longer actively developed. We should remove it from the CLI to avoid misleading new users.
This will naturally run only once ("echo LOL" one time):
foo: #up: [
op.#DockerBuild & {
dockerfile: """
FROM dubodubonduponey/debian
ARG TARGETPLATFORM
RUN echo LOL $TARGETPLATFORM
"""
},
]
This will run twice (echo LOL multiple times):
foo: #up: [
op.#DockerBuild & {
dockerfile: """
FROM dubodubonduponey/debian
ARG TARGETPLATFORM
RUN echo LOL $TA...
I guess it's a requirement for #447 ?
Here is my environment:
dmp@macArena:~/Projects/Go/src/github.com/blocklayerhq/dagger$ cat ~/.dagger/store/docker-elastic/deployment.json
{
"id": "4d81378c-db20-45fb-a15d-df768d5d336e",
"name": "docker-elastic",
"plan": {
"type": "dir",
"dir": {
"path": "/Users/dmp/Projects/Distribution/docker-images/infrastructure/docker-elastic",
"include": [
"*.cue",
"cue.mod"
]
}
},
"inputs": [
{
"key": "Root",
...
Warning: this will break if merged as-is. The docs deployment infrastructure must be changed too.
- Internal repo
launch.dagger.io - Github Action
cc @slumbering
dagger now stores its state alongside the plan.
To initialize an environment, dagger init must be run once:
$ cd ~/work/dagger/examples/react
$ dagger init
$ tree -a .dagger
.dagger
โโโ .gitignore
โโโ state
โย ย โโโ computed.json
โโโ values.yaml
$ cat .dagger/values.yaml
name: react
# share the state
$ git add .dagger
$ git commit -m ...
NOTE .dagger/state contains internal state (e.g. computed values).
A .gitignore is automatically generated to avoid committing it.
This...
Environment
- Macbook Pro Catalina
- Dagger Release Version
dagger version --check
dagger version 0.1.0-alpha.5 darwin/amd64
dagger is up to date.
Issue
The dagger version CLI doesn't require the Global Flags and hence should be removed.
dagger help version
Print dagger version
Usage:
dagger version [flags]
Flags:
--check check if dagger is up to date
-h, --help help for version
Global Flags:
-e, --environment string ...
Bumps google.golang.org/grpc from 1.37.0 to 1.37.1.
Release notes
Sourced from google.golang.org/grpc's releases.
Release 1.37.1
client: fix rare panic when shutting down client while receiving the first name resolver update (#4398)
client: fix leaked addrConn struct when addresses are updated (#4347)
xds/resolver: prevent panic when two LDS updates are receives without RDS in between (#4327)
Commits
64031cb Change version to 1.37.1 (#4415)
ef64e13 v...
TL;DR
Fix #460
Fix
dagger help version now print :
Print dagger version
Usage:
dagger version [flags]
Flags:
--check check if dagger is up to date
-h, --help help for version
Implements fetch-git arguments from buildkit. Adds keepGitDir (+ tests) option.
Also masks URL auth from logs when https auth is used in the git url.
This implements the one-liner workaround mentioned in #445
Bumps github.com/rs/zerolog from 1.21.0 to 1.22.0.
Commits
3c3b4a3 Add ability to customize level values
4de2fcc Fix handling of printing caller with Write, Print, and Printf. (#315)
f09463f Migrate to github actions
19c98f6 If LevelFieldName is empty don't log level (#313)
0f923d7 Fix: mutil.fancyWriter.ReadFrom records number of bytes written (#256)
582f0cf add Disabled to String and ParseLevel; add tests (#307)
98f889f Revert "Fix Typo (#306)"
f...
Consider:
#Pull: {
image: string | *"debian"
#up: [
op.#FetchContainer & {
ref: image
},
]
}
firststep: #Pull & {}
#Image: {
image: _ | *firststep
#up: [
op.#Load & {
from: image
},
]
}
finalimage1: #Image & {
image: firststep
}
finalimage2: #Image & {
}
finalimage1 will work fine.
finalimage2 will fail with `failed to up environment: buildkit solve: task fai...
in aws/s3.#Put, if target does not end with a trailing slash /, the url field will have the s3:// scheme instead of https:// scheme.
This leverages the @dagger(input) attribute to detect inputs.
Please don't merge this yet.
Changes
- Remove
skip_unless_file_existhelper - Minor change on
docker example
Bumps cuelang.org/go from 0.4.0-beta.1 to 0.4.0-beta.2.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. ...
As requested on the community discord: when a command is listed in the CLI, but fails with a scary โpanic: not implementedโ message, it complicates the learning experience.
Better to hide these commands until they are implemented.
There is an upcoming change in the Cue language, that could impact Dagger. The change concerns โclosedness rulesโ: what operations are or arenโt allowed on a value once it is closed. Currently a closed struct may not have new fields added, but it may have new definitions added. With the proposed change, new definitions will not be allowed either. This could potentially affect Dagger since we rely on the ability to add a definition called #up to values.
Currently the impact of this change...
Bumps github.com/cli/cli from 1.9.2 to 1.10.0.
Release notes
Sourced from github.com/cli/cli's releases.
GitHub CLI 1.10.0
Improvements to JSON exporting
Add gh repo list/view --json support cli/cli#3627
Add gh release view --json support cli/cli#3656
Fixes numerous issues with gh pr list/view --json support #3547
fetching closed field
fetching date fields like createdAt or mergedAt
fetching files field
Fix exporting milestone field for issues and PR...
Bumps cuelang.org/go from 0.4.0-beta.2 to 0.4.0-rc.1.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You ...
Bumps google.golang.org/grpc from 1.37.1 to 1.38.0.
Release notes
Sourced from google.golang.org/grpc's releases.
Release 1.38.0
API Changes
reflection: accept interface instead of grpc.Server struct in Register() (#4340)
resolver: add error return value from ClientConn.UpdateState (#4270)
Behavior Changes
client: do not poll name resolver when errors or bad updates are reported (#4270)
transport: InTapHandle may return RPC status errors; no longer RST_...
quick update for the Github action regarding docs
TL;DR
- Add
#Rundefinition indocker stdlibpackage. - Add tests for
#Run
Changes
I've add #Run definition to docker stdlib package. Thanks to DOCKER_HOST variable and docker in docker image, we can run any image to a remote host.
Example
Here is an example of cue configuration to run a docker container in a remote host
docker.#Run & {
host:
ref:
name: (optional)
key:
}
Re...
Recent buildkit changes broke the cache, causing double execution when an Exec has cache volumes. This breaks aws.#ECR auth and aws.#ELB.
This change removes the cache volume to workaround this problem.
Bumps github.com/uber/jaeger-client-go from 2.28.0+incompatible to 2.29.0+incompatible.
Release notes
Sourced from github.com/uber/jaeger-client-go's releases.
Release 2.29.0
Update vendored thrift to 0.14.1 (#584) -- @โnhatthm
Changelog
Sourced from github.com/uber/jaeger-client-go's changelog.
Changes by Version
2.29.1 (unreleased)
Nothing yet.
2.29.0 (2021-05-20)
Update vendored thrift to 0.14.1 (#584) -- @โnhatthm
2.28.0 (2021-04-30)...
Re-implemented input detection based on @dagger(input) attribute.
There is stilla known bug where a value merging with a definition triggers duplicates detection (not considered as a reference by the cue api).
Changes
Add random.cue file on each tests that needs random string generation.
Reason
In order to avoid code duplicate directly in tests, I've move the random string generation in his own file that we could repleace later with a private tests library for example.
One big selling point of Cue is that you can catch many errors in your configuration before applying it. However, certain errors which should be caught before running (or at least feel like they should be caught), are not.
So far I have identified one category of errors which is not caught by Cue before execution:
- References to a non-existing definition in an imported package
For example, I expect this snippet to fail before execution, but it does not:
import (
โstr...
NOTE: this PR depends on #459
- Modify daggerโs own dagger environment to use the new gitflow UX
- Fix broken stdlib along the way
We've had a regression since BuildKit 0.8.3: when using cache mounts, pipelines get executed multiple times (each time they're referenced).
This is because we use the cue value path as "cache key".
https://github.com/dagger/dagger/pull/414 implemented a workaround: if the value is a reference, use the reference as the "cache key" rather than the value itself (see https://github.com/dagger/dagger/pull/414/files#diff-12b33ae88bbb59b957748ffced0d12a6a6248013fc81d342fdc44343ce2c692cR441 `ca...
Bumps github.com/uber/jaeger-client-go from 2.29.0+incompatible to 2.29.1+incompatible.
Release notes
Sourced from github.com/uber/jaeger-client-go's releases.
Release 2.29.1
Remove dependency on "testing" in "thrift" (#586) -- @โyurishkuro
Changelog
Sourced from github.com/uber/jaeger-client-go's changelog.
Changes by Version
2.29.2 (unreleased)
Nothing yet.
2.29.1 (2021-05-24)
Remove dependency on "testing"...
This allows dagger.io/dagger/op.#Op to be maintained in regular
uncommented CUE, but not participate by default in normal dagger
evaluation (given the performance problems it currently introduces).
The "full" #Op can be enabled by passing the "-t fullop" flag:
cue eval -t fullop ./examples/react
(which currently triggers the performance issue).
- Go to https://docs.dagger.io
- Scroll to the bottom
- Click on link โNext: Introductionโ
- Back to step 1.
- Simplify install instructions
When navigating to https://docs.dagger.io/devel the right version (devel) is selected in the list.
However, when going to a link directly (https://docs.dagger.io/devel/programming/), the first version is selected (v0.1.0-alpha.1)
split dagger into sub-packages
- With gitflow, the
daggerpackage was becoming massive
move packages to top level, change vanity URL. This changes dagger.io/go/dagger to go.dagger.io/dagger
- use a separate domain for golang, not to interfere with dagger.io
dagger.io/goreferred togithub.com/dagger/daggerwhereasgo.dagger.iotogithub.com/dagger, which allows to use the URL for other repositories
- input list: Adds support for cue doc string in new Description field
- Implemented integration tests
- Added
@dagger(input)and@dagger(output)annotations in stdlib pkgs
- Updated
./examples/README.md - Various UX improvements on commands instructions + help
When arriving on the doc website, we can see, on the left, the menubar of all the accessible pages for a given version
Devel version, left menubar
Introduction
Install Dagger <==
Dagger vs. Other software <==
GUIDES
Programming Guide
Operator Manual
COMMUNITY <==
GitHub
Discord
All other alpha versions, left menubar
Introduction
Getting started <==
Dagger vs. Other <==
GUIDES
Programming Guide
Operator Manual
MORE <==
GitHub
Discord
Plus...
When visiting an unknown URL (https://docs.dagger.io/devel/deqs), the 404 page retrieved seems in debug mode, showing some S3 infos :
Message: The specified key does not exist.
Key: devel/deqs
RequestId:
HostId: ```
Bumps cuelang.org/go from 0.4.0-beta.1 to 0.4.0.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also t...
Bumps filippo.io/age from 1.0.0-rc.1 to 1.0.0-rc.2.
Release notes
Sourced from filippo.io/age's releases.
The last (?) v1.0.0 release candidate!
This is it! The v1.0.0 milestone is empty. Will let it simmer for a bit and then we'll have v1.0.0. Please test it!
Main changes
๐ค Reject RSA keys (for ssh-rsa) smaller than 2048 bits (#266)
๐ฅ Remove limit of 20 recipients per file (#139)
๐ Add age(1) and age-keygen(1) man pages (#131) โ read them here!
โ๏ธ ...
Secrets are never exposed in plaintext in the Cue tree. dagger query
won't dump secrets anymore, Cue errors won't contain them either.
BuildKit-native secrets support through a new mount type. This
ensures secrets will never be part of containerd layers, buildkit
cache and generally speaking will never be saved to disk in plaintext.
Updated netlify as an example
Added tests
Changed the Cue definition of a secret to:
@dagger(secret)
id: string
}
This is to ensure ...
- Implements user inputs checks when
dagger up - Only triggers when a tty is detected
- Supports
--forceto bypass the check
@quaint umbra ^^^ FYI, added discord notifications for the example repo
versioning was broken by https://github.com/dagger/dagger/commit/af776b8abe5a18290abfcdcce2aa1c5b3b6e507b
cc @aluzzardi
Happens only with the official release (not if I compile myself), due to cgo that is disabled.
Workaround:
export USER=$LOGNAME
cc @aluzzardi
dagger input edit: view and edit all inputs in bulk using a text editor
Day to day it's a PITA to use dagger input *, I ended up relying a lot on my shell history. Changing an input requires re-typing the whole thing (e.g. adding an exclude to a dir, changing the branch of a git input, ...). Renaming an input requires unsetting it and then setting it back to a different key with the same arguments, etc.
Demo:
[](h...
Example:
secure: dagger.#Secret
test: os.#Container & {
mount: "/secure.txt": secret: secure
command: "echo hello $(cat /secure.txt)"
always: true
}
$ dagger input secret secure world
$ dagger up
test | computing
test | #4 0.102 hello ***
test | completed duration=200ms
Changes
There was a problem with #512 CI, some tests were skipped because sops couldn't read inputs.yaml.
Here is the trace
With @grouville, we explored that problem and discovered that it was just the sops version that was outdated 3.6.1 -> 3.7.1.
:warning: We should take care about that kind of problems because there were some tests that failed for people with the latest version (but not in the...
Bumps ws from 7.4.2 to 7.4.6.
Release notes
Sourced from ws's releases.
7.4.6
Bug fixes
Fixed a ReDoS vulnerability (00c425ec).
A specially crafted value of the Sec-Websocket-Protocol header could be used
to significantly slow down a ws server.
for (const length of [1000, 2000, 4000, 8000, 16000, 32000]) {
const value = 'b' + ' '.repeat(length) + 'x';
const start = process.hrtime.bigint();
value.trim().split(/ *, */);
const end = process.hrtime.bigi...
First PR :
- Implements a small fix for the out-of-bound parsing error on
dagger compute --input-[string,dir,git], when a wrong command is written - First test usage
Signed-off-by: Guillaume de Rouville
- implements command
dagger output list - implements tests
- list outputs at the end of a
dagger up
added outputs and inputs annotations
When running dagger list in a workspace for which I donโt have the private key, the following error appears: failed to load environment: unable to decrypt state: Error getting data key: 0 successful groups required, got 0.
To reproduce, fetch the dagger examples repo, and try to use it:
$ git clone https://github.com/dagger/examples
$ cd examples/helloapp
$ dagger list
3:14PM ERR system | failed to load environment: unable to decrypt state: Error getting data key: 0 successf...
When running dagger up on a machine with no docker client installed:
$ dagger up
3:26PM ERR system | failed to run docker: exec: "docker": executable file not found in $PATH output=
3:26PM FTL system | unable to create client: exec: "docker": executable file not found in $PATH
Tested on https://github.com/dagger/examples
since .dagger directories have a special meaning now because of gitflow,
it's better not to have a ~/.dagger since it's not a workspace and
it confuses dagger (e.g. dagger new from $HOME).
We don't store state there anymore, just keys and the last version
check, so it's okay to be in ~/.config IMO
Looking at my system, in ~/.config there's gcloud, gatsby, gh,
yarn, and others so it seems like a pretty common location.
Signed-off-by: Andrea Luzzardi
sourceis now optionalsourceInlinerenamed tomanifestkubeconfigis astringrather than adagger.#Secret
Signed-off-by: Andrea Luzzardi
First pass at a Kubernetes example
In some situations, Dagger needs to talk to a Docker daemon to setup its buildkit environment. Currently it does this by shelling out to the docker command. If the docker command is not installed, Dagger will fail (see #533). This can be solved by implementing the required Docker client calls directly in the dagger binary (using a Docker client library). This will guarantee that Dagger can always setup its buildkit environment when a Docker daemon is available, even when the docker cl...
Currently it is difficult for multiple Dagger environments to share code, whether by specialization or package import.
Currently Daggerโs โuniverseโ (standard library of packages) does not follow the standard Cue development workflow:
- If a cue configuration imports a Dagger Universe package (
dagger.io/โฆ),cue evalwill not work out of the box. - Packages cannot be installed or upgraded individually
- Packages are loaded at runtime by the
daggercommand. They are not available to the developer in theircue.moddirectory, or anywhere else on the local system. - There is no versioning. Users cann...
Most packages in the Dagger โuniverseโ are not yet stable, and the import path should reflect that. One possible inspiration is kubernetes resource naming. When resources are not yet stable, their canonical name includes โalphaโ or โbetaโ, for example: service.beta.kubernetes.io/aws-load-balancer-ssl-cert.
In our case, instead of this:
import (
โdagger.io/osโ
โdagger.io/daggerโ
โdagger.io/dagger/opโ
)
We might use:
import (
โdagger.io/alpha/osโ
โ...
$ make integration
go build -race -o ./cmd/dagger/dagger-debug ./cmd/dagger/
You need sops. On macOS: brew install sops
make: You: No such file or directory
make: *** [Makefile:44: integration] Error 127
$ cd voteapp
$ dagger up
1:33PM FTL system | failed to query environment: plan config: #ECSApp.cfnStack.outputs.#up.6.env.AWS_ACCESS_KEY_ID: conflicting values string and {id:string} (mismatched types string and struct):
/config/cue.mod/pkg/dagger.io/aws/aws.cue:14:13
/config/cue.mod/pkg/dagger.io/aws/cloudformation/cloudformation.cue:78:28
/config/cue.mod/pkg/dagger.io/dagger/dagger.cue:18:10
/config/cue.mod/pkg/dagger.io/dagger/op/op.cue:46:18
/config/ecs.cue...
Dagger distributes several Cue packages for use by the Dagger developer community. There are 2 types of packages:
-
Core packages, which expose features of Dagger itself. Currently those are
dagger.io/os,dagger.io/file,dagger.io/daggeranddagger.io/dagger/op. This may change in the future (possible by merging these packages into one). -
โUniverseโ packages, which implement useful components to be reused by others in the community. For example AWS integration, Netlify integrat...
$ cd voteapp
$ dagger up
3:11PM FTL system | failed to query environment: plan config: #KubeApp.deploy: field not allowed: sourceInline:
/config/cue.mod/pkg/dagger.io/kubernetes/kubernetes.cue:47:9
/config/gke.cue:95:10
/config/gke.cue:98:3
$ cd voteapp
$ dagger up
3:14PM FTL system | failed to query environment: infra.gcpConfig.project: conflicting values string and {id:"secret=infra.gcpConfig.project"} (mismatched types string and struct):
1:1
/config/cue.mod/pkg/dagger.io/gcp/gcp.cue:12:11
Secrets support in os.#Container is not consistent with other mounts:
- Tmpfs:
#Container.tmpfs[โ/fooโ] - Cache:
#Container.cache[โ/fooโ] - Artifact:
#Container.mount[โ/fooโ] - Secret:
#Container.mount[โ/fooโ]. Following the pattern from the other 3, it should be#Container.secret[โ/fooโ]
- Fix issue with kubernetes package (breaking API change upstream)
- Fix broken values.yaml (infra.gcpConfig.project was incorrectly set to type secret, should be text)
After spawning some tty blocking #op.Exec commands inside Buildkit for some tests. I wasn't sure if they were still processing in the background.
I manually stopped the buildkit container, then dagger up again. However, it didn't reconnect to the built container, neither creates a new one. The only way is to delete the container and rerun dagger up. Btw, jaegertracing also interferes with the rerun, it shall be stopped also
Proof :
โ docker ps
CONTAINER ID IMAGE ...
1. docker.#Push doesn't support authenticated pushes (and therefore cannot work, unless pushing to a local unauthenticated registry).
Support was added in #373 in op.#DockerLogin.
Either docker.#Push takes credentials or we expose a docker.#Login component.
2. docker.#Push doesn't export the reference/digest of what was just pushed. Support was added to op.#PushContainer in #303. I suggest docker.#Push takes a name as an input and has a ref and digest as outputs...
This happens with js/yarn (e.g. examples/react), at the end of dagger up (and when using dagger output list), artifacts (docs.build is a dagger.#Artifact) are displayed like this:
docs.build {\n from: {\n image: {\n package: {\n bash: "=~5.1"\n yarn: "=~1.22"\n }\n version: *"3.13.5@sha256:69e70a79f2d41ab5d637de98c1e0b055206ba40a8145e7bddb55ccc04e13cf8f" | stri...
Signed-off-by: slumbering
Changes
Add always: true keyword on random.cue to always generate random string for test
Small non-breaking cleanup of universe packages.
Changes
With #503, secrets are now stored in .dagger/ directory.
I've changes docker.#Run secrets management to use this new feature and update tests to verify that nothing has been broken.
Signed-off-by: slumbering
Adds support for generating whole stdlib documentation (markdown, json, text) and write it to the local filesystem:
Example:
dagger doc --output ~/my-docs --format json
cc @aluzzardi @slumbering for docs website integration
PS: I've also fixed formatting and typos on aws packages (aws/rds and aws/elb were broken, cc @TomChv)
dagger up now prints the environmentโs outputs, which is very useful. However, there is no distinction between โfinalโ outputs and โintermediaryโ outputs (because we donโt know how to make that distinction). As a result, a typical configuration will print intermediary outputs which are 1) not useful to the end user, and 2) confusing because their contents are obscure.
For example, see voteapp in https://github.com/dagger/examples
- docs: rename tools/daggosaurus to website
- docs: use the path config option rather than symlinks
Code convention: use nouns instead of verbs whenever possible.
Reasoning: One can apply just about anything to Kubernetes via this:
deployment, load balancer, RBAC policy, a custom CRD resource, etc.
Upstream those are called resources: You give kubectl apply one or more
manifests and it will create the corresponding resources.
Signed-off-by: Andrea Luzzardi
- This PR adds a new mount type:
docker.sock(in addition tocache
andtmp) - It's then able to mount the LOCAL (as in, from the machine running
dagger) docker socket inside the container by pretending to be an SSH
Agent (hijacking the SSH agent forwarding support of buildkit)
support running containers on the local docker engine
DEPENDS ON #570
TL;DR
- Add
#Upindocker/composepackage - Add
#Clientindocker/composepackage - Add tests for docker-compose package
Changes
I took hover the docker.#Run definition to create compose.#Up. There is nothing different about ssh connection.
:warning: We musn't forget to update both packages if we modify something about
sshconnection.
We can call ``#Up docker-compose one (or both) of those arguments :
- a
docker-compose.yamlincomposeFile
...
Changes
Refactor random generation into a definition #Random to make it reusable.