#github-feed

1 messages · Page 19 of 1

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11685 core: ModTree infrastructure for dynamic checks

Summary

  • Replace walkObjectChecks with ModTree abstraction for traversing module trees
  • Rename FnTreeNode to ModTreeNode with simplified structure
  • Add support for walking arrays of objects with +check methods
  • Add --all/-a flag to dagger check to enumerate dynamic checks
  • Include dynamic checks in aggregate listing even without --all
  • Fix TUI display for aggregate check execution
  • Add modtree path matching tests

Context

This PR adds the core infrastructure needed to support d...

#
shykes
[dagger/dagger] Pull request opened: #11686 toolchains: dynamic checks for go and engine-dev

Summary

  • Add dynamic test enumeration to Go toolchain using +check decorator
  • Add test splitting to engine-dev toolchain for parallel test execution

Changes

Go toolchain

  • Tests() returns []*TestDirectory grouped by directory
  • Each Test has a +check decorated Run() method
  • Uses Source.Search() for fast regex-based test discovery
  • Pre-builds container and shares across tests via WithWorkdir()

Engine-dev toolchain

  • Tests() returns []*TestDirectory with...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11692 Revert "fix: support kernels without legacy xtables (6.17+) (#11608)"

This reverts commit 79c2341d21f7499269a4667d093bb7f32e1d157d.

I saw some strange slowness in https://github.com/dagger/dagger/pull/11608 but thought it went away after re-runs. However, I did some more testing before starting the release and realized it actually is quite noticeably slower. It's hard to tell in CI since our infra is quite noisy, but my laptop is much more consistent and is consistently several minutes slower to run TestModules with the nftables change vs. without it.

...

vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #11695 'dagger check': ignore checks not from installed toolchains

Problem

When running 'dagger check' in a module, there is no distinction between:

a) checks meant to be run in the context of another module (by installing the current module as a toolchain)
b) checks meant to be run in the context of the current module

As a result, running 'dagger check' in a toolchain module, is likely to fail in confusing and unpredictable ways.

This complicates out-of-the-box CI for every Dagger module - because that requires a simple way to run every check on ...

vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

Here's a strawman proposal that leans hard into pragmas to support sequencing and context-conditional execution.

This proposal has some controversial aspects even with me, but it seemed worth pulling on the thread and seeing how much it addresses.

In particular, I had to convince myself to press on after deciding it might be OK for functions to reference other functions in pragmas. And to press on after inventing a bit of special notation inside those pragmas. But, in the whole picture,...

vivid lintelBOT
ornate vigilBOT
#
eunomie
[dagger/dagger] New comment on discussion #11653: +ship

Very interesting 👍

Regarding referencing functions by name or by their return type: type looks more elegant, but also more complex. You have to define a type, a specific type, that might only be used to create this link. For instance if your ship function must be run after tests to pass, the result value of test might be discarded anyway as we only want to synchronise execution, we have nothing to do with the value itself.
So in that case it means to create a type, return a value, add an...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
henkhofs
[dagger/dagger] Pull request opened: #11704 feat(sdk/java): add checks support to Java SDK

Summary

This PR adds support for the checks feature to the Java SDK, bringing it to feature parity with Python, TypeScript, and Go SDKs.

Changes

  • New @Check annotation: Marks functions as checks that can be executed for validation
  • Updated FunctionInfo record: Added isCheck field to track check status
  • Updated annotation processor: Detects @Check annotation and generates withCheck() call
  • Test module added: hello-with-checks-java demonstrates usag...
ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

So in that case it means to create a type, return a value, add an argument, all that for nothing.

Yeah, I agree - the arg-passing trick is neat when there is something actually worth propagating, but it's really clunky to require it for all forms of sequencing, especially if/when the values being propagated are deterministic (i.e. keyed on commit or branch) or not interesting to the target function (shipping probably doesn't care about test reports, even if you make your checks return ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11705 engine: make logs more useful

The engine logs above Debug have over time devolved to be mostly useless. Many of the Debug level logs are also not particularly helpful. This commit is a grab bag of improvements that makes Info level logs useful without being too overwhelming:

  • warnings about deprecated APIs are rm'd; there's zero utility for these to be in engine logs vs client telemetry output
  • "CoreModObject.ConvertFromSDKResult: got nil value" happens in expected scenarios, lowered to debug
  • "collectPBDefinitions:...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #11708 feat: use the remote library on versioned engine instead the local one

This removes the generation of internal/telemetry and internal/querybuilder that can be installed from the local library.

Add a new flag to --generate-module to pull the libraty that matches the engine version's.

Add a new _EXPERIMENTAL_DAGGER_DEV_ENGINE to force the engine to act as a dev engine even if its tagged so we don't break engine_test files if we do some breaking changes in telemetry or querybuilder package.

Related to #11515 and #11547

vivid lintelBOT
ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

Just realized +labels wouldn't be enough for preview environments, since we'd need to handle removing them too. :thinking:

Currently we handle that by looking for a unlabeled or closed event and that the PR is missing the labels:

on:
  pull_request:
    types:
      - unlabeled
      - closed
    paths:
      - ...

name: Cloud preview environment deleted

jobs:
  add-github-pr-comment:
    # add the check in case a user remove a different set of labels
  ...
vivid lintelBOT
ornate vigilBOT
#
eunomie
[dagger/dagger] New comment on discussion #11653: +ship

I was thinking about some use cases, that are don't know how we should handle them.

So for instance, let's say we have a check function, defined in a toolchain. And we want to notify a system when the check is passing or failing (I'm thinking about notification and tracking, but that could be anything else). So in this case, a ship/publish function makes total sense. But we want, we need the check results.
I'm seeing different ways to achieve that:

1/ define a ship function tha...

vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger] New discussion #11710: Using Dagger like Testcontainers

Context

Dagger is great at building and running integration test harnesses. This is done in two parts:

  1. Execution environment: the container that will run the test code
  2. Service dependencies: services that the tests need to connect to while they run

Orchestrating the execution environment with Dagger is pretty straightforward: build the container on-the-fly, mount the application code, optionally build, run the test tool.

But there are several ways to orchestrate servi...

ornate vigilBOT
#
kpenfound
[dagger/dagger] New comment on discussion #11710: Using Dagger like Testcontainers

I think a good DX would look something like this:

import { render, screen, fireEvent } from '@testing-library/react';
import Counter from './Counter.jsx';
import dag from '@dagger.io/dagger';

describe('Counter Component', () => {
  // Create service container
  let postgres = dag().container().from('postgres:latest').asService();

  // Jest beforeAll hook to start services.
  // Can also do beforeEach() to start a new one for each unit in this suite
  beforeAll(async ()...
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #11713 CI: orchestrate test service dependencies from test code (aka "testcontainer model")

NOTE: this is about a change to how we test the Dagger project. It is NOT about how Dagger runs tests.

For our e2e tests, we orchestrate service dependencies outside our test code. Recently we have discussed the benefits of orchestrating service dependencies in the test code. This made us realize that, for our own project, we should switch to this technique.

Task list

Core integration tests

  • [ ] Migrate test execution: from `to...
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
frantjc
[dagger/dagger] New comment on discussion #11710: Using Dagger like Testcontainers

Hi! :wave:

re: "Docker API compatibility bridge", aka Dogger. The (limited) work I've done on it is here. It's worth keeping in mind that I didn't start this with just Testcontainers in mind. The goal I had was to be able to use the docker CLI from within Dagger's execution environment without having to pull down and run the Docker engine, while also taking advantage of Dagger's cache. The primary use-case I had in mind was `ki...

ornate vigilBOT
#
verdverm
[dagger/dagger] New comment on discussion #11710: Using Dagger like Testcontainers

I recently implemented (what i understand to be) the (2) service orchestration at the client level, as one of those rare examples from the community :]

https://github.com/hofstadter-io/hof/blob/_next/lib/env/cmd/run.go

pseudo-code

all = `user selected sync points`
services = [for $item in all if $item.kind == "service"]
containers = [for $item in all if $item.kind == "container"]

for $svc in services:
  $svc.up()
for $ctr in containers:
  if isLast:
    $ctr.terminal()
...
ornate vigilBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #11710: Using Dagger like Testcontainers

Another obstacle I'd identified is that docker run -p 8080:80 nginx wouldn't actually expose the container's port on localhost, but on Service.endpoint()

If you call Service.up(), the port is in fact exposed on localhost. This is strictly better than Docker/testcontainers, because Docker cannot actually guarantee what the listening hostname + ports are - testcontainers has to make educated guesses.

vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11717 chore: bump astral-sh/ruff from 0.14.11 to 0.14.13 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.14.11 to 0.14.13

Release notes
Sourced from astral-sh/ruff's releases.

0.14.13
Release Notes
Released on 2026-01-15.
This is a follow-up release to 0.14.12. Because of an issue publishing the WASM packages, there is no GitHub release or Git tag for 0.14.12, although the package was published to PyPI. The contents of the 0...

ornate vigilBOT
#
marcosnils
[dagger/dagger] New comment on discussion #11683: +up

Initially the proposal SGTM. The only question that comes to my mind is if you have thought about how the UX around customizing exposed ports might be. Given that dagger up could potentially start multiple services, it could happen that those services might be trying to expose an invalid (< 1024) or overlapping ports so some sort of client-side customization will be needed here similarly to the --ports flag that dagger call $svc up has today.

ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

@eunomie Having tried something like that in Concourse, I'm inclined to not mix the notifications use case in with ships:

  1. Notifications tend to be unilateral, like "alert the team when ANY function fails." In Concourse this meant copy-pasting ~6 lines of YAML on every Job, and with @ship it seems like that would mean one @ship per @check, or a new notation like @any-check - so it starts to feel a bit forced.
  2. A lot of the time people what notifications to be on _state transit...
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11720 internal dev `cache-expert` agent skill

Had some promising but not quite fruitful trials w/ latest LLM coding models (codex 5.2-xhigh + opus 4.5) when it comes to debugging super gnarly engine issues around caching.

Seemed like one bottleneck was that it had to learn (and re-learn after context compaction) the whole codebase every time (which is often necessary for particularly tricky problems). So giving it detailed developer docs that focus it in on the parts that truly matter when debugging cache stuff may save a lot of file ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
easyas314159
[dagger/dagger] Issue opened: #11724 🐞 github.com/wazero/wazero indirect dependency causing panic on older ARM processors

What is the issue?

I am deploying a GitHub self-hosted actions runner on some older Nvidia Jetson hardware that uses an ARM Cortex-A57 that the software we are developing must support. Every version of the dagger cli that' I've tried panics immediately.

From the digging I've done so far the panic appears to be an issue with the github.com/wazero/wazero dependency. Updating it from 1.9.0 to 1.11.0 seems to resolve the issue when I tested against main. I unfortunately don't have the t...

vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11725 sdk(python): fix annotations with `from __future__ import annotations\

Fixes #11554

Context

When from __future__ import annotations (PEP 563) is enabled, param.annotation becomes a string instead of an actual type object. This causes all metadata extraction (DefaultPath, Doc, Name, Ignore, Deprecated) to fail silently: the annotations are simply ignored.

Fix

The fix uses get_type_hints(..., include_extras=True) to resolve string annotations back to actual types while preserving Annotated metadata.

To test

dagger ...
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11726 core: add MainModule to Env

Introduce MainModule field to Env to explicitly represent the project
being worked on, as distinct from toolchain modules. This provides a
clear semantic separation:

  • MainModule: the project/context module
  • installedModules: toolchain modules (via withModule, now deprecated)

Changes:

  • Add MainModule field to Env struct
  • Add Env.WithMainModule() method and GraphQL binding
  • Update currentEnv() to use withMainModule instead of withModule
  • Env.Checks() now delegates to MainModule.Checks()...
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
eunomie
[dagger/dagger] New comment on discussion #11653: +ship

Well, I think you could have a toolchain with a notify @ship function that will run after all checks, and deals with it. So you just install the toolchain, probably add some config in the dagger.json to customize it, and that should be enough. So no copy-pasting. But more than that, notification was just one easy example, there's probably other use cases. And I'm not really sure why we should make notification a special case.

But the transition based notification looks interesting. But...

#
eunomie
[dagger/dagger] New comment on discussion #11683: +up

I have not, but that's a good point.
A first idea could be to add the port to the +up pragma, but that's probably wrong. We can imagine to have that in the dagger.json, the same way we are customizing checks for instance. That way it stays decoupled from the implementation of the service and is really just a host-side configuration.
But we start to have more and more things in this dagger.json and I don't know if that's a good thing.

ornate vigilBOT
#
idlsoft
[dagger/dagger] New comment on discussion #11653: +ship

Here's a strawman proposal that leans hard into pragmas to support sequencing and context-conditional execution.

There is an inherent limitation with using pragmas.
In our company most github workflows do the following:

  • on pull request run checks and pushes concurrently, because we need to push an image for further testing regardless of unit tests passing
  • on merge we push only after checks are successful

Pragmas are unconditional, so something like this becomes very tricky to d...

vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

@idlsoft Isn't that just a matter of setting different pragmas on different functions to handle the different configurations? Parts 1 and 2 are meant to address the fact that all these sorts of different configurations need to be supported, and at least with this proposal the idea is that you would just define different functions for them, which shouldn't be too toilsome, since they can trivially share code. (Much easier than duplicating entire GHA workflows, and IMO still better than templat...

ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #11653: +ship

Didn't have time for a detailed review yet... But my 2 cents: if we're going in the direction of a full-blown declarative workflow DSL built on dagger functions (not a bad idea, although the devil is in the details)... then the starting point IMO is to move that declarative config out of the function code, and into a separate layer, possibly a yaml/toml config file in the project module, or something equivalent.

A few reasons:

  1. If you're referencing another function from code, that ...
#
idlsoft
[dagger/dagger] New comment on discussion #11653: +ship

@idlsoft Isn't that just a matter of setting different pragmas on different functions

Well, maybe, but all the extra attributes are basically a replacement for good old if statements. Which still has value, is convenient to use, and may satisfy 90% of use cases, but is also limited.

To give another example, after pushing a docker image we post a request to an external service to scan the image, and get approval that no critical vulnerabilities were found. That's a post-push action.
...

ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

@idlsoft

To give another example, after pushing a docker image we post a request to an external service to scan the image, and get approval that no critical vulnerabilities were found. That's a post-push action.

If you're humor me for a moment, that's just another pragma:

@ship(branches: ["@default"]) # defaults to `after: ["@checks"]`
pub publishImageAfterChecks: Void {
  publishImage
}

@check(after: ["publish-image-*"]) # * to match PRs and main, or just spell ...
ornate vigilBOT
#
idlsoft
[dagger/dagger] New comment on discussion #11653: +ship

I suppose if dependencies between checks and ships go both ways - yes, this is a lot more flexible.
For some reason I assumed that ships can depend on checks, but not the other way around.

On a different note, it may be useful to have 3 possible outcomes instead of 2: success, failure, ignored. And, perhaps a way to consume them in the "after" functions?

ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship
  1. Yeah, the encoded function names were a hesitation that I also had. It would really come down to DX. If that's the cost of entry and the DX is better in every other way than the alternatives we come up with, for me it's not a dealbreaker, since it's at least using string values that we're already expecting the user to be familiar with outside of any particular codebase, in particular as a way to interact with the very functions that have these pragmas. But, it's subjective like you said.
    ...
vivid lintelBOT
ornate vigilBOT
#
vito
[dagger/dagger] New comment on discussion #11653: +ship

I suppose if dependencies between checks and ships go both ways - yes, this is a lot more flexible.

Yeah exactly. :)

On a different note, it may be useful to have 3 possible outcomes instead of 2: success, failure, ignored. And, perhaps a way to consume them in the "after" functions?

Interested to hear more on that - what's your use case?

ornate vigilBOT
#
idlsoft
[dagger/dagger] New comment on discussion #11653: +ship

Interested to hear more on that - what's your use case?

It'd be similar to how in unittesting you can to skip a test under certain conditions that can only be evaluated at runtime, typically by throwing a specific exception/error.
You can then see separate counters for succeeded/failed/skipped. The distinction is important, I think.

An example... Only push an artifact if remote repository credentials were provided? idk, I'm sure there are better ones.

Also... if pragmas control co...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11729 finish separating content digests out; refactor ids/cache keys

Couple interrelated goals here:

  1. Fix the root cause of the TestConstructor flakes by more fundamentally squashing the possibility of content digest overlap resulting in cross-client cache confusion
  2. Setup some cache key + ID changes that will be needed for the imminent removal of the buildkit solver
  3. Finish what I originally started in https://github.com/dagger/dagger/pull/11625 but had to partially back out of due to needing a fix ASAP

Main thing is to manage "content digests" ...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11730 initial draft of agent skill for daggerizing a repo

This is a rough first draft based purely on a single instance where I daggerized a particular repo. I daggerized it via prompting and then asked the LLM to create a skill out of what it learned. Likely useful enough to merge as is and iterate on.

I don't think the opinions it expresses are likely what everyone shares, but I also don't have time to get bogged down in refining details right now since this was just a quick extracurricular thing. If there's preference to not merge I'll just cl...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11731 (updated+fixed) support kernels without legacy xtables (6.17+)

This is an update to the reverted https://github.com/dagger/dagger/pull/11608 which fixes the performance regression that introduced.

After investigation, the previous PR was fine except that it removed the ip6tables package.

When I run TestModule (pretty heavy integ test suite):

  • With the previous PR: 7m39s
  • With the previous PR but leave in ip6tables: 5m56s

These numbers are pretty consistent on my laptop, not noise.

It's not 100% clear to me yet why that is, but m...

ornate vigilBOT
#
eunomie
[dagger/dagger] New comment on discussion #11653: +ship

It might be a little bit off topic, but if we think about a separate config layer, would it also make sense to include the toolchain customization that is right now inside the dagger.json?
Or should all that in fact be the equivalent of the dagger.json but maybe using a different format (like toml)? To also avoid to have multiple config files.

ornate vigilBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #11653: +ship

@vito whether it's a @ship in a 100% reusable toolchain, or in a more project-specific toolchain, it's the same problem. It makes it impossible to break down your project's dagger functions in a set of toolchains - you have to keep a '.dagger` in your project with embedded functions, and also install toolchains when that's possible. And we have to explain all of that, navigate when to use which, etc. I don't like that UX because it's too many concepts to understand on day one. I would p...

ornate vigilBOT
ornate vigilBOT
#
idlsoft
[dagger/dagger] New comment on discussion #11653: +ship

Or should all that in fact be the equivalent of the dagger.json but maybe using a different format (like toml)? To also avoid to have multiple config files.

As long as non-json formats are being considered, PKL may be an interesting contender. It has interesting built-in features to help avoid duplication.

vivid lintelBOT
#
kpenfound
[dagger/dagger] Pull request opened: #11732 refactor toolchain proxying

This refactors/simplifies toolchain proxying. I've felt like the original proxying was too hacky, especially if we're going to lean on toolchains even more. This can all potentially go away if we solve contextual Envs (@shykes @vito ), but in the current implementation this refactor seems like a more straightforward approach

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11735 ci: make test binaries faster to build

We were running all tests with -race, which means that we don't hit earlier go build cache and is generally quite slow to build. It also is not incredibly valuable in that it applies to the client side of integ tests, which isn't the main point of the integ tests.

You could see this in telemetry because there was often a large gap between the start of the withExec for go test ... and the actual start of spans for tests.

This was exacerbated by the fact that we defaulted to running go ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11738 chore: bump the sdk-java group in /sdk/java with 2 updates

Bumps the sdk-java group in /sdk/java with 2 updates: org.assertj:assertj-core and org.testng:testng.

Updates org.assertj:assertj-core from 3.27.6 to 3.27.7

Release notes
Sourced from org.assertj:assertj-core's releases.

v3.27.7
:lock: Security
Core

Fix XXE vulnerability in isXmlEqualTo assertion (CVE-2026-24400)

See GHSA-rqfh-9r24-8c9r for details; many thanks to @​wxt201 and @​Song-Li for responsibly repor...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Issue opened: #11744 Go generated bindings should depends on the released core bindings

Every Go module contains a bunch of boilerplate to work properly and integrate with the IDE.

A regular dagger go module is composed of the following generated files:

├── dagger.gen.go <--- module support static generation + dagger global client used in the module
├── internal
│   ├── dagger/dagger.gen.go <--- Client bindings to query the dagger engine, include core and dependencies bindings.
│   ├── querybuilder/
│   └── telemetry/

Note: querybuilder and telemetry are staticall...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11748 filesync: fix .gitignore handling

Mark gitignored entries in stats instead of excluding them from the walk, and have the server skip deletes/updates under those paths.

This avoids cross-client conflicts caused by confusing overlapping requirements of having .gitignore in the synced fs tree but also needing to exclude it sometimes.

Fixes CI flake about change kind changed from "add" to "delete" during sync and variations

  • repro'd with `dagger call engine-dev test --parallel=12 --pkg ./core/integration --run='TestHo...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
cbochs
[dagger/dagger] Issue opened: #11750 🐞 The `+ignore` pragma is not always respected during module construction

What is the issue?

Follow-up of Discord discussion: https://discord.com/channels/707636530424053791/1465717553530671389

I have a context directory, which looks like (simplified)

    // +defaultPath="/"
    // +ignore=["**","!dagger.json"]
    source *dagger.Directory,

When calling a module without any arguments, the source is resolved as a "context" directory and the correct filters (exclude) are applied:

$ dagger call test

dagger.json

However, if a --source...

#
shykes
[dagger/dagger] Issue opened: #11751 'dagger client install go' overwrites replaced dagger.io/dagger

When installing a generated go client ('dagger client install go'), if the target module has dagger.io/dagger installed and replaced with a go.mod replace directive, then the replaced version will be overwritten with the generator's bundled version (matching the current engine's version).

This will break whatever behavior was enabled by the replaced version. In our case, github.com/dagger/dagger, we need the replace directive because we're targeting a dev version of the library.

As discus...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11761 chore: bump qs and express in /sdk/rust/crates/dagger-sdk/examples/build-the-application/app

Bumps qs and express. These dependencies needed to be updated together.
Updates qs from 6.11.0 to 6.14.1

Changelog
Sourced from qs's changelog.

6.14.1

[Fix] ensure arrayLength applies to [] notation as well
[Fix] parse: when a custom decoder returns null for a key, ignore that key
[Refactor] parse: extract key segment splitting helper
[meta] add threat model
[actions] add workflow permissions
[Tests] stringify: incre...

vivid lintelBOT
vivid lintelBOT
#
vmaffet-fast
[dagger/dagger] Issue opened: #11765 🐞 Engine tries to reuse ssh-agent that does not exist

What is the issue?

We occasionally get errors from the engine trying to load a remote module through SSH in our CI environment (multiple ephemeral GHA runners accessing a single dagger engine).

We receive 128 exit code with a permission denied; however, our SSH socket looks correctly started and has our key in the runner.

It appears that the engine is trying to reuse a socket from a previous runner that does not exist anymore.

Dagger version

dagger v0.19.8 (image://registry.d...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
kpenfound
[dagger/dagger] Pull request opened: #11768 go generated clients get their own go.mod

This gives generated clients in go their own go.mod.

Doing this means we don't compete with the project's own go.mod at all

It includes backwards compat for existing generated clients so that we don't break anyone's working setup.

We are also directly handling the integration test's need to replace the local SDK (read: library) rather than making that part of client generation. In the integration tests we handle the replace for the local library. Because the generated clients have th...

#
tiborvass
[dagger/dagger] Pull request opened: #11769 Make dagger connect faster

Skip Available() call for drivers, getting rid of docker version altogether.
If Provision() or Connect() fails, try the next driver.
Skip Wait() loop when establishing the buildkit connection.
Instead, if Info() fails, error out.
Delay setting up buildkit session to before Run.

This approximately halves the connect time for me on a docker+ssh context.

vivid lintelBOT
#
tiborvass
[dagger/dagger] Pull request opened: #11770 engine-dev: replace alpine with wolfi

I've had trouble running dagger call engine-dev introspection-json as-json contents due to possible Alpine issues.

load http(url: "https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/libcrypto3-3.5.4-r0.apk", refID: "qbyaq3p936rlfmt7shw4nbi3z"): invalid response status 404 Not Found

I'm not exactly sure what happened: https://dagger.cloud/tiborvass/traces/22fa61bd3737d64e844dad167bd733c5?listen=5f98acc260801256&listen=0a627df8013f68cd&listen=f86b388a509b75ae&listen=8ec8af...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11775 telemetry: avoid O(n^2) space usage in published telemetry

Every call made in the engine is sent back to the client (and from there forwarded to cloud). Each call also was including the entire ID payload of the call.

That takes total O(n^2) space, where n is the size of the DAG, due to the fact that each call payload duplicates the parts of the DAG already sent in earlier call payloads.

e.g. for a very simple DAG like A->B->C, the telemetry sent would be

  1. ID for A
  2. ID for A->B
  3. ID for A->B->C

We don't seem to be using the payload at...

vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11777 chore: bump the sdk-elixir group across 1 directory with 2 updates

Bumps the sdk-elixir group with 2 updates in the /sdk/elixir directory: credo and ex_doc.

Updates credo from 1.7.15 to 1.7.16

Release notes
Sourced from credo's releases.

v1.7.16
Check it out on Hex: https://hex.pm/packages/credo/1.7.16

Fix compatibility & compiler warnings with Elixir 1.20.0-rc.1
Credo.Check.Refactor.PassAsyncInTestCases add new param :force_comment_on_explicit_false (defaults to false)
Cre...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11781 chore: bump preact from 10.26.9 to 10.28.3 in /docs

Bumps preact from 10.26.9 to 10.28.3.

Release notes
Sourced from preact's releases.

10.28.3
Fixes

Avoid scheduling suspense state udpates (#5006, thanks @​JoviDeCroock)
Resolve some suspense crashes (#4999, thanks @​JoviDeCroock)
Support inheriting namespace through portals (#4993, thanks @​JoviDeCroock)

Maintenance

Update test with addition of _original (#4989, thanks @​JoviDeCroock)

10.28.2
Fixes

Enforce strict equality for VNode object construct...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11782 chore: bump cross-spawn from 7.0.3 to 7.0.6 in /sdk/rust/crates/dagger-sdk/examples/multi-stage-build/app

Bumps cross-spawn from 7.0.3 to 7.0.6.

Changelog
Sourced from cross-spawn's changelog.

7.0.6 (2024-11-18)
Bug Fixes

update cross-spawn version to 7.0.5 in package-lock.json (f700743)

7.0.5 (2024-11-07)
Bug Fixes

fix escaping bug introduced by backtracking (640d391)

7.0.4 (2024-11-07)
Bug Fixes

disable regexp backtracking (#160) (5ff3a07)

Commits

77cd97f chore(release): 7.0.6
6717de4 chore: upgrade standard-version
f700743 fix: upda...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11785 Fix case where disk full prevents client DB open

When storage fills up, opening the client telemetry DB can fail, but aborting the session prevents telemetry from flushing or cleanup from progressing — a catch-22 where the engine can’t proceed to recover.

This change logs and continue on keepalive DB open errors, so clients are at least not stuck waiting for a clientdb prune.

There's probably a lot more we could do here, but this seems like a reasonable start.

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11786 cli: embed default license to avoid network dep

If a LICENSE isn't found during dagger init, we create one for you by downloading it from spdx.org.

This creates a network dep, which is especially problematic when there's networking errors from spdx.org and our tests flake out.

It also is probably somewhat impolite to download hundreds of copies of the same license file each CI run.

This changes the default license only (apache 2.0) to be embedded so we can avoid the network dep in that case.

A likely better long-term fix would ...

ornate vigilBOT
#
vmiko
[dagger] New discussion #11787: How to export files or directories from the runtime container

Hi everyone,

Does anyone know how to export files or directories from the runtime container?

I followed these cookbook examples (thanks for them, btw):

https://docs.dagger.io/cookbook#copy-a-file-to-the-dagger-module-runtime-container-for-custom-processing
https://docs.dagger.io/cookbook#export-a-directory-or-file-to-the-host

However, I can’t find any way to chain an export from within the runtime container.
More specifically, I have an entrypoint module that runs and manages ot...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11789 engine: workaround ancient kernels in container start

Turns out some infra providers like AWS CodeBuild are somehow still on Linux kernel 4.14 (despite that one officially being 2y past EOL).

The experience on these kernels is not gonna be ideal in terms of performance but right now the only hard blocker is our use of open_tree and related syscalls during container start (to avoid overlayfs leak issues).

It's pretty easy to just skip that when starting containers, so we may as well to unblock users on CodeBuild. Overlay leaks result in var...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11793 Add support for sockets in Directory.dockerBuild

Sockets have to be explicitly provided as arguments to the API. If so, then RUN --mount=type=ssh will use that socket.

So if you want to have a function that does a DockerBuild with an SSH socket the code might be like:

func (m *MyModule) DoDockerbuild(dir *dagger.Directory, sshSock *dagger.Socket) *dagger.Container {
  return dir.DockerBuild(dagger.DirectoryDockerBuildOpts{SSH: sshSock})
}

And can then be called like this (or similar):

dagger call do-dockerbuild ...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11795 fix case where cache mounts become unmountable after restart

When the engine started, it was checking mutable mounts for any leftover volatile dirs. The way it did this resulted in the "shared" mount pool needed by overlay cache mounts being skipped. This also ended up being saved as the mounts to use for that ref going forward, which meant any concurrent use of the overlay cache mount would be invalid due to duplicate work/upper dirs and also fail due to volatile existing.

Fix is to just ensure the shared mount pool is correctly used in all cases.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11799 speed up module loading in large monorepos

Module loading was specifying an include that had a **/* appended, which appears to create a huge slowdown in monorepos with large directory trees in terms of number of files/dirs.

This specifically applies to a case like:

  1. Big monorepo
  2. A dagger module subdir that has relatively few files

Before this change, the **/* pattern seemed to be possibly be resulting in walks/checks of the full tree. I didn't take time yet to diagnose why exactly that happens and if it's expected or ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
chrisjpalmer
[dagger/dagger] Issue opened: #11802 🐞 Go codegen is non deterministic sometimes

What is the issue?

When running dagger develop multiple times on the same module, ocassionally the output is different. Specifically the ordering of a particular module type's .MarshalJSON, .UnmarshalJSON functions is changing in the /dagger.gen.go.

I actually found this issue rarely happens so I don't have an exact repro, however I believe I have isolated the root cause of the non-determinism and can show it [using this example repo](https://github.com/chrisjpalmer/dagger-co...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
rrati
[dagger/dagger] Issue opened: #11805 🐞 lchown operation not permitted

What is the issue?

When doing a dagger install .. in a CI pipeline the task can fail with:

error setting dir metadata for / lchown / not permitted

The full error from logs looks like:

✘ .export(path: ""): String! 0.0s ERROR
✘ export directory / to host  0.0s ERROR
! error from receiver: error setting dir metadata for /: lchown /: operation not permitted

In this use case, the dagger modules each have tests in a tests directory. i.e. daggerverse/module/tests.

As part ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
fredleger
[dagger/dagger] Issue opened: #11814 ✨ [DAGGERVERSE] why daggerverse is not currated

What are you trying to do?

After a few talks with some fellow daggernauts, we are all wondering why there is not some kind of curation on the daggerverse.

  • Docker Hub has pulls + stars + last updated criteria
  • Github has stars and forks
  • Packagist has Installs Stars Watchers Forks
  • You've got the idea...

IMHO, the daggerverse desperately needs an addition of more criteria or a manual curation to go one step further.
This is a break in the adoption of dagger generally, as I see it.
...

vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #11815 0.19.11 breaks modules importing generated telemetry package

Problem

As of 0.19.11, generated clients are smaller: they import dagger.io/dagger for core types, instead of duplicating it. This includes dagger.io/dagger/telemetry.

This broke a small number of modules (all internally used by us) that manually imported their generated dagger/telemetry package, to use our experimental otel features (attribute definitions etc). As a result these modules fail to load with v0.19.11.

Known affected modules:

Directly affected:

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11818 fix: core/integration test binary compiles on macOS

Summary

  • Export CopyFileContent on darwin in internal/fsutil/copy to match the linux API
  • Extract syscall.Mount/syscall.Unmount bind-mount calls from core/changeset.go into platform-specific helpers
  • Move 4 Linux-only functions from core/git_remote.go (runWithStandardUmaskAndNetOverride, unshareAndRun, overrideNetworkConfig, runProcessGroup) into git_remote_linux.go
  • Move mountIntoContainer from core/service.go into service_linux.go
  • Move `pathResolverForM...
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11819 chore: update uv-build requirement from <0.10.0,>=0.8.4 to >=0.8.4,<0.11.0 in /sdk/python in the sdk-python group

Updates the requirements on uv-build to permit the latest version.
Updates uv-build to 0.10.0

Release notes
Sourced from uv-build's releases.

0.10.0
Release Notes
Since we released uv 0.9.0 in October of 2025, we've accumulated various changes that improve correctness and user experience, but could break some workflows. This release contains those changes; many have been marked as breaking out of an abundance of caution. We expect most users to be able t...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11820 chore: bump the sdk-java group in /sdk/java with 2 updates

Bumps the sdk-java group in /sdk/java with 2 updates: com.palantir.javapoet:javapoet and io.opentelemetry:opentelemetry-bom.

Updates com.palantir.javapoet:javapoet from 0.10.0 to 0.11.0

Release notes
Sourced from com.palantir.javapoet:javapoet's releases.

0.11.0
No documented user-facing changes

Commits

035c1f3 Release 0.11.0
a020cf1 [High Priority] Excavator: Upgrades baseline-error-pron...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11823 chore: bump astral-sh/ruff from 0.14.14 to 0.15.0 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.14.14 to 0.15.0

Release notes
Sourced from astral-sh/ruff's releases.

0.15.0
Release Notes
Released on 2026-02-03.
Check out the blog post for a migration guide and overview of the changes!
Breaking changes

Ruff now formats your code according to the 2026 style guide. See the formatter section below or in the blog post ...

vivid lintelBOT
#
cbochs
[dagger/dagger] Issue opened: #11824 🐞 Module loading slows down significantly with modest `.env` files

What is the issue?

I have a larger repository which I've adopted to use Dagger. This repository contains a .env file with ~220 env variables, all unrelated to Dagger, which is copied from a .env.template for local dev. I've noticed significant slow downs to module loading when the .env file is present (~1.9s w/o .env, ~26s w/ .env).

This directly impacts all uses of Dagger within this repository.

There looks to be some prior discussion on configuring the env file loading in h...

#
cbochs
[dagger/dagger] Issue opened: #11825 🐞 Engine panics with `unknown type` (`GetOrInitializeWithCallbacks`, `/app/engine/cache/cache.go`)

What is the issue?

I'm not certain how to this occurred, but I wanted to record the error here. I was refactoring an existing module, moving/deleting code (a Release struct) from one module (mod A) to a new one (mod B). When I called dagger functions on mod A the CLI errors with:

$ dagger functions

✘ load module: . 2.2s ERROR
✘ initializing module 0.3s ERROR
! Post "http://dagger/query": command [docker exec -i dagger-engine-v0.19.11 buildctl dial-stdio] has
  exited with exit ...
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #11826 feat: correctly pin the go pkg in module genera

Update the go_sdk to send a pinned lib version to a commit from dagger-go-sdk.

⚠️ This change the release process, now we will need to update the const in go_sdk to the latest commit of dagger-go-sdk, this commit will not be the one tagged because it's done before the release.
At some point, it would be much better to release the SDKs before the engine so we can set the commit of the released instead of the one before.

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11828 core: use content digests for secrets+sockets; fix git SSH caching

Fixes #11765

Couple of interrelated high-level goals here.

One goal is to reduce usage of "custom recipe digests" and replace them with content digests. This is important for the dagql cache upgrades as recipe digests will be changed to always be purely "recipe", with any additional digests (content, "semantic", etc.) modeled as separate digests. That in turn makes the whole system much simpler algorithmically and makes digests less opaque ("is this a true recipe digest or a cus...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11830 let local-cache prune use explicit space limits

Before this change, manual local-cache pruning only accepted useDefaultPolicy, so teams running with gc=false could not reuse their configured space thresholds and had to choose between all-or-nothing pruning behavior.

After this change, engine.localCache.prune accepts explicit space overrides and applies them per call, enabling controlled manual pruning windows without re-enabling automatic GC.

Added test coverage in core/integration/localcache_test.go, test case `TestLocalCachePru...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
matipan
[dagger/dagger] Pull request opened: #11835 engine: fix +ignore pragma for constructors

Fixes #11750.

The issue is caused by a check on setCallInputs:

if len(arg.metadata.Ignore) > 0 && !arg.metadata.isContextual() { // contextual args already have ignore applied

In this context an argument is considered contextual if either the DefaultPath or DefaultAddress is set. If it is then setCallInputs will not go through the ignore process of directories. The problem: arguments that are contextual (they have a default path) but a user specified a value ...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11837 Avoid WithExec panics and surface underlying exec errors

Before this change, Container.WithExec could panic with "index out of range" while handling exec failures. The failure-handling path assumed mount and output slices were always aligned, and a deferred block could run even when mount preparation had already failed.

e.g.

 panic: runtime error: index out of range [0] with length 0

 goroutine 4176537 [running]:
 github.com/dagger/dagger/core.(*Container).WithExec.func2()
         /app/core/container_exec.go:299 +0x1198
 githu...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11838 [wip] make dagql cache ID and Result aware

This is unrefined slop atm, but directionally correct. Just want to kick off some full CI checks for errors and/or perf impact. Will do human cleanup before bringing out of draft.

Goal is to make the base cache aware of more dagql primitives, specifically of IDs and Results (whereas previously cache was decoupled and generic).

This is another crucial step for replacing the buildkit solver and enables a lot of simplifications + new features.

This is one of those situations wher...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11842 fix(container): make PrepareMounts clean up its own refs on error

Follow-up of https://github.com/dagger/dagger/pull/11837

PrepareMounts allocates refs in a loop and can fail partway through. Before, callers were responsible for cleaning up partial refs on error. This was fragile: a caller refactor could silently introduce leaks.

Now PrepareMounts cleans up after itself:

  • on success, caller owns the refs
  • on failure, PrepareMounts releases any refs it created

Includes a regression test for the partial-failure path.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #11844 add `dagger trace` for viewing a trace
  • retire dagger watch - stopped working ages ago I think?
  • add dagger trace [--org ]

Currently, trace streams payloads via the existing Cloud GraphQL APIs, converts them back into OTLP data, and exports them back into the local frontend. Alternatively we could add OTLP fetching endpoints to Cloud, and I had a separate PR for that, the main advantage being that you don't need to specify an org, but this PR immediately works without having to bikeshed that and coordinate merging, so...

vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11845 cmd/dagger: hint that module ref is overridden on errors

Context

@nipuna-perera hit a Jenkins-only failure on dagger toolchain install.

The command was resolving the module from DAGGER_MODULE (remote git ref) instead of the local workspace. For local-mutating commands, that leads to: kind must be "local", got "git".

Problem

The error did not explain where the module ref came from, so the fix path was unclear.

Approach

Keep behavior unchanged, impro...

vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11846 feat: expose OriginalModule on checks and generate functions

Store and expose the reference to the original module in which the check or generate has been defined (and by extension to any ModTreeNode).
For instance if a check is defined in a Toolchain, Module will remain the root module while OriginalModule will be the Toolchain's module.
That way we can access properties like ModuleSource from a check or generate function.

The OriginalModule is exposed in the schema, I'm not sure it's needed or not. If it's not required I'd be...

ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11850 engine: harden BoltDB settings to avoid page-freed panics under NoSync

Before this change, several engine BoltDBs ran with NoSync only, and crashy/shutdown-edge cases could still surface bbolt corruption panics like page ... already freed.

After this change, every BoltDB in this tree that already uses NoSync also sets NoFreelistSync and NoGrowSync, aligning the option set across all DB open paths and matching the mitigation pattern used in containerd.

This updates all four relevant init points: snapshotter metadata store options, containerd metad...

vivid lintelBOT
#
kevinschoonover
[dagger/dagger] Issue opened: #11851 🐞 dagger cache invalidation not respecting intermediate steps

What is the issue?

I have a dagger module where I am
(1) creating a hashicorp/vault server with -dev flag
(2) initializing that vault server with a bunch of policies
(3) initializing a nomad instance running on my local docker using

but I am running into an issue where (2) and (3) aren't always run. Upon further investigation, it looks like its just skipping those initialization steps because its being cached. I created a minimal reproduction of ...

vivid lintelBOT
vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11853 ✨ Use generate functions

Replace manual generate function exposed in the main .dagger module by dagger generate command:

  • add +generate pragma and @generate directive

  • rename some of the generate functions to better express what they are doing

    $ dagger generate -l
Name                        Description
changelog:generate          Generate the changelog with 'changie merge'. Only run this manually, at release time.
docs:references             Regenerate the API ...
vivid lintelBOT
vivid lintelBOT
#
moltar
[dagger/dagger] Issue opened: #11854 Solver cache poisoning: stale session IDs in actives map cause permanent engine lock-up

Bug Description

After a Dagger session completes (or fails), the BuildKit solver's in-memory vertex cache (actives map) can retain stale session ID references. When a new session connects and resolves the same LLB vertex digests (same Dagger module, same code), the solver reuses the cached vertex state. The reused state's NextSession() iterator walks parent states and finds the dead session ID, causing a 5-second timeout per cache key lookup. Every subsequent job fails within ~10 seco...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11855 engine: fix safeToPersistCache propagation for nullables

Nullable return types from functions weren't propagating from the Nullable-wrapped result to the de-ref'd un-null result.

This meant nullable return types from functions would not get cached. It seems most SDKs end up using Non-nullable types for the most part in practice, but we just hit this in Dang where you can easily specify a type as Nullable.

Along the way I noticed a quirk with the sqlite db TTL checks in that we were always following the "TTL expired" branch even when there act...

vivid lintelBOT
vivid lintelBOT
#
kpenfound
[dagger/dagger] Pull request opened: #11857 ci: gracefully handle git worktrees

This is so that we can develop dagger/dagger with git worktrees. This should greatly simplify agent setups. I tested locally and it looks good, I can dagger-dev -m version call version from a regular branch or a worktree and it works correctly. Using main dagger the same command does not work in a worktree.

Summary:
In git worktrees, .git is a file (not a directory) containing a gitdir reference. This caused Dagger to crash when trying to sync .git as a directory. This change adds workt...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11862 core(changeset): fix subdirectory scoping in changeset layers

Fixes #11656

Users expectchangeset.layer to follow 3 simple rules for scoped changesets:

  1. diff only what is inside the selected scope
  2. keep the resulting layer shape at that same scope
  3. if there are no scoped changes, the layer must be empty

Before this change, changeset.layer called before.Diff(after) directly. For scoped inputs, that bypassed the resolver diff path that normalizes both sides before diffing, which could lead to inconsistent scoped output, breaking 1 and ...

vivid lintelBOT
#
AsocPro
[dagger/dagger] Pull request opened: #11863 Add a local-defaults documentation for .env files

I was helped on discord learn about the .env file features for defaulting of arguments and since there was no documentation covering the feature I offered to add a quick page and was told that would be great.

I verified that the docs build and that they look correct and all links are functional when running the doc server. I also validated that all of the cases I documented here function the way they are documented.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11871 fix(sdk/go): make Go codegen ordering deterministic when positions are equal

Fixes #11802

Go module codegen had a nondeterministic edge case: when types or methods share the same source position (notably token.NoPos), Pos()-only sorting cannot define a stable order.

This showed up as occasional reordering of generated blocks (e.g. MarshalJSON / UnmarshalJSON) between identical runs, producing noisy diffs in generated code.

Add shared comparators with explicit tiebreakers (position, package path, name) and use them consistently across all codegen sorting...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
moltar
[dagger/dagger] Issue opened: #11873 Engine ignores OTEL_EXPORTER_OTLP_METRICS_ENDPOINT environment variable

What is the issue?

Setting OTEL_EXPORTER_OTLP_METRICS_ENDPOINT on the Dagger engine process has no effect — metrics are never exported to external OTLP collectors.

Root cause

cmd/engine/telemetry.go:53-55 calls telemetry.Init() without Detect: true:

ctx = telemetry.Init(ctx, telemetry.Config{
    Resource: otelResource,
})

When Detect is false (zero value), Init() at sdk/go/telemetry/init.go:398 skips the block that calls ConfiguredMetricExporter() — t...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
IljaKroonen
[dagger/dagger] Issue opened: #11882 🐞 Cannot pass a numerical-looking value to a string argument in toolchain customizations

What is the issue?

Error when trying to pass in a default that looks like a number

Dagger version

dagger v0.19.10 (docker-image://xxxinternalurl:0.1.115) linux/amd64

Steps to reproduce

  • Create a dagger module dagger init --sdk=go
  • Create a toolchain in a subdir toolchains/test-toolchain using dagger init --sdk=go again
  • In the first module you created run dagger toolchain install ./toolchains/test-toolchain/
  • in dagger.json add the customization below
  • run `dagge...
#
IljaKroonen
[dagger/dagger] Issue opened: #11883 🐞 dagger.json toolchain customizations uses surprising argument and function name casing

What is the issue?

The casing of the function and argument field seem to be the casing of the actual implementation.

For example in the go default example, the functionName is ContainerEcho. I don't think this is by design because it is surprising, considering other views on function names are standardized for the environment they are being used in. My first instinct was to use the names as shown by the CLI when using help commands or invoking things.

If it is by design and shoul...

vivid lintelBOT
#
IljaKroonen
[dagger/dagger] Issue opened: #11884 ✨ Toolchain-wide settings

What are you trying to do?

We are trying to build toolchains that offer a very standardized out-of-the-box experience with our templates. While doing this we noticed that it's not easy to design these without some arguments being repeated between the functions we expose.

For example, for our dotnet toolchain, we want to offer an override for the SDK image to be used. This is needed for pretty much all the features we expose and we end up with a large amount of repeated defaults in custo...

vivid lintelBOT
#
moltar
[dagger/dagger] Issue opened: #11885 fatal error: stack overflow in S3 cache export Finalize on second run

Description

When using _EXPERIMENTAL_DAGGER_CACHE_CONFIG with an S3 backend, the Dagger engine crashes with a Go stack overflow during the cache export phase on the second run of a pipeline. The first run succeeds and populates the S3 cache. Any subsequent run that imports this cache and then tries to re-export crashes the engine.

Environment

  • Dagger version: v0.19.11
  • Engine: Remote engine on EC2 (Amazon Linux 2023, c7i.xlarge24, 192GB RAM)
  • Cache config: `type=...
vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11886 chore: improve python lint checks
  • return container instead of an error. As returning a container the engine will sync to return the error, that's exactly what was done. By making the container available, it's easier to debug as we can dagger call python-sdk lint terminal
  • limit lint-docs-snippets check to lint docs... Previously it was also linting sdk/python that added confusion in check results as the same error appeared multiple time
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Issue opened: #11887 🐞 `check --scale-out` never fails

What is the issue?

dagger check --scale-out currently shows a passing check and exit status 0 even if checks failed.

There's a different in the local (to the engine) vs. scale-out paths.

The local path directly calls the underlying function, which will bubble up an error if the call returns it (or returns a Syncable object that fails on sync):

https://github.com/dagger/dagger/blob/42bb3b568c0a38671561a1f40e7a40cba49cde60/core/modtree.go#L151-L171

The scale-out path however just q...

vivid lintelBOT
vivid lintelBOT
#
Dudesons
[dagger/dagger] Issue opened: #11889 🐞 Json value is not correclty set in envfile object

What is the issue?

When I'm setting a json value in a variable for an envfile quotes are stripped.

ENV_JSON_ISSUE={foo: bar}
ENV_JSON_GOOD={"foo": "bar"}

Dagger version

dagger v0.19.11 (image://registry.dagger.io/engine:v0.19.11) linux/amd64

Steps to reproduce

import dagger
from dagger import dag, function, object_type

import json


@object_type
class DaggerBug:
    @function
    async def env_file_bug(self) -> str:

        return await (
            dag.
...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
idlsoft
[dagger/dagger] Pull request opened: #11891 feat(cmd/dagger): support path filters (include, exclude, gitignore) for dagger.Directory arguments

This is a simple client-side feature to complement the more comprehensive server-side dynamic filtering.

It will allow specifying include/exclude options for arguments of type dagger.Directory using a url-like format.
For example:
/path/to/monorepo?include=apps/app1,libs/lib1,libs/lib2&exclude=libs/lib1/testdata

Can be useful for builds that need a subset of directories.
For example a golang build, with replace support.

Works for command line args, as well a...

vivid lintelBOT
#
shepherdjerred
[dagger/dagger] Pull request opened: #11892 sdk/typescript: surface detailed error context in UnknownDaggerError

Summary

When a non-ClientError exception occurs during a GraphQL request (e.g. transport failures with kube-pod://, Docker socket issues, or container exec errors that don't come back as proper ClientError), the TypeScript SDK throws a generic:

Encountered an unknown error while requesting data via graphql

This makes debugging CI failures extremely difficult. Real build failures (like exited with code 1 or binary already exists in destination) get hidden behind this ge...

vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11893 chore: bump com.jayway.jsonpath:json-path from 2.10.0 to 3.0.0 in /sdk/java in the sdk-java group

Bumps the sdk-java group in /sdk/java with 1 update: com.jayway.jsonpath:json-path.

Updates com.jayway.jsonpath:json-path from 2.10.0 to 3.0.0

Release notes
Sourced from com.jayway.jsonpath:json-path's releases.

json-path-3.0.0
What's Changed

adding jackson3 provider (#1066) by @​bsa01 in json-path/JsonPath#1067
Prepare Version 3.0.0 - Java 17 baseline with Jackson 3 by @​kallestenflo in json-path/JsonPath#1072

New Contributors

@​bsa01 made their ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11894 chore: bump astral-sh/ruff from 0.15.1 to 0.15.2 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.15.1 to 0.15.2

Release notes
Sourced from astral-sh/ruff's releases.

0.15.2
Release Notes
Released on 2026-02-19.
Preview features

Expand the default rule set (#23385)
In preview, Ruff now enables a significantly expanded default rule set of 412 rules, up from the stable default set of 59 rules. The new rules are mostly...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11902 Fix accessing nodes (check, generate) from objects

[!NOTE]
This is a follow up to https://github.com/dagger/dagger/pull/11827 that was (partially) solving https://github.com/dagger/dagger/issues/11811

Problem | State

check and generate functions must be listed and callable.

When check and generate functions are defined on the main object of a module, it's working.

#11827 was solving the case where check and generate functions are defined on a different object of a module. But this was working only on the "main"...

vivid lintelBOT
vivid lintelBOT
#
vfouqueron
[dagger/dagger] Issue opened: #11904 🐞 failed to get type defs json during module sdk codegen

What is the issue?

I try to run tests for a pipeline written with the Typescript SDK following the documentation (https://docs.dagger.io/reference/best-practices/modules). When I try to launch the test, it fails with a failed to get type defs json during module sdk codegen error

Dagger version

dagger v0.19.11 (image://registry.dagger.io/engine:v0.19.11) darwin/arm64/v8

Steps to reproduce

I followed the steps described:

mkdir tests
cd tests
dagger init --name=tests --sd...
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11905 fix(typescript): avoid ts-introspector crash on empty nested src dirs

Fixes #11904

Problem

A user reporting getting this module init failure:

failed to get type defs json during module sdk codegen
TypeError: reduce of empty array with no initial value

Root cause

getTsSourceCodeFiles recursively scans src/, and each level reduces readdirSync(dir).map(...) without an initial accumulator.

To trigger the error, the module does not need to be empty: any empty nested directory under src/ can trigger this panic.

...

vivid lintelBOT
vivid lintelBOT
#
verdverm
[dagger/dagger] Issue opened: #11907 Errors hidden by `dagger run`

What is the issue?

# tony @ hydrogen in ~/repros/dagger/obf-errs [12:15:27] C:1
$ dagger run --progress report go run ./main.go 
✔ connect 0.2s

Full trace at https://dagger.cloud/verdverm/traces/8316244cfb3c1372a1510d8bd1c68d24

A new release of dagger is available: v0.19.10 → v0.19.11
To upgrade, see https://docs.dagger.io/install
https://github.com/dagger/dagger/releases/tag/v0.19.11

# tony @ hydrogen in ~/repros/dagger/obf-errs [12:15:43] C:1

package main

impor...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11909 core/git: default remote fetches to `--no-tags`, restore SHA compatibility fallback (~80% faster)

Problem

The remote fetch hot path currently uses git fetch --tags unconditionally. That is expensive on tag-heavy repos and makes branch/commit/tree flows pay global tag transfer cost even though tag APIs and ref resolution already come from ls-remote.

There is a real drawback to removing --tags: some Git servers/proxies reject direct SHA fetches (not our ref, unadvertised object, couldn't find remote ref). BuildKit historically handled this with conditional behavior (`--n...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11910 python-sdk-dev: split tests by py version and type

Introduce a specific object type exposing unit and slow check function executing tests.
Instead of relying on the python version matrix and handling the parallelism when calling the tests, use the built-in check behaviour. All the check functions will be run in parallel.

This change gives more visibility and more control to the developer to run python sdk tests.

Previously only one single check was available regarding those tests:

$ dagger check -l
...
python-sdk...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
matipan
[dagger/dagger] Pull request opened: #11916 drivers: add TLS driver for engine clients

Adds a TLS driver for clients behind a new tls:// protocol for _EXPERIMENTAL_DAGGER_RUNNER_HOST.

Main motivation for this change was in the context of Native CI. We've been debating for more direct ways to send smart information to our scheduler that do not require going through the CLI with custom cloud-only parameters. Connecting to a cloud engine today requires a TLS certificate, and we are already doing that in the Cloud driver. So I thought: easier to just provision the engine, get ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11918 fix invalid gorelease config

Previous attempt to remove windows/armv7 builds of CLI wasn't quite right because the goreleaser config had a duplicate name in it. I ran with --dry-run at the time but apparently that skipped goreleaser. CI also only runs goreleaser on main, so missed there too.

Fixed the problem, this time I tricked goreleaser into actually running (without actually uploading anything), so more confident in the fix this time around.

Will need this before next release

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
ornate vigilBOT
#
marcosnils
[dagger/dagger] New comment on discussion #11924: Can dagger be scaled across multiple machines? If so, how?

Hey there!

Dagger's design and architecture has the ability to scale acorss multiple machines but there's currently no user-facing when to set this up. This is mostly because it involves a considerable amount of effort between infrastructure, observability and the runtime.

We are currently in the testing phase a of a Dagger hosted CI service which provides this capability. If this is something you think you're interested, we'd like to chat.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #11928 dagger native dockerfile implementation

Required for https://github.com/dagger/dagger/pull/11856 but technically can be done independently off main and saves that PR from more diff

Still WIP but very good progress so far. Almost all the Dockerfile integ tests in our repo are passing already.

The diff here is not as bad as it seems; there's thousands of lines of redundant tests from earlier iterations of it that can be deleted. Also almost a thousand lines for the WHITEBOARD.md file used to persist LLM work+state. The converte...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
nipuna-perera
[dagger/dagger] Pull request opened: #11929 Implement OIDC integration for vault secrets provider

Summary

Adds OIDC support to the vault secretsprovider. This falls back to oidc auth of other forms of auth envs are not provided (making it the default). It supports opening the browser and waiting for response before proceeding to obtain vault token.

AI Generated Summary

This pull request introduces comprehensive support for Vault OIDC (OpenID Connect) authentication in the secret provider, including robust token caching, fallback, and end-to-end testing. The main focus is to...

vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11930 chore: bump org.mockito:mockito-core from 5.21.0 to 5.22.0 in /sdk/java in the sdk-java group

Bumps the sdk-java group in /sdk/java with 1 update: org.mockito:mockito-core.

Updates org.mockito:mockito-core from 5.21.0 to 5.22.0

Release notes
Sourced from org.mockito:mockito-core's releases.

v5.22.0
Changelog generated by Shipkit Changelog Gradle Plugin
5.22.0

2026-02-27 - 6 commit(s) by Joshua Selbo, NiMv1, Rafael Winterhalter, dependabot[bot], eunbin son
Avoid mocking of internal static utilities (#3785)
Bump graalvm...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11933 chore: bump astral-sh/ruff from 0.15.2 to 0.15.4 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.15.2 to 0.15.4

Release notes
Sourced from astral-sh/ruff's releases.

0.15.4
Release Notes
Released on 2026-02-26.
This is a follow-up release to 0.15.3 that resolves a panic when the new rule PLR1712 was enabled with any rule that analyzes definitions, such as many of the ANN or D rules.
Bug fixes

Fix panic on access to ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11936 chore(llm): refresh default model versions

The built-in LLM defaults have drifted behind current provider recommendations. This updates the core Anthropic/OpenAI defaults and keeps CLI/docs examples aligned so fresh sessions start with sensible, current model names.

What changed

  • update core default Anthropic model to claude-sonnet-4-6
  • update core default OpenAI model to gpt-5.2
  • keep core default Google model at gemini-2.5-flash
  • add OpenRouter alias mapping for claude-sonnet-4-6 -> anthropic/claude-sonnet-4.6
  • u...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
eunomie
[dagger/dagger] Pull request opened: #11940 Merge changesets function

Add a root function that allows to merge an array of changesets.

Before, with changes an array of changesets:

changes[0].WithChangesets(changes[1:])

After:

dag.MergeChangesets(changes)

The benefit is essentially to not have to worry about the size of changes (and so if changes[0] will panic).

[!NOTE]
I don't know about the name, I was also thinking about dag.Changesets(changes)

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11943 chore: bump posthog-js from 1.356.2 to 1.357.1 in /docs in the docs group

Bumps the docs group in /docs with 1 update: posthog-js.

Updates posthog-js from 1.356.2 to 1.357.1

Release notes
Sourced from posthog-js's releases.

posthog-js@1.357.1
1.357.1
Patch Changes

#3149 91223c5 Thanks @​adboio! - avoid re-checking URLs if they have not changed
(2026-03-02)
Updated dependencies [5e8d5fc]:

@​posthog/core@​1.23.2
@​posthog/types@​1.357.1

posthog-js@1.357.0
1.357.0
Minor Changes

#3169 4f885c0 Thanks @​marandaneto! - f...

vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #11945 codegen: include internal/telemetry/

Currently loadPackage fails with no packages found in . when we make changes to go.mod in the root of the repo. To debug, I added a go mod tidy immediately before the packages.Load, suspecting that it's silently failing. go mod tidy revealed that the module's generated code was referring to
github.com/dagger/dagger/modules/go/internal/telemetry which indeed did not exist. Including it in this spot fixes the issue, but it's still a bit mysterious as to how it's been working up u...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #11946 cmd/dagger: batch `check -l` fetch and drop noisy query subtree tracing

Switch dagger check -l to a single bulk GraphQL request (dag.Do with CheckGroupListDetails) instead of per-check field fetches. When using --cloud, this avoids a roundtrip from client server.

This is the simple fix to a more deep issue: following the Dagql-ification, we dropped the eager return of object fields. At the moment, accessing the fields requires a roundtrip to the engine. [It would be nice to have a way to specify that some fields as safe to be returned as part of ...

vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11947 chore: bump the docs group across 1 directory with 2 updates

Bumps the docs group with 2 updates in the /docs directory: posthog-js and markdownlint-cli.

Updates posthog-js from 1.356.2 to 1.358.0

Release notes
Sourced from posthog-js's releases.

posthog-js@1.358.0
1.358.0
Minor Changes

#3165 0e08337 Thanks @​dustinbyrne! - feat: Tree-shake surveys, toolbar, exceptions, conversations, logs, experiments
(2026-03-03)

Patch Changes

#3164 20c1ff2 Thanks @​...

vivid lintelBOT
#
grouville
[dagger/dagger] Issue opened: #11948 Python SDK codegen does not handle nullable object fields and crashes on subfield access

Context

Container.dockerHealthcheck (as an example, but this might probably be hit by other API methods) can resolve to null, with the docker equivalent of:

FROM alpine
HEALTHCHECK NONE

When exposed using the Dagger API, the Python codegen exposes docker_healthcheck() as a concrete HealthcheckConfig type and scalar accessors (shell(), args()) as non-null values.

This leads to an error, at runtime, in which the Python SDK raises **Required field got a null re...

#
eunomie
[dagger/dagger] Pull request opened: #11950 cmd/dagger: batch `generate -l` fetch and drop noisy query subtree tracing

Switch dagger generate -l to a single bulk GraphQL request (dag.Do with GeneratorGroupListDetails) instead of per-generator field fetches.

Run that list-query subtree with a discarded tracing context so per-generator name/description spans are not emitted, while keeping CLI behavior/output unchanged.

This is a follow up to https://github.com/dagger/dagger/pull/11946 that did it for dagger check -l

#
alexcb
[dagger/dagger] Pull request opened: #11951 feat: use HEALTHCHECK commands with services

If a docker healthcheck command is defined (either from a Dockerfile HEALTHCHECK command, or by using the WithDockerHealthcheck() api), it will be used instead of the exposed ports to determine when a service is ready.

To revert to the old behavior of using the exposed ports, one can use the WithoutDockerHealthcheck() call before defining a service.

e.g.

srv := c.Container().
    From("some-image-with-a-healthcheck").
    WithExposedPort(8080).
    WithoutDockerHealthchec...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #11952 tui: switch to tuist

This PR reimplements our "pretty" TUI frontend using Tuist, a new TUI framework. The main benefit is scrollback handling. Bubbletea is constrained to your viewport, and requires you to painfully track items as they go out of view and print them on your own, where they become "frozen" - completely out of bounds of Bubbletea, unable to be repainted or interacted with at all. Tuist, on the other hand, supports an infinite scrollback, with an efficient rendering...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #11953 feat: add schema manipulation utilities in codegen

Add utilities functions to include/exclude types in the GraphQL schema that comes from dependencies.
This is a follow up on https://github.com/dagger/dagger/pull/11896 that uses these changes.

The ultimate goal is to follow up that PR with another one that will generate dependency bindings in specific files instead of being in dagger.gen.go.

The tests were written by getting the introspectionJSON schema locally on a module Test that has a dependency Dep then I simplified that fil...

vivid lintelBOT
#
Neirda24
[dagger/dagger] Issue opened: #11954 ✨ Add multi context docker builds

What are you trying to do?

I'd like to be able to have more than one context for docker build from Dockerfile.

Why is this important to you?

It would ease quite a lot the adoption of dagger on projects with massive Dockerfile to migrate over to dagger. That way we could focus on orchestrating other parts of the CI instead of the build itself.

How are you currently working around this?

I am not.

vivid lintelBOT
vivid lintelBOT
#
kpenfound
[dagger/dagger] Pull request opened: #11956 add --failfast flag for `dagger check` and WithFailFast in util/parallel

This extends util/parallel to have WithFailFast which uses github.com/sourcegraph/conc/pool's WithCancelOnError to cancel remaining checks as soon as one check fails.

My main motivation with this is to give agents a faster feedback loop when running dagger check. I can't really see the use of this in CI or for humans.

The next step would be to enable failfast for the go tests inside our checks through an optional flag so that we dont need to wait for entire suites to fail.

vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
hayaiti
[dagger] New discussion #11961: Dagger for CI/CD of a financial data platform — stock data pipeline automation

Using Dagger to automate the CI/CD pipeline for a stock market data platform. The programmable pipeline approach works great for complex data workflows.

Why Dagger for Financial Platforms

  1. Reproducible builds — Same pipeline locally and in CI
  2. Caching — Don't rebuild unchanged data processors
  3. Parallel steps — Run tests, lint, and data validation concurrently
  4. Programmable — Complex deployment logic in real code, not YAML

Pipeline Steps

1. Build API serve...
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11963 workspace: require absolute workspace paths and enforce repo boundary

Summary

  • define the Workspace API desired state in spec/workspace-api.md
  • enforce absolute path inputs for Workspace.directory, Workspace.file, and Workspace.findUp(from:)
  • resolve workspace paths directly in host/repo context and reject paths outside the workspace repo root
  • keep host filesystem access routed through the owning workspace client metadata
  • remove allowOutsideRepo for now
  • regenerate GraphQL/docs/SDK bindings for updated API docs

Why

This tightens works...

vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
hayaiti
[dagger] New discussion #11966: Stock Market Data Pipeline Architecture - CI/CD for Financial Data

Financial Data Pipeline Architecture

Sharing the CI/CD and data pipeline architecture for a stock market platform:

Pipeline Components

Data Sources (5 providers)
    ↓
Ingestion Layer (BullMQ workers)
    ├── Stock sync (6K+ symbols, 30s)
    ├── Options sync (99 Tier 1 chains)
    ├── News ingestion (22 RSS feeds + Finnhub + SEC)
    ├── Fundamentals sync (Finnhub basic_financials)
    └── Bar backfill (5yr daily for 290 stocks)
    ↓
Processing Layer
    ├── Price aggregation (...
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11967 skill: add dang-dagger-modules for authoring .dang modules

Summary

  • add a new dang-dagger-modules skill for creating, editing, and debugging Dagger modules written in Dang
  • include a full Dang language reference focused on Dagger module development patterns
  • add editor setup instructions (Zed, VS Code, Neovim) and troubleshooting guidance
  • document nullable type-handling patterns and common pitfalls seen in real module work

Testing

  • not run (docs/skill content only)
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
alexcb
[dagger/dagger] Issue opened: #11972 🐞 dockerfile RUN heredoc fails

What is the issue?

dockerfile-build is unable to build a dockerfile that runs a heredoc script.

Dagger version

dagger v0.20.0 (image://registry.dagger.io/engine:v0.20.0) linux/amd64

Steps to reproduce

Consider the following Dockerfile:

# syntax=docker/dockerfile:1
FROM alpine
RUN < data

Then try to build it with dagger:

dagger -c 'directory | with-file . $(host |file "Dockerfile") | docker-build | sync'

it fails with:

✘ sync 1.2s ERROR
! p...
vivid lintelBOT
vivid lintelBOT
#
langri-sha
[dagger/dagger] Issue opened: #11974 ✨ Add support for custom `tsc` configuration file

What are you trying to do?

I'd like if we could have support to specify a custom tsconfig.json for the TypeScript SDK, something along the lines of:

{
  "sdk": {
    "source": "typescript",
    "config": "tsconfig.dagger.json"
  },
  "source": ".dagger"
}

Why is this important to you?

I have a monorepo that is using composite projects, so I'd like if my language server and tools can run type checks and include my Dagger sources as well, without having to have a separate ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
lukas-mertens
[dagger/dagger] Issue opened: #11976 🐞 Hash mismatch after newest release

What is the issue?

With the newest release I am getting a hash mismatch in my pipeline:

Run dagger/dagger-for-github@v8.3.0
  with:
    version: latest
    cloud-token: ***
    dagger-flags: --progress plain
    verb: call
    workdir: .
    enable-github-summary: false
  env:
    _EXPERIMENTAL_DAGGER_RUNNER_HOST: tcp://my-url:1234
Run set -o pipefail
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Tota...
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Issue opened: #11978 ✨ Reproducible builds Dagger itself

What are you trying to do?

Building Dagger release archives should produce bit-for-bit identical assets on every run.

Why is this important to you?

For v0.20.1 our publish job flaked out, and we had to delete the GitHub release in order to re-run it (not idempotent - separate issue). When publish finally succeeded, we had a mix of opinions on what the checksums should be for our archives, between Nix, our checksums.txt on dl.dagger.io, our CDN, our S3 bucket, etc. - a huge ...

vivid lintelBOT
#
moltar
[dagger/dagger] Issue opened: #11979 Add option to disable CLI update notification

The Dagger CLI checks for new versions on every invocation and prints a notification to stderr when an update is available. There's currently no way to disable this.

This can be noisy in CI environments and automated scripts where upgrading isn't desired or is managed externally (e.g., pinned versions in Dockerfiles, Nix, Homebrew).

A common pattern in other CLI tools (Terraform, GitHub CLI, etc.) is to support an environment variable like DAGGER_NO_UPDATE_CHECK=1 to suppress the notifica...

ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
#
Jaredw2289-svg
[dagger/dagger] Pull request opened: #11981 cli: add DAGGER_NO_UPDATE_CHECK to suppress update notifications

Fixes #11979

Summary

  • Adds support for the DAGGER_NO_UPDATE_CHECK environment variable to suppress the CLI update notification that runs on every invocation
  • When set to any non-empty value (e.g., DAGGER_NO_UPDATE_CHECK=1), the automatic version check is skipped entirely, avoiding both the network request and the stderr output
  • This is useful in CI environments and automated scripts where CLI versions are pinned and managed externally

Details

The change is a 4-line early retu...

vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #11985 chore: bump credo from 1.7.16 to 1.7.17 in /sdk/elixir in the sdk-elixir group

Bumps the sdk-elixir group in /sdk/elixir with 1 update: credo.

Updates credo from 1.7.16 to 1.7.17

Release notes
Sourced from credo's releases.

v1.7.17
Check it out on Hex: https://hex.pm/packages/credo/1.7.17

Credo.Check.Readability.ModuleDoc add new param :ignore_modules_using (defaults to [Credo.Check, Ecto.Schema, Phoenix.LiveView, ~r/.Web$/])
Credo.Check.Warning.UnusedOperation update :modules param: instead of a list of functions to check, :all...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #11986 chore: bump astral-sh/ruff from 0.15.4 to 0.15.5 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.15.4 to 0.15.5

Release notes
Sourced from astral-sh/ruff's releases.

0.15.5
Release Notes
Released on 2026-03-05.
Preview features

Discover Markdown files by default in preview mode (#23434)
[perflint] Extend PERF102 to comprehensions and generators (#23473)
[refurb] Fix FURB101 and FURB103 false positives when I/O varia...

vivid lintelBOT
#
perwestling
[dagger/dagger] Issue opened: #11988 🖐️ Frequent Error 429 when using dagger-for-github

What happened? What did you expect to happen?

Have reported this issue on Discord

We do use some GitHub workflows that has jobs that run several actions that use dagger.
And this for a number of workflows that are run in parallel.

This quite often result in Error 429, which we presume is due to rate limits. And as the workflows typically uses the same IP this causes all workflows to fail.

Have tried to remedy this,...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
danil2308
[dagger/dagger] Issue opened: #11992 🐞 <is not possible to add custom liveness and readiness probes>

What is the issue?

https://github.com/dagger/dagger/blob/f8c23f77772249bb5a480e7044b5345df89f5f62/helm/dagger/templates/engine-daemonset.yaml#L127-L146

Hey all, I tried to configure a custom liveness probe on my dagger engine but ended up having next error,


rror while running post render on files: map[string]interface {}(nil): yaml: unmarshal errors: │
│ line 110: mapping key "exec" already defined at line 104

Reason is: it is not possible to have two readinessprobes for the ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #11995 workspace: engine-side workspace plumbing and legacy compat

Context

This PR is a spinoff of #11812 (Workspaces, modules v2). That PR grew large enough that it benefits from being split into two focused, reviewable PRs:

  • This PR (foundation + compat): the lower half — engine-side plumbing, runtime workspace model, legacy compatibility, lockfile substrate
  • Follow-up PR (UX + authoring): the upper half — .dagger/config.toml support, dagger migrate, explicit workspace targeting, workspace management commands

Together these two PRs depr...

vivid lintelBOT
#
mglazer
[dagger/dagger] Issue opened: #11996 🐞 dagger 0.20.1 fails with sha fetch not supported by remote

What is the issue?

It seems that ever since: https://github.com/dagger/dagger/pull/11909 merged, and 0.20.1 was released, all of our CI pipelines are failing to pull a module that exists inside of our GitHub Enterprise instance.

Specific versions: GitHub Enterprise 3.18.4

Error I'm seeing:

 load module: .
9   : ┆ finding module configuration
10  : ┆ ┆ moduleSource(refString: "."): ModuleSource!
11  : ┆ ┆ ┆ load SDK: go
11  : ┆ ┆ ┆ load SDK: go DONE [0.0s]
12  : ┆ ┆ ┆ moduleSource(...
rocky idol
# vivid lintel

@rough birch let me know if you need any other information for this ticket

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
moltar
[dagger/dagger] Issue opened: #11998 🐞 `dagger call` exits 0 when module function errors (Void return type with failing service)

What is the issue?

A TypeScript module function returning Promise that runs multiple parallel checks via Promise.all — including one that uses .withServiceBinding() to bind an .asService() container — fails when the service's underlying withExec exits non-zero.

The Dagger TUI output (--progress=dots) clearly shows the error:

): Void 5m17s ERROR

✘ withExec sh -c '...' 1m12s ERROR
! exit code: 1

✘ withExec sh -c '...' 2m23s ERROR
! start  (aliased as ): exit code: 1
```...
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #12000 Harden cases where CLI may exit 0 even during failure

Two commits with separate approaches:

  1. if err != nil but the exit code was 0, still have the CLI exit 1. It's not clear the exact sequence of events where this might happen, but it seems plausible since 0 is the Go zero value (duh).
  2. If the primary span is marked as failed, then never exit 0
    • I'm less convinced by this one, but it should handle the case where err == nil somehow but the telemetry still knows the main span was failed. cc @vito dunno if we're gonna be breaking any...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #12002 core(changeset): use structured diff stats instead of client-side patch parsing

Problem

Changesets larger than 128MB couldn't show diff summaries at all — the raw patch had to be loaded into memory as a string, which hit the max file contents size limit. Even below that threshold, the TUI would freeze while diffparser chewed through the full patch just to count lines.

Solution

The fix is to stop parsing patches for metadata that git already knows.

git diff --numstat returns per-file line counts directly, without loading patch content. We expose...

vivid lintelBOT
#
hdhdn
[dagger/dagger] Issue opened: #12003 [Security Alert] Exposed API key(s) detected: AWS Access Key

Hi,

An automated responsible-disclosure scan found pattern(s) matching the following API key type(s) in this file:

https://github.com/dagger/dagger/blob/8d322d0aeb150b214d79f83cf34b9e8098d7b854/docs/current_docs/partials/cookbook/secrets/_use-secret-variable.mdx

Keys detected

Recommended actions

  1. Revoke each key immediately using the links above
  2. Remove the key(s) from the file and commit the cha...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
grouville
[dagger/dagger] Pull request opened: #12016 WIP(core/cache): allow specifying checks cache seed

Context

This PR is adds an optional cacheBuster argument to the checks API:

  • Check.run(cacheBuster: String)
  • CheckGroup.run(cacheBuster: String)

The contract is:

  • no cacheBuster: current behavior, normal cache lookup
  • same cacheBuster: reuse the same isolated cache branch
  • different cacheBuster: run on a different cache branch

In other words, this gives the caller a way to choose the cache branch used for a check run.

It is shipped to open the discussion, stil...

vivid lintelBOT
vivid lintelBOT
#
alexcb
[dagger/dagger] Issue opened: #12018 🐞 dagger init fails when `.env` exists but is a directory

What is the issue?

dagger init fails if there's a directory called .env in any of the parent directories.

Dagger version

dagger v0.20.1 (image://registry.dagger.io/engine:v0.20.1) linux/amd64

Steps to reproduce

sh -c 'cd $(mkdir -d) && pwd && mkdir .env && mkdir example && cd example && dagger init'

Log output

✔ connect 0.1s

✘ moduleSource(refString: ".", disableFindUp: true, allowNotExists: true, requireKind: LOCAL_SOURCE): ModuleSource! 0.0s ERROR
┇...
vivid lintelBOT
vivid lintelBOT
#
majiayu000
[dagger/dagger] Pull request opened: #12019 fix: dagger init fails when .env exists but is a directory

Fixes #12018

Changes

When .env exists as a directory (e.g. Python venvs create .env/), dagger init fails because innerEnvFile() and outerEnvFile() in core/modulesource.go try to read it as a file.

Fix: detect the "is a directory, not a file" error from the Host.file() call and skip gracefully instead of propagating it.

Test Plan

Added two integration tests in core/integration/envfile_test.go:

  • TestEnvDirectory: .env directory in the module directory
  • `TestEnvD...
vivid lintelBOT
#
majiayu000
[dagger/dagger] Pull request opened: #12020 fix: dockerfile RUN heredoc fails

Fixes #11972

Problem

Dockerfile heredoc syntax fails with "command not found (exit 127)" when using dagger docker-build.

Root Cause

In engine/buildkit/executor_spec.go, the mount filtering logic separates OCI spec mounts into those kept for runc and those pre-applied to the rootfs. The /dev/pipes/ bind mount used by heredoc processing was being pre-applied to the rootfs. However, runc subsequently mounts /dev as tmpfs (a default OCI spec mount), which shadows everything prev...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #12021 Handle missing cached git auth secrets

It seems that it's still possible in some cases for a contextual directory sourced from a private git repo to end up in function cache in a state where a client hitting the cache reloads it but with different credentials than before and then the engine complains it doesn't have the creds anymore (since the http auth secret becomes and explicit arg to the git operation).

All these problems are going away in the new cache implementation, but in the meantime we can patch this by just checking...

vivid lintelBOT
vivid lintelBOT
#
kpenfound
[dagger/dagger] Pull request opened: #12022 refresh on docs cloud page

The page at https://docs.dagger.io/reference/configuration/cloud/ was super out of date, so this is a refresh

Quick summary:

  • Link to cloud reference page from cloud section on features page
  • mention cloud checks on features page
  • improve github onboarding instructions on reference page
  • add cloud checks section to reference page
  • add secret provider configuration section on reference page
  • clean up modules section that was overly verbose
  • improve instructions in Traces secti...
vivid lintelBOT
#
timplifier
[dagger/dagger] Issue opened: #12023 ✨ Dagger Rust Community SDK

What are you trying to do?

I wonder if Rust Community SDK is going to be promoted to official SDK. Currently it doesn't have proper codegen ( dagger-codegen library was last updated two years ago ) and as Rust users we can't use modules properly. I know there is an option to opt in for Go or other officially supported languages but it would be awesome if we could use Rust for that case

Why is this important to you?

Currently we only can talk to the Dagger client and fully replace D...

vivid lintelBOT
#
perwestling
[dagger/dagger] Issue opened: #12024 🐞 CVE GHSA-p77j-4mvh-x3m3 reported from Dagger engine/SDK in dagger v0.20.1

What is the issue?

The CVE GHSA-p77j-4mvh-x3m3 (Critical gRPC vulnerability, fixed in v1.79.3) is coming from the Dagger engine/SDK for dagger v0.20.1.

This appeared today for dagger modules built.

Dagger version

dagger v0.20.1 (docker-container://dagger-engine) linux/amd64

Steps to reproduce

  1. Create go container for dagger module using dagger SDK for v0.20.1
  2. Run Grype scanning for the module

Log output

NAME                              INSTALLED  FIXED IN  TY...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
tupe12334
[dagger/dagger] Issue opened: #12034 feat: add block-no-verify PreToolUse hook to protect git hooks

Problem

Claude Code agents can silently bypass git hooks by running git commit with the hook-skip flag. This disables pre-commit, commit-msg, and pre-push hooks — the same hooks that enforce linting, type-checking, and commit message conventions.

Solution

Create .claude/settings.json with block-no-verify@1.1.2 as a PreToolUse Bash hook. It reads tool_input.command from the Claude Code hook stdin payload, detects the hook-bypass ...

#
tupe12334
[dagger/dagger] Pull request opened: #12035 feat: add block-no-verify PreToolUse hook to protect git hooks

Summary

Creates .claude/settings.json with block-no-verify@1.1.2 as a PreToolUse Bash hook to prevent Claude Code agents from bypassing git hooks via the hook-skip flag.

Details

When an agent runs git commit or git push with the hook-bypass flag, it silently disables pre-commit, commit-msg, and pre-push hooks. block-no-verify reads tool_input.command from the Claude Code hook stdin payload, detects the hook-bypass flag across all git subcommands, and exits 2 to block. T...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
rrati
[dagger/dagger] Issue opened: #12039 🐞 WithExec Expand option does not apply to RedirectStdout

What is the issue?

Running WithExec with these options:

dagger.ContainerWithExecOpts{
   Expect:         expectedResult,
   RedirectStdout: reportPath,
   Expand:         true,
},

Where reportPath = $HOME/artifacts

Results in an error open redirect stdout file: open /tmp/rootfs1507444385/$HOME/artifacts

It looks like Expand is not applying to RedirectStdout option.

Dagger version

0.20.1

Steps to reproduce

Run WithExec with the options and configuration in the desc...

vivid lintelBOT
ornate vigilBOT
#
manimovassagh
[dagger/dagger] New comment on discussion #11787: How to export files or directories from the runtime container

The trick is to use Directory as the shared state between your modules, and then call .Export() from your top-level entrypoint.

Here is the pattern:

1. Have each sub-module return a Directory

Instead of writing files to the runtime container's filesystem directly, have each module function return a *dagger.Directory containing its outputs:

// In your sub-module
func (m *SubModule) Build(ctx context.Context, src *dagger.Directory) *dagger.Directory {
    return dag.Containe...
#
manimovassagh
[dagger/dagger] New comment on discussion #10150: How to manage multiarch builds which need to be executed on separate runners?

The core issue is that when dag.Container().From("registry/image:tag") pulls an already-published image, Dagger re-exports it with a fresh timestamp in the config, producing a different digest. This is by design -- Dagger treats From() as constructing a new container state, not as a reference to the remote manifest.

Workaround: use the registry directly for the manifest merge

Skip Dagger for the final combine step and use crane or docker manifest instead. Since you have already ...

ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #12042 chore: bump com.fasterxml.jackson:jackson-bom from 2.21.1 to 2.21.2 in /sdk/java in the sdk-java group

Bumps the sdk-java group in /sdk/java with 1 update: com.fasterxml.jackson:jackson-bom.

Updates com.fasterxml.jackson:jackson-bom from 2.21.1 to 2.21.2

Commits

10e12a5 [maven-release-plugin] prepare release jackson-bom-2.21.2
d754903 Prep for 2.21.2 release
63e1b3b Post-release dep version bump
716ab0d [maven-release-plugin] prepare for next development iteration
See full diff in compare view

[![Dependabot compatibility score](https://depe...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #12045 chore: bump astral-sh/ruff from 0.15.6 to 0.15.7 in /modules/ruff/build in the sdk-python-docker group across 1 directory

Bumps the sdk-python-docker group with 1 update in the /modules/ruff/build directory: astral-sh/ruff.

Updates astral-sh/ruff from 0.15.6 to 0.15.7

Release notes
Sourced from astral-sh/ruff's releases.

0.15.7
Release Notes
Released on 2026-03-19.
Preview features

Display output severity in preview (#23845)
Don't show noqa hover for non-Python documents (#24040)

Rule changes

[pycodestyle] Recognize pyrefly: as a pragma comment (E501) (#24019)

Server...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #12046 lockfile: add lookup resolution substrate and initial consumers

Summary

This stacks on workspace-plumbing and extracts the lockfile lookup substrate into its own PR.

It adds:

  • the generic tuple-based lockfile substrate in util/lockfile
  • lock policy and lock mode plumbing through the CLI, client metadata, nested clients, and module runtimes
  • workspace-local lockfile read/write helpers and currentWorkspace.update(): Changeset!
  • dagger lock update for refreshing existing entries
  • initial lock-aware consumers for container.from and Git ref...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
httpete-broadcom
[dagger/dagger] New comment on discussion #12050: 800+ Issues - Whoah

OK thanks - I felt the same - it is certainly getting uptake and I see why. The doc is beautiful and examples great. The only hitch I had was trying to do the codegen, the container doesn't accept any arguments (like our own custom npm registry from artifactory) and it can't get out to the npm registry, so I was stumped. Other than that - the programmatic approach is great.

ornate vigilBOT
ornate vigilBOT
#
fredleger
[dagger] New discussion #12051: ✨ [DAGGERVERSE] why daggerverse is not currated

What are you trying to do?

After a few talks with some fellow daggernauts, we are all wondering why there is not some kind of curation on the daggerverse.

  • Docker Hub has pulls + stars + last updated criteria
  • Github has stars and forks
  • Packagist has Installs Stars Watchers Forks
  • You've got the idea...

IMHO, the daggerverse desperately needs an addition of more criteria or a manual curation to go one step further.
This is a break in the adoption of dagger generally, as I see it.
...

vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #12051: ✨ [DAGGERVERSE] why daggerverse is not currated

@fredleger I agree, we need to improve curation. We've been stretched thin but are looking at this problem again.

The root cause of the issue is that we've made it much easier to write and share modules, than to reuse them. As a result, there is a lot of noise on Daggerverse, since it's basically a search engine for every dagger module available anywhere on the internet. As a result, it's very difficult to find a high quality module that you can trust is following all the best practices, e...

#
verdverm
[dagger] New discussion #12052: Dagger should enable network resilience everywhere

My current build system often fails when a remote network / service behaves badly. 502's, host hangs up, services being down... Today, I have to add custom logic at all of these points as we discover them, or we move to replicating these remotes internally.

  • git clone
  • go mod download
  • docker pull
  • apt update
  • the list goes on and on...

It would be great if there was retry / expo decay options across the building blocks for these situations. Detecting which are network ...

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger/dagger] New comment on discussion #12052: Dagger should enable network resilience everywhere

Just as another example of this, @jpadams hit a remote cache import error (failed to compute cache key: failed to copy: invalid status response 503 Egress is over the account limit) due to a 503 from GHA that failed the entire run. In general Buildkit tries to treat these errors as cache misses, but due to the way this failed Buildkit didn't. Have an issue open there now, though it's actually a pretty hard problem: https://github.com/moby/buildkit/issues/2836

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #12053: `dagger bug` would open a new github issues with metadata filled in

I love this. A few follow-up thoughts:

  • We can probably broaden this to also allow suggesting features, asking for help on a configuration, discussing a use case, reporting an issue with a specific package, search the knowledge base... A proposal to get us started: dagger support ?

  • This feature works in tandem with a backend service. That service should be hosted by dagger.io, not github :) Of course we can still leverage github as much as we want, for example to create issues as y...

#
Jitsusama
[dagger] New discussion #12054: ✨ Support Scheduling Build Steps on K8s Nodes

What are you trying to do?

We have the circumstance where a build could engage a large amount of expensive steps to all be engaged in order to assemble the pieces necessary to create a final build artifact. A number of these steps are large enough that even a well equipped machine might have a hard time coping with it.

My intent is to be able to schedule some/all actions as separate Pods scheduled on a K8s cluster. We could define their resource constraints so that K8s would be able t...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
aluzzardi
[dagger] New discussion #12056: Cache invalidation explain

Cache invalidation explain

It's very common for an action (or a set of actions) to be re-executed given that "something" in the build cache changed. While speaking with @gerhard today across several topics, we came up with the idea that it'd be great if Dagger could output some optional detailed information about why the cache is being invalidated. I think this could potentially help to prevent mistakes and/or find issues/improvements in already executed pipelines.

As side nice-to...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
nvuillam
[dagger] New discussion #12057: ✨ MegaLinter package

What are you trying to do?

I'd like to create a reusable dagger package to run MegaLinter

Would such PR be accepted ?
If yes, where in dagger sources could I implement it ?

Why is this important to you?

It could allow dagger users to easily integrate MegaLinter within their dagger pipelines :)

How are you currently working around this?

I could just do docker run but i'd prefer to have a reusable item :)

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
chopanpma
[dagger/dagger] New comment on discussion #12058: 🖐️ How to run dagger inside a container ?

I would like to see an example of it, especially in docker because this
should be part of the developer tools, and if we can have all dev tools in
a repo, dockerized it is easy to keep the same environment in the team,
versions, etc. Not so many devs have a k8s or the likes running locally
however I would like to understand the use case for you guys that have
asked for it.

On Fri, Apr 28, 2023 at 4:04 AM thecooldrop @.***>
wrote:

I would like to see this as well. Especially as to ho...

#
chopanpma
[dagger/dagger] New comment on discussion #12058: 🖐️ How to run dagger inside a container ?

awesome thanks

On Thu, May 18, 2023 at 1:01 AM Miranda Carter @.***>
wrote:

Hi @thecooldrop https://github.com/thecooldrop , @chopanpma
https://github.com/chopanpma , and @kajogo777
https://github.com/kajogo777 ! We are actively working on this. I
highly recommend joining the Dagger Community call on June 1st. @gerhard
https://github.com/gerhard and @kpenfound https://github.com/kpenfound
will be doing a demo about this topic.

You can register here:
https:...

#
thecooldrop
[dagger/dagger] New comment on discussion #12058: 🖐️ How to run dagger inside a container ?

For me it would be sufficient to see how we can run dagger within a rootless container. Namely a container where I do not have to have access to root. This would then enable me to run the dagger in any containerized environment. Mostly that would mean that I can use dagger in my Jenkins, while using Kubernetes Pods as agent nodes, where I can then have elastically scaling Jenkins server as load increases or decreases, while still retaining the capability to run the pipelines locally.

#
senare
[dagger/dagger] New comment on discussion #12058: 🖐️ How to run dagger inside a container ?

I am also looking for this !

I can add my 2 use cases also

Again using Argo Workflow for CI/CD (i.e Argo suite with ArgoCD Argo Workflows Argo Events ...)

Also would like to have builds local inside k8s i.e something like ...

https://www.talos.dev/v1.4/talos-guides/install/local-platforms/docker/

throw in challenge's with multi platform i.e arm/amd with rise of M1, M2 .. I am having a hard time working out how this should work would love some more examples !

#
ahaeber
[dagger] New discussion #12059: ✨ Azure Pipelines tool installer for Dagger

What are you trying to do?

Using Dagger with Azure Pipelines. The documentation has a helpful example for Azure Pipelines. It starts with a script to install dagger:

  - script: |
      cd /usr/local
      curl -L https://dl.dagger.io/dagger/install.sh | DAGGER_VERSION=$(DAGGER_VERSION) sh
    displayName: Install Dagger $(DAGGER_VERSION)

then there is a different approach for Windows agents.

Azure Pipelines has the concept of 'tool installer' tasks that help to accompl...

vivid lintelBOT
ornate vigilBOT
#
dolanor
[dagger] New discussion #12060: ✨ support code coverage report for Go

What are you trying to do?

Get some Go code coverage directly from dagger

Why is this important to you?

100% coverage doesn't mean you're safe, but having 0% is worst. Having an idea where you are in between is useful.
What would be great would be to be able to track the difference between the last run in the main branch to track the evolution more than the absolute value.

At a first step, erroring if the full coverage value <XX% would already be interesting.

How are you...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
senorsmile
[dagger] New discussion #12061: ✨ Jenkins: way to align dagger build steps as Jenkins Build steps

What are you trying to do?

It would be great to be able to hook into the DAG created by dagger to create build steps in a Jenkins that align to visualize the progress of a long running job via Blue Ocean.

Why is this important to you?

Seeing the output visually is often helpful vs just seeing streaming text output.

How are you currently working around this?

I don't see a way to work around this currently. Everything just ends up in a single build step in Jenkins.

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger] New discussion #12062: ✨ Standard error reporting (e.g. JUnit XML)

What are you trying to do?

Clearly export and see errors from a Dagger do execution using a standard format. Someone I talked to after my PlatformCon talk mentioned this as a good way to integrate with existing systems:

Usually an HTML or something like JUnit XML, which can be displayed in most of the CI/CD web interfaces.

They in the past have struggled with trying to parse errors out of BuildKit's (custom, non-standard) output, something like this seems like it would help them a ...

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger] New discussion #12063: Support `no-cache` execution

There are use cases that call for always running a certain step of a pipeline. Examples include an step that checks to see if a remote resource has drifted from the last known cached state, in which case other steps may need to run in order to correct it.

This requires being able to specify that a step should always run. A few open questions:

  1. Who configures this: the extension author (or us as dagger devs in the case of core api)? the api invoker? both?
  2. Do we use buildkit's `Ignore...
vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger/dagger] New comment on discussion #12063: Support `no-cache` execution

Some updated thoughts on this, another benefit to having official support for this is in our API that we can have our remote caching skip these execs (which would obviously be a waste of time+space to cache).

I think a good potential implementation here would be to re-use @vito's idea w/ services where we stop buildkit from caching these by forcing them to fail. In this case we wouldn't cancel them though, we can instead have the shim always exit non-zero so buildkit thinks it's failed and...

#
dolanor
[dagger] New discussion #12064: ✨ GitHub action to run regularly to find flaky tests

What are you trying to do?

I want to merge PR but sometimes test fail for no good reason except they are flaky.

Why is this important to you?

Flaky test can be a productivity killer. It can lead you to the wrong rabbit hole when you don't need it.

I know go test accepts the -count flag. We could use it with 10 or 30 to run those tests many times on main, and if it fails, it means there must be some race condition or some flaky test.
We're better investigating this problem...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger] New discussion #12068: Container: support read-only mounts (and/or directories)

Useful because:

  • Users sometimes want to prevent files from changing somewhere
  • Performance benefits (buildkit doesn't need to create new layers)
  • Better content-based caching (there are some really convoluted corner cases that cause the caching behavior to be slightly different when using a rw mount, I can look them up again if needed - suffice it to say there can be differences that a few users might care about someday)

Could model this as

  1. allowing a mount to be specified as re...
vivid lintelBOT
ornate vigilBOT
#
marcosnils
[dagger] New discussion #12069: Multi-tenant engine

What are you trying to do?

Having two different clients taking to the same Dagger engine to have different views of the API depending on the extensions that each decided to include, currently the engine exposes a single global API for everyone.

Why is this important to you?

So we can provide multi-tenancy clients in the same engine

How are you currently working around this?

N/A

cc @sipsma @aluzzardi

vivid lintelBOT
ornate vigilBOT
#
timplifier
[dagger] New discussion #12070: ✨ Dagger Rust Community SDK

What are you trying to do?

I wonder if Rust Community SDK is going to be promoted to official SDK. Currently it doesn't have proper codegen ( dagger-codegen library was last updated two years ago ) and as Rust users we can't use modules properly. I know there is an option to opt in for Go or other officially supported languages but it would be awesome if we could use Rust for that case

Why is this important to you?

Currently we only can talk to the Dagger client and fully replace D...

vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #12070: ✨ Dagger Rust Community SDK

@timplifier good question, it really depends on a maintainer stepping up to get it done. I agree it would be really nice to have full module support in Rust, unfortunately the core Dagger team doesn't have the bandwidth to develop one, but we are happy to provide support it some Rustaceans volunteer :) Perhaps the new generation of coding AIs makes this exponentially easier?

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger] New discussion #12071: Unintuitive behavior when combining Build+Stdout

User reported that they expected this to result in evaluation:

id, _ := g.DaggerClient.Container().
        Build(src, dagger.ContainerBuildOpts{Dockerfile: "docker/Dockerfile")}).
        Stdout().Contents(context.Background())

But it didn't. The implementation reason is that Stdout only prints the output of previous Execs, not Build. However, that doesn't really make any intuitive sense.

Possible solutions:

  1. Stdout could return the progress output of the previous bui...
vivid lintelBOT
ornate vigilBOT
#
dolanor
[dagger] New discussion #12072: ✨ support mounting device into the build

What are you trying to do?

I need to access /dev/ttyUSB0 to be able to flash some stm32 microcontroller.
Once I built an app with tinygo, I need to use stm32flash on /dev/ttyUSB0 with it.

Why is this important to you?

We could simplify the whole flashing process by removing the dependencies on the host to be able to hack on those devices.

How are you currently working around this?

I do it directly on the host (and have a bunch of microcontroller tools on it)

vivid lintelBOT
ornate vigilBOT
#
daniel-naegele
[dagger/dagger] New comment on discussion #12072: ✨ support mounting device into the build

Hey is there any update on this? This would be really awesome in order to be able to start virtual machines using Qemu for services. Then the /dev/kvm device could be passed through the container..
Usually one would of course use docker containers to manage services, but sometimes its just necessary to have a VM. For example running a macOS VM using docker-osx, because building iOS apps strictly require macOS with Xcode.

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger] New discussion #12076: Mounts under symlinks may result in unexpected behavior for `directory`

(This needs to be verified, it should be true in theory but hasn't been tried yet)

When using the Container API, if a mount is placed under a path containing a symlink, it's possible to create situations where:

  1. a directory in the container during an exec op shows some contents
  2. use of the directory API to get the contents of that path returns something completely different

Described more here: https://github.com/dagger/dagger/pull/3217#discussion_r987479638

Possible soluti...

vivid lintelBOT
ornate vigilBOT
#
vito
[dagger] New discussion #12077: HTTP API should support validating checksums

We should support validating checksums for HTTP downloads. LLB already supports it, thankfully: llb.Checksum.

Here are some discussion points:

# should it be required or optional?
http(url: String!, digest: String!): File!
http(url: String!, digest: String): File!

# usage:
http(url: "https://dagger.io", digest: "sha256:4f559663fb9d7d49db9c17b4ef81fb41c85fe8a3c7dca127a18af4818f07b807")

# should we have ...
vivid lintelBOT
ornate vigilBOT
#
dolanor
[dagger/dagger] New comment on discussion #12077: HTTP API should support validating checksums

I think it should be optional, we don't always know the checksum in advance.
It depends on the scenario I guess.
For SBOM, it makes sense to make sure to get exactly what you expect, and make for more idempotent builds.
For testing/experimenting, you don't want to get a checksum beforehand every time.

On the format, I'm not sure. We've retired sha1 a while ago, I don't know when sha256 will. As most of the tools we use are still on sha256 as well, maybe it makes sense to just use that. ...

#
vikram-dagger
[dagger] New discussion #12078: ✨ Add extension to deploy a container from a registry to a Kubernetes cluster

What are you trying to do?

Assume that I have built an application image with Dagger/Cloak and pushed this to a registry like Docker Hub. As part of my deployment process, I would like to deploy this image from the registry to a K8s cluster. This could be either a local minikube cluster for testing, or a cloud K8s provider like GKE.

Why is this important to you?

Being able to deploy to K8s should be supported in Dagger as many deployments now happen on K8s.

How are you curren...

vivid lintelBOT
ornate vigilBOT
#
bluebrown
[dagger/dagger] New comment on discussion #12078: ✨ Add extension to deploy a container from a registry to a Kubernetes cluster

@vikram-dagger, Kubernetes pulls images by itself. So you are not giving it the image really via dagger. You would only tell it where it can get the image.

The most simple thing you could do would be mounting your local kubeconfig to a kubectl image and apply some manifest referencing your image in some pods container list.

Once you have an action in place to run a command against your cluster, you need to ensure that your cluster is allowed to pull from the registry in case of private ...

#
jpadams
[dagger/dagger] New comment on discussion #12078: ✨ Add extension to deploy a container from a registry to a Kubernetes cluster

@bluebrown could you advise us on what this could look like from Dagger/Cloak to AKS?

Guessing we can start with this example your submitted using kubectl:
https://github.com/dagger/dagger/blob/main/pkg/universe.dagger.io/alpha/azure/test/test.cue

or this one with helm:
https://github.com/dagger/dagger/blob/main/pkg/universe.dagger.io/alpha/azure/aks/helm/test/test.cue

WDYT?

#
sipsma
[dagger] New discussion #12079: Support for executing on non-Linux OSes

Multiplatform support is coming in cloak, but it will only support architecture emulation and help out users/extensions that want to perform cross-compilation.

However, there are many use cases where cross-compilation is not really a viable option or where binaries targeting non-Linux kernels need to be executed rather than just compiled. Dagger doesn't support those use cases yet.

It's most likely an enormous undertaking, but opening this as a tracking issue for supporting execution on...

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger/dagger] New comment on discussion #12079: Support for executing on non-Linux OSes

From a discord user:

We are talking about building big C++ codebases, where build performance is a top priority.
Also putting aside build performance, I don't see cross-compiling C++ to macOS, having to link to native macOS libraries, as an easy path either... 🙃

I understand the current limitations and I understand the "in the very long term" 🙂

As a data point, I can confirm that there are still companies that, although they follow modern best practices regarding CI/CD, hav...

vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger] New discussion #12080: ✨ Log warning when using deprecated APIs

Summary

Present a warning to the user when using a deprecated API field or argument.

Motivation

Logging a warning makes it more visible whenever you're actually running a pipeline, no matter if you use a smart IDE or consult the reference docs during development. Removing the warning acts as encouragement to migrate to the new behavior.

Rationale

There's two sides to this:

  • In each SDK, so it's shown when using the IDE and when browsing the reference documentation.
    ...
vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger/dagger] New comment on discussion #12080: ✨ Log warning when using deprecated APIs

Yeah, been thinking about this because we have pending deprecations. For example:

I added this warning:

https://github.com/dagger/dagger/blob/fc6a74297956c41801d36081d841c7410ac9651e/core/schema/container.go#L582

The intent was for the warning to show for the user when this API is used, both in the TUI and hopefully in cloud traces as well. Nothing was shown last time I ran this.

I defer to @vito to help point out what needs to be done...

#
vito
[dagger/dagger] New comment on discussion #12080: ✨ Log warning when using deprecated APIs

@helderco Yeah, I think it's time to start integrating with OpenTelemetry logging properly, which I take to mean integrating slog with ctx and setting OpenTelemetry's severity levels etc. instead of mimicking stdout/stderr.

I had an idea for a "global logger" that we could use sort of like Progrock's previous logging system:

https://github.com/dagger/dagger/blob/4ec5818fab06e4ac42e4b0a74dd913e0d2c02a85/telemetry/logging.go#L40-L47

...but it's half-baked at the moment. I wanted to...

#
TomChv
[dagger/dagger] New comment on discussion #12080: ✨ Log warning when using deprecated APIs

@helderco I'm not sure I understand how to implement that, if I just log deprecation warning in the function I'm not sure it's going to be enough, for example

func (s *socketSchema) Install() {
	dagql.Fields[*core.Query]{
		dagql.Func("socket", s.socket).
			Doc("Loads a socket by its ID.").
			Deprecated("Use `loadSocketFromID` instead."),
	}.Install(s.srv)

	dagql.Fields[*core.Socket]{}.Install(s.srv)
}

type socketArgs struct {
	ID core.SocketID
}

func (s *socketSc...
#
TomChv
[dagger/dagger] New comment on discussion #12080: ✨ Log warning when using deprecated APIs

@vito Do you have any hints on where I could add this kind of logic, I see that we configure the TUI in the dagger/main.go

https://github.com/dagger/dagger/blob/7e99ef68b6338953096580e1a73c622e79f9bf13/cmd/dagger/main.go#L277-L312

But I do not understand how I could create a static view at the top of the logging, like a permanent span or an extra global logger like you mentionned in https://github.com/dagger/dagger/issues/4855#issuecomment-2098721537

I have an idea but I'm not sure...

#
helderco
[dagger] New discussion #12081: codegen: Selecting sibling fields

I know that we're not yet optimizing for single queries by allowing to select multiple sibling fields but my brain keeps bringing it up and I'm fond of discussing possibilities here if anyone has ideas to add this feature.

Uncached Exec?

The common example that comes to mind for this is when you want to select both stdout and stderr from an exec, although there's a separate issue to attempt to simplify that:

  • #3496

In any case, those are two different scalars that need t...

vivid lintelBOT
ornate vigilBOT
#
stobias123
[dagger] New discussion #12082: ✨ Dockerignore syntax.

What are you trying to do?

The new WithoutDirectory syntax is great, but I am looking for something closer to .dockerignore syntax.

I don't know the name of all directories I'd like to ignore when I read the host dir, but I know they will all contain a folder foo that will be quite heavy. I'd like to ignore all folders under **/foo

Why is this important to you?

See description. I have many many monorepos of this style.

How are you currently working around this?

Se...

vivid lintelBOT
ornate vigilBOT
#
sheldonhull
[dagger/dagger] New comment on discussion #12082: ✨ Dockerignore syntax.

While defining ignore directly in dagger is great, I'd suggest that the same functionality where Dagger can define a build directly or use a DockerFile would be great here.

Since Dagger might be an incremental adoption for folks, setting WithDockerIgnore(ignorefile) as a method would be fantastic to allow incremental adoption while respecting the incremental adoption of Dagger.

#
aluzzardi
[dagger] New discussion #12083: Document Lazy DAG Model

Related #3555

There's been confusion over the "lazy" model, what is sync vs what is async (few discord threads, feedback forum, live YouTube video, ...)

Not only it's confusing, it's one of great advantages of dagger and it's not being understood: write "procedural" code (e.g. container1.Exec, container1.Exec, anotherGraph.mount(container1, container2) and have everything run in parallel, as fast as possible)

I think we should:

  • Document in the getting started (it's a major featu...
vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger/dagger] New comment on discussion #12083: Document Lazy DAG Model

I agree this is a high-priority DX item. So a general 👍to spending time on all this now.

Embrace the laziness

+1 it's a strength of our compute model. We should embrace it and educate people upfront about it. Of course that is work :)

Handling of IDs

This one feels like a special case, I would like to discuss it a bit more before we cross any one-way doors.

My feeling so far:

  • Developers seem to find the ID handling part of our Go API confusing
  • Hiding it is probabl...
#
dolanor
[dagger/dagger] New comment on discussion #12083: Document Lazy DAG Model

@slumbering was also thinking that for the TS SDK, where you can start typing functions, but if you just stop there, you're actually running nothing.
I was thinking about maybe some linter that would analyze that and underline some code that would execute nothing.

But I agree with the fact that this way of lazy evaluation is a strength and we should document it more no matter what we decide to do on the other hand.

#
sipsma
[dagger] New discussion #12084: Replace session stickiness with source pinning

Continuing the discussion here: https://github.com/dagger/dagger/pull/3421#issuecomment-1296089591

This is all just my interpretation and summary of that discussion, @vito please correct me if I misrepresent anything.

Problem

Buildkit sessions are hard to work with as is. They result in certain state being sticky for the duration of a client connection. But once that connection is closed, you can't resume the session or otherwise reobtain that sticky state.

Examples of state as...

#
sipsma
[dagger/dagger] New comment on discussion #12084: Replace session stickiness with source pinning

Does this also include the problem of scoping cache volume names, or is that unrelated?

Good question, I think this isn't necessarily tied into that namespacing problem, but there might be potential for them to have overlapping or at least conceptually similar solutions.

Thinking out loud:

For example, if you submit a query that internally gets the ID of a cache volume, then maybe the ID returned to you can be namespaced somehow (very similar to what's described in Solution A abo...

vivid lintelBOT
ornate vigilBOT
#
sipsma
[dagger/dagger] New comment on discussion #12084: Replace session stickiness with source pinning

@jlongtine

I'm guessing this change (https://github.com/moby/buildkit/pull/3332) will be helpful for implementing source pinning? Looks like that dropped in the v0.11.0-rc2 buildkit release

Yep! That's the follow up of the PR I linked in the description, glad to see it got in for v0.11. Still need to figure out whether+how we can integrate this w/ dagger, but it's pretty powerful and nice to have that flexibility when needed.

#
shykes
[dagger] New discussion #12085: Explain the engine architecture

Problem

In the documentation, we talk about the Dagger Engine in these terms:

Using the SDK, your program opens a new session to a Dagger Engine: either by connecting to an existing engine, or by provisioning one on-the-fly.

graph LR;

subgraph program["Your Go program"]
  lib["Go library"]
end

engine["Dagger Engine"]

oci["OCI container runtime"]
subgraph A["your build pipeline"]
  A1[" "...
vivid lintelBOT
ornate vigilBOT
#
aluzzardi
[dagger] New discussion #12086: Improve Logging

It's hard to figure out which operation logs belong too -- especially with lazy evaluation.

Something I'm discovering as I continue building is that the lazy execution makes debugging tricky -- it's sometimes difficult to tell what code in my build app generated which build instructions as expressed in the output.

#1034221596853932084 message

Proposal:

  • Bring back contextual logging from Europa (e.g. wrap logs with gra...
vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger/dagger] New comment on discussion #12086: Improve Logging

Proposal: Always dump logs to a file. Wrap errors in "See full logs at `

I think this deserves more discussion. In a production setting it may not be acceptable to have a "secret log file" accumulate somewhere. But maybe there's a way to mitigate this.

That's how npm/yarn does it, don't remember when exactly. You get an error and it says "go look at this file". It's a bit annoying but there's precedent in production.

#
jpadams
[dagger/dagger] New comment on discussion #12086: Improve Logging

From users trying Dagger:

Complaints about the enormous stack trace that's produced. Why have the base64 encoded content in the output? It could use some TLC so that it makes the error you're interested in easy to find.

It's unclear to others why layers are out of order. I presumed this was because it was some sort of multi stage docker build thing, but I don't actually know either 🙂

#
helderco
[dagger/dagger] New comment on discussion #12086: Improve Logging

It's unclear to others why layers are out of order. I presumed this was because it was some sort of multi stage docker build thing, but I don't actually know either 🙂

If that's regarding engine logs getting out of order, I've assumed it's because of the async nature of the engine (concurrency and parallelism). 🙂

#
shykes
[dagger/dagger] New comment on discussion #12086: Improve Logging
  • Proposal: Always dump logs to a file. Wrap errors in "See full logs at `

I think this deserves more discussion. In a production setting it may not be acceptable to have a "secret log file" accumulate somewhere. But maybe there's a way to mitigate this.

IMO it would be better (but perhaps more difficult?) to 1) extract the relevant error message in the error itself, and 2) separately let users configure where and how to log

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
marcosnils
[dagger] New discussion #12088: ✨ Add commenting capabilities to docs

What are you trying to do?

Browsing the Haxe docs the other day (https://haxe.org/manual/introduction.html) I've found it's commenting system quite friendly which lead me to this GH project https://github.com/utterance that allows enabling native GH issues as comments to docs / blogpost pages.

Since we don't have that in Dagger docs or blogposts I was thinking that it might be a good idea to give it a try

cc @slumbering @gerhard @shykes

Why is this important to you?

Get rap...

vivid lintelBOT
ornate vigilBOT
#
nipuna-perera
[dagger] New discussion #12089: Document creating and using a sidecar docker service

What is the issue?

It will be useful to have an example cookbook for adding a docker dind sidecar. It is slightly different from the other service examples in the cookbooks as you have to add the DOCKER_HOST to the consumer for it to work. The only example is in the dagger code here but it is set to invalidate cache every run - https://github.com/dagger/dagger/blob/aacce090406f99fed23c969fcef769d532f2b587/core/integration/container_test.go#L3260

More details in this discussion - h...

vivid lintelBOT
ornate vigilBOT
#
marcosnils
[dagger] New discussion #12091: How to configure Go to use private module dependencies

What is the issue?

ref: https://discord.com/channels/707636530424053791/1134415015198330981

We have a cookbook that explains how to clone private repositories here https://docs.dagger.io/710884/private-repositories/. Having said that, in some cases like go private modules a bit more tweaking is necessary. Here's an example of how I had to configure my dagger pipeline to make it work:

   _, err = client.Pipeline("build").
        Container().
        From("golang:1.20").
 ...
vivid lintelBOT
ornate vigilBOT
#
marcosnils
[dagger/dagger] New comment on discussion #12091: How to configure Go to use private module dependencies

:wave: adding another option by using Githu's fine-grained tokens. ref: https://discord.com/channels/707636530424053791/1275877411497447466

func (m *Lala) Test(src *dagger.Directory, token *dagger.Secret) *dagger.Container {
    return dag.
        Container().
        From("golang:1.23").
        WithDirectory("/app", src).
        WithWorkdir("/app").
        WithSecretVariable("GITHUB_TOKEN", token).
        WithExec([]string{"bash", "-c", `git config --global url."https:...
vivid lintelBOT
ornate vigilBOT
#
kpenfound
[dagger] New discussion #12092: [Docs] Document effective abstraction patterns

What is the issue?

We should provide documentation/guidance for effective methods of abstracting + reusing (within a repo or org):

  • pipeline components. wrappers for building parts of a container that are reused across multiple parts of a pipeline
  • subpipelines. Reusing entire subpipelines across multiple pipelines like a github action
  • entire pipelines. Reuse common pipelines across multiple repos without reimplementing
vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger] New discussion #12093: 📖 Multi-language concurrency guide

What?

Create a multi-language guide to cover the importance of thinking about concurrency for improving performance. It can make a significant impact, but many users aren't familiar with it. No need to be exhaustive, just the essential for dagger, an example, and sharing resources to learn more.

### Common
- [ ] Simple concurrency explanation (an analogy and images may help)

### Language specific
- [ ] Example
- [ ] Explanation
- [ ] Lear...
vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger/dagger] New comment on discussion #12093: 📖 Multi-language concurrency guide

#5833 doesn't resolve this since it's just a cookbook.

The point of the guide is for users to understand that concurrency matters, otherwise you can hit a bottleneck pretty easily without realizing. That's the point of the above examples: it takes much longer to execute tasks in sequence when they can run concurrently.

Additionally, the guide can provide resources to learn more about concurrent programming in each language.

#
TomChv
[dagger/dagger] New comment on discussion #12093: 📖 Multi-language concurrency guide

https://github.com/dagger/dagger/pull/5833 doesn't resolve this since it's just a cookbook.

@vikram-dagger asked me to transform it into a cookbook, what was missing for it to be a guide?

I could add a more diagrams and resources by I feared that it would be too overwhelming for the users and make it look really complex to add concurrency to Dagger where it's actually something pretty simple. This is why I focused the PR on code example than theoretical explanations.

vivid lintelBOT
ornate vigilBOT
#
vic3lord
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

Hey @grouville,
They issue is that host.docker.internal isn't available for some reason on Github actions but it does exist on "docker for desktop"

  • My github action is just calling the go binary dagger workflow.
  • It's trying to access the postgres container but it's timing out due to the issue with host networking.
  • I setup the postgres container via _test.go file and https://golang.testcontainers.org
#
grouville
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

dagger

Nice, I was to propose that. The other things that should work (I was about to explore tomorrow, as it's easter today), is to rely on Colima as your container runtime.
Following their docs, it totally seems compatible: https://github.com/abiosoft/colima/blob/65ee3d284b81bfebc09783f5e35a186584aa4bc1/docs/FAQ.md?plain=1#L148-L161

On CI: https://github.com/dagger/dagger/blob/4be9d6ee11cb5d3a66ec3e0f846436535009fd5e/.github/workflows/publish.yml#L110-L116

It would require some ...

#
mircubed
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

reopening to ensure that we solved for @vic3lord 's problem.

@vic3lord is there anything else that you'd like to see from Dagger to make this experience easier? Is there documentation that you'd expect to see for this workaround?

I moved away from test containers to dagger service containers just for CI (although it's a bit of a pain to manage different versions of code between go test to when it's a dagger workflow)

@vikram-dagger , I am adding a docs tag since I think we should docum...

#
vic3lord
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

@mircubed I'm not sure changing the runtime to colima is a good alternative for the basic docker we already get in github actions.

  • We either need a way to get the docker gateway using the SDK (which isn't available in docker for linux it's a trick for docker for mac/windows only)
  • Or we should be able to run long running containers for development and dagger services alongside those for TDD cycles without killing those at the end
#
paul-prdca
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

I'm running into this same issue, let me see if I can clarify my use case.

I've got code running with a "testcontainers" container (in my case Vault). While I could probably adjust this to use Dagger service containers, it works currently with go test outside of Dagger. Most importantly to me, that means it works easily through my IDE (where I'd like to get quick response for testing and coverage).

Rearranging the IDE test process to require Dagger and plumbing the coverage results...

#
gerhard
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

If I understand correctly, you want to run integration tests with https://golang.testcontainers.org/ in GitHub Actions using Dagger. This requires for the Docker instance running in GitHub Actions to be available to the container running within Dagger. This diagram captures what I described:

graph TD
    subgraph "GitHub Actions"
        subgraph "Docker"
            socket(unix:///var/run/docker.sock)
            subgraph "Dagger"
                go-test
            end
...
#
king-11
[dagger/dagger] New comment on discussion #12094: 🖐️ Running a `go test` container and calling test containers fail to reach nested containers

I still think this is still an issue am not able to run testcontainer test inside of dagger runtime, which spawn up postgres and etcd containers.

I am using colima and macOS M4. The error I see is related to cgroups, I tried the approach mentioned here https://github.com/dagger/dagger/issues/5292#issuecomment-1593750070 but for some reason the entrypoint doesn't really get picked up, not sure what am I missing over here

ENTRYPOINT_CGROUP_V2 = """#!/bin/sh

set -o errexit
s...
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger] New discussion #12099: Add Merge function to Directory

Moved from issue #6476

When working with build results and generating final build artifacts, it's common to need to merge directories. While dag.Directory().WithDirectory("", other) works, a dedicated Merge method would better express intent (similar to Diff).

Originally filed by @sagikazarmark on 2024-01-22

vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger] New discussion #12100: Global install

Moved from issue #6883

Proposal: when calling dagger install outside a module, use the home directory as a pseudo-module. This would make dagger install always work regardless of the current working directory.

Benefits:

  • Standardize best practices: "install first" is the recommended way
  • First call of a module is faster
  • Example commands are shorter

Originally filed by @shykes on 2024-03-14

vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger] New discussion #12101: Create Dockerfile from Dagger

Moved from issue #7357

Dagger is great at reading and executing Dockerfiles, but it would be useful to generate a Dockerfile from Dagger as well. This would help users who rewrite their Dockerfile in Dagger but still need docker-compose compatibility downstream.

Originally filed by @levlaz on 2024-05-10

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
#
shykes
[dagger] New discussion #12104: Explicit dependencies between containers without an artifact

Moved from issue #9109

Request for a way to express explicit ordering dependencies in the DAG between two containers without requiring an artifact to be passed between them. Current workaround is creating a dummy file in one container and mounting it in the other.

Use case: integration test setups where a service needs to be initialized before other containers can use it.

Originally filed by @joeshaw on 2024-12-04

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT