#github-feed

1 messages · Page 8 of 1

vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6237 [zenith] New `source` field in dagger.json

Problem

When creating a Dagger module, the source of the module has to be at the root of the module (ie. in the same directory as dagger.json). This is not always practical. In particular, when embedding a Dagger module in an existing software project, the ideal location is not the same for dagger.json and for the module source:

  • Ideal location for dagger.json: the root of the parent software project. This could be the root of the repository, or in the case of a monorepo, the ...
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6238 ✨ Custom service readiness health check

What are you trying to do?

Some software may need more complex readiness checks in addition to checking port availability (eg. to deal with eventual consistency).

Most common example is calling an HTTP endpoint and checking it's result.

Why is this important to you?

Both Docker Compose and Kubernetes have these kinds of checks. When moving from any of those to a Dagger based developer environment, this may be an important bit when checking feature parity.

How are you curren...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6240 core: set endpoint map when initializing APIServer

Without this we're getting nil panics when using dagger shell.

Went unnoticed because we haven't added test coverage for dagger shell to the integ tests yet due to the fact that testing tty programs in our current testing setup is very non-obvious.

I will add whatever best test I can in a follow up to this PR, but sending the fix separately since it is quite trivial and probably an order of magnitude less time consuming than figuring out a test.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6242 CI: override URL components in provisioning tests rather than full URL

During the most recent release we caught a typo in the URL paths used to download cli/checksums in the Python SDK during the bump engine PR (fix).

This worked as expected in that it caught the problem before we published the SDK, but it would have been slightly more convenient if we caught it earlier than the bump engine PR (which is the last step before publishing the SDKs).

The problem is th...

#
sipsma
[dagger/dagger] Issue opened: #6243 CI: split up engine publish and CLI publish

Sometimes the engine publish job needs to re-run independent of the CLI publish, but right now because it all happens in the same job you can't run one without the other, which can create headaches where a re-run results in partial success (re-running engine publish) but failure during CLI publish.

If the jobs were separate but dependent this could be a lot cleaner to deal with.

cc @gerhard, creating issue for what we mentioned during the last release process

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Pull request opened: #6245 Fix Nix publishing

This is attempt numero due to fix Nix package publishing to dagger/nix.

https://github.com/dagger/dagger/pull/6063 added Nix to the GitHub Actions workflow, but GoReleaser actually runs in Dagger using a GoReleaser Pro image.

This PR removes Nix from the GHA workflow and adds Nix to the GoReleaser image.

(Reminder: we only need nix-prefetch-url for GoReleaser to work, but it's not bundled with their image and installing Nix from the Alpine package repo seems to be the easiest way t...

vivid lintelBOT
#
aweris
[dagger/dagger] Issue opened: #6246 ✨ Improve logs for focused mode

What are you trying to do?

I'm trying to show nonexecution messages in my module execution in focus mode to give the user meaningful information. However, it's impossible to do it without using the container WithFocus + WithExec at the moment. It would be amazing to have some logging support in generated client code.

Why is this important to you?

Some useful or important information to make execution is lost in execution.

How are you currently working around this?

Wrapping ...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6247 ci: always print engine logs on job failure

Previously, these logging messages were only added on the publish job provisioning tests: https://github.com/dagger/dagger/blob/9a0f81a7367a675b13854f0be82694d4ebc44dd3/.github/workflows/publish.yml#L251-L256

This meant that failures in the engine and cli jobs didn't trigger this type of message, and so makes crashes more difficult to debug. We already print the kernel logs, we just don't grab the engine logs.

This would have been useful when encountering https://github.com/dagger/dagge...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6248 chore: add git binary to go sdk

When building a module that has a "go.mod" that depends on a commit sha from a repository, we may require the git binary to be present.

Since this is not included in "golang:alpine" official images, we should explicitly install it.

I seem to remember a discord discussion around this, but given discord's wonderous search, I can't seem to find it :scream:

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6249 go: decouple type parsing from code generation in module

This is a small-ish scale refactor that results in the code we use to parse various types/functions/etc. in the go module codegen code. There should be no functional changes, just code refactor+cleanup.

The gist of the idea is that rather than having single methods that do both the type parsing and then return generated code for typedefs, we have a method that does the type parsing and then returns a type that implements an interface which has separate methods for getting the generated cod...

#
jedevc
[dagger/dagger] Pull request opened: #6250 chore: mage consts package cleanup

This PR has two distinct parts (I can split up if it helps, but they made sense to bundle together to me).

In part 1, I merge a couple more consts into the shared common internal/mage/consts package which was recently introduced in https://github.com/dagger/dagger/pull/5557/files#diff-1d8655625df99d708502c2f351eef188e9079cfbd8b3634adc614ca7ddd41f04. Previously, we were just duplicating certain magic string values, which was error-prone. There's also some general cleanup here.

In part ...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6251 Backfill integ tests for dagger shell + assorted fixes

Main goal here was to add integ test coverage for dagger shell after missing a really straightforward nil panic as a result of not having coverage: https://github.com/dagger/dagger/pull/6240

We didn't have coverage before because integ testing TUI programs is not usually straightforward. The solution here uses the https://github.com/Netflix/go-expect library to drive the program and run some simple commands in an alpine shell and verify their output. It worked out reasonably well and base...

vivid lintelBOT
#
aweris
[dagger/dagger] Issue opened: #6252 🐞 Module constructor returns pointer for empty object instead of returning nil for *Directory type

What is the issue?

When I created a module constructor with // +optional=true missing flag returned a pointer of the empty object instead of nil.

Example Code:

package main

func New(
	// +optional=true
	src *Directory,
) *A {
	return &A{Src: src}
}

type A struct {
	Src *Directory
}

func (m *A) IsEmpty() bool {
	return m.Src == nil
}

Log output

dagger call is-empty
✔ dagger call is-empty [1.38s]
┃ false
• Cloud URL: https://dagger.c...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6254 Support interfaces in core + Go SDK modules

This adds initial support for interfaces: https://github.com/dagger/dagger/issues/6213. See that issue for a description of the problems it's solving.

The support here is very much MVP and has a lot of restrictions (mentioned below). However the PR is already borderline too big so I think we can merge with approximately the current feature set and fill in gaps as needed in follow ups.

**NOTE: it's getting a bit late and taking a while to finish documenting everything, so some sections s...

vivid lintelBOT
#
vbehar
[dagger/dagger] Issue opened: #6255 🐞 dagger doesn't work with sysbox

What is the issue?

I've tried to run Dagger in docker-in-docker on top of sysbox, but it doesn't work - because it seems like sysbox doesn't allow Dagger to create a new network namespace

Log output

time="2023-12-12T09:19:13Z" level=info msg="detected mtu 1500 via interface eth0"
dnsmasq[89]: started, version 2.89 cachesize 150
dnsmasq[89]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-nftset auth no-cryptohash ...

vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6256 ✨ Default action for modules

What are you trying to do?

When grouping various actions in a "submodule", there is no way to assign an action to the parent at the moment.

Given the following module:

type Ci struct{}

func (m *Ci) Lint() *Lint {
	return &Lint{}
}

type Lint struct{}

func (m *Lint) All(ctx context.Context) error {
	var group errgroup.Group

	group.Go(func() error {
		_, err := m.Go().Sync(ctx)
		if err != nil {
			return err
		}

		return nil
	})

	group.Go(func() error...
#
jedevc
[dagger/dagger] Pull request opened: #6257 Fix various input normalization issues

Fixes #6252 (the equivalent #6227, but for core types instead of user module types).

In #6167, it looks like we missed another nil check, which manifests as core module types not being set to their zero value but to an empty struct.

Additionally, it looks we regressed on #6057, and we weren't correctly converting field names back into the field names that the module requested. Thankfully, this is a pretty simple fix, and I've verified manually this works. I'm not quite sure of a test ...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6258 feat: skip namespacing when already prefixed

This goes some way to fixing #6071 (@sipsma up to you if you want to leave it open for tracking the subpackage aspect, or splitting up core)

This allows a module "foo" to define it's own "FooContainer" type that won't be additionally namespaced into "FooFooContainer".

Previously, we skipped namespacing if the object was named the same as the module.

However, with this patch we can now additionally skip namespacing if the object if prefixed with the namespace.

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6259 chore: prevent duplicate LLB names

This prevents a tiny visual glitch which is confusing in the TUI output where we get identical custom names.

I noted this in a debugging session with @marcosnils where we were seeing the same message in the TUI and in the cloud. Eventually, it would be nice to remove this, but adding this in at least helps to remove the visual confusion as to why there are duplicates.

#
jedevc
[dagger/dagger] Issue opened: #6260 ✨ dagger cli should have `cache` subcommands

What are you trying to do?

Discussed in discord with @vito and @marcosnils: #maintainers message

It would be nice to have some subcommands for the dagger cli that can help inspect and prune the cache. At a basic level, it would be nice to have similar behavior to buildctl du and buildctl prune.

Future extension ideas:

  • Nice visualization of cache records and how they link to each other
  • Listing and pruning o...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vikram-dagger
[dagger/dagger] Pull request opened: #6262 docs: Restructures Zenith quickstart

The current Zenith quickstart uses a pre-built module and focuses on teaching the user the various sub-commands/features of the Dagger CLI. While this is a good learning path for the CLI itself, it does not do Dagger justice as it fails to capture the power of creating reusable modules and leveraging the Daggerverse. It is also less impactful than the pre-Zenith quickstart, which provides a more hands-on learning experience by having the user build a new pipeline from scratch.

This commit ...

vivid lintelBOT
#
purkhusid
[dagger/dagger] Issue opened: #6263 ✨ Support for stateful processes

What are you trying to do?

I've been looking into using Dagger for our CI pipeline in which one of the main tools we use is Bazel.

The issue with using Bazel in Dagger is that the Bazel startup time(Analysis phase) grows as your repository grows.
Most organizations using Bazel end up using persistent CI workers to lower this startup cost since Bazel keeps the repository state in memory and this makes incremental build very fast once the analysis phase is over.

This model is kind...

ornate vigilBOT
ornate vigilBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6265 Fix flakiness in Python test

Reported by @jedevc in https://github.com/dagger/dagger/pull/6258#issuecomment-1853740495

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-0/cache_home0/dagger/dagger-0.0.0'

The automatic provisioning garbage collector probably needs a lock:

https://github.com/dagger/dagger/blob/970bcfe512cfc699005c46992f52a8f1d17c2af3/sdk/python/tests/engine/test_download.py#L136-L142

https://github.com/dagger/dagger/blob/970bcfe512cfc699005c46992f52a8...

vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6266 🐞 <title>

What is the issue?

Noticed during #6117 that some integration tests were causing a panic to show in the TUI. @sipsma reported in https://github.com/dagger/dagger/pull/6117#pullrequestreview-1778643232.

The tests pass fine and the SDK gets a correct dagger.ExecError, but the panic pollutes the screen and makes it seem like there's a bigger problem.

The issue is in the shim:

https://github.com/dagger/dagger/blob/970bcfe512cfc699005c46992f52a8f1d17c2af3/cmd/shim/main.go#L285-L293...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6267 Dagger `GetStarted` Interactive Shell Module

We could create a GetStarted module that opens an interactive shell and teaches you how to use Dagger.

You could take this really far, but as a baseline it could just be a container w/ the dagger CLI and some example module code. It then presents you with prompts for dagger CLI commands to run that show off the various things you can do.

More interesting stuff you could layer on top of that baseline:

  1. Per-SDK tutorials (i.e. specific ones for Go, Python, Node, etc.)
    • You could...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #6270 feat: add tests for Node Typescript SDK

Fixes argOrder regexp when default arg is set.
Add test on overriding default arg on registry test. Set return type as optional when input type is void. Add tests for Typescript in module_test.go.

Fix ModuleRuntime to include codebase.

  • This fixes the dagger mod install command
  • This fixes using a module without sdk generated in the host.

Fix loading arg to handle array and strings.
Fix entrypoint to load parent state too.
Check for defined result.
Serialize fields that ha...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
gerhard
[dagger/dagger] Issue opened: #6272 🐞 runtime error: index out of range [0] with length 0

What is the issue?

A module that used to work in Dagger v0.9.3 doesn't work in v0.9.4:

dagger-0.9.4 -m github.com/vito/daggerverse/test call testcontainers

Log output

┃ Error: response from query: input:1: main.testcontainers failed to get function output directory: process "/runtime" did not complete successfully: exit code: 2
✘ testcontainers ERROR [3.01s]
✘ exec /runtime ERROR [3.01s]
┃ input:1: testcontainers.dockerService failed to get function output d...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6275 Document how to run dagger in a private intranet

What is the issue?

Assume closed intranet, with no Internet access.

I’ve seen questions about this in Discord and it’s essentially what this OP is asking for:

There’s a few pieces of documentation to put this together and a few responses in different places but it would be helpful for those with this use case to have a unified body of documentation.

If not a guide on https://docs.dagger.io, it can be a section in the [operator manu...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6276 fix: omit buildkit/createdat timestamp from blob source

When calling GetRemotes to get a descriptor for the local source, so we can put into our custom BlobSource, buildkit creates a custom buildkit/createdat annotation on the descriptor that represents the timestamp that the blob was created at.

By default, we were pulling all of the descriptor annotations into the source identifier, which as a result changed the LLB definition (and the vertex digest, which obviously causes all of the dependents to change as well). This can be observed i...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6279 More field manipulation in go

Oh look it's #6181 #5972 #6068 again.

This behavior regressed in v0.9.4 (with #6181 lol).

Fields can have zero names - in which case we shouldn't accidentally discard the entire field, woops.

I've added a pretty comprehensive test for all these scenarios, getting very annoyed having to fix them all.

Also some other smallish related fixes I ran into while trying to get this working.

#
helderco
[dagger/dagger] Pull request opened: #6280 BREAKING: Clear `defaultArgs` on `withEntrypoint` by default

Fixes https://github.com/dagger/dagger/issues/6044

⚠️ Warning
Based on https://github.com/dagger/dagger/pull/6278 so don't merge before that. To review, filter for last commit. Also, perhaps better to merge only after the christmas holidays.

What?

When Container.withEntrypoint is set, it will clear Container.defaultArgs, unless withEntrypoint(args: [...], keepDefaultArgs: true) is set.

vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6281 BREAKING: Make default args parameter required

Fixes https://github.com/dagger/dagger/issues/6051

⚠️ Warning
Based on https://github.com/dagger/dagger/pull/6280 so don't merge before that. To review, filter for last commit. Also, perhaps better to merge the breaking change only after the christmas holidays.

What?

The args field of Container.withDefaultArgs is no longer optional:

- ctr = ctr.WithDefaultArgs(ContainerWithDefaultArgsOpts{Args: []string{"foo"}})
+ ctr = ctr.WithDefaultArgs([]string{"foo"}...
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6282 fix: only add the main module object at the top-level

See https://github.com/dagger/dagger/issues/6272#issuecomment-1858003650:

https://github.com/dagger/dagger/commit/77760c2754575695f6a9329c79c4c6a2d6078a65#diff-42df7f581f0c9e41a3320a7be6c79027f0b373d77e57231d767b3539643a4da8L155-R152 modified the logic so that the following struct would be code-gened:

type Foo struct {
    X string
}

However, if Foo is not reachable from methods/fields on the main struct, then we should not include it on our code-gen and field decla...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6283 fix: ensure declForFunc also matches receiver

Continuing my journey through the forest of a thousand go-sdk papercuts :tada: (sorry for all the small PRs, I figure they're easier to review and merge separately, and I'm just opening them as I find them and have time to fix them :cry:)

Previously, if two methods had the same name (but on different struct receivers), it was accidentally possible to confuse the two.

With this patch we now ensure that as well as matching the names, we also match the receiver name.

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6284 tests: run dagger shell test in isolation + pre-cached

Saw while reviewing PRs that the TestModuleDaggerShell test was flaking sometimes specifically in the testdev workflow. Have never seen this flake in the other "normal" integ test workflow or locally.

From the logs I saw it looks like it is running as expected but just extremely slow, to the point that the generous timeouts weren't enough.

The testdev workflow is extra slow and overloaded for various unrelated reasons (doesn't actually run on the shared runner and doesn't have the same ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6287 Helder/oss 2 zenith camelcase automagic issues with pythonsdk

⚠️ Warning
Based on https://github.com/dagger/dagger/pull/6268.

Will fix https://github.com/dagger/dagger/issues/6286, but no actual fix yet. Just added test to reproduce reported use case.

Problem is that the parent value of a function call is sending the GraphQL cased field names instead of original names.

Example

from dagger import field, function, object_type

@object_type
class HelloWorld:
    my_name: str = field(default="World")

    @func...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6288 feat: lay foundations for engine drivers

This patch starts to simplify the connhelpers and docker-image helper into specific engine drivers (see https://github.com/dagger/dagger/issues/5583) - adding new drivers now becomes easier.

Each driver registers itself under a name, and is called when connecting to a URL with that scheme. Additionally, during connection, we provide a set of additional parameters.

This is mostly just a refactor around the connhelpers, which we can't use directly out of the box, since we want to support ...

vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6289 Go SDK: support pre-connected global `dag` client outside of Modules

In Modules using the Go SDK you get a pre-made dagger client called dag that's ready to use at init.

For non-module Go SDK code (i.e. SDK code that runs directly on a caller's host), you still have to explicitly create a dagger client with dagger.Connect and use that.

We should retain support for dagger.Connect since it has useful options like configuring the destination of log output that users may sometimes want to customize. However we can also add support for a pre-configured ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6291 Modules: replace `root` in `dagger.json` with `source`

Currently, by default the engine loads the directory tree at a module's dagger.json and nothing else. However, to support cases where there are files/dirs the module relies on that are outside that directory tree (i.e. a go.mod, package.json, another dependency module that exists relative to the module, etc.) there exists a root field in dagger.json that allows you to specify a parent directory that the engine will load instead.

This technically works but is inherently very confusi...

#
sipsma
[dagger/dagger] Issue opened: #6292 Cache efficient checkouts of subdirectories of large git repos

Right now when checking out git repos (whether users of our Git API in core or Dagger's internal use for e.g. loading modules), even if only one subdirectory of a git repo is needed, the full repo still needs a clone (to my knowledge).

We use buildkit's Git source implementation, which does jump through hoops to use sparse checkouts and performance optimizations, but it doesn't yet support features in newer versions of git that enable only pulling the files needed for a certain subdirec...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6293 Groundwork for exposing core API in CLI

This lays out of some of the low-level plumbing needed to support https://github.com/dagger/dagger/issues/6229 :

  1. Adds the ability for clients to inspect the entirety of the current schema as a set of TypeDefs.
    • Previously callers could only do this for a single module; now it can for everything loaded into the schema, including core APIs.
  2. Adds an internal flag to the CLI plumbing to optionally include the core type defs it now has access to thanks to the above
  3. Adds a f...
vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6294 Fix python provisioning test flakiness

Fixes https://github.com/dagger/dagger/issues/6265

I considered adding a file lock to the provisioning which would also prevent downloading the CLI multiple times, but it's not done in the other SDKs. Quick fix here is to just not complain if the file to remove doesn't exist anymore (race condition).

The synchronization primitive is not necessary to fix the issue but since I was looking at the Go implementation this matches it a bit more closely.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6296 Update docs to use global client

Summary

Update our docs to use the simpler global client—on SDKs that support it—instead of requiring the client instance to be passed around.

Background

The suggestion was initially made by @aluzzardi in https://github.com/dagger/dagger/pull/5060#discussion_r1189242491 regarding a simpler DX in the (experimental) Zenith modules:

Expand quote

[…] since we're in the process of radically changing the DX, I'm going to put it out th...

vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6297 wip: integrate DagQL

This is an early draft PR, will add more context soon, just want to get a PR up early.

This PR swaps out our GraphQL server infrastructure for DagQL.

goals

  • better telemetry / visualization
  • a stable, useful format for IDs
  • shorter IDs
  • saving and loading pipelines (#3923)
  • global query cache to replace ad-hoc caching (modules, container build/import)
  • a clean purpose-built GraphQL stack to replace our two forks
vivid lintelBOT
#
siulcode
[dagger/dagger] Issue opened: #6298 🐞 Getting: "AttributeError: 'Progress' object has no attribute 'status'" on the Getting started for Python

What is the issue?

I'm trying to follow the getting started guide for Python and getting the below error.

Guide: https://docs.dagger.io/sdk/python/628797/get-started

Error:

dagger run python test.py

┣─╮
│ ▽ init
│ █ [1.47s] connect
│ ┣ [0.18s] starting engine
│ ┣ [0.23s] starting session
│ ┃ OK!
│ ┻
█ [0.58s] ERROR python test.py
┃ Traceback (most recent call last):
┃ File "/Users/lulopzpe/build/apps/lm/eNext/POC/core-ui/test...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6304 tests: flaky rust test

cc @dagger/sdk-rust :heart:

I frequently see this rust test failing: https://github.com/dagger/dagger/actions/runs/7263920431/job/19790293305 (which passes upon a re-run):

Diff  :
 failed to query dagger engine: domain error:
 Look at json field for more details
pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Not quite sure why this shows up, but it appears reasonably frequently, so wou...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6306 fix: install nix using script

Fixes https://github.com/dagger/dagger/pull/6245#issuecomment-1866175753 (cc @sagikazarmark)

The apk package for nix is only available in alpine:edge atm, and goreleaser does not use alpine:edge.

So, we can install it using the same method we were previously using in github actions - the logic is borrowed from https://github.com/cachix/install-nix-action/blob/master/install-nix.sh#L76 and https://github.com/cachix/install-nix-action/blob/master/install-nix.sh#L104.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6312 Modules: `dag.Host()` name is extremely confusing

When module code uses dag.Host(), the "host" it refers to is the module container's "host", not the CLI caller's host.

While this is technically consistent, it's very non-obvious at first glance and a constant source of confusion.

I think at a minimum we need to change Host to something with a more obvious name. TBD if this change also impacts non-module SDKs or if it's somehow scoped just to the codegen for modules.

There's certainly larger scope changes to the API that could a...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6314 CLI: support secrets args by source rather than by plaintext

Private tracker related issue: https://linear.app/dagger/issue/DEV-2944/cli-pass-secret-arguments-by-name-not-by-value

This PR changes how secrets are passed as args in the dagger CLI.

Whereas previously a function arg of type Secret was passed in the cli by plaintext, now secrets are passed on by specifying a source of the secret. This is intended to be much more secure and convenient to users.

This PR adds support for:

  • env:MY_TOKEN - read the secret from the env var `MY_TOKE...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6316 docs: add redirect for old nodejs sdk location

Noticed by @gerhard during the v0.9.5 release:

Noticed that the https://docs.dagger.io/current/sdk/nodejs/reference/modules/api_client_gen link is now broken, most likely after the Docusaurus v3.0 refactoring. The links comes from https://github.com/dagger/dagger/releases/tag/sdk%2Fnodejs%2Fv0.9.5

In #6311, I've fixed the docs link for future releases, but we should add a redirect for previous changelogs. It looks like this was changed during #6164.

vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Pull request opened: #6317 Improve "dagger functions"

Before

dagger functions dumps all functions available in all types for a given module, regardless of arguments. So I can't see what functions are available at a given stage of the pipeline.

Example:

$ dagger -s -m github.com/shykes/daggerverse/wolfi functions
object name                                                         function name   description                                    return type
*Wolfi                                                            ...
ornate vigilBOT
#
skycaptain
[dagger] New discussion #6318: Zenith: How to use private dependencies with the Python SDK

I would like to utilize Dagger along with the Python SDK to configure my CI pipelines across multiple projects. In order to avoid unnecessary repetition, I intend to externalize certain functions into a shared Python library called my-private-library. However, since this code is intended to remain private, I opt to utilize a private PyPI registry. My pyproject.toml looks like this:

[project]
name = "main"
version = "1.0.0"
description = ""
dependencies = [
    "my-private...
vivid lintelBOT
vivid lintelBOT
#
chrira
[dagger/dagger] Pull request opened: #6320 feat: Java SDK upgrade maven to 3.9.6

Upgrade Maven to get rid of validation issues from an old maven version.

[WARNING]  Plugin validation issues were detected in 1 plugin(s)
[WARNING] 
[WARNING]  * org.apache.maven.plugins:maven-plugin-plugin:3.7.1
[WARNING]   Declared at location(s):
[WARNING]    * org.apache.maven:maven-core:3.9.2:default-lifecycle-bindings @ line -1
[WARNING]   Used in module(s):
[WARNING]    * io.dagger:dagger-codegen-maven-plugin:1.0.0-SNAPSHOT (dagger-codegen-maven-plugin/pom.xml)
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6321 Deprecate export functions?

Overview

I regularly see developers get confused (sometimes severely) by File.Export, Directory.Export, and Container.Export, to the point of failing to "get" Dagger, and sometimes even giving up entirely.

I think we should consider the possibility of deprecating these functions.

The timing matters, because we are currently discussing a new CLI experience for Zenith (dagger call etc) which might rely on these export functions. So it seems important to discuss the future of these e...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
sondrelg
[dagger] New discussion #6324: Speeding up Rust SDK execution

Hi!

Thanks, first of all, to the creators of dagger. Really cool project 👏

Background

I have a short list of operations I would like my CI run to perform, for my Rust monorepo:

cargo clippy -p {{ function-name }} -- --deny warnings
cargo test -p {{ function-name }}
cargo deny --manifest-path "./functions/{{ function-name }}/Cargo.toml" check
cargo +nightly udeps -p {{ function-name }}
cargo audit

To try and make this reasonably efficient, I've first de...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6325 feat: add llb to progress output

The idea behind this PR is to attach the JSON representation of the underlying LLB to each vertex in the progress output. This data should be available to be ingested by the cloud service, so it can be used for better visualizations.

This piece is just the client-side part. To see an example, you can use _EXPERIMENTAL_DAGGER_PROGROCK_JOURNAL:

$ _EXPERIMENTAL_DAGGER_PROGROCK_JOURNAL=/tmp/progrock.json dagger call -m github.com/jedevc/daggerverse/hugo build --target git@github.com:...
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6326 feat: global go client

Fixes #6289, and enables #6296.

As suggested in #6289, we create a new dagger.io/dagger/dag package which contains all the same methods that a client does. The client itself is constructed lazily behind the scenes on first access.

Opening since I've got something working, but I've also broken a few other things that need tidying up :tada:

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6328 Fix module name conflicts with core types

Related discord question: #1190052885686407179 message

Right now a module that has the same name as a core type (e.g. Module, Container, etc.) will eventually result in an error. This is not only due to conflicts in the SDK code (depending on the SDK language) but ultimately also on the graphql schema level.

At a bare minimum, we should error out in dagger mod init if a module has a name that will result in a conflict. T...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
samalba
[dagger/dagger] Issue opened: #6329 dagger mod init fails with confusing error message when there is already a go.mod

How to reproduce

go mod init github.com/samalba/sandbox/test
dagger mod init --name=test --sdk=go

Output

✘ generatedCode ERROR [0.57s]
✘ exec /usr/local/bin/codegen --module . --propagate-logs=true --introspection-json-path /schema.json ERROR [0.34s]
┃ Error: load package ".": err: exit status 1: stderr: go: go.mod requires go >= 1.21.5 (running go 1.21.3; GOTOOLCHAIN=local)     
✘ generating go module: test ERROR [0.04s]
├ [0.00s] go mod tidy
┃ writ...
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6331 chore(deps): bump pytest-httpx from 0.27.0 to 0.28.0 in /sdk/python

Bumps pytest-httpx from 0.27.0 to 0.28.0.

Release notes
Sourced from pytest-httpx's releases.

0.28.0 (2023-12-21)
Changed

Requires httpx==0.26.*

Changelog
Sourced from pytest-httpx's changelog.

[0.28.0] - 2023-12-21
Changed

Requires httpx==0.26.*

Commits

0abf1eb Merge pull request #130 from Colin-b/develop
ef6fc63 Merge pull request #129 from Colin-b/feature/latest_httpx
7a611eb Merge remote-tracking branch 'origin/develop' into feature...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6333 chore(deps): bump anyio from 4.1.0 to 4.2.0 in /sdk/python

Bumps anyio from 4.1.0 to 4.2.0.

Release notes
Sourced from anyio's releases.

4.2.0

Add support for byte-based paths in connect_unix, create_unix_listeners, create_unix_datagram_socket, and create_connected_unix_datagram_socket. (PR by Lura Skye)

Enabled the Event and CapacityLimiter classes to be instantiated outside an event loop thread

Broadly improved/fixed the type annotations. Among other things, many functions and methods that take variadic...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6335 chore(deps): bump mypy from 1.7.1 to 1.8.0 in /sdk/python

Bumps mypy from 1.7.1 to 1.8.0.

Changelog
Sourced from mypy's changelog.

Mypy Release Notes
Next release
Mypy 1.8
We’ve just uploaded mypy 1.8 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:
python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.
Type-checking Improvements

Do...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6336 chore(deps): bump golang.org/x/oauth2 from 0.13.0 to 0.15.0

Bumps golang.org/x/oauth2 from 0.13.0 to 0.15.0.

Commits

6e9ec93 go.mod: update golang.org/x dependencies
e067960 go.mod: update golang.org/x dependencies
4c91c17 google: adds header to security considerations section
See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/oauth2&package-manager=go_modules&previous-version=0.13.0&new-version=0.15....

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6337 chore(deps): bump github.com/containerd/stargz-snapshotter from 0.14.3 to 0.15.1

Bumps github.com/containerd/stargz-snapshotter from 0.14.3 to 0.15.1.

Release notes
Sourced from github.com/containerd/stargz-snapshotter's releases.

v0.15.1
Notable Changes

Removed support for CRI v1alpha2 API that was deprecated in containerd v1.7 (#1175)
Make timeout per-request (#1181), thanks to @​Kern--
fix: rollback snapshot to prevent bolt deadlock (#1326), thanks to @​goller
Protect node.ents and node.entsCached with a mutex (#13...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6338 chore(deps): bump github.com/google/uuid from 1.4.0 to 1.5.0

Bumps github.com/google/uuid from 1.4.0 to 1.5.0.

Release notes
Sourced from github.com/google/uuid's releases.

v1.5.0
1.5.0 (2023-12-12)
Features

Validate UUID without creating new UUID (#141) (9ee7366)

Changelog
Sourced from github.com/google/uuid's changelog.

1.5.0 (2023-12-12)
Features

Validate UUID without creating new UUID (#141) (9ee7366)

Commits

4d47f8e chore(master): release 1.5.0 (#145)
9ee7366 feat: Validate UUID without creating new ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6339 chore(deps): bump google.golang.org/grpc from 1.59.0 to 1.60.1

Bumps google.golang.org/grpc from 1.59.0 to 1.60.1.

Release notes
Sourced from google.golang.org/grpc's releases.

Release v1.60.1
Bug Fixes

server: fix two bugs that could lead to panics at shutdown when using NumStreamWorkers (experimental feature).

Release 1.60.0
Security

credentials/tls: if not set, set TLS MinVersion to 1.2 and CipherSuites according to supported suites not forbidden by RFC7540.

This is a behavior change to bring us into better ali...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6340 chore(deps): bump github.com/charmbracelet/bubbles from 0.16.1 to 0.17.1

Bumps github.com/charmbracelet/bubbles from 0.16.1 to 0.17.1.

Release notes
Sourced from github.com/charmbracelet/bubbles's releases.

v0.17.1
What's Changed

feat: upgrade bubbletea and remove deprecated code by @​aymanbagabas in charmbracelet/bubbles#448

Full Changelog: https://github.com/charmbracelet/bubbles/compare/v0.17.0...v0.17.1

Thoughts? Questions? We love hearing from you. Feel free to reach out on Twitter, The Fediverse, or on Discor...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6341 chore(deps-dev): bump eslint from 8.52.0 to 8.56.0 in /sdk/nodejs

Bumps eslint from 8.52.0 to 8.56.0.

Release notes
Sourced from eslint's releases.

v8.56.0
Features

0dd9704 feat: Support custom severity when reporting unused disable directives (#17212) (Bryan Mishkin)
31a7e3f feat: fix no-restricted-properties false negatives with unknown objects (#17818) (Arka Pratim Chaudhuri)

Bug Fixes

7d5e5f6 fix: TypeError: fs.exists is not a function on read-only file system (#17846) (Francesco Trotta)
74739c8 fix: suggestion w...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6342 chore(deps-dev): bump prettier from 2.8.7 to 3.1.1 in /sdk/nodejs

Bumps prettier from 2.8.7 to 3.1.1.

Release notes
Sourced from prettier's releases.

3.1.1
🔗 Changelog
3.1.0
diff
🔗 Release note
3.0.3
🔗 Changelog
3.0.2
🔗 Changelog
3.0.1
🔗 Changelog
3.0.0
diff
🔗 Release note
3.0.0-alpha.6
What's Changed

Update .d.ts files of plugins to use export default ... by @​fisker in prettier/prettier#14435

Other changes since v2, see 3.0.0-alpha.1 release notes
Full Changelog: https://github.com/prettier/prettier/compare/3.0....

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6343 chore(deps-dev): bump typescript from 5.2.2 to 5.3.3 in /sdk/nodejs

Bumps typescript from 5.2.2 to 5.3.3.

Release notes
Sourced from typescript's releases.

TypeScript 5.3.3
For release notes, check out the release announcement.
For the complete list of fixed issues, check out the

fixed issues query for Typescript 5.3.0 (Beta).
fixed issues query for Typescript 5.3.1 (RC).
fixed issues query for Typescript 5.3.2 (Stable).
fixed issues query for Typescript 5.3.3 (Stable).

Downloads are available on:

NuGet package
...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6344 chore(deps-dev): bump eslint-config-prettier from 9.0.0 to 9.1.0 in /sdk/nodejs

Bumps eslint-config-prettier from 9.0.0 to 9.1.0.

Changelog
Sourced from eslint-config-prettier's changelog.

Version 9.1.0 (2023-12-02)

Added: [unicorn/template-indent], (as a [special rule][unicorn/template-indent-special]). Thanks to Gürgün Dayıoğlu (@​gurgunday)!
Changed: All the [formatting rules that were deprecated in ESLint 8.53.0][deprecated-8.53.0] are now excluded if you set the ESLINT_CONFIG_PRETTIER_NO_DEPRECATED environment...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6345 chore(deps): bump smallrye-graphql.version from 2.5.1 to 2.6.1 in /sdk/java

Bumps smallrye-graphql.version from 2.5.1 to 2.6.1.
Updates io.smallrye:smallrye-graphql-client-api from 2.5.1 to 2.6.1

Updates io.smallrye:smallrye-graphql-client-implementation-vertx from 2.5.1 to 2.6.1

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6346 chore(deps-dev): bump org.assertj:assertj-core from 3.24.2 to 3.25.0 in /sdk/java

Bumps org.assertj:assertj-core from 3.24.2 to 3.25.0.

Release notes
Sourced from org.assertj:assertj-core's releases.

v3.25.0
:jigsaw: Binary Compatibility
The release is binary compatible with the previous minor version.
:no_entry_sign: Deprecated
Core

Deprecate the following date/time related assertions in favor of isCloseTo:

isEqualToIgnoringHours
isEqualToIgnoringMinutes
isEqualToIgnoringSeconds
isEqualToIgnoringMillis
isEqualToIgnoringNanos
isInS...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6347 chore(deps-dev): bump org.mockito:mockito-core from 5.6.0 to 5.8.0 in /sdk/java

Bumps org.mockito:mockito-core from 5.6.0 to 5.8.0.

Release notes
Sourced from org.mockito:mockito-core's releases.

v5.8.0
Changelog generated by Shipkit Changelog Gradle Plugin
5.8.0

2023-12-01 - 15 commit(s) by Andreas Turban, Mikaël Francoeur, dependabot[bot], jfrantzius
#3000: fix ArrayIndexOutOfBoundsException (#3190)
Bump com.diffplug.spotless from 6.23.1 to 6.23.2 (#3188)
Bump com.diffplug.spotless...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6348 chore(deps): bump de.m3y.maven:inject-maven-plugin from 1.4 to 1.5 in /sdk/java

Bumps de.m3y.maven:inject-maven-plugin from 1.4 to 1.5.

Release notes
Sourced from de.m3y.maven:inject-maven-plugin's releases.

1.5
This release migrates from previous javassist to bytebuddy for injecting byte-code.

Migrate from javassist to bytebuddy #148
Note: As a side effect, the plugin now also works with latest JDK 21

See release milestone for details.

Commits

77fa2a8 [maven-release-plugin] prepare release inject-maven-plugin-1...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6350 chore: remove dockerfile

cc @marcosnils as the last person to edit this file :eyes:

As far as I'm aware, this Dockerfile isn't actually used for anything? It's not used in our own internal/mage package, and it isn't referenced anywhere in our docs, I think this is legacy from before dagger was built in dagger? Also, since go 1.21, this dockerfile no longer even builds! (gotta love the best error message of all time invalid go version '1.21.5': must match format 1.23)

Additionally, the constants for go version...

vivid lintelBOT
ornate vigilBOT
#
marcosnils
[dagger/dagger] New comment on discussion #6324: Speeding up Rust SDK execution

That said, a cached run still takes ~18 seconds to execute, whereas just running the steps outside of dagger takes ~4 seconds.

that's quite strange. It definitely shouldn't take that much time. Were you able to spot which operation in your pipeline is the one that takes 18 seconds? I'd assume there's a misconfiguration somewhere that's causing the caching not to be kicking in accordingly.

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6354 feat: improve service stop semantics

This PR changes how services are stopped:

  1. Calls to Service.Stop now block until the service exits
  2. Services are now stopped using SIGTERM instead of SIGKILL
  • This can be changed by setting the new kill option to true, which sends SIGKILL instead
  1. Services that are automatically cleaned up (e.g. client disconnection, all bindings are detached) are now gracefully stopped using SIGTERM+SIGKILL after a default 10 seconds has passed.
  • This isn't really ideal reall...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
kjuulh
[dagger/dagger] New comment on discussion #6324: Speeding up Rust SDK execution

From what I can see. It can be because there is a lot of data to be sent between the different docker contexts.

When you got the 4 sec time, was that just running pure cargo, or was it an image and then the commands?

It may also be because when you load the directory. It by default loads everything. I can see that it is used above, but you should be able to see it in the output how much it is loading it will say something like Transferring context (x Mbs)

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6355 `.Export` partially transfers files and then gets stuck

This seems to be the same issue as upstream in buildkit: https://github.com/moby/buildkit/issues/2950 (cc @sipsma)

We've seen from a customer, as well as reported on discord: #1191805052378161192 message:

I've not seen the issue happening without a large amount of files being exported, so this might be related indeed

We should try and commit some debugging resources to trying to track down the deadlock upstream, so that we ...

vivid lintelBOT
#
Juneezee
[dagger/dagger] Pull request opened: #6356 fix(cmd/shim): fix panic when exec-ing an unknown command

Fixes https://github.com/dagger/dagger/issues/6266.

When an unknown command is executed, the returned error is *exec.Error.

Example code (Go playground: https://go.dev/play/p/ybni3fJpo46):

package main

import (
	"log"
	"os/exec"
)

func main() {
	if err := exec.Command("foobar").Run(); err != nil {
		log.Printf("%T: %+v", err, err)
	}

	if err := exec.Command("false").Run(); err != nil {
		log.Printf("%T: %+v", err, err)
	}
}

Output:

2009/11/...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6357 CLI: support for `-o` flag

As described in this comment, we want to add a -o flag for smartly redirecting output in dagger call.

This flag would be applicable for any return type and result in that result being written to the caller's filesystem rather than written to stdout.

For simple cases like a string return type, this would have the same end effect as a shell redirection. The difference is that we would allow some special...

#
sipsma
[dagger/dagger] Issue opened: #6358 CLI: improve display of core API in `--help` and `dagger functions` output

https://github.com/dagger/dagger/pull/6293 exposed the core API in dagger call, which results in --help and dagger functions output to be pretty overwhelming and messy when dealing with core types like Container.

There's most likely going to be a lot of little things needed to address this, possibly including but not limited to:

  1. displaying multi-line doc strings better
  2. sorting of functions more cleanly

I think we should start out trying to do this purely from the CLI wit...

vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6359 CLI: update handling of various core types

As described in this comment (and that issue generally), we need to make some adjustments to the CLIs behavior when dagger call ends in various core types.

There's a handful of related adjustments needed here. Creating as a checklist for now to save issue spam; can break down into more issues if useful though. I'm including my initial suggestions on how to handle these, but to be clear these are not finalized decisio...

#
sipsma
[dagger/dagger] Issue opened: #6360 CLI: decide whether to support `id` or to continue hiding

Right now https://github.com/dagger/dagger/pull/6293 went with hiding id fields on types from the CLI because IDs are not generally safe to re-use across sessions.

However, https://github.com/dagger/dagger/pull/6297 will drastically improve the ID system and opens the possibility of allowing IDs to be re-used in some circumstances. We can thus consider whether to embrace that in the CLI and add some mechanism for getting the IDs of objects and re-use those.

To be clear, this isn't _su...

#
sipsma
[dagger/dagger] Issue opened: #6361 CLI: enable `shell` and `up` functionality as chainable APIs

As described in this comment, we want to centralize call+shell+up to all be part of a common CLI command via exposing shell and up functionality as chainable APIs from Container/Service.

The original idea described there was to update the core API with "side-effect-only" APIs that result in the relevant behavior being triggered (similar to how export APIs work), e.g. as strawman starter ideas

  1. A `she...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6362 go sdk: comment the new codegen w/ docs+examples

Follow-up for previous PR that added a lot of this fairly intricate new codegen (https://github.com/dagger/dagger/pull/6254#issuecomment-1877837257)

I was a bit torn on whether to include as many concrete examples in the comments since it technically adds a small maintainance burden to keep the comments up to date whenever we change things.

However, in practice as working on this code I always ended up writing temporary throw-away comments like this (to help reason through the code to w...

ornate vigilBOT
#
cathaysia
[dagger] New discussion #6363: Is dagger suitable for me?

hi. Hello everyone. I'm currently looking for a cicd tool that will suit my needs. I hope someone can give me some opinions.

My team currently uses Jenkins as the cicd tool, but you know what. Jenkins is not very convenient to use. In the past week, I spent a lot of effort getting jekins in docker and docker in jenkins running. Soon, I will have to write many, many scripts (or Makefiles) on it. It makes me miss the fun days of using Github Action.

I have been trying to replace jenki...

vivid lintelBOT
#
keax
[dagger/dagger] Issue opened: #6364 engine: cannot enter cgroupv2

What is the issue?

I encounter errors when I run Kubernetes engine (k3s) inside a Dagger pipeline.

It works well with a previous version of Dagger (0.6.4).
It seems to be related to this issue : #5593 and with a potential fix https://github.com/moby/buildkit/pull/4308
I don't understand very well what this fix does.
Dagger version 0.9.5 uses the buildkit version github.com/moby/buildkit v0.13.0-beta1.0.20231107172747-c4f191410a41
So it is supposed to be fixed but I think I missed ...

vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
samalba
[dagger/dagger] New comment on discussion #6363: Is dagger suitable for me?

Hello @cathaysia,

Before digging into the details, yes, you can easily execute your tests, build and run docker containers, manage artifacts, call APIs, etc... from a Dagger pipeline. You can then run this pipeline either locally from your machine or from Jenkins or from any infrastructure. Dagger Cloud can then allow you to visualize the running status (logs and errors) from a Web UI.

I would go over the following resources, in that order (relevant to the point...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6366 Go SDK: support objects from a user's module being passed as interface values

Follow-up to https://github.com/dagger/dagger/pull/6254

Right now there's a fairly annoying limitation in that in Go modules it's not possible to use an object you've defined in your module to implement an interface you've defined in your module.

This is due to the fact that interfaces are passed by ID by right now objects defined in a module are still passed by raw json in the context of themselves (some previous discussion with more context on why [here](https://github.com/dagger/dagg...

ornate vigilBOT
#
samalba
[dagger] Discussion #6363 marked answered: Is dagger suitable for me?

Hello @cathaysia,

Before digging into the details, yes, you can easily execute your tests, build and run docker containers, manage artifacts, call APIs, etc... from a Dagger pipeline. You can then run this pipeline either locally from your machine or from Jenkins or from any infrastructure. Dagger Cloud can then allow you to visualize the running status (logs and errors) from a Web UI.

I would go over the following resources, in that order (relevant to the point...

vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6369 🐞 Optional type in module field yields error

What is the issue?

When using Optional in a module field type (combined with constructors) Dagger yields the following error:

Error: query module objects: json: error calling MarshalJSON for type *dagger.Module: returned error 400 Bad Request: failed to get schema for module "ci": failed to create field: failed to get mod type for field "githubActor"

Dagger version

dagger v0.9.5 (registry.dagger.io/engine) darwin/arm64

Steps to reproduce

Create a module with optional f...

ornate vigilBOT
#
helderco
[dagger/dagger] New comment on discussion #6318: Zenith: How to use private dependencies with the Python SDK

Yes, that's what I'd suggest for now, the problem I see with this is how do pass the secret value (for gitlab's token)? You'll have to cache that token into the runtime container in the my-cutom-runtime module as a normal env var or explicitly in the source code if you're using Dagger's native secrets. I'd suggest using an external secret store like infisical ([website](https://i...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6370 fix: forbid go optional wrapper in struct definition

Fixes https://github.com/dagger/dagger/issues/6369.

This is easier than attempting to handle this case, which would make the optional handling logic spill out to even more places (which I'd really like to avoid for now).

@sipsma @vito what do you think about removing the Optional type and relying entirely on // +optional for the RC? If we're going to break some things, feels like this is a good candidate.

ornate vigilBOT
ornate vigilBOT
#
skycaptain
[dagger/dagger] New comment on discussion #6318: Zenith: How to use private dependencies with the Python SDK

Thank you for your insights. At the moment, I am in the process of exploring Dagger modules to ascertain their suitability for our needs and to identify areas where adjustments to my CX libraries may be necessary. My use case is that I have a Python library so that I can reuse code (I know that is kind of contrary to the modules approach) in my CX pipelines in multiple projects with the CX pipelines being Dagger modules. The question now is, how do I include my library from a private reposito...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6371 feat: move ExecError definition to codegen

Fixes DEV-2853.

This allows modules to also access ExecErrors, so they can analyze return codes, etc.

Sidenote: this adds to our little template collection. In light of the discussion around #6213 and trying to move more to jennifer where possible for our Go SDK codegen, I think the following divide makes sense:

  • All "static" code can be defined in templates
  • All "dynamic" code generated from the graphql introspection / typedefs s...
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6372 deps: update containerd to v1.7.11

Diffs:

I started by bumping conta...

vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #6373 Rename nodejs to typescript

Update

  • sdk/nodejs -> sdk/typescript
  • All mentions of NodeJS SDK to Typescript SDK
  • Generator, Mage and Runtime wording
  • Docs reference
  • Docs snippet tab went from Node.js to Typescript
  • Relative path and filenames.
  • Release process (@jedevc I'll be with you for the next release because this PR might break things in our release)

Note

  • I kept mention of a Node.js environment, since it's not related to the SDK but to the environment (or runtime) required ...
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6374 Update buildkit past ca71a446d9f4e31ca7c4fa30fb8fb8861b8b0117

Currently, we are stuck on the current version of buildkit, since https://github.com/moby/buildkit/pull/4347#issuecomment-1872217571 seems to have introduced a deadlock.

To reproduce: update buildkit in go.mod to ca71a446d9f4e31ca7c4fa30fb8fb8861b8b0117, and run a large number of integration tests. The engine will lock up after some time - after sending SIGUIT to the engine, you can observe the stack trace:

goroutine 441 [sync.Mutex.Lock, 4 minutes]:
runtime.gopark(0xc01d5eb...
vivid lintelBOT
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6375 ✨ Allow pulling refs from a git repo

What are you trying to do?

While working on replacing a GHA workflow with pure Dagger (ie. no initial checkout step the GHA workflow), I noticed the directly pulling a commit from a ref not being fetched by default (ie. belonging to a PR: refs/pull/5/merge) fails.

This is quite understandable, but I had a hard time figuring out a workaround.

Here is what I tried:

source = dag.Git("https://github.com/openmeterio/benthos-openmeter.git", GitOpts{
	KeepGitDir: true,
}).Co...
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6377 deps: update cloudflare/circl to v1.3.7

This mitigates GHSA-9763-4f94-gfch, and fixes the failing scan engine job on main: https://github.com/dagger/dagger/actions/runs/7460295076/job/20298322166#step:3:307.

github.com/cloudflare/circl is required by go-git:

$ go mod why -m github.com/cloudflare/circl
# github.com/cloudflare/circl
github.com/dagger/dagger/cmd/dagger
github.com/go-git/go-git/v5
github.com/ProtonMail/go-crypto/openpgp
github.com/ProtonMail/go-crypto/openpgp/internal/ecc
github.com/cloudflare/...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #6381 Transition node to typescript in zenith

Replace #6373 with a simpler set of changes that only concerns our codebase and Zenith documentation.

List of updates:

  • Our GitHub Actions (/cc @gerhard)
  • Our release doc (/cc @jedevc)
  • sdk/nodejs -> sdk/typescript
  • Mage sdk:nodejs -> sdk:typescript
  • Our generator
  • I created a label sdk/typescript on GH

I also deleted the reference directory in current_docs, since these are generated files.

This PR should be way easier to review than the previous one si...

ornate vigilBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6382 chore: refactor go codegen templates

This patch refactors the go codegen templates so as to allow a more template-centric approach, further improving upon the approach in #6326.

Essentially, the codegen now looks for every *.go.tmpl file in the templates directory and attempts to render it. This allows for easily adding new code-gen files, which will be useful for when we want to put the server API into a sub-package, i.e. dagger/dagger.gen.go.

The main change here is to rework the visiting, so instead of having the ca...

vivid lintelBOT
#
skycaptain
[dagger/dagger] Issue opened: #6383 🐞 Zenith: Using secrets in SDKs results in error after re-run from cache

What is the issue?

Since SDKs are modules themselves, it is possible to use a custom SDK. In #6318, a custom SDK is used to install additional Python dependencies from a private repository, with the required credentials passed from a .env file. You can find an example at https://github.com/skycaptain/dagger/blob/python-sdk-runtime-with-env/sdk/python/runtime/main.go.

In this example, dag.SetSecret and ctr.WithSecretVariable are used to set the credentials before installing the P...

vivid lintelBOT
#
kjuulh
[dagger/dagger] Pull request opened: #6384 chore(rust-sdk): update-dependency and version

It has been a while. But here are some updates to some dependencies. The update to the version should also include some quite major improvements I didn't release in December. Such as better support for lifetimes in rust.

Also did some minor refactoring with the dependencies, and moved them and the versions to the main Cargo.toml, this should make it easier in the future to bump the version, as well as holistically update dependencies.

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6386 WIP: replace root w/ sources in dagger.json

Fixes https://github.com/dagger/dagger/issues/6291

Very rough draft (been much trickier than I originally imagined), but got some manual tests of init/install/sync working so sending out a checkpoint.

TODOs:

  • [ ] Update the existing integ tests
  • [ ] Add test coverage for new cases
  • [ ] Handle old dagger.json (at min error out w/ helpful message, maybe automatic update for simple cases)
  • [ ] Double check that impact on daggerverse will be minimal
    • From what I've look...
vivid lintelBOT
#
yann-soubeyrand
[dagger/dagger] Issue opened: #6387 🐞 dagger export --output <mydir> doesn’t repect umask when creating <mydir> if it doesn’t already exist

What is the issue?

When running dagger export --output , Dagger creates directory if it doesn’t already exist. However, it does so without respecting the configured umask: on my system, umask is set to 022, so permissions should be 755, but it’s actually 700.

Dagger version

dagger v0.9.5 (registry.dagger.io/engine) linux/amd64

Steps to reproduce

  1. mkdir test
  2. cd !$
  3. dagger mod init --name test --sdk go
cat < main.go
package main

type Te...
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6388 fix: protect against concurrent map access in APIServer

Fixes a potential concurrency issue introduced in https://github.com/dagger/dagger/pull/6167 - it looks like during the lock refactors, we missed a couple cases.

In a past testdev run (https://github.com/dagger/dagger/actions/runs/7459047868/job/20294230186), the tests failed with:

fatal error: concurrent map read and map write

goroutine 569315 [running]:
github.com/dagger/dagger/core/schema.(*APIServer).ServeHTTP(0xc0197f3900, {0x1f9c9a0, 0xc0100aa460}, 0xc00eed5800)
	/app/cor...
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6390 replace shell command with chainable core api

This is one part of our efforts to consolidate dagger CLI commands. It removes dagger shell in favor of using a shell api chained from the core Container type.

For example, previously you'd do:
dagger shell my-ctr

Now you do:
dagger call my-ctr shell

This required some adjustments in the core API. Namely, shellEndpoint: String! is replaced with
shell(args: [String!]) InteractiveShell! on the Container type.

The new InteractiveShell type currently only has a fiel...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6394 chore(deps): bump golang.org/x/tools from 0.15.0 to 0.16.1

Bumps golang.org/x/tools from 0.15.0 to 0.16.1.

Commits

2acb2e6 gopls/internal/test/marker: minor clean up of marker test doc
28b92af internal/typeparams: eliminate remainining compatibility shims
ee35f8e gopls/internal/lsp/source: hovering over broken packages is not an error
67611a1 internal/typeparams: eliminate type aliases
23c86e8 internal/typeparams: delete const Enabled=true and simplify
e46688f gopls/internal/analysis/fillstruct: don't panic with i...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6395 chore(deps): bump github.com/rs/zerolog from 1.30.0 to 1.31.0

Bumps github.com/rs/zerolog from 1.30.0 to 1.31.0.

Commits

8344fc0 Bump actions/checkout from 3 to 4 (#588)
4cb8cc5 Update dependencies
ae9b265 Update Build Status Badge (#589)
1bac5cc added support for NO_COLOR (#586)
b81cc57 Fix the standard logger output example (#584)
ad77222 chore: improve coverage report
95cf29c chore: switch to go-goverage-report action as gocover.io is closing
802c88f chore: adding any function to context (#580)
158e4ad Update logben...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6396 chore(deps): bump golang.org/x/crypto from 0.17.0 to 0.18.0

Bumps golang.org/x/crypto from 0.17.0 to 0.18.0.

Commits

dbb6ec1 ssh/test: skip tests on darwin that fail on the darwin-amd64-longtest LUCI bu...
403f699 ssh/test: avoid leaking a net.UnixConn in server.TryDialWithAddr
055043d go.mod: update golang.org/x dependencies
08396bb internal/poly1305: drop Go 1.12 compatibility
See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6397 chore(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.16.0

Bumps golang.org/x/oauth2 from 0.15.0 to 0.16.0.

Commits

39adbb7 go.mod: update golang.org/x dependencies
4ce7bbb google: add Credentials.GetUniverseDomain with GCE MDS support
1e6999b google: add UniverseDomain to CredentialsParams
See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/oauth2&package-manager=go_modules&previous-version=0.15.0&ne...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6398 chore(deps): bump github.com/spf13/cobra from 1.7.0 to 1.8.0

Bumps github.com/spf13/cobra from 1.7.0 to 1.8.0.

Release notes
Sourced from github.com/spf13/cobra's releases.

v1.8.0
✨ Features

Support usage as plugin for tools like kubectl by @​nirs in spf13/cobra#2018 - this means that programs that utilize a "plugin-like" structure have much better support and usage (like for completions, command paths, etc.)
Move documentation sources to site/content by @​umarcor in spf13/cobra#1428
Add 'one required flag...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6399 chore(deps): bump gql from 3.4.1 to 3.5.0 in /sdk/python

Bumps gql from 3.4.1 to 3.5.0.

Release notes
Sourced from gql's releases.

v3.5.0
IMPORTANT: graphql-core has been reverted to the stable versions 3.2.x
See #427
A new pre-release version will be made shortly with the graphql-core versions 3.3.x
New features

Add execute-timeout argument for gql-cli (#349)
Add HTTPX transport (#370)
Don't try to close the aiohttp session if connector_owner is False (#382)
Feature allow to set the content-type of file ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6400 chore(deps): bump smallrye-graphql.version from 2.6.1 to 2.7.0 in /sdk/java

Bumps smallrye-graphql.version from 2.6.1 to 2.7.0.
Updates io.smallrye:smallrye-graphql-client-api from 2.6.1 to 2.7.0

Updates io.smallrye:smallrye-graphql-client-implementation-vertx from 2.6.1 to 2.7.0

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You...

vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6402 chore(deps-dev): bump prettier from 2.8.7 to 3.1.1 in /sdk/typescript

Bumps prettier from 2.8.7 to 3.1.1.

Release notes
Sourced from prettier's releases.

3.1.1
🔗 Changelog
3.1.0
diff
🔗 Release note
3.0.3
🔗 Changelog
3.0.2
🔗 Changelog
3.0.1
🔗 Changelog
3.0.0
diff
🔗 Release note
3.0.0-alpha.6
What's Changed

Update .d.ts files of plugins to use export default ... by @​fisker in prettier/prettier#14435

Other changes since v2, see 3.0.0-alpha.1 release notes
Full Changelog: https://github.com/prettier/prettier/compare/3.0....

vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6405 add DagQL subtree

This PR merges in the full DagQL repository as a toplevel dagql/ subdirectory, keeping its history intact.

I know we usually do squashed commits for PRs but this feels like a bit of a special case where I'd prefer to keep the git blame since some of the commits have a lot of explanation. In total it's 99 commits which doesn't seem too crazy. I'll probably add another on top to remove the redundant LICENSE file, which would bring us to a pleasant :100:.
...

vivid lintelBOT
#
vito
[dagger/dagger] Issue opened: #6406 🐞 Blobs referenced by private fields in a module object can disappear

What is the issue?

Right now linkDependencyBlobs walks over an object's map[string]any and uses the object's field definitions to convert each any to a HasPBDefinitions so it can find blobs to link to the current session.

The problem is private fields don't actually have a field definition, so we don't know how to convert it to a HasPBDefinitions-able value.

Maybe we need to also register private fields, and just mark them private so they don't show up in the schema?

###...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6407 replace up command with chainable core api

This is one part of our efforts to consolidate dagger CLI commands, along side https://github.com/dagger/dagger/pull/6390. It removes dagger up in favor of using a up api chained from the core Service type.

For example, previously you'd do:
dagger up my-svc --port 8080:80 --port 2222:22

Now you do:
dagger call my-svc up --ports 8080:80 --ports 2222:22
or
dagger call my-svc up --native
or just
dagger call my-svc up (use any available local port)

This change seems...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6408 [WIP] Change `Host` to more clear name

Fixes https://github.com/dagger/dagger/issues/6312

Opening this purely to have a discussion right now, the changes amount to find+replace on the host name right now, but not even entirely yet (need to update tests, etc.)

Background

The name Host has become extremely confusing in the context of modules. So far, it seems that more often than not users very reasonably assume that in a module when they use dag.Host they are interacting with the original "host" where the module was ...

vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6409 ✨ Allow cache volumes as CLI parameters

What are you trying to do?

I have a module with the following constructor:

func New(
	// ...

	// Disable mounting cache volumes.
	disableCache Optional[bool],

	// Module cache volume to mount at /go/pkg/mod.
	modCache Optional[*CacheVolume],

	// Build cache volume to mount at ~/.cache/go-build.
	buildCache Optional[*CacheVolume],
) *Go {
	// ...
}

Currently, running this module in the CLI fails due to CacheVolume being an unsupported CLI argument:

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6410 sdk: Bump engine dependency to v0.9.6

This PR was not auto-generated. The v0.9.6 publish job failed: #1193910832166342777 message. However, only nix failed, everything else succeeded - so I've just opened this PR manually to continue with the release (similar to as in https://github.com/dagger/dagger/pull/6309).

The nix issue seems related to branch protection, I suspect that dagger/nix has some branch protection that dagger/homebrew-tap doesn't have (the ...

vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6412 Bump golang.org/x/tools from 0.15.0 to 0.17.0

Bumps golang.org/x/tools from 0.15.0 to 0.17.0.

Commits

0b1f1d4 gopls/internal/lsp/cache: (re-)ensure clean shutdown
706525d gopls/internal/lsp/source/completion: support postfix completion (iferr,
581c0b3 gopls/internal/lsp/source: add receiver name to stubbed methods
c95fa0f gopls/internal/test: skip marker tests on darwin builders if -short
25a0e9d go.mod: update golang.org/x dependencies
c9c95f9 internal/refactor/inline: improve a confusing error messa...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jpadams
[dagger/dagger] Issue opened: #6415 🐞 Seeing `CLI_VERSION = "0.9.5"` in python and typescript SDKs after `dagger mod init` or `sync`

What is the issue?

When creating a new module via dagger mod init or updating a module with dagger mod sync, under dagger CLI 0.9.6 I'm ending up with files like

python:
sdk/src/dagger/_engine/_version.py

# Code generated by dagger. DO NOT EDIT.

CLI_VERSION = "0.9.5"

typescript:
sdk/provisioning/default.ts

// Code generated by dagger. DO NOT EDIT.
export const CLI_VERSION = "0.9.5"

Dagger version

dagger v0.9.6 darwin and windows arm64

...

vivid lintelBOT
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6416 ✨ Add name field to File

What are you trying to do?

Files passed to dagger call currently does not preserve the name of a file passed to it.

Why is this important to you?

Some tools often require the original file name (for example: Spectral requires the file name do determine the file type)

Discussed it with @sipsma here: #daggernauts message

How are you currently working around this?

I pass the original file name to the mod...

vivid lintelBOT
vivid lintelBOT
#
giovannicandido
[dagger/dagger] Issue opened: #6417 🐞 Dagger cli messes up with mouse in windows terminal

What is the issue?

After running the command dagger run go run xxx/main.go when it finishes, the mouse becomes useless.
This is annoying especially because you can't scroll and copy the output or any information in terminal after that.

Dagger version

dagger v0.9.6 (registry.dagger.io/engine) windows/amd64

Steps to reproduce

Use any terminal in windows terminal. Tested with Powershell but other shells affected

Log output

https://github.com/dagger/dagger/assets/9158...

vivid lintelBOT
#
skycaptain
[dagger/dagger] Issue opened: #6418 🐞 entrypoint evaluated when calling with_exec

What is the issue?

Consider the following example

dag.container().from_("registry.dagger.io/engine:v0.9.6").with_exec(["apk", "add", "--no-cache", "nodejs"])

This example errors with:

218: exec dagger-entrypoint.sh apk add --no-cache nodejs npm
218: [0.06s] mkdir: can't create directory '/sys/fs/cgroup/init': Read-only file system
218: exec dagger-entrypoint.sh apk add --no-cache nodejs npm ERROR: process "dagger-entrypoint.sh apk add --no-cache nodej...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6420 chore: ensure git clone step in sdk publish always runs

Found this during the release of v0.9.6.

This step should always run - otherwise, it can accidentally cache, and we wouldn't run the rest of the pipeline, even when we would have new tags/branches/commits.

The "correct" solution long term is to ensure that we use the top-level client.Git method here - however, due to a weird upstream bug, we end up confusing tags and branches when cloning, which causes the filter-branch command we use to fail. (I'm following up with this upstream).

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6421 `WithMountedDirectory`/`WithMountedFile` does not use content-based cache in `withExec`

See https://github.com/moby/buildkit/blob/a09112603a36532b8e44ade74ee3fd8383f4b708/solver/llbsolver/ops/exec.go#L260-L262.

This behavior is explicitly disabled in buildkit - mounts only use slow content-based cache when both:

  • The mount is not the root mount
  • The mount is read-only

This is to avoid computing expensive content-based checksums. The root mount case often makes sense for us as well: I really struggle to imagine a scenario where caching this would be super useful. It w...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6425 services: key services by server id; auto start cli service flags

Before this, services provided as CLI flag values were not automatically started, so user module code needed to explicitly start them (or implicitly start them via a service binding), which is somewhat surprising and boilerplatey since there's not really any cases where a user wouldn't want the host service to already be running. This is debatable though.

Now, the CLI calls start on host services provided as flag args.

In order for this to work, there was also a change needed in the int...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vikram-dagger
[dagger/dagger] Issue opened: #6426 🐞 Excessive wait time on first command execution following version upgrade

What is the issue?

After upgrading to a new Dagger version, attempting to create a module fails on first attempt. The new engine container is downloaded and started, but command execution seems to "freeze". The only workaround is to cancel and reattempt the command and in this case it executes correctly and immediately.

Dagger version

dagger v0.9.6 (registry.dagger.io/engine) linux/amd64

Steps to reproduce

Initial setup:
Dagger 0.9.5 installed and 0.9.5 container running.
...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6430 🐞 Service stop semantics work differently between SDK and Modules

What is the issue?

Initially repoted by @matipan here: #daggernauts message

Given the current function

func (m *Foo) Nginx(ctx context.Context) error {
	svc := dag.Container().
		From("nginx").
		WithExposedPort(80).
		AsService()
	tunnel, err := dag.Host().Tunnel(svc).Start(ctx)
	if err != nil {
		return err
	}
	defer tunnel.Stop(ctx) 

	endpoint, err := tunnel.Endpoint(ctx)
	if err != nil {
		r...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #6433 fix: typescript module init in existing project

This commit improve the entrypoint registration:

  • Use reflect-metadata to register modules' class constructor.

This commit includes several changes around the runtime:

  • Add a proxy entrypoint to execute dagger entrypoint as a sub-package.
  • Install the TypeScript SDK in the project package.json if required.
  • Update project tsconfig.json if the paths alias is missing.
  • Add src/index.ts if missing in the codebase.
  • Simplify template install dagger as a sub package.

Al...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
#
skycaptain
[dagger] New discussion #6436: Zenith: How to use shell command with entrypoint?

I'm trying to understand how the dagger-shell command works with entrypoints. Consider the following example:

...
@function
def debug() -> dagger.Container:
    return (
        dag.container()
        .from_("debian:bookworm")
        .with_exec(["apt-get", "update"])
        .with_exec(["apt-get", "install", "-y", "python3", "python3-venv"])
        .with_exec(["python3", "-m", "venv", "/venv"])
        .with_new_file(
            "/usr/bin/local/entrypoint.sh",
   ...
vivid lintelBOT
ornate vigilBOT
#
helderco
[dagger/dagger] New comment on discussion #6436: Zenith: How to use shell command with entrypoint?

There's an issue in shell with reporting errors. So a failed execution of the entrypoint just exits without a proper non-zero return code and error message. In your case, you can debug manually with --entrypoint, which forces the command to use:

❯ dagger shell debug --entrypoint=bash
root@urjt6mj2624m0:/# /usr/bin/local/entrypoint.sh /bin/bash
/usr/bin/local/entrypoint.sh: line 2: !#/bin/sh: No such file or directory
root@urjt6mj2624m0:/# which sh
/usr/bin/sh

This exp...

ornate vigilBOT
#
skycaptain
[dagger/dagger] New comment on discussion #6436: Zenith: How to use shell command with entrypoint?

Indeed, the command not returning an error misled me. I was finally able to get it to work. The error in the hashbang was just a typo when I wrote the ticket, nevertheless a good catch. There was another issue with the sourced script. Thanks for pointing me into the right direction and showing how to debug.

As for virtual environments, I'm aware of this approach. I only used it here as an example to simplify things, so I apologize for any confusion. There are other ecosystems, such as Yoct...

vivid lintelBOT
#
AnaisUrlichs
[dagger/dagger] Issue opened: #6438 🐞 Error: failed to automate vcs -- when trying to install different Dagger integrations

What is the issue?

I get the following error message when I try to install the Trivy integration:

❯ dagger mod install github.com/jpadams/daggerverse/trivy@21f9211e5727a0b6acce8a0ab6c35d01896dc642
✘ generatedCode ERROR [1.24s]
✘ exec /usr/local/bin/codegen --module . --propagate-logs=true --introspection-json-path /schema.json ERROR [0.64s]
┃ Error: generate code: template: module.go.tmpl:74:3: executing "_dagger.gen.go/module.go.tmpl" at : error calling ModuleMainSrc: cann
...
ornate vigilBOT
#
helderco
[dagger] Discussion #6436 marked answered: Zenith: How to use shell command with entrypoint?

There's an issue in shell with reporting errors. So a failed execution of the entrypoint just exits without a proper non-zero return code and error message. In your case, you can debug manually with --entrypoint, which forces the command to use:

❯ dagger shell debug --entrypoint=bash
root@urjt6mj2624m0:/# /usr/bin/local/entrypoint.sh /bin/bash
/usr/bin/local/entrypoint.sh: line 2: !#/bin/sh: No such file or directory
root@urjt6mj2624m0:/# which sh
/usr/bin/sh

This exp...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
TomChv
[dagger/dagger] Pull request opened: #6440 feat(tsSDK): fix constructor registration

This commit updates entrypoint loading by:

  • Loading args type in constructor

This commit fixes argument loading by:

  • Correctly serializing input and parentJSON
  • Fixing loadArg to correctly supports list
  • Fixing loadArg to load object with ID only
  • Refactor load to avoid repetition

This commit fixes register by:

  • Correctly assigning constructor if there's

Fixes DEV-3309

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6442 Improved logging from Modules

Questions around how to write progress directly from module code come up fairly often. Right now all modules can do is write to stdout+stderr and it will show up in the progress output in the same way other exec output shows up. Very barebones.

Various problems that have been raised w/ the current situation:

  1. Can't differentiate between debug + normal + error logs. You just write every log or no log.
    • In theory you could make the "log level" of your function a parameter, but that c...
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6444 Don't mix core API into module API

Problem

When the Dagger Engine loads a module and serves its API, that API is 1) the module's functions and types, mixed with 2) the engine's core functions and types. This makes the resulting API noisy and difficult to explore, because the core API is huge and drowns out the rest.

Solution

  1. Only expose the module's functions and types in its API.
  2. If we want to support calling core functions from the CLI, expose the core API as a module that can be loaded the usual way: ...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6452 Potential issues introduced by `fix: skip head merge commits on pull requests`

A couple small issues potentially introduced by https://github.com/dagger/dagger/pull/6253 (cc @grouville @marcosnils)

So, prior to this PR, we didn't perform any network operations when starting a dagger session. However, after this, we start doing git operations. My worry is that this has introduced subtle bugs with how this works.

Firstly, we could get a potential timeout after 30s if the git operation takes longer than 30s (can't find a CI example off the top of my head, but ...

vivid lintelBOT
vivid lintelBOT
#
dariusj1
[dagger/dagger] Issue opened: #6454 ✨ <Feedback>

What are you trying to do?

I started looking into Dagger after I saw it on one DevOps Engineers's video in Youtube. I liked the idea: juggling containers, which are supposed to be the same everywhere: cloud, CI runners, localhost, server, etc. And so I started playing with it and I've been playing with it for a few days now. Here is my feedback

  • The Good
    • Everything is in containers. This allows me to have complete control over what's happening where
    • Programming language sup...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6456 propagate progrock parents

These are the necessary changes for the Zenith visualization experiments I've been working on. I figure we can sneak these out even if we don't have anything that uses them yet, just to have the ability to record a journal file and test the new UI/TUI against it.

The only real change is that Progrock now supports parent/child vertex relationships, which are analogous to parent/child spans in OpenTelemetry. The DagQL telemetry now uses this instead of the weird group-that's-really-a-vertex ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6462 🐞 Dagger transfers file to build context always regardless of caching

What is the issue?

Seems like Dagger is always transferring files to the build context even if they are cached.

Dagger version

v0.9.6

Steps to reproduce

given the following pipeline where bar is a very large file:

	_, err = client.Container().From("alpine").WithMountedFile("/tmp/image", client.Host().File("bar")).
		WithExec([]string{"ls", "-la", "/tmp/"}).Sync(ctx)
	if err != nil {
		return err
	}

first time the pipeline runs I get the following out...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6468 Bump prettier from 2.8.7 to 3.2.4 in /sdk/typescript

Bumps prettier from 2.8.7 to 3.2.4.

Release notes
Sourced from prettier's releases.

3.2.4

Fix .eslintrc.json format #15947

🔗 Changelog
3.2.3

Format tsconfig.json file with jsonc parser #15927

🔗 Changelog
3.2.2
🔗 Changelog
3.2.1
🔗 Changelog
3.2.0
diff
🔗 Release note
3.1.1
🔗 Changelog
3.1.0
diff
🔗 Release note
3.0.3
🔗 Changelog
3.0.2
🔗 Changelog
3.0.1
🔗 Changelog
3.0.0
diff
🔗 Release note
3.0.0-alpha.6
What's Changed

Update .d.ts files of plugins t...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6469 Module engine version tracking

Currently based on top of https://github.com/dagger/dagger/pull/6386, 3 most recent commits are unique

Goals:

  1. Add engine version module was most recently updated with to dagger.json for the purposes of tracking as metadata
    • Currently updates on dagger mod init/install/sync
    • This also opesn up a path to serving modules compatible schemas based on their version
      • This PR doesn't include the all the internal functionality for doing so, but in the future if we make backwa...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sagikazarmark
[dagger/dagger] Issue opened: #6474 ✨ Add WithName to File

What are you trying to do?

File preserves it's name when mounted to a directory like this:

dag.Directory().WithFile("", file)

It doesn't always make sense to accept a file name when it's being created though.

For example, when building a binary, I may not care about the file name in 99%, so adding an optional name parameter just makes the API more complex with little benefit:

dag.Go().WithSource(dir).Build(GoWithSourceBuildOpts{Name: "mybinary"})

...

#
sagikazarmark
[dagger/dagger] Issue opened: #6475 ✨ Add WithFiles to Directory

What are you trying to do?

I find myself writing code like this lately:

dir := dag.Directory()

for _, file := range files {
    dir = dir.WithFile("", file)
}

I'd like to see a shorthand where I can easily add a list of files to a directory:

dir := dag.Directory().WithFiles(files...) // OR: WithFiles(files)

Why is this important to you?

When using CLI tools (or even in Dockerfiles) files and directories are usually described by a path without h...

#
sagikazarmark
[dagger/dagger] Issue opened: #6476 ✨ Add Merge function to Directory

What are you trying to do?

When working with build results and generating final build artifacts (packages), I often need to create directories and merge them.

In theory, this works using the following:

dag.Directory().WithDirectory("", dag.Directory())

I'm not entirely sure it always merges directories though.

In any case, a function called Merge would reflect better what it does (similarly to Diff)

Why is this important to you?

No response

How are ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6478 [WIP] Gracefully handle name conflict corner cases

Opening as draft to start discussion while I work on proposed solution in parallel, only change so far are some new tests that exemplify the problem (and commits from the dagger.json updates PR, which this will be based on).

Related to:

The need for better handling of corner case conflicts between a module's objects and dependencies/core has b...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6479 fix: avoid holding server map lock

We should avoid holding the server map lock for any long period of time. This should help to prevent a total deadlock in the case where a rogue server holds the lock for too long during shutdown (for currently unknown reasons).

By reducing our usage of the global map lock to only simple operations that won't block, we ensure that this rogue server doesn't block every operation on the controller.

However, because of this refactor, we need to introduce a more precise per-server-id lock. T...

vivid lintelBOT
ornate vigilBOT
ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
taylormonacelli
[dagger/dagger] Issue opened: #6483 🐞 running demo `dagger functions` returns error

What is the issue?

I'm Starting to learn about modules from here:
https://docs.dagger.io/zenith/user/quickstart/120928/setup

but ran into error:

Error: failed to get loaded module ID: input: resolve: host: directory: asModule: failed to call module "example" to get functions: call constructor: process "go build -o /runtime ." did not complete successfully: exit code: 1

when running dagger functions command.

I expected to see something like this:

✔...
#
helderco
[dagger/dagger] Pull request opened: #6484 Add field to populate a module's description

We had a Module.description field in the schema but no way of populating it. Rather than make the main object's description the module description, we can cleanly separate those two in the SDKs. The consumer can, for example, default to the main object's description if there's no module description.

Still needs tests, and only Python sets the field for now.

Having issues with elixir and php codegen.

\cc @grouville @jpadams

vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6486 How to scale Dagger in production?

Problem

We haven't conclusively answered the question: "what is the best way to scale Dagger in production?". This is in part because there is a wide variety of requirements and preferences, depending on the use case and constraints.

So far there are 2 competing approaches: horizontal scaling and vertical scaling. We have not conclusively decided which one is better, and why.

Solution

Using our collective experience running Dagger in production since the cloak release (0...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6489 ci: remove ready_for_review trigger for prs

These were added in https://github.com/dagger/dagger/commit/e4e5fa65ac5ebc0ed58b92f6cd9d29156a63e132.

opened, synchronize and reopened make sense to me - these are all cases when changes are made to the PR (and re-opened is quite rare, but given that main may have moved on since then, it can be useful to re-run).

However, ready_for_review often doesn't mean any actual changes to the content of the PR, and happens much more frequently, so it makes sense to skip this.

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6491 ci: add dry-run targets for SDK publishing

Follow-up to #6463 (and related to #6488)

This PR introduces DRY_RUN to all SDK publish mage jobs, allowing any of them to be run and tested locally.

  • Adds DRY_RUN to python, go, typescript and php
  • Renames HEX_DRY_RUN to DRY_RUN for elixir
  • Renames CARGO_PUBLISH_DRYRUN to DRY_RUN for rust
  • Renames MVN_DEPLOY_DRY_RUN to DRY_RUN for java

The idea is to prevent issues like #6464 from appearing only during the release process, and instead to show up as soon as possi...

vivid lintelBOT
vivid lintelBOT
#
gerhard
[dagger/dagger] Pull request opened: #6492 ci: Export GitHub Actions Workflows traces to Honeycomb.io

[!NOTE]
This is an experimental PR which may not end up in main. It was created from a branch so it has access to secrets. The new workflow is restricted to only run on the dagger/dagger repository.

The goal is for us to better understand where our workflows spend time before Dagger gets invoked.

Jobs may be queued for longer than normal, steps which run before or after Dagger may be problematic, etc.

Without this, it's very difficult to know where our workflows as a whole s...

vivid lintelBOT
#
kpenfound
[dagger/dagger] Issue opened: #6493 Networking errors after calling `Sync()` on Containers that have dependent Services

I have a pipeline with multiple Services and Containers being passed around. If a Container that has a dependent service is Sync()'ed, future calls to the Service will fail with something like:

lookup 5r4dg8qetp2iq.r4mh9kocl49fi.dagger.local on 10.87.0.1:53: no such host

psuedocode example:

foo := dag.Container().From("postgres").AsService()

bar := dag.Container().From("asdf").WithServiceBinding("db", foo).WithExec(["some", "commands])

_, err := bar.Sync()

baz := ...
#
sipsma
[dagger/dagger] Pull request opened: #6494 cache: make first synchronous import non-fatal and timed out.

Before this, if there was an error when synchronously importing layer cache at engine start, the engine failed to start entirely. Now, the engine will continue running with just the local cache and retry import periodically in the background.

Additionally, that initial synchronous import did not have a timeout previously. It now has a timeout of 1 minute. If that's hit, then the above logic kicks in to gracefully handle the error.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6496 hack: send SIGQUIT to testdev if we run too long

:scream: we're still seeing the weird deadlocks from the buildkit upgrade, even after pulling in https://github.com/dagger/dagger/pull/6479 (it looks like other non-blocked tests keep running now, but the blocked tests are still an issue).

This PR tries to dump a ton more information in these scenarios.

The weird sleep and "docker exec" to send SIGQUIT should be removed once the source of the deadlock is resolved, since it's very very hacky, but should work to at least grab some inf...

vivid lintelBOT
#
migmartri
[dagger/dagger] Pull request opened: #6497 docs: bump CLI version during installation

While reviewing the upcoming version documentation, I noticed that the CLI installed through other installation methods, except homebrew, was outdated.

This prevented me (I use Linux and the curl method) from progressing through the following sections in the quickstart, such as this one failing due to the missing functions command.

I bumped the docs to point to 0.9.7, which seems to be the latest release, also reference...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Issue opened: #6501 🐞 Logs are duplicated when a vertex is used multiple times

What is the issue?

This issue is only on main; the current release, v0.9.7, is not affected.

If you run a pipeline with a vertex that is used multiple times in the same session, the log output will be duplicated, possibly with each extra use (i.e. cache hit).

Dagger version

main

Steps to reproduce

Once #6456 is merged, run a function that calls another function twice; you should see duplicate logs in its codegen vertex.

Log output

main:

![image](https://gith...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6505 fix: dedupe uncached metadata across a session

Fixes #6501.

Thank goodness it's Friday, lol.

The root cause of #6501 is that every single ExecOp gets a unique digest. This means that the buildkit solver is not able to dedupe them, and instead, to ensure it only runs once, we rely on the edge merging behavior. This was kiiiinda hacky, and not super great - edge merging is supposed to be for different vertexes that happen to produce similar output (detected by their cache keys). We were using it for pretty much everything.

Since ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
skycaptain
[dagger/dagger] Issue opened: #6510 🐞 `with_user` breaks `with_exec`'s `stdin`

What is the issue?

with_exec's stdin parameter enables writing content to the command's standard input before it closes. However, if you switch to another user, it breaks without any notification, preventing the command from executing.

Dagger version

dagger v0.9.7 (registry.dagger.io/engine) darwin/arm64

Steps to reproduce

This works:

import anyio
import dagger
from dagger import dag


async def main():
    async with dagger.connection():
        out ...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6514 query builder (go): support selecting siblings

This is different from https://github.com/dagger/dagger/issues/3577, which is about figuring out a DX for doing this in the SDKs.

Python’s query builder is from a third-party library, and supports everything from GraphQL. The one in Go is a custom solution, built entirely for our chainable API. Because of this and the issue above, it doesn’t support sibling selection. You can only select one field from an object.

However, dropping down to the query builder lever can unblock certain use ...

#
helderco
[dagger/dagger] Issue opened: #6516 API: convention for short and long descriptions

Because of core chaining in dagger call --help:

The inconsistency in documenting the API is really noticeable. Since we want to list a short description next to commands and flags in the CLI, I suggest we apply a convention of writing the most concise short description in the first line, a blank line, and a more elaborate description.

Prior art

Many languages use this simple convention for docstrings (for example, [Python](https:/...

#
helderco
[dagger/dagger] Issue opened: #6517 CLI: skip adding `sync` as a command in `dagger call`

Background

With https://github.com/dagger/dagger/issues/6359, If the last argument in dagger call is a Container, Directory, or File, the sync field is selected automatically, while the returned ID is ignored (not printed to screen). You can, however, select sync explicitly, in which case the ID is printed.

Motivation

For consistency, we want to just do the forced evaluation, without printing the returned ID to the user because there’s no meaning in this context. Ho...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6520 support passing cache volumes as flag

Prior to this PR, attempting to run a module that takes a CacheVolume in its constructor (or likely also any function) would fail with this:

Error: unsupported object type "CacheVolume" for flag: mod-cache

The flag value is just the cache volume name. Can't think of an immediate reason for it to be any more complicated than that.

  • [ ] test
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6521 chore: create test client at lowest Parallel level

We should avoid calling Parallel in a sub-test after creating a client, since it can block for a completely indeterminate amount of time, and in that time, we can end up with a huge number of just unused clients.

Not only is this (really only a) little wasteful, it makes stack traces and dumps in the case of test timeout very hard to decipher. Are we waiting for a client because it got deadlocked? Or because it was open since the beginning of the test run.

We do this in two ways:

  • M...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jdhaines
[dagger/dagger] Pull request opened: #6523 docs(command order)

I think the command has to come after the -m flag. When it comes before dagger can't find the module... most other places (for instance in the bottom tip) the command is after the module flag.

This is a tiny update, but it cost me at least 10 minutes before I found another example with them flipped and then mine started working locally. Hope it saves someone else a few minutes!

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6524 chore: update buildkit to master@ef61ad133d7f

Diff:

Lots of interesting potential fixes for some of the recent buildkit instability from deadlocks:

There's a few scarier changes in there, but nothing I'm overly worried about.

cc @sipsma

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6525 chore(deps): bump follow-redirects from 1.15.3 to 1.15.5 in /docs

Bumps follow-redirects from 1.15.3 to 1.15.5.

Commits

b1677ce Release version 1.15.5 of the npm package.
d8914f7 Preserve fragment in responseUrl.
6585820 Release version 1.15.4 of the npm package.
7a6567e Disallow bracketed hostnames.
05629af Prefer native URL instead of deprecated url.parse.
1cba8e8 Prefer native URL instead of legacy url.resolve.
72bc2a4 Simplify _processResponse error handling.
3d42aec Add bracket tests.
bcbb096 Do...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6526 chore(deps): bump h2 from 0.3.22 to 0.3.24 in /sdk/rust

Bumps h2 from 0.3.22 to 0.3.24.

Release notes
Sourced from h2's releases.

v0.3.24
Fixed

Limit error resets for misbehaving connections.

v0.3.23
What's Changed

cherry-pick fix: streams awaiting capacity lockout in hyperium/h2#734

Changelog
Sourced from h2's changelog.

0.3.24 (January 17, 2024)

Limit error resets for misbehaving connections.

0.3.23 (January 10, 2024)

Backport fix from 0.4.1 for stream capacity assignment.

Commits

7243ab5 Prepa...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6527 chore(deps): bump com.jayway.jsonpath:json-path from 2.8.0 to 2.9.0 in /sdk/java

Bumps com.jayway.jsonpath:json-path from 2.8.0 to 2.9.0.

Release notes
Sourced from com.jayway.jsonpath:json-path's releases.

json-path-2.9.0
What's Changed

Fix for CVE-2023-51074.
update dependencies by @​SingingBush in json-path/JsonPath#965
JPMS: define Automatic-Module-Name as json.path by @​SingingBush in json-path/JsonPath#966
Bump json-smart version from 2.4.10 to 2.5.0 by @​shoothzj in json-path/JsonPath#945
Fixed rendering error on $..book[?(@...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6528 chore(deps): bump aiohttp from 3.9.1 to 3.9.2 in /sdk/python/requirements

Bumps aiohttp from 3.9.1 to 3.9.2.

Release notes
Sourced from aiohttp's releases.

3.9.2
Bug fixes

Fixed server-side websocket connection leak.
Related issues and pull requests on GitHub:
#7978.

Fixed web.FileResponse doing blocking I/O in the event loop.
Related issues and pull requests on GitHub:
#8012.

Fixed double compress when compression enabled and compressed file exists in server file responses.
Related issues and pull requests on GitHub:
...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
nipuna-perera
[dagger/dagger] Issue opened: #6532 🐞 Confusing error message when `Context` is not the first parameter of a function

What is the issue?

Mostly what the title says. The error message I get from putting the Context in the wrong place is cryptic.

Dagger version

dagger v0.9.7

Steps to reproduce

Create any Zenith module function and put the context.Context in any location but the first parameter.

Log output

Error: failed to automate vcs: failed to get vcs ignored paths: input: resolve: module: withSource: failed to get modified source directory for go module sdk codeg...
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6533 module CLI commands polish

This implements a bare minimum subset of the polish described here and discussed separately: https://github.com/dagger/dagger/issues/6291#issuecomment-1911592394

Specifically, it's mainly just the stuff that's backwards incompatible, with the addition of dagger update since it was trivial to support and needed as a follow-up to https://github.com/dagger/dagger/pull/6386

  • sync is renamed to develop
  • dagger update can be used to set name+sdk of module
    • currently only allow...
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6534 Update zenith dev guides with incoming breaking changes.

The most important updates here are in anticipation of breaking changes in the next release. There are also some more opinionated cleanups, but for now just leaving the overall structure the same.

The big breaking changes to adapt for are:

  1. dagger mod init/install/etc. moving to be right under dagger
  2. dagger up replaced by chainable dagger call ... up
  3. dag.Host() being replaced with dag.CurrentModule().Source()

The more opinionated tweaks are:

  1. Don't use raw grap...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
cubeee
[dagger/dagger] Issue opened: #6540 🐞 sdk(elixir): wrong os type and architecture detection on Windows for cli downloader

What is the issue?

Elixir SDK CLI downloader's :os.type() pattern matching expects {:windows, :nt} but is actually {:win32, :nt} as documented in Erlang.

https://github.com/dagger/dagger/blob/7350f0b8dd5ff74b7ae0ddfd05e35c37bf16712b/sdk/elixir/lib/dagger/internal/engine/downloader.ex#L120-L126

Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:1] [jit:ns]

Interactive Elixir (1.16.0) - pres...
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6541 Revert "rip out progrock socket plumbing"

Turns out this is still needed by legacy Dagger (dagger run, go run + auto-started dagger session, etc).

It's unfortunate to bring all this plumbing back but we obviously can't break that yet.

Kept everything as-is, including propagating parents, because #6505 prevents the cache-busting issue that we originally ripped all this out to avoid.

This reverts commit 3309a8e799e1f4d66506bdc5e540613ae4bab90a.

#
kpenfound
[dagger/dagger] Pull request opened: #6542 refs v2 lite

Refs V2 lite

Allows releasing modules individually in a monorepo with subpath tags, e.g. "github.com/foo/daggerverse/proxy" can be tagged with proxy/v1.0.0 and installed with dagger mod install github.com/foo/daggerverse/proxy@v1.0.0.

This does not affect non-monorepo module tags or repo-wide tags in monorepos.

This does not add any advanced version matching logic, but takes a step towards enabling that.

https://linear.app/dagger/issue/DEV-3338/simplest-possible-refs-v2-impl...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6545 core: fix support for custom SDK modules

The recent updates accidentally left in dag.Select calls that referred to old APIs in the codepath for custom (i.e. 3rd party) SDKs. We didn't catch this because we didn't have any integ tests for this feature.

This fixes the bug and adds an integ test. It also required exposing Module.Runtime as a graphql field in order to write the test case, but that seems like it will only be beneficial in general to any other SDK authors trying to do something similar to this.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6549 CLI: Improve display of core API in `--help`

Fixes #6358

Not all of these are focused on the core API, most impactful is the wrapping on flags and commands. I’m submitting what I have now to be cut in next release but will continue in a separate PR for general --help improvements.

Putting individual improvements in separate commits.

Todo

  • [x] Group commands in dagger root (modules, execution, cloud, additional)
  • [x] Group function commands in dagger call
  • [x] Wrapping on flag usages
  • [x] Wrapping on available...
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6552 Secret censoring should include common transformations

When we had an issue with https://github.com/dagger/dagger/pull/6488, the only thing saving us was github censoring it's own token in the logs.

We didn't pick up on this, because the printed token was base64 encoded.

Our new secret censoring mechanism https://github.com/dagger/dagger/pull/6034 should ideally handle this - we should try and filter our base64/hex/etc encoded versions of secrets (though for base64 this may be tricky(er), since it's not perfectly byte-aligned).

Also *sor...

#
jedevc
[dagger/dagger] Issue opened: #6553 ✨ Access Service stdout/stderr and exit code

What are you trying to do?

Currently, once a service starts, there's not a ton of interaction points for it. We can stop it... and that's about it.

One key thing I'd like is access to the stdout/stderr of a service. At the moment, these are exclusively streamed to the progrock output, but it would be nice to be able to output them somewhere else.

[!NOTE]
In addition to this, I'd really like to have a way to suppress the default progrock service output. These in combinati...

vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6554 ✨ Raise a warning if /var/lib/dagger is an overlayfs mount inside the engine container

What are you trying to do?

Whenever the engine is provisioned within a container - default strategy by the SDK's - as highlighted in the operator's manual here (https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-operator_manual.md#execution-requirements), it's imperative to set /var/lib/dagger as a volume so it doesn't form part of the overlayfs of the container. Otherwise, this will have a severe performance penalty when running pipelines.

We should raise a warning if we d...

vivid lintelBOT
vivid lintelBOT
#
tomasmota
[dagger/dagger] Pull request opened: #6556 feat: add WithFiles to Directory and Container

Fixes: #6475

The issue mentioned only Directory, but I also added it to Container, not sure that was a good idea. Also, this is my first contribution so I might be missing a few things.

With regards to the sdk changelog, it seems like that is something I should do after creating the PR? go through every sdk and run it?

Also a question: in Directory WithFiles, I wasn't sure whether or not I should have re-used WithFile, some feedback on that would be great :)

vivid lintelBOT
#
wingyplus
[dagger/dagger] Pull request opened: #6559 feat(sdk/elixir): rework on Elixir codegen

This is a big change but let me explain to catch up easier.

Codegen move out from the SDK

Previously, the dagger codegen was embedded in the SDK, this caused a problem in that it could not be run when the codegen generated a bad code. In this change, introduces a new Elixir module called dagger_codegen which's responsible for generating code and uses the escript to build a binary instead of the mix task.

The way to fetch introspection is now split out of the codegen. This way...

#
jedevc
[dagger/dagger] Pull request opened: #6560 chore: refactor customer dns sources to avoid hacks

This removes our smuggling hack where we smuggle properties through existing ones - we no longer need to do this, since we can create a custom identifier that wraps the upstream ones. (Hope it's ok I took your TODOs @vito :laughing:)

This is hopefully part 1 of several to try and clean up our custom sources - I'd love to be able to remove most of the copy-pasta code from buildkit, to avoid future drift (as I noticed in https://github.com/dagger/dagger/pull/6376#issuecomment-1919427939).

#
jedevc
[dagger/dagger] Issue opened: #6561 Strange error when querying core type without sub-selection

Given a very simple go module:

package main

type Playground struct{}

func (m *Playground) Alpine() *Container {
	return dag.Container().From("alpine:latest")
}

Using dagger call without a sub-selection works as expected:

❯ dagger call alpine
✔ dagger call alpine [2.19s]
┃ Container evaluated. Use "dagger call alpine --help" to see available sub-commands.                                                                   
• Engine: e254c39a3421 (version de...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6564 chore(deps): bump pytest-httpx from 0.28.0 to 0.29.0 in /sdk/python

Bumps pytest-httpx from 0.28.0 to 0.29.0.

Release notes
Sourced from pytest-httpx's releases.

0.29.0 (2024-01-29)
Added

Add support for pytest==8.* (pytest==7.* is still supported for now). (many thanks to Yossi Rozantsev)

Changelog
Sourced from pytest-httpx's changelog.

[0.29.0] - 2024-01-29
Added

Add support for pytest==8.* (pytest==7.* is still supported for now). (many thanks to Yossi Rozantsev)

Commits

fcb442a Merge pull request ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6565 chore(deps): bump urllib3 from 2.1.0 to 2.2.0 in /sdk/python

Bumps urllib3 from 2.1.0 to 2.2.0.

Release notes
Sourced from urllib3's releases.

2.2.0
🖥️ urllib3 now works in the browser
:tada: This release adds experimental support for using urllib3 in the browser with Pyodide! :tada:
Thanks to Joe Marshall (@​joemarshall) for contributing this feature. This change was possible thanks to work done in urllib3 v2.0 to detach our API from http.client. Please report all bugs to the urllib3 issue tracker.
🚀 urllib3 is ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6566 chore(deps): bump com.spotify.fmt:fmt-maven-plugin from 2.21.1 to 2.22.1 in /sdk/java

Bumps com.spotify.fmt:fmt-maven-plugin from 2.21.1 to 2.22.1.

Release notes
Sourced from com.spotify.fmt:fmt-maven-plugin's releases.

2.22.1
There are no changes in this release and it was done purely for verifying problems with our release process.
Full Changelog: https://github.com/spotify/fmt-maven-plugin/compare/2.22.0...2.22.1
2.22.0
What's Changed

This upgrades google-java-format to 1.19.2 which adds support for formatting Java 21
Adds t...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6567 chore(deps-dev): bump org.mockito:mockito-core from 5.8.0 to 5.10.0 in /sdk/java

Bumps org.mockito:mockito-core from 5.8.0 to 5.10.0.

Release notes
Sourced from org.mockito:mockito-core's releases.

v5.10.0
Changelog generated by Shipkit Changelog Gradle Plugin
5.10.0

2024-01-24 - 8 commit(s) by Andre Brait, dependabot[bot]
Bump org.shipkit:shipkit-auto-version from 1.2.2 to 2.0.2 (#3248)
Bump org.assertj:assertj-core from 3.25.1 to 3.25.2 (#3247)
Bump org.shipkit:shipkit-changelog fro...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6568 chore(deps): bump github.com/google/uuid from 1.5.0 to 1.6.0

Bumps github.com/google/uuid from 1.5.0 to 1.6.0.

Release notes
Sourced from github.com/google/uuid's releases.

v1.6.0
1.6.0 (2024-01-16)
Features

add Max UUID constant (#149) (c58770e)

Bug Fixes

fix typo in version 7 uuid documentation (#153) (016b199)
Monotonicity in UUIDv7 (#150) (a2b2b32)

Changelog
Sourced from github.com/google/uuid's changelog.

1.6.0 (2024-01-16)
Features

add Max UUID constant (#149) (c58770e)

Bug Fixes

fix typo in version ...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6569 chore(deps): bump github.com/google/go-containerregistry from 0.15.2 to 0.19.0

Bumps github.com/google/go-containerregistry from 0.15.2 to 0.19.0.

Release notes
Sourced from github.com/google/go-containerregistry's releases.

v0.19.0
What's Changed

Work around docker v25 tarballs by @​jonjohnsonjr in google/go-containerregistry#1872

Full Changelog: https://github.com/google/go-containerregistry/compare/v0.18.0...v0.19.0
v0.18.0
What's Changed

fix: goreleaser config by @​caarlos0 in google/go-containerregistry#1764
Al...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6571 chore(deps): bump google.golang.org/grpc from 1.60.1 to 1.61.0

Bumps google.golang.org/grpc from 1.60.1 to 1.61.0.

Release notes
Sourced from google.golang.org/grpc's releases.

Release 1.61.0
New Features

resolver: provide method, AuthorityOverrider, to allow resolver.Builders to override the default authority for a ClientConn. (EXPERIMENTAL) (#6752)

Special Thanks: @​Aditya-Sood

xds: add support for mTLS Credentials in xDS bootstrap (gRFC A65) (#6757)

Special Thanks: @​atollena

server: add grpc.WaitForHandlers...

#
dependabot[bot]
[dagger/dagger] Pull request opened: #6572 chore(deps): bump github.com/charmbracelet/bubbles from 0.17.1 to 0.18.0

Bumps github.com/charmbracelet/bubbles from 0.17.1 to 0.18.0.

Release notes
Sourced from github.com/charmbracelet/bubbles's releases.

v0.18.0
Textarea, but faster
This release features several fixes and big performance improvements for the textarea bubble.
What's Changed
New

Optional File Permissions and File Size by @​maaslalani in charmbracelet/bubbles#471
Add Paginator OnFirstPage method by @​maaslalani in charmbracelet/bubbles#463

Improved

...

vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6574 `Container.From` should ignore image entrypoint by default

Problem

As discussed on Discord:

I find myself wishing that we did not honor OCI image entrypoints by default...
I would much rather know that, by default, From("foo").WithExec("bar") will always actually execute the binary "bar" in the image "foo", unless I specifically tell Dagger that I care what the image author put in the entrypoint (most of the time I don't).

Kind of like the way Kubernetes manifest ignores a bunch of fields in the docker image, and just makes you re-...

vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6575 fix module include/exclude applying to essential files

This started as a fix to a problem where syncing a module using
"include" in dagger.json was resulting in the dagger.json being reset to
an empty module. The root cause was that the include did not have
dagger.json itself in it, so when the module was loaded into the engine
it looked like an uninitialized module that needed to be initialized.

That bug in particular was a regression caused by the recent changes
around module loading and configuration.

This change fixes that bug, but...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6576 Clean up started services on StartBindings failure

While investigating something (hopefully) unrelated to this, I found that StartBindings wasn't handling error cases well. There is an attempt to try and call detach on failure, but it wasn't being used.

Because we use a channel to pull the binding results, if one of the services failed to start for some reason, we would exit early, and none of the started services would be in running, so when we called detach, none of the services would be stopped. This could easily result in dangli...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6577 fix: avoid client panic on ctx cancel

A super weird find (I think this was introduced recently?)

https://github.com/dagger/dagger.io/actions/runs/7756389308/job/21153589598?pr=3359#step:4:2365

24: exec go test ./cmd/ci/ -v -timeout=30m
24: [244.0s] 2024/02/02 13:12:31 http: panic serving 127.0.0.1:42432: context canceled
24: [244.0s] goroutine 863 [running]:
24: [244.0s] net/http.(*conn).serve.func1()
24: [244.0s] 	/usr/local/go/src/net/http/server.go:1868 +0xb9
24: [244.0s] panic({0xfcadc0?, 0x1b79170?})
24: [24...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
bdols
[dagger/dagger] Issue opened: #6589 WithoutDirectory absolute paths not handled

What is the issue?

I have found that the following does not do like I intended and it's not documented, as it is expected to be a relative path:
containerImage.Directory("/foo").WithoutDirectory("/foo/bar")
and instead this needs to be:
containerImage.Directory("/foo").WithoutDirectory("bar")

 func (*dagger.Directory).WithoutDirectory(path string) *dagger.Directory                                                                                           
                     ...
vivid lintelBOT
#
benjaminstrasser
[dagger/dagger] Pull request opened: #6590 WIP: ci: added ci test for bunjs

Purpose

This PR introduces tests for bun to ensure its compatibility with TypeScript/Node SDK and to prevent future breaking changes.

Key Changes

  • Added a test suite for bun using TypeScript/Node SDK.
  • Updated CI pipeline to include new tests

Goals

  • Verify bun works as expected with TypeScript/Node SDK.
  • Automate testing to catch breaking changes before they're merged and ensure bun works in the furure.

TODO

  • Discuss what tests should be added
  • add ...
vivid lintelBOT
#
benjaminstrasser
[dagger/dagger] Issue opened: #6591 ✨ Bun

What are you trying to do?

Add official support for bun in dagger.

Below I propose a initial list of potential tasks I fell would help users to use bun with dagger

  • [ ] Add tests for bun #6590
  • [ ] Update the documentation to show how bun is working with dagger
  • [ ] Add at least one example showcasing bun compatibility with dagger

Why is this important to you?

Bun is a new JavaScript runtime promising to solve many of the issues found in the current Node ecosystem that ...

vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6592 docs: generate cli reference

This adds CLI reference generation for the main docs. Previously we only had it for zenith docs.

Since there’s now two different front matters, I thought it’s better to decouple from being hardcoded anyway (--frontmatter), which also reduced the complexity over parsing the file name to get the right slug for it. This also allowed printing the result to stdout because the file name no longer matters (--output flag).

Fixed an issue where dagger completion was being excluded from usa...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6596 Closing a service should implicitly close all dependent tunnels *first*

This is a fun one. Let's suppose I'm running a dagger engine as engine:

// This works fine :tada:
tunnel.Stop(ctx)
engine.Stop(ctx)

If we swap the order, we block forever, yay!

// This blocks forever!
engine.Stop(ctx)  // we never return
tunnel.Stop(ctx)

This is kinda tricky, but essentially, there is still an open connection for the tunnel open to the server, so the server's Stop/GracefulStop can get stuck. We get stuck trying to do a handshake...

#
jedevc
[dagger/dagger] Pull request opened: #6597 chore: cleanup grpchijacked connections

There is a comment suggesting this? I'm not quite sure why it's not turned on, it definitely feels important.

This also clogs up every stack dump, with literally thousands of copies, since it never closes.

@sipsma is there a good reason to not do this? The stream connection can never outlive this Session call, so we should just be able to call this.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
Vermillionaire
[dagger/dagger] Issue opened: #6599 🐞 Dagger modules break in corporate environments

What is the issue?

For security reasons many companies man-in-the-middle all SSL traffic to the internet with their own certificate. This causes a x509: certificate signed by unknown authority issue in two spots for dagger:

  1. Dagger engine. This is easy to solve by mounting the certificates in the container.
  2. Dagger modules. The modules appear to be loading a tar container /usr/local/share/dagger/go-module-sdk-image.tar and building a go program when running any module command...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6605 `dagger install` polish

Follow ups from https://github.com/dagger/dagger/pull/6533 around install (and managing dependencies generally):

  • [ ] add support for dagger update to update existing deps
    • Currently this implicitly happens if you dagger install an existing dep at a different version, but it would be better to have an explicit command for it.
    • We could then consider whether we want dagger install to fail if dep at different version already exists or to continue supporting the implicit upgr...
#
sipsma
[dagger/dagger] Issue opened: #6606 `dagger config` polish

(Follow ups from https://github.com/dagger/dagger/pull/6533 related to dagger config)

dagger config currently only prints the configuration of a module, needs to be expanded with subcommands for actually setting the values not settable elsewhere. Currently those values are just:

  • include
  • exclude

But we should keep in mind that list may grow over time.

Quoting @shykes from the previous PR around some possibilities here:

  • git config style. I don't remember what it is...
#
sipsma
[dagger/dagger] Issue opened: #6607 `dagger develop` support for re-naming/re-sdk'ing existing module

(Follow-up to https://github.com/dagger/dagger/pull/6533)

dagger develop currently only allows you to set the name/SDK of a module that does not already have one set.

It's possible to support changing the name/SDK too, but there's a few things to consider here:

  • If you rename your module, you'd need to also update your code to use that name in the structs, but obviously much better would be if we did that automatically somehow
vivid lintelBOT
vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6608 ✨ CLI: Don't show functions that can't be callable

What are you trying to do?

I'm writing a module that exposes a function with complex arguments which are not currently supported via the dagger call command. I would really like that function not to be callable or displayed in the dagger functions given that it's only intended to use via an SDK and not the CLI.

Why is this important to you?

To provide a clearer API to my module users.

How are you currently working around this?

No response

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6611 fix: new engine drivers should allow multiple connection attempts

See https://github.com/dagger/dagger/pull/6288#issuecomment-1931971234 - this fixes the borked provision job: https://github.com/dagger/dagger/actions/runs/7814283652/job/21315752098.

Previously, we would only connect once. However, when calling buildkit's client.Wait we might attempt multiple connections over and over. If we only have one connection, this isn't possible. Ideally, we should have some better error messages in buildkit, since this was annoying to debug.

The right call her...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6621 dagger functions/call/etc should fail if none present

@kpenfound noticed this fun one while migrating to new the source fields:

❯ dagger init --sdk=go --name=playground --source .
Initialized module playground in .
❯ ls
dagger.gen.go  dagger.json  go.mod  go.sum  main.go  querybuilder
❯ rm main.go
❯ dagger functions
Name             Description
container-echo   example usage: "dagger call container-echo --string-arg yo stdout"
grep-dir         example usage: "dagger call grep-dir --directory-arg . --pattern GrepDir"

Eve...

vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6623 Need integ tests for git modules

Right now all our integ tests only use local refs because git modules are locked into github repos and we've been trying to avoid tests depending on modules in an external git repo. However, I think we should just bite the bullet on that at this point since missing that test coverage is too big a gap.

#
kpenfound
[dagger/dagger] Issue opened: #6624 🐞 Typescript module dependency not included in dagger.gen.go

This is happening on main, but not on version 0.9.8

Repro:

git clone -b daggerize https://github.com/kpenfound/example-voting-app
cd example-voting-app
dagger functions

The module has 4 local module dependencies and 1 remote. Of the local dependencies, there are 2 Go modules (Seed, Worker), 1 Python module (Vote), and 1 Typescript module (Result).

The codegen is not including Result for some reason, and any call results in:
`dag.Result undefined (type *Client has n...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6626 core: fix native always true for up even when ports are set

Think this happened when switch up to default to native.

The logic was accidentally setting native to true even when ports were manually specified which resulted in each port being applied twice. In the case where the same values of frontend and backend were specified (i.e. 5000:5000) this resulted in an error since you'd try to listen on the same port twice.

Should (re-) fix https://github.com/dagger/dagger/issues/6303

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6630 `slice bounds out of range [1:0]` panic in shim secret scrubbing

Hit this while testing publish things in my fork: https://github.com/sipsma/dagger/actions/runs/7837271347/job/21386972933#step:4:2866

Stack:

62: [0.09s] panic: runtime error: slice bounds out of range [1:0] goroutine 1 [running, locked to thread]:
62: [0.09s] runtime/debug.Stack()
62: [0.09s] 	/usr/local/go/src/runtime/debug/stack.go:24 +0x5e
62: [0.09s] main.main.func1()
62: [0.09s] 	/app/cmd/shim/main.go:63 +0x38
62: [0.09s] panic({0x1105620?, 0xc000044c78?})
62: [0.09s] 	/u...
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6631 fix runaway memory usage via wonky simplification

An unfortunate series of events can cause the TUI to rapidly consume all available memory. With whitespace. Oopsie.

  1. When you call ModuleSource.resolveDependency(gitModSource), it returns gitModSource, since there's no need to make a Git module source relative to anything else.
  2. This meant the resolveDependency call recorded gitModSource as an Output: https://github.com/dagger/dagger/blob/5ee49b9e9a955432d688157893b299f8a6180b1d/tracing/graphql.go#L119
  3. The TUI uses this ...
vivid lintelBOT
vivid lintelBOT
#
d3rp3tt3
[dagger/dagger] Pull request opened: #6633 Update Cloud docs for local runs

This PR includes updates to the Cloud docs

  • [ ] Clarify config and UX for local and CI
  • [ ] Update cache instructions to include config for pushing the volumes to distributed caching
  • [ ] Update reference and getting started for recent UI improvements
  • [ ] Update all Cloud docs for new plans

Status

  • Getting started guide: Ready for review
  • Reference: In progress

We will also need to copy these docs into the Zenith/latest later. That will be in a separate PR.

vivid lintelBOT
vivid lintelBOT
#
gianpaolof
[dagger/dagger] Issue opened: #6635 🖐️ nullpointer exception dagger.spi.shaded.androidx.room.compiler.processing.XTypeElement.getAnnotation

What happened? What did you expect to happen?

I've been trying to add Hilt in my app but I run into this issue when building:

Execution failed for task ':app:hiltJavaCompilePreprod'.

java.lang.NullPointerException: Cannot invoke "dagger.spi.shaded.androidx.room.compiler.processing.XTypeElement.getAnnotation(com.squareup.javapoet.ClassName)" because the return value of "dagger.spi.shaded.androidx.room.compiler.processing.XProcessingEnv.findTypeElement(com.squareup.javapoet.TypeNam...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
matipan
[dagger/dagger] Issue opened: #6640 🐞 CLI: dagger listen --disable-host-read-write fails in v0.9.9

What is the issue?

There is a regression on dagger v0.9.9 with the dagger listen command when the flag --disable-host-read-write is specified. Prior to v0.9.8 this was working correctly. Since v0.9.9 it fails with:

Error: failed to get configured module: failed to get local root path: input: resolve: moduleSource: resolveContextPathFromCaller: failed to stat source root: failed to receive file bytes message: rpc error: code = Unimplemented desc = unknown service moby.filesync.v1....
vivid lintelBOT
vivid lintelBOT
#
d3rp3tt3
[dagger/dagger] Pull request opened: #6642 Update Cloud docs for local runs and plan updates

Note

This is a new PR to replace 6633. Something got off in the commit history for that PR after a rebase. It's easier to start afresh in a new PR. Original PR: https://github.com/dagger/dagger/pull/6633

Overview

This PR includes updates to the Cloud docs

  • [x] Clarify config and UX for local and CI
  • [x] Update cache instructions to include config for pushing the volumes to distributed caching
  • [ ] Update reference and getting started for recent UI improvements
  • [ ] Update a...
vivid lintelBOT
vivid lintelBOT
#
jpadams
[dagger/dagger] Issue opened: #6644 CLI: `dagger functions` should work after `init` and `install` of one or more modules.

I'm surprised I can't do this flow for installing a bunch of modules and then seeing what functions I could run from CLI with dagger functions:

  1. dagger init
  2. dagger install
    4 dagger functions # to see what I've got to play with...

throws error

Error: input: resolve: moduleSource: resolveFromCaller: asModule: initialize: module name and SDK must be set

Figure we need to make that flow work or provide another way to get list of functions available from installed modules since...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
dependabot[bot]
[dagger/dagger] Pull request opened: #6647 chore(deps-dev): bump prettier from 2.8.7 to 3.2.5 in /sdk/typescript

Bumps prettier from 2.8.7 to 3.2.5.

Release notes
Sourced from prettier's releases.

3.2.5
🔗 Changelog
3.2.4

Fix .eslintrc.json format #15947

🔗 Changelog
3.2.3

Format tsconfig.json file with jsonc parser #15927

🔗 Changelog
3.2.2
🔗 Changelog
3.2.1
🔗 Changelog
3.2.0
diff
🔗 Release note
3.1.1
🔗 Changelog
3.1.0
diff
🔗 Release note
3.0.3
🔗 Changelog
3.0.2
🔗 Changelog
3.0.1
🔗 Changelog
3.0.0
diff
🔗 Release note
3.0.0-alpha.6
What's Changed

Update .d.ts ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6650 bump midterm for better TUI support

This bumps midterm to fix a ton of rendering issues with TUI apps, primarily with how scrolling regions are handled.

Vim now mostly works, but only with TERM=tmux-256color. With TERM=xterm-256color for whatever reason it prints \r\n when it should be printing \n which totally butchers the UI. Easy to trigger with just deleting a line (dd). Something similar happens when selecting lines upward (e.g. GV{{{). Currently compiling neovim to fig...

#
shykes
[dagger/dagger] Issue opened: #6651 Go SDK: Discontinue +default pragma

I want to flag something in the new Go DX for optional arguments. I don't like encoding my default values in comments with +default, because they are no longer in my code. I can't use them if I need any value more trivial than a string. I can't use them in any way when calling functions intra-module. At this point I think we should remove +default and keep only +optional. It's not a one-way door: we can add it later. But if we keep it now, we will fragment how devs manage their default va...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6652 feat: set GOTOOLCHAIN to auto for go module sdk

Without this, the Go module SDK would only ever be able to build modules with the single go version that our CI had permitted - this made for a really confusing and frustrating experience when running a newer go version locally (e.g. being on a bleeding edge distro :smile).

I noticed this when using a go.work in my own daggerverse repo:

$ dagger call get-latest-release --repo="dagger/dagger" body
✘ ModuleSource.asModule: Module! 1.8s
  ✘ Module.withSource(
      source: ✔ Modu...
#
slumbering
[dagger/dagger] Issue opened: #6653 🐞 Goland modules error while executing ./dagger version

What is the issue?

If I follow the section export-the-build-directory from the docs: https://docs.dagger.io/zenith/quickstart/853930/directories/#export-the-build-directory
I'm getting an error when trying to check the Dagger version:

$ ./dagger version
zsh: exec format error: ./dagger

cc @jedevc

Dagger version

dagger v0.9.9 (registry.dagger.io/engine) darwin/arm64

Steps to reproduce

To reproduce the issue, execute the following commands on an Apple Silic...

vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6654 Generated gitattributes should be generated in module source, not module root

If I have a module with a source key set to a ./ci, when I run dagger develop, I expect that the generated .gitattributes file should be generated into ./ci, and not at the top-level where my dagger.json is.

Alternatively, if we do decide that the top-level is the right place to put it, then the file should contain the right paths. Today, it looks something like:

/dagger.gen.go linguist-generated
/querybuilder/** linguist-generated

Even though dagger.gen.go...

#
slumbering
[dagger/dagger] Issue opened: #6655 Dagger module error message with mistyped --sdk flag

When I initialize a new module with dagger init and I mistyped the --sdk, I'm getting the following error.

$ dagger init --name=my-module --sdk=typescrip
Error: failed to generate code: input: resolve: moduleSource: withName: withSDK: withSourceSubpath: resolveFromCaller: failed to collect local module source deps: failed to collect local sdk: missing config file /Users/slumbering/workdir/dagger-zenith/trivy/typescrip/dagger.json

I'm wondering if we could improve this mes...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6659 Error running `dagger init`: `package name is empty`

To reproduce this, try initializing a dagger module in a directory that already contains a go.mod. For example:

$ git clone https://github.com/dagger/dagger.git
$ dagger init --sdk=go
✘ ModuleSource.asModule: Module! 2.0s
  ✘ Module.withSource(
      source: ✔ ModuleSource.resolveFromCaller: ModuleSource! 0.0s
    ): Module! 2.0s
    ✘ Container.directory(path: "/src"): Directory! 1.4s
      ✔ rm /dagger/dagger.gen.go 0.4s
      ✘ exec /usr/local/bin/codegen --module-context...
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6661 Cannot have utils sub-package with Go Modules SDK

I want to have a module with a directory structure like this:

dagger
├── dagger.gen.go
├── go.mod
├── go.sum
├── main.go
├── querybuilder
│   ├── marshal.go
│   └── querybuilder.go
└── util
    ├── engine.go
    ├── generated.go
    └── util.go

3 directories, 9 files

Essentially, I want a custom util directory with some helpers. However, if these utils require accessing dagger types like Client or Container, then there's no way of expressing this.

The ty...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6662 Using embedded fields with go modules fails with unhelpful error

Take the following code:

type Playground struct {
	// This breaks
	*Directory

	// This doesn't
	// Directory *Directory
}

func New() Playground {
	return Playground{Directory: dag.Directory()}
}

When attempting to evaluate anything on the Playground object, we get the following error:

Error: response from query: input: resolve: playground: failed to convert return value: unexpected result value type string for object "Playground"
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6663 `test:race` job should enable race checker on the engine as well

At the moment, we're only checking for race conditions in the client - but ignoring race conditions in the engine.

https://github.com/dagger/dagger/blob/bf262aca916e91a69b2191582c306c1e3ec7b633/internal/mage/engine.go#L332-L375

This is useful, and it has caught a number of issues in the client, but there are also potentially race conditions in the engine! e.g. see https://github.com/vito/progrock/issues/12 (cc @vito)

Note: running with the race checker is likely going to have a *...

#
kpenfound
[dagger/dagger] Issue opened: #6664 Unable to reference remote modules by short name

What I expect to work:

dagger init
dagger install github.com/shykes/daggerverse/hello
dagger -m hello functions

What I get:
Error: failed to get configured module: failed to get local root path: input: resolve: moduleSource: resolveContextPathFromCaller: cannot resolve non-local module source from caller

Additionally, when I run dagger -m hello functions, I get local files created mimicking the module's remote ref:

$ tree github.com
github.com
└── shykes
...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
samalba
[dagger/dagger] Issue opened: #6669 🐞 installed module is unusable (codegen'd with wrong capitalization)

What is the issue?

Created module for running remote commands via SSH https://daggerverse.dev/mod/github.com/samalba/dagger-modules/ssh@35ed3e343d7e6faa3eab44570ee7531914dd4e65

I initialized the module with:
dagger init --name ssh --sdk go --source .

The module code uses a struct named Ssh. It works fine as standalone. However when you install the module from another module, it's available via dag.SSH() (different capitalization), which then fails to compile, because it cann...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6672 🐞 Exec shim doesn't allow background processes to be terminated successfully

What is the issue?

Trying to execute the following pipeline:

func main() {
    defer dag.Close()
    dag.Container().From("alpine").
        WithExec([]string{"apk", "add", "bash"}).
        WithNewFile("/entrypoint.sh", dagger.ContainerWithNewFileOpts{
            Contents: `#!/bin/bash
sleep 10 &
echo "foo"`,
            Permissions: 0755,
        }).
        WithEnvVariable("BUST", time.Now().String()).
        WithEntrypoint([]string{"/entrypoint.sh"}).
       ...
vivid lintelBOT
#
grouville
[dagger/dagger] Issue opened: #6673 🐞 `dagger up --ports` doesn't bind host address when multiple `WithExposedPort` are present in a module

What is the issue?

It seems that host ports are not binded properly to the host when multiple WithExposedPort are present (and at least one port is binded to the host with --port)

Dagger version

dagger v0.9.10

Steps to reproduce

Context

Given this example module:

func (m *MyMod) Package(
) *Service {
	return dag.Container().
		From("localstack/localstack:latest").
		WithExposedPort(4566).
		AsService()
}

I can run this command: `dagger cal...

vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6675 `platformVariants` is unwieldy in modules

Using platformVariants is extremely unwieldy to use: https://github.com/dagger/dagger/blob/fca9e788f6d605ef9d256beace4656c11d7f6fe5/docs/docs-graphql/schema.graphqls#L196-L201

The idea is that you can construct multiple Container objects, and then compose them to together on Publish/Export to push a multi-platform image.

However, this feels like a weird abstraction - quite often, in our own CI in this repo, we end up calling a single function multiple times with different arch...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6680 feat: split out go module defs into package

Finally! This patch splits up the codegen for go modules to produce the main external struct defs into a separate sub-package. There are a couple of reasons for doing this:

  • Tighter control over exposed structs and their internal private methods
  • Ability to import the generated structs from a sub-package

Initially this might sound simple - and prior to the addition of interfaces, it was, however, these add a lot of confusion. Specifically, interface codegen is added alongside the main ...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
scriptnull
[dagger/dagger] Issue opened: #6682 Access files in parent and sibling directories

What happened? What did you expect to happen?

Hi, I am just getting started with dagger. My project structure is like this

.
├── ci
│   ├── go.mod
│   ├── go.sum
│   └── main.go
├── service1
│   ├── Dockerfile
│   ├── go.mod
│   ├── go.sum
│   ├── internal
│   └── main.go
├── service2
│   ├── Dockerfile
│   ├── go.mod
│   ├── go.sum
│   ├── internal
│   └── main.go

The dagger program is written inside the ci folder. What I would like to do is "access the...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
benjaminstrasser
[dagger/dagger] Pull request opened: #6685 ci: Update Node.js version for the typescript SDK and test LTS and Maintenance versions

As discussed with @TomChv in #6591.

Changes:

Our CI would test with

  • node:18-alpine
  • node:20-alpine
    and not with node:16-alpine, which is discontinued and reached EOL.

All other tasks, such as lint, bump, publish, build, utilize the LTS version of node, which is currently 20.

Updated the package.json to reflect we only test for node>=18.

Node 18 is maintained until 30 Apr 2025.
Thats when we should potentially look into upgrading again.

vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Issue opened: #6687 🐞 Directory.glob with recursive patterns (`**/*.go`) returns duplicates and is very slow

What is the issue?

It looks like Buildkit returns weird results when you ReadDir with a glob like **/*.go. It actually seems to find everything recursively, but then trim each result to only have the first path segment. So for a/1.go a/2.go b/3.go it'll return a a b. We then recurse into a twice, which will also have this issue, so it quickly explodes into a huge number of results.

We could work around this issue by deduping, but it would probably be better to fix this upstr...

ornate vigilBOT
ornate vigilBOT
#
marcosnils
[dagger/dagger] New comment on discussion #6688: Mounted directory with withMountedDirectory is not updated on host

hey there! WithMountedDirectory works the same way as the RUN --mount instruction that you see in Dockerfiles.

It's not bind mounting the files in your machine to the build, what it does is mounting the files from your build context into your pipeline so your resulting container doesn't contain those files in the layered filesystem. If you want the files that you change in your pipeline in your host, you need to use the Export function. There's more info about that here: https:/...

ornate vigilBOT
#
marcosnils
[dagger] Discussion #6688 marked answered: Mounted directory with withMountedDirectory is not updated on host

hey there! WithMountedDirectory works the same way as the RUN --mount instruction that you see in Dockerfiles.

It's not bind mounting the files in your machine to the build, what it does is mounting the files from your build context into your pipeline so your resulting container doesn't contain those files in the layered filesystem. If you want the files that you change in your pipeline in your host, you need to use the Export function. There's more info about that here: https:/...

ornate vigilBOT
ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
#
wingyplus
[dagger/dagger] Issue opened: #6689 🐞 `dagger init` returns internal server erroro when inititate with git sdk

What is the issue?

Found on the PR https://github.com/dagger/dagger/pull/6559 which is already rebased on main. When create a module with git repository SDK (not a builtin). The engine returns internal server error instead of error from the SDK

Dagger version

dagger 4b825dc642cb6eb9a060e54bf8d69288fbee4904 (registry.dagger.io/engine) darwin/arm64

Steps to reproduce

$ mkdir first-module
$ cd first-module
$ ~/src/github.com/dagger/dagger/hack/with-dev dagger init --sdk=...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
ornate vigilBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6693 backfill integ test coverage for git ref modules

fixes https://github.com/dagger/dagger/issues/6623

Most of the recent bugs from the last week ended up being simple things related to using git module refs, which was due to the fact that we didn't have integ test coverage of those.

We didn't have coverage previously because it's slightly less obvious than it seems due to the limitation around git modules only being allowed to be from github.com/. That essentially rules out authoring the tests in the repo itself (there's various shena...

vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6694 Python: Refactor methods that return lists to not prefetch and select id instead

Noticed a performance impact in a module when iterating over a list returned by a dependent module.

The calling module was requesting a field (state), which should be returned directly by the API server, but was actually calling out (exec /runtime) on a bunch of functions that return a simple string. So these were flagged as simple fields to prefetch, even though I didn't need them, but it's the act of executing the runtime that slowed things down unnecessarily.

By loading from ID inste...

vivid lintelBOT
vivid lintelBOT
#
helderco
[dagger/dagger] Pull request opened: #6696 Python: Improve error message when using unsupported scalar or enums as arguments

Before:

TypeError: Expected collection type to be subscripted with 1 subtype, got 0: 

TypeError: Expected collection type to be subscripted with 1 subtype, got 0:

After:

NotImplementedError: Scalar types are not supported yet. Define argument as a string and convert to the desired scalar type in the function body.

NotImplementedError: Enum types are not supported yet. Define argument as a string and convert to the desired enum type in...
#
jedevc
[dagger/dagger] Pull request opened: #6697 fix: close service in pipeline test

This test is super annoyingly flaky :(

It seems like calling Close is racy, and sometimes we don't get the fully progress logs - to work around this for now, we can try and call service close explicitly, which should hopefully make this failure a bit more unlikely.

Yes, we should fix the underlying issue at some point - cc @grouville, I know you were also working on this.

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
vito
[dagger/dagger] Pull request opened: #6704 More visualization plumbing and polish
  • Rather than always marking ID vertices internal to avoid cluttering Cloud and the old TUI, we mark them internal just before sending downstream to either.
  • Better panic handling in DagQL + the TUI, plus fixed a couple of panics
  • Better errors out of DagQL in general

TODO:

  • [ ] Update TUI to handle new init and primary vertices specially
  • [ ] Just move meta fields to progrock.Vertex so we don't have to deal with the time period where they haven't arrived (super annoying)
vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Issue opened: #6706 Go SDK issues with newer go.mod than packaged into engine

Writing up a brain dump, so might be a bit messy.

Currently the go version used by the Go SDK is 1.21.3. However, a user module can set a custom go version (using go.mod/go.work) to a higher version, like 1.22.0.

Now, this can cause fun issues it seems? I'm not quite sure what happens, but essentially in https://github.com/dagger/dagger/blob/b2450fc1c61dfff5f04fe1679bc09b412c33a7e0/cmd/codegen/generator/go/generator.go#L292-L302

We can get a package returned that:

{...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6710 CLI: don't hide `id` from core types

Problem

When calling a function that returns a core type, like Container or Directory, I can then call any function from that core type... except id.

This makes it unnecessarily difficult to stitch together calls in a shell script, for example:

dagger call deploy --build="$(dagger call build --src . id)"

One workaround is to call sync, but that's confusing if I'm trying to get the object's ID to pass it to another function call. id is exactly what I'm l...

#
vito
[dagger/dagger] Pull request opened: #6711 idtui: print \n to tui if no trailing \n

Receiving a simple string value from a module is a pretty common use case, and most of the time there's no trailing linebreak, so if we print it verbatim the user will either have a broken prompt or see a funny little character if their shell is polite and adds a linebreak for them.

Rather than potentially encourage module authors to add a linebreak, we'll add it ourselves, but only if stdout is a TTY, since we want piping and redirects to always "tell to the truth."

#
shykes
[dagger/dagger] Issue opened: #6712 `WithDefaultTerminalCommand` doesn't support privileged or nesting

Problem

Container.WithDefaultTerminalCommand allows configuring the command to be executed by Container.Terminal. But some execution options are not available:

Option Description Available in withExec Available in withDefaultTerminalCmd
experimentalPrivilegedNesting Allow the executed command to make API calls to the Dagger Engine running it ✅ ❌
insecureRootCapabilities Execute the command in a privileged linux container ...
vivid lintelBOT
vivid lintelBOT
#
shykes
[dagger/dagger] Issue opened: #6713 Check if a file or directory exists

Problem

The Dagger API does not support checking if a file exists. There is an SDK-specific workaround (maybe? not documented) which is to 1) call any function that requires the file to exist, like Sync or Contents, 2) check for error, and 3) check for the error message returned when the file doesn't exist. This possibly can be wrapped in a convenience type by the SDK (again hard to tell as it is not documented).

This makes for a bad DX.

Solution

Add a new core function ...

vivid lintelBOT
#
sipsma
[dagger/dagger] Pull request opened: #6714 core: speed up loading of builtin SDKs

@vito noticed a while back that we were spending a lot of time loading the builtin Go SDK tarball. This fixes that bottleneck by avoiding use of local imports for built-in SDKs and instead using OCI store sources, which avoid the various problems with local sources (re-importing when used in parallel, need to always re-read tarball when importing containers, etc.).

Seeing some huge improvements when running subsets of tests, e.g. TestModuleGoInit takes 47s whereas it previously took 1m16...

vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6715 fix: ensure go types have MarshalJSON implementation

Fixes https://github.com/dagger/dagger/issues/6662.

With the below case, the object would inherit it's implementation of MarshalJSON from Directory:

type Foo struct {
    *Directory
}

We need to define our own MarshalJSON so we don't end up with this issue.

To do this, we just define another method next to our implementation of UnmarshalJSON, using the same trick of having an intermediate concrete struct (but we can handle module interfaces in a much simpler way, ...

vivid lintelBOT
vivid lintelBOT
#
jedevc
[dagger/dagger] Pull request opened: #6716 fix: remove global Client property on generated structs

See discussion in discord: #daggernauts message

https://github.com/dagger/dagger/pull/6680 made Client and Query reserved fields on all objects - but it's quite common to define Client on objects.

This patch is a little bit of a refactoring I've wanted to do for a while - it essentially allows binding clients to querybuilder.Selections, which means we can pretty much do away with Client+Query combinations, no...

vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6719 Very high memory usage in dagger engine

I noticed that on main when running go test -run=TestModule ... the engine ends up using over 6GB of RSS at times. By eye, it seems like it particularly spikes during TestModuleLotsOfFunctions (but did not fully confirm yet).

pprof is showing:

File: dagger-engine
Type: inuse_space
Time: Feb 22, 2024 at 10:18am (PST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1811.45MB, 71.51% of 2533.09MB total
Dropped ...
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6723 `dagger call` is tedious with too many flags+args

Somewhat related to https://github.com/dagger/dagger/issues/6112 but a bit more general

Passing a large number of flags and args when using dagger call can become way too tedious to type out by hand. One example would be support for filtering dirs as mentioned here. But really any time a function accepts more than a few flags and/or the arguments to the flag end up being verbose are a problem.

This leaves users ...

vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6724 Support for testing modules

(We had an internal Linear issue for this but creating a public GH one)

You should be able to write test code for your module code and easily run those tests. Currently, you really just have to invoke the dagger CLI to "test" a module (that's what our integ tests do for the most part), which isn't nearly as expressive+simple as writing actual test code like you normally would in any language.

Various ideas on how to approach this:

  1. SDKs could add support for it via another Function a...
vivid lintelBOT
#
sipsma
[dagger/dagger] Issue opened: #6725 Support passing modules as arguments in the CLI

Modules can accept other modules as arguments to functions. Use cases:

  1. SDKs implemented as modules do this already by accepting ModuleSource as an arg
  2. Support for running tests of modules could use this: https://github.com/dagger/dagger/issues/6724
  3. Support for re-executing modules in specialized hosts (i.e. run my module on a "gpu-as-a-service" platform)
  4. Combined with support for interfaces, you could pass a module as an implementation of an interface argument (if it matches...
#
stobias123
[dagger/dagger] Issue opened: #6727 ✨ Manually set Dagger Cloud run Title

What are you trying to do?

I'd like to be able to set the title of my runs in dagger cloud. I execute multiple pipelines per commit, so it's a little hard to find what I'm looking for based on commit title.

Example:
I run a (sequential) pipeline that

  1. Builds everything
  2. Runs terraform for each environment

Right now each of the above is a separate dagger call. It's hard to find say, the terraform dev run when it all looks like this

![image](https://github.com/dagger/dagge...

vivid lintelBOT
#
helderco
[dagger/dagger] Issue opened: #6728 Zenith: default to use a class instead of top-level functions in Python

Summary

Change the dagger init template for Python to use a class instead of just functions. Also update documentation snippets to do the same.

Motivation

In an earlier iteration of Dagger modules, when the concept of objects that contain functions were introduced, the Python SDK kept the convenience of being able to define simple top-level functions, without having to convert into a class. Python allows this by transparently “creating” the object and associate the top-level ...

vivid lintelBOT
vivid lintelBOT
#
benjaminstrasser
[dagger/dagger] Pull request opened: #6730 Draft: Added detection for bun/node inside typescript SDK runtime

As discussed by #6591

Changes

Tries to detect bun/node runtime through the following rules. If a detected runtime is not node or bun an error is raised.

If a package.json is present and the dagger.runtime field is set, the configured runtime is used
If a package-lock.json file is detected node is used a the runtime
if a bun.lockb file is detect bun is used as the runtime
if no runtime can be detected node is used by default

vivid lintelBOT
#
marcosnils
[dagger/dagger] Issue opened: #6731 🐞 [CLI]: `dagger functions $function_name` returns an error if return type is not a dagger object

What is the issue?

calling dagger functions foo on the following function definition raises the following error:
Error: function 'foo' returns non-object type STRING_KIND

package main

type Testmodule struct{}

func (m *Testmodule) Foo() string {
	return "bar"
}

Dagger version

dagger v0.9.11

Steps to reproduce

run dagger functions foo on the code given above

Log output

Error: function 'foo' returns non-object type STRING_KIND

vivid lintelBOT