Parallel builds across chained actions | Dagger | Page 1

stone rune May 10, 2024, 3:15 PM

#

Hello

I noticed that Dagger is building things sequentially when I'm chaining Dagger Functions.

I'm running "dagger call job-group out".
The first function (job-group) returns multiple objects. A second chained function (out) wants to evaluate everything. And the build process will not build the objects in parallel.

Is this expected?

I added an optional sync to the first function ("dagger call job-group --sync out") to work around this issue.

Is there a better way to fix this?

#

📎 main.go

zealous spindle May 10, 2024, 3:31 PM

#

This is interesting! I don't think this is expected behavior. Let me play around with a repro locally
cc @hazy yarrow @cobalt agate @odd harbor

stone rune May 10, 2024, 3:43 PM

#

By the way: I'm using dagger v0.11.2 (registry.dagger.io/engine) darwin/arm64

#

And registry.dagger.io/engine:v0.11.1 locally.

zealous spindle May 10, 2024, 3:45 PM

#

Thanks! Yeah I'm seeing the same thing on latest

#

just to confirm the behavior is specific to chaining, I created this function:

func (r *MyModule) Hack(ctx context.Context) *Directory {
    jobs := r.JobGroup(ctx, false)
    return jobs.Out()
}

and indeed it runs the jobs concurrently. So things aren't working as expected with cli chaining

#

created https://github.com/dagger/dagger/issues/7353

GitHub

DAG loses concurrency with module chaining · Issue #7353 · dagger/d...

When chaining module functions from the CLI, it appears that the expected DAG concurrency is lost. Here is an example: package main import ( "context" "fmt" "math/rand"...

stone rune May 10, 2024, 4:02 PM

#

Thank you!

#Parallel builds across chained actions