calling steps in parallel

[tidal carbon]

Hey, hopefully a quick question… I've got this step:

func (m *Beeps) All(
    ctx context.Context,
    // +optional
    // +defaultPath=.
    source *dagger.Directory,
) []*dagger.Container {
    return []*dagger.Container{
        m.Build(ctx, source, false),
        m.Clippy(ctx, source),
        m.Typos(ctx, source),
    }
}

I'd like those calls to run in parallel. Is there a nice way to do this that doesn't involve the boilerplate of an error group?

[thick walrus]

@swift star any ideas to share? https://github.com/dagger/dagger/issues/4766

@tidal carbon I think an error group is what you need today for building up your slice from function results otherwise you will get serial function execution instead of parallel.
The good news is that caching should kick in on subsequent runs, but if caching doesn't help you there (because of source changes, for example), then I'd look at an error group approach. I haven't tried this much with modules TBH, but would like this pattern fleshed out.

@jagged flame Did these examples go away? Having trouble finding in current cookbook.
https://github.com/dagger/dagger/pull/5833

[jagged flame]

It should have been ported to the new cookbook but looks like it wasn't, I pinged @past kiln internally to ask about this

[tidal carbon]

that's helpful stuff. Thank you!

[jagged flame]

@tidal carbon this is now in the cookbook. I hope it's helpful! https://docs.dagger.io/cookbook#execute-functions-concurrently also fyi @thick walrus

[tidal carbon]

whoo, nice! thank you 😄

[valid blaze]

There is also a trick you can use if you want to make your Build container dependent on a successful Clippy and Typos.

doneClippy := ctrClippy.withExec([]string{“touch”, “.done”}).File(“.done”)
ctrBuild = ctrBuild.withFile(".doneClippy", doneClippy)
doneTypos := ctrTypos.withExec([]string{“touch”, “.done”}).File(“.done”)
ctrBuild = ctrBuild.withFile(".doneLint", doneTypos)
return ctrBuild

This makes it lazy and dagger takes care of concurrency.