How do I chain function calls without losing stdout from previous steps ?

[velvet glacier]

Say i have a pipeline with 2 functions returning the module for chaining. How can I see logs from all previous .WithExec calls ?

e.g : if i do the following i'm only seeing "hello world from function2 "
dagger call function-1 function-2 container stdout

package main

import (
    "context"
    "dagger/test-chaining/internal/dagger"
)

type TestChaining struct {
    Container *dagger.Container
}

func (m *TestChaining) Function1(ctx context.Context) *TestChaining {
    m.Container = dag.Container().From("alpine").WithExec([]string{"sh", "-c", "echo \"hello from function 1\""})
    return m
}

func (m *TestChaining) Function2(ctx context.Context) *TestChaining {
    m.Container = m.Container.WithExec([]string{"sh", "-c", "echo \"hello from function 2\""})
    return m
}

I know stdout only returns the output from the last .WithExec() but what would be the right way to see stdouts from all WithExecs ?

[weak stratus]

👋 Since you're keeping the same alpine container around to exec in, you could append to a file in that container, or store the state in a string in the struct, perhaps. Let me give it a quick go.

One option...

package main

import (
    "context"
    "dagger/test-chaining/internal/dagger"
)

type TestChaining struct {
    Container *dagger.Container
    Stdout string
}

func New() *TestChaining {
    return &TestChaining{
        Container: dag.Container().From("alpine"),
        Stdout: "",
    }
}

func (m *TestChaining) Function1(ctx context.Context) *TestChaining {
    m.Container = m.Container.WithExec([]string{"sh", "-c", "echo \"hello from function 1\""})
    out, _ := m.Container.Stdout(ctx)
    m.Stdout = m.Stdout + out
    return m
}

func (m *TestChaining) Function2(ctx context.Context) *TestChaining {
    m.Container = m.Container.WithExec([]string{"sh", "-c", "echo \"hello from function 2\""})
    out, _ := m.Container.Stdout(ctx)
    m.Stdout = m.Stdout + out
    return m
}

dagger call function-1 function-2 stdout

hello from function 1
hello from function 2

[velvet glacier]

Thanks @weak stratus that works ! very cumbersome though, could be handy to be able to optionally pass a *string or *strings.stringbuilder to Stdout() or WithExec() ?

[weak stratus]

@last niche related to the complex of issues around stdout, stderr.

In this case, getting the cumulative stdout of chained WithExec()s

Guessing a new issue would be helpful, or is there a good exisiting one?

https://github.com/dagger/dagger/issues/3559
https://github.com/dagger/dagger/issues/3496
https://github.com/dagger/dagger/issues/6553

[wet talon]

@velvet glacier here's another way to refactor @weak stratus 's code and use a more generic approach 🙏

type Lala struct {
    Container *dagger.Container
    // +private
    StdoutBuf string
}

// Returns a container that echoes whatever string argument is provided
func (m *Lala) F1(ctx context.Context) *Lala {
    m.Container = dag.Container().
        From("alpine:latest").
        With(m.logExec(ctx, []string{"echo", "hello"}))
    return m
}

func (m *Lala) F2(ctx context.Context) *Lala {
    m.Container = dag.Container().
        From("alpine:latest").
        With(m.logExec(ctx, []string{"echo", "world"}))
    return m
}

func (m *Lala) logExec(ctx context.Context, args []string) dagger.WithContainerFunc {
    return func(c *dagger.Container) *dagger.Container {
        out, _ := c.WithExec(args).Stdout(ctx)
        m.StdoutBuf += out
        return c
    }

}

func (m *Lala) Stdout(ctx context.Context) string {
    return m.StdoutBuf
}

Having said that, having a way to provide a Writer like interface to RedirectStdout (and equivalents) would be nice. It's not trivial though since we'd need to do some sort of SDK changes so this behavior is handled the same way everywhere. The approach above should allow you to move forward without too many boilerplate needed