Extracting messages from stderr from progress output?

[glass canopy]

I noticed that if a program emits messages to stderr, these messages won't appear in the TUI.

This is unfortunate; typically logging frameworks by default log to stderr, so these messages disappear in the execution of a dagger workflow.

Is there a way to configure dagger to also emit messages from stderr?

For reference, I'm using v0.20.2.

Just to add that I confirmed that stderr messages were indeed emitted if I injected a terminal into the workflow at a step I know for a fact logs are emitted to stderr. Running this workflow step as usual does not emit these log messages.

Even with verbosity increased to 5, stderr messages are not emitted. Stdout messages are.

[flint wedge]

Do you have an example of code I can use to reproduce it?
I did try and I have both stdout and stderr visible.
I prefixed them with stdout/stderr and added (func) for the ones emitted by the function itself, and (ctr) for the ones emitted by the container.

The output:

$ dagger call container-echo --string-arg plop combined-output -E
▼ ✔ MyModule.containerEcho(stringArg: "plop"): Container! 2.4s
  ● ✔ container: Container! 0.0s
  ● $ .from(address: "alpine:latest"): Container! 0.5s CACHED
  ▶ ✔ withExec sh -c 'echo "stdout(ctr): $1" && echo "stderr(ctr): $1" >&2' -- plop 0.1s

  stderr(func): plop
  stdout(func): plop


✔ connect 0.2s
✔ load module: . 0.6s
✔ parsing command line arguments 0.0s

✔ myModule: MyModule! 0.0s
✔ .containerEcho(stringArg: "plop"): Container! 2.4s
✔ .combinedOutput: String! 0.1s

stdout(ctr): plop
stderr(ctr): plop

The module

package main

import (
        "fmt"
        "os"
        "dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Returns a container that echoes whatever string argument is provided
func (m *MyModule) ContainerEcho(stringArg string) *dagger.Container {
        fmt.Fprintln(os.Stdout, "stdout(func): "+stringArg)
        fmt.Fprintln(os.Stderr, "stderr(func): "+stringArg)
        return dag.Container().From("alpine:latest").WithExec([]string{"sh", "-c", "echo \"stdout(ctr): $1\" && echo \"stderr(ctr): $1\" >&2", "--", stringArg})
}

I also used combined-output to capture both outputs at once

$ dagger call container-echo --string-arg foo stdout -E
▼ ✔ MyModule.containerEcho(stringArg: "foo"): Container! 0.7s
  ● ✔ container: Container! 0.0s
  ● $ .from(address: "alpine:latest"): Container! 0.4s CACHED
  ▶ ✔ withExec sh -c 'echo "stdout(ctr): $1" && echo "stderr(ctr): $1" >&2' -- foo 0.1s

  stderr(func): foo
  stdout(func): foo


✔ connect 0.2s
✔ load module: . 0.2s
✔ parsing command line arguments 0.0s

✔ myModule: MyModule! 0.0s
✔ .containerEcho(stringArg: "foo"): Container! 0.7s
✔ .stdout: String! 0.1s

stdout(ctr): foo

In this case, I only get the stdout output from the returned container, but it also made visible the stderr and stdout outputs created during the function call
(works also with stderr)

[glass canopy]

Thanks for a quick reply and sorry for a late reply from me 😄 .

It turns out that the way logging was structured in this specific program was incorrect. It's a Rust binary that used indicatif, a library for emitting progress reporting.

I'm not sure how this library detects if the process is invoked from a terminal emulator, but this library seems to at least reason that we are not inside a terminal when running dagger workflows, so no output is emitted.

I did a simple work-around to verify that output was written to stderr by using the eprintln in this binary.

So this turned out to not be a dagger issue, just incorrect setup of this custom logger.