Getting the logs of Dagger-engine

[grizzled kindle]

Hey, we are running some Dagger workloads on Gitlab runners that we can't acccess. Is there any way to fetch the logs of Dagger-engine through Go DSK?

[normal bear]

Hi. You can use LogOutput in Connect to send the logs to os.Stderr or a file. https://pkg.go.dev/dagger.io/dagger#WithLogOutput

All our docs examples have the LogOutput set to stderr.

[silk cobalt]

_EXPERIMENTAL_DAGGER_JOURNAL: "/tmp/journal.log" will write the logs to this file. FWIW:

• https://github.com/dagger/dagger/blob/1636fd2fa753b978191d7e8c4661f561ca63ee8d/.github/workflows/test.yml#L33C11-L33C59
• https://github.com/dagger/dagger/blob/1636fd2fa753b978191d7e8c4661f561ca63ee8d/.github/workflows/test.yml#L64-L66

Is this something worth adding to our cookbooks @brisk tree ?

[brisk tree]

We could, but with the caveat that the recipe relies on an experimental variable which may be unsupported in future. WDYT?

[normal bear]

The journal is machine-readable, used by our GUI. You can write a program that reads from that and presents a more human readable output, but is that what OP is asking? It's not the same thing as what you get with LogOutput.

[silk cobalt]

Good point. I think you are right, this may be less useful - WithLogOutput is definitely what is needed here. Here is an example from our own dogfooding:

• https://github.com/dagger/dagger/blob/1636fd2fa753b978191d7e8c4661f561ca63ee8d/internal/mage/cli.go#L27

There is a lot of insight in that journal.log that would be super helpful if it could be replayed with the dagger CLI in different ways (UI, TUI, plain logs, etc.).

[normal bear]

Yeah, Alex has mentioned it several times. It's on his scope.

But I don't know if there's an issue for it.

The idea is to start the interactive TUI with it to help debug a past run.

[silk cobalt]

This is another one variable that I would like us to remove _EXPERIMENTAL from, right after _EXPERIMENTAL_DAGGER_RUNNER_HOST. They are both essential for many pipelines.