Recommended patterns to handles logs from pipeline steps? | Dagger | Page 1

copper sparrow Jun 10, 2024, 2:16 PM

#

Is there any recommene patterns to send logs from each stesp of dagger piepline, main goal of this is it collect logs of each step into files?

marble acorn Jun 10, 2024, 2:42 PM

#

copper sparrow Is there any recommene patterns to send logs from each stesp of dagger piepline,...

if you're setting up OTEL (https://discord.com/channels/707636530424053791/1249642165278343230) , logs are being attached to the OTEL spans following the OTEL spec.

copper sparrow Jun 10, 2024, 2:52 PM

#

Yeah I agree 👍
But In my use case we also need logs to be saved in files in order to show for customers using gitlab artifacts

neat lance Jun 10, 2024, 4:29 PM

#

I think Buildkit stores its log somewhere (/cc @topaz cairn) ? I'm not sure though it will be super human readable.
I wonder if there's a way to capture the engine stdout into a file

copper sparrow Jun 10, 2024, 4:44 PM

#

i can think of one possible way is by storing and passing logs file for each steps and returning to the caller as directory and at the end may be orchetrator can save it, but thought that may be there's any out of box support 🙂

topaz cairn Jun 10, 2024, 5:16 PM

#

you can configure the Dagger CLI to send data to any OTel-compatible provider with the standard OTEL_ env vars - maybe you could configure a collector that writes to local disk?

#Recommended patterns to handles logs from pipeline steps?