Using "configure_logging(logging.DEBUG)" in Python SDK has major impact on performance

[uneven quest]

Hi all

This for information.

One month ago, I've posted this question https://discord.com/channels/707636530424053791/1357325635160244374 and the workable solution was to add the configure_logging(logging.DEBUG) in my code.

But I've not paid attention to this: this has a very huge impact on performances.

In concrete terms: I'm running time dagger call [my_own_flags] [my_action] where f.i. my_actionis nothing more than purely a return 1.0. Nothing more than that.

With configure_logging(logging.DEBUG) in my code, when I run the time dagger call [my_own_flags] version command, time reports me a duration of 72 seconds. By rerunning the same command again and again, I got 66 seconds or 67 seconds.

By commenting the configure_logging(logging.DEBUG) line and running again, the execution time falls to 13 seconds.

I've experienced this to another actions like lint_docker (where I'm running Hadolint on my codebase); with configure_logging(logging.DEBUG) successive execution time are 72 seconds, 67 seconds, 67 seconds, 66 seconds. (Successive: I'm rerunning the exact same command again; without any changes to my codebase so cache system should be used).

Without, I drop to 21 seconds for the first run then 7 seconds for the second and 6 for the third.

Is this normal?

Ping @wary tinsel

(I fully understand such configure statement shouldn't stay in my production codebase)

[wary tinsel]

It doesn't surprise me but I also haven't spent any time optimizing the performance with debug logs enabled. They were initially added to help develop the SDK and then kept to help debug issues that aren't clear otherwise. It's not meant to be left turned on. I suggest opening an issue if you're getting an error that you think should show on normal circumstances (e.g., at runtime, from function body/workflow rather than signature).

Exception traces are usually very verbose and early on we found it was adding quite a lot of output on our integration tests so I dialed it back, but maybe a balance can be achieved here like excluding part of the trace stack when it comes from certain libraries like asyncio/anyio or httpx. Or maybe just showing the part of the stack that comes from your own code could be enough, perhaps also from the dagger-io library. 🤷‍♂️