#Hey all I am experimenting with Dagger

1 messages · Page 1 of 1 (latest)

silent epoch
#

For example given this code
await test_process.exit_code()
which exits 1, I am seeing

Traceback (most recent call last):
  File "/Users/user/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/dagger/api/base.py", line 159, in _handle_execute
    yield
  File "/Users/user/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/dagger/api/base.py", line 105, in execute
    result = await self.session.execute(query)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/gql/client.py", line 1231, in execute
    raise TransportQueryError(
gql.transport.exceptions.TransportQueryError: {'message': 'process "rspec spec/models/lab_spec.rb" did not complete successfully: exit code: 1\nStdout:\nAn error occurred while loading ./spec/models/lab_spec.rb.\nFailure/Error: raise(MissingConfig, message)\n\nGlobalConfig::MissingConfig:\n  Coul
silent epoch
#

Do reduce to a base case

    async with dagger.Connection(dagger.Config()) as client:
        await client.container().with_exec(["exit", "1"]).exit_code()

Surely this should not throw anything, but instead return something

formal burrow
#

The good news is that there is a workaround (not clearly explained in the issue, we should fix that cc @calm abyss @ebon terrace ).

#

I believe the workaround is to catch the error, and get the contents of stdout/stderr from the error message.

Obviously this is not as clean, but should at least get you unstuck until we implement a cleaner fix

silent epoch
#

Is it accurate to say this is not limited to exit_code? I'm just playing around and it seems to me that for a failed process, stdout() and stderr() suffer the same issue

formal burrow
#

that's probably true - @calm abyss @vernal girder @lusty mason can confirm

silent epoch
#

The workaround appears to omit output

If I look at the output that comes from Config log_output i.e.

dagger.Connection(dagger.Config(log_output=sys.stderr))

It contains information that

try:
    exit_code = await process.exit_code()
except Exception:
    print(sys.exception())

misses. It looks to me like printing the exit code exception only includes STDERR, not STDOUT

formal burrow
#

cc @ebon terrace this one should definitely be on your DX radar...

lusty mason
calm abyss
ebon terrace
silent epoch
limber tapir
# silent epoch

Hey @silent epoch, do you have a simple example I can use to reproduce this? Are you actually seeing missing output or just output from the engine and python mixed together? Truncation usually means missing output after a certain point.

#

If you use log_output=sys.stderr at the same time as print(), the output from each may get mixed which can be a bit confusing. There's multiple ways to solve this. For example, you can use the rich library to print messages (e.g., rich.print(...)) which will give them a distinctive look, or use a file in log_output instead of dumping to the terminal which will still allow you to debug what the engine is returning but still keep the terminal output only for your code. You can also use print("xxx", flush=True) to send all output at once to the terminal, or you can even take advantage that one is being sent to your stderr while the other is going to stdout to redirect them to different places with the shell.

#

It looks like it's the case when I see your second output:

<bunch of engine logs>
process "bundle exec rspec" did not complete successfully: exit code: 1
Stdout:
<more engine logs here>
Top 0 slowest examples (0 seconds, 0.0% of total time):

Finished in 0.00004 seconds (files took 2.96 seconds to load)
0 examples, 0 failures, 1189 errors occurred outside of examples
<more engine logs here>
Stderr:
#

Another thing worth noting is that if you're using log_output, then you'll see some engine logs finishing output after the with dagger.Connection() block ends, so if your last print() statement isn't outside of that block, it won't be the last thing you see.

#

Consider this example:

import sys

import anyio
import dagger


async def main():
    cfg = dagger.Config(log_output=sys.stderr)

    async with dagger.Connection(cfg) as client:
        version = (
            await client.container()
            .from_("python")
            .with_exec(["python", "-V"])
            .stdout()
        )

-       print(version)
+   print(version)


anyio.run(main)

Inside the with block you'll see something like:

<bunch of engine output>
Python 3.11.2
<last of engine output>

If you move it outside the with block, then:

<bunch of engine output>
Python 3.11.2
#

I know it can be helpful to have these considerations documented, I have it on my list of issues to create.

silent epoch
#

I'm off 10% time today so I won't have a chance to look at this for a couple weeks, but I will follow up.

If its at all helpful I suspect a minimal example would simply be running a container that echo's > 100 lines (or >10k characters) and then exits 1.

But I'll set that up if someone doesn't beat me to it

limber tapir
limber tapir
#

Is this what you mean?

import sys

import anyio
import dagger
import rich
from rich.panel import Panel


async def main():
    cfg = dagger.Config(log_output=sys.stderr)

    async with dagger.Connection(cfg) as client:
        ctr = (
            client.container()
            .from_("bash")
            .with_exec(["bash", "-c", 
                'for i in {1..100}; do echo "Line <$i> output"; done; exit 1'
            ])
        )
        try:
            await ctr.exit_code()
        except dagger.QueryError as e:
            rich.print(Panel(str(e)))


anyio.run(main)

I've put a border around the print in except to make it more visible. Seems to be all there. As for the engine logs, looking at the last lines seem like it's getting cut, but if you look more closely the lines are simply coming out of order (not sorted).