#hey My demo went great today I ve

1 messages · Page 1 of 1 (latest)

edgy star
#

👋 the reason why the connection gets closed is because most of our examples are using python's with statement (https://docs.python.org/3/reference/compound_stmts.html#the-with-statement) to execute the pipeline withing a context. So in order to use the same connection across multiple files, you need to pass the same client variable you get in the with context

Python documentation
8. Compound statements

Compound statements contain (groups of) other statements; they affect or control the execution of those other statements in some way. In general, compound statements span multiple lines, although i...

wooden knoll
#

Yes, you have to call the rest of your code within the body of the with connection block (calling a function, passing client). There’s other ways to do it if you can’t wrap the connection over the rest (e.g. ExitStack). If you can share how you’re structuring your files even if a simple dummy example I can give more specific guidance. Are you using Typer or Click?

stone fjord
#

Hey! thanks for the replies I am using neither - but i can look into wrapping the call in ExitStack but I am unsure if this will work. I am not using typer or click I am calling dagger run and the individual python file. I am using neither typer or click. I can absolutely share some dummy code -
https://gist.github.com/seanrclayton/19ac38a6ee0f847b3e38c1738620914f

Gist
build_rpm.py

GitHub Gist: instantly share code, notes, and snippets.

#

dagger run -- build_rpm_docker.py is what I run. Curious on trying ExitStack. I'll give that a go - unless I'm missing osmething obvious.

wooden knoll
# stone fjord Hey! thanks for the replies I am using neither - but i can look into wrapping t...

Thanks. There's a lot that can be done here, I'll focus on the next simplest thing.

First, you should remove the async in def dagger_connection since you're not doing any await inside it. This way you don't have to handle a coroutine when you don't need to.

Secondly, if you want to be able to execute each script manually but also reuse those functions, I suggest you add this boilerplate to your scripts:

async def build_docker_rpm(client: dagger.Client):
    ...

async def main():
    async with dagger_connection() as client:
        await build_docker_rpm(client)

if __name__ == "__main__":
    anyio.run(main)

Same idea for the other file. main establishes connection and passes client as a dependency. Now you can import build_rpm and pass it the client, without going through another with dagger_connection.

# 
async def build_rpm(client: dagger.Client):
    ...

async def main():
    async with dagger_connection() as client:
        await build_rpm(client)

if __name__ == "__main__":
    anyio.run(main)
#

You can remove a bit of boilerplate by sharing main:

# dagger_connection.py
import sys
import anyio
import dagger

def run(func):
    async def main():
        async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client:
            return await func(client)
    return anyio.run(main)
      

# build_rpm.py
import dagger
from .dagger_connection import run

async def build_rpm(client: dagger.Client):
    rpm_commands = (...)
    ...

if __name__ == "__main__":
    run(build_rpm)


# build_docker_rpm.py
import dagger
from .dagger_connection import run
from .build_rpm import build_rpm

async def build_docker_rpm(client: dagger.Client):
    rpm_container, application_rpm_name, application_name, docker_image_tag = await build_rpm(client)
    secret = client.set_secret("reg_user", os.environ["GITLAB_REGISTRY_TOKEN"])
    ...

if __name__ == "__main__":
    run(build_docker_rpm)
stone fjord
stone fjord
#

So running the second blocks build_docker_rpm.py - It connects to the dagger engine and exits (also the import statement should be from dagger connection import run correct? In any case I suspect the return here exits the event loop which causes the remaining code not to run. I will tinker with the code you provided - but I will also look into using a context manager (something I've never done dagger )

wooden knoll
#

As for exiting after connection, you can share what you have. Maybe there's a step missing.

stone fjord
#

sure

wooden knoll
#

Re: the context manager. You definitely can, but dagger.Connection is already one, there's no benefit wrapping in another for this simple case.

stone fjord
#

Apologies for not posting directly.

wooden knoll
#

No, that's fine 🙂 You say that code exits early?

stone fjord
#

correct running python_build_docker_rpm.py just returns Connected to engine d3268f8f3700 then exits

wooden knoll
#

In build_rpm_docker.py the if __name__ is not well indented.

#

From what you've shared, there's no executions. It's all lazy building.

#

(i.e., no awaits)

stone fjord
#

thats a copy paste issue - updated. in build_docker_rpm.py I am awaiting on build_rpm. Do I need to await something in build_rpm.py?

wooden knoll
#

Yes. In this case, you need to await something in dagger, otherwise nothing will actually be sent to the engine to run.

stone fjord
#

oh! I added an await to the dagger connection file. python def run(func): async def main(): async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client: return await func(client) return anyio.run(main)

#

that seems to work. Gitlab is having issues at the moment - So I cannot be sure. But this is good enough. Again - I am so thrilled with the dagger community. So much thnaks.

fallow gust