How to execute python code inside a container within a pipeline?

[icy sleet]

Hello to everyone! I've just discovered Dagger yesterday and now I'm trying to implement a simple pipeline. I want to execute some python logic inside the container, but I see only exec option to execute shell commands. Is it possible to execute python functions from the same file or imported from elsewhere?

[rich bear]

Welcome! 👋 Have you checked out the python getting-started guide? It's a great example to get up and running with the Dagger Python SDK. It includes examples of executing python in a container, using the exec call as you mentioned using a container that has python preinstalled
https://docs.dagger.io/sdk/python/628797/get-started

[icy sleet]

Hello @rich bear , I saw the getting-started guide but the only way I found there is doing it like exec(["python", ...]), am I understand correctly that this is the only way? I was hoping to find something similar to Jenkins style groovy functions invocation 🙂

[rich bear]

Can you describe what one of these functions might do in your pipeline so I can better understand your needs?

[coarse elk]

You can abstract into a function a container that runs a python script by mounting a file with your contents. Then just use that whenever you need:

import textwrap

import anyio

import dagger
from dagger.api.gen import Container


def script(client: dagger.Client, contents: str) -> Container:
    return (
        client.container()
        .from_("python:alpine")
        .with_mounted_directory(
            "/mnt",
            client.directory().with_new_file("/script.py", textwrap.dedent(contents)),
        )
        .exec(["python", "/mnt/script.py"])
    )


async def main():
    async with dagger.Connection() as client:
        zen = (
            await script(
                client,
                """
                import this
                """,
            )
            .stdout()
            .contents()
        )
    print(zen)


anyio.run(main)

[icy sleet]

Just looping through some files and checking the content for example. In general executing some repository or environment dependent logic.

[icy sleet]

Thanks a lot! But do you think is possible to achieve the plain invocation like for example client.container().from_("python:alpine").exec_method(lambda x: do something) ?
Now looking at that structure I think it would be pretty complicated to implement, because you'd need to setup the python environment in the container with all dependencies.

[coarse elk]

Yeah, dagger runs everything in containers but the SDK allows you to use python libraries that run in your host. If you do want to run a snippet of python code in a container, you need to put it in a string that can be fed into a python executable.

[icy sleet]

Okay, I see, thank you for your help!

[near haven]

@icy sleet you could probably create your own helper function that does what you describe, and translates it under the hood into lower-level Dagger calls. Could be pretty awesome