dagger run python fails when using python directly works | Dagger | Page 1

rocky pelican Oct 26, 2023, 12:15 PM

#

Hello, I'm trying to follow the tutorial at https://docs.dagger.io/quickstart/593914/hello

When I run dagger run python main.py I get an error:

▶ dagger run python main.py                                                                                                         14:07:29
┣─╮
│ ▽ init
│ █ [1.38s] connect
│ ┣ [1.27s] starting engine
│ ┣ [0.11s] starting session
│ ┃ OK!
│ ┻
█ [0.00s] ERROR python main.py
┻
• Engine: 898f38986522 (version v0.9.0)
⧗ 1.39s ✔ 3 ✘ 1
exec: "python": executable file not found in $PATH```

If I do `python main.py` directly then it's working correctly:

```▶ python main.py                                                                                                                    14:08:02
⠙ Creating new Engine session1: connect
1: > in init
1: starting engine
⠹ Creating new Engine session1: starting engine [2.42s]
1: starting session
⠼ Creating new Engine session1: [2.61s] OK!
1: starting session [0.18s]
1: connect DONE

6: resolve image config for docker.io/library/python:3.11-slim
6: > in from python:3.11-slim
6: resolve image config for docker.io/library/python:3.11-slim DONE

11: pull docker.io/library/python:3.11-slim
11: > in from python:3.11-slim
11: resolve docker.io/library/python:3.11-slim@sha256:d36d3fb6c859768ec62ac36ddc7397b5331d8dc05bc8823b3cac24f6ade97483
11: resolve docker.io/library/python:3.11-slim@sha256:d36d3fb6c859768ec62ac36ddc7397b5331d8dc05bc8823b3cac24f6ade97483 [0.01s]
11: pull docker.io/library/python:3.11-slim DONE

9: exec python -V CACHED
9: exec python -V CACHED
Hello from Dagger and Python 3.11.6```

Is the doc wrong or is there a step I'm missing?

Write your first Dagger pipeline | Dagger

Write your first Dagger pipeline

restive raven Oct 26, 2023, 12:39 PM

#

Hi! What's the output of which python?

rocky pelican Oct 26, 2023, 12:41 PM

#

Hello, ▶ which python 14:40:02 python: aliased to /usr/local/bin/python3

#

I'm on MacOS, dagger installed with brew

restive raven Oct 26, 2023, 12:42 PM

#

Is that the system python?

#

Seems like you have a shell to alias python -> python3. Are you in an activated virtual env?

rocky pelican Oct 26, 2023, 12:43 PM

#

I do have alias python=/usr/local/bin/python3 in my .zshrc you're right, I had forgotten about it

restive raven Oct 26, 2023, 12:43 PM

#

Ok, then try dagger run python3 main.py

rocky pelican Oct 26, 2023, 12:43 PM

#

yes I just did this it works

#

The error message could be improved I think but I get it now

restive raven Oct 26, 2023, 12:44 PM

#

dagger run finds the executable in $PATH, it doesn't know about shell specific things.

rocky pelican Oct 26, 2023, 12:46 PM

#

yeah it was not clear to me that it was using locally installed python

#

I thought dagger run python was a builtin command

restive raven Oct 26, 2023, 12:47 PM

#

dagger run is a wrapper around any executable you have. It could be a local script.sh as well or any other language.

#

You should create and activate a virtual env for python so you don't run off of the system environment though. Then you should have python in your $PATH, but I'd remove that alias or you'll get the wrong pyhon interpreter every time.

#

I mean even without dagger run. If you activate a virtual env, python should point to that. If you have the alias, it'll override what's on $PATH and get you the system one instead. You'd have to do the same thing of choosing python3 to get the virtual env's.

#

Removing the alias avoids confusion.

rocky pelican Oct 26, 2023, 12:53 PM

#

Thanks for your help, I removed the alias 😉

restive raven Oct 26, 2023, 12:54 PM

#

https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments

rocky pelican Oct 26, 2023, 12:54 PM

#

Yes I have the venv but you're right it was shadowing python

restive raven Oct 26, 2023, 12:55 PM

#

Ok 👍

rocky pelican Oct 26, 2023, 12:59 PM

#

Although now that I'm working in the venv I'm getting this permission error:

┣─╮
│ ▽ init
│ █ [1.55s] connect
│ ┣ [1.51s] starting engine
│ ┣ [0.04s] starting session
│ ┃ OK!
│ ┻
█ [0.00s] ERROR infra/dagger/main.py
┻
• Engine: 898f38986522 (version v0.9.0)
⧗ 1.56s ✔ 3 ✘ 1
fork/exec infra/dagger/main.py: permission denied```

restive raven Oct 26, 2023, 1:04 PM

#

You need the python interpreter after dagger run. Otherwise you need to add the bang to your script and make it executable.

rocky pelican Oct 26, 2023, 1:05 PM

#

haha I'm wasting your time here facepalm

#

sorry

restive raven Oct 26, 2023, 1:06 PM

#

🙂

rocky pelican Oct 26, 2023, 1:09 PM

#

While I have you, is there a way to pass arguments to my pipeline?

#

didn't find it in the docs

#

This is what I tried:


import anyio
import dagger


async def main(test_files: str):
    async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client:
        db_password = "postgres"

        postgres = (
            client.container()
            .from_("dashdoc/postgres:14.4")
            .with_env_variable("POSTGRES_USER", "dashdoc")
            .with_env_variable("POSTGRES_DB", "dashdoc")
            .with_env_variable("POSTGRES_PASSWORD", db_password)
            .with_exposed_port(5432)
            .as_service()
        )

        redis = client.container().from_("redis:6-alpine").with_exposed_port(6379).as_service()

        # set build context
        context_dir = client.host().directory("backend")
        django = await context_dir.docker_build(target="test-image")

        test = await (
            django.with_service_binding("db-test", postgres)
            .with_service_binding("redis", redis)
            .with_env_variable("DJANGO_SETTINGS_MODULE", "dashdoc.settings.pytest")
            .with_env_variable("DB_NAME", "dashdoc")
            .with_env_variable("DB_USER", "dashdoc")
            .with_env_variable("DB_PASSWORD", db_password)
            .with_env_variable("DB_HOST", "db-test")
            .with_env_variable("REDIS_URL", "redis://redis:6379")
            .with_env_variable("DEBUG", "false")
            .with_exec(
                [
                    "pytest",
                    "-n=4",
                    "-vvvv",
                    "-p",
                    "no:sugar",
                    "--nomigrations",
                    "--no-cov",
                    "--maxfail=3",
                    test_files,
                ]
            )
            .stdout()
        )

    print(test)


if __name__ == "__main__":
    test_files = sys.argv[1:]
    anyio.run(main, test_files)

restive raven Oct 26, 2023, 1:10 PM

#

With dagger run you have to do it yourself by using something like https://typer.tiangolo.com.

#

In #daggernauts that's done for you.

rocky pelican Oct 26, 2023, 1:11 PM

#

nevermind it's the sys.argv[1:]

#

I need sys.argv[1]

restive raven Oct 26, 2023, 1:12 PM

#

Yeah, you can parse that manually or use a library to make it nicer. That's what Typer does.

#

You also have the built-in argparse if you don't want an additional dependency.

Python documentation

argparse — Parser for command-line options, arguments and sub-comma...

Source code: Lib/argparse.py Tutorial: This page contains the API reference information. For a more gentle introduction to Python command-line parsing, have a look at the argparse tutorial. The arg...

rocky pelican Oct 26, 2023, 1:12 PM

#

Yes the graphql error threw me off

#dagger run python fails when using python directly works