#Error loading python module in 0.20.5

1 messages · Page 1 of 1 (latest)

trim wharf
#

@ember sage reported:

Where is the best place to ask for help for and upgrade issue?

I have a module that works on v0.20.3 but on v0.20.5 I am getting a cannot import name 'Python' from 'dagger' error for context Python is a module that is developed by our team. It is installed as a toolchain in my local module (python sdk) and used as a type in my local python code? It does not seem to like from dagger import Python, but again it is all good on v0.20.3

#

cc @exotic girder @lilac peak @queen zephyr @oblique nimbus @lethal agate

#

@ember sage we've had some known issues in 0.20.5, we're preparing a 0.20.6 release right now... Want to make sure we confirm it solves your problem too

exotic girder
lilac peak
#

@ember sage If you have a repro it would be super helpful!

trim wharf
#

Yeah I was going to ask the exact command you ran that triggered the error...

#

In particular did you load the module locally, or remotely with eg. dagger -m ....

lilac peak
#

Bonus points if you have a dagger cloud trace

lethal agate
#

It is installed as a toolchain in my local module (python sdk) and used as a type in my local python code?
What's "in my local python code"? Is it you local module written in python? Or is it something else?
If I understand it correctly and you install the toolchain in your local module, it shouldn't be accessible inside this module. In that case you would need to declare it as a dependency instead.
(I can be wrong, but I saw a similar issue some time ago, but it was in go)

ember sage
#

If needed I can create a public project tomorrow but here is what I did as best as I can remember:

# create temp folder, cd into it
# while using v0.20.3 CLI
dagger init --sdk python --source .dagger
dagger toolchain install github.com/act3-ai/dagger/python@python/v0.11.1
# add check function to my main.py (shown below)
dagger check -l
dagger check 
# all good

# stop engine container and switch to v0.20.5 cli
$ dagger check -l
▶ ✔ connect 2.4s
✘ fetch check information 48.6s ERROR
! loading modules: loading module "/home/tgilkerson/temp/dagregress": resolving module source "/home/tgilkerson/temp/dagregress": failed to call module "dagregress" to get functions: ModuleLoadError: cannot
  import name 'Python' from 'dagger' (/src/xxh3:e36bc9511bc8ee53/.dagger/sdk/src/dagger/__init__.py) [traceparent:00000000000000000000000000000000-0000000000000000] [traceparent:00000000000000000000000000000000-
  0000000000000000]
Error: loading modules: loading module "/home/tgilkerson/temp/dagregress": resolving module source "/home/tgilkerson/temp/dagregress": failed to call module "dagregress" to get functions: ModuleLoadError: cannot import name 'Python' from 'dagger' (/src/xxh3:e36bc9511bc8ee53/.dagger/sdk/src/dagger/__init__.py) [traceparent:00000000000000000000000000000000-0000000000000000] [traceparent:00000000000000000000000000000000-0000000000000000]
#

main.py shown below:

from typing import Annotated

import dagger
from dagger import (
    DefaultPath,
    Ignore,
    Python,
    check,
    dag,
    function,
    object_type,
)

SrcDir = Annotated[
    dagger.Directory,
    DefaultPath("/"),
    Ignore(
        [
            ".venv",
            "**/__pycache__",
            ".dagger",
            ".mypy_cache",
            ".pytest_cache",
            ".ruff_cache",
        ]
    ),
]

@object_type
class Dagregress:

    async def get_py(
        self,
        src: SrcDir,
    ) -> Python:
        """Test"""
        return dag.python(src)


    @check
    @function
    async def check_py(self, src: SrcDir) -> dagger.Container:
        """Check test"""
        src = src.filter(gitignore=True)
        py = await self.get_py(src)
        return py.project()
#

dagger.json shown below

{
  "name": "dagregress",
  "engineVersion": "v0.20.3",
  "sdk": {
    "source": "python"
  },
  "toolchains": [
    {
      "name": "python",
      "source": "github.com/act3-ai/dagger/python@python/v0.11.1"
    }
  ],
  "source": ".dagger"
}
#

sorry for the multiple messages but all in one was too big.

ember sage
lethal agate
#

Ok, I think I see (I'm trying to repro on my side)
dag.python(src) shouldn't work as you installed the module as a toolchain, not as a dependency.
If you changed toolchains by dependencies in your dagger.json it should work

ember sage
#

are you saying it shouldn't work in v0.20.5 but did in v0.20.3?

lethal agate
#

My understanding is the fact is was working before was a bug and as we are working on modules a few things have changed, and we fixed this at the same time (maybe without noticing it)

#

It should not have work in v0.20.3

ember sage
#

I will change toolchains to dependencies and see if that does fix it

#

btw it does work on v0.20.3 so i guess that is a bug

#

ok I can confirm that if I change toolchains to dependencies and retest on v0.20.5 it now works

trim wharf
ember sage
#

I need to think about what that means to our users. I think it means that they need to use dependencies instead but that will force them to have to wrap all our functions and not be able to use the CLI to run our toolchain functions, right?

trim wharf
#

FYI in 0.21 we're taking this distinction even further, with a first-class concept of workspace, with its own human-friendly config file. "toolchains" will move there, and out of dagger.json.

(Note this will be backwards compatible and there will be a dagger migrate command to smoothly create a workspace config file from your existing dagger.json)

If you're curious about that: https://github.com/dagger/dagger/pull/11812

GitHub
Workspaces by shykes · Pull Request #11812 · dagger/dagger

Introduce workspaces: project-level configuration separate from module definitions.
Dagger currently conflates project configuration and module definitions in a single dagger.json. This PR separate...

ember sage
#

OK I will take this back to my team, we have been looking forward to the workspace stuff, this is just growing pains. You all have been a great help this is a great project... keep up the good work!

trim wharf
#

Appreciate it. Yeah Workspaces are 100% what you want to expose to your users... If you're considering rolling out some sort of change to your end users, I would definitely make sure to do that on top of workspaces. The result should be better (that's the goal anyway) and it will avoid having to roll out 2 distinct change. We all know how users LOVE change.

ember sage
#

oh before I go, I just want to let you know.... this is forcing us to not upgrade our engine on our runner untill all the project/users have upgraded... that is the pain. do you see any options for me

lethal agate
trim wharf
#

Yes in fact when we designed toolchains, we intended it that way - so it's explicitly supported to have the same module as a dependency and toolchain

ember sage
#

I really appreciate the suggestions but I am trying to figure out how I can upgrade the engine without forcing my users to make any changes right now because they are in the middle of their sprint and have some deadlines so I just want to allow them to remain pined to v0.20.3 and upgrade when they are ready....

trim wharf
#

Ah I see. The constraint is not making changes to the module (including to its dagger.json)

trim wharf
#

@exotic girder @lethal agate do you know how much work it would be to re-enable the 0.20.3 behavior of codegen on dependencies? Might not be an option at all with all the other structural changes ongoing. But, just in case it turns out to be really easy... Worth asking 🙂

lethal agate
lethal agate
ember sage
#

Thanks, I will track that PR to see what happens.

lethal agate
#

@ember sage Just to share with you that the previous behavior has been reintroduced in main. So you should be able to see the toolchain module as a dependency. This has not yet been released.