Poetry | Dagger | Page 1

tribal ledge Oct 31, 2023, 8:50 PM

#

Here's a minimal poetry config:

[tool.poetry]
name = "main"
version = "0.0.0"
description = "Minimal Python module using Poetry"
authors = ["helderco"]

[tool.poetry.dependencies]
python = "^3.11"

[tool.poetry.group.dev.dependencies]
dagger-io = {path="./sdk", develop=true}

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Notice description and authors is required in Poetry.

restive kindle Oct 31, 2023, 8:51 PM

#

I have both keys, though description is blank

tribal ledge Oct 31, 2023, 8:52 PM

#

readme = "README.md"

Shouldn't be on src/.

restive kindle Oct 31, 2023, 8:54 PM

#

Resolved that, progresses further to: Building wheel for pytest-ci-module (pyproject.toml) did not run successfully.

tribal ledge Oct 31, 2023, 8:56 PM

#

Have you specified anything that changes where poetry finds the packages?

#

Your code, that is.

restive kindle Oct 31, 2023, 8:56 PM

#

Not that I'm aware of, no

tribal ledge Oct 31, 2023, 8:57 PM

#

What does poetry check tell you?

restive kindle Oct 31, 2023, 8:57 PM

#

"All set!"

tribal ledge Oct 31, 2023, 8:57 PM

#

Do you have anything extra besides my minimal example?

#

Have you kept the source in src/main.py?

restive kindle Oct 31, 2023, 8:58 PM

#

Only readme = "src/README.md". Source is in src/main.py yep

tribal ledge Oct 31, 2023, 8:58 PM

#

Why isn't README.md in the root of the module? src/ is meant for python packages.

restive kindle Oct 31, 2023, 9:00 PM

#

Testing it in the root of the module now

#

So my pyproject.toml is, except obvious differences like description, identical to the above

tribal ledge Oct 31, 2023, 9:02 PM

#

This is what I have:

.
├── src
│  └── main.py
├── dagger.json
├── poetry.lock
├── pyproject.toml
└── README.md

restive kindle Oct 31, 2023, 9:02 PM

#

Still getting poetry.core.masonry.utils.module.ModuleOrPackageNotFound: No file/folder found for package pytest-ci-module

tribal ledge Oct 31, 2023, 9:02 PM

#

Where have you set pytest-ci-module?

restive kindle Oct 31, 2023, 9:03 PM

#

In pyproject.toml? No-where

tribal ledge Oct 31, 2023, 9:03 PM

#

Oh, I guess it's you project name. Change it to main.

restive kindle Oct 31, 2023, 9:03 PM

#

tribal ledge This is what I have: ``` . ├── src │ └── main.py ├── dagger.json ├── poetry.loc...

Ohhh

#

I had that same setup

tribal ledge Oct 31, 2023, 9:04 PM

#

If not main, then you have to tell Poetry to look for packages in src/. The default is to collapse based on the project name, so it has to match main.py or main/__init__.py.

restive kindle Oct 31, 2023, 9:04 PM

#

But not "main" in pyproject.toml's name key

tribal ledge Oct 31, 2023, 9:05 PM

#

Supporting other package names is planned but not ready.

restive kindle Oct 31, 2023, 9:05 PM

#

That's done it, thanks!

#

I realise I'm testing this before docs cover these gotchas so thanks for debugging with me 🙂

tribal ledge Oct 31, 2023, 9:06 PM

#

Thank you too! Now I won't forget to highlight keeping "main" as project name 😁

#

If you want to test changing anyway, you need to set packages: https://python-poetry.org/docs/pyproject/#packages

restive kindle Oct 31, 2023, 9:07 PM

#

If it helps, this is going to form the basis of my demonstrating Dagger to a second client who has answered "how do we run the same commands locally as we do in CI" by making the mother of all monstrosity hierarchical Makefiles with many includes, so I'm excited to show this

tribal ledge Oct 31, 2023, 9:08 PM

#

Something like:

packages = [
    { include = "main", from = "src" },]

restive kindle Oct 31, 2023, 9:08 PM

#

So support for different package names is planned and will eventually be documented or default?

tribal ledge Oct 31, 2023, 9:09 PM

#

Eventually be documented yes, but easiest to default to main.

#

Right now, what's in src/ needs to be main because that's imported internally by the SDK (hardcoded).

#

It can be main.py or main/__init__.py (for multiple files).

restive kindle Oct 31, 2023, 9:11 PM

#

tribal ledge Something like: ```toml packages = [ { include = "main", from = "src" },] ``...

This results in: ValueError: /src/src/main does not contain any element

tribal ledge Oct 31, 2023, 9:11 PM

#

What did you change?

#

Looks like it didn't find any functions.

restive kindle Oct 31, 2023, 9:12 PM

#

name back to what it was originally, and added the packages key

#

[tool.poetry]
name = "pytest-ci-module"
version = "0.1.0"
description = "Test"
authors = ["Mjb141"]
packages = [{ include = "main", from = "src" }]

tribal ledge Oct 31, 2023, 9:13 PM

#

Oh. Who's giving the ValueError, can you tell?

#

Is it Poetry or the SDK?

restive kindle Oct 31, 2023, 9:13 PM

#

Looks like Poetry

tribal ledge Oct 31, 2023, 9:13 PM

#

I guess you would need to make it a package to use it like that: main/__init__.py. There's another setting...

restive kindle Oct 31, 2023, 9:14 PM

#

┃       ValueError: /src/src/main does not contain any element                        
┃       [end of output]                            
┃                                                   
┃   note: This error originates from a subprocess, and is likely not a problem with pip.                                                     
┃   ERROR: Failed building wheel for pytest-ci-module                                                                                    
┃ Failed to build pytest-ci-module       
┃ ERROR: Could not build wheels for pytest-ci-module, which is required to install pyproject.toml-based projects

tribal ledge Oct 31, 2023, 9:14 PM

#

Try this:

packages = [{ include = "src/main.py" }]

#

Curious if this works:

packages = [{ include = "main.py", from = "src" }]

restive kindle Oct 31, 2023, 9:16 PM

#

tribal ledge I guess you would need to make it a package to use it like that: `main/__init__....

This works

restive kindle Oct 31, 2023, 9:16 PM

#

tribal ledge Try this: ```toml packages = [{ include = "src/main.py" }] ```

This results in:

✘ exec /runtime ERROR [0.93s]
┃ ╭─ Error                                                         
┃ │ The "main" module could not be found. Did you create a "src/main.py" file in │                                                           
┃ │ the root of your project?

tribal ledge Oct 31, 2023, 9:17 PM

#

Ah, that's probably because it gets installed under src/main.py in site-packages. I suspect the second form should work.

#

packages = [{ include = "main.py", from = "src" }]

restive kindle Oct 31, 2023, 9:18 PM

#

tribal ledge Curious if this works: ```toml packages = [{ include = "main.py", from = "src" }...

Also doesn't work:

┃       ValueError: /src/main.py does not contain any element                                    
┃       [end of output]                                  
┃   note: This error originates from a subprocess, and is likely not a problem with pip.                                                     
┃   ERROR: Failed building wheel for pytest-ci-module                                             
┃ Failed to build pytest-ci-module                  
┃ ERROR: Could not build wheels for pytest-ci-module, which is required to install pyproject.toml-based projects

tribal ledge Oct 31, 2023, 9:18 PM

#

Did you add from = "src"?

restive kindle Oct 31, 2023, 9:18 PM

#

Nope, I just realised

restive kindle Oct 31, 2023, 9:19 PM

#

tribal ledge ```toml packages = [{ include = "main.py", from = "src" }] ```

This works

tribal ledge Oct 31, 2023, 9:19 PM

#

Awesome

restive kindle Oct 31, 2023, 9:19 PM

#

That's the simplest one for sure, I'm going to leave it like that

tribal ledge Oct 31, 2023, 9:20 PM

#

🙂

restive kindle Oct 31, 2023, 9:27 PM

#

Super simple to switch this over to pytest now it's running. Few bits left to do in work tomorrow to apply these to some lambda function pytests and it'll be ready to demonstrate to my team. Glad I got that working 🙌

tribal ledge Oct 31, 2023, 9:28 PM

#

Great!

#

Now I'm wondering about the dev group. That should probably be an optional group and in your host you'd have to say poetry install --with dev.

#

Not sure if that's being handled automatically for dev.

#

Since it's now a group like any other.

restive kindle Oct 31, 2023, 9:32 PM

#

Does it need special consideration? poetry install installs all groups by default

tribal ledge Oct 31, 2023, 9:34 PM

#

Ah, I guess it's ok because you need poetry to install from any group that's not the main one.

#

In dagger it's a pip install, not poetry install, so you should only get the main dependencies there.

restive kindle Oct 31, 2023, 9:35 PM

#

Ah I see

tribal ledge Oct 31, 2023, 9:35 PM

#

A caveat here is that the lockfile is ignored. So you can't pin dependencies yet.

restive kindle Oct 31, 2023, 9:36 PM

#

I presume since you've said 'yet' that's also planned

#

Worth documenting that as well, else it's likely to lead to odd errors people can't diagnose

tribal ledge Oct 31, 2023, 9:37 PM

#

Yes, it's planned. I'm considering using micropipenv for that.

restive kindle Oct 31, 2023, 9:38 PM

#

Oh that looks useful. I've never seen that

#

I've simply added poetry to base images in the past and not really considered that I might not need it. Added that to the "test this at some point" list 🙂

tribal ledge Oct 31, 2023, 10:18 PM

#

https://github.com/helderco/daggerverse/tree/main/poetry

#Poetry