Language Dependencies | Dagger

[wheat zenith]

cc @fathom coral - I got things working with Poetry below, but want to confirm that we need to keep the Hatching build-system for Dagger to function properly. That was my experience so far.

[wheat zenith]

@ocean narwhal looks like we need to add some more details to docs.

The pyproject.toml that is created by default is not ready for poetry (lack of [tool.poetry] section, for example).

You can convert an existing project to use poetry by running poetry init (https://python-poetry.org/docs/basic-usage/#initialising-a-pre-existing-project), but it will complain about the existing [build-system] section because it wants to use its own (https://python-poetry.org/docs/pyproject/#poetry-and-pep-517).

A pyproject.toml file with a defined build-system already exists.

So to keep poetry happy, I just comment out the build-system section:

[project]
name = "main"
version = "0.0.0"
dependencies = []

#[build-system]
#requires = ["hatchling"]
#build-backend = "hatchling.build"

then run poetry init.

Then I remove or comment out the Poetry build-system section and restore the original Hatchling one that Dagger seems to require (I couldn't run dagger call without it in place).

Poetry doesn't seem to mind the Hatchling config and poetry add works just fine for me.

[fathom coral]

No, you don't need hatchling if you're using Poetry. Any build-system is supported, just not custom lock file formats (besides uv).

Only two "lock" formats are supported: a requirements.lock file with pip-compile compatible requirements, or a uv.lock. Here's a an example of a module managed by Poetry: https://github.com/helderco/daggerverse/blob/main/poetry/pyproject.toml

You can add dependencies as normal, under [tool.poetry.dependencies], but will need to poetry export the pinned dependencies to a requirements.lock file, otherwise it'll use the open constrains in the pyproject.toml file instead.

You can use poetry init before dagger init, just make sure the package name is "main". For now, Dagger requires that your module's code is importable via import main.

[ocean narwhal]

Thank you

name = "codegen"
version = "1.0.0"
description = "CLI tool is designed to help developers analyze their code coverage reports and generate tests for their code."
authors = ["Kambui Nurse <[e-mail]>"]
license = 'Proprietary'
readme = 'README.md'
packages = [ { include = "codegen"}]

[tool.poetry.scripts]
codegen = "codegen.cli:agent"

[tool.poetry.dependencies]
python = ">=3.10.12 <3.10.14"
tenacity = "^8.2.3"
langgraph = "^0.0.24"
langchain = "^0.1.7"
beautifulsoup4 = "^4.12.2"
docker = "^6.1.3"
langchain-community = "^0.0.20"
langchain-openai = "^0.0.5"
gitpython = "^3.1.41"
langchain-experimental = "^0.0.50"
langchainhub = "^0.1.14"
pygithub = "^2.2.0"
python-dotenv = "^1.0.1"
pyright = "^1.1.350"
ruff = "^0.2.1"
pytest-asyncio = "^0.23.5"
jsonschema = "^4.21.1"
click = "^8.1.7"
pyyaml = "^6.0.1"


[tool.poetry.group.dev.dependencies]
autopep8 = "2.0.2"
flake8 = "6.0.0"
pytest = "^8.0.2"
pytest-sugar = "0.9.7"
pytest-cov = "4.1.0"
pytest-html = "3.2.0"
pytest-mock = "^3.14.0"
pytest-emoji = "^0.2.0"

[tool.pyright]
useLibraryCodeForTypes = true
exclude = [".cache"]

[tool.coverage.run]
branch = true
source = [ "codegen" ]

[tool.coverage.report]
exclude_lines = ['if TYPE_CHECKING:']
show_missing = true

[tool.pytest.ini_options]
addopts = "--cov --cov-report html:'../../coverage/apps/codegen/html' --cov-report xml:'../../coverage/apps/codegen/coverage.xml' --html='../../reports/apps/codegen/unittests/html/index.html' --junitxml='../../reports/apps/codegen/unittests/junit.xml'"
python_files = ["*_test.py"]
python_classes = ["Test", "*Test", "Describe"]
python_functions = ["*_test.py", "test_"]
testpaths = ["codegen", "tests"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

I can into issues using this

i'll let you guys know

i'll try again

[wheat zenith]

@ocean narwhal according to @fathom coral you need the package name to be “main” instead of “codegen” above ☝️ I think

[ocean narwhal]

ok

um that worked

🙂

muhahahahaha

thanks guys

everything installs

but I get this error

  ╰─▶ Because the current Python version (3.11.9) does not
      satisfy Python>=3.10.12,<3.10.14 and main==1.0.0 depends on
      Python>=3.10.12,<3.10.14, we can conclude that main==1.0.0 cannot be
      used.
      And because only main==1.0.0 is available and you require main, we can
      conclude that the requirements are unsatisfiable.```

when I run dagger functions

I think I can fix

yup I got it

no worries

everything is working

thanks guys

[fathom coral]

Note that you'll want to add the dagger sdk as a part of the dev dependencies, in order to properly get autocompletions in the IDE:

[tool.poetry.group.dev.dependencies]
dagger-io = {path = "sdk", develop = true}

[wheat zenith]

Thanks so much, Helder!

cc @modest plank

[ocean narwhal]

i added this

after you posted the example

[wheat zenith]

I get issues running
poetry init before dagger init

poetry init --name main

...

dagger init --sdk python

[fathom coral]

I think that's poetry's fault. It generates a pyproject.toml referencing a README.md but doesn't generate the readme. Isn't that the issue you're getting?

You can either create the file or remove the readme = "README.md" line .

[wheat zenith]

FileNotFoundError: [Errno 2] No such file or directory: '/src/README.md'

Yep, I was looking for the perfect invocation of poetry init, but I think a template file example will be better to start folks with. Since I couldn't get the dev-dependency in as a one-liner either.

[fathom coral]

They have flags for name, description, license... but no readme 🙂

[wheat zenith]

not for package version either

I like the interactive package search and selection, though. Pretty slick.

[fathom coral]

dagger init --sdk=python
poetry add --group=dev -e ./sdk

[wheat zenith]

right, once you past the poetry init and remove the readme line 😄

For poetry add --group=dev -e ./sdk

I get: Unable to create package with no name

[fathom coral]

Yeah. The reason touch README.md isn't working is because it's failing on the step that generates the requirements.lock file, and the runtime pipeline only copies the pyproject.toml files for that. I've avoided adding all the sources there because of caching, but might be better in case there's other custom stuff referenced that is needed for the build.

[fathom coral]

So you have the sdk dir locally?

[wheat zenith]

Strange it doesn't get it the name from sdk/pyproject.toml:

[project]
name = "dagger-io"

[fathom coral]

Not clear what you mean 🙂

What's your poetry --version?

[wheat zenith]

Poetry (version 1.2.2)

poetry init --name main --no-interaction
awk '!/^readme/' pyproject.toml > pyproject.toml.temp
mv pyproject.toml.temp pyproject.toml
dagger init --sdk=python
poetry add --group=dev -e ./sdk

gives me: Unable to create package with no name error

[fathom coral]

That's from 2 years ago! Latest is 1.8.3.

[wheat zenith]

haha. There you go! classic problems 🙂 Clearly we need a Dagger module for this!

Of course we'd want to support someone's version of poetry within reason, but if we assume a modern version then we could even have dagger init --sdk python --poetry (or similar)

[fathom coral]

Yeah, there's been discussions around offering multiple templates, but it can be done with a module.

[ocean narwhal]

I seem to be fighting with Dagger and Nx

for a folder structure that makes sense for both tech

@wheat zenith can we pair on this tomorrow ?

[wheat zenith]

Sure! @wind jetty have you tried making Nx and Dagger work well inside an Nx monorepo? IIRC you had Dagger pulling in a whole monorepo and running nx commands with Dagger.

[wind jetty]

yes,

I had the conventional Nx structure and the dagger module in a nested dir that was ignored by Nx.

Let me dig up this branch when I get to work.

[ocean narwhal]

oh nice

[wind jetty]

got caught up at work 🤦‍♂️

I found the branch but the code is probably outdated and wont run with the latest Dagger version. But hopefully it helps as a perspective on how this could be done 😄

This was an experiment when I was trying out Dagger for the first time and I see that I commented out the client.Git(...) invocation and just used the base dir of the project to mount the NX files. You could try to invoke the fetchRef function

I take no responsibility for this mess 🙈

https://gist.github.com/busla/d6e68919dd6be95ccff6163ec6ec42c1

[ocean narwhal]

i'll have look thanks

[modest plank]

@wheat zenith @fathom coral this [tool.poetry.group.dev.dependencies] configuration is already in our docs: https://docs.dagger.io/manuals/developer/module-structure#module-layout. Was there something you wanted to add or change?

[fathom coral]

It's on my list for a while to revisit those docs. There's a few changes I want to make to it.

[modest plank]

OK. I suggest waiting until https://github.com/dagger/dagger/pull/8087 is merged though.

[ocean narwhal]

How can I point Dagger to the toml file in a different location?

[fathom coral]

Can you give an example?