dagger.json issues | Dagger | Page 1

untold vortex Feb 12, 2024, 10:26 PM

#

What happens when you try that command?

mortal fjord Feb 12, 2024, 10:28 PM

#

oops sorry i pasted in the main chat

#

let me move it

untold vortex Feb 12, 2024, 10:28 PM

#

@mortal fjord seeing this error:

Error: load package ".": package name is empty

Do these subdirectories exist? daggerverse/java, daggerverse/docker, daggerverse/oauth?

mortal fjord Feb 12, 2024, 10:28 PM

#

Error: failed to generate code: input: resolve: moduleSource: withName: withSDK: withSourceSubpath: resolveFromCaller: asModule: failed to create module: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: process "/usr/local/bin/codegen --module-context /src --module-name sonar --propagate-logs=true --introspection-json-path /schema.json" did not complete successfully: exit code: 1

Stderr:
Error: load package ".": package name is empty

untold vortex Feb 12, 2024, 10:28 PM

#

Are you able to paste the output of find . or equivalent?

mortal fjord Feb 12, 2024, 10:29 PM

#

untold vortex <@163822683799158784> seeing this error: > Error: load package ".": package nam...

These do exist. They all work too when I dagger call them

untold vortex Feb 12, 2024, 10:31 PM

#

can you run dagger functions from the root module?

mortal fjord Feb 12, 2024, 10:31 PM

#

This is find . from daggerverse dir

.
./docker
./docker/go.mod
./docker/querybuilder
./docker/querybuilder/marshal.go
./docker/querybuilder/querybuilder.go
./docker/dagger.gen.go
./docker/go.sum
./docker/.gitignore
./docker/dagger.json
./docker/.gitattributes
./docker/main.go
./oauth
./oauth/go.mod
./oauth/querybuilder
./oauth/querybuilder/marshal.go
./oauth/querybuilder/querybuilder.go
./oauth/dagger.gen.go
./oauth/go.sum
./oauth/.gitignore
./oauth/dagger.json
./oauth/.gitattributes
./oauth/main.go
./java
./java/go.mod
./java/querybuilder
./java/querybuilder/marshal.go
./java/querybuilder/querybuilder.go
./java/dagger.gen.go
./java/resources
./java/resources/settings.xml
./java/go.sum
./java/.gitignore
./java/dagger.json
./java/.gitattributes
./java/main.go
./sonar
./.gitattributes

untold vortex Feb 12, 2024, 10:31 PM

#

That will tell us if the module is currently in a good state

#

OK, so at the moment there is no root module (no dagger.json at the root)

mortal fjord Feb 12, 2024, 10:31 PM

#

untold vortex That will tell us if the module is currently in a good state

let me run that. I renamed that dagger.json to dagger.json.back just to see if removing it helps

untold vortex Feb 12, 2024, 10:31 PM

#

ah I see

mortal fjord Feb 12, 2024, 10:32 PM

#

untold vortex OK, so at the moment there is no root module (no dagger.json at the root)

There is, I am unable to find . that one as that's a fairly big Go project

untold vortex Feb 12, 2024, 10:32 PM

#

ok 🙂

mortal fjord Feb 12, 2024, 10:33 PM

#

output of dagger functions from root (after setting back dagger.json)

Error: input: resolve: moduleSource: resolveFromCaller: asModule: initialize: module name and SDK must be set

untold vortex Feb 12, 2024, 10:34 PM

#

What's the output of dagger version?

mortal fjord Feb 12, 2024, 10:34 PM

#

dagger v0.9.9 (registry.dagger.io/engine) darwin/arm64

untold vortex Feb 12, 2024, 10:35 PM

#

are you able to share the contents of that dagger.json?

mortal fjord Feb 12, 2024, 10:35 PM

#

I did, let me repaste, I am redacting the name of the module but here it is

{
  "name": "mymodule",
  "dependencies": [
    {
      "name": "docker",
      "source": "daggerverse/docker"
    },
    {
      "name": "java",
      "source": "daggerverse/java"
    },
    {
      "name": "oauth",
      "source": "daggerverse/oauth"
    }
  ],
  "engineVersion": "v0.9.9"
}

#

here's the contents of daggervers/docker/dagger.json

{
  "name": "docker",
  "sdk": "go",
  "source": ".",
  "engineVersion": "v0.9.9"
}

#

hmm, if I create a new folder in my home directory called daggertest/sonar and run dagger init --sdk=go --name=sonar it works.

untold vortex Feb 12, 2024, 10:41 PM

#

mortal fjord I did, let me repaste, I am redacting the name of the module but here it is ```...

thanks, and sorry for the repaste

#

total brainfart on my side

mortal fjord Feb 12, 2024, 10:42 PM

#

no worries!

untold vortex Feb 12, 2024, 10:43 PM

#

So, it actually makes sense that dagger functions doesn't return anything useful, since your root module doesn't have any functions of its own. Obviously we need to add more polish, that error is too rough, but it's probably not what's blocking you.

#

@mortal fjord what are you trying to do where you are stuck?

mortal fjord Feb 12, 2024, 10:44 PM

#

right

untold vortex Feb 12, 2024, 10:44 PM

#

For example this should work: dagger -m ./daggerverse/docker functions

#

And ideally, this should also work since you installed it as a dependency: dagger -m docker functions. Although @strong summit has been having issues getting that to work

mortal fjord Feb 12, 2024, 10:45 PM

#

So basically I am trying to build my own daggerverse. It was working fine pre-0.9.9. What I tried to do with the root is make it a module so I can call them via dagger -m docker ... etc.

untold vortex Feb 12, 2024, 10:46 PM

#

Ah I see. Well to make your own daggerverse, you don't need a root dagger.json at all. As long as your individual modules load correctly, you shouldn't need to change anything.

mortal fjord Feb 12, 2024, 10:46 PM

#

I don't care too much about the root. I can live without it. But the biggest problem now is dagger engine has gotten into some bad state. i can't even initialize individual modules within my daggerverse folder

untold vortex Feb 12, 2024, 10:46 PM

#

My own daggerverse repo hasn't changed in that regard in months. The only thing I need to fix at the moment, is how my individual modules handle optional arguments in Go. Other than that, no breaking change.

untold vortex Feb 12, 2024, 10:47 PM

#

mortal fjord I don't care too much about the root. I can live without it. But the biggest pro...

Ah, I see.

mortal fjord Feb 12, 2024, 10:47 PM

#

my individual modules had root specified. So when 0.9.9 came along I went into each module and did a dagger develop which restructured the dagger.json and removed root

untold vortex Feb 12, 2024, 10:48 PM

#

mortal fjord my individual modules had `root` specified. So when 0.9.9 came along I went into...

Ok so far so good. Did you need that root to access shared files in the parent repo?

mortal fjord Feb 12, 2024, 10:49 PM

#

That was the idea yes. but based on recent changes to dagger I resorted to passing whatever needed via CLI args

untold vortex Feb 12, 2024, 10:49 PM

#

I tried this in my daggerverse:

$ dagger version
dagger v0.9.9 (registry.dagger.io/engine) darwin/arm64
$ dagger init --sdk go ./sonar
WARNING: no LICENSE file found; generating one for you, feel free to change or remove license="Apache-2.0"

Initialized module sonar in ./sonar
$ cat dagger.json
cat: dagger.json: No such file or directory
$ cat sonar/dagger.json
{
  "name": "sonar",
  "sdk": "go",
  "source": "dagger",
  "engineVersion": "v0.9.9"
}
$

mortal fjord Feb 12, 2024, 10:50 PM

#

yes I see the same if I dagger init outside of this project..

untold vortex Feb 12, 2024, 10:50 PM

#

(Don't love the default dagger/ in that context, but that's a separate design conversation with its own tradeoffs...)

mortal fjord Feb 12, 2024, 10:50 PM

#

yeah same

#

my default is .

#

 cat daggerverse/java/dagger.json
{
  "name": "java",
  "sdk": "go",
  "dependencies": [
    {
      "name": "docker",
      "source": "../docker"
    }
  ],
  "source": ".",
  "engineVersion": "v0.9.9"
}

visual perch Feb 12, 2024, 11:18 PM

#

mortal fjord yeah same

Hello Nipuna, jumping in also. Did you manage to repro on the side apart your current daggerverse ?

From what I understand, you have a root module as such:

{
  "name": "mymodule",
  "dependencies": [
    {
      "name": "docker",
      "source": "daggerverse/docker"
    },
    {
      "name": "java",
      "source": "daggerverse/java"
    },
    {
      "name": "oauth",
      "source": "daggerverse/oauth"
    }
  ],
  "engineVersion": "v0.9.9"
}

So a tree like this:

daggerverse
 /docker
 /java
 /oauth

Now, each of these modules work with 0.9.9 when calling them from the root, and each of them have the engineVersion set as v0.9.9

dagger call -m ./docker
dagger call -m ./oauth
dagger call -m ./java

Now, the docker have such dagger.json:

{
  "name": "docker",
  "sdk": "go",
  "source": ".",
  "engineVersion": "v0.9.9"
}

The java has:

{
  "name": "java",
  "sdk": "go",
  "dependencies": [
    {
      "name": "docker",
      "source": "../docker"
    }
  ],
  "source": ".",
  "engineVersion": "v0.9.9"
}
``` and references the docker

What about the oauth, is it referencing any of these modules ?

To summarize, whenever you do a `dagger init newmodule`, in that configuration it breaks ?

mortal fjord Feb 12, 2024, 11:21 PM

#

visual perch Hello Nipuna, jumping in also. Did you manage to repro on the side apart your cu...

Thanks for jumping in! And yes I think you summarized it pretty well. I haven't tried exactly to replicate yet outside of this structure yet. That's my next goal. I am going to copy the individual modules + code into a brand new folder and see what happens

#

the oauth module does not have deps. but there's another one in there that depends on another mod

visual perch Feb 12, 2024, 11:23 PM

#

so oauth is like this:

{
  "name": "oauth",
  "sdk": "go",
  "source": ".",
  "engineVersion": "v0.9.9"
}

mortal fjord Feb 12, 2024, 11:24 PM

#

yes exactly

#

Heres the other module with deps

{
  "name": "(redacted)",
  "sdk": "go",
  "dependencies": [
    {
      "name": "docker",
      "source": "../docker"
    },
    {
      "name": "java",
      "source": "../java"
    }
  ],
  "source": ".",
  "engineVersion": "v0.9.9"
}

#

It all looks very vanilla. Hence why i'm stumped this is not working

visual perch Feb 12, 2024, 11:25 PM

#

hmm, so two modules are depended on docker, one with some depth. Ok, trying a repro from scratch on the side 👀

mortal fjord Feb 12, 2024, 11:31 PM

#

hmm, i cannot repro it in a new folder.. so strange

#

same structure

#

same modules

visual perch Feb 12, 2024, 11:33 PM

#

looks a lot like a state issue then. Did you try to kill the engine, remove its cache ?

mortal fjord Feb 12, 2024, 11:34 PM

#

by removing cache you mean remove the local volume? Is there anything else I need to clear?

visual perch Feb 12, 2024, 11:35 PM

#

mortal fjord by removing cache you mean remove the local volume? Is there anything else I nee...

it shall be enough ✅

mortal fjord Feb 12, 2024, 11:41 PM

#

tried that, didn't make a difference 😦

#

I wonder if my existing project being a go project has something to do with it. That's the only difference between my repro attempt and original

visual perch Feb 12, 2024, 11:42 PM

#

mortal fjord tried that, didn't make a difference 😦

Actually it's better like this, it shall be reproducable 👼

mortal fjord Feb 12, 2024, 11:51 PM

#

interestingly dagger init --sdk=python --name=newmodule worked.. So something to do with Go.

visual perch Feb 12, 2024, 11:52 PM

#

mortal fjord interestingly `dagger init --sdk=python --name=newmodule` worked.. So something ...

and its not a conflict of the name of the module and the top level package name in go ?
dagger init --sdk=go --name=newmodule fails

mortal fjord Feb 12, 2024, 11:52 PM

#

I get the error regardless of the name of the module

visual perch Feb 13, 2024, 12:05 AM

#

mortal fjord I get the error regardless of the name of the module

what command did you use to make docker a deps to java ?

#

In the codebase, the issue is as such:

case 1:
        if pkgs[0].Name == "" {
            // this happens when loading an empty dir within an existing Go module
            return nil, nil, fmt.Errorf("package name is empty")
        }
        return pkgs[0], fset, nil

mortal fjord Feb 13, 2024, 12:43 AM

#

I think dagger install. I stepped out for a bit will check when I get back. Really appreciate you looking into this!

verbal marlin Feb 13, 2024, 9:50 AM

#

visual perch In the codebase, the issue is as such: ``` case 1: if pkgs[0].Name == ""...

link for future reference:
https://github.com/dagger/dagger/blob/88a813f01b578c876ca1e5ee7c06457127a49fb6/cmd/codegen/generator/go/generator.go#L101-L104

verbal marlin Feb 13, 2024, 9:52 AM

#

mortal fjord This is `find .` from `daggerverse` dir ``` . ./docker ./docker/go.mod ./docke...

so this is weird, this implies that the parent directory somewhere has a go.mod
but the find you've done doesn't show that

#

this looks specific to the Go SDK, as opposed to all the SDKs - what's weird to me is why this ever worked, this specific code path is quite old (like when we started zenith)

mortal fjord Feb 13, 2024, 1:16 PM

#

Sorry I am back to testing this. @verbal marlin I didn't get what you meant. My root dir is an existing Go project are you saying that's a conflict somehow?

verbal marlin Feb 13, 2024, 1:32 PM

#

yeah it looks like it - thankfully, i managed to reproduce this!

#

i'll open an issue

#

https://github.com/dagger/dagger/issues/6659

GitHub

Error running `dagger init`: `package name is empty` · Issue #6659 ...

To reproduce this, try initializing a dagger module in a directory that already contains a go.mod. For example: $ git clone https://github.com/dagger/dagger.git $ dagger init --sdk=go ✘ ModuleSourc...

#

i left a suggested workaround, i'll try and come back and address this later - i'm a bit caught up today with an internal hackathon ❤️

mortal fjord Feb 13, 2024, 1:45 PM

#

awesome! I can do the workaround for now as it only fails on new module creation. I won't have to do that many times.

#

Yes! That worked. Seems like folks developing CI modules for Go apps will run into this fairly often.

verbal marlin Feb 13, 2024, 5:37 PM

#

wow lol, it really is ridiculously easy to hit this everywhere isn't it 😄
can't believe it was such a pain to find it 😛

mortal fjord Feb 13, 2024, 6:20 PM

#

haha. I was the first unlucky soul to hit it I guess. Glad you figured it out though!

untold vortex Feb 13, 2024, 6:20 PM

#

Thank you for tracking this down guys, and for your patience Nipuna

verbal marlin Feb 14, 2024, 4:18 PM

#

@mortal fjord I should have a fix for you in https://github.com/dagger/dagger/pull/6678

GitHub

fix: add generated go modules to go.work by jedevc · Pull Request #...

Should fix #6659 (cc @nipuna-perera).
This is a pretty common use case - we have a go.work at the root of a repo, and we want to create a dagger module for it with dagger init.
However, naively, th...

#

If you're comfortable testing against a dev engine, I'm curious if this resolves your issue

#

I think the issue is with parent go.works causing issues, but potentially there could be multiple things here

#

(also bleh, those couple of lines with the actual fix, if you average them out, i guess it's a nice average code-writing speed of 2 lines an hour 😢)

mortal fjord Feb 14, 2024, 4:48 PM

#

Woot!! I haven't done a lot of testing with the dev engine. I know at some point there was a engine module. Is there an updated document on how to test with the dev engine?

verbal marlin Feb 14, 2024, 4:49 PM

#

this is generally how I do it: https://docs.dagger.io/204441/contributing/#how-to-run-a-development-engine

Contributing to Dagger | Dagger

mortal fjord Feb 14, 2024, 5:02 PM

#

easy enough, let me try that

#

bleh, didn't work. ./hack/dev failed because the internal Go image does not have my company CA certs. So it fails to go mod download 😦

mortal fjord Feb 15, 2024, 4:21 PM

#

I've been trying to use the workaround for this but I can't do much. When I rename the go.mod and go.work files, my autocompletion breaks. dagger develop seems to work but then it actually generates the code for the example module and not the one I have coded. When I name go.mod and go.work back, dagger develop fails with the same error I saw originally. So I am effectively stuck

verbal marlin Feb 15, 2024, 4:30 PM

#

hm, can you make sure that the go.work includes the go.mod defined by your dagger module

#

this is essentially what my fix does - makes sure it's included if you're using a go.work

mortal fjord Feb 15, 2024, 4:31 PM

#

yeah my go work has the new module. Let me double check

verbal marlin Feb 15, 2024, 4:32 PM

#

hmmm if your go.work has the dagger module included, and you're still getting the issue, then there's something else going on

mortal fjord Feb 15, 2024, 4:32 PM

#

go 1.21.3

toolchain go1.21.5

use (
        .
        ./daggerverse/docker
        ./daggerverse/java
        ./daggerverse/lwc
        ./daggerverse/oauth
        ./daggerverse/sonar
)

./daggerverse/sonar is the module I am trying to build

verbal marlin Feb 15, 2024, 4:33 PM

#

and dagger develop in the sonar directory gives the package name is empty error?

mortal fjord Feb 15, 2024, 4:34 PM

#

yes.. just checked again

#

what's weird is, it doesn't error out in my other modules

verbal marlin Feb 15, 2024, 4:35 PM

#

hm, what's ls -lha ./daggerverse/sonar give?

mortal fjord Feb 15, 2024, 4:36 PM

#

I cropped where it shows my user

verbal marlin Feb 15, 2024, 4:37 PM

#

are there significant differences between that and the other dirs?

mortal fjord Feb 15, 2024, 4:39 PM

#

hmm.. other modules have a .gitignore but not the dagger folder. I thought i moved everything out of the dagger folder the other day

#

so that dagger folder also has a go.mod. I wonder if that's causing the issue

#

removing the dagger folder didn't change anything FWIW

verbal marlin Feb 15, 2024, 4:47 PM

#

oh, did you also update the source value in the dagger.json for that module

#

if you removed the dagger folder

mortal fjord Feb 15, 2024, 4:47 PM

#

oh I did not

#

let me update it

#

ok changing the source fixed it.

#

That error message could be better

verbal marlin Feb 15, 2024, 4:49 PM

#

yeah, i'm not actually sure how you got it though

mortal fjord Feb 15, 2024, 4:49 PM

#

So I guess the issue was, the dagger folder was not in go.work

verbal marlin Feb 15, 2024, 4:49 PM

#

ohhhh

#

yes

mortal fjord Feb 15, 2024, 4:50 PM

#

it was the top level ./daggerverse/sonar folder that was in go.work

#dagger.json issues