Dagger doesn't refresh code generation for updated function signatures in local module dependencies

[lucid radish]

Hi everyone, I’m encountering an issue with Dagger during local development:

I have two modules where one depends on the other (Module-B is installed in Module-A). After updating the function signature in Module-B and running dagger develop, the internal code generation doesn’t reflect the updates. I’ve tried clearing the cache with dagger core engine local-cache prune and deleting internal files, but it still generates the old version of the code.

Has anyone experienced a similar issue or know how to force Dagger to regenerate the updated code?

I opened this github issue: https://github.com/dagger/dagger/issues/9304

I made a video to explain the situation https://www.youtube.com/watch?v=9tAhftPotq8

Thanks in advance!

[jagged vessel]

could you please provide dagger.json for both the modules?

i see dagger.json in video. let me try

I tried it, for me it fails to add dependency from a sibling directory. (which I expected).

would it be possible for you to push "test-deps" dir as a repo somewhere I can clone/try from pls.

[lucid radish]

https://github.com/prefapp/dagger-test-dep

https://github.com/prefapp/dagger-test-dep/blob/8eeb65e5f64d322e502a469f9a9561d3df97b095/module-b/main.go#L25

changing the signture return values to this ->

    return dag.Container().From("alpine:latest").WithExec([]string{"echo", stringArg}), nil
}

[jagged vessel]

Thnx, i will try out and report back in a bit

[lucid radish]

Thank you!

[jagged vessel]

ok... my first observation (and possibly a TIL moment). if you remove the .git directory from dagger-test-dep (basically make them regular dir instead of git repo), the dagger develop command fails inside module-a dir.

@lucid radish when you have chance, could you please confirm that?

[lucid radish]

yes it fails if it is not a git repository, i could notice that too

I consider that normal behavior, given that dagger install uses Git to install module dependencies

[jagged vessel]

Marcos is here, so he can fact check me.

but what do you mean by dagger install uses Git to install module dependencies

[lucid radish]

Originally, the modules were installed by referencing Git repositories. Now that it's possible to install them locally as well, I assume it would still use some form of git mechanism to handle the local installation, but this is just my assumption

[jagged vessel]

gotcha, thank you for that info. My understanding is that local deps are just retrieved using local path. but seems like there is some gap in my understanding here

i am able to reproduce the issue that you reported originally. I will check if i can find more info, but maybe in the meantime some dagger expert can chime in and let us know what is happening here and if that is expected or not somehow.

i will keep you posted.

ah... thawt might be expected

could you try changing moduleB to return *dagger.Service instead of *dagger.Container instead

you would notice that module-a starts complaining about the mismatch in schema.

[lucid radish]

i change the function, ok, it returns a Service and an error

but see the the code gen

However, if i use a primitive type

[jagged vessel]

yeah, I am trying to find in code where it is generated, but this seems like expected so far to me.

[lucid radish]

there

[iron token]

i found this error as well, maybe @round tangle knows what sort of bug may lead to that behaviour

seems like whenever u try to return a (dagger.something, error) from the dependency it just does not work

only on primitive types

[jagged vessel]

this seems like expected and intentional: https://github.com/dagger/dagger/blob/44356172c3443c9fa58b69a689d101affaa0252f/cmd/codegen/generator/go/templates/functions.go#L289

but we should wait for Marcos to confirm

[lucid radish]

if it is a list it generates the code properly

the function in the module-b

[jagged vessel]

yep, if you look at the code I linked, it checks if it is a scalar or a list, and generated code slighly differently

[iron token]

The point is that it adds error return type by default for scalars and lists

for whatever reason for other cases it just ignores them

u can return something like []*.dagger...., error to make it work the way it should

what i dont understand is why is not adding errors by default in dagger types

[jagged vessel]

it could be a tradeoff for implementing something else. I know dagger relies on generating a lot of dynamic code for SDK and so I rely on IDE auto-completion to tell me function signatures based on generated code

rather than looking at dependencies function signatures strictly.

[iron token]

/cc @ember hornet

[ember hornet]

Yeah, this is working as intended - it's a byproduct of how the laziness rules work. Any API that returns an object is lazy on the client-side; the client-side generated code isn't expected to exactly reflect the server-side. The server-side function won't actually be called until a scalar value is requested from its return value, so there's never an error to return from a client-side API that returns an object, even if the server-side implementation says it returns an error.

See also:

• https://github.com/dagger/dagger/issues/3617

[iron token]

by client you mean the sdk and server the graphql server?

[ember hornet]

Yeah sorry, hazy terminology there - by client I meant the client codegen, and by server I meant the backing module implementation, which the client does indeed call through the GraphQL server, but at that layer there's no notion of whether a field resolver can/can't return an error - all resolvers can return an error

[round tangle]

thx Alex for replying here. We need to update this in our docs to clarify that. cc @little meteor (https://docs.dagger.io/api/return-values)

[urban scroll]

Hi, joining the thread here for similar problem.

I have module A exposing locally this fn

func (m *Goland) GitCodeBase(gitGo GitGo) *dagger.Container 

Installing in Module B via dagger install
In the gen code , the public fields of the GitGo structure become private, and any change to it is not reflected and it is not clear what are the steps to keep the code of the module A in sync in module B?

• dagger install in module B
• dagger install in module A then in module B
• How can I deal with an exported custom struct?

[urban scroll]

when changing the exported function signature in module A, a ne dagger develop seems to work.

the problem seems with custom structs but I have not understood how to deal with them from here
https://docs.dagger.io/api/custom-types

[round tangle]

I'm not sure if the SDKs currently support generating function input types for arguments. cc @uneven moat will probably have a better understanding about this 🙏

[uneven moat]

Hey. The implementation code and the client bindings will look different. They're used differently. So I wonder if you have that in mind. As for the custom struct, you'll need to have a function that returns that type.

[round tangle]

i.e: If I have this function:

type One struct{}

type TestIntput struct {
    Name string
}

// Returns a container that echoes whatever string argument is provided
func (m *One) Test(input *TestIntput) *dagger.Container {
    return dag.Container().From("alpine:latest").WithExec([]string{"echo", input.Name})
}

then, the generated code for the caller is:

// Returns a container that echoes whatever string argument is provided
func (r *One) Test(input *OneTestIntput) *Container { // one (../../one/main.go:28:1)
    assertNotNil("input", input)
    q := r.query.Select("test")
    q = q.Arg("input", input)

    return &Container{
        query: q,
    }
}

type OneTestIntput struct { // one (../../one/main.go:23:6)
    query *querybuilder.Selection

    id   *OneTestIntputID
    name *string
}

Where the name is a private field. My understanding is because we haven't implemented function inputs in the SDKs yet. Is that accurate?

[uneven moat]

Yes, I understood. The SDKs know how to generate client bindings for input types (see port forward, or build args for example) but we left it out of modules on purpose because they introduce some headaches. That is, only core has them, you can't create new ones with modules. So for modules you need to have a function to construct any object type. Starting with the main object, which if you don't create a constructor, the SDK will assume a default empty one.

[uneven moat]

So in this case, you need to have some function to create the TestInput instance.

[round tangle]

Yep, that makes sense

[round tangle]

I think we should document this. Cc @little meteor

I haven't seen anything in the arguments section of our docs

[uneven moat]

Yeah, the difference from an input object type and a normal object type is that input objects don't have an ID and can only be used as inputs. But normal objects can be used anywhere, they just need to be constructed by a function in order to pass the object instance's ID around.

With the automatic object to ID conversion that SDKs make, it's a smoother experience compared to raw GraphQL.

I think the main issue with using input object types is the duplication. You'd have to have two object types that are basically the same if you want to return it, not just use as input. We could hide it in the schema with a suffix but we thought it wasn't worth it.

[round tangle]

Makes sense. You'd have to do the same in GraphQL right?

I mean.. you could use interfaces to avoid duplication I guess

Seems like quite some effort though...

[uneven moat]

You can't avoid the duplication with input types, not with GraphQL. In GraphQL they're meant as a way to group several function arguments in one argument. They're not meant to represent an object that you'd return.

[round tangle]

Oh, right.. input type can't implement interfaces 😬

[uneven moat]

See this: https://www.graphql-js.org/mutations-and-input-types/. You have Message and MessageInput.

[round tangle]

Yeah.. takes me back to old Java DAO VO days

[uneven moat]

🙂

[round tangle]

POJO DAO VO 😧

[uneven moat]

If what you're doing is dynamic than it's less noticeable. But if you have a typed language, it's a pita. 🙂

[urban scroll]

thank @round tangle @uneven moat to clarify that.
It is clear the limitation, documenting in the fn argument section with an fn example would be great, because the workaround is just there.
📿

[little meteor]

I'll open a docs issue for this