How to extend an existing module?

[placid bramble]

Hey!

I'd like to extend the functionality of an existing module but cannot find a great way of doing this.

I am currently adding the main module as a dependency to other modules, e.g.
Original module:

type ModuleA struct {}
func (m *ModuleA) Foo() string { ... }
func (m *ModuleA) Bar() string { ... }

Extended Module:

type ModuleB struct {}
// Redefined Foo
func (m *ModuleB) Foo() string { ... }
// Reusing Bar
func (m *ModuleB) Bar() string {
    return dag.ModuleA().Bar()
}

This gives me a lot of agency but means that I have to redefine all the functions of ModuleA inside ModuleB to be able to call them 😦

I have tried defining interfaces inside ModuleA so that Foo could accept a module defining PreFoo and PostFoo but then I cannot inject the implementation from outside ModuleA...

Is there a pattern I'm missing or nicer way to extend modules?

Thanks a lot!

[tender niche]

hey @placid bramble ! I'm wondering if Go's embedding could work in this case? How about if your ModuleB embeds dag.ModuleA and then you could also add whatever new methods you want?

[placid bramble]

The struct embedding would be great indeed!

However it did not work when I tried. Doing this:

type ModuleB struct {
    dagger.ModuleA
}

Yields this error:

✔ connect 1.0s
✘ initialize 2.5s
! input: module.withSource.initialize resolve: failed to initialize module: failed to add object to module "module-b": failed to validate type def: object "ModuleB" field "ModuleA" cannot reference external type from dependency module "module-a"
  ✔ resolving module ref 0.1s
  ✘ installing module 2.4s
  ! input: module.withSource.initialize resolve: failed to initialize module: failed to add object to module "module-b": failed to validate type def: object "ModuleB" field "ModuleA" cannot reference external type from dependency module "module-a"
    ✔ ModuleSource.resolveFromCaller: ModuleSource! 0.1s
    ✔ ModuleSource.asModule: Module! 1.7s
    ✘ Module.initialize: Module! 0.6s
    ! failed to initialize module: failed to add object to module "module-b": failed to validate type def: object "ModuleB" field "ModuleA" cannot reference external type from dependency module "module-a"

Error: input: module.withSource.initialize resolve: failed to initialize module: failed to add object to module "module-b": failed to validate type def: object "ModuleB" field "ModuleA" cannot reference external type from dependency module "module-a"

[tender niche]

hmm I see. In that case, I don't think it's currently possible. cc @vague parrot . Mind opening an issue @placid bramble please?

[vague parrot]

We purposely don't allow you to return types from other module dependencies (it exposes your module dependencies to your callers, which unleashes dependency hell versioning problems since the whole transitive DAG becomes visible to every module).

But I do agree this would be very nice to have conceptually if we can avoid that problem.

One thing that's possible today in Go is to use interfaces: https://docs.dagger.io/manuals/developer/interfaces

They are underdocumented at the moment, but the idea is that you can define an interface in your module for the parts of your dependency that you want to expose and then you can return that interface (with the dependency object as the implementation).

That still is overly verbose and boilerplate-y at the moment though. So yes agree an issue would be a good place to discuss and figure out some better patterns here

[tender niche]

nice! you can always use something like https://github.com/vburenin/ifacemaker and automatically generate the interface from the struct? Seems quite convenient in this case.

[placid bramble]

Thanks for detailed answer! That makes a lot of sense 🙂

The interfaces are quite nice, and I'm finding more ways to use them through trial and error.
I think they'll cover my entire use case in the end as I can do some dependency injection now.
Some more documentation around them would do the trick I think 🙂

[idle dagger]

@placid bramble were you able to find an acceptable solution, otherwise I've been fiddling around for many hours and still can't get it... but found this issue.
@vague parrot is there any plan to expand the documentation for this case?

[swift meteor]

Yes, we are going to add docs for interfaces, for sure. I know they're a bit experimental at the moment, but still we need to doco them.
https://github.com/dagger/dagger/issues/9492