private dagger module 1 -> private dagger module 2 + private go modules

[cerulean dagger]

I seem to be having an issue resolving private go packages, but only when I chain one dagger module to another, and only when running in GitHub Actions for some reason I can't quite yet figure out..

Setup:

• GitHub Actions -> private dag module #1 ("entrypoint")
• private dag mod #1 -> private dag mod #2 ("do-stuff")
• both private dagger modules have private go module dependencies
• goprivate is set in the dagger.json for both dagger modules to "github.com/my-org/*"
• first module codegen is fine
• second module codegen seems to no longer have access to the ssh key / ssh agent socket needed to complete its go mod tidy step

I've tried putting all the dagger modules into a monorepo and installing the "do-stuff" module as a dependency of the "entrypoint" module, but same results as when the two dagger modules are in different repos.

I've tested it w/out GitHub Actions, and it seems to work fine locally.

If the SSH_AUTH_SOCK wasn't being forwarded at all, even the first module would not build, as it also has private go module dependencies. (a different one, in case that matters)

For now, the ssh key I'm using is the same locally and in github actions, and it has access to all the repos (private dagger modules and private go modules) needed.

Is there something special that happens when I chain the modules like that, and run it inside of GitHub Actions Runners that I'm missing / something I need to tweak for this case?

Many thanks if anyone can take a moment and just point me in a sane direction here. (or an insane one, as long as I can move forward, that's fine too)

Not sure if matters, but my monorepo does not have a dagger.json at the root dir - its just a normal git repo and the current only subdirs are the entrypoint and do-stuff modules.

Should I maybe try a different approach? Right now I'm using dagger install and then trying to call the one module directly from the Dagger API, which, again, works fine locally.. but, maybe I need to be fancier and instantiate the module from a directory, cloned via git and do all that manually inside the entrypoint?

Not sure, but figured I'd ask the wise folks here if you have any pointers ("you're doing it wrong" is a valid pointer IMO) before hacking myself into a deeper hole.

[cerulean dagger]

well, for now, I'll just cram all the code together in one place and ditch calling across the Dagger API. I had wanted to have a single entrypoint repo that just took the event trigger as some JSON, and then looked at that and dispatched the appropriate secondary module's function call to handle the event, but, for now, it's fine to just copy/paste the entrypoint function everywhere. That seems to work fine.

I'll keep digging later when I have some more time - I'm almost positive this is a n00b mistake and I'm missing something obvious.

[vocal pilot]

Hey, thanks for the thread. Will take a look later today or tomorrow if it's ok with you 🙏 cc @vocal pilot (self-pinging to have the notification)

[cerulean dagger]

no rush, I'm just copying the entrypoint file into the submodules for the time being and its fine. Would be nicer to have that part in its own little repo, but the Dagger code is just getting started, so it is very manageable for the moment.

Whenever you have a moment, just would be nice to have a pointer saying "there's this one weird trick" or "this is expected to work, something's up w/ your setup". (I suspect the latter as I'm very new here still..)

[vocal pilot]

Hey @cerulean dagger,

I'm not sure to have understood everything. Let me reframe what I understand, just to make sure 😇

It seems that you have:

• Module #1 ("entrypoint") - has private Go deps → codegen works ✅
• Module #2 ("do-stuff") - has (the same) private Go deps, installed as dependency of module #1
• When module #1 calls module #2 via the Dagger API → module #2's codegen fails ❌
• Works "locally" (meaning that you can call them separately and they both work?)

My suspicion: The SSH socket isn't being forwarded to the codegen step when loading a chained module dependency. Module #1 gets the socket, but when it triggers loading module #2, that codegen runs without SSH access ?

I though we had a test for that, but it seems that: https://github.com/dagger/dagger/blob/main/core/integration/module_test.go#L7080 only tests a module with private deps in direct, not though a dependency edge case

So there's a big chance it's a bug, could you please file an issue ? I started working on the private deps and i still don't have a clear opinion at the moment on what to expect for those use-cases, what would you expect ?

[cerulean dagger]

Thanks for the follow up here @vocal pilot ! I got sucked into some other deliverables for the last few days - but will definitely be getting back to this in the New Year.

[worn raven]

The SSH socket isn't being forwarded to the codegen step when loading a chained module dependency. Module #1 gets the socket, but when it triggers loading module #2, that codegen runs without SSH access ?
This is true, the SSH socket is not forwarded. Our current workaround is to run Dagger in Dagger and invoke the cli in that container, which we have passed the socket to (not shown). I did find another way using the current module dependencies (example 3), but I haven't implemented it in our repo.

For example, "Test" is a module with another module nested in it's source private-ssh-dep - this mod contains a dependency in a private repo. Example 1 & 2 fail regardless of private-ssh-dep being installed as a dependency. Example 3 requires it to be installed as one.

func New(
    Src *dagger.Directory,
) *Test {
    return &Test{
        Src: Src,
    }
}

type Test struct {
    Src *dagger.Directory
}

// dagger call --src . codegen
// FAIL
func (m *Test) Codegen(ctx context.Context) *dagger.Directory {
    return m.Src.AsModuleSource(dagger.DirectoryAsModuleSourceOpts{SourceRootPath: "private-ssh-dep"}).GeneratedContextDirectory()
}

// dagger call --src ./private-ssh-dep codegen-2
// FAIL
func (m *Test) Codegen2(ctx context.Context) *dagger.Directory {
    return m.Src.AsModuleSource().GeneratedContextDirectory()
}

// dagger call --src . codegen-3
// SUCCESS
//
// Requires that the "private-ssh-dep" module is installed as a dependency.
func (m *Test) Codegen3(ctx context.Context) *dagger.Directory {
    deps, err := dag.CurrentModule().Dependencies(ctx)
    if err != nil {
        panic(err)
    }

    if len(deps) == 0 {
        panic("no dependencies found")
    }

    return deps[0].GeneratedContextDirectory()
}

[vocal pilot]

Hi, it's still part of my todo 😇