confusing secret error with cross repo module invocation | Dagger | Page 1

modest solstice Feb 3, 2025, 9:58 PM

#

so I am able to reproduce this with two completeliy new and bare modules within my existing repos.

A. github.com/myrepo/a
has file in ./certs/test.crt (plaintext file with some dummy text. not an actual cert)

package main

import (
    "dagger/testmod/internal/dagger"
)

type Testmod struct{}

func New() *Testmod {
    return &Testmod{}
}

// Returns a container that echoes whatever string argument is provided
func (m *Testmod) ContainerEcho(
    //+defaultPath=./certs/test.crt
    cert *dagger.File,
) *dagger.Container {
    return dag.Container().
        From("alpine:3.12").
        WithFile("/src/cert.crt", cert).
        WithWorkdir("/src").
        WithExec([]string{"cat", "/src/cert.crt"})
}

#

B. github.com/myrepo/b
dagger install github.com/myrepo/a@pull/1/merge

package main

import (
    "dagger/tests/internal/dagger"
)

type Tests struct{}

// Returns a container that echoes whatever string argument is provided
func (m *Tests) ContainerEcho() *dagger.Container {
    return dag.Testmod().ContainerEcho()
}

#

CC @empty coral (since I didn't reply to your message with the thread)

#

In this case they are both v0.15.3

#

There is no secret involved here at all. So the error is super confusing

failed to add client resources from ID: failed to add secret from source client 4sfuyti2ov0duk9p8xlz90b4d: secret xxh3:a1b22fd50fdb4b95 not found in other store

#

Also notice, I am using a PR pull/1/merge. I haven't tested otherwise, but I wonder if there's a relationship

#

confusing secret error with cross repo module invocation

craggy ingot Feb 5, 2025, 6:06 PM

#

fyi @tardy path, since we both dug into this kinda thing before

#

i'll dig a bit tomorrow, but just wanted to flag in case you wanted to poke around first (was going to today, and ran out of cycles)

#

(also fyi @modest path, i know we changed the purity of some functions, mayyy have had some unexpected impact)

craggy ingot Feb 6, 2025, 10:30 AM

#

okay, so i can't repro with this

#

is there anything special about your setup at all?

#

(i'm just running this in a fresh engine)

#

i'm doing what you're doing, and running dagger call container-echo

#

🤔 by any chance are these repos private and using PAT authentication

#

If so, I wonder if the way the default path is loaded from context is using the secret which we then try and transfer - but for some reason we're not actually adding it into the session for that client

modest solstice Feb 6, 2025, 3:06 PM

#

craggy ingot 🤔 by any chance are these repos private and using PAT authentication

Yes, and that could be it!

craggy ingot Feb 6, 2025, 5:17 PM

#

okay okay 🙂 so with my shot in the dark, the theoretical fix would be to add the secret from those kind of loaded files to the client session secret store

#

(note to self/whoever picks up a fix for this)

#

summarized this into an issue: https://github.com/dagger/dagger/issues/9524

GitHub

Cannot use context directories from git alongside PAT auth · Issue ...

Based on this thread with @nipuna-perera in discord: #1336068640168218756 message Module setup: // testmod package main import ( "dag...

modest solstice Feb 6, 2025, 8:13 PM

#

Hmm, can you think of a way to work around it?

craggy ingot Feb 7, 2025, 1:55 PM

#

modest solstice Hmm, can you think of a way to work around it?

thinking about it, not realllllly

#

i mean, it depends on what you're returning

#

i think as long as the thing you return is in anyway related to the defaultPath loaded input, you're gonna have this problem

#

but, say, if you have a File, you could load the contents of it, create a new file, and return that instead

#

since that's not actually linked at the dagger api level (specifically, the ID of the new File doesn't connect to the secret)

#

anyways, there's a fix here: https://github.com/dagger/dagger/pull/9530
we're planning a release for early next week, so hopefully should be out in that

modest solstice Feb 7, 2025, 3:39 PM

#

woot!! thank you! Is there a way for me to test this locally?

craggy ingot Feb 7, 2025, 3:52 PM

#

you can do a local engine build (with ./hack/dev, or can build the engine container with dagger call engine container) but i seem to remember you've had some problems with that before in your environment?

modest solstice Feb 7, 2025, 6:02 PM

#

I did, but Marcos helped resolve that. I'll try doing that 🙂

modest solstice Feb 21, 2025, 5:48 PM

#

hey @craggy ingot and @empty coral , I just upgraded to v0.16.1 and I'm an still seeing this 😦

#

target module (in different repo) is v0.15.4 and the calling module is v0.16.1.

modest solstice Feb 21, 2025, 7:29 PM

#

interestingly, this didn't happen when the versions matched. fwiw

#

let me upgrade my target repo to 0.16.1

modest solstice Feb 21, 2025, 9:36 PM

#

nope, upgrading target version did not help.

modest solstice Feb 22, 2025, 11:29 PM

#

looks like other's are seeing the same - https://github.com/dagger/dagger/issues/8989#issuecomment-2676349710

timid garden Feb 23, 2025, 4:54 PM

#

Hi there. I'm jumping into this thread. Surprisingly, I never encountered this issue with v0.15.2 or v0.15.3. Everything worked fine using GrAT on GitLab up to v0.15.4. However, I'm now seeing this issue with v0.16.1. Let me know how I can help solving this.

empty coral Feb 24, 2025, 1:14 PM

#

I'll be back tomorrow from PTO and can take a look at this 🙏

craggy ingot Feb 24, 2025, 3:45 PM

#

modest solstice looks like other's are seeing the same - https://github.com/dagger/dagger/issues...

@modest solstice did engine version v0.15.4 fix this for you? the fix i made here fixed the repro i had for v0.15.4, but we did a bit refactor in v0.16.0 that might have changed a couple things.
if even v0.15.4 didn't fix it, then i suspect that the repro i made was for some reason not your actual issue 😦

#

i'm happy to work on a fix here, but yeah, kinda stuck for how to repro this one, since i was able to get mine working fine 🤔

modest solstice Feb 24, 2025, 4:26 PM

#

v0.15.4 worked for me, FWIW.

#

so I've had to revert to v0.15.4 for now

craggy ingot Feb 24, 2025, 4:41 PM

#

ohhh okay, that's good to know actually!

#

(bleh) lemme spend some time on this

#

oh okay, i think i see what went wrong

#

yeah, it's this: https://github.com/dagger/dagger/pull/9697

GitHub

fix: move fix for context transfer secret by jedevc · Pull Request ...

Looks like a recent refactor moved this - it should be for local sources, as well as dir sources.
Re-applied from #9530.
Warning
This absolutely needs a test - sloppy work from me, since I didn&amp...

#

i'll add tests this time 😢

modest solstice Feb 24, 2025, 5:30 PM

#

that was fast! Thank you! I'll wait for next release 🙂 Or test when it's merged to main

modest solstice Feb 24, 2025, 7:05 PM

#

I tried to build the PR locally but was unsuccessful. It's probably related to something in my internal network

 ✔ Container.withWorkdir(path: "elixir_sdk_dev"): Container! 0.0s
│ │ │ │ │ │ │ ✘ .withExec(args: ["mix", "deps.get", "--only", "dev"]): Container! 1.1s | Disk Read: 0 B | Disk Write: 221 kB | CPU Pressure (some): 93µs | CPU Pressure (full): 93µs | Memory Bytes (current): 684 kB | Memory Bytes (peak): 4.4 MB | Network Rx: 108 B | Network Tx: 4.0 kB
│ │ │ │ │ │ │ ┃
│ │ │ │ │ │ │ ┃ 19:04:16.429 [notice] TLS :client: In state :certify at ssl_handshake.erl:2183 generated CLIENT ALERT: Fatal - Unknown CA
│ │ │ │ │ │ │ ┃
│ │ │ │ │ │ │ ┃ Failed to fetch record for jason from registry (using cache instead)
│ │ │ │ │ │ │ ┃ {:failed_connect, [{:to_address, {~c"repo.hex.pm", 443}}, {:inet, [:inet], {:tls_alert, {:unknown_ca, ~c"TLS client: In state certify at ssl_handshake.erl:2183 generated CLIENT ALERT: Fatal - Unknown CA\n"
│ │ │ │ │ │ │ ┃ ** (Mix) Unknown package jason in lockfile
│ │ │ │ │ │ │ ! process "mix deps.get --only dev" did not complete successfully: exit code: 1

modest solstice Feb 24, 2025, 9:22 PM

#

Well, I have bad news 😦 I am still seeing this issue on v0.15.4 too.

modest solstice Feb 25, 2025, 5:59 AM

#

When I copied the contents to a new file as you suggested instead of adding the file directly, it worked. So this is definitely related and you are on the right path. There may be a missing piece to the puzzle somewhere. Happy to help debug.

modest solstice Feb 25, 2025, 5:36 PM

#

I created an issue to better track this - https://github.com/dagger/dagger/issues/9713

GitHub

🐞 Module file referencing doesn't work when using private Git auth ...

What is the issue? When calling an external module (callee), and that external module references files that are local to that module, the engine throws an error. Dagger version dagger v0.15.4 upwar...

craggy ingot Feb 25, 2025, 5:37 PM

#

cheers 🙏

#

@grave yacht - i'm already looking at the specific error in https://github.com/dagger/dagger/pull/9697

#

i have a proper fix, it's the tests that are taking a while

#confusing secret error with cross repo module invocation