ModuleSource digest

[steel bay]

@sage patrol, what are the inputs to a ModuleSource digest? I'm trying to understand why the first functions call has a different one:

1. dagger init --sdk=python --name=benchmark
2. dagger functions (xxh3:c083daec3e945cec)
3. dagger functions (xxh3:8dcb858d74fd0040)
4. dagger functions (xxh3:8dcb858d74fd0040)

[sage patrol]

It's calculated here: https://github.com/dagger/dagger/blob/312255260f8d640c8260b72f51d4487989228987/core/modulesource.go#L258

[steel bay]

Thanks 🙏

[sage patrol]

let me know if you find something, definitely weird it changes like that (assuming no local file changes, etc.)

[steel bay]

I'm pretty sure it's because the modsource digest is appended to /src when mounting the context directory.

[sage patrol]

I believe you but the dots don't fully connect in my head yet 😄

[steel bay]

Well.. I mean I think that's why there's a caching issue there, but still the digests on modsource change if I remove the digest from the mount path. I'm going to check on another SDK (TS) to see of the same thing happens.

[sage patrol]

Oh I think I see what you're saying. That's why the cache busts during the function calls made by the sdk but still doesn't fully explain why the input digest changes in the first place

?

[steel bay]

As caching goes, you can imagine that if the modsource digest changes, everything after WithWorkdir that has that digest busts the cache from then on.

Yes

Ok, also happens for other SDKs then:

1. https://v3.dagger.cloud/Correia/traces/2f2a9cbb325b792c2636e1ba7e45782e?span=d2f28d526aaa5bfb
2. https://v3.dagger.cloud/Correia/traces/d791c8084172c3eefea1a0f9195b51dc?span=ef1896a3d2970c7c

It's just Python is affected cache-wise because of that namespace.

[sage patrol]

Yep that makes sense, the thing is we really want to fix whatever is causing the arg input digest to be different. Cause then with persistent function caching that being stable will result in the whole call getting cached

[steel bay]

Then there's a general, not Python specifc reason why two consecutive module loads have different modsource digests.

[sage patrol]

I don't have time to pivot from what I'm working on atm, but might be worth println debugging that CalcDigest func to see what the diff is

[steel bay]

Yeah