Hey @left void! 👋 What are span "effects"?
https://github.com/dagger/dagger/blob/ef3400788bf37c1c96cf41d1890b5d82f556c5f7/dagql/dagui/spans.go#L498
#Telemetry: cache miss reasons
dreamy cloak
Trying to understand the reason for cache misses in graphviz. Is the output of CachedReason() always accurate, or only in certain cases?
left void
Hey <@108011715077091328>! 👋 What are span "effects"?
https://github.com/dagger...
"effects" are usually e.g. the LLB ops installed by withExec (the "cause") - they are correlated by cause/effect ID which for LLB ops is the vertex digest, but the string can be anything
nowadays span links are mostly used for this instead of cause/effect attrs but the code is still there for UI backwards compat. with older engines
dreamy cloak
So for a newer engine, you'd look into span links for cache miss reasons?
left void
eh no effect IDs are probably better for that - since they say "here are all the possible effects you should see for me in the future"
span links only appear when the effect's span arrives - which might be a cache hit, might not, and it might never actually arrive (if it's a dependency of a cache hit for example, you won't see a span for all the transitive dependencies)
left void
Trying to understand the reason for cache misses in graphviz. Is the output of `...
it should be pretty accurate, but it's complicated as hell to implement so there's always a chance I missed a fringe scenario
left void
span links only appear when the effect's span arrives - which might be a cache h...
this comment is related to the last bit I mentioned: https://github.com/dagger/dagger/blob/ef3400788bf37c1c96cf41d1890b5d82f556c5f7/dagql/dagui/spans.go#L506-L511
dreamy cloak
it should be pretty accurate, but it's complicated as hell to implement so there...
Ok, that's good to know 🙂 I can see you get a span from an effectID via span.db.EffectSpans[effect]. Can it match a span from a function call, or is it always to something lower level like an LLB op?
left void
it's always something "lazy" like an LLB op. also we don't even show the effect spans, the UI just merges them with the originating/causal span instead, so e.g. withExec(sleep 30) shows a 30s duration
dreamy cloak
Would you be surprised if a "higher level" span (say in terms of the nodes in the graphviz output from the CLI) has cached parents, but not be itself cached?
Assuming all function args are pure
left void
it's technically possible - sometimes the effects just never run. for example, a module whose constructor applies a default like container.From("golang:1.20") that never ends up being used, will make the constructor look like it's pending forever
dreamy cloak
So a pending span shows up as not cached in CachedReason()?
left void
So a pending span shows up as not cached in `CachedReason()`?
it should yeah - they're derived from the same data. (there's also a PendingReason())
btw - if you press '?' in the TUI, or add ?debug to the URL in the web UI, it'll surface all that info
dreamy cloak
Ooh, didn't know about ?debug!
left void
a little buggy though, the debug icon shows up for nested spans since it's far off in the margin 😛
dreamy cloak
Ok, I can se the effectIDs in that debug, but how can I relate them to something?
left void
it also lists the effect spans and their statuses below the <pre> - they don't show up normally so can't really relate it to other spans
dreamy cloak
I'm looking at a span in cloud ui where the snapshot shows a bunch of effectids, but in the Effects section only one is shown. Which ones go in there?
dreamy cloak
Some IDs are in xxh3 and others are in sha256. Does that tell you something about what kind of "span" it is?
left void
noticed that too, I think it's just a side effect of @bitter ibex's recent refactor (https://github.com/dagger/dagger/pull/9204) - if it's sha256 that means it's a custom digest i guess?
part of the march towards dagger/dagql native content-addressing
hopefully that only shows up in the inner plumbing like this
hmm looks like it's used for the context dir too
basically anywhere that you used to see blob(...)
dreamy cloak
If a span HasLogs, it's not cached?
left void
yeah, since that implies something ran
dreamy cloak
Oh, I see.
left void
hmm it seems like the list of effects is incomplete
for example this span beneath codegen
dreamy cloak
Can't tell why it ran, though.
left void
same digest is listed in the effect IDs of the parent codegen call, but the uv lock span isn't listed for some reason
left void
Can't tell why it ran, though.
yeah, this part is implicit with cache misses still, theoretically it comes down to a change in the inputs from last time
are you trying to determine why codegen re-ran?
dreamy cloak
Yeah, this withExec
left void
i see
wonder if it has to do with this? https://github.com/dagger/dagger/issues/8955
cc @brisk walrus - saw it mentioned in another thread