Like "x directory has changed" or something?
#Is it possible to find out why a `WithExec` wasn't cached?
vocal coyote
bold ether
do you have a dagger cloud URL for that run? 🙏
vocal coyote
I'm working on making it more cachable by only copying in files/folders that are needed. and his one has me stumped
bold ether
@vocal coyote by any chance is the code of this pipeline open?
I'd love to check something
vocal coyote
yeah.
https://github.com/grafana/grafana-build/
on my branch km/improve-caching.
I'm running:
dagger run go run ./cmd artifacts --version=v10.4.0-test --grafana-dir=$HOME/Work/Grafana/grafana -a targz:grafana:linux/amd64
Here's the relevant code:
https://github.com/grafana/grafana-build/blob/main/artifacts/backend.go#L71
bold ether
@vocal coyote any chance you could update to the latest Dagger version? I see that you're still using v0.8.7 which a bit old
additionally, could you verify if you get cache misses consistently? i.e running the same pipeline 2/3 times 🙏
vocal coyote
Yeah that particular step never gets cached. I should have multiple runs right before it. I'll look at updating it!
bold ether
perfect, I'll take a look and come back with some more info
bold ether
testing this now locally Kevin with the latest Dagger version 🙏
vocal coyote
🥲 thank you. It's not super urgent!
Is there was an easy way to see the reason that it was a miss?
bold ether
think I've found it
validating
yep, issue found: problem is here: https://github.com/grafana/grafana-build/blob/main/containers/withenv.go#L7
since iterating a map is not deterministic, you get random different orders each time and cache gets invalidated
that's why you might have seen that sometimes you get a cache hit and some others you don't.
Unfortunately, it's quite difficult to figure this out either via the TUI or Dagger Cloud. On the other hand, with @merry vault we've spoken about these kind of situtations in the past and we have some work that will land soon to be able to show this in cloud
vocal coyote
Doh. Thanks so much!! That makes sense. I appreciate that a lot!
vocal coyote
I knew I was doing something wrong
bold ether
np, we need to improve the visibility for these kinds of things. Thx for being an awesome Daggernaut 🤗
bold ether
cc @broken saffron for TUI visibility
vocal coyote
With your advice I found some other places where I was using iterating over a map. I corrected those and now I have this. Very awesome.
