Caching of failing commands

[uncut topaz]

I wrote a pipeline for a bigger company project. The first run without cache takes about 30min. During the execution there are several commands which are failing. Some of the commands are not that important to execute successfully. For example linter steps or doc generation.
If I run the pipeline again, it takes around 5min. Without any changes on the filesystem! I observed that Dagger is trying to rerun the failed commands again. Which, obviously will fail again.

So is there an option to throw something like a warning instead of an error, cache the result and skip the execution of the command in the next run?

[cobalt vale]

Hi, sorry for the slow response here...

• At the moment, Dagger will never cache a command that fails
• We could possibly make that configurable, for example with an optional cacheOnFail argument or something like that. What do you think @elfin nymph @hard iron @sage ermine ?
• Meanwhile, you can achieve the same result by wrapping your command in a shell expression. Something like:

• Before: container { withExec(args: ["cat", "nosuchfile.txt"]) { stderr } }
• After: container { withExec(args: ["sh", "-c", "cat nosuchfile.txt || true"]) { stderr } }

[sage ermine]

Worth mentionining this is/will be resolved by Zenith already; if you define an Check it can do whatever it wants (including running things that fail) and then return a dag.StaticCheckResult(false, "lint output"), which will be cached.

In general it's safer to not cache failures by default, since things like network flakes happen all the time in test suites, and folks often want to be able to just retry. But I'd still be supportive of adding an API for it independently of Zenith, if we can figure out something intuitive

[cobalt vale]

That makes me wonder if we could add an optional retry config, maybe on withExec? or a custom caching strategy where it only caches after N executions?