#Cache-stable parameters without Secret log sanitization

1 messages · Page 1 of 1 (latest)

rich prairie
#

We're running Playwright E2E tests inside Dagger containers in CI. We need to pass git/CI metadata (commit SHA, branch ref, author name, commit message) into the container for test reporting (Currents.dev integration).

The problem: All function parameters contribute to Dagger's cache key. These values change on every commit, so passing them as regular string params invalidates the entire E2E test cache — tests re-run from scratch even when no test code changed.

Our current workaround: We use Secret type with a static ?cacheKey= identifier:

--commit-sha=env://CURRENTS_COMMIT_SHA?cacheKey=currents-commit-sha --commit-info-author=env://COMMIT_INFO_AUTHOR?cacheKey=commit-info-author --github-ref=env://CURRENTS_GITHUB_REF?cacheKey=currents-github-ref

This keeps the cache key stable — great. But Dagger now treats these as secrets and sanitizes their values across all downstream logs. If the commit author is "John", every occurrence of "John" in the entire CI output gets replaced with ***. Short values like PR numbers or single digits are especially bad — they get redacted everywhere, making logs unreadable.

These values are not sensitive — they're public git metadata. We just need them excluded from cache key computation.

What we're looking for:
Is there a way to mark a parameter as cache-transparent without using Secret? (e.g., some annotation or decorator we're missing)
Is there a way to disable log sanitization for specific Secret values?
Has anyone solved a similar problem with a different pattern?
We looked through the source (dagql/call/id.go) and it seems like isSensitive controls both cache exclusion and log scrubbing with no way to separate them. Would a feature like @argument({ cacheTransparent: true }) be something that's been discussed or planned?

Running Dagger v0.19.3, TypeScript SDK.

brisk ridge
#

Even if the top-level function is invalidated, it doesn't automatically follow that the whole DAG should get cached... If your actual test execution is getting invalidated, it means you have a problem with the inputs to that underlying execution, not to the top-level function

#

Do you have a dagger cloud trace URL you can share?

rich prairie
#

Thanks for the suggestion! We ran the experiment — changed the 7 metadata params from Secret to plain string and ran two consecutive CI runs (second one was an empty commit, so only git metadata changed).
All non-E2E steps (lint, format, type-check, unit tests, integration tests, endpoint tests) were cached in the second run — no output for them in the logs at all. But E2E tests were NOT cached — they re-ran from scratch.

So the DAG-level caching works as you described for everything except the container that receives these changing env vars. Is there a way to set env vars on a container without affecting its cache key? We need these values available at test runtime (Currents reads them from env), but they shouldn't invalidate the cached test execution.

This is the trace of second run on empty commit:
https://dagger.cloud/fv-ci/traces/ff7b11c9497344438d5f415ba7ef4149

brisk ridge
buoyant ice
#

Hey 👋 having a look. I'm not too familiar with playwright beyond the surface level, can you explain how those input values contribute to the test runtime if we want the caching to be independent of those values?

brisk ridge
#

Also can you point out the specific span of the E2E test execution in that trace 🙏 The individual spans are linkable

rich prairie
rich prairie
# buoyant ice Hey 👋 having a look. I'm not too familiar with playwright beyond the surface le...

These env vars are read by the @currents/playwright reporter plugin at test runtime. Currents uses them to associate test results with the correct commit in its dashboard

The actual Playwright test execution doesn't depend on these values at all — tests pass or fail regardless. It's purely reporting metadata. But since @currents/playwright reads them from process.env during the test run, we need them set as env vars before withExec runs Playwright.

Ideally we'd inject them in a way that doesn't affect the container's cache key, since they don't influence test outcomes.

brisk ridge
# rich prairie These env vars are read by the `@currents/playwright `reporter plugin at test ru...

But did't you mention that even without setting those variables, the e2e exec still doesn't cache? So, those env vars are not the cause, there is another undiscovered cause for invalidating those withExecs? Or did I misunderstand your earlier message?

changed the 7 metadata params from Secret to plain string and ran two consecutive CI runs (second one was an empty commit, so only git metadata changed).
All non-E2E steps (lint, format, type-check, unit tests, integration tests, endpoint tests) were cached in the second run

rich prairie
#

Sorry for confusion
So caching of results works if these environment varables are passed as Secrets, but it's not working if those are strings and passed in .withEnvs

brisk ridge
#

@rich prairie I will look more into the trace to confirm (and confer with @buoyant ice 🙂 but my guess is that the e2e run is too monolithic, because of the bundled report export to currents.dev (I guess a proprietary test management product?)

Ideally there would be a way to decouple test execution & report generation & upload. I don't know if the tool supports that.

I would be surprised if currents didn't support that one way or the other - it would be very bad design if they didn't

#

Separately, we would love to help you break down your mega-CI function into smaller individual checks, it would make your traces easier to debug, and also probably execution faster 🙂

buoyant ice
#

Yeah I do feel like the underlying problem is dagger being at odds with how currents expects execution to happen (+ github actions in the mix for extra confusion). If we get caching working for the e2e test, that would mean currents would not get any data for that test run. Hopefully thats expected. So currents would only get reporting for uncached runs. If you want currents to have all the data it expects, keep the pipeline like this so that the git metadata correctly busts the cache. If you want caching to prevent reruns of identical source, then technically the github metadata should not be inputs to the dag. If it really must work like this though, I'm sure we can come up with a better workaround than making each parameter a secret!

brisk ridge
#

What I don't fully understand is: if the tests are cached, it means that the same code was tested before - doesn't it also mean that the same commit and associated metadata were tested before? Or is the issue that the git history is rewritten - rebase, force-push etc, and the same git directory tree is wrapped in a different git commit ID?

buoyant ice
#

It could also be a commit to code that isn't part of the playwright tests though

brisk ridge
#

Ah of course

quaint forum
#

I think the question is here more about how NOT to cache when using non-secret, but also non-deterministic values.

#

E.g. we need to pass "github run ID" for example to the e2e env so that it can pass it to currents. We cannot put it into a secret, because then any ID will be obfuscated in the log. But only secrets have the "cache key" option on the args. So we cannot stabilize this value. We don't care if the step does not run and does not produce output to currents, when it skipped due to cache.

brisk ridge
#

@quaint forum so you're looking for a way to guarantee that your e2e run will not be cached?

That's easy, you can use cache control - @cache("never")

#

The reason that e2e exec doesn't cache properly is because it mixes a check (pure, cacheable) and a "ship" (side effect, not cacheable). You're both testing and publishing a test report to an external service in the same low level
operation, so Dagger can't treat them differently.

The solution (unknown difficulty) is to separate your test run from your publishing of test report.

rich prairie
#

I think there's a slight misunderstanding — we actually want E2E tests to be cached. @cache("never") would be the opposite of what we need.

To clarify our goal:

  • When no test-relevant code changes → E2E should be cached (skip execution). There should be no reports from Currents as well, as no actual e2e tests were run.
  • When test code changes → E2E runs normally, Currents gets the report naturally.

Separating test execution from reporting is possible in theory, but @currents/playwright is a Playwright reporter plugin that runs during test execution (capturing traces, videos, screenshots in real-time), not a post-hoc upload tool. So decoupling them would mean loosing part of functionality as well and creating separate pipeline with storing tests results and uploading them.

But the underlying issue is broader than Currents — it's about Dagger not having a way to pass env vars to a container without affecting its cache key. Today the only options are and we're looking for a third option: env var that doesn't affect cache key and isn't treated as a secret.

Is that something Dagger could support?

brisk ridge
#

env vars affecting a container's cache key is by design... we'll think about ways to give you an escape hatch, but I don't recommend it except as a last resort.

#

I guess there is a precedent with cache volumes in a way

#

Container.withEnvVariable(cacheNeutral: true)?

brisk ridge
#

While we look into the above (looks doable and I understand the rationale for supporting it).

It looks like there is a way to decouple execution from upload in currents. you don't need a whole new pipeline, just to split that withExec into 2 withExecs: a playwright run, and currents upload. Then you don't need the escape hatch: the text execution is pure, the upload is not. You do lose real-timr streaming during test execution though.

rich prairie
#

Thanks for looking into cacheNeutral: true — I think that would be the good solution for our case.

Regarding the split approach: we could use @currents/cmd to upload results in a separate withExec after test execution. But when the test execution step is cached, the upload would still run and send the cached results to Currents tagged with the new commit SHA — so Currents would report results for a commit where tests didn't actually execute. This is something which we also want to avoid

brisk ridge
brisk ridge
sturdy solstice
brisk ridge
#

Based on the discussion above 👆 it looks like we encountered the first use case (that I know of) where that seems like a legit use of it, and no good alternative exists

#

If you're on board with the idea, I can take the PR

sturdy solstice
#

Yeah that SGTM. Otherwise users will just hack it anyways via secrets or other stuff like that.

brisk ridge
#

Oh I'm sure users would NEVER do that! Right @rich prairie 😇

brisk ridge
brisk ridge
quaint forum
#

I think a better solution would be having support for arg having a cache key just like secrets do. Where you pass the ?cache key flag and it makes it stable. Why can’t any arbitrary arg have the same option? We ultimately pass those currents envs thru the args down to env anyways

brisk ridge
quaint forum
#

Cli args

brisk ridge
#

The core issue is the caching of that expensive withExec.. Better control over the caching of your wrapper function may also be something we can help with, possibly with a new primitive... But regardless of whether your wrapper function is cached or not, the first order of business is giving you better control of the caching of that withExec

brisk ridge
rich prairie
#

Amazing, thank you!
I hope it could be pushed soon 🙏

brisk ridge
rich prairie
#

Hi @brisk ridge , I've seen that PR was closed, are you planning to merge & release it soon, still?

brisk ridge
rich prairie
#

Great to hear, thanks for the update!