#Expiration in Env in tests

31 messages · Page 1 of 1 (latest)

karmic thunder
#

Should we expect the Env to hide storage that's expired in tests?

I have a test where I create an item, then change the ledger to be later, well after the data should have expired. The data was still available.

cerulean jay
#

I'm not sure TBH. preflight 'restores' the expired entries and returns them in response. basically host invariant is that the storage snapshot it gets is always valid, so it doesn't deal with expiration. if necessary, we could emulate the expiration for tests specifically, though it's not immediately clear how much value that would add

karmic thunder
#

On one hand I agree, on the other, this is another way that the tests don't represent true behaviors. And tests are one way that folks can learn about how the real system behaves. Similar problem to auth.

#

Tests don't really present a clear notion of ledgers either. Everything is running in this pseudo ledger of sorts. I think for us to address this, we'd need to, or should, address that issue more holistically.

#

So I guess we could try answering that question: Should tests make ledgers explicit somehow? Or should they're experience of ledgers be improved in some way, such as every top-level invocation results in a ledger bump?

#

The latter doesn't always make sense, as you pointed out recently, because someone might want multiple txs/invocations in a single ledger.

cerulean jay
#

yeah, ledger seq is only relevant for the contracts that care about time, and these probably would do sequence/timestamp bumps manually to exercise their logic

#

tests don't cover certain parts of the full stack because these lie outside of the scope of contract influence

#

like we could let entries 'expire', but then what? in reality, you'd do restore operation. we could of course add a test fn to restore entries, but at that point you're basically building a test-only infrastructure that is fully self-contained and doesn't cover almost anything that your contract does

#

(besides the bumps... and for bumps I think we need a way to assert that certain entries have been bumped or have certain expiratoin)

karmic thunder
#

Right.

karmic thunder
cerulean jay
#

yeah, that's something I wanted to have and we're missing - we need a test util function to return an expiration for a given storage key

karmic thunder
#

Got it. I think I can add that without too much effort. However, everything we add as a testutil we should ask why not in a contract, since everything we add is one more difference between test and real. It wouldn't be super useful to expose this function to non-test contract code, but it wouldn't be harmful and may have edge case uses?

cerulean jay
#

that's a fair point. I think it should be fine. but let me think for a moment. FWIW initially we wanted to be as opaque as possible about the expiration ledger due to potential concurrency implications. but I don't think that's an issue with the current design

karmic thunder
#

Yeah conceptually we sort of broke the seal when we said contracts could bump internally. If there was no bump, arguably none of this should be testable within contract tests.

#

I don't really want to rehash any of this, but maybe we've gone too granular. If instead of bump calls to bump individual entries, if a contract just had a meta config that said "any entry created by this contract should live for Low to High".

We got rid of auto bumps before we had the Low to High. Does Low to High address the concerns we had with auto bumps? cc @lone charm

cerulean jay
#

"any entry created by this contract should live for Low to High".
this unfortunately seems too restrictive - first and foremost doesn't work for temp entries (which have to have very specific lifetime), also doesn't play well with contracts like tokens where there are different 'tiers' of storage (e.g. every user would probably need to bump the instance just a bit, but balances for more etc.)

lone charm
cerulean jay
#

but we're basing fees on this value, I thought that the value that every contract observes is just the bump performed by its own transaction

lone charm
#

No. Each partition begins on the same snapshot, but within a partition, the TXs execute sequentially. I.e., within a given partition, if TX 0 bumps and entry, TX 1 sees the bump

cerulean jay
#

hmm, still not sure I understand. what the contract would see is the value that is a function of the snapshot (stable within logical partition) and its own bump operations. it can't observe bumps from another contracts, thus there shouldn't be concurrency issues. the same logic already applies for the fee computations

lone charm
#

Suppose we have TX A which calls InvokeHostFunction. The first line of the function is ASSERT(wasm.lifetime > 10). When TX A is invoked, the wasm lifetime is 9. On that same ledger, TX B calls BumpFootprintOp(100) on the WASM. There is no dependency between these two operations, so they are scheduled in seperate paritions. TX A fails, while TX B succeeds. However, we must enforce the invariant that execution order of partitions does not affect execution. This invariant is broken by the assert

#

This would also break meta, since either TX could produce meta first. This means that downstream systems see 0: WASM lifetime bumped 100, 1: TX failed, WASM lifetime < 10

#

We have a resolution step in the meta that resolves conflicts ibetween expirationLedgerSeq values, but it only resolves the values. It could not resolve the runtime dependency that was introduced by the assert

#

All that to say, I don't think this feature would be useful at all to developers (Why would they want fees to be burned when the TX could have succeeded?) and seems to introduce some concurrency complexity later on, even if there is a solution

cerulean jay
#

but we already have the code that does pretty much that - compute the difference between the new lifetime and old lifetime and compute the fee. the point that is missing is that wasm.lifetime can't be modified by tx B

#

yeah, I don't particularly care about exposing this as host fn. but I did get confused by the concurrency implications again (because I thought we've worked around those)

lone charm
#

TLDR concurrency is as follows:
Within a partition, charge fees and bump lifetimes sequentially, such that within the parition TX B sees side effects from TX A
After all partions execute, take the max of all lifetime bumps, write this value as the "conflict resolution" step

lone charm