#Test Recording

7 messages · Page 1 of 1 (latest)

marble oracle
#

I'm looking for feedback on some new functionality for the Rust SDK that records the ending state of the Env after tests are complete.

https://github.com/stellar/rs-soroban-sdk/pull/1137

The goal is make it really easy to record a snapshot of the Env's major forms of output (events and storage) as well as a snapshot of what functions returned and errored with (as captured in diagnostic events).

For the majority of tests due to work done recently to omit random data these values being outputted should be constant, and you could commit them to version control as a way of catching if things change unexpectedly. I think this will be especially helpful for events such as SDK upgrades and future protocol upgrades as a way to sanity check your contracts behavior will continue to be consistent.

For the tests that produce random or unpredictable outputs, simply ignoring the outputted in gitignore or similar files should suffice for ignoring them.

This feature currently has no configuration. It's always on for all tests.

GitHub
Autosave a test snapshot file on every test exit by leighmcculloch ...

What
Autosave a test snapshot file on every test exit.
Why
To encourage the over-time consistency testing of contracts. To provide a mechanism where any developer who commits the generated files ca...

marble oracle
#

@solid dawn Interested to know what your experience of this would be like in your development workflow.

solid dawn
#

I have a feeling these files are going to be MASSIVE for something like Blend (lots of setup / contracts involved), however, having snapshots does seem like a good way to debug what is going on (not sure if there was an existing way to easily pretty print the meaningful portion of the env object).

My gut reaction is that this is mostly useful for integration level tests, and I would be turning on the feature for a select number of tests and turning it off for the rest. (see https://github.com/blend-capital/blend-contracts/blob/main/test-suites/tests/test_pool.rs#L19 as something this could replace)

I can try and update Blend to this commit later this week and share any findings

marble oracle
marble oracle
marble oracle
#

After experimenting with ☝🏻 it really wasn't clear it was better. But definitely interested to hear what folks think who have larger outputs. It could make it easier to browse when unexpected changes occur, but unclear to me.

solid dawn