Ledger Snapshots and Testing

[waxen pond]

There's a PR open on the stellar-cli that contains a new ledger snapshot subcommand:

The command generates a snapshot given a ledger sequence number, and that snapshot can be loaded into contract tests as a beginning state for the test to start with.

The goal is to make it possible to write tests that interact with complex production state, either for debugging or for writing tests that cover complex scenarios.

Feedback appreciated on the feature before it is release.

You can try it out by installing from source:

cargo install --locked --git https://github.com/stellar/stellar-cli --branch snap stellar-cli

Then run a command like this to create a snapshot:

stellar snapshot --contract-id CCW67TSZV3SSS2HXMBQ5JFGCKJNXKZM7UQUWUZPUTHXSTZLEO7SJMI75 --out snapshot.json

Demo: https://www.youtube.com/watch?v=mnY7AMm8MRk
PR https://github.com/stellar/stellar-cli/pull/1368

[subtle wedge]

This is so sick @waxen pond!! This was partly on my next sprint's mercury roadmap but you've pretty much already wrapped 90% of what I wanted to do (rest is just some server-level caching for high demand buckets). Will try this out later, not sure if it's there already but would be great to add certain ledgers in time support.

[waxen pond]

certain ledgers in time support
Are you thinking where you can say give me ledger from a certain date, timestamp, relative time ago? Or do you mean something else?

The command supports a --ledger where you can specify any ledger that has been stored in history arcvhives. (That means every 64th ledger.)

[subtle wedge]

yeah, part of what I wanted to do is reaching a snapshot of any ledger, even those that aren't checkpoints. That means, getting the checkpoint for the ledger, and applying the transactions (I was thinking soroban only so that I wouldn't have to rewrite the core logic and rely on the vm's execution only + apply changes to my snapshot) until the target ledger was reached

(saying this as it's probably where using archives comes more in handy, since current ledger snapshots would be much faster done with other services since it could already be structured in the service's end. For example, I could get a full snaphost of two contracts with a simple zephyr program, I wouldn't be able to get it for a certain ledger that has already closed since the current entries are updated for each close on the server.)

[bronze sail]

How do you list multiples of these filters? Just comma separated?

Can I combine filters?

[waxen pond]

List the option multiple times --account-id G... --account-id G..., and yeah they all combine.

[bronze sail]

Just tried this out for real but I'm getting an error

running 1 test
test test::test ... FAILED

successes:

successes:

failures:

---- test::test stdout ----
thread 'test::test' panicked at /Users/tylervanderhoeven/.cargo/registry/src/index.crates.io-6f17d22bba15001f/soroban-sdk-21.3.0/src/env.rs:1267:52:
called `Result::unwrap()` on an `Err` value: Custom { kind: InvalidData, error: Error("missing field `generators`", line: 539, column: 1) }
stack backtrace:
   0: rust_begin_unwind
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:652:5
   1: core::panicking::panic_fmt
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/result.rs:1654:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/result.rs:1077:23
   4: soroban_sdk::env::Env::from_snapshot_file
             at /Users/tylervanderhoeven/.cargo/registry/src/index.crates.io-6f17d22bba15001f/soroban-sdk-21.3.0/src/env.rs:1267:29
   5: hello_world::test::test
             at ./src/test.rs:24:15
   6: hello_world::test::test::{{closure}}
             at ./src/test.rs:23:10
   7: core::ops::function::FnOnce::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/ops/function.rs:250:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    test::test

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s

error: test failed, to rerun pass `-p hello-world --lib`

 *  The terminal process "cargo 'test', '--package', 'hello-world', '--lib', '--', 'test::test', '--exact', '--show-output'" terminated with exit code: 101. 
 *  Terminal will be reused by tasks, press any key to close it.

All I have in my test is
let env = Env::from_snapshot_file("snapshot.json");

stellar snapshot --contract-id CALCROAXSHD3HWE3O2EBJIGGWFMXD24725XIQL5P3IZHA6DE3ETO3NU2 --contract-id CAKX2ZMAKMID6PEDSUHBMU4NAHWUIEFXTXHESCSN7IRVG5E4QKAWSGLU --account-id GDT3KJMJIQWDOZWERC3K5SSGQGYALT2VSUAQP2YEGDK7YDPSQWUCIHYZ --account-id GBT7SVY2S6KUQA4C2MKIN3XKGFKRHZUDEITEGCRQITUATD6ZVANT2LW7 --out snapshot.json --network testnet --format json

[workspace]
resolver = "2"
members = [
  "contracts/*",
]

[workspace.dependencies]
soroban-sdk = "21.3.0"
ed25519-dalek = { version = "1.0.1" }
rand = { version = "0.7.3" }
stellar-strkey = { version = "0.0.8" }

[profile.release]
opt-level = "z"
overflow-checks = true
debug = 0
strip = "symbols"
debug-assertions = false
panic = "abort"
codegen-units = 1
lto = true

[waxen pond]

You need to do Env::from_ledger_snapshot_file

[bronze sail]

Ha

[waxen pond]

It's unfortunate we have two competing concepts. A snapshot in the SDK contains a complete snapshot of the test environment.

Where-as a ledger snapshot only contains the ledger.

Thinking about what we can do to make this less confusing.

[bronze sail]

from_network_snapshot_file vs ledger? Could clue into the fact this is from the network vs just a snapshot

idk

[waxen pond]

I could make the from_snapshot function at least return a nicer error like "maybe you wanna call this other fun"

[bronze sail]

Yeah that works too

[waxen pond]

It should be possible to detect this.

Opened issue for improving this: https://github.com/stellar/rs-soroban-sdk/issues/1305

[bronze sail]

If you seed the env from a ledger snapshot should you still be able to do things like this?
let seed_contract_id = env.register_contract(None, Contract);

I'm seeing some errors I'm not quite sure how to debug

---- test::test stdout ----
thread 'test::test' panicked at /Users/tylervanderhoeven/.cargo/registry/src/index.crates.io-6f17d22bba15001f/soroban-env-host-21.2.0/src/host.rs:768:9:
HostError: Error(WasmVm, MissingValue)

Event log (newest first):
   0: [Diagnostic Event] contract:CAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD2KM, topics:[error, Error(WasmVm, MissingValue)], data:"escalating error to panic"
   1: [Diagnostic Event] contract:CAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD2KM, topics:[error, Error(WasmVm, MissingValue)], data:["contract call failed", balance, [CAKX2ZMAKMID6PEDSUHBMU4NAHWUIEFXTXHESCSN7IRVG5E4QKAWSGLU]]
   2: [Failed Diagnostic Event (not emitted)] contract:CAKX2ZMAKMID6PEDSUHBMU4NAHWUIEFXTXHESCSN7IRVG5E4QKAWSGLU, topics:[error, Error(WasmVm, MissingValue)], data:["invoking unknown export", balance]
   3: [Diagnostic Event] contract:CAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD2KM, topics:[fn_call, Bytes(157d658053103f3c83950e16538d01ed4410b79dce490a4dfa2353749c828169), balance], data:CAKX2ZMAKMID6PEDSUHBMU4NAHWUIEFXTXHESCSN7IRVG5E4QKAWSGLU

Oops I was pointing my token client to the wrong contract_id

Everything is working now except for a sequence number that I'm setting for __check_auth

[waxen pond]

In any case that error is really difficult to understand, so it's worth opening an issue on the rs-soroban-env repo with an example of your error.

[bronze sail]

Even just let signature_expiration_ledger = env.ledger().sequence() + 1; results in "signature expiration is too late"

[waxen pond]

invoking unknown export
This should say invoking unknown function, since people know what functions are but probably doesn't realise a function is an export.

[bronze sail]

let signature_expiration_ledger = env.ledger().sequence() - 1; results in "signature has expired" which is expected

Just let signature_expiration_ledger = env.ledger().sequence(); seems to work but I'm not sure why I can't extend when building my auth entry

Where do buckets get cached? Do they ever get flushed? I'm generally a little concerned with how large test_snapshots are and this increases my concern for unbound growth of relatively secret files and directories

[waxen pond]

The SDK won't cache them.

The CLI won't cache generated snapshots. It will cache buckets it downloads, and those cached entries can be cleared by clearing the CLIs cache with stellar cache clean.

[waxen pond]

What do you mean by extend when building the auth entry?

[bronze sail]

I build an auth entry and part of that is setting the expiration ledger

Currently I can let set an expiration ledger for anything beyond the current ledger

[waxen pond]

Makes sense, anything earlier and it would be expired.

[bronze sail]

I can’t set it later either

#1251054730281750578 message

[waxen pond]

Could you link to the code example?

Could you open an issue with an example on the rs-soroban-sdk or rs-soroban-env repos? That doesn't look right that the auth has to have the current ledger. Although mabye the test env isn't setup right.

[bronze sail]

I’ll link you privately as it’s code for a new puzzle I’m working on. Don’t want to leak the answer! 😋

[bronze sail]

@waxen pond is it reasonable I'd need to wait awhile before this tool would pick up changes to testnet?

I push stuff to testnet like deploying a contract but it seems to take ~5 minutes before snapshotting will actually find that new contract

Is that due to this tool pulling from checkpoints vs ledgers themselves?

[waxen pond]

Yup, this tool gets state from archives, not from validators, so it only has access to checkpoints which occur every 5 minutes on deployed networks, and every 8 seconds on quickstart.

[bronze sail]

https://github.com/stellar/rs-soroban-sdk/issues/1307

[waxen pond]

Are you seeing any large disk usage or explosive growth here? I haven't seen that in my own use, but curious if that's what led to your comment above.

[bronze sail]

Nope, I just like to know where things are going and how to get rid of them. Rust projects are notoriously big.

And I thought node_modules were bad

[waxen pond]

stellar snapshot create is released in stellar-cli v21.3.0.

[hardy frigate]

How much work would it be to have a standalone network be initialized from a snapshot? My assumption is that it would be fairly hard, but in the event it's not I think the benefit could be cool.

Use case would be setting up integration tests for external services (like a JS program). Could snapshot mainnet/testnet, start an standalone network / rpc from the snapshot, and run your test against the RPC. Would save a lot of effort in running sometimes lengthy setup scripts, and allow external services to reproduce bugs easily.

cc: @waxen pond