SorobanClient `getLedgerEntries` throwing error in 1.0.0-beta.4 | Stellar Developers | Page 1

weary coral Nov 15, 2023, 6:52 PM

#

Hello!

Wanted to flag an error I've been seeing in soroban-client 1.0.0-beta.4.

The Blend-Sdk makes use of getLedgerEntries pretty extensively to fetch contract data from chain. (this is mostly so we can fetch multiple entries at once from the RPC)

See this block of example code reading from testnet for the error:

  const backstop_id = 'CDTPZ5PCK5COX73S5MABIZ3G3A6CLC3HDYQOHSXH6ZMN2VWHVRXAJO74';
  const pool_id = 'CAMUSVYFN6LUKIZTKCFVP536GDU3GOBIFVGG6OJKU4EDUW6OMDTM3UZK';
  const poolEpsDataKey = xdr.LedgerKey.contractData(
    new xdr.LedgerKeyContractData({
      contract: Address.fromString(backstop_id).toScAddress(),
      key: xdr.ScVal.scvVec([
        xdr.ScVal.scvSymbol('PoolEPS'),
        Address.fromString(pool_id).toScVal(),
      ]),
      durability: xdr.ContractDataDurability.persistent(),
    })
  );

  const backstopPoolDataPromise = await stellar_rpc.getLedgerEntries(poolEpsDataKey);
  console.log('Result: ', backstopPoolDataPromise);

with 1.0.0-beta.2 this produces:

Result:  {
  entries: [
    {
      key: 'AAAABgAAAAHm/PXiV0Tr/3LrABRnZtg8JYtnHiDjyuf2WN1Wx6xuBAAAABAAAAABAAAAAgAAAA8AAAAHUG9vbEVQUwAAAAASAAAAARlJVwVvl0UjM1CLV/d+MOmzOCgtTG85KqcIOlvOYObNAAAAAQ==',
      xdr: 'AAAABgAAAAAAAAAB5vz14ldE6/9y6wAUZ2bYPCWLZx4g48rn9ljdVsesbgQAAAAQAAAAAQAAAAIAAAAPAAAAB1Bvb2xFUFMAAAAAEgAAAAEZSVcFb5dFIzNQi1f3fjDpszgoLUxvOSqnCDpbzmDmzQAAAAEAAAAKAAAAAAAAAAAAAAAAABt3QA==',
      lastModifiedLedgerSeq: '2535782'
    }
  ],
  latestLedger: '2537751'
}

with 1.0.0-beta.4

TypeError: expiration not set
    at ChildUnion.get (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1727:95)
    at ChildUnion.get [as expiration] (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1836:25)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:636:104
    at Array.forEach (<anonymous>)
    at mergeResponseExpirationLedgers (/Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:626:135)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:230:24
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async temp (file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:164:37)
    at async file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:191:1

Suggestions on how to move forward? My current thought is patching the sequence number bug manually on the 1.0.0-beta.2 release.

#

Update - it appears this failure is not consistent. I am able to sometimes load the information and sometimes not.

Here is a quick output of my running the script a few times every couple seconds:

mootz12@Alexs-MacBook-Pro blend-utils % node ./lib/scripts/temp.js testnet
Result:  {
  latestLedger: '2537934',
  entries: [
    {
      lastModifiedLedgerSeq: '2535782',
      key: [ChildUnion],
      val: [ChildUnion],
      expirationLedgerSeq: 2777702
    }
  ]
}
mootz12@Alexs-MacBook-Pro blend-utils % node ./lib/scripts/temp.js testnet
/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1727
    if (this._arm !== _void__WEBPACK_IMPORTED_MODULE_0__.Void && this._arm !== armName) throw new TypeError(`${armName} not set`);
                                                                                              ^

TypeError: expiration not set
    at ChildUnion.get (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1727:95)
    at ChildUnion.get [as expiration] (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1836:25)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:636:104
    at Array.forEach (<anonymous>)
    at mergeResponseExpirationLedgers (/Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:626:135)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:230:24
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async temp (file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:164:37)
    at async file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:191:1

Node.js v18.18.0
mootz12@Alexs-MacBook-Pro blend-utils % node ./lib/scripts/temp.js testnet
/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1727
    if (this._arm !== _void__WEBPACK_IMPORTED_MODULE_0__.Void && this._arm !== armName) throw new TypeError(`${armName} not set`);
                                                                                              ^

TypeError: expiration not set
    at ChildUnion.get (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1727:95)
    at ChildUnion.get [as expiration] (/Users/mootz12/dev/blend/blend-utils/node_modules/js-xdr/lib/xdr.js:1836:25)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:636:104
    at Array.forEach (<anonymous>)
    at mergeResponseExpirationLedgers (/Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:626:135)
    at /Users/mootz12/dev/blend/blend-utils/node_modules/soroban-client/lib/server.js:230:24
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async temp (file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:164:37)
    at async file:///Users/mootz12/dev/blend/blend-utils/lib/scripts/temp.js:191:1

Node.js v18.18.0
mootz12@Alexs-MacBook-Pro blend-utils % node ./lib/scripts/temp.js testnet
Result:  {
  latestLedger: '2537938',
  entries: [
    {
      lastModifiedLedgerSeq: '2535782',
      key: [ChildUnion],
      val: [ChildUnion],
      expirationLedgerSeq: 2777702
    }
  ]
}

wheat mortar Nov 15, 2023, 7:54 PM

#

You can try migrating to stellar sdk but lets tell @vivid dragon also

vivid dragon Nov 15, 2023, 8:05 PM

#

lol wtf - that's so weird

#

it implies that the returned val is not an ExpirationEntry but there's a very explicit check beforehand that the key is a LedgerKeyExpiration 😕

wheat mortar Nov 15, 2023, 8:08 PM

#

Yep

weary coral Nov 15, 2023, 8:08 PM

#

the code snippet above should be everything needed to reproduce, FWIW.

wheat mortar Nov 15, 2023, 8:09 PM

#

I was just looking at the commits briefly but i dont see what would cause it. Can check again later when more time

#

Can u test it with stellar-sdk @ latest?

weary coral Nov 15, 2023, 8:11 PM

#

wheat mortar Can u test it with stellar-sdk @ latest?

Sure, gimme a few

vivid dragon Nov 15, 2023, 8:11 PM

#

trying to repro locally. can you try beta.4, first? should be easier than porting to the sdk right now (but you should do that, btw)

weary coral Nov 15, 2023, 8:11 PM

#

The error was on beta.4

vivid dragon Nov 15, 2023, 8:11 PM

#

oh you said beta.2

#

oh wait nvm

#

reading comprehension 🤦

#

what's the soroban rpc testnet url again 🙈

weary coral Nov 15, 2023, 8:13 PM

#

confusingly different than futurenet, lol -> https://soroban-testnet.stellar.org

vivid dragon Nov 15, 2023, 8:13 PM

#

lol yeah I remembered it was diff but not what it actually changed, ty!

#

ok I repro'd on third try

#

wtf it's a contract data entry

#

this makes no sense

weary coral Nov 15, 2023, 8:18 PM

#

Tested with "stellar-sdk": "11.0.0-beta.6" and am getting the same results

#

My guess would be some race condition between loading the expirationLedgerSeq and the contract data entry? Maybe the expiration is null in the failure case? not sure what else would make it fail randomly

vivid dragon Nov 15, 2023, 8:20 PM

#

bless up, thanks for checking that

#

(btw you should port to stellar-sdk permanently when you have time lol, soroban-client is gonna be deprecated)

#

(but that's an aside)

weary coral Nov 15, 2023, 8:21 PM

#

vivid dragon (btw you should port to stellar-sdk permanently when you have time lol, soroban-...

noticed this today, I'll get around to it soon.

vivid dragon Nov 15, 2023, 8:21 PM

#

the migration guide is in the release notes, also here: https://gist.github.com/Shaptic/5ce4f16d9cce7118f391fbde398c2f30

Gist

A migration guide for stellar-sdk@v11.0.0

A migration guide for stellar-sdk@v11.0.0. GitHub Gist: instantly share code, notes, and snippets.

wheat mortar Nov 15, 2023, 8:22 PM

#

Maybe theres an await missing

#

I just dont see any errors in the diff but maybe im not looking hard enough

#

Or ya lack of null handling but i dont see any handlers in the old not in the new

vivid dragon Nov 15, 2023, 8:24 PM

#

nah - the workaround we did was basically "insert lookups for the corresponding expiration key for each ledger key when you call getLedgerEntries, then inject the resulting expiration into the response as if it was alongside the original key"

wheat mortar Nov 15, 2023, 8:24 PM

#

Ah

vivid dragon Nov 15, 2023, 8:24 PM

#

the fact that it's returning a contractdatanetry instead of an expirationentry is pure bizarro world

#

here's the pr: https://github.com/stellar/js-soroban-client/pull/161

GitHub

Include expiration entry on ledger entries in getContractData/getLe...

This is a client-side approach to eagerly fetching the ledger entry expiration info(ExpirationEntry) automatically for any LedgerDataEntry requested through getLedgerEntries and getContractData, an...

wheat mortar Nov 15, 2023, 8:25 PM

#

Do update to release beta 12 :laughing

vivid dragon Nov 15, 2023, 8:25 PM

#

point is this should never happen

wheat mortar Nov 15, 2023, 8:25 PM

#

Interesting. Well looks like a fun problem to solve. I envy you?

#

Is that the pr where the error started?

vivid dragon Nov 15, 2023, 8:31 PM

#

well that's the PR that adds expiration info to getLedgerEntries requests

#

btw @weary coral there's a Server.getContractData so you don't have to build the entry yourself

#

(again an aside but just wanted to point it out)

weary coral Nov 15, 2023, 8:42 PM

#

Yeah I saw that, I like the getLedgerEntries as it lets me fetch a bunch of things in one rpc call.

wheat mortar Nov 15, 2023, 8:50 PM

#

Would it work to get ledger entries with the preflight and using a getter and then in simulation it will look up the key? Not on topic sorry

weary coral Nov 15, 2023, 8:52 PM

#

I don't expose any "frontend" getters on the pool to minimize the contract size, so some of this information is only accessible via directly reading contract entries.

wheat mortar Nov 15, 2023, 8:53 PM

#

Makes sense. But i struggle to figure out how to retrieve the key and the value in a single call?

#

Ill look at your repo probably has an example. Ive gotten so much help from your code you dont even know

vivid dragon Nov 15, 2023, 9:24 PM

#

whelp I'm almost certain this is an rpc bug

#

there appear to be instances in which key.type != val.type when there are multiple keys requested

#

yep, confirmed

wheat mortar Nov 15, 2023, 9:38 PM

#

Ordering issue?

vivid dragon Nov 15, 2023, 9:46 PM

#

I think rpc assumes entries are returned in the same order they're requested

#

see here: https://github.com/stellar/soroban-tools/issues/1084

GitHub

soroban-rpc: `getLedgerEntries` may interleave incorrect keys and v...

Bug Report What version are you using? Testnet, which I believe is v20.0.0-rc.4.1. What did you do? I requested two ledger entries: a ContractData key alongside its corresponding ExpirationData key...

wheat mortar Nov 15, 2023, 9:48 PM

#

We were discussing an ordering issue with scmaps the other day and them not matching the lexical ordering of scval.maps

#

I dont have a clue if it could be related bht just something thst popped in my mind

vivid dragon Nov 15, 2023, 9:48 PM

#

nah that's def unrelated here

weary coral Nov 15, 2023, 9:50 PM

#

vivid dragon see here: https://github.com/stellar/soroban-tools/issues/1084

I've noticed this as well. It would be great if the return values are always in the same order as the ledgerKey array for the JS function as a result of fixing this.

vivid dragon Nov 15, 2023, 9:51 PM

#

yeah I mean that would be ideal but that may be beyond the RPCs control

#

this is an actual bug where the val belongs to a different key

wheat mortar Nov 15, 2023, 9:53 PM

#

Did u make an issue for this bug mootz

craggy zodiac Nov 15, 2023, 10:55 PM

#

vivid dragon (btw you should port to stellar-sdk permanently when you have time lol, soroban-...

would you mind sharing why, or give an announcement on this with more information, please? it would be great. Ty!

vivid dragon Nov 15, 2023, 11:31 PM

#

sure! we'll write something more formal up but the basic idea is that many stellar apps need both horizon and soroban rpc, especially if they've been around since before protocol 20. so stellar-sdk will have everything you need to use the network. of course, if you just need soroban rpc, you'll still be able to use just those pieces. so this shouldn't affect anyone negatively (besides the small amount of effort needed to adapt your imports)

#

having them in separate places causes bloated, incompatible codebases for people who use both parts as well as confusion about why certain things are available in some places but not others

craggy zodiac Nov 16, 2023, 12:00 AM

#

It's a reasonable decision. Thank you for the explanation @vivid dragon

weary coral Nov 16, 2023, 3:11 PM

#

vivid dragon see here: https://github.com/stellar/soroban-tools/issues/1084

just double checking that this fix is targeted for the current testnet release? I know another release candidate is around the corner, so I was curious if this will make it in testnet before then.

wheat mortar Nov 16, 2023, 3:19 PM

#

weary coral just double checking that this fix is targeted for the current testnet release? ...

Its not a host env update so you can update your local rpc to implement it just fyi

#SorobanClient `getLedgerEntries` throwing error in 1.0.0-beta.4