#Request for comments! Generating TS/JS bindings with Soroban CLI

86 messages · Page 1 of 1 (latest)

prime zinc
loud sage
#

It's really hard to tell when building tooling if this contract method is a write function or a read function

#

I really like that it generates a full npm module. Prevents us from being limited on what we can generate in these clients. 👏🏻

#

The fact that contract docs in the contract interface show up in the generated client is 😍.

dry ridge
#

maybe there is more value in introducing a method annotation that does address.require_auth() than we thought. still won't help with cases that need to e.g. read from storage though, but probably would cover a lot of common cases

#

is running preflight when doing tools like this not an option?

prime zinc
dry ridge
#

well, the benefit is that it's generic

#

annotations (even built into SDK) are optional

#

I guess more proper thing to ask would be why the generated client does not use preflight itself?

#

the recording auth mode is specifically made for building the signature payloads without the need of bespoke client code

loud sage
#

I don't think running preflight for generating bindings is an option, because running preflight would require making some assumptions about function arguments.

dry ridge
#

yeah, that makes sense. which is why preflight should be called from the client itself

coarse basin
#

@forest gulch

dry ridge
#

without preflight you can't set the right resources/footprint anyway, so without some Wasm interpretation/parsing, it doesn't seem feasible to me to auto-generate full-fledged clients without preflight

prime zinc
#

Got it, ok, I think this may have been a misunderstanding on my part based partly on how I saw other tooling handle requests. But maybe that tooling was designed with old auth in mind.

I'd like to see if I can rework the invoke logic to only sign if the preflight requires it. (I'll probably make separate threads/GH Issues with questions & feedback on how js-soroban-client currently handles this!) invoke logic currently lives here, if people want to read and leave other feedback on how to improve it: https://github.com/stellar/rs-soroban-sdk/pull/871/files#diff-8789e45f0f17ae6863f70c3cc1b28364c0def99bf52a3a030b04ae0294e055ecR42-R111

However, I think even with the current version of invoke and requiring some @signme comments, this is already a 10x improvement over the approach taken by soroban-dapp-example. Maybe we can:

  • merge this more-or-less as-is
  • make issues to remove the need for @signme comments
  • let early adopters know that this is an alpha version of soroban contract bindings ts, and that future versions may change significantly
  • after it's merged and we teach some early adopters about it, circle back and remove the need for @signme comments in future versions.
#

I'm also curious if people had any feedback related to other aspects of this design!

forest gulch
#

I love this!

Also +1 to death of silly comments!!

Ship it!

dry ridge
#

another important note on any sorts of annotations - these can't go deeper than the top-level call, so if you do not a token, but e.g. an AMM/DEX swap, you will lose the token transfer from signature payload. so this really can't scale without preflight

split crag
#

@prime zinc This is awesome! Few thoughts (hoping to get a chance to try this out on Blend soon)

  1. Is the package linked to a specific RPC / network? And, is the contract id also hardcoded in the package? I imagine this package will be used for both testnet and mainnet applications.

  2. Is this an NPM package per contract? Does it seem feasible to allow for a set of contracts per NPM package?

loud sage
dry ridge
#

no, I mean that client should talk to preflight by default

#

a bonus would be an ability to override and extend that behavior. e.g. allow adding keys to the footprint or override the whole footprint, but still use preflight to find out the resource consumption etc. probably this would be a separate library with some 'base client' though and generated client would inherit from it. not sure what are the best practices for ts/js libraries though, so this of course can be adjusted as needed

loud sage
#

Got it. So preflight wouldn't help us know statically if signing was required as an argument, it would help us know at runtime when calling the generated client if signing was required.

dry ridge
#

yeah, exactly. and it would tell what actually needs to be signed

rocky scarab
#

Wonderful, can't wait to test this out!

prime zinc
#

yeah, the invoke behavior can definitely 1. be improved over time and 2. be extracted to a separate library at some point, and these generated per-contract clients can depend on it. invoke itself could either be incorporated into js-soroban-client or, my preference, obsolesce it and go right from dev-friendly params like { to: 'G…' } to XDR encoded as Uint8Array in a new, lighter-weight library.

prime zinc
# split crag <@104558308068704256> This is awesome! Few thoughts (hoping to get a chance to t...
  1. Is the package linked to a specific RPC / network? And, is the contract id also hardcoded in the package? I imagine this package will be used for both testnet and mainnet applications.
    Yes, it's hard-coded for now. We went back and forth on this. I like the hard-coded approach for a couple reasons:
  • super simple starting point
  • testnet and mainnet versions of contracts sometimes (often?) have important differences
  • generating separate js bindings/libraries for each might be good enough
  • if they really are the same interface, you can override CONTRACT_ID, RPC_URL, etc using environment variables (see the example generated README in the video at 5:17: https://youtu.be/H-jV3ih0TJI?t=317)
  1. Is this an NPM package per contract? Does it seem feasible to allow for a set of contracts per NPM package?

Yes, we think it makes sense to start with a separate NPM package per contract. They're pretty lightweight packages, so using several of them in one app seems fine enough, as a starting point. Down the road, it might make sense to expand the bindings command to work with multiple contracts.

prime zinc
loud sage
#

Is there a way to generate not with it being deployed? So that folks can generate the bindings before deploying, and test bindings against local quickstart?

#

Or so that we can generate bindings in CI, without running a network in CI.

forest gulch
prime zinc
# loud sage Is there a way to generate not with it being deployed? So that folks can generat...

yes, that's actually the only thing that's supported right now. See 1:16 in the video for the current command we're using https://youtu.be/H-jV3ih0TJI?t=76

./target/bin/soroban contract bindings ts \
  --wasm ./target/wasm32-unknown-unknown/release/abundance_token.wasm \
  --id 2c6c3b8ba9923d029d8ef7eb80080384b1da32bcf0698290119fdfbf3f2a01de \
  --root-dir ./target/js-clients/abundance \
  --contract-name abundance-token

However, there's currently a bug: we don't allow specifying the RPC_URL or NETWORK_PASSPHRASE in any way! they're hard-coded into the template. That's obviously a blocker for initial release.

loud sage
#

It would be interesting if we found a way to flag functions are "View" functions, like how Solidity supports, and then even prevent calling methods that modify state. I think we might be able to do that in Rust, but it would probably be very complicated. How we'd need to do it is the Env would be different, then every SDK type would need to be generic/parametized over the two types of envs, the one that allows modifying state or not.

loud sage
forest gulch
#

What's the underlying logic for the @signme comment @prime zinc? Does it signal to the client something that calling the footprint doesn't?

loud sage
dry ridge
#

that would be great to have actually! do note though, that the sets of read-only and read-write functions and sets of auth-required and no auth required fns are not equivalent

prime zinc
# forest gulch What's the underlying logic for the `@signme` comment <@104558308068704256>? Doe...

The tooling that we took our lead from all have a signTransaction: boolean argument exposed to the app, making it the app's job to decide whether or not to sign the transaction. So we just went with this approach, which meant that we needed to know at build time if a given method requires signature.

However, the discussion here helped us realize that we can do this at runtime by inspecting the preflight data. See my earlier comment about this #1110323110277828639 message

dry ridge
#

there can be RO functions that require auth (e.g. consider an oracle that only works for 'subscribers' - I've seen an idea like that around) and there are RW functions that don't require auth

forest gulch
prime zinc
forest gulch
prime zinc
# forest gulch Tossed as in tossed before shipping v1

Unexplored, i guess. need to spend some time with it. if you know of an example app/tool that inspects the preflight to decide whether to sign, and then updates the transaction with the signature before calling sendTransaction, send it to me! That'll speed me up a lot.

forest gulch
#

I do not though I think the platform team may be working on some apps/interfaces/examples for stuff like this
cc @naive hill any insight here into the message above?

#

I assume it's as "simple" as inspecting the preflight response to see what's included in the auth array?

dry ridge
#

I suppose there needs to be the respective wallet support to sign contract auth instead of the whole tx

#

(when needed, that is)

naive hill
split crag
# prime zinc > 1. Is the package linked to a specific RPC / network? And, is the contract id ...

if they really are the same interface, you can override CONTRACT_ID, RPC_URL, etc using environment variables
I imagine most dApps want to be able to change their RPC URL on the fly, for example maybe to whatever RPC Url the connected wallet is using.

AFAIK for a statically generated site I don't think these env values can be easily update. I'm not a web dev, so maybe some1 can tell me otherwise

Yes, we think it makes sense to start with a separate NPM package per contract.
Cool, I'm OK with this! Will likely be easy to take the generated code to create a single package if necessary anywho.

prime zinc
#

I imagine most dApps want to be able to change their RPC URL on the fly, for example maybe to whatever RPC Url the connected wallet is using.

AFAIK for a statically generated site I don't think these env values can be easily update. I'm not a web dev, so maybe some1 can tell me otherwise
Good point. Environment variables are a build-time concern for static frontends, so updating them requires a redeploy. They will be overridable on a per-call basis, but that requires a lot of passing around rpcUrl and networkPassphrase arguments that imo ought to be global most of the time.

Maybe getting them from the connected wallet might make sense, and then app authors would switch contract ID based on network (or display an error). Given that we're already hard-coding the reliance on Freighter, maybe we just use that for now? We'll think about this more and see how we can improve these ergonomics in a follow-up version, but I think the current environment-variables-based version is good enough for the first release?

Will likely be easy to take the generated code to create a single package if necessary anywho
For sure 👍🏼

prime zinc
loud sage
#

@dry ridge Do you know what that is ☝🏻 ?

prime zinc
#

The footprint is also inscrutable:

AAAABAAAAAYsbDuLqZI9Ap2O9+uACAOEsdoyvPBpgpARn9+/PyoB3gAAABAAAAABAAAAAQAAAA8AAAAIRGVjaW1hbHMAAAAGLGw7i6mSPQKdjvfrgAgDhLHaMrzwaYKQEZ/fvz8qAd4AAAAQAAAAAQAAAAIAAAAPAAAABVN0YXRlAAAAAAAAEwAAAAAAAAAA0rvDCRTln3A2NXcPgyxFoQnEq7bnr4NKJzCFIC7AprMAAAAGLGw7i6mSPQKdjvfrgAgDhLHaMrzwaYKQEZ/fvz8qAd4AAAAUAAAAB0lIvizZq7gW/6EfFG2vCNvJB5WZCLD3IgIUYb6o4BqKAAAAAQAAAAYsbDuLqZI9Ap2O9+uACAOEsdoyvPBpgpARn9+/PyoB3gAAABAAAAABAAAAAgAAAA8AAAAHQmFsYW5jZQAAAAATAAAAAAAAAADSu8MJFOWfcDY1dw+DLEWhCcSrtuevg0onMIUgLsCmsw==

Can't figure out how to decode that in the Stellar Laboratory at all.

dry ridge
#

auth is ContractAuth

#

footprint is LedgerFootprint

#

both work just fine for me in the lab (with Futurenet, of course)

coarse basin
#

hey i just watched your video... this was awesome. Soroban needs more video content, I'm glad you published it.

prime zinc
dry ridge
#

not sure I understand

prime zinc
prime zinc
# dry ridge not sure I understand
  • get rid of @signme comments
  • if preflight returns an auth array, sign the whole transaction
  • otherwise, return simulated results

my question is on the second step. is signing the whole transaction the best way to do it, or is there a more specific thing to sign?

dry ridge
#

get rid of @signme comments
sure
if preflight returns an auth array, sign the whole transaction
not necessarily. ContractAuth entries do need a signature to be filled in. if addressWithNonce is missing, then it's invoker auth and can be signed as a part of tx. otherwise, the respective Address has to sign the payload corresponding to ContractAuth (you can find more details in this doc: https://soroban.stellar.org/docs/how-to-guides/invoking-contracts-with-transactions#authorization-data)
otherwise, return simulated results
not sure what does this have to do with auth necessarily. the client may return preflight results which sometimes might be relevant, but in general there should be a way to send the tx to the network

#

also, in the next release we're going to revamp xdr quite a bit (there are no plans to change the general principle though)

coarse basin
loud sage
# prime zinc The footprint is also inscrutable: ``` AAAABAAAAAYsbDuLqZI9Ap2O9+uACAOEsdoyvPBp...

@prime zinc The XDR you posted decodes okay for me using the stellar-xdr cli (cargo install --locked stellar-xdr --version 0.0.16 --features cli):

echo -n 'AAAAACxsO4upkj0CnY7364AIA4Sx2jK88GmCkBGf378/KgHeAAAACXRva2VuX3BsegAAAAAAAAEAAAATAAAAAAAAAADSu8MJFOWfcDY1dw+DLEWhCcSrtuevg0onMIUgLsCmswAAAAAAAAAA' | stellar-xdr +next decode --type ContractAuth --output json-formatted

echo -n 'AAAABAAAAAYsbDuLqZI9Ap2O9+uACAOEsdoyvPBpgpARn9+/PyoB3gAAABAAAAABAAAAAQAAAA8AAAAIRGVjaW1hbHMAAAAGLGw7i6mSPQKdjvfrgAgDhLHaMrzwaYKQEZ/fvz8qAd4AAAAQAAAAAQAAAAIAAAAPAAAABVN0YXRlAAAAAAAAEwAAAAAAAAAA0rvDCRTln3A2NXcPgyxFoQnEq7bnr4NKJzCFIC7AprMAAAAGLGw7i6mSPQKdjvfrgAgDhLHaMrzwaYKQEZ/fvz8qAd4AAAAUAAAAB0lIvizZq7gW/6EfFG2vCNvJB5WZCLD3IgIUYb6o4BqKAAAAAQAAAAYsbDuLqZI9Ap2O9+uACAOEsdoyvPBpgpARn9+/PyoB3gAAABAAAAABAAAAAgAAAA8AAAAHQmFsYW5jZQAAAAATAAAAAAAAAADSu8MJFOWfcDY1dw+DLEWhCcSrtuevg0onMIUgLsCmsw==' | stellar-xdr +next decode --type LedgerFootprint --output json-formatted
#

The auth (first command above) outputs:

{
  "address_with_nonce": null,
  "root_invocation": {
    "contract_id": "2c6c3b8ba9923d029d8ef7eb80080384b1da32bcf0698290119fdfbf3f2a01de",
    "function_name": "token_plz",
    "args": [
      {
        "address": {
          "account": {
            "public_key_type_ed25519": "d2bbc30914e59f703635770f832c45a109c4abb6e7af834a273085202ec0a6b3"
          }
        }
      }
    ],
    "sub_invocations": []
  },
  "signature_args": []
}
#

The footprint (second command above) is too long to post here.

prime zinc
# loud sage The auth (first command above) outputs: ```json { "address_with_nonce": null, ...

Is there a way to get that nice structure in js-soroban-client? I'm trying this: ```js
soroban.xdr.ContractAuth.fromXDR('AAAAACxsO4upkj0CnY7364AIA4Sx2jK88GmCkBGf378/KgHeAAAACXRva2VuX3BsegAAAAAAAAEAAAATAAAAAAAAAADSu8MJFOWfcDY1dw+DLEWhCcSrtuevg0onMIUgLsCmswAAAAAAAAAA', 'base64')

But the object it returns is much less readable & harder to work with than the simple JSON structure the `stellar-xdr` CLI returns.
loud sage
restive carbon
#

we could (should?) create a higher-level abstraction in js-stellar-base (for this and other Soroban-related constructs), but that kind of work hasn't been on the roadmap. in general, though, xdr -> json is not really a thing (though you aren't the first to request it: https://github.com/stellar/js-xdr/issues/25)

GitHub
Issues · stellar/js-xdr

Read/write XDR encoded data structures (RFC 4506). Contribute to stellar/js-xdr development by creating an account on GitHub.

loud sage
#

The Rust lib has it, but it isn't canonical, or easy to represent identically across sdks, which I think is the main problem. It would take significant effort to have all SDKs generate the same JSON.

dry ridge
#

I would also very much enjoy an SCVal<->JSON library, as even given the XDR library building SCVals is quite painful

bleak jasper
#

Yeah, generating the jsonToScVal and scValToJson conversions into the ts bindings is fine for now, but it would be great to generalize and extract these into js-stellar-base so they are re-useable

#

Just catching up on this. Awesome video, and super nice with the typeahead and docs. 🎉 Dumping a bunch of thoughts below.

#

It would be nice to see better support for running with a custom: rpc url, network passphrase, contract id. Particularly for testing stuff locally where contracts will have a different address than on futurenet/pubnet. A couple random ideas in that vein:

Maybe generating a:

class Contract(contractAddress: Address, {rpc_url?: string, network_passphrase?: string}?: Options)
// type thing, which you'd use like:
let abundance = new abundance.Contract(env.ABUNDANCE_CONTRACT_ID, { rpc_url: "http://localhost:8080/..." });

You could augment this for convenience by exporting a pre-instantiated version of the contract, so users could just do:

import { futurenet } from "abundance-token";
await futurenet.symbol();
#

I guess fees, source account, and other options like that, would be handled with a second optional param?

await token.balance({of: "G123...ABC"}, {fee: 200})
#

There's also some broader questions around who should be the one controlling the RPC_URL, the dapp or the wallet. Because that is very related to who users should trust to simulate & submit their txns. but that's probably outside the scope of this

#

FYI, as a side note. I've also been considering making network_passphrase in rpc clients an optional setting. soroban-rpc now has getNetwork which gives you the network_passphrase which is expected. Seems cleaner to just fetch that from the RPC. But I'm not 100% on the security implications of that yet.

#

It might be possible to simplify the example a bit by:

  • use off-the-shelf soroban-cli (once that is available, duh)
  • don't use just, just use npm. It's one less tool to install & learn. But might make the npm stuff gnarly, not sure...
prime zinc
prime zinc
bleak jasper
#

also, fwiw I agree with @forest gulch, we should ship this as soon as you're comfortable, @prime zinc . It's miles better than what we have now, and we can start getting real feedback