Contract Metadata

[gilded osprey]

Where we are with Contract Metadata. Note that there will be two separate types of metadata - one for the contract code, and one for the contract instance.

For Code Metadata (which will live in a WASM custom section) we'll need SDK support to compile information in. We could also save space by just compiling in the metadata hash, and the contract developer could distribute the metadata off chain, but this is a choice for the contract developer to make.

For Instance Metadata, we'll have to store additional information in the executables ContractData. The ContractExecutable xdr can be updated to include an optional metadata field. This does mean that metadata needs to be provided when the contract instance is created. The host can fill in metadata for the SAC executable. This metadata should contain read-only information relevant to the contract instance like token symbol and the number of decimals.

Some questions we still need to answer -

1. What happens if a contract is nativized? What about the SAC's metadata?
2. Should Instance metadata be mutable if we allow instances to be mutable for proxy support?
3. What should the structure of both Code and Instance metadata look like? Arbitrary key value pairs? Do we even enforce a structure?

@dusk seal @pulsar valley

[dusk seal]

for some generic metadata in a vacuum only minimal structure is required, similarly to the events. but if we actually want to use something on-chain, then we need to do someting more specific

2 is an interesting question; I think that the instance metadata should be set in the constructor and constructor should probably run just once. alternatively, we could just make the instance metadata mutable - I suppose this depends on how exactly do we anticipate it to be used

[gilded osprey]

The instance metadata should only be accessible by the instance itself (for example,decimals() will retrieve the value from metadata.). Are there any other on-chain use cases?

[dusk seal]

you can skip decimals as function as use metadata instead...

(unless we want attributes like decimals to have custom logic, that is)

I suppose a key val interface still works though...

[gilded osprey]

That's true, but contracts could still have functions that return information from metadata. I guess the decimals example isn't great because it's not that useful on chain, but some contracts may still want to call it.

Unless you're saying we should allow contracts to access metadata for any other contract?

[dusk seal]

yeah, that's something I'm not sure about

if we expose this in the interface, might as well expose it as metadata

(and it is usable on-chain now)

[gilded osprey]

Yeah I guess we could expose the metadata, but now the format would then be part of the interface. That sounds like a good thing as it'll standardize the metadata, but I think the contracts will have to validate the format (the token init call will fail if the format isn't followed.

[dusk seal]

well, the same is true for the contract interfaces in general

it is trickier from the static verification standpoint though, but still testable

so if we just store a map in the contract instance entry, we should resolve question 1

(and if the code metadata is just a wasm section; it doesn't need any protocol support and can just be implemented in the sdk)

for mutation, I think it would be easier to just make it mutable, otherwise we would need to implement some tricky rules without much benefit

(unless there is some strong requirement for immutability, that is)

and for 3 the answer is the same as for 1 - just store a map in contract instance entry

this doesn't provide any way for implementing fast on-chain getters (like get a token balance without spinning up a vm), but I'm not sure how this can be done in general

[boreal horizon]

maybe a wild idea: should instance metadata even be a first class feature in the environment, or can this be an ecosystem standard that defines a canonical key and entry layout for the metadata ledger entry?

(we can add convenience getters/setters in both in the contract sdk and in the dapp sdk)

[dusk seal]

it has to be the first class feature to some degree

[boreal horizon]

why?

[dusk seal]

because the only way to store the data currently is in the contract data entries, which aren't readable by another contracts

so the 'metadata' now has to be implemented via contract functions (like decimals() in the token interface)

if we add a map the the contract instance entry, we have the following benefits:

• contract usage will be cheaper for the users (e.g. token pair for the AMM will be automatically fetched together with the contract instance)
• the map can be shared with other contract without spinning up a vm (if needed)
• off-chain processors know where to look for it

[boreal horizon]

ok. I understand you. Maybe we need to align on the instance metadata use case because the main thing I had in mind was for downstream systems to be able to consume this information without having to run any wasm code. How common do we think the contract-to-contract use case is?

token pair for the AMM will be automatically fetched together with the contract instance
what do you mean here?

[dusk seal]

there is also the contract-to-itself use case...

I mean that there are certain contract attributes that are conceptually immutable and are always needed for its operation, such as token pair for an AMM

if we go for off-chain only approach, then I really don't see a point in doing anything

[boreal horizon]

you mean the actual pool values or the token contract addresses?

[dusk seal]

I mean the ids of tokens that are being swapped, not the balances

[dusk seal]

doing anything on-chain, that is

[dusk seal]

I don't think that's a bad approach either - this will allow us to spend more time on another features/polishing

[gilded osprey]

I don't think optimizing for cross-contract retrieval of metadata is important, and I don't think the cost benefits mentioned here will matter much given that most contracts will not be retrieving constant read-only data from other contracts. If that information should be exposed, a contract function is fine. I think a much more general solution would be to just let contracts read data from other contracts to bypass VM calls, but that's a different discussion.

If we assume any contract will have at most one metadata entry, we could just provide a new ScVal key (similar to ScVal::LedgerKeyContractExecutable) to make it easy to look up metadata for a contract and let the ecosystem determine what should be stored in that entry.

[dusk seal]

a contract function is fine
contract functions are actually really expensive (much more expensive then every other operation), so reducing the number of vm calls is one of the most relevant optimizations out there. that said, I agree that 'flat' keys don't seem that useful, compared to 'multi-value' keys (like balances).
FWIW reducing the number of ledger keys used is not completely irrelevant either - that reduces the footprint and likely reduces IO. but that's not critical either

we could just provide a new ScVal key (similar to ScVal::LedgerKeyContractExecutable)
I don't think we need to do that. we special-case only the keys that are meaningful for the protocol. I don't think there is any benefit of special-casing this for the off-chain usage. it's not really any better than, say, agreeing that instance metadata is stored in Symbol("METADATA") entry or something like that (that also allows for some flexibility and allows ecosystem to come up with more standards).

implementing the special key would require some host and SDK machinery. I don't think it's worth the time investment, given that we can't force developers using any particular key and we can't force them to use any specific value structure for that special key

[gilded osprey]

I agree that VM calls are expensive. My point was that I think a VM call to retrieve metadata will be rare so my opinion was that there's no need to optimize for this.

Yeah an agreed upon key works as well. I was just thinking that the host and SDK machinery for this would make metadata easier to use.

[dusk seal]

well, we still need to introduce the helpers for the events at the sdk level and I think that's a good candidate for a library that exposes such 'standardized' constants

like just do pub const CONTRACT_METADATA_KEY: Symbol = Symbol::short("METADATA");

the metadata structure needs to be agreed upon on a per-contract interface basis though

so maybe it only makes sense to expose event and metadata that SAC uses

creating a special xdr key is much more involved

[gilded osprey]

So to summarize the option discussed here for Instance Metadata - add no explicit protocol support, but add tooling in the sdk to allow writing a specific metadata entry to some key like Dima mentioned for the token interface. Metadata format for other contracts can be determined by an ecosystem standard. This would confine all metadata work to the SDK.

The one drawback is that metadata can only be accessed on chain through cross-contract calls, but I think that tradeoff here is fine since we don't expect metadata to be accessed on chain that often.

[dusk seal]

that sounds reasonable. one downside we haven't mentioned is the state expration - if the metadata is for off-chain use purely, then there is no good way to bump the rent for it. bundling it with the contract instance entry would help solving that issue as well.

[gilded osprey]

Isn't that a problem that needs to be solved for all non-user owned data in contracts like the admin entry? I was thinking that whatever solution is used there would be used for these entries as well.

[dusk seal]

I think it's a bit different here; admin entry is useful on-chain (otherwise, there wouldn't be one), while off-chain metadata isn't useful at all and chances are won't even be readable by another contracts

which means we never touch it at all on-chain

there is of course a case of the admin entry that's never touched either, but it probably would appear less frequently; for metadata we explicitly have something not intended for on-chain use

[gilded osprey]

I think this will be a more general problem with rarely-touched entries as well like you mentioned, but yeah an entry that is never touched on chain is a little different. @icy fable would be interesting to get your thoughts on how your rent interface could be used to pay rent for an entry that is never accessed on chain. Maybe the contract developer can tie certain entries to the contract instance so rent is topped of together?

[dusk seal]

that would be trivial to achieve if we used the approach with a small map inside the contract instance entry

[gilded osprey]

Yeah that's one way to do it

[icy fable]

Yeah in the current rent proposal, contract instance rent is automatically bumped on each access, with the intention being that any shared or important state be stored in the contract instance map

Otherwise the contract would need to define a rent top up function and explicitly call it to make sure meta stays on the ledger, since right now all rent bumps are handled whenever the entry is accessed

[gilded osprey]

Your proposal already has this concept of a "contract instance map"?

[icy fable]

I don't specialize for a "contract instance map", but I make a few mentions for it. In particular for things like auth data or shared state that's used by many different accounts, a map inside a contract instance would be very useful from a rent perspective

[dusk seal]

alternatively, it could be a flag that makes entries to point at the contract rent pool. the benefit is that there won't be data size limits and that the storage can be used in the same fashion as the regular storage

[boreal horizon]

@dusk seal It sounds like what you're describing here is more of a per instance "global state" than it is "metadata". I'm a bit worried of presenting so many different types of storage for developers

[dusk seal]

hmm, I think the current idea is actually to not special-case the metadata on-chain

with the main issue being the ledger rent, which we discuss here

(this issue indeed has to be solved for the 'global state' - again, metadata is not that special here, besides the potential lack of on-chain access)