#Fees consumed by Soroban - how much to worry about

1 messages · Page 1 of 1 (latest)

teal vine
#

Hello, I've been working on developing an Oracle contract (GitHub link) for Soroban over the past year. In my implementation, prices are stored in a single Map, which is a ledger entry stored in the "instance" storage. There are other implementations that store one price entry per ledger entry as a temporary storage key, but my question applies to both types of implementations.

After some tests, I noticed that a contract invocation call to lastprice(), one of the most critical functions of the contract and likely to be frequently called by consumer contracts, consumes between 200,000 and 290,000 stroops, equivalent to around 0.02 XLM. This is for Testnet.

I tested both my implementation and a few other Oracle implementations that use different approaches to storage. However, the lastprice() invocation consumes almost the same in all contracts.

These fees seem a bit high for just a simple lastprice() call, and I don't observe any implementation that can consume less than 0.02 XLM for invoking that function. Are these fees high because it's Testnet? Is there any estimation of how much this cost may increase or decrease when deployed to Pubnet? I'm a bit concerned about the usability of Oracles inside Soroban due to these high fees. It might become impractical.

prisma quarry
#

From what I see, you're reading prices for all assets even if you need only check the price for 1 asset. There is a read limit of 130 KB, so you'll exceed it quickly with actual data.
Also, maps consume a lot of CPU instructions in Soroban.

rose bone
#

After some tests, I noticed that a contract invocation call to lastprice()
how did you measure the cost? does this single invocation use about 20M instructions?
in any case, yes, fees are kind of high now. we might reduce them, but the write fees will have to stay pretty high in order to be on par with the classic write fees (we don't want to make populating ledger cheaper than on classic).
I'm also curious what would you consider a 'low enough' fee (just as a data point)
cc @spiral nimbus

spiral nimbus
#

I tested both my implementation and a few other Oracle implementations that use different approaches to storage.

Along with storage, also worth looking at what consumes cpu instr in that function like @prisma quarry mentioned.

Yeah I'm also curious about what fee amount would be 'low enough'. Like, why is 0.02 xlm too high?

rose bone
#

FWIW with the current network config tx that consumes maximum possible amount of instructions (100M) is 0.1 XLM. so chances are the costs are coming from IO/history etc. basically without considering the ledger writes, maximum possible transaction would cost about 0.16 XLM. writes really are currently up to several orders of magnitude more expensive (I've actually been thinking about offseting the write cost to rent costs, so that it's at least cheap to create a temp entry)

prisma quarry
rose bone
#

ah, sorry, you're right - I've missed one 0

#

FWIW I don't have a strong opinion on where we should start at. the end goal for non-write fees is to promote rational use of ledger resources, but if everyone thinks that, say, 0.01/max tx is already a lot, then this should be a sufficient fee. again, writes are trickier as we have a baseline of 0.5 XLM/classic entry created

spiral nimbus
rose bone
#

I think we should be able to start with generally lower fees. I'll work on a proposal to share - stay tuned

spiral nimbus
#

I'm working on a proposal as well! DM'ing you

prisma quarry
#

I just did some testing, and I have a question about write fee:
Let's say I'm storing a vector with 100 u64 numbers. Then, I'm calling a contract function that removes 10 numbers from the vector and saves it. Setting data to the same key doesn't increase the expiration ledger. So, what is the purpose of the write fee here if I'm decreasing the ledger size?

rose bone
#

that's a fair question. in reality, any ledger modification increases its size, even if it's an entry removal. the changes will eventually be merged, where 'eventually' might be a few months. current fee model takes a simplistic and conservative approach and considers every write in the same fashion (well, we don't account for some entry metadata, which is why complete removal of entry is free).

#

what I think we should do to address this issue is to offset write costs to rent costs, so that it's cheap to write/modify entry, but it's expensive to keep it around.

#

stay tuned for that as well

prisma quarry
#

That would be nice. Right now, it's cheaper to remove the key before setting it. In that case, you will at least bump the expiration.

rose bone
#

hmm, you're right

#

you do need 2 transactions for that though

#

so it's hard to argue what's cheaper. but this does seem like a weird/unintended pattern

prisma quarry
#

Why 2 transactions? This worked for me to bump the expiration. The difference between with/without remove is 16K CPU instructions.

Screenshot_2023-12-08_at_10.25.12_PM.png
rose bone
#

hmm, insns don't really matter much, I'm talking about the write fees which are computed on the 'final' results of your invocation (i.e. this vs direct set would result in the same amount of writeBytes charged)

#

the CPU diff is still counter-intuitive though

teal vine
#

my add_prices function is consuming 1.5 XLM if I remember correctly, and it seems that some other contracts also consume a lot of fees for adding prices, more than 1 XLM for bulk-add prices is too expensive to maintain, I might have to switch to other kind of oracle implementation that maybe stores prices off-chain

rose bone
#

I would suggest not rushing the changes as the futurenet/testnet fees aren't realistic. we've started doing fee tuning in preparation to mainnet. one of the main goals is to reduce the cost of operations that don't increase the ledger state size (so something like a price update should become cheaper). I wonder though what is (roughly) the maximum fee that you'd find usable for your project?

teal vine
# rose bone I would suggest not rushing the changes as the futurenet/testnet fees aren't rea...

We're currently operating with 10min resolution for the oracle, this means adding prices every 10min to the blockchain. We have a bulk-add function named add_prices, that can add up to 30 prices in a single contract call, we did that to save fees. If we call that function every 10min, and if every call costs 1 XLM for example, we'd be spending 144 XLM per day. Calling this "expensive" or "cheap" really depends on how much we can afford to maintain the Oracle. But atm for us is not viable to spend so much each day.

We're currently thinking about how we can make it less expensive. Maybe we'll increase the resolution. We're not sure yet.

#

Retrieving the lastprice() is not really an issue, as we're pretty much comparing ourselves to other Oracle implementations and 0.02 XLM is kind of the average for most calls. Our issue is adding prices.

#

We're storing all prices in a Map (and we're aware of the scaling limitations) in instance storage. We likely made a huge mistake implementing it like that, maybe storing each price in a temporary ledger entry is the right approach, as other Oracles did.

rose bone
#

well, what I'm interested in whether, say, 0.1 XLM update call or 0.01 XLM call would be feasible. of course the implementation can be changed/updated, but I'm just curious about this as a data point.

teal vine
#

I see, well, it seems like it can be made cheaper, is there a reason it's not cheaper already?

rose bone
#

using temp entries might be a better idea (it might also make calling your oracle cheaper if the client needs just a few entries). that said, it probably won't have an effect with the current (pretty arbitrary!) testnet fees

#

the reason is time - testnet 'economy' is not really representative of the real network, so we haven't spent much time on fee tuning until now.

teal vine
#

makes sense

#

0.01 XLM for a bulk-add operation seems fair to me

#

0.02 XLM max

#

but I have no idea if that's possible

rose bone
#

I see. thanks for the data point!

teal vine
#

out of curiosity, in terms of hardware costs and availability, is storage or CPU the most expensive (or "disputed") resource to use?

because if storage is the most expensive, maybe Soroban can make fees more expensive for storage-heavy contracts (super high fees for persistent storage, fair fees for instance storage, and ultra low for temporary storage), and less expensive for CPU-heavy

#

because imo CPU instructions are kind of inevitable

rose bone
#

it's a bit more complicated; the storage costs are significant and they have to be high from the get-go. everything else doesn't really have to be high as we're limiting the ledger capacity. the point is to have the resource fees high enough to discourage spam/bad actors for taking ledger capacity from legitimate dapps (and also to promote mindful usage of the ledger capacity in general)

#

(and again, the storage modification doesn't have to be expensive and we want to reduce it)

teal vine
#

right! that makes total sense

we'll, I guess there isn't much I can do own my side besides re-implementing with temporary storage and without the Map, then waiting for Prod launch and measure the fees, thanks a lot for the explanations!

#

ah and let me ask something I've been wanting to know for a super long time but was kind of ashamed to ask: how is concurrency handled in Soroban? if two transactions in same ledger try to change a balance of a token (via token contract call for example), what prevents duplicate balance consumption?

#

in other words, what prevents two instances of a contract altering the same ledger key without "mutex lock" logic

rose bone
#

FWIW we might have time to push the fee changes to testnet before mainnet launch.

#

in protocol 20 transactions are executed sequentially. but the concurrency support is already built into the protocol: every transaction has a footprint which indicates which entries the transaction can modify. then the transactions that modify the same key (or read it) will be scheduled sequentially

teal vine
rose bone
#

in both cases the sequential execution order is pretty arbitrary - it is shuffled based on the hash of every other transaction included into ledger. so if both tx1 and tx2 want to, say, arbitrage the same DEX trade, it's hard to predict who will get it

teal vine
#

I see, also there's the TRY_AGAIN errors I get once in a while, will that be likely common in Pubnet?

rose bone
#

the ledger capacity and thus effective will be low initially after mainnet launch (we do plan to increase it soon though). but your error actually suggests that you're sending several transactions from the same source account (which we don't allow). you could try using channel accounts instead

teal vine
#

understood, yes I'm familiar with channel accounts
I got those TRY_AGAIN mainly when trying to call add_price too many times in a row (in separate transactions)

#

but now that we have the bulk add_prices(), it doesn't happen often

#

I appreciate all the explanations and clarifications, it helped me A LOT
leaving now, thanks @rose bone