Does o1js(kimchi) support wasm64?

[copper vale]

Nodejs can already enable wasm64 support through the experimental option. Can o1js support wasm64? According to my testing it doesn't work, so not sure if I'm missing something.

If I am not mistaken, I understand that many zk applications developed on mina will be rollup-based applications, but the two major obstacles to development based on o1js are circuit size limitations and wasm memory limitations, especially the wasm memory limit. Even if my circuit is far from reaching the circuit size limit, there will still be a high frequency of insufficient wasm memory. This makes many current rollup-based applications unavailable. We will probably only be able to develop an experimental application based on rollup with very low throughput.

So I think this issue is very important and urgent. If wasm64 is not currently supported, does the team have any plans to move forward to solve this problem? Looking forward to the team's reply. It is worth mentioning that many of my ideas were stalled by the memory limitations of wasm, even if I tried my best to optimize circuits.😅

[tame trout]

i have never ran into insufficient memory issues with nodejs with circuits right on the edge of 2^16 constraints 🤔 but i dont remember doing anything specific for my node configuration to make it work. I’m on a 64gb ram machine tho

[quick pond]

wdym support wasm64?

it's written in JS

you could compile snarky to wasm though

oh for kimchi

if you're hitting the memory limit you should probably be splitting up the circuit IMO

but adding wasm64 support is probably trivial

[tame trout]

@quick pond there’s already a 2^16 constraint limit for circuits, and i had no issues memmory wise with the largest circuits

[quick pond]

yeah but it will be raised

[copper vale]

This has nothing to do with the memory size of your machine. The existing wasm32 supports up to 4gb of memory.
According to my tests, when the circuit size approaches the limit, the wasm behind o1js will easily become unstable.
Even if your circuit compiles normally, there is a certain probability that wasm-related errors will occur when you execute proof generation in the same node process, and wasm memory usage will increase. If you do some stress testing and perform proof generation multiple times in the same process, it is possible to discover the instability of wasm, this has a lot to do with insufficient memory used by wasm.

[tame trout]

yeah thats a sound argument, just didnt happen YET in my personal experience with 2^16 constraint circuits

[copper vale]

Does o1js(kimchi) support wasm64?

[copper vale]

Thank you for your clarification. Maybe it would be more appropriate to change the question to whether kimchi supports wasm64.
I think it makes sense to reduce the circuit size under the current o1js implementation by splitting the circuit and recursively generating proofs, but this does not reduce the memory footprint of wasm to a large extent.

In the current o1js implementation, if I split a large circuit into several circuits, there will be dependencies between several circuits, which will result in the need to compile all the circuits in one process. After the compilation is completed, the memory usage of wasm will increase very quickly.
Later, you will find that when executing proof generation in the same process, there is a high probability of encountering errors caused by insufficient wasm memory. We currently cannot support normal throughput applications with common business logic, and I think adding wasm64 support is an obvious solution to the problem.

[copper vale]

Thank you for your reply. I am interested in your use case. What does your application mainly do? Have you conducted any performance testing?

[tame trout]

yeah it was a circuit that calculated root hashes in large merkle trees, barely could fit into 2^16 constraints, i ran it many times with no issues tho

[copper vale]

Have you tried compiling the circuit in a persistently running background process and continuously accepting the task of generating proofs in the same process (instead of restarting a process each time to compile the circuit and only handle the task of generating proofs once).

[tame trout]

i merely had it run once in a test compile -> prove -> exit

[copper vale]

So this is the reason why you basically don’t encounter wasm problems. When we run rollup circuits in the production environment and process the generation of proofs, it is not a one-time processing model.If each process can only stably handle the generation of a proof once, such availability cannot meet the requirements of a production environment.

[quick pond]

sounds like a memory leak

look at the bindings repo

should be trivial to change to wasm64

[copper vale]

It is indeed possible that such a problem exists, but currently the memory occupied by compiling the entire application circuit (which has been split into several dependent circuits) in the same process is indeed very large. It is easy to encounter the situation where wasm does not have enough memory to generate proofs in the same process. Many existing zk-rollups consume very high amounts of memory, which is something that cannot be avoided at present.

[trim haven]

Yes, I've already encountered the exact issue you are describing:
#1139330728384024577 message

Either while compiling too large circuits, or by keeping generating proofs within the same process and eventually reaching the inherent wasm32 memory limit. Often there are workarounds to these limits, but these workarounds can be ugly and inefficient (and sometimes there may not even be workarounds at all), so getting wasm64 would really power up zkApps, although I'm not sure if wasm64 is production ready yet (plus changing it for my own project would probably result in multiple compatibility, security, and other unexpected issues, not to mention the extra work associated with deviating from the standard Mina stack)

[frigid geyser]

My first question would be does Rust even have an option to compile to wasm64 currently?

[quick pond]

it does AFAIK

[copper vale]

As far as I know, rust has made preliminary support for wasm64, which is documented in the following link: https://doc.rust-lang.org/rustc/platform-support/wasm64-unknown-unknown.html. However, please note that the provided documentation may be outdated, and there might be more recent and improved support available now.

Here are some time estimates for wasm64 implementation progress: https://github.com/WebAssembly/memory64/issues/36

Due to the current uncertainty about the implementation of wasm64 in browsers, I wonder if it is feasible for us to use wasm32 for the circuit running on the browser client and use the experimentally supported wasm64 for the rollup circuit running on the server.

[quick pond]

simpler solution:
compilation caching and/or vk sideloading

[copper vale]

I fully agree that vk sideloading is a valuable approach to mitigate the memory usage of wasm (although it addresses the symptoms rather than the underlying issue), but I'm not sure to what extent the compilation caching (you mean serializing the prover keys?) you mentioned can alleviate this problem.

[trim haven]

And I think this wouldn't address the case for when you reach the memory limit for simply creating multiple consecutive proofs 🤔

[quick pond]

it does

[copper vale]

Yes, the solutions mentioned above can only alleviate this problem.

I made a brief investigation and found that the wasm-related packaging and compilation tools currently used by kimchi do not seem to support wasm64, so switching to wasm64 may not be an easy task. With the hard fork approaching, this may have a huge impact on the adoption of o1js, and developers cannot do much under this restriction.

[frigid geyser]

I wouldn't be so pessimistic 😄 we have other ideas for addressing the memory restrictions.

• the crypto team is working on "chunking" which is a way to prove an arbitrarily large circuit using an SRS restricted to the current size 2^16
  • this should lower memory reqs for large proofs a lot

• we are thinking about lowering memory reqs by only loading part of the SRS into memory and leaving the majority on disk

• for the nodejs environment, we are considering exposing bindings to a native prover. I.e. you'd get the native Rust binary which would generate proofs much faster and wouldn't be bound by any of the memory restrictions of Wasm. This is arguably much more powerful than moving to wasm64

because of the last point and the fact that wasm64 doesn't currently apply to the browser, I don't see wasm64 as an urgent milestone

[quick pond]

Does it not work in the browser? I thought it did but just didn't support more than 4G

[frigid geyser]

wasm64 hasn't shipped in the mainstream release of any major browser, see https://webassembly.org/roadmap/
also the whole reason for the 4GB restriction is that memory addresses are limited to 32 bits. 32-bit numbers only go up to 4 billion, hence 4GB. this would be solved with 64-bit memory addresses

[quick pond]

Seems like it has?

it says i64 is supported

well, what does wasm64 even mean

it was as I thought

i64 but no more than 4GiB of memory

[weak delta]

Memory64 is what you want to look at https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md

i64 is just 64 bit integer

[frigid geyser]

"i64 is supported" is not the same thing as "you can address memory locations with an i64". wasm64 means the latter

in current wasm, there are certain opcodes for loading and storing memory, which take a memory address as input, which has to be an i32

with wasm64 we mean the proposal to also let it be an i64. on the roadmap page it is called "Memory64" and it hasn't shipped in any browser

[brisk valley]

hi， @frigid geyser , which solutions are on going? any time schedules on these listing solutions? which solution could be carried out before hardfork??

actually we Anomix team also are meeting same challenges as @copper vale
https://github.com/o1-labs/o1js/issues/1135
https://github.com/o1-labs/o1js/issues/1088

[frigid geyser]

@brisk valley chunking is ongoing, native prover is in the early planning phase but should get in before hard fork

[copper vale]

Awesome!👍 These solutions are unlikely to be launched in the near term. So, is there a timeline for the implementation of vk sideloading? Can it be supported in the short term? The project I'm currently working on is hindered by the issue of insufficient wasm memory, and supporting vk sideloading should help alleviate this problem.

[frigid geyser]

how would sideloading help?

[copper vale]

I split the circuit into several sub-circuits. There are dependencies between them, and all circuits need to be compiled in one process. I think this is the main reason for the lack of memory in wasm.

If vk sideloading was supported, I wouldn't have to compile so many circuits in one process, so a single process might have enough memory to compile the circuits and perform proof generation

Looks like I might miss the latest mina-navigator hackathon 🥲

[quick pond]

you'd need compilation caching too

well maybe not

you could make the proofs in separate processes

and communicate the intermediate proofs across

[copper vale]

Based on my understanding of the current implementation of o1js, I think what you said is not feasible. As long as these proofs have dependencies, their corresponding circuits must be compiled in the same process.

[frigid geyser]

yes, provers are functions

[quick pond]

You always can. How does Mina verify proofs uploaded by others? You can serialise the proof.

you can export the vk

[frigid geyser]

yeah but that's also because Mina uses side-loaded vks

[quick pond]

shouldn't that be easy to support in O1JS?

[quick pond]

@copper vale you can always just use OCaml snarky directl

y

it's not that bad actually

[copper vale]

I don't know much about ocaml, and it's difficult to find collaborators using ocaml 😅

[copper vale]

@frigid geyser Hey! It looks like we are getting close to the time for Mina's hard fork, so may I ask if there is any progress with the native prover?

[frigid geyser]

Sure! There is no progress on native prover, we chose to focus on other issues to fix in time before the hf