WASI 0.2 completely lacks friendly documentation

[blissful cargo]

WASI completely lacks any good documentation on how to use it directly. I don't care about C++/Rust or whatever, I just want to play around with it directly in WebAssembly Text Format and perhaps eventually create a library/framework/programming language that compiles to WASI. Even for preview1, the only somewhat official resource I could find was this hello world example in the wasmtime docs: https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-tutorial.md#web-assembly-text-example
What would be the equivalent code for a hello world in WASI 0.2, in WebAssembly Text Format, and ideally without using the component model? Once I know how to "convert" the wit definitions to actual WASM imports & calls, I'm good, but at the moment I can't even figure out how to make a simple hello world!

[cosmic swift]

Yeah, you're right. We don't have enough documentation for working at this level yet.

One thing you can do if you want to see what the core-wasm imports look like is to use wit-bindgen with eg. the Rust or C bindings generators, like wit-bindgen rust test.wit, and you can find the Canonical ABI calls being generated inside the bindings.

[blissful cargo]

I tried out a ton of things now, but I still can't get WASI 0.2 to work in Wasmtime without using the component model. Imports like

(import "wasi:clocks/[email protected]" "now" (func $now (param i32)))
```all fail with an `unknown import: 'wasi:clocks/[email protected]::now' has not been defined` error. Am I missing something, or does Wasmtime simply not yet support executing WASI 0.2 core wasm files? The component model is extremely complex (especially with a non-existent documentation) and overkill for my use cases and is the opposite of what I liked about preview1 (simple import functions that don't reinvent the wheel). Also, adapting a simple hello world WASI preview1 module to preview2 and the component model with `wasm-tools component new ... --adapt ...` (as described in https://github.com/bytecodealliance/wasi) yields a 4000+ lines long unreadable mess that is extremely hard to reverse-engineer and thus learn about how WASI 0.2 and the component model works in practice.

[cosmic swift]

Wasmtime indeed does not support executing WASI 0.2 core Wasm files. The overall goal is, to work to improve the documentation so that people can more easily understand it, improve the tools to produce simpler binaries that are easier to understand, and overall focus on making components better. That's admittedly not the easiest path, however it is the most plausible path to a unified ecosystem that doesn't fragment between lots of different people doing the simplest thing for their own use case.

[blissful cargo]

This is sad to hear! Is Wasmtime at least planning to support WASI in core WASM files or will that never be the case? WASI preview1 was so cool because it just worked with any language, since anyone could implement their own WASI library that calls the wasi_snapshot_preview1 functions internally; that's also how e.g. languages like AssemblyScript that do not natively support WASI got unofficial support for it (see https://github.com/jedisct1/as-wasi). The current state of WASI is very sad; everything is completely undocumented and I have yet to figure out how to make a simple hello world in a WebAssembly Text format file. With the overly complicated component model (that absolutely no language supports and that I also doubt will get implemented in browsers anytime soon, or even anytime at all), the WebAssembly on the browser side has drifted even more apart from the WebAssembly on the server/local computer side.
I have to say that I really don't quite get all of this, since especially you, Dan, have always advocated for clear, human-readable documentation (I love your wasm-reference-manual!), yet now with WASI it seems to be the exact opposite (absolutely no documentation, just the confusing component model and WIT explainers plus documentation about how to target WASI 0.2 with C++/Rust). Instead of already working on version 0.3, clear documentation about the actual usage of WASI 0.2, with simple examples in WebAssembly text format, should be prioritized.
I love the concept of WASI so much, and I envision a world in the future where developers, instead of compiling their programs to 17 different OS+architecture combinations, simply compile one single .wasm file that everyone can execute everywhere and that is forwards-compatible with new architectures and operating systems in the future. For that to happen, however, the underlying internal structure of WASI must be clear and understandable - hard things should be possible, while simple things should be simple!

[cosmic swift]

This is sad to hear! Is Wasmtime at least planning to support WASI in core WASM files or will that never be the case? WASI preview1 was so cool because it just worked with any language, since anyone could implement their own WASI library that calls the wasi_snapshot_preview1 functions internally; that's also how e.g. languages like AssemblyScript that do not natively support WASI got unofficial support for it (see https://github.com/jedisct1/as-wasi).

One of the big lessons we learned from launching WASI 0.1 and talking to lots of users is that porting existing Unix applications to Wasm is cool, but by itself isn't enough to make Wasm-on-the-server really great. Building an ecosystem where the fundamental connective tissue is C ABIs with a global linear memory and raw pointers everywhere is really limiting. Wasm modules are designed to be sandboxes, but we can't make use of that if modules share a linear memory and pass around raw pointers.

We looked at what we learned from WASI 0.1 and asked, "do we really want to define Wasm on the server in terms of these limitations?", and said: No. No, that would be throwing away so much of the opportunity, so much of what makes Wasm special.

So we set out fix this, and ended up on the journey that would become the component model. It is, in many ways, the logical completion of many of the ideas hinted at in the Wasm MVP. The Wasm MVP has this idea that we have Modules whose only interaction with the outside world is through their Imports and Exports and it's super super cool, with many implications for security, virtualizability, and composability. However, in practice, core Wasm almost never gets to take advantage of those properties. With the component model, that vision is now realized and we can use those properties in practice today.

It also means that we have a much bigger scope than what we had before. There's more work to do, in implementing it, supporting it in language toolchains, and yes, in documenting it. And it's different from the way people coming from eg. C ABI environments are used to seeing, so things aren't as easy for people to just pick up and figure out on their own. And the structure of the work is different: instead of writing bindings by hand, the approach we're encouraging people to take is to use bindings generators. By-hand bindings are manageable if all you want to do is Unix with its limited feature set, but it doesn't scale well when you want more APIs. Bindings generators are more work up front, and sometimes more complex up front, but once written, allow languages to support many more APIs with higher-quality more secure and easier-to-use bindings, because you're not hand-writing code to raw C ABIs anymore. So it's a stronger foundation, even though the higher layers and ecosystem are taking more time to build.

With the overly complicated component model (that absolutely no language supports and that I also doubt will get implemented in browsers anytime soon, or even anytime at all), the WebAssembly on the browser side has drifted even more apart from the WebAssembly on the server/local computer side.

As a point of comparison, WASI 0.1 had absolutely no prospect of being implemented natively in browsers. Being based on C ABIs, exporting linear memory and passing around i32 "pointers", and being 100% synchronous, it would have been a non-starter for browser folks, had we ever proposed it.

It's fair to say the component model isn't going to be implemented natively by browsers any time soon either. But, you can already run components in browsers today, using jco. Looking forward, many of the people working on the component model have connections to browsers (former Mozillian here :-)), and while it's too early to make any promises, I would say, the component model is more plausible than anything based on WASI 0.1 ever was.

In terms of language support, it's early, but we already have support for Python, JavaScript, Rust, C, and more are in development.

I have to say that I really don't quite get all of this, since especially you, Dan, have always advocated for clear, human-readable documentation (I love your wasm-reference-manual!), yet now with WASI it seems to be the exact opposite (absolutely no documentation, just the confusing component model and WIT explainers plus documentation about how to target WASI 0.2 with C++/Rust). Instead of already working on version 0.3, clear documentation about the actual usage of WASI 0.2, with simple examples in WebAssembly text format, should be prioritized.

I and most of the people I work with are still working primarily on 0.2.

I still do believe in clear, human-readable documentation. And I am planning on doing a hello-world show-all-the-pieces document. We'll get there :-).

I love the concept of WASI so much, and I envision a world in the future where developers, instead of compiling their programs to 17 different OS+architecture combinations, simply compile one single .wasm file that everyone can execute everywhere and that is forwards-compatible with new architectures and operating systems in the future. For that to happen, however, the underlying internal structure of WASI must be clear and understandable - hard things should be possible, while simple things should be simple!

This concept is still alive, and it's even stronger than before!

new [operating systems]

With WASI 0.1, this was mostly only true if you only mean new Unix-like operating systems. But WASI 0.2 is turning heads and sparking some really interesting discussions about, well shucks, if you're going to be running Wasm anyway, what kind of an OS would you want?

It's an interesting question. Unix is shaped by starting with C, which is wildly memory-unsafe so they built this really big box called a process, which is then so opaque that code can't talk to the outside world, so they added Inter-Process Communication (IPC), which is pretty heavy-weight but everyone just pays for it because what else are you going to do?

What if... we started with Wasm? Straight off the bat, Wasm is sandboxed and has a trusted call stack. A caller can call a callee without sharing an address space with the callee. So it kind of looks like Wasm might not even even need a concept of IPC. I don't have a complete OS design here, but we can already see how starting from Wasm can lead to a very different OS.

[wet smelt]

Documentation is definitely something people care about. There was a new Bytecode Alliance special interest group created recently for documentation. I think SIG-Docs' current priority over the next few days is to update the component model book so they can point people to it at WASM I/O this week, and then potentially working on a WASI book with the wasi.dev domain. (If you'd like more info on what they're up to, here's their meeting minutes for last week which also links to their current planning document, or there's a recording of the same meeting on the Bytecode Alliance youtube channel. They're also on Zulip.)

[tacit bone]

As a point of comparison, WASI 0.1 had absolutely no prospect of being implemented natively in browsers. Being based on C ABIs, exporting linear memory and passing around i32 "pointers", and being 100% synchronous, it would have been a non-starter for browser folks, had we ever proposed it.

@blissful cargo - I'd keep an eye on the JSPI work being done by the Chrome team. this will make more of core wasm easier to integrate into async JS code. https://developer.chrome.com/blog/webassembly-jspi-origin-trial?hl=en

I have to echo most of your sentiment about WASI 0.2 -- sadly it seems like its real goal is to force components onto unassuming wasm users. and with no cross-runtime plans, it kind of locks you into wasmtime (which is a great piece of software! but not the answer for lots of use cases).

As Dan mentioned, the component model will likely not be implemented in the browser anytime soon (read: likely never as its not actually a wasm core spec -- it's also unsupported by Binaryen, which makes CM a tough target to build universal wasm tooling for). better to track what the browser teams are doing to make wasm better integrated into the web.

[iron spire]

@tacit bone I can totally appreciate complaints that 0.2 is still quite rough around the edges and in need of documentation and more widespread implementations, but some of the things you are saying are simply false and so I'd like to ask you attempt to exercise some moderation and nuance when making general statements about the future.

In particular:

• It is quite explicitly the goal of the goal of the component model to be implemented in as many runtimes as possible, including browsers. Already WasmEdge has talked to the WASI subgroup about their work-in-progress and, based on talking to other runtime folks at Wasm I/O last week, I expect we'll see more before long. The current lack of runtime support isn't surprising either; it's what you'd expect for an in-progress standard proposal that is releasing incremental milestones.
• While browsers have not yet started working on the Component Model, this is due to the fact that there are much higher-priority features for browsers to implement which I personally agree with: the current browser work on shared-everything-threads and stack-switching is all core functionality that is difficult to polyfill and just as useful outside the browser. In contrast, WASI 0.2 components are polyfillable and thus runnable in browsers today (via jco transpile). Browsers have a long history of leaning on polyfills for developer-experience features for years, while they build up the motivation to provide a native implementation (see: ESM, import maps), and so we should expect no different for the Component Model. We can also see public evidence of openness to one day implementing the Component Model with the recent addition of the Component Model to the W3C WebAssembly WG's official charter, the acceptance of which involves all the browsers' AC Reps. Thus, as a person who used to work on a browser and continues to frequently talk to browser engineers and leaders, my expectation is that there will indeed be a native implementation one day (it's just a matter of waiting until there is sufficient demand and win over the polyfill). Which again, with the working polyfill, is not a huge blocker to the ecosystem.

In the future, please consider this and the fact that making cross-language, cross-toolchain, cross-company open standards is a challenging (and slow) process, and one that a bunch of people have put a bunch of work into before making such casually dismissive statements.

[tacit bone]

I don't think anything I have said is false. I'm not mis-informing anyone, but I do believe others are...

Believe me, these are not casually dismissive... I've tried and tried to support components and get behind the mission, but around so many corners throughout the process, it's felt forced and incomplete and more of a control tactic than a viable long-term community benefit. I'm not the only one!

I am sad to see many folks concerned about the future of Wasm because of the complexity brought by the component model. See this (albeit rather extreme) example of someone who is so put off by the current state of things https://www.youtube.com/watch?v=cvB-ivBkcKk that they actually recommend people just completely disregard Wasm entirely.

To me, this poses a significant risk for Wasm, which is just starting to gain more wide-spread adoption, particularly of core modules, as more runtimes and use cases emerge. It's confusing to folks who are misled to believe components are "the way" to work with Wasm, and in many cases, these users are met with incomplete or non-functional tooling / examples / documentation. I fear it's actually causing harm. Now, with Wasi 0.2 plastering components onto implementors, it's not unlikely that the recent entrants to the runtime space will be percieved as "falling behind", if they don't support Wasi 0.2... which I can only believe is intentional, to position Wasmtime "ahead" even if that is artificial.

[wet smelt]

The view I attribute to lots of people working on the component model is, that it's complex and may take a while to implement, but would ultimately be of huge benefit to the wasm ecosystem. With that in mind, what do you think they should have done differently to ensure that no runtime falls 'behind'?
In my current way of thinking, I don't doubt that WASI designers were acting in good faith, so it seems like Wasmtime is just 'accidentally' ahead because the designers took the fastest path they knew towards stabilising the changes eventually. I ask my question because that'll help me understand what you think was done wrong.

[silent pagoda]

@tacit bone how does wasip2 deal with 64 bit?

the wasi libc still assumes the pointer size to be 32

[tacit bone]

until memory64 spec, it can only address 32bit so unless there’s some WIT magic I don’t think it does. do the docs say anything about that?

[tacit bone]

yea I agree - it will take time & it could be beneficial to some of the wasm ecosystem. however, the rollout of wasip2 has not been done in a way I (and many others who build for the wasm ecosystem) find particularly graceful. again, it appears to be a way to force components on everyone now vs. when they are actually ready and useful to most. being so limited to the Rust compiler (please anyone point me to another well-adopted language that can target wasip2) also makes it feel narrowly thought-through. the majority of the few users behind wasip2 and the component model are cloud/serverless companies who are largely solving their own problems, while marketing the solutions as the “best for everyone” future.

I’ve talked to many users and language/runtime implementers who are realizing that for many use cases they just won’t work. Think graphics, IoT, browser (jco and polyfills are not optimal or ideal), & embedded. Maybe with stack switching, shared memory (not needing to copy between every module boundary), etc it could in the future — but it’s not really good for the ecosystem or adoption to market CM as the way everyone should use wasm when it doesn’t really make sense outside these serverless / cloud companies use cases.

My main issue is it’s just feeling like there’s a massive & premature push for rather incomplete technology that is going to cause harm for many users outside the targeted serverless/cloud use cases.

[wet smelt]

I think I see your point: of the two worlds in WASI 0.2, one is CLI which already had WASI preview 1, and the other is HTTP for cloud. When you say the solutions being promoted won't work very well for graphics, embedded etc, are you thinking more about current WASI interfaces or about the state of the component model in general?

[tacit bone]

CM in general

[wet smelt]

Right. And is it to do with what's currently not possible, like async and GC, or to do with permanent philosophical choices, like putting hard boundaries between components or using higher level types than in C APIs?

[tacit bone]

no it’s none of that really — its that the approach taken by the Bytecode Alliance to roll out wasip2 (effectively forcing CM on users), and the premature marketing & promises of CM are quite likely harmful to the greater & near term adoption of Wasm.

if the messaging was not so aggressively and falsely advertised as “the next version of wasm” (in so many words), I’d have no problem with it. but it’s absolutely positioned to users that CM is the standard and anything else should basically be ignored.

now, if there was already substantial support across languages and runtimes for it - that would be a different story. but it’s quite evident that many runtimes don’t want the complexity, their users don’t need the features, and the languages are not supporting it at the pace users deserve in order to advertise CM the way it’s being done.

[wet smelt]

Correct me if I'm mistaking, but are you thinking that the design of the CM is theoretically fine, but needs a lot of tooling support (from compilers, from docs, from WASI, from runtimes etc) to be practically good enough for most use cases — and Bytecode Alliance looked at the design, decided that it's theoretically good enough for lots of use cases, then endorsed it for everything and everyone without waiting for the tooling and ecosystem to catch up?

[tacit bone]

yea that pretty much sums it up.

to add, it seems to be a very intentional move to capture users into Bytecode Alliance solutions because there are no alternatives… so much for an open system.

“this is the best solution, and the future of wasm, and by the way, we’re the only ones who support it”

[wet smelt]

I'm thinking about where to go from here. If Bytecode Alliance doesn't outright stop promoting the component model, would you consider it a step in the right direction if they instead pivoted towards only recommending to implementers and tooling that they support it, rather than that users use it? And perhaps at the same time if they put effort into making it easier for implementers to support it, by keeping the spec as simple as possible and improving documentation?

[tacit bone]

I don’t think it’s going to matter much in the long run.. I just fear the CM is going to cause harm in the near-term as users will face friction due to lack of support, contrary to the messaging they’ve received about it being ready.

From a different point of view, unless significant adoption and funding comes, the primary promoters aren’t going to be able to push it or maintain it. the majority of cloud / serverless platform adopters lack users / customers and can disappear overnight. the one large company backing it is approximately one single poor financial quarter away from being overtaken by some private equity firm who will slash innovation budgets first - meaning CM efforts grind to a halt. this is admittedly speculative, but based on reality. just look at the financials. it may not seem immediately important, but applying a practical business-oriented model to look at the longevity of things here, and there are fundamental risks to adoption and future support too.

[wet smelt]

It might be naïve, but forgive me if I continue to hope that the component model might work. I haven't seen enough evidence of harm for me to (what feels like) abandon the component model to die. Treating it as harmful would a self-fulfilling prophecy, because it feeds back into the lack of adoption which is the concern. The potential of it working seems worth the risk of it failing, but personally I feel that Bytecode Alliance should take your warning on-board, spending more effort on laying out a sustainable pathway for implementers before they ask for adoption from users. That at least would contain the damage if the component model should fail.
Which leads us back to the topic of this forum thread: improving the documentation situation, reducing friction.

[tacit bone]

not naïve at all -- i hope it works out, and that wasm flourishes! thanks for your time and the chat 👍

[silent pagoda]

they are just ignoring my concerns. I said if the api is x, then its 64 bit version should be named as x_wasm64. Or it will be extremely tough for the VM to support it. if an api does the same thing for 32bit and 64bit, then 64 bit does not need to be suported

then they told me hey it is just a wasip1 thing

lol