Decentralized OpenUSD-based 3D scene sync ('Metaverse')

[quaint mulch]

Hey all, thought I'd show off something I've been working on for a while. Since discovering that there are usable Rust bindings to OpenUSD (https://github.com/anderslanglands/bbl-usd-rs) I've been itching to use them for something, and I've come up with this.

The idea is that each node maintains their own copy of a USD stage and renders it via the built-in OpenGL storm renderer. They use the layering system to place objects on top of the base scene such as avatars etc, and serialize these layers and send them over a network to other nodes. I'm using the iroh-net crate for this, which provides hole-punched QUIC connections. No central servers are required, and there's a heavy emphasis on permissions and privacy. A node's local layers can override the remote layers, and each node has a private, non transmitted layer that they can write changes into (this is already required to make the avatar invisible for the local user but not remotes).

Things are still in a very early stage but I'm excited about the potential. Next thing I want to add is basic manipulation of objects in the scene, e.g. enough to play chess

[worldly needle]

Interesting stuff!

As a glTF shill I don't have a very solid understanding of USD; what's the thinking behind using USD instead?

[quaint mulch]

USD has a whole API and runtime associated with it which is what I'm using for rendering and combining the data that comes from different nodes.

I personally don't see USD and glTF as being in competition at all really. There's some stuff that you can use both for but generally they have well defined roles.

USD is much better for content creation tools and small scripts, e.g. I had to write a thing a while ago that took point clouds in some bespoke format and converted them into USD files and the python API made that incredibly easy. It's also really good for the stuff I'm doing here where you're keeping a persistent scene open and making modifications to it in real time.

GlTF is much better for when you need to transmit a model over the web or load it directly into a game etc.

I'm actually planning on using both via the guc plugin for USD:

https://github.com/pablode/guc

This lets you load glTF files as if they were USD files by converting them on the fly.

[rancid arrow]

Seems like hazel engine look a like, but interesting.