SwarmUI SharpSplat - ML Sharp Gaussian Splats in SwarmUI

[distant sandal]

SwarmUI-SharpSplat

A SwarmUI extension that turns any generated image into a 3D Gaussian Splat directly inside the browser.

It uses Apple's ml-sharp to reconstruct a 3D scene from a single image, converts the output to the compact .splat format, and renders it interactively in a dedicated Splat Viewer tab powered by gsplat.js.

---

How It Works

Generated image
      │
      ▼
[Generate 3D Splat button]
      │
      ▼  (sends image as base64 to server)
SharpGenerateSplat API (C#)
      │
      ├─► ml-sharp  →  .ply  (3D Gaussian Splat, standard PLY format)
      │
      └─► ply2splat  →  .splat  (compact 32-bytes-per-splat binary)
                               saved to Output/{user}/splats/
      │
      ▼  (returns /View/... HTTP URL)
Splat Viewer tab (browser)
      │
      └─► gsplat.js  →  interactive WebGL viewer

Step by step

1. Generate an image in the SwarmUI Generate tab as normal.
2. Click the Generate 3D Splat button that appears on the image.
3. The image is sent to the SwarmUI server. The extension:
   • Writes the image to a temp directory.
   • Runs sharp predict via ml-sharp to reconstruct a 3D Gaussian Splat as a .ply file.
   • Converts the .ply to .splat using ply2splat (avoids browser-side PLY parser limitations).
   • Saves the .splat file permanently to Output/{user}/splats/.
   • Returns the HTTP URL of the saved file.
4. SwarmUI automatically navigates to the Splat Viewer tab and loads the result.
5. Orbit, zoom, and pan around the scene with the mouse.

Previously generated splats are listed in the sidebar and can be reloaded at any time.

I promise I'll stop spamming extensions btw

[olive lily]

No, don't. But speak with @kind eagle about adding them to SwarmUI's official list.

So, does this extension work with boobies

And, where is the extension URL

[distant sandal]

I edited the post but here: https://github.com/GlenCarpenter/SwarmUI-SharpSplat/tree/master

[distant sandal]

I should work with anything!

It is 'monocular' so not a full on 3D scene, but still gives the effect. I plan on adding an export feature to export the image from different angles.

[olive lily]

That's pretty neat. It's got a very low max zoom though

I'm trying to zoom to look at some ... eyes. but seems I can't go past what was the original 100% zoom

[distant sandal]

the controls take a little getting used to.
Orbit: left-drag · Zoom: scroll · Pan: right-drag

those are the built-in gsplat.js controls

[olive lily]

Yeah, I can zoom out, but not really zoom in

and, the delete icon should respect user setting for delete confirm

[distant sandal]

The native output is 1536 x 1536, so square inputs may give you better results

[olive lily]

Can't zoom in further than that

[distant sandal]

I may replace gsplat.js

https://superspl.at/editor renders the same splats with much better detail

https://github.com/mkkellogg/GaussianSplats3D looks promising

[distant sandal]

@olive lily when you have a chance can you verify the GaussianSplats3D is working for you with the latest update

this one has more controls too:

https://github.com/mkkellogg/GaussianSplats3D#controls

[olive lily]

It does work, and I can zoom in much further now.

delete confirm is also respecting my settings 👍🏽

[distant sandal]

gsplat.js was pretty lightweight, I was using CDN.

This one I converted to a node project, I'm just shipping the bundled js file when I commit a new release. I couldn't figure out how to get Swarm to build it automatically without trying to use the system installed npm

can't expect users to have it installed

[kind eagle]

to add new extensions you just PR em to https://github.com/mcmonkeyprojects/SwarmUI/blob/master/launchtools/extension_list.fds

[torpid glacier]

Does this work with vr to view?

[distant sandal]

The splat files can be loaded into any compatible application. So depending on your headset, you'd have to install a corresponding app and load in your files. This is designed to work with swarm UI in a web browser, so you might have some compatibility with different browser based viewers.

[distant sandal]

I'm going to start playing around with this to see if it is supportable: https://github.com/facebookresearch/vggt

Since it requires multiple inputs I will probably look at a dropzone area in the Splat Viewer tab with the option to use ML-Sharp or VGGT

Would be nice to have some kind of workflow to generate multiple angles of the same scene. Flux Klein might be a good candidate, would be nice if there was a Klein LoRA similar to this Flux2 https://huggingface.co/lovis93/Flux-2-Multi-Angles-LoRA-v2

[distant sandal]

Official- the extension now supports the blazing fast multi image VGGT model.

Hot take cool for scenery and landscape but not nearly as clean as ml-sharp, which is limited to monocular single image outputs

[distant sandal]

Official - the extension now supports exporting the current viewport with framing