ToolKitty

[dark bay]

Code gen accumulates in 3 spots at the moment, then they get joined in the end.

• global scope
• sdf local scope
• colour local scope

Each node generates code for any of those scopes.

[dark bay]

---

Oh well... got 1 candle on the 7000 post cake... messed up repeat rotational :p

[zenith void]

🎂 congrats on the 7000

[dark bay]

There we go:

Colours don't flow through repeat rotational properly yet.

And a cone would be better for the tip of the flame than a sphere.

[dark bay]

Good practice :p

Repeat rotational operates on the XY plane, but the candles were in the XZ plane.

Rotate X 90deg --> Repeat Rotational--> Rotate X -90deg

Feels like solving a rubic cube.

[zenith void]

published [email protected] with compile.toSchema and compile.toString

[dark bay]

I'll check that one out tomorrow.

[dark bay]

Colour fix in repeat-rotational

Granted we need a different shader for flames.

[dark bay]

Also, there is an ID number, the x/y/z repeat index. The purpose of those are to allow each repeated object to have a different look, and to provide non-equal spacing to break up the pattern visually.
Gotta work out a sensible way to visually include that in the node graph.

(E.g. a forest of trees, where each tree looks different.)

[dark bay]

---

This one is interesting... a sponge:

(A sphere minus the distance field for sin(x)*sin(y)*sin(z))

Allows ya to put a tonne of holes in things.

Its like you can create a lot of detail with a small distance field formula.

Displacement node not quite working yet:

[dark bay]

With a bit of tweaking in the ray marcher, it should look like this:

[dark bay]

No pc time yet, but I'll keep throwing in primitives via Termux... Bend node:

Handy for sculpting.

[dark bay]

---

@bigmistqke/view.gl integration complete!

---

Now the image node (for tracing 3d graphics off of images) needs to not be in perspective project i think... not 100% sure.
For now I might just include it in the scene as a textured quad.

Or maybe I need the option for a non-perspective ray marching. (A toggle). To make it easier to trace off the images.

That's what I need... Orthogonal Projection during tracing.

[zenith void]

wdym with tracing 3d graphics off of images?

[dark bay]

Just having the ray marching in orthogonal projection (2d), rather than perspective projection. Then have a reference image on a plane underneath, that can be used for positioning 3d primitives for the head, the arms, the limbs etc. using the knobs.

Then I can rotate the whole scene 90 degrees, then re-adjust from the side view with a secondary side view image.

Not really tracing with lines, just tracing with primitives ig.

[zenith void]

Gotcha!

[dark bay]

And we can already blend solids together for joints via soft union.

If we make anything cool, we can throw in an export-to-shader-toy.

Would be very challenging to compete with what's there though.

[zenith void]

It's cool w ur node editor that u can use the compiled output for previews

No need to evaluate the graphs

[dark bay]

Yeah... sort of takes the plugin approach, each node defines its own translation to glsl.

You can have 3rd party plugins (as js files) you can upload at runtime to add additional nodes for example.

But it wasn't the focus, that's just how the code turned out.

One thing it is missing is an IR for an optimisation pass, but not really worried about that yet.

I think opengl optimises its shaders while compiling/linking them anyway. (compileShader / linkProgram)

[dark bay]

Not much done yet, been gardening ½ the day.

[dark bay]

If being strict, compilation is a form of evaluation. The code being the value produced when evaluating the graph.

Add style 1 (intepreted):

function add(a: number, b: number): number {
  return a + b;
}

Add style 2 (compiled)

function add(a: string, b: string): string {
  return `${a} + ${b}`;
}

[dark bay]

---

This is pretty cool, we can just about copy-paste our compiled code directly into shader toy. I just had to manipulate the produced code a tiny bit by hand:

allows us to share our created artwork with the world

[dark bay]

---

Orthogonal projection is in now via a combo box (<select>).

Just need to get that image/texture node in.

Probably need an infinitePlaneNode and a applyTextureNode too.

[dark bay]

Textures might be a bit weird. I think opengl let's you use only a small finite number of textures at one in one draw call.

Yet the node graph let's you use unlimited textures at once.

That's one area where triangle soups win over SDFs.

If there is reflective and transparent solids in the ray marcher, you technically need all the textures bind even if its over the limit to get the correct result visually.

The limit appears to be 8 textures at once in 1 draw call.

It will be plenty for my use-case. Most the time I just need 2 or 3 at once.

[zenith void]

seems so little

[dark bay]

That's the minimum.

Desktop opengl i think is 32 textures max per draw call

You usually batch together all meshes using the same texture, draw them together and switch to the next texture, and repeat.

Three.js hides all the ugly details.

Allowing unlimited textures on a model

[zenith void]

ye that's the handy part of the triangle ig

[dark bay]

And I have felt SDF get slow for certain scenes.

Still... sdf is fun 😁

[zenith void]

ye!

as a modeller it's great

[dark bay]

You say... I got this formula:
f(x,y,z) = 0
Plot it and get something amazing.

f being your SDF.

Oh.... there is 3d texture support too.

That can be used as a hack to squeeze in a lot of 2d textures.

[zenith void]

const symbol = Symbol()

type Generic<T> = T

function fn<T>(a: T): T {
  return a
}
const result = fn(symbol)

type A = Generic<typeof symbol> // typeof x 
type B = typeof result     // symbol
``` ugh bummer, symbols work nicely with type generics, but not with function generics 🙁

weirdly when u hover over the function itself, u do see the narrow type, but then on the variable itself it gets widened to symbol

[zenith void]

then fully namespaced and typesafe snippets isn't possible i think

[dark bay]

Oh well... can't be typesafe everywhere ig :p

[zenith void]

apparently not 🙂

i guess with your technique of scoping it, it will actually still work 🤔

it reminds me a bit of svelte: this thing that only works within the function scope but you have to make wrappers if you want to expose it to the outside

not ideal, but workable

you do lose a bit this single source of truth that you have with getting everything from the compile call

[dark bay]

Bit low on time. I'll get one texture working, and work on a plan of attack a bit later for multiple textures.

[dark bay]

---

No compile errors or shader errors, but the texture came up black. Bit more debugging.

using texture / applyTexture pattern, same as done for colour / applyColour:

Maybe its working and I just need to scale my texture coordinates.

[dark bay]

---

Looks promising:

Little touchy:

Bit of a hack, had to do a 1x1x1 cube, then scaling it up after applying the texture, because the texture coordinates range from 0 to 1.
Would be better, if I had extra nodes for scaling and positioning the texture.

Actually does enough now for me now, to start modelling Pomni.

[dark bay]

Not there yet, I need to scale texture coordinates. Because non-square textures have been stretched into a square:

You get the idea though. Use the sliders to line up 3d primitives with reference image:

[dark bay]

Another thing that would be handy is a visibility checkbox in the Display nodes to mimic layers we can switch on and off.

Or even just a layer node.

[dark bay]

(After placing the head, difficult to see position of eyes.)

I threw in scales and offsets to fix up the aspect ratio of the applyTexture:

[zenith void]

lesgoooo

[dark bay]

Layers: :p.... the lazy way.

The hat I think I can blend two bended cones together.

[dark bay]

So I'll add a cone primitive soon.

And a donut primitive for the trim on the gloves and collar erc.

[dark bay]

Tricky... one bend cone:

A mirror operation would probably be helpful.

Let's get that torus in while we are going.

[zenith void]

it's so satisfying to see the view.gl commit in toolkitty

cool how it fitted right in

[dark bay]

Torus added via termux... slowly adding more primitives:

Probably got enough there to get Pomni done without colours. No energy left for today though. :p

Somewhere to set a filename when saving on mobile would be nice.

[dark bay]

Just gotta keep playing with the sliders:

[dark bay]

Don't know if discord has a storage limit. :p... getting closer:

Definitely do-able with knobs/sliders.

[dark bay]

Inigo also has a SDF function for 2D b-splines on his site. I was thinking we can use that to make edges selectable with the mouse, because we can calculate how far the curve is from the mouse pointer. (used for object selection, rather than rendering)

edge selection has been on my TODO list for a long time.

currently to remove an edge, you have to make a dummy node and plug it into the input pin you want to remove, then delete the dummy node.

better to use math for object selection rather than SVG events, because then you can switch your renderer at any time for the graph nodes.

(E.g. rendering graph nodes with webgl for better performance.)

And I decided against using mirror for modeling just half the character.
Otherwise if you want to animate a wink, it will animate a blink.

[dark bay]

Its also possible to click/tab objects in the 3d and have the corresponding graph node get selected. That might help with navigation.

[zenith void]

makes sense, but also maybe it's still possible to wink with a mirror? mb just needs an additional transform after the mirror?

[zenith void]

that would be sick

at the moment that you can start interacting with the graph by interacting with the preview instead of the graph something special happens

[dark bay]

Yeah I suppose we can make the flow of time (time uniform) different in the reflection... pretty trippy though. 🍄 🐸

[dark bay]

Actually yeah... E.g. if p.x > 0.0, then make your reflection not use wink transform.
Something like that.

---

I'll throw in some quick orbital camera controls soon. Have a feeling I have to rotate or bend cones back a bit, and wanna see how it looks from all angles.

Gotta model up how the hat connects to the head:

Maybe another solid in the top middle to blend with.

The rest of the body seems pretty easy. Maybe the fingers can just be smooth unioned spheres.

[zenith void]

let's goooooo

[dark bay]

Basical orbital camera control:

Will sneak in a mirror next ig.

[dark bay]

For object selection in the 3D viewport rather than just the node graph, requires extra code added to every node. So I will hold off that one for a little while.

[dark bay]

We can probably sneak in bounding box nodes too, for strategical optimisation.

(Skip chunks of calculations of ray misses bounding box)

[dark bay]

We got some interest in the model editor, people are asking for a link to program.

Which means it will be bug fixing time soon... sweeping over all the old bugs.

Adding new features is more exciting than fixing bugs though. :p

[dark bay]

---

So I got the mirror node to just make an independent mirror copy of the object (rather than both sides of the object at once)
It will allow you to do soft unions against the reflection if we want to:

or difference against the reflection or anything we want.

[dark bay]

---

Yeap... Mirror is gonna save some time:

[dark bay]

There is a hierarchy to object selection too. How deep you want to go determines what node gets selected.

To get viewport object selection going. I will need a code gen for the sdf for javascript as well as the code gen for glsl. So that javascript can also evaluate the sdf.

That's why its extra code per node.

Granted the generated code will look almost identical. So it's not a hard task.

An intermediate representation would of made that easier. Because I can have a glsl and a JavaScript backend for the IR.

[dark bay]

always getting closer

There is no way I would write all that glsl code by hand. So the tool is helping.

[dark bay]

Very small tweak to the ray marcher makes it so much smoother (performance wise). I will have to expose the parameters for the ray marcher so they user can tweak them while modelling.

[dark bay]

---

Right... can not make your max iterations of your for-loop a uniform:

ModelEditor.tsx:425 An error occurred compiling the shaders: ERROR: 0:30: 'i' : Loop index cannot be compared with non-constant expression

That makes sense.
So I will have to generation a whole new program each time the max number of iterations change.

[dark bay]

Changing the max iterations to 20, and the tolerance to 10 in the ray marcher makes the frame rate silky smooth again without a noticeable difference to the quality.
I exposed those rendering params in the interface.

[dark bay]

---

Yeah... definitely smoother with a few parameter tweaks:
https://youtube.com/shorts/B3CDbOarZDc?si=hkZUb5xPwxPpgPlT

[zenith void]

really cool there is interesct! but tbh i would be a bit careful with connecting toolkitty with ur daytime job.

ur boss sounds a bit sketchy and would be a bit stoopid if he gets to pick the fruits of ur spare time work. would be a lot of unpaid overtime.

[dark bay]

Surprised you remember that 😅... was just swinging the idea with my boss that time, it never caught traction.

I mean interest from random people out in the interweb asking for a link.

[zenith void]

aa siick

[zenith void]

haha i remember all the 7151 messages 🤣

but that's 🧢

[dark bay]

One guy is asking for sample json files so he can learn to use it.
Was thinking of doing a base64 of the compressed as an import/export, so that people can pass around model in chat using a small amount of text.

I would have to lower the entropy though, by replacing the UUIDs with counter IDs to make the files more compressible.

[dark bay]

A context menu. So I don't loose screen space from too many buttons. That is what I will do.

Inigo's ray marcher is much better, he uses 1000 iterations without loss of performance. Gotta study it a bit. Mine gets slow on 100 iterations sometimes. (More iterations, more quality)

[dark bay]

And a ply export for 3d printing would be nice too. (Marching Cubes).
I do not yet own a 3d printer though. But sure my daughter would like one for her birthday.

I know there is a lot of good software out there already for 3d printing. But it is a different feeling if your using software you have written. 😉

$250 AUD for a cheap one to play with. (Basic one)

[dark bay]

Marching cubes will be fun anyway, and a way to boost performance if we need it. Kinda. Boost performance for static geometry. Dynamic geometry, ray marching will be faster for complex smoothing.

Can borrow some lookup tables to make the implementation less of a headake:
https://github.com/ivanwang123/marching-cubes-v2/blob/main/src/triangulation.ts

[dark bay]

I have written the 256 branches of that if-statement in the past in C. It's not really something I want to do again. Will reuse an existing lookup table. Although, I can probably code-gen that lookup table.

[dark bay]

interesting... By configuring a max step in the ray macher, I was able to do 1000 max iterations without performance loss.

[dark bay]

  let exportToClipboard = async () => {
    let isUUIDRegEx = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
    let shortIdMap = new Map<string,string>();
    let nextId = 0;
    for (let entity of props.world.entities()) {
      if (isUUIDRegEx.test(entity)) {
        shortIdMap.set(entity, `${nextId++}`);
      }
    }
    let data1 = JSON.stringify(props.world.toJson());
    for (let [ entityId, shortId ] of shortIdMap.entries()) {
      data1 = data1.replaceAll(entityId, shortId);
    }
    let data2 = gzip(data1);
    let data3 = uint8ArrayToBase64(data2);
    await window.navigator.clipboard.writeText(data3);
  };

Still 400 characters for a simple scene with one sphere. Was hoping for something smaller.

[dark bay]

suppose there is a little bit of overhead for the header information in a gzip file. Maybe the compression will scale well.

I'll throw in the XOR node and give that guy an example with all four boolean OPs.

I will get back to ToolKitty as a game dev tool eventually. Just thinking of some 3D support after rigging and animating some characters. SDFs are so intuitive to use for 3D modelling, I am not skilfull enough to do the same thing in Blender that I can do with SDFs.

Pixel editor for 2D, SDFs for 3D.

Yeah... sequencer I will need too (placing notes to make music from scratch)

Big job a game engine.

23,000 characters for pomni on the clipboard with base64 gzipped json. Might need a binary json bson.

I failed a 4K demo by a mile 😅

[dark bay]

Ohhh.... XOR just looks like OR from the outside for a cube and a sphere. (One or the other, but not both). Bad demo. 😜

Not sure how to show xor on a 3d shape in a meaningful way.

On a 2d shape it is clear though.

But we can do 2d in 3d to demonstrate XOR:

Smooth xor is interesting... although I am not sure if smooth xor makes sense, because it has more than one solution:

[dark bay]

I tried to paste the union, intersection, different demo here (as a base64 gzip). But discord thought it was a virus and blocked it. 😅

Here's the file anyway:

Oh well. We'll see if this guy can copy base64 data, and click "import from clipboard" in the program.

[dark bay]

Tried to copy paste the text on my phone from the comment to test it, it was nearly impossible. Will be more of a desktop thing.

---

Now that that is all sorted out.
Back to Pomni's feet.
I think I can ½ an octahedron shape to form them up.

[dark bay]

... or should be do marching cubes 2nyt?

[dark bay]

GitHub is slow for me 2nyt. Might be time for a break.

[zenith void]

it's so funny because they are almost these polar opposites of each other

would be fun when you start combining them

render a 2d pixel game on an sdf blob

[dark bay]

Raster 2d, and Vector 3D 😜

[zenith void]

exactly 🤣

data and computation

[dark bay]

You can also do 2D in the 3D ray marcher too. Its kinda like having hardware accelerated SVGs.

[zenith void]

you could build the pixel editor in ur model editor

but you can not build the model editor in ur pixel editor

[dark bay]

Yeah ig

I saw something absolutely unbelievable from Inigo the other day on YouTube. I can't seem to fine it now though, its not on his website.

It was like a wilderness scene with flowing creeks, and you can see the suns rays through the trees from the dust in the air.

All running at real time just in a vertex shader.

He is some sort of shader God or something.

Will see if I can find it again. 😜

[zenith void]

🧙‍♂️

[dark bay]

Weird... I am not seeing it his youtube account either.

[dark bay]

One of the coolist ways to do a tutorial:
https://youtu.be/9g8CdctxmeU?si=kl26lebYzjyz1Tsr

[dark bay]

---

Time to throw in some 2D shapes too, along with extrusion and revolution, to make them 3D:

[dark bay]

This is pretty neat.
You can pretend a 3d shape is a 2d shape, feed it to revolution, and it will revolve the parts of the 3d shape that intersect with the xy-plane:

So all our 3d primitives can be used to 2d primitives too.

E.g. soft union two spheres, do a revolution of their intersection with the xy-plane, and the you have an hour glass.

Apply an onion operation, add transparency, refaction a d reflection to the ray marcher, and then you got a really good looking hour glass.

[dark bay]

---

So to do a 3d egg shape, I can simple take a cone intersecting the xy plane a certain way, and then do a revolution operation.

Or a cylinder intersecting the xy-plane will give a different egg like shape, but more like a stretched circle.

I've got a offset op called rounded in there too.
I could call rounded on a 2d line segment to make finger segments.

Quite a lot of flexibility already.

Still haven't worked out how to do Pomni's shoes though.
I thought ½ an octahedron would work, but its too wide.

... maybe a 4D prism intersected with the 3d space can provide a stretch octahedron, just the same way a cylinder intersection the xy plane can provide a stretched circle.
... there is no way I can visualise it, but it feels possible.

Time to google hyper prisms

[dark bay]

Or just CSG intersect 5 boxes together would work too.

[dark bay]

Ahhh.... that's how you get displace node working properly. You have to lower the step size on the ray marcher:

[zenith void]

sick!

lathe-like

[dark bay]

Now that I think about it, decreasing the step size makes perfect sense.
When you distort space using something like displacement, you are compressing and expanding space. The compressed regions of space will need a smaller step size in their ray marching, so that you do not overshoot the solution.

As a cheat, I could distort space around an octahedron in order to stretch it in 1 direction, but I would have to lower step size of the ray marcher for that solid. I could put a bounding box around that stretched solid to prevent reducing the step size everywhere though.

[dark bay]

Yeah, we can do pots and vases. :p

[dark bay]

Maybe I can throw in extra nodes with a little warning logo on them ⚠️... "warning, may hurt performance"

---

⚠️... so that works, I can scale in 1 direction at the cost of performance:

A bounding box could be formed around that to prevent the performance loss everywhere.

[dark bay]

Might pass as a Pomni shoe ig :p

Nah... different shape now I go back to look at it.

[dark bay]

Ohh... refract is built into glsl these days:
https://registry.khronos.org/OpenGL-Refpages/gl4/html/refract.xhtml

Was trying to work out the formula again, but it is built-in.

[dark bay]

---

Bit of a bug in the sphere. But that is the start of glass:

You can see the lensing effect start to take form. Although buggy at the moment.

At the moment, anything transparent has the refraction index of glass. But we can add a node for that.

[zenith void]

let's gooo that's pretty sick!

[dark bay]

Not sure what bug is yet. It will have to be placed behind a runtime feature flag:

Not quite right yet.

---

Win:

Just had to flip the normal when exiting the glass solid from the other side.

[zenith void]

Let's gooooooooo

[dark bay]

Triangle soups can do it too via rendering the scene to a texture, including it in the fragment shader, and then using refract(). That must be why refract() is a built-in function in glsl.

Need a proper material properties node eventually. There is reflection and shadows to sort out too. And probably countless other things that I don't know well.

Also had another play with that music waterfall thing.
I found there is GC pauses still even though no memory will have to be freed. When pointers/references are reassigned, the GC pass still has to run to check if there is memory to be freed, even if there isn't.

Only escaped from that might be wasm for performance critical parts.

And the audioworklet thread seems to freeze the main ui thread for that GC pass, not the audio thread.

[dark bay]

And a load of render settings ⚙️.
The current go-through-transparent-thingy limit is 1 at the moment.

[dark bay]

Ohh... I get it now... water flows downstream, so in a reactive graph things that update later in time are downstream, not upstream. That makes sense now. I've always had that backwards.

[dark bay]

How the ray marching works here by the way, is once the ray hits a transparent object, you negative your distance field to keep ray marching until you exit the transparent object, then restore the sign of your distance field as you exit.

(Invert the density of space while inside a transparent object)

Although, inside/outside is a bit confusing... air is a transparent object too.

[dark bay]

---

Let's get point to point capsules going... finger time 👉

Actually a vertical capsule of given height might be better. To help maintain constant length finger segments.

Let's pull out everything we have in our bag of tricks and put triangle soups to shame! 😀

[dark bay]

---

Capsules added, I'll keep trying to finish of Pomni, and work out what features I need along the way. (The model is the todo list.)

[dark bay]

Feels like the next feature will be functions. Would be nice to design the parts independently and re-use them where needed.

[dark bay]

---

Hands are hard:

[dark bay]

couple of more segments. I'll share:

[dark bay]

or maybe auto-format graph next... I'm really getting lost in a sea of spaghetti.

[dark bay]

this will do for now:

[zenith void]

hands are legendarily hard

each artist's biggest nightmare

it's funny that ai had such a hard time with it too

[zenith void]

beautiful how the shape of the code mimicks a bit the shape of the hand 🙂

[dark bay]

Not intentional. Auto format comming soon I think.

I got the start of a palm in there using my phone:

Just have to soft union the other 3 fingers.

Slow work though. (Dragging around the screen forever to find the right nodes :p)

Soft unions definitely seem like the way to go.

The palm is a small rounded cylinder.

[dark bay]

After reloading and looking at the graph again after a break. I am completely lost 🤣

[dark bay]

Bit more of a play on my phone... not too shabby:

[dark bay]

For fun:
https://youtube.com/shorts/gu3Kro8Agxc?si=4Yiebk8NT4PitOeG

[dark bay]

Ohh... react flow uses 3 different layout engines you can switch between:

Looks like automatic layout is a pro feature:

Sometimes its just better to write your own node graph tools. :p

---

Will try this 1st i think:
https://github.com/dagrejs/dagre

Even with automatic layout, we will still need functions too. And maybe we can ctrl click them to decend into their source.

Sort of have a global graph view, and function graph views. A hierarchy of graphs. To make it more organisational.

The main problem, too much information on the screen at once, and no code reuse.

[dark bay]

---

Here goes nothing:

  const format = () => {
    let graph = new dagre.graphlib.Graph();
    graph.setGraph({});
    graph.setDefaultEdgeLabel(() => ({}));
    for (let node of nodesSystem.nodes()) {
      graph.setNode(
        node.node.nodeParams.entity,
        {
          label: node.node.nodeParams.entity,
          width: node.renderSize()?.x ?? 100.0,
          height: node.renderSize()?.y ?? 100.0,
        }
      );
    }
    for (let node of nodesSystem.nodes()) {
      for (let input of node.node.inputPins?.() ?? []) {
        let source = input.source();
        if (source != undefined) {
          graph.setEdge(source.target, node.node.nodeParams.entity);
        }
      }
    }
    dagre.layout(
      graph,
      {
        rankdir: "LR",
      },
    );
    for (let nodeId of graph.nodes()) {
      let node = graph.node(nodeId);
      let node2 = nodesSystem.lookupNodeById(nodeId);
      if (node2 == undefined) {
        continue;
      }
      node2.setSpace(
        Transform2D.create(
          Vec2.create(node.x, node.y),
          Complex.rot0,
        )
      );
    }
  };

---

well, it did work.... but it doesn't seem to make the graph easier to read.

Like... it's layed it out pretty, but the flow direction for the order of calculations is all over the place, instead of just in the one direction.
Gotta read the document a bit more.

There must be somewhere to control the ranks for each graph node, so that dagre can lay them out in execution order.

[dark bay]

.... ohhh.... Dagre's coordinate system is likely different from mine.

I'm in the Cartesian coordinate system with nodes positioned by their bottom left corner, rather than their top left corner.

[dark bay]

It's doing a pretty good job now. Occasionally it has nodes overlapping though.

[zenith void]

Ye... it's tricky!

Graph order is not exactly the same as intent

With text there are all these different syntax to communicate intent, but with the graph there is only a single form to communicate everything (node in a graph)

It's counterintuitive bc as a coder u do think in terms of data flow, and a graph is such a pure representation of that

But somewhere u lose the trees in the forest

A true graph formatter would probably need to analyze the graph too, and generate an ast of intent from that. What that exactly would entail: I do not know. Would be a good PhD.

[dark bay]

Probably user node functions will help with organising the data.
E.g. the user could make a node function called finger segment with a few params, then another node function called finger. Then use finger 4 times, and maybe have a separate one for thumb.
Then user combines all of that into a hand node function, that can be reused.

If we where writing regular text-code with no variables, no functions, just one line with a display function taking calls to all the other functions as parameters, and thoses functions doing the same, then I suspect understanding the text-code would be just as challenging as understanding the node-code.

[dark bay]

---

This one has 3 issolated sub-graphs, each of which could be a user node function to help show intent:

(One for eyes, one for teeth, one for body)

Pre-knobs/sliders that one.

Dagre actually did a pretty good job with this one.

[zenith void]

isn't that jsx?

jk (but also not)

[dark bay]

Your not wrong.

[zenith void]

the edges make it more complicated

it being a graph, not a tree

linking with attributes always a bit messy: <label for="..."/> type beat

[dark bay]

JSX would make a good DSL for sure.

[zenith void]

mb there are language parrallels that could be made

like imagine u compile the node graph to code

and then u format the generated code

and from that u could hypothetically extract some rules for how the graph should be organized in a way that makes semantically sense

lisp mb

or some functional language

[dark bay]

It compiles to glsl, but maybe not human readable.

float x0(vec3 p) {
          return length(p) - 300.0;
        }
      
        float x1(vec3 p) {
          vec3 q = abs(p) - 0.5 * vec3(3000.0, 3000.0, 3000.0);
          return length(max(q,0.0)) + min(max(q.x,max(q.y,q.z)),0.0);
        }
      
        float x2(vec3 p) {
          return x0(p - vec3(-600.0, 800.0, 1500.0));
        }
      
        void x3(vec3 p, out vec4 c) {
          defaultColour(p - vec3(-600.0, 800.0, 1500.0), c);
        }
      
        float x4(vec3 p) {
          vec3 q = abs(p) - 0.5 * vec3(4000.0, 500.0, 4000.0);
          return length(max(q,0.0)) + min(max(q.x,max(q.y,q.z)),0.0);
        }
      
        float x5(vec3 p) {
          vec3 q = abs(p) - 0.5 * vec3(500.0, 1500.0, 500.0);
          return length(max(q,0.0)) + min(max(q.x,max(q.y,q.z)),0.0);
        }
      
        float x6(vec3 p) {
          vec3 q = abs(p) - 0.5 * vec3(500.0, 1000.0, 500.0);
          return length(max(q,0.0)) + min(max(q.x,max(q.y,q.z)),0.0);
        }
. . .

[zenith void]

lol ofcourse

[dark bay]

Ahh... but doesn't compile to a language on the DSL level.
Only the low level.

[zenith void]

ye, it's sort of meshed together i suppose

u would need some type of sourcemap that maps one form to the other

or an intermediary form

idk, just spitballing

[dark bay]

Good to throw ideas around.

I ultimately went node-based, so a non-programmer can just link some boxes together with lines to get things happening.

I think it's a data management / organisational problem as the graph gets bigger.

Just need a way for the user to combine chunk as single nodes, and reuse those chunks.

In theory that should be just as expressive as a text based language.

[zenith void]

For sure, u can always give users tools to manage the organization themselves

But i think that's the power of autoformatting: you don't have to become a graphic designer to write maintainable logic.

[dark bay]

When we write text code. We can get away without comments a lot of the time, because the names we give to functions behave like the comments, and the code becomes self documenting.

The node graph at the moment isn't self documenting one bit.

[zenith void]

https://observablehq.com/@erikbrinkman/d3-dag-sugiyama satisfying to play with the options

[zenith void]

Very true

And naming is not without labor

Probably what i spend most of my time on

[dark bay]

I'd hate to think what a JSX UI would look like in the node graph editor :P... I bet it is messy as.

[dark bay]

When I get time, I'll try that... user node functions.
The same feature will be usable in the instrument editor, the model editor, and the AI editor. (In anything using that node graph thing.)

[dark bay]

I'd be pretty happy with dagre if it doesn't overlap nodes. I wonder if I have not used it correctly. It gets very close to no overlapping nodes, but the occasional one still occurs.

We'll still make a text based DSL in a separate tab to accompany the graph too. (With optional node coordinates in it's params)

We can auto-generate .d.ts files from the graph nodes for the text based DSL, and fire open code mirror.

The plus side to a TypeScript DSL... we already have user functions.

[dark bay]

A big question ❔️... how do we keep the node code and the text code in sync?

I'll do some research and see what others have done.
If there are typescript if-statements and for-loops in the mix, then synchronisation is not straightforward.

[dark bay]

Maybe I can have them separate and save them separate until I can work it out.

[dark bay]

The main difficulty is typescript has so many more language features than the node graph, so there is no 1:1 conversion.

I suppose the node graph could simply disappear, if the TypeScript code uses any features not supported by the node graph. And that might be the best solution. Because it will always be in sync, and if any incompatibilities arrise, then it drops to one representation.

I will need to store variable names for the graph edges, is one addition I need.

It would be advantageous to have sliders in the text editor too. For numbers.

[zenith void]

I think if the dsl is ur api and not the whole language of typescript u will have a better time

Like if the api also includes controlflow

Then u don't need to rely on if/else and the rest of the bunxh

If u do need full typescript, you could have a hook for a graph node that hooks into the lifecycle of the graph generation

[dark bay]

I suppose the moment incompatibility arrives, the graph has 1 special typescript node, that is a textbox containing the typescript.

Tomorrow I should be able to find some time to have a play.

A TypeScript for loop is kinda handy, because it is like a loop at the macro level.

[zenith void]

But that node will only run during the initial transpilation of the shader right? Bc u cannot run typescript on the GPU (except if u would use typegpu 😅)

[dark bay]

Unless it's designed so the execution of the TypeScript DSL, generates the glsl 😉

[zenith void]

Yes, exactly. If it's part of the graph generation it's fine.

[zenith void]

If u haven't yet, u should have a look at https://github.com/software-mansion/TypeGPU creator is in this discord too and is really chill

[dark bay]

Yep. I've seen her.

[zenith void]

I promised I would have a look at a solid integration, but haven't gotten around it yet

Solid version of https://github.com/iwoplaza/JotaiGPU?tab=readme-ov-file

[dark bay]

Here another nice.... yet old one:

https://github.com/ku-fpg/sunroof-examples/blob/master/examples/canvas/Main.hs

Learning sunroof shows how its all possible.

[zenith void]

That's Canvas API right?

[dark bay]

Your creating and working with types that belong to another domain.

[dark bay]

Haskell js DSL... allowing you to generate js from Haskell.

Same idea as type gpu

Domain types

And when you execute functions on those domain types, it builds up a AST under the hood, exposed as another domain type for the return type.

All single pass, builds an AST, then that AST is pretty printed for the codegen for the given target.

function add(a: GLSL<vec4>, b: GLSL<vec4>): GLSL<vec4> {
  return GLSL.of<vec4>({
    type: "add",
    a,
    b,
  });
}

That's all you need. AST tree builder, an optional optimiser, and a pretty printer.

[dark bay]

That's a bit more tricky... mixed domains.

[dark bay]

Actually no... TypeGPU uses a babel compiler iirc 😅

[dark bay]

Maybe to begin with:

• One source of truth at all times.
• A convert to TypeScript button.
• A convert to graph button.
• Only one, graph or text source shown at a time.

The reason for the one source type at a time is.
A conversion from typescript to graph to typescript will not end up with the same typescript you started with.

[dark bay]

I'll probably enjoy the TypeScript DSL so much, that I will have a hard time returning to the node graph. 😅

[dark bay]

---

I have started on the node graph to typescript converter... Its not quite right yet, but here is a sample output:

let { out: x0 } = Number({  });
let { out: x1 } = UnboundKnob({  });
let { out: x2 } = UnboundKnob({  });
let { out: x3 } = Colour({  });
let { out: x4 } = Colour({  });
let { out: x5 } = Number({  });
let { out: x6 } = UnboundKnob({  });
let { out: x7 } = UnboundKnob({  });
let { out: x8 } = UnboundKnob({  });
let { out: x9 } = Cylinder({ radius: x8, height: x7 });
let { out: x10 } = Vec3({ x: x5 });
let { out: x11 } = ApplyCheckers({ model: x9, colour1: x4, colour2: x3, size: x2 });
let { out: x12 } = Vec3({ y: x0 });
let { out: x13 } = Rotate({ model: x11, axis: x12, angle: x1 });
let { out: x14 } = Rotate({ model: x13, axis: x10, angle: x6 });
let {  } = Display({ in: x14 });

that's kinda what the DSL looks like

[zenith void]

have you looked at https://github.com/mrdoob/three.js/wiki/Three.js-Shading-Language ?

[dark bay]

Yes, I've seen that too. It's impressive.

[dark bay]

Basically all those node functions will be defined in global scope, and executing them reconstructs the node graph, allowing for a conversion back again.

Its done in the general node editor. So the conversion to TypeScript will work for both instrument editor as well as the model editor. (Works for multiple domains.)

[zenith void]

i can see that

generic enough

{ out: ... } is so you can support multi-outputs?

[dark bay]

Multiple output pins are support. The instrument editor has a few with multiple output pins.

But yeah, its kinda ugly in the generated source.

Maybe if its only { out: ... }, then it should default to a single output in the generated TypeScript.

Adding type signature to our node pins is one missing thing.
To give us type safety in the generated TypeScript.

Phantom types... types that represent the types of the values in another domain.

[zenith void]

you can accomplish this with type branding i think

for floats and ints p.ex

or just a class ig

or anything haha

[dark bay]

Lots of ways yeah.

declare even

Or making classes/types for them all.

---

I am actually mistaken... for every single node types I have, all the outputs are single pin only.

[zenith void]

it's kind of nice to support multi-outputs

although technically each function only has a single output too

if u are not yielding

you wrap it in an object then

[dark bay]

Yeah... and makes the generated output look a little ugly for multi outputs.

[zenith void]

from a user POV it is nice tho, having multiple node outputs

but

from a dsl pov it could be that it still looks like a single object

it could be return { type: 'multi-output', out: Record<string, ...> }

[dark bay]

Yeah... I was just gonna check the number of outputs the node type supports. And if its only 1, then do single output instead of multi.

And change the first letter of the nodes functions to lower case 😅

Don't like C# style.

Although number is already taken.

Might have to go the import * as $ from "prelude"; path again.

($.number(...))

jQuery lives on 😁

[zenith void]

🤑 got my mind on my $ and my $ on my mind 🤑

[dark bay]

import * as $1_000_000 from "prelude"; cha-ching.

[zenith void]

if it's a dsl and they are just data-containers, mb it could be valibot

then u got free validation and types

but they are not just data-containers

u were mentioning to have it contain typescript

[dark bay]

Types will be enough i think. There is nothing to validation because the actual code is executed remotely.

[zenith void]

i mean u can still connect wrong datatypes right?

[dark bay]

I can fix that, it's on my todo.

[zenith void]

ye, but to say that u do need to validate

[dark bay]

I need to define types for the pins in the node definitions.

TypeScript's type system will do it for me via the phantom types 👻

[zenith void]

oo i see

codegen the dsl

typecheck the dsl

[dark bay]

Same approach as sunroof.

[zenith void]

then u will need to map the type errors back to the nodes

it's possible

not sure if it's the most optimal approach

[dark bay]

That's interesting.

[zenith void]

but it's an interesting one for sure

[dark bay]

It means TypeScript can type check the node graph...

[zenith void]

exactly

[dark bay]

Saves a lot of trouble :p

Bit of a weird way to do it, but it works.

[zenith void]

to create some other trouble hahaha

[dark bay]

Stolen type system, so we don't need to implementation our own. 😆

[zenith void]

but i think if u go the dsl route u will need to solve the issue anyway: some type of sourcemap that maps in between dsl and the graph

[zenith void]

if u have that, then the error mapping becomes relatively trivial

but is a big if

[dark bay]

It's just a matter of storing the line numbers for the code generated for each generated node. Seems easy.

[zenith void]

ye i think it's more when u let people mess with the generated code that it becomes tricky

you will mb have to statically analyse the graph that the nodes generate

[dark bay]

I've studied the implementation of the type system for PureScript, I can implement my own if I need to.

[zenith void]

that's the spirit 😉

[dark bay]

Monad support 🤣

[zenith void]

that monad gonna haunt me the rest of my life isn't it

but halloween around the corner, so makes sense

👻

[dark bay]

Its a type of a type with a few methods and laws, nothing more.

(* -> *) -> *

[dark bay]

---

And of course to go from TypeScript to glsl, this path is taken under the hood:

• execute TypeScript to generate node graph.
• do the normal compilation of the node graph to glsl.

Its pretty neat!

The node graph becomes the intermediate representation.

[zenith void]

big brain

[dark bay]

My daughter is likely to get a 3d printer next month for her birthday... will have to get that triangle soup export going soon.

[zenith void]

Siiiiick gift!!

That's so cute

3d modeling in the editor that ur dad made and then printing it out on ur own 3d printer: that's a cool kid 😎

[dark bay]

---

The conversion to TypeScript is getting more tidy:

let x0 = unboundKnob({ value: 266, sensitivity: 1, });
let x1 = unboundKnob({ value: 1975, sensitivity: 1, });
let x2 = box({ size: x1, });
let x3 = round({ model: x2, radius: x0, });
display({ visible: true, in: x3, });

Starting to look sensible.

need to work out a way I can still drag the knobs though, now that it is all text

---

Looks OK... onto the prelude.

[dark bay]

For types, I'll just have non-generic concrete types for now. It will be all I'll need.... maybe type constraints too, we'll see.

The Box's size pin actually takes vec3 | float. So a union type.

Concrete types and union types will probably be enough.

[elfin lintel]

interesting

[dark bay]

One of the methods auto-generated in the prelude. I don't have all the type information there yet, but you can see the pattern:

export function applyColour(params: { colour?: any, model?: any }) {
  let entity = world.createEntity([
    applyColourComponentType.create({
    colour: params.colour == undefined ? undefined : { target: params.colour.target, pin: params.colour.pin, },
    model: params.model == undefined ? undefined : { target: params.model.target, pin: params.model.pin, }
    }),
    transform2DComponentType.create({
      transform: Transform2D.create(
        Vec2.zero,
        Complex.rot0,
    }),
  ]);
  return {
    target: entity,
    pin: "out",
  };
}

The world will be captured using a context pattern like you'd find in solidjs.

[dark bay]

Some node input pins are optional, some are not. But I haven't flagged them in the NodeType definitions yet.

[dark bay]

Hooking up code mirror now.
The autocomplete seems slower on code mirror, compared to monaco, not sure why that is.

It's like I hit . and it takes 10+ seconds to show the auto complete on my old lappy connected to TV.

On document. anyway, on user types it is faster.

Maybe it is lazy loading the type signatures or something.

Or maybe I should not be doing stuff on a 10 year old laptop 🤣

[dark bay]

---

I always struggle with code-mirror hookup... but it's hooked up again...

we have some auto-complete going for the code-gen prelude.

the types need fixing, but we will get there.

[dark bay]

---

For now, onto ts -> js repl thing, so I have something to execute to regenerate the graph to compile.
I should also diff the new generated graph against the old one, so a change to a number value only in the source, will just update a uniform value, rather than do a gl compile.

In order to diff them. I need to reuse the same entity IDs on the next graph generation, so I am able to diff the graphs somewhat.
If the number of nodes/entities change, it will need a full recompile (gl compile) anyway.

I should also find a way to put in proper varargs support into the nodes too. So we can use a single union node on more than two inputs at a time.

[dark bay]

---

A little bit of dependency injection to get the world into the prelude:

  let preludeModule: Accessor<{
      withWorld: <A>(world: EcsWorld, k: () => A) => A,
  } | undefined>;
  {
    let preludeModule_ = createMemo(() => {
      let preludeUrl = repl.getExecutable("node_modules/prelude/index.ts");
      if (preludeUrl == undefined) {
        return undefined;
      }
      let [ result, setResult, ] = createSignal<{
        withWorld: <A>(world: EcsWorld, k: () => A) => A,
      }>();
      (async () => {
        let module = await import(/*vite-ignore*/preludeUrl);
        setResult(module);
      })();
      return result;
    });
    preludeModule = createMemo(() => preludeModule_()?.());
  }

Powered by @bigmistqke/repl

(Users TypeScript gets imported/executed after our ECS world gets injected into the prelude module that it imports.)

[dark bay]

Might have to do some small alterations. I believe EcsWorld class will have all its methods mangled to short-names, from the compilation process done to ToolKitty, which the prelude will not know about.

[zenith void]

Don't minify?

[zenith void]

I never managed 😅

[dark bay]

I'll work around it. I'll mangle the generated code to suit i think.
Actually I'll just disable minify in certain places like you suggested.

[zenith void]

handier for debugging anyway!

[dark bay]

I honestly don't know if it will mangle... it's all untested code. I'm just guessing it will.

Keep getting kicked off 😅

[zenith void]

ye actually if it's about method names, i m pretty sure they stay unmangled

and unminified

although

now i'm less sure about it

haha

[dark bay]

Cuz you could access via myObj[key] and there is no way it would know how to mangle it.

[dark bay]

---

Seems to be playing the game so far:

(the code is doing as it's told)

I'd probably allow for direct numbers, as well as making number nodes.

[zenith void]

let's goooo

[zenith void]

i can remember looking at minifed code and wondering why method names weren't being minified. I think the idea of minification is that all public api is unmangled.

Internally it would be minified and call minified versions of itself, but it would export the unminified versions: const _Re = ...; const _Ye = (arg) => _Re(arg); export { _Re as createSignal, _Ye as createSignalWrapper }. From that heuristic it then also results that method names can not be minified, as they can be accessed by the user.

i wonder if it does minify private properties like class Class { #privateProperty: any }

[dark bay]

Ah... I didn't try compiling it, to see if it still works. I was in vite serve mode.

[zenith void]

😅 classic

[dark bay]

Still works with vite preview after compiling. I think its all good.

I just remember the pain we went through with our mario clone repl hookup.

[dark bay]

A conversion to TypeScript does not directly make the graph easier to understand, but atleast we can organise reusable part into functions to create some structure to the code:

I'd argue the direct TypeScript conversation is no easier to understand than the original graph.
It is still spaghetti in need of some organisation.

---

And I go to type, and the virtual keyboard covers all the source 🤣

[dark bay]

For now. I'll just swap top and bottom ig.

[dark bay]

And maybe a update/refresh button as gl compile gets slow for large source files.

[dark bay]

... the variable number is the line number... a navigational hack.

I will need to take inspiration from that shader editor for Android and overlay the source on top of the rendered image.

[dark bay]

That's a quick fix:

I can see it compiling the typescript. Executing it to produce the new graph. But no change in the model yet. A bit more debugging to do.

[zenith void]

nice progress!

[dark bay]

Then we can be cheeky, and say: "look TypeScript running on the graphics card".... (not really)

---

Weird, I didn't change/debug the code yet. But it appears to be working.

I changed the TypeScript and the display changed.

Yes... not mobile friendly yet. Being able to see 4 lines of code at once editing is better than zero i guess.

Ohh... the gzipped TypeScript is probably a more compact representation when saving than the gzipped json of the graph.

[dark bay]

---

Cool thing though, the TypeScript compilation works for the instrument editor too with no extra code. Because its just a graph builder.

Like an all purpose DSL creator.

[dark bay]

.... And we have unlocked recursion... we can do interesting fractal stuff. (GLSL doesn't support recursion.)

We could also do tricky things, like use two floats under the hood to represent one float with extra precision. (Fractal zooms)

[dark bay]

---

Sneaking in a "Convert to Node Graph" in the menu too, in order to go back from TypeScript to node graph.

---

That's working... I might have a bit of a play with the new TypeScript feature. Try and do some fractals with recursion.

Recursion works, because its a macro-like recursion, not a glsl recursion.

Which is pretty neat!
(It inlines the recursion calls)

[dark bay]

---

Rust E is probably more sensible in TypeScript too. You have notes and delays that are clearly visible:

[dark bay]

If I get import working while in TypeScript mode, that will solve my slow SVG problem with huge graphs.

[zenith void]

u unroll it?

[zenith void]

do u statically analyse it? or do u run it and serialize the graph that it generates?

[dark bay]

Yeap

[dark bay]

Run it, and use the graph it generates.

So a lossy conversion.

[zenith void]

good enough for now ig

assuming a static graph

[dark bay]

The lines of code in a text source file are list of static characters.
So maybe static is OK.

E.g. the source code for solidjs are static text files.

Code in our TypeScript thing is executed at the macro/compile level, and not really executed at runtime.

Any more advanced features will need to be supported by the graph node types.

[zenith void]

ye sure

i mean

if there is no IO u probably can't do anything meaningfully dynamic

but u can make the compilation indeterministic at least

if(Math.random() < 0.5){ ... }else{ ... }

[dark bay]

Yeah 🤣

Not very random when it hits that line.

[zenith void]

date based stuff mb?

but ye

fair to say to just not support it 🙂

[dark bay]

The runtime is in a very different environment. (GLSL)

[zenith void]

i know 🙂

it's about the compilation

of the graph

was just thinking of ways to break it 🙂

but it doesn't need to be solved

it can be a feature!

[dark bay]

Was hoping to have you a cool fractal demo before you were online. But I wasn't quick enough 😁

[zenith void]

recursion is really sweet!

is it with a recursion node?

curious how u abstracted it

[dark bay]

And I didn't have a save for the TypeScript stuff, and I locked up my phone. 🤣

[zenith void]

The Struggle

[dark bay]

Plain macro recursion... the produced graph isn't recursive at all.

You just have a depth limit for your macro recursion.

[zenith void]

i meant how does it look like in the graph

[dark bay]

So the code that produces the code is recursive, but the resulting code isn't recursive.

Just lots and lots of nodes.

[zenith void]

oo i see

[dark bay]

Too painful to do by hand. But easy with a macro.

[zenith void]

u did it in the dsl?

[dark bay]

Yeap

In the TypeScript DSL

[zenith void]

ok ok, i see that's kinda neat

[dark bay]

It magically works around a glsl limitation. (No recursion.)

[zenith void]

so when u see these gpu fractal thingies it's something different?

like space warping w a modulo somehow?

[dark bay]

Normally yeah... I am doing it a very different way, that I think no-one has tried. (But I could be wrong.)

Have you seen this?
Who needs a graphics card?... people power 🤣 :

https://humanshader.com/

You have to shoot a photo of your calculations you made by hand on paper, to prove you didn't use a computer or calculator to cheat.

[zenith void]

Reminds me of how they used to do game of life on grid papers before they had easy access to computer time

I just had a cool idea: for ur audio graph editor, it would be sick of u could visualize intermediary states as spectrograms.

That would be very interesting as an audio guy to be able to visually see the results of ur nodes, and spectograms are such an elegant and cool looking visualization of the frequency spectrum over time.

For graphic node editors having this intermediary visualizations is quite common

But haven't seen it with audio yet

U can add spectrograms to max and pd ofcourse

But having it as this optional default changes how you would approach it, I think

Also in terms of getting overview of the graph: intermediary states will really help i think

Pulls it out of the abstract and into something you can get an intuitive understanding of

[zenith void]

amazing to see that image

ig those wonky pixels are miscalculations?

really amazing project lol

https://humanshader.com/humanshader_worksheet.pdf

[dark bay]

Yeap... people make mistakes 😉

But not bad, you can tell what the image is.

[zenith void]

ye it's amazing

those wonky pixels are icing on the cake

they tell the story

reminds me of this brian eno quote: Whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided. It’s the sound of failure: so much modern art is the sound of things going out of control, of a medium pushing to its limits and breaking apart. The distorted guitar sound is the sound of something too loud for the medium supposed to carry it. The blues singer with the cracked voice is the sound of an emotional cry too powerful for the throat that releases it. The excitement of grainy film, of bleached-out black and white, is the excitement of witnessing events too momentous for the medium assigned to record them.

[dark bay]

Yeah. I remember that quote.

[dark bay]

It's doable.

It's even possible to provide an secondary interpreter for the music DSL to be interpreted in glsl for visual effects that are in time with the music.

Tricky but possible.

[zenith void]

oof

that is kind of sick

not serializing the outputs

but letting the data flow through the same graph

just different environment

[dark bay]

Yeah, pure data in a different environment.

... in Haskell that's called a Free Monad...

[dark bay]

You don't have to learn this. Just incase you find it interesting:

https://typelevel.org/cats/datatypes/freemonad.html

Gotta sleep now. Catch ya later.

[zenith void]

Sleep well random!

[zenith void]

Implementing fft from scratch on gpu is not going to be so straightforward tho, if u wanna visualize the frequency domain

Someone else prob already did the work before

[dark bay]

Recursion demo... the performance was worse than I expected:

import * as $ from "prelude";

let t = $.number({ value: 20000.0, });

function holes(
  level: number,
  size: number,
  offsetX: number,
  offsetY: number,
  offsetZ: number,
) {
  let s = $.number({ value: size / 3.0, });
  let b1 = $.box({
    size: $.vec3({
      x: s,
      y: s,
      z: t,
    }),
  });
  let b2 = $.box({
    size: $.vec3({
      x: s,
      y: t,
      z: s,
    }),
  });
  let b3 = $.box({
    size: $.vec3({
      x: t,
      y: s,
      z: s,
    }),
  });
  let b = $.union({
    model1: b1,
    model2: $.union({
      model1: b2,
      model2: b3,
    }),
  });
  if (level == 1) {
    return $.translate({
      model: b,
      offset: $.vec3({
        x: $.number({ value: offsetX, }),
        y: $.number({ value: offsetY, }),
        z: $.number({ value: offsetZ, })
      })
    });
  }
  for (let i = -1; i <= 1; ++i) {
    for (let j = -1; j <= 1; ++j) {
      if (i == 0 && j == 0) {
        continue;
      }
      for (let k = -1; k <= 1; ++k) {
        if ((i == 0 || j == 0) && k == 0) {
          continue;
        }
        b = $.union({
          model1: b,
          model2: holes(
            level - 1,
            size / 3.0,
            offsetX + i * size / 3.0,
            offsetY + j * size / 3.0,
            offsetZ + k * size / 3.0,
          )
        })
      }
    }
  }
  return $.translate({
    model: b,
    offset: $.vec3({
      x: $.number({ value: offsetX, }),
      y: $.number({ value: offsetY, }),
      z: $.number({ value: offsetZ, })
    })
  })
}

let b = $.box({
  size: $.number({ value: 4000.0, }),
})

$.display({
  visible: true,
  in:
    $.difference({
      model1: b,
      model2: holes(
        2,
        4000,
        0,
        0,
        0,
      )
    })
})

That's probably why no-one else does it this way... it tanks the performance 🤣

---

This blue wing is the unrolled recursion:

Going one more level deeper in the recursion seems to freeze it up on my phone.

[dark bay]

Its probably one area where bounding boxes to speed up ray marching will help.

[zenith void]

loool

[dark bay]

And that fractal is probably simple enough for the modulo trick anyway.

[dark bay]

---

I guess on to marching cubes when I can find the energy.

[zenith void]

exciting!

random thought: what would sdf based physics look like

feels like there is overlap between sdf and like surface pressure

you can think of it as describing force fields

[dark bay]

I have no idea.

Force fields yes, they can be represented as distance fields ig.

I know voxels can be used to simulate water and flames.

[zenith void]

mb they are already quite similar actually

like bounding box hard physics

and box sdf

both describe the shape in mathematical terms and then to run it through some more calculations

[dark bay]

I was pleased to see SDFs used for rendering text. It's a really good use case.

The text never gets blurry ot pixelated. It remains crisp from all viewing distances and angles.

Really cool.

[zenith void]

100%

that's how i learned of them first

rendering text in unity

[dark bay]

I learnt about them 1st from those crazy 4K demos from Inigo.

[zenith void]

really like those maps that it makes of the fonts. very brat coded

the ones w rgb are cool too

[dark bay]

There is probably nothing you can't do graphically with SDFs.
Just a question of how to optimise them.

Bounding boxes will not always help. Because the GPU bulk operates fragments though the instructions. If two simultaneous fragments take two different branches of an if-statement, the fragment shader still has to wait until both branches complete before it can keep going with it's bulk fragment operations.

Would be nice if the GPU chooses groups of simultaneous fragments to operate on such that each group is tightly bundled together screen distance-wise.

[zenith void]

Can u elaborate on this?

[dark bay]

GPUs are like SIMDs. If you branch, you wait for both sides of a branch to complete, if the data your processing happens to fall down both sides of the branch.

Probably why Inigo says the more branching you can avoid in your shaders the better.

He showed an example of an SDF for a box using branching, then he optimised it by reformulating it to avoid branching.

(He did 1/8th of the box and mirrored it into the other octants by taking the absolute value of the input point)

That mirror trick gives him an 8x speed boost on the box SDF, because of 1/8th of the number of instructors to process per fragment.

[dark bay]

---

But branching can be an advantage too if the amount of time spent on each side of the branch is balanced.

Early bail if-statements make less sense in the shader. But balanced if-statements are OK.

So any bounding box tree should be balanced for best performance.

[dark bay]

---

Bounding box trick almost worked... its smooth to rotate, but not rendering correctly yet.

[dark bay]

Will be a little tricky to solve.

[zenith void]

what's the bug?

[dark bay]

Just a silly error on my part.

I miss-used the bounding box.

Having more luck now, but gl compile is taken a long time.

And I think their is a time limit for compiling a shader.

Moves smoothly, but if I change the depth from 3 to 4, then gl compile refuses to compile it.

Bounding box trick is working anyway.

I guess Inigo must encode his bounding box tree in a texture for rendering triangle soups using ray marching (without hitting the code limit)

I know there is a maximum amount of code your allowed to have in a shader.

I keep forgetting that I am on a mobile too. The mobile probably has tighter limits. :p

Well the node graph is a little insane, when zoom out to see it all, it is just a thick blue line:

I'll work her out. Just gotta minimise the amount of code/nodes produced.

I made a balanced union helper, to help produce a balanced tree:

function union(bs: any[]): any {
  if (bs.length == 1) {
    return bs[0];
  } else {
    let i = bs.length >> 1;
    return $.union({
      model1: union(bs.slice(0, i)),
      model2: union(bs.slice(i)),
    });
  }
}

You can see the graph nodes have a balanced pattern from that helper:

I'll work out how to go deeper without bugs. Pretty sure it can be done.

It's probably because I am unioning the frame boxes instead of subtracting the cross boxes from a box. Thus producing too much code for gl compile to handle.

[dark bay]

---

I did a bit of research. GLSL fragment shaders seem to have a limit of 65K gpu asm instruments once gl comile compiles your shader.

[dark bay]

Many people online saying that you shouldn't construct massive shaders, and you should rely more of triangle soups via vertex buffers. 😜

They say if your shader is big, you will hurt your FPS...
But it does not seem to be the case when you are using a bounding box tree.

[dark bay]

There is however something that will not take up a lot of lines of glsl code, and is a natural fit for a bounding box tree..... voxels.

We could do a minecraft rendering engine without the use of a triangle soup.

A voxel model editor might be fun too.

Of course... Inigo has already done it... that legend:

https://www.shadertoy.com/view/4dfGzs

Geez thats cool!

And it's dynamic... you can see all the blocks changing (constructed/deconstruction) at the same time... triangle soups optimisations would struggle with that.

[zenith void]

Siiick

Really need to start binging inigo articles

[dark bay]

Inigo's online activity has reduced lately, but the stuff he did the past is still amazing by today's standards.

[dark bay]

just a link for myself... the old laptop connected to the TV is not powerful enough:
https://pastebin.com/raw/GqaXXCbM

Adding the bounding boxes must be pushing it over the gpu instruction count limit or something for going 1 level deeper.

Trying the other way with fractal junctions and then subtracting them from a box at the end:

But 3 levels deep. I get a linking error from gl. :p

Might give that fractal thing a miss. I think I am pushing my luck with gpu shader instruction limits.

[dark bay]

---

Onto the Marching Cubes... again... 😜... been putting it off because of that 256 branch if-statement.

If possible, I wanna code-gen those branches, so I know every branch is free from human error. (Without the need to test every branch.)

Just gonna find the pattern that can be used to derive the code gen.

Lots of combinations should be able to be reduced via rotational and reflective symmetries.

With a bit of care, there should be very few combinations to handle with all the symmetries in play.

If was only 2D marching squares, we only have two cases, and the rest are a rotation or reflection of those two cases.

I suppose to hit the ground running, I could implement the rotations and reflection transformations, throw in an if-statement for a couple of cases, and iterable all the combination to discover the uncovered cases under the symmetries.

Because it is a bit hard to mentally visualise how many cases are covered under those symmetries.

[dark bay]

Made a start and got kicked off :p
Variables rotation for codegen for marching cubes:

function rotateX(corners: SDFCornerVariables) {
  let tmp = corners[MIN_X_MIN_Y_MIN_Z_IDX];
  corners[MIN_X_MIN_Y_MIN_Z_IDX] = corners[MIN_X_MIN_Y_MAX_Z_IDX];
  corners[MIN_X_MIN_Y_MAX_Z_IDX] = corners[MIN_X_MAX_Y_MAX_Z_IDX];
  corners[MIN_X_MAX_Y_MAX_Z_IDX] = corners[MIN_X_MAX_Y_MIN_Z_IDX];
  corners[MIN_X_MAX_Y_MIN_Z_IDX] = tmp;
  tmp = corners[MAX_X_MIN_Y_MIN_Z_IDX];
  corners[MAX_X_MIN_Y_MIN_Z_IDX] = corners[MAX_X_MIN_Y_MAX_Z_IDX];
  corners[MAX_X_MIN_Y_MAX_Z_IDX] = corners[MAX_X_MAX_Y_MAX_Z_IDX];
  corners[MAX_X_MAX_Y_MAX_Z_IDX] = corners[MAX_X_MAX_Y_MIN_Z_IDX];
  corners[MAX_X_MAX_Y_MIN_Z_IDX] = tmp;
}

I can use those to help cover many cases with a few cases.

So I got rotateX, rotateY, rotateZ and reflectX. If I cycles over all the combinations of those four. It should be enough to discover all the reusable cases.

I have a feeling for marching cubes there is only 4 cases to handle, and everything else is a rotation of reflection of those 4 cases. Which is more manageable than 256 cases.

[vocal saffron]

That's hard! Auto skinning is impossible to get right without manual touch up. At some point there will be AIs to smooth out weight paints.

Btw my friend got https://webgl.com and https://webgpu.com and we're gonna start getting content onto there. Would love to feature projects on there, such as the 3D node graph thing you're making.

[dark bay]

How on earth did your friend get those domain names?

[vocal saffron]

$$$$

He also snatched https://motions.io!

Nothing there yet tho

[dark bay]

Something like webgl.com has a mozilla development network or similar ownership sort of vibe to it. Like the web standards ppl. Was surprised he could grab that at all.

[dark bay]

---

I threw in some code for marching cubes, unused / untested. Hopefully it will open up the doors to 3D printing some SDF models:

https://github.com/clinuxrulz/tool-kitty/blob/main/packages/tool-kitty-marching-cubes/src/index.ts

I went the table way, rather than the codegen way. I was getting too confused 😛

[dark bay]

... just hit a roadblock... the code gen for the SDF function is in glsl, not JavaScript.
I will have to code gen the JavaScript code as well. 😅

Thought I was almost done, then realised my SDF can only be executed on the graphics card.

Well... atleast it also opens up the doors to selecting the SDF nodes in the 3d rendered model with the mouse.

[zenith void]

i mean 🤔 u can also render to a buffer and read from this? like a compute shader?

but javascript code gen probably smarter in the long end

reading from gpu buffer is a bit costly

with webgpu it would be more of an option though

i was thinking of webgpu's approach and keeping as much of the data within the gpu as possible

u could do something similar w webgl too ig

render to a texture in shader 1, read from texture 1 in shader 2, ...

mb that's also how u could bypass shader complexity bugs?

split it up

brainfart: could be interesting if u would write code like all these small modules and then the compiler arranges it into shaders itself, sort of like a build process, inlining as much as it can into a single shader, and only splitting up if it's necessary/more performant for some reason.

[dark bay]

I'd probably need a proper IR as a immediate format.

But I won't worry about that for now. I'll just have a pair of codegen callbacks per node type. One for glsl, one for javascript.

[zenith void]

yes, maybe it's a DSL representing a graph 😉

[dark bay]

2 sources of truth 😅

[zenith void]

or the same source of truth?

if each module is also a single node

that's kind of interesting about DSLs ig, u can always change what they compile to

i never really played around too much with webgpu, fun how javascript-y it is inside the expressions https://www.w3.org/TR/WGSL/#intro

[dark bay]

But yes, we could use a texture to evaluate the SDF function on the graphics card for many points at once and receive the result off the display buffer.

The actual code for each of the SDF node is very basic. Probably won't hurt having code-gen in pairs.

[zenith void]

just generating a bit of text in the end

[dark bay]

There will be a refactor comming later down the line too.

I realised I don't need to have a unique ECS component per node type.

I can just have an ECS component called node. A general one. And still have the separate node type registry.

The node type registry will be responsible for the up comming type system for the node graph.

[zenith void]

what do you need the ECS for?

i thought that was for game logic

but you are also evaluating the graph with it?

[dark bay]

Just handy... the ECS will automatically work with automerge down the line.

Just with a one line change.

[zenith void]

that is nice

[dark bay]

Ecs is not really required here.

[zenith void]

graph on a graph

[dark bay]

An overkill tool. :p

[zenith void]

graphs all the way down

[dark bay]

And serialization and validation is already built into it.

Plus it's reactive.

[zenith void]

no ye that's nice to have a single way of modelling graphs throughout toolkitty

[dark bay]

Buy yeah... if it was split off as a separate library... ppl be like "whhhyyy?"

[dark bay]

That idea is probably faster with a bit of care.

Like a buffer of the right size could be prepared in the beginning. And the gpu just keeps overwriting that buffer.

(An animating marching cubes)

[zenith void]

ye exactly

u would need a way to calculate these buffer sizes

but i think u already have all the information u need to do that

[dark bay]

Yeah, can do worst case.

[zenith void]

could probably also have a big scratch buffer

resize if needed

[dark bay]

I think worst case is 4 triangles per cube, but would need to double check.

[zenith void]

it's like an rpc type thing

smalltalk

u just need the buffer for the hand off

then u can re-use as u please

[dark bay]

That's what I will do then. Seems to make sense.

Can even just do multiple visits to the gpu if the buffer happened to be too small. Breaking it up into parts.

It probably be a Termux job, but doesn't seem hard.

[zenith void]

o ye also true

that's kind of nice w the gpu stuff

ur so pushed to thinking in parrallel

[dark bay]

map reduce 😜

Except doing reduce is a little painful with webgl i think.

Can't help but wonder if the actual marching cubes algorithm could happen in the vertex shader.

But kinda want the mesh result out for printing.

Actually that would probably be the geometry shader, not the vertex shader maybe.

Most of what I know is just the fragment shader. :p, because it is the most fun ig.

[zenith void]

the most psychedelic one

[dark bay]

I should use OffscreenCanvas for the sdf evaluator.

[zenith void]

True

Mb some sharedarraybuffer baffoonery

Or pass ownership of the buffer around

[dark bay]

Likely a 2mrw job. Bit on the tired side. :p

Pretty cool that marching cubes is not far off.

[zenith void]

Very cool!

[dark bay]

It's even possible to do a partial marching cubes to implement real-time destructive terrains.

Would also be another way to sculpt.

[zenith void]

i have seen it also where it's only used for the physics

so you do all the collision stuff w the marching cubes

but render the sdf

[dark bay]

oh... webgl version 2 for uniform buffer objects. Luckly webgl v2 has good browser support (since 2022?)

it's weird that webgpu has better support on mobile devices than desktop systems. Normally that stuff is the other way around.

I might stick with a texture... feels like too much mucking around 😛

or could I do better?, the points in the texture for the evaluation will be spaced evenly in a lattice shape... I can just compute the points inside the shader that I want to evaluate the SDF against.

[dark bay]

---

Here is the first draft:

  sdfEvaluatorCodeGen(params: {
    numCubesX: number,
    numCubesY: number,
    numCubesZ: number,
  }) {
    return glsl`precision highp float;
${uniform.ivec2("uResolution")}
${uniform.vec3("uEvalMin")}
${uniform.float("uCubeSize")}

float dot2( in vec2 v ) { return dot(v,v); }
float dot2( in vec3 v ) { return dot(v,v); }
float ndot( in vec2 a, in vec2 b ) { return a.x*b.x - a.y*b.y; }

void defaultColour(vec3 p, out vec4 c) {
  c = vec4(1.0, 1.0, 1.0, 1.0);
}

${this.globalCode}

float map(vec3 p) {
  float d = 100000000.0;
  ${this.mainBody}
  return d;
}

vec4 packFloatToRGBA(float v) {
    // This function relies on the underlying bit representation and
    // works for both positive and negative floats when read back correctly in JS.
    const vec4 bitShifts = vec4(1.0, 256.0, 65536.0, 16777216.0); 
    const vec4 bitMask  = vec4(1.0/255.0, 1.0/255.0, 1.0/255.0, 0.0);

    vec4 enc = fract(v * bitShifts);
    enc -= enc.yzww * bitMask; // Remove the overflow from the next component
    return enc;
}

void main(void) {
  int idx = gl_FragCoord.y * uResolution.x + gl_FragCoord.x;
  int cubeZIdx = floor(idx / ${params.numCubesX * params.numCubesY});
  idx = idx - cubeZIdx * ${params.numCubesX * params.numCubesY};
  int cubeYIdx = floor(idx / ${params.numCubesX});
  idx = idx - cubeYIdx * ${params.numCubesX};
  int cubeXIdx = idx;
  vec3 p = uEvalMin + vec3(cubeXIdx, cubeYIdx, cubeZIdx) * uCubeSize;
  float r = map(p);
  gl_FragColor = packFloatToRGBA(r);
}
`;
  }

No idea if it compiles.

That packFloatToRGBA was AI generated... no idea if I can trust it.

[dark bay]

That packFloatToRGBA just looks too simple to possibly work. I think AI is daydreaming again.

I want to get float points back instead of -1, 0 and 1 so that I can interpolate the points from the ray marcher for a better fit with the SDF surface.

[dark bay]

Nah... I reckon AI is on drugs... I'll have to derive my own packFloatToRGBA.

Really don't like how AI's answer is at the top of Google search. Especially when it is blatantly wrong.

This looks more sensible:
https://github.com/mikolalysenko/glsl-read-float/blob/master/index.glsl

[dark bay]

This is tidy:
An offline canvas of size 1000x1000 can handle 100x100x100 SDF eval chunks at a time. Nice round numbers. Unusually base 10 friendly.

• 8x8 -> 4x4x4
• 27x27 -> 9x9x9
• 64x64 -> 16x16x16
• 125x125 -> 25x25x25
• 216x216 -> 36x36x36
• 343x343 -> 49x49x49
• 512x512 -> 64x64x64
• 729x729 -> 81x81x81
  . . .
  There is a pattern here:
• (n³)x(n³) -> (n²)x(n²)x(n²)

(3x2 == 2x3)

[dark bay]

The 512 is attractive being a 2 power (bit shifts), I'll probably use it for my chunks.

[dark bay]

---

Some more untest code... an sdf eval chunk fetcher... fetches sdf values one 64x64x64 chunk at a time from the gpu:

https://github.com/clinuxrulz/tool-kitty/blob/main/packages/tool-kitty-model-editor/src/MarchingCubes.tsx

Just gotta glue all the parts together, debug it and see what we get.

I do wonder if it would be better todo several marching cubes of smaller chunks and join the results together, or do one large marching cubes to get the result in one go. The former would have lower memory usage.

[dark bay]

Wonder what app this guy is using to sculpt with, it looks really neat:
https://youtu.be/JZJp-glUB1k?si=bxIF8ikTyxylTUZ5

[dark bay]

Not quite there yet. The same positive float point number got returned from the sdf for all points for the ray marcher.
The flow is all there, just gotta debug her a bit.

[dark bay]

---

Some success, we're now reading float point values from the SDF calls back from the graphics card via the offscreen canvas:

our very own webgl 1.0 compute shader

[dark bay]

OK... something looks odd... marching cubes generated a mesh with 256,224 triangles for just a simple sphere, using 100x100x100 grid of cubes for the algorithm.

Minor bug fix. I'm now down to 140,265 triangles for a sphere. Still feels excessively high for a 100x100x100 grid generating a mesh for a surface of a sphere. Will need to render it and see what it looks like.

Well.. its 4.pi.r², iirc for the surface area of a sphere. So 4×pi×50×50 = 31,415. So I'd expect around 60,000 triangles roughly.

Maybe it is in the ballpark, but its not an ideal tessellation.

[dark bay]

Oh ... hooked up threejs and added it to the scene. Can not see anything yet though. More debugging.

[dark bay]

Ohh... I see something forming:

A small spec of something in the bottom left.

[dark bay]

---

Hi Pomni 👋:

Bit more bug fixing todo.

[dark bay]

Definitely some bugs in he generated triangle soup:

[zenith void]

sick progress! let's gooo

[zenith void]

very cool!

[dark bay]

The breaks in the meshes are likely I got my X,Y,Zs in the wrong order in the marching cubes lookup table. And the cut off repeating block, a bug in writing the result chunks back from the graphs card to the grid storage.

[dark bay]

Switching off interpolation for marching cubes shows we are close:

I'll need interpolation back on to make the mesh smoother though. (Less lumpy). Otherwise its gonna be an ugly 3d printing experience 😅

[zenith void]

Let's goooooooo

Very cool

Did the 3d printer for ur daughter already arrive?

[dark bay]

Still deciding to be honest. (Not ordered yet.)

[dark bay]

This one seems pretty cheap:

https://www.creality.store/products/creality-ender-3-v3-se-3d-printer?srsltid=AfmBOooP7M8HGVRQx3_bNceYaSpMh9pyR5paHntkj3LkVmiP405k6x1_dnI

Could just print in white and she could paint it.

[dark bay]

I think it's more likely my daughter would use something like "Tinkercad".
ToolKitty is a bit far off user friendliness. 😅

[dark bay]

Interpolation in marching cubes fixed:

Time to work out why there is that missing slice.

I wonder if it is because gl_FragCoord is in the centre of the pixels instead of the corner. That might be messing up my calcs.

(Bottom left gl_FragCoord starts at (0.5,0.5) instead of (0,0))

Yeap! That was it!... centre of pixels vs corner of pixels:

We now have fully working marching cubes. 🥳 🎉

[zenith void]

siiiiiick

that's a milestone!

[zenith void]

damn looks really good!!!!

[zenith void]

ToolKitty is a bit far off user friendliness. 😅
i mean... it is now! but now that u have the marching cubes and can get data out of the shader u pretty much have everything u need to get screen-interaction working too

i don't think ur that far off from having a neat little sdf modeller

[dark bay]

Yeah... flow is kinda there.

I like how 512×512 = 64×64×64 ... each dimension a power of 2.
(Divisions and modulos become bitshifts and bitmasks)

[dark bay]

Marching cubes is able to do a pretty good job:

(Has a mix of sharp flat surfaces interfacing curved surfaces.)

And garenteed manifold.

[dark bay]

Almost forgot.
You have my permission to host the sdf modeller on webgl.com however you see fit.
I haven't assigned a licence. But I intend it to be fully open source and free to use.
Aiming for the most permissive licence that does not hold me liable for anything.

There are a few user friendliness things to sort out. I'll sweep over it when I get time.

iframe or shadow dom is probably best to avoid conflicting CSS.

[dark bay]

Maybe MIT licence is the best permissive one to use.

• Use, modify, and distribute freely.
• No warranty or liability.
• Allows use in closed source / proprietary software.

GPL kinda sucks, because any code uses it also needs to be open source from what I understand.

Let's go the "The Unlicensed" License

Fully public domain

The "The Unlicensed" License had been added.

[dark bay]

@zenith void If you have a moment, I have some type errors to work out when using @bigmistqke/view.gl. At runtime the code still works though.

Just under this file here:
https://github.com/clinuxrulz/tool-kitty/blob/main/packages/tool-kitty-model-editor/src/ModelEditor.tsx

I have this GLSL type I take in through props for a solidjs component:

sdfEvalCode: GLSL<[{
  type: "uniform";
  kind: "ivec3";
  key: "uNumCubes";
}, {
  type: "uniform";
  kind: "ivec2";
  key: "uResolution";
}, {
  type: "uniform";
  kind: "vec3";
  key: "uEvalMin";
}, {
  type: "uniform";
  kind: "float";
  key: "uCubeSize";
}, any, any]>,

It then gets compile to a opengl program:

  let vs = glsl`attribute vec4 aVertexPosition;
uniform mat4 uModelViewMatrix;
void main(void) {
  gl_Position = uModelViewMatrix * aVertexPosition;
}`;
  let program = compile(
    gl,
    vs,
    props.sdfEvalCode,
  );

Here is the inferred type signature of the program from vscode:

let program: {
    program: WebGLProgram;
    schema: MergeGLSLSchema<GLSLSlotsToSchema<[]>, GLSLSlotsToSchema<[{
        type: "uniform";
        kind: "ivec3";
        key: "uNumCubes";
    }, {
        type: "uniform";
        kind: "ivec2";
        key: "uResolution";
    }, {
        type: "uniform";
        kind: "vec3";
        key: "uEvalMin";
    }, {
        type: "uniform";
        kind: "float";
        key: "uCubeSize";
    }, any, any]> | GLSLSlotsToSchema<[{
        type: "uniform";
        kind: "ivec3";
        key: "uNumCubes";
    }, {
        type: "uniform";
        kind: "ivec2";
        key: "uResolution";
    }, {
        type: "uniform";
        kind: "vec3";
        key: "uEvalMin";
    }, {
        type: "uniform";
        kind: "float";
        key: "uCubeSize";
    }]> | GLSLSlotsToSchema<...>, ViewSchemaPartial>;
    view: {
        ...;
    };
    vertex: string;
    fragment: string;
}

However the inferred type of program.view is:

{
    attributes: {};
    interleavedAttributes: {};
    buffers: never;
    uniforms: {
        [x: string]: {
            set(arg: Float32Array<ArrayBufferLike>): void;
        } | {
            set(...args: any): void;
        };
    };
}

And that causes these 3 to fail their type check, even though they work at runtime:

program.view.uniforms["uNumCubes"].set(
  chunkNumCubesX,
  chunkNumCubesY,
  chunkNumCubesZ,
);
program.view.uniforms["uResolution"].set(
  width,
  height,
);
program.view.uniforms["uCubeSize"].set(
  cubeSize,
);

maybe the vertex shader also needs to declare the same uniforms?

The , any, any]> part of the first GLSL type signature is from injecting text source code into the glsl' block I believe.

[dark bay]

Could be a tsconfig thing too. (Type recursion limit maybe.)

[dark bay]

I might focus on edge selection now to help improve user friendliness. Inigo has an SDF for a quadratic bezier curve (using SDF for mouse/touch selection), however the s-bend edges are cubic bezier curves. I will need to research a bit more for mouse/touch selection of cubic beziers.

Apparently producing an SDF function for a cubic bezier involves solving 5th degree polynomials, of which no general solution exists.

However. Maybe s-bends are a special case where a general solution exists.
Or even under a different distance type (non-euclidean) a general solution may exist, which would be perfectly suitable for mouse selection of curves.

Doing a google search, they say to just break the cubic bezier curve up into 100 straight line segments and just measure the mouse distance from the straight line segments. Feels a bit barberic, but maybe it is the only solution out there that is practical.

Obviously you'd do your bounding box tests first, so your not testing again 100x lines.

---

Come to think of it... why not just draw the s-bend edges using multiple quadratic beziers instead of a single cubic bezier where a general sdf function exists.

Bounding boxes + quadratic bezier curves for sdf for mouse selection seems to be the way to go.

[dark bay]

.... oh.... three.js already had marching cubes in its add-ons 😅 :

https://threejs.org/examples/?q=marching#webgl_marchingcubes

Oh well.

Selecting 100 blobs in here reminds me of the fluids in the game "Portal 2":

https://webglsamples.org/blob/blob.html

Seems like the must use marching cubes for their liquid animations.

[zenith void]

ye 🤔 somewhere it gets widened

i remember i also hit a limit on what was inferable, but that was the due to symbols

[zenith void]

i cannot seem to find that snipppet ur describing

[zenith void]

no, that shouldn't be necessary

[dark bay]

Not too worried about it yet.
Might try that other way compile.toSchema.

[dark bay]

Sorry sent you the wrong link. 1 sec.

[dark bay]

This is the file:
https://github.com/clinuxrulz/tool-kitty/blob/main/packages/tool-kitty-model-editor/src/MarchingCubes.tsx

They both started with M and I picked the wrong one, sorry 😅

[zenith void]

no prob!

o

so it's coming from ```tsx
GLSL<[{
type: "uniform";
kind: "ivec3";
key: "uNumCubes";
}, {
type: "uniform";
kind: "ivec2";
key: "uResolution";
}, {
type: "uniform";
kind: "vec3";
key: "uEvalMin";
}, {
type: "uniform";
kind: "float";
key: "uCubeSize";
}, any, any]>

[dark bay]

Maybe the anys tricking it up?

[zenith void]

ye exactly

what are they for again?

o ye

for sure

it's the any

can't u replace it with string?

[dark bay]

They are the code injections 💉

From the nodes.

[zenith void]

with GLSL then?

[dark bay]

Yeap, GLSL in GLSL

[zenith void]

it's definitely this

the inference problem

[dark bay]

I'm not on a computer. But if I replace the anys with GLSL, maybe I'll be fine.

Ah... because it merges uniforms and attributes together from all blocks recursively.

[zenith void]

exactly

kinda cool

u could have control over the whole type if u want to

like what type of GLSLs can be injected 💉

typing your children

[dark bay]

Its a bit random for the injected GLSL types, because they originate from a plugin like system.

[zenith void]

ye in ur case it doesn't make sense

or right now it doesn't

not sure if i can filter out any

maybe unknown could also work

[dark bay]

I suppose GLSL<[]> from the injected might be ok.

[zenith void]

ye or GLSL, it has some default generic too

[dark bay]

Its weird, because it is like they have private uniforms.

[zenith void]

GLSLSlot[] probably

[dark bay]

Like private fields of a class for example, but as uniforms.

[zenith void]

exactly!

especially w the symbol stuff

[dark bay]

Not overly worried. But it is interesting to think about how it fits together.

Always have type casts as bandaids.

[zenith void]

ye i m pretty happy w the lib

it's so tiny

but it's powerful

[dark bay]

Existential types....

exists A: GLSL<A> is sort of what we need.

[zenith void]

the other day i logically ended up with Continuations when trying to solve a type question

i wanted to invert the branching of a condition

[dark bay]

Interesting. Yeap Cont definitely pop ups everywhere.

Cont is the mother of all monads.

[zenith void]

it's sort of like the tokenize and evaluate approach, but then it is in a single step inside the data-structure itself instead of being these 2 clearly defined, separate steps

[dark bay]

<exists A>: GLSL<A>

becomes

type ExistsGLSL = <R>(k: <A>(glsl: GLSL<A>) => R) => R

Thats the exists turned into forall encoding.

[zenith void]

Cont feel like a balloon being blown up and then being turned inside out

[dark bay]

Yeah... the remainder of the computation is the input parameter to Cont.

That's what gives it crazy superpowers.

It can skip the remainder of the computation, repeat the remainder of the computation, place the remainder in a list for later execution, ....

All sorts of stuff.

Obviously you can overflow the stack easy without care. And it does get confusing when reasoning about its control flow.

Like a function, but instead of returning a value, it is feed the next line of code in the input parameter... kinda.

Seems like it would make sense for unknown to be an existential type parameter. But not sure if it can work that way.

Gemini says... no... unknown can not really represent an existential type, but does get close.

[dark bay]

The constructor for this encoding might clear things up.

function makeExistsGLSL<A>(glsl: GLSL<A>): ExistsGLSL {
  return (k) => k(glsl);
}

And the runner:

function runExistsGLSL<R>(glsl: ExistsGLSL, k: <A>(glsl: GLSL<A>) => R): R {
  return glsl(k);
}

Thats how to wrap up the exists, and unwrap the exists using that encoding.

The type parameter <A> magically dissapears and reappears without breaking the type system, and without doing type casting.

.... OK.... other ppl stole my encoding too 😅 :

https://andreasimonecosta.dev/posts/parsing-raw-data-with-existential-types/

[dark bay]

You know what?....

type ExistsGLSL = GLSL<any>
🤣

Why not?

[dark bay]

You can still pull out a type parameter:

function runExistsGLSL<R>(glsl: ExistsGLSL, callback: <A>(glsl: GLSL<A>) => R) => R {
  return k(glsl);
}

Feels sensible.

Reason for this is so the type system knows which any types are the same and which any types are different when more complex logic is involves.

[dark bay]

I.E. associates anonymous type names to any types (via generics), thus achieving existential types.

Simple as and does not involve Continuations... but kinda does still, but appears as a callback so the scary stuff is hidden.

[dark bay]

... just use GLSL<any>... :p

[dark bay]

Here is a weakness of unknown:

let x: Foo<unknown> = ...
let y = x;

function bar<A>(x: Foo<A>, y: Foo<A>) {
  . . .
}

bar(x, y);

Will not type check. Even though it is certain x and y have the same unknown type. This is what existential types solve. unknown is not quite there yet.

[dark bay]

And using any has the opposite problem where it will make stuff type check that shouldn't type check.

Why can't the TypeScript team just throw in exists and be done with it. :p

[dark bay]

---

Anyway... doing sdf of beziers for edge selection now.

[dark bay]

OK... so:

• blue is the original cubic bezier curve, that does not have a well defined SDF function.
• green is the new two quadratic bezier curves that do have a well defined SDF function.
  The new green ones allow for a function to easily test for the mouse pointer being a certain number of pixels within the edge. (say within 10 pixel distance for edge selection.)

The green lines are very close to the blue lines. They should make a good substitute.

[zenith void]

lesgoo

[dark bay]

Yeah... its not as exciting, but user friendliness needs to improve 😅

The grunt work.

[dark bay]

Why I think this is better than comparing the mouse position from a bunch of line segments is beyond me 😅 :

  sdf(pt: Vec2): number {
    let A = this.start;
    let B = this.controlPoint;
    let C = this.end;
    let a = B.sub(A);
    let b = A.add(B.multScalar(2.0)).add(C);
    let c = A.multScalar(2.0);
    let d = A.sub(pt);
    let kk = 1.0 / b.dot(b);
    let kx = kk * a.dot(b);
    let ky = kk * (2.0*a.dot(a)+d.dot(b)) / 3.0;
    let kz = kk * d.dot(a);
    let res = 0.0;
    let p = ky - kx*kx;
    let p3 = p*p*p;
    let q = kx*(2.0*kx*kx-3.0*ky) + kz;
    let h = q*q + 4.0*p3;
    if(h >= 0.0) { 
        h = Math.sqrt(h);
        let x = Vec2.create(h - q, -h - q).multScalar(0.5);
        let uv = Vec2.create(
          Math.sign(x.x)*Math.pow(Math.abs(x.x), 1.0 / 3.0),
          Math.sign(x.y)*Math.pow(Math.abs(x.y), 1.0 / 3.0),
        );
        let t = clamp(uv.x+uv.y-kx, 0.0, 1.0 );
        res = dot2(d.add(c.add(b.multScalar(t)).multScalar(t)));
    } else {
        let z = Math.sqrt(-p);
        let v = Math.acos(q / (p*z*2.0) / 3.0);
        let m = Math.cos(v);
        let n = Math.sin(v) * 1.732050808;
        let t = clamp(
          Vec3.create(
            m+m,-n-m,n-m
          ).scale(z).sub(Vec3.create(kx, kx, kx)),
          0.0,
          1.0,
        );
        res = Math.min(
          dot2(d.add(c.add(b.multScalar(t.x)).multScalar(t.x))),
          dot2(d.add(c.add(b.multScalar(t.y)).multScalar(t.y))),
        );
    }
    return Math.sqrt(res);
  }

There is a bug in that formula, now to find it.

[dark bay]

yeah... definitely messed up that SDF formula for the quadratic bezier:

(debugging by plotting the SDF)

[dark bay]

---

TADA:

We can now determine if a mouse event or a touch events occurs over an edge.

Visual debugging beats staring at numbers. :p

[dark bay]

Pretty wild looking SDF formula, but most of the formula can be constant folded for a fixed bezier. (Precompute everything that does not directly depend on pt)

[zenith void]

Sooooo sick!!!

That is really handy..........

[dark bay]

Stolen Inigo's formula for edge selection 😅

[zenith void]

And now I want to steal it from u!

I wonder if we can abstract these pieces out

Make a view.gl/components or something

[dark bay]

Its all cool. I have the "The Unlicenced" licence. It's official now.

Most permissive licence of all.

[zenith void]

What is difference w MIT?

That u can also remove copyrights?

[dark bay]

Yep

Bcuz legal stuff is a pain :p

[zenith void]

I probably should be more aware of these things tbh. Chances are big I broke rules around this stuff.

[dark bay]

I would of broke a heap of rules. So hard to keep track of.

E.G. GNU licence libraries can only be used in open source software.
I never saw that one comming.

[zenith void]

Like do u include inigo's license in toolkitty bc u used their formula?

[zenith void]

Ye true FOSS

[dark bay]

Not yet, but I should.

[zenith void]

It's kinda cool too