Rosy

[cloud rivet]

I am trying out alienbrain too

just going on a software adventure I guess

my green github will go dark though if I like alienbrain

doesn't work at all for me

idk haven't been able to get a perforce trial and https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage is pretty limited

[wraith urchin]

Are we doing version control bikeshed now?

Shouldn't you be working on robot controls now?

You can always come back to this

[cloud rivet]

I don't have way to version any of the things

ok ok

[wraith urchin]

I just throw all my shit in git lol

I don't even use LFS

[cloud rivet]

^^ you don't get impacted by those limites I posted?

[wraith urchin]

Those are limits for LFS no?

I'm not doing LFS

[brisk dagger]

Don't worry this will change quick enough when you will see how unstable it can be

WER sits next to each other, so it's handy to quickly switch between TRS mode

Sofitmage was doing the same

[cloud rivet]

yay

ok ok

uh

I guess I need to figure out what mesh is what now

oh the texture isn't right on the cube

hrm

looks like the texture coordinates are wrong on that

[cloud rivet]

fixed

made the box a little bigger

I kind of liked it better just orange though

but it was a mistake withour I configured it

this is how it looks in houdini

I dn't have a sky in my engine and none of the fancy lighting yet

this like 2 days of using houdini level of skill I don't think this is the way to do it

anyway

I got enough for a scene to figure out how to move things

hrm

I think I need to know how to identify this mesh as an object in my engine so I can translate it

huh

like maybe a gltf attribute or something

I don't want to like hard code a mesh id

or it's a node actually not a mesh

no just meshes

I need them to be nodes

[brisk chasm]

normals fixed too already?

[cloud rivet]

actually

it's all one mesh and the grid and cube are primitives

[cloud rivet]

there wasn't anything wrong with my normals

I need these to be separate nodes

[brisk chasm]

am i in the wrong channel? 😄

i remember some non properly coloured cube faces

[cloud rivet]

I am really over any lighting nonsense at the moment, sorry, focused on things that won't make me miserable

it's all looking ok to me right now

[brisk chasm]

ah

then i didnt say anything

as you were!

[dry apex]

lmao he sounds like loris

[cloud rivet]

that is loris

it's always interesting when I change something I do to generate a gltf and my app breaks because I am not supporting something

thank god for debuggers

it returned an error it didn't crash or anything at least

MissingExtensions

//< One or more extensions are required by the glTF but not enabled in the Parser.

KHR_lights_punctual

nice I have nodes

[cloud rivet]

I don't have to edit my shader anymore to do some light debugging. Once I add better lights I'll make all that configurable too

[cloud rivet]

this is the light direction, hrm, just doesn't look right

I need an option to switch off tangent space

[cloud rivet]

this is getting weird

I need an option to inverse tangent.w ??

[cloud rivet]

this is my view direction in tangent space, it is completely fucked along the x axis

[cloud rivet]

pretty cool

[cloud rivet]

my view direction is static in sponza :/

lengyel uses an object space camera position

I'm flying around with camera positions in the -5 to 5 range

these fucking sponza vertices are in the thousands

[astral hinge]

modelling tools have scale conventions

[astral hinge]

maybe that model was made with one where 1 unit = 1mm

[cloud rivet]

yeah

[astral hinge]

or maybe 1cm lol

[cloud rivet]

ok

well

that is a bug in my math I didn't know I had

the light direction is also in object space

ok

rm

[astral hinge]

wtf

why does this guy like object space so much

I'm in a hurry to get tf into world or view space

[cloud rivet]

ok well it's like my first progress on lighting issues in days

sponza looks so cool like this

even if it is a bug

I guess I will transform the camera position to object position then

hrm

so that's inverse model to view space

[cloud rivet]

well

I think that fixed a bunch

I still have that seam in the circle banner

[astral hinge]

Have you processed that sponza at all

If not, it may just be the model

And a re-export might help

[cloud rivet]

yeah so I started on mikktspace but then I realize I had it all wrong

I don't understand anything about "processing" models or using mesh optimizer or welding

I don't understand it at all actually

[astral hinge]

In this case I literally just just importing and exporting from blender or something

Nothing more

[cloud rivet]

how you preserve tangents without accounting for the indices

oh I did that

yeah I exported this from blender

[astral hinge]

Ok so it probably is your fault then 😄

[astral hinge]

In what context?

By "accounting for the indices", are you referring to the other triangles that may share the vertex?

[cloud rivet]

the mikktspace docs say something about how the tangents that it creates are not indexed

so you get a bunch of tangents that you do what with

I don't understand it

it has a welder

jake said he uses mesh optimizer

idk anything about it

view space no longer looks broken

let me see how it now looks at my little scene

well it looks cool even when it's not bugged

[astral hinge]

idk either, I've never used it

[cloud rivet]

here's how all these banners look in the various views without lighting/shadows enabled

the third one is just the pure tangent I got from blender

it just comes with a seam

I think maybe I should try run it through houdini or maya

instead of blender

[astral hinge]

do you have back face culling?

[cloud rivet]

yes

I can change it too

doesn't do anything in this case

you can see it on the tangent though that it has a seam

[astral hinge]

yeah

[cloud rivet]

this doesn't look right at all

those are the tangents I got from houdini

[astral hinge]

maybe you forgot to multiply by the tangent.w or something

but shouldn't it be the same for the whole model?

[cloud rivet]

float3 bitangent = cross(normal, tangent) * new_tangent.w;

[astral hinge]

what happens if you don't multiply by w?

[cloud rivet]

looks the same

I think

houdini is not setting the w

oh maybe I need to configure that

all the w's I get from houdini are just 1

oh

I lost my tangent node

my grid still has its

my box lost it

that fixed it

[astral hinge]

https://tenor.com/view/noice-nice-click-gif-8843762

[cloud rivet]

oh man

well

my lighting is unfucked except for that sponza seam

I think maya cannot export gltf

or import it

houdini can open it

I'm gonna try that

nah it tries to transform to obj and then back to gltf and it loses all the materials

[brisk chasm]

why do you keep looking at tangents

visualize normals and uvs, those 2 are input for the former

[cloud rivet]

I am looking at tangents because they're used for normal mapping

and if the tangents are bad then the normal mapping is bad

[brisk chasm]

but its easier to tell if the normals are kaputt and if they are, how kaputt they are

[cloud rivet]

I don't understand what you are suggesting I do

[brisk chasm]

you visualise normals and uvs, as mentioned above

[cloud rivet]

I don't think would do a good job building my own tangents

[brisk chasm]

???

tangents are exported and if the exporter implemented the gltf spec properly are part of the gltf file

[cloud rivet]

houdini gets it wrong in the other axis

[cloud rivet]

the tangents that I am getting from blender and houdini are bad tangents

they are part of the gltf file

and the contents in the gltf file are bad

they have seams

[brisk chasm]

do you multiply w?

[cloud rivet]

yes

[brisk chasm]

how do you load the normals?

[cloud rivet]

by loading them?

[brisk chasm]

i remember like 10 people having the same problem with normal seams because they used the wrong format

[cloud rivet]

the format of which?

[brisk chasm]

normal maps

[cloud rivet]

I am using fastgltf

oh

they are ktx files

it's just those banners that are broken

nothing else

[brisk chasm]

i dont mean the file format, but the image format 🙂

RGBA8_SRGB, R16G16..._FLOAT etc

[cloud rivet]

they are jpegs

but

[astral hinge]

erm

deccer means the API format

[cloud rivet]

maybe the conversion

[astral hinge]

VK_FORMAT

[brisk chasm]

this

[cloud rivet]

libktx does that

let me go look

[brisk chasm]

😛

[cloud rivet]

what was the cause?

[brisk chasm]

#1155133989338746990 message

[cloud rivet]

those looked as fucked mine in the same way

huh

[astral hinge]

wait are you using normal maps?

[cloud rivet]

yes

[astral hinge]

ruh roh

[brisk chasm]

https://tenor.com/view/nani-surprised-what-shocked-scared-gif-17557349

[cloud rivet]

what

[astral hinge]

I saw the ktx command you sent earlier

[brisk dagger]

Thx for the laugh

[astral hinge]

it had srgb and you even mentioned it

[cloud rivet]

no

[brisk chasm]

xD

[cloud rivet]

that was for some old cube map

[astral hinge]

oh

[cloud rivet]

I didn't run ktx

I used the nvida texture tool

[astral hinge]

il al au it

[brisk chasm]

i remember seeing a big fat SRGB in that ktx cmd line too

but perhaps that was for something else

[cloud rivet]

yes that was for something I wrote months ago

[astral hinge]

looks like one of those baboons with a buttface

[brisk chasm]

that seam s wrong, badumm tss 😄

[cloud rivet]

it does

let me redo that

[brisk chasm]

and the alpha channel is also weird

the green part should just be 0.5, 0,5, 1,0, blueish purple

[astral hinge]

better known as ourple

[brisk chasm]

albedo/diffuse, emission, skyboxi are "color" maps, normals are normal :), roughness/metallic/ao are also not coleurs, or mixed ones like ARM

we should have a normal seams incident counter

bjorn, was that the reason that you installed all of those tools?

[silver slate]

It will forever be a mystery.

[cloud rivet]

I wanted to

[cloud rivet]

someone should let martty know that Theory of Mind is a thing and that no one has direct access to the mind of another

yeah so I checked in renderdoc and even after I save them as normal mapped libktx is still using an srgb format

and I looked at the api and it's fully blackbox

so to fix it would require to not use libktx for normal maps

or maybe I can use the ktx tool via some cli arg to tell it do the right thing instead of using the nvidia texture tool

oh

[brisk chasm]

: )

https://tenor.com/view/sad-pull-leg-stick-man-gif-13137776

[cloud rivet]

even when I tell it to not gamma correct libktx still creates the image with srgb

cool

mystery solved, thanks deccer

I need to fix the image loading at some point I guess

I'm not going to worry about it right now though

[brisk chasm]

de rien

[cloud rivet]

so to recap, my lighting issues were:

1. Lengyel's math uses object space camera position and light and I missed that. I fixed and that fixed a whole bunch of things.
2. libktx loads normal maps as SRGB, unresolved
3. I didn't export tangents on one of my meshes in houdini

all good now though, and I have a bunch lighting debug tools now

[brisk chasm]

kewl

[cloud rivet]

normal maps

anyway

now I need to get these meshes to move individually and click in 3d

🇬🇳

[brisk chasm]

normals still wrong, if the camera is sitting at x < 0 then i shouldnt see a bright red facing +x

[cloud rivet]

which picture?

[brisk chasm]

the middle one where the plane is green

[cloud rivet]

what's wrong about it?

[brisk chasm]

if you ignore everything so far, and picture a unit cube with normals

and turn each cube's face into the color of the normal value

then the right face (the one facing +x) shall be bright red

the one on the opposite site facing -x shall be dark red

[cloud rivet]

you're saying this is wrong?

[brisk chasm]

+y is bright green, -y less green

[brisk chasm]

and z is blue ish

[cloud rivet]

you're saying this is wrong ^^ ?

[brisk chasm]

yes

[cloud rivet]

ok

[brisk chasm]

look at your camera positions

[cloud rivet]

so I am at -5 x

right

[brisk chasm]

if that is your dark red

[cloud rivet]

so I would expect the normal to be pointing -1 ya

[brisk chasm]

then the opposite face musut be brighter

[cloud rivet]

is the other side

[brisk chasm]

if you take the right face, facing +x, its normal is 1, 0, 0

if you do the * 0.5 + 0.5 trick to normalize the values into 0..1 range for debug display, that normal value becomes 1.0, 0.5, 0.5, some piglet red, the opposite side, the face facing -x, its normal is -1, 0, 0, if you normalize it it becomes, 0, 0.5, 0.5, thats some sort of teal

[cloud rivet]

outFragColor = float4(normalize(basicVertex.normal) * 0.5 + 0.5, 1.f);

[brisk chasm]

hmm

either your camera position display is fucked

or something else is

if the camera.x is at +x then i should see the piglet red, not teal

if the camera.x is at -x then i should see teal not the piglet red

what is this

2 times normal values?

[cloud rivet]

2 times?

I think my translate is incorrect

transform*

[brisk chasm]

idk what the left 4 values are

[cloud rivet]

that's the interpolated position the vertex shader saw

[brisk chasm]

normals look like normals do

[cloud rivet]

before model to world

[brisk chasm]

ah

[cloud rivet]

yeah but you're right

the box is pointing the wrong way

I wonder if this is another houdini thing I am not doing right

[brisk chasm]

ah so all the tools using a fucked coordinate system somehow, or dont respect gltf spec 😄

[cloud rivet]

well

I have a gltf to ndc I used with deccer cubes

and that' what I am using here

and it's not working

so I think it's me just not knowing how to use houdini

[brisk chasm]

hehe thats also possible

but we will get that fixed eventually

go to bed now 🙂

[cloud rivet]

there's this checkbox

you started this lol

[brisk chasm]

"[ ] export gltf correctly"

[cloud rivet]

I said gn

[brisk chasm]

hehe

[cloud rivet]

and then you said my normals were wrong

so this is all your fault

[brisk chasm]

https://tenor.com/view/cute-baby-head-massage-tickle-gif-26829434

[cloud rivet]

what's this check box

I will press it

[brisk chasm]

oh

i mean

gltf assumes +y up

[brisk chasm]

the tooltip doesnt really help

[cloud rivet]

yeah

[brisk chasm]

blender has a similar stupid thing info, and there was some discussion with don to have it removed iirc

[cloud rivet]

idk will try it

[brisk chasm]

aye

[cloud rivet]

that didn't seem to do anything

[brisk chasm]

lets pick that up later

[gaunt drum]

hey! happy to see that you created another post

[wraith urchin]

That looks correct to me actually

[cloud rivet]

@brisk chasm I just realized that my normals look that way because I do all my lighting in tangent space, I don't transform my normals. I transform camera and light into object space and then transform them to tangent space, my normals are correct and it's not useful to look at

[brisk chasm]

i see

[cloud rivet]

I think I will add a debug view that does do that transformation because it's still valuable to see normals

I do have an inverse transpose matrix I send with each draw, just haven't been using it

I mean they could be wrong still, but they're never going to be correct in world space without the transform

yeah

that's really valuable, so I will add normal to world transform for debugging purposes

maybe they are broken, how would I know

[brisk chasm]

yeah my normals are also fucked for some raisin

[cloud rivet]

it's good to see some pixels from your project though

why isn't it red though

yeah it should clearly be red right because it's x and x is in the first channel

[brisk chasm]

we could check in renderdoc ackchually

[cloud rivet]

yes

ws trying to but visual studio decided to report nonexiting errors

[brisk chasm]

ah

[cloud rivet]

couldn't include filesystem

[brisk chasm]

set(CMAKE_CXX_STANDARD 23) in your root CMakeLists.txt

[cloud rivet]

yeah I was on the wrong release, I use msbuild, I want to switch to premake at some point

it's not red because -1, 0, 0, * 0.5 = -0.5, 0, 0 and if you add 0.5 to that you get 0.0, 0.5, 0.5

[brisk chasm]

yeah that should be the -x face

[cloud rivet]

yes I am at a negative x coordinate

[brisk chasm]

that is correct then

and its not 0 128 128 because your color picker is srgb probably

[cloud rivet]

I thought you said it should be light red

[brisk chasm]

+x should be piglet red

because 1, 0, 0 * 0.5 = 0.5, 0, 0 and then + 0.5 is 1.0, 0.5, 0.5 which is moo deng in action red 🙂

[cloud rivet]

oh great

ok so the last lighting thing is then to fix how I create normal map images in vk

but I am not going to worry about it

at this moment

I will create a github issue

[brisk chasm]

i envy your shadows

[cloud rivet]

they are not that good tbh

[brisk chasm]

doesnt matter 🙂

even shit shadows add a lot to ze atmosphere

[cloud rivet]

yeah they make such a huge difference

so right now I spam create all the images I find in a gltf at the very start, because I put my faith and trust in to libktx and it betrayed me, I will instead have to get the materials first, and create normal mapped images manually

but I just want to work on gameplay the rest of the month 🙏

[brisk chasm]

then do that

i will try to fix my normals in the meantime hehe and get asset loading done too, i never want to touch that shit again

[cloud rivet]

alright

so where I left gameplay off is I now have gltf nodes for all my meshes

and my gameplay code doesn't know anything about the scene graph yet

my transform hiearchy is just sent once right now too

hrm

also I don't want to break loading sponza

so I can't hard code assumptions for the game scene

[brisk chasm]

perhaps try to turn s ponza into a game object(s)

[cloud rivet]

I want to annotate my nodes in gltf

[brisk chasm]

were each flower pot or pillar is an object with mesh and material

[cloud rivet]

so I can treat some as special

[brisk chasm]

fastgltf supports Extras

[cloud rivet]

so I already have a python post render script

that runs after houdini generates the gltf

so I can do whatever I want there, but houdini can maybe do this

[brisk chasm]

with blender you can already add custom attributes

[cloud rivet]

oh it exports custom attributes

let's try that

[brisk chasm]

i have no idea how it actually works, but sean's project is just a room away

[cloud rivet]

yes

[cloud rivet]

these are just geometry attributes

was not the right play

https://www.sidefx.com/docs/houdini/hom/nodeuserdata.html

    FASTGLTF_EXPORT using ExtrasParseCallback = void(simdjson::dom::object* extras, std::size_t objectIndex, Category objectType, void* userPointer);

I will add a tags list of strings to extras and those become flecs tags, and the first tag will be "mob" which I will then make it show up in ui as such and then I can add interactivity with that gltf node

then I will have mobs!

I will do some adulting, as the millennials say, and then come back to work on identifying mobs in a scene graph

that's houdini, not my engine

[hushed creek]

that's why houdini is better than blender, their cubes are sentient

[cloud rivet]

lol

[cloud rivet]

all these tools use python for scripting, houdini also has a custom scripting language, and both maya and houdini also offer a C++ SDK

created a separate repo for all these scripts https://github.com/btipling/rosy_scripts

where all these is just one so far 😅

[cloud rivet]

I didn't even need extras

just used the gltf node hiearchy

I think eventually I'll need extras though

man

my build used to take 79 seconds

still under 6 seconds and more capable than what I had previously

[cloud rivet]

-                       size_t res = fread(n.name.data(), sizeof(uint32_t), num_node_name_chars, stream);
+                       size_t res = fread(n.name.data(), sizeof(char), num_node_name_chars, stream);

this UB caused low level SDL code to fail to init a DXI dll

like

the only reason I was able to fix it is because I had just written this code

this code path where the bug was had absolutely nothing to do with SDL init

[cloud rivet]

this was so much work

that's me drawing on a screenshot

that green stuff is not in game

but just getting the game to know there was a box was a lot of work

it still can't be moved or anything yet

but at least it exists as a game object now

and it does it without breaking sponza

[cloud rivet]

Added a Node.cpp to represent things in the world

[cloud rivet]

so for things that can move around

I am going to sort their buffer data to be at the end after all the static geometry, store that offset

and every frame use the same buffer pipeline barrier I use for scene data and write to that buffer at that offset

I have a "graphics object" buffer that stores per node data that already is sent via a bda with the draws

it's been static until now

I am creating a separation between static nodes that don't need their transform hierarchy updated and mobs, and I assume that there will be way more static geometry

I can use that buffer also to run animations, and signal status effects and stuff, just any per node data the shader needs, just a bda into this buffer, and because it's sorted at the end I can actually I guess a different struct so I don't store a bunch of unused stuff for static geometry that doesn't need all that

I think this will work for now

[cloud rivet]

I like this camera distance hrmmm

looks like it has zoom

also the point lights seem to do shadows within a radius

I like that too

a small radius

[cloud rivet]

this is the ascents camera distance for comparison https://www.youtube.com/watch?v=vZSwPt1mIN0

the angle of view changes based on the surrounding

it's also much closer

[cloud rivet]

is it weird that imgui uses char* instead of std::string even though it is a C++ project

like why can't I give it just vectors

[true moon]

No

It would be nice if it used string_view

[cloud rivet]

hrm

[true moon]

But char* is so convenient

Even though it's shit

[cloud rivet]

no?

it's not?

[true moon]

String literals

[astral hinge]

making the interface use string_view would be just as convenient tbh

and std::string can be implicitly constructed from const char*

[cloud rivet]

I have a std::vector<my_struct> and my_struct has one field with a string and I wish I could just give it a lambda or something to convert my vector to the list for ImGui::Combo

[true moon]

Then it would allocate everywhere

If it used std string

[cloud rivet]

hrm

[true moon]

You could use sv literals though

[astral hinge]

yeah std::string would be bad but not from a convenience perspective

[cloud rivet]

sure

I don't want any allocation that I didn't myself allocate

unless I could provide an allocator

[true moon]

If you dig into the API there might be an internal function that lets you specify a stride or something

Or has a callback

[cloud rivet]

it's fine, I'm just going to convert it I was just thinking it was a not very C++ like C++ library

[astral hinge]

I think you should just deal with it and make a vector<const char*> like imgui wants

[true moon]

Yeah

[cloud rivet]

that's what I am going to do

just was complaining about having to

[true moon]

Generally quick and ugly is the way to be productive with imgui

[cloud rivet]

yeah

I don't want to use imgui for my game's ui

I think I will do something myself

[astral hinge]

imgui is my "temp" solution for game ui

[cloud rivet]

yes same

ah ImGui::BeginCombo is better than ImGui::Combo when using vectors

I like my files having this green check mark on all of them

no warnings, green checkmark, it's good

imgui_demo.cpp:

idk if that's a resharper thing or visual studio

I like visual studio

[cloud rivet]

everything is always so much work than I expect

[true moon]

Amen to that

I maintain that this is one of the most difficult things you can do with programming

[cloud rivet]

it's just like ok almost done, but wait...oh right it's a scene graph tree with all the children and I have to recurse, oh but I didn't recurse in my graph I made earlier...oh and I need to now handle this other thing....

yeah

it's hard

I should know by now

[astral hinge]

I'm a lazy ass so everything is less work than I expect

[true moon]

That's why I have such a dumbass programming style

[astral hinge]

I implemented a feature that I was putting off and it only took like 20 loc

[true moon]

I just do everything in the most straightforward and easy way even if it's ugly, as long as it remains fairly decoupled

[cloud rivet]

that's not lazy that's skill lol

[cloud rivet]

same

that's the goal anyway

just pure imperative code with as few abstractions as possible

can avoid a graph when you're dealing with graph data though

[true moon]

You can a little bit

My hierarchy is pretty lazily evaluated

But yeah you need at least one walker function to traverse it

[cloud rivet]

I made a recursive structure that just call the same operation on all its children

I don't like recursion but it works here I guess

lazy

I'll fix it if I hit a stack overflow I guess or when I think of something better

I'm going to have a small scene graph though I think

I'm not making an open world game

[astral hinge]

I just build things up in my mind

[cloud rivet]

I could have just hard coded updating this one box I know I have with no children, but I had to go update the full thing as if it was a giant scene graph

[cloud rivet]

this was so hard lol

[true moon]

Hm why do you need to update a scene graph for that

[cloud rivet]

because every node in gltf is a graph

[true moon]

What has that got to do with this scene

[cloud rivet]

the buffer for all the data is built by the graph

[vagrant musk]

Einstein’s theory of relativity (things can move relative to other things without necessarily affecting other things or smth)

[cloud rivet]

if I update something in the graph, I have to update that part of the graph

[wraith urchin]

I just flatten the whole thing on import

[true moon]

Yeah but I don't understand why the leaf nodes need updating, can't you just update the root

[cloud rivet]

the full graph

[vagrant musk]

tldr in a more sane way
you can update the data for that specific node

[cloud rivet]

I don't want to update the whole scene

[true moon]

I don't understand why you'd have to

If you're just moving one object and its children

[cloud rivet]

yes

that's the graph

that one object and its children

that I update

[true moon]

But its children are updated implicitly

[cloud rivet]

no

[true moon]

so you just need to update the root

it should be

[cloud rivet]

do you pass all of the matrices that transform a child node into the shader? do you do all that math every draw per primitive?

the buffer contains the model to world transform for each primitive

[true moon]

The nodes don't really exist as far as the renderer is concerned in my engine

[wraith urchin]

You probably shouldn't have a node per primitive

[cloud rivet]

I don't have a node per primitive

I have the same structure that gltf has

the primitives have to index into the buffer for their matrix

their transform

[true moon]

But yeah I do that math every frame per entity, at the start of each frame in the rendering system it walks the tree and calculates the world-space transformation of each drawable entity

[cloud rivet]

I do not do that

I stick it into the buffer

if it is updated I update that part of the buffer

that's the work I did today

[wraith urchin]

Definetly don't update the buffer, thats costly

[cloud rivet]

it's not at all

[true moon]

He's talking about a buffer of transforms

[wraith urchin]

ah

[cloud rivet]

it's tiny

[true moon]

you need to update it, but you can do it all at once rather than constantly updating every subgraph every time something moves

That's expensive

[cloud rivet]

yes I do it all at once, but the child transforms all need to be transformed by their parent transforms

[wraith urchin]

Yeah I just do all the updates all the time

Trying to cache stuff often leads to bugs

[cloud rivet]

I'm not caching

It's just the data is in the buffer

[true moon]

That's what it sounds like when you talk about updating the node's child graphs

[cloud rivet]

you guys have like uniforms

I don't have uniforms

there are no uniforms

I have 128kb

I have to update a buffer

[true moon]

I'm not talking about the rendering really at all I'm talking about the scene representation

Generally speaking child nodes in a hierarchy don't move relative to their parent and only a small number of them will need to be queried spatially so it's not really worth trying to keep a global transformation for each one updated all the time

It's better to just store their local transform relative to their parent and calculate the world space transform on demand for rendering or queries

It's a lot less work and error prone

[cloud rivet]

that's what I am doing

[true moon]

Alright I am confused by what you said you implemented today then

[cloud rivet]

ok

so for each draw

whatever is being drawn needs to be in the right place, it finds its right place by pulling its transform out of the buffer

that transform needs to put it in the right place, that transform needs to be the full transform based on where it is in the hiearchy because that's how gltf works, the parent transform * the object's transform

if the parent moves and the parent has a hat

the parent's translation needs to be applied to the hat as well

that final hat transform is in the buffer

when the parent is transformed I have traverse its graph

so that all the things it has will have the right transform on draw

idk how else to do this

this is how vkguide does it too

[wraith urchin]

Unless the hat moves independently I would just flatten the transform on load

[cloud rivet]

I see

[true moon]

Hm I don't use gltf so to me every model only has one mesh, my separate meshes are separate entities altogether, and that's where the hierarchy lives

[cloud rivet]

yeah that could be a thing

hrm

it would still have to be a graph

[true moon]

Yeah it is

Mine looks something like

[wraith urchin]

Yeah each gltf I load becomes one Mesh component in engine (with a submesh array)

[cloud rivet]

gltf is more complex than just meshes, it's nodes with a bunch of meshes which each their primitives and then the nodes can have child nodes

[wraith urchin]

Yeah but my engine doesn't care about the gltf heirarchy

[cloud rivet]

hrm

[wraith urchin]

or rather it get flattened at load time

[cloud rivet]

https://vkguide.dev/docs/new_chapter_5/gltf_nodes/

I do this

well

not that, my own way

but builds the same graph

[wraith urchin]

gltf is not a sophisticated enough format to handle all the data my engine needs so I don't try to use it for entire scenes

[true moon]

struct ChildOf
{
    ent_t parent;
};

void main()
{
    ent_t ent1 = reg.create_ent();
    ent_t ent2 = reg.create_ent();

    reg.emplace<LocalPosition>(ent1) = x;
    reg.emplace<LocalPosition>(ent2) = y;
    reg.emplace<ChildOf>(ent2).parent = ent1;

    // Game
    while (true)
    {
        // move parent
        reg.get<LocalPosition>(ent1) += glm::vec3(0.1, 0, 0);
        // render scene (child automatically follows parent)
        render();
    }
}

glm::mat4 get_global_transform(ent_t child)
{
    pos_t localPos = reg.get<LocalPosition>(child);
    if (reg.has<ChildOf>(child))
    {
        ent_t parent = reg.get<ChildOf>(child).parent;
        return localPos + reg.get<LocalPosition>(parent);
    }
    else
    {
        return localPos;
    }
}

struct DrawCmd
{
    glm::mat4 transform;
    ent_t entity;
    Step drawStep; // which pipeline does this entity use to draw
};
std::vector<DrawCmd> draws;

void render()
{
    // step 1: collect select scene state into draw list
    for (ent_t e : reg.view<Drawable>())
    {
        DrawCmd cmd
        {
            .transform = get_global_transform(e),
            .entity = e,
            .drawStep = which_bucket(e)
        };
        draws.push_back(cmd);
    }

    // Step 2: sort draws by draw step

    // Step 3: draw stuff
    int i = 0; 
    for (; i < draws.size(); i++)
    {
        if (draws[i].drawStep != Step::Opaque) { break; }
        draw_opaque(draws[i].entity, draws[i].transform);
    }
    for (; i < draws.size(); i++)
    {
        if (draws[i].drawStep != Step::Transparent) { break; }
        // ...
    }
}

Approximate sketch with lots of omissions but you get the idea

If you bake out all the global transforms at the beginning of the rendering, your children don't need to be updated at all unless their relationship to their parent changes

Actually they don't need to be updated period

Since you just set their local transform (in this case I only included translation, rotation left as an exercise to the reader) and it will get picked up automatically

otherwise you only need to move the parent around and everything else just follows automatically

edited to add a simple system in the main game loop

In my current naive implementation I basically just pass uniforms in the draw_x() functions (in fact those functions just fetch arbitrary extra data from the registry) but you can bake the scene data into as digested of a format as you like, or even just write it into the buffers immediately in step 1 (that's how I've done it in Vulkan in the past)

[cloud rivet]

I see

[true moon]

In this snippet the global transform calc is super brute force in reality you'd want to walk the graph in order so you only touch each node as many times as you need

Although this is what I do in my game lol the super lazy way

But there isn't just one way to do this, whatever works

[cloud rivet]

I think even if I flatten it the code wouldn't be different it would just do less work

idk it works

your approach makes sense too

[true moon]

Mine is like the opposite of flattening, my meshes are just a single part so all my compound objects need to be composed out of multiple entities

Generally just hard-coded offsets although I will eventually probably have some sort of config file format to represent assemblies

[cloud rivet]

I'm adding flecs today

going to add it as a git submodule and make all my dependencies besides the vulkan sdk git submodules now, SDL, fastgltf, KTX, flecs, tracy

after I have added flecs I will add being able to click on objects in the scene and drawing a ring where clicked, and then I will add a flecs system to move the box where clicked, first instantly, and then I will have to figure out how to move it like it should

[wraith urchin]

Woo submodules! They work great for external libraries. Just don't be tempted to use them for different components of your own app (this is a nightmare)

[hushed creek]

it's one way to make your repo usable by just you that's for sure XD

[wraith urchin]

At work one of the ancients decided to do that for a few things and its a nightmare if you have to make a change inside one of them. You have to make 2 PRs and pray that they are merged in the right order

[hushed creek]

at work the idea floated in my mind a bit too but after experimenting with both submodules and the other subrepo thing, I just didn't do any of it

[brisk chasm]

subtrees are great, but those things only work when everyone in the team involved with these repos knows how to use them, same applies to rebase and other "more advanced" gitisms

[hushed creek]

I wrote a CLI tool that pulled both repos, did its job with both of them, done

[wraith urchin]

I use submodules for Bullet, SDL, SoLoud, etc. They're great for that

[hushed creek]

the thing is if you're in a team, unless the solution is braindead simple it's not good

and even if you take the time to train everyone and teach everything

if it's more than one braindead simple step someone somewhere WILL fuck up at a critical moment

it's guaranteed

[brisk chasm]

ye

[wraith urchin]

Well we will fuck them up solo too, its just the consequences are not as bad lmao

[brisk chasm]

also true hehe

[cloud rivet]

yeah I'm just going to use submodules for third party dependencies

I have a separate repo for my python scripts for houdini, blender etc

but I won't like import it

I really need versioning for my houdini files, I don't know what to do about it, I think I will just locally use svn

and use onedrive to back it up idk

I have to relearn svn I guess

I'm not using git for large files

[brisk chasm]

why not

for your one man army its fine

then you dont have to relearn things

[cloud rivet]

because svn is better for that

[brisk chasm]

i do want to redo my homelab as well : >

[cloud rivet]

I just have a pc

[brisk chasm]

same, with proxmox running on it, ah i mean a separate pc

[cloud rivet]

and a laptop

I don't want to run any servers

hrm

[brisk chasm]

you have that one server you dont use 😛

hint hint

[cloud rivet]

I have a digital ocean droplet

that's it

[brisk chasm]

ah i thought its some sort of vps or root server

[cloud rivet]

idk it's just going on my pc and if my pc blows up I guess my game is gone :/

maybe onedrive will save me

[brisk chasm]

zip it once a day/week and copy it to your dropbox

[cloud rivet]

ya

[hushed creek]

I dunno how big you want your files to be

but I had no issues using git with a unity game where I downloaded gigabytes worth of assets

synced to bitbucket

bitbucket's really good at compressing your repo

I think it brought a 10+ GB unity project down to less than 1GB

be clever if you own the binary formats to split the parts that change very often from the parts that don't, that way you have fewer undiffable changes happening all the time

[wraith urchin]

Git works absolutely fine for all my assets yeah. I don't do any LFS stuff either

[hushed creek]

I can see it being an issue in AAA where everything is measured in terabytes but I think you'll be fine 😛 and besides you can always add perforce later if you want to

[cloud rivet]

ok ok I use git

[wraith urchin]

Don't solve a problem you don't have yet basically

[brisk chasm]

its only an issue with github repos really, due to their size limits on the free plan

[wraith urchin]

What is the size limit btw?

[cloud rivet]

GitHub blocks files larger than 100 MiB.

https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github

I have a paid plan

[wraith urchin]

Oh interesting

[cloud rivet]

I pay $48 a year I guess, I forgot about it

[brisk chasm]

i upgraded to a 10gb plan at some point too, i think thats the smallest one

5 bucks per month

ah yeah thats what bjorn has, annual is cheaper than monfly

[cloud rivet]

oh you can pay for LFS

[brisk chasm]

yeah

[cloud rivet]

hrm

ok well I'll just use git and if they block me I will turn on LFS and if that gets blocked I guess I'll pay for more storage

[brisk chasm]

you could keep the big assets on your drop box perhaps, and keep code and the rest in git

[cloud rivet]

I don't use dropbox

I use onedrive

[brisk chasm]

or whatever file storage 🙂

[cloud rivet]

but it's already on a onedrive folder

but it's not being versioned

[brisk chasm]

so its backed up and versioned too already

[cloud rivet]

I don't think it has versioning

I will pull it out of onedrive

idk

[brisk chasm]

i think that was a thing at one time at least

[cloud rivet]

a github repo on onedrive doesn't sound fun

[brisk chasm]

hehe no

[cloud rivet]

my asset folder is a mess

[brisk chasm]

yeah

[wraith urchin]

How many files do you have bigger than 100mb?

I'm not on GH but I don't think I even have any

It's not like I'm putting video files in there

[hushed creek]

try bitbucket I don't think they have 100mb file limits

but it's not gh so it won't look as shiny 😛

[brisk chasm]

bitbucket is atlassian and atlassian turned into a shit company

i wouldnt trust them with any of my data anymore

even if bitbucket "is good"

[wraith urchin]

I mean Microsoft may not be any better on that front

[brisk chasm]

every other week you read about vulnerabilities in service xyz from atlassian

[wraith urchin]

Well they're at least probably secure

But MS is definitely scraping your data to train AI

[hushed creek]

atlassian is likely doing the same because every corpo wants AI

atlassian has AI in jira and confluence where it tries to tell you what acronyms mean

among other things

[brisk chasm]

yes but bjorn is on github already

[wraith urchin]

Jira is already shit, AI is not gonna make it better lol

[hushed creek]

the only 100% private way is to self host :<

[wraith urchin]

I have a linode server, maybe I should put my repo on that

I could also run my own machine but I would like to have something off site

[steady venture]

LFS may need additional fees

You have a limited bandwidth even if you are a pro user

[cloud rivet]

I don't have any big files right now

I'm just going to use straight up git until that stops working I guess

[steady venture]

Solid plan

It will warn you via e-mail when you hit certain a certain quota

[cloud rivet]

I don't read email

[hushed creek]

that said you should totally also keep your repo on onedrive/dropbox/anything else as an extra backup

I have this .bat file that I run at the end of every work day

git archive --format=zip --output backup.zip master
xcopy "backup.zip" "onedrivepath\backup.zip" /y
xcopy "backup.zip" "dropboxpath\backup.zip" /y
del backup.zip

set /p DUMMY=Hit ENTER to continue...

it creates a .zip of the repo, then copies the zip over

[brisk chasm]

many many many years ago i lost a lot of nice and cool code ive written over the years, never backed this stuff up. but i was only sad for a day. the loss provided a good opportunity to start from scratch

[cloud rivet]

it's good to kick it out

the past belongs in the past

[cloud rivet]

fully building with submodules now (except vulkan sdk)

still have to manually run all the cmake commands

ok now I can add flecs

I'm going to statically link flecs I think

[brisk chasm]

why do you have to run everything manually

you just add_subdirectory your gitmodules

[cloud rivet]

idk

[brisk chasm]

and link them with your main project as usual

then cmake/ninja will figure out what to build

[cloud rivet]

I don't use cmake

[brisk chasm]

ah

[wraith urchin]

You can add arbitrary commands to your VS build as pre or post build steps

[cloud rivet]

this is more of a one time ever kind of command

I should create a powershell script

idk, this is progress though

[wraith urchin]

Well you need to rerun it if you update your dependencies

[true moon]

Honestly before you get too deep into flecs you should make sure it's what you want

vblanco tends to complain about it being inflexible

[wraith urchin]

EnTT is also very good

and probably lighter weight

[cloud rivet]

I have used flecs

I used in blockens

I like it

[true moon]

Alright

[cloud rivet]

vblanco is super active on the flecs discord

seems pretty happy with it on there

he's also building a mega open world open ended randomly generated rpg game

[true moon]

Yeah I know I follow his thread and he complains about flecs sometimes

[cloud rivet]

idk I like it

[true moon]

The thing that EnTT or your regular hand rolled ECS lets you do is just dynamically add or remove components on demand

[cloud rivet]

you can do that in flecs

that's like a key thing

[true moon]

Apparently not in the way you can in EnTT idk not from how it sounded

[cloud rivet]

well any change you make isn't written right away unless you force it

I don't know anything about entt

I'm sure there are trade offs that entt made differently and might work better for vblanco's game, but he spams the flecs discord like daily

he's probably more active there than his thread here

actually no

he's way more active here

it just seems like it because there's less activity on that discord

I used to boost the flecs discord, but I stopped after I stopped working on blockens and wasn't using it anymore. I quite like flecs

I also sponsor the project on github

that second one is Sanders

so I am fan

4 of those are zig projects that I sponsored in the past for a year and haven't expired yet, and then flecs

[astral hinge]

flecs seems cool but I don't feel a need to switch ecs libs right now

[cloud rivet]

yeah I just like it for me idk what's best for other people

[astral hinge]

idk what it excels at compared to entt

entt works well for me

[astral hinge]

is entt really better at nothing vs flecs

I mean both libs offer non-ecs features afaik. I'm using a bunch of entt's non-ecs stuff too

[cloud rivet]

well that's a flecs document so not unbiased

[astral hinge]

yeah

oh wait I misread that thing

it does admit some things are faster in entt

I don't think I am or will be bottlenecked by my ecs either way, so I guess usability is the focus

[cloud rivet]

heh

[astral hinge]

link?

[cloud rivet]

that's from entt's documentation

https://github.com/skypjack/entt/wiki/Frequently-Asked-Questions#frequently-asked-questions

[astral hinge]

lol I enabled /JMC and did notice that it appears in the profiler now

[cloud rivet]

what's /JMC?

[astral hinge]

it inserts a little bit of code into every single function call to check if it's a "user" function

it's so the debugger can skip non-user code when stepping through code

you'd think a debugger could just look at the source association and skip stl files instead of having to change the binary

[cloud rivet]

one thing I love about flecs is how will just immediately quit the application if you use it wrong

like it validates everything you do

[astral hinge]

any good library should do that imo

entt does the same

[cloud rivet]

nice

yay flecs working

[cloud rivet]

wow this ai is validating the issues I linked and figured out I skipped on adding a powershell script for my git submodules github issue

hrmmmmm

oh no

this git submodule change makes it so I can't git bisect beyond this point anymore

or built the project at all anymore for old commits

hrm

well

I hope nothing is broken I have to go beyond this commit

I should create a git tag

oh my build is a little bit faster now

ok so next up is determining location in 3D clicks with mouse

I think that's a reverse projection or something, I have read this before, will reread

[wraith urchin]

You need a ray cast to do 3d picking

or rather that is one way to do it

you can find the line that the mouse is one with simple linear algebra though

[cloud rivet]

right, but first I have to convert the screen coordinates where the mouse clicked into 3D coordinates, and then ray cast

hrm

I think I have done this before

[wraith urchin]

inv_viewprojection * vec3(screen_coord.xy, 0.0)

[cloud rivet]

do I need an AABB for this

[wraith urchin]

Depends how accurate you wanna get

I just use the built in raycast from bullet

[cloud rivet]

well there are just two meshes right now

I'll do something fast and dumb to start

[wraith urchin]

If you are gonna integrate a physics engine, you can probably do the same

[cloud rivet]

and just "iterate" over my single static object

yeah I will integrate jolt and recast pretty soon

[wraith urchin]

yeah you need a AABB ray intersect code, thats easy to find examples for

[cloud rivet]

I think I read that bullet was not great, do you like it?

[wraith urchin]

It has problems

I can't exactly call it "bad" though

I made the choice long ago, I'm too deep now

Jolt seems to be a popular choice though

[cloud rivet]

I did my own collisions in blockens with ray casting against voxel data, other than that I have never done anything with physics

I was so proud of that though, when I could climb around my voxels 🤗

man I really enjoyed working on blockens

I hope this project is as fun

so

I think I will do a move in the direction of the mouse

instead of a go to where I clicked actually

because that's more of how gamepads work ya

and that avoids having to build pathing ai for my character at the beginning

but I also do want to do a click to go somewhere

[wraith urchin]

Is you camera gonna be fixed?

or will it be able to rotate?

[cloud rivet]

it has a fixed angle, but you can rotate is what I am thinking

I'm not sure yet

I would like to rotate it though, but it's less assets to create and simpler to not do that

I think I will though

I haven't made my mind up yet, all I have is a box that I can move by typing a number into an imgui form

[true moon]

You may end up needing both physics engine intersection tests and your own intersection test

Physics tests won't include every sort of object you might need to select if you're building an editor here

[cloud rivet]

cool

I got it

I'm gonna do some cheating for this first attempt

no physics engine required

the click takes place on the plane of projection, I convert the screen space coordinates to NDC and then to view by accounting for the aspect ratio used in the projection. Now I have a point on the projection plane, I get my ray from the camera position to that point on the projection plane in view space

here's the cheap trick: I just care about where I click on my floor

so I conver the ray into the floor's object space by converting the ray to the inverse of the floor's object to view, now the ray is in the floor space

the floor is at y 0

[astral hinge]

quick maths

[cloud rivet]

I find where the ray intersects at y 0

see if that's within the rectangle dimensions of the floor

and whatever point that is where I my debug circle's origin is

then I use the floor's matrix to view transform on the circle

bam

I think this works

for the circle

I haven't thought about moving the box to the circle yet

I guess I convert the box to floor space and translate it with a lerp

ya

I think this all works

and then next step is lerp the orientation of the box towards the point

and then my first gameplay task is achieved?

I think this all checks out in my brain

will see when I write code

it's not even a click it's just where ever my mouse is down while I hold it

I like this

I gotta exercise and think about all this

[astral hinge]

You could write it on paper

A bit more organized than scrolling text

[cloud rivet]

ya true

once I have stuff in the world I will have to consider collisions but I don't have to yet

maybe I don't even need a physics engine for this game

hrm I think so though for someof the puzzles I want to do

where there's like projectiles that push the player back

in blockens I did my own falling logic

it was bad, but it was mine and I liked it

I did my own jumping too

[astral hinge]

Physics engines are really fun to play with

It's like your game suddenly transforms into Garry's Mod

[cloud rivet]

I was thinking of a puzzle where like boxes are being ejected from some facitilty automation that prevent the player escaping a room and I want those things to have physics and bounce around so I will use a physics engine eventually

unless I can do that myself too

and the game is sort of like mega man, where in mega man you get the defeated boss's power

[astral hinge]

oh that's a mechanic in Valheim too

you can choose which boss buff you get from any of the ones you've unlocked

[cloud rivet]

but in this game you learn about some facitity function to solve a puzzle and then you can use that automation to solve later problems, so you can eventually use that box projectile mechanism to solve other puzzles and push other things around to escape next levels

but as you gain all these automations you also have to start to manage the facilities resources

because it won't be free

so there's like a physical escape puzzle situation, and also a resource management puzzle

anyway, that's the idea I have right now

and then the decisions you make have consequences, because you have to use some resources that were being used for something else that might have bad or good effects for other npcs and areas

it's all a pipe dream right now

right now I have box I can move with an imgui form

[cloud rivet]

I have decided the robot's name

[cloud rivet]

heavily commenting the code I wrote yesterday before I forget, holy moly

it is complex

[cloud rivet]

I got an initial hello world flecs system running using the flecs C api

was hoping to get further than that, but it's progress

[wraith urchin]

Line count?

Me when I wrote culling.cpp

Only I forgot to actually write the comments

[cloud rivet]

Level.cpp

almost every line in the function that loads the scene graph is now commented

Level.cpp is going to be thousands of lines

it's where I am going to put all the ecs

I should rename it to Game.cpp

[wraith urchin]

You are loading the graph from gltf? Or do you have an actual level format

You might wanna split it up a bit

Like separate format loading from level update stuff

[cloud rivet]

I have my own format

[wraith urchin]

I load stuff in json_serializer.cpp but then I have the rest of the stuff elsewhere

My world format is in json

[cloud rivet]

mine is I just write the actual cpp types I made to file and load them back

there's a header

and each section has a header

with counts

it's working for me atm

[wraith urchin]

Right but I assume you also have a really long loading function?

Maybe it should go in its own file

[cloud rivet]

no it's not that big actually https://github.com/btipling/Rosy/blob/master/src/Packager/Asset.cpp

it is in its own file

[wraith urchin]

Cool

[cloud rivet]

it actually is also split up

the renderer gets all the geometry data

Level.cpp is getting the actual scene graph

none of the images or vertices or any of that

[wraith urchin]

Yeah so what I do is have a world class and the loading function basically call world->create_entity() whenever it loads something

[cloud rivet]

oh I see, that's similar to me

I don't have a world, I just load levels so I think that's kind of the same

[wraith urchin]

Have you considered how you are gonna handle changing struct layouts as you develop the engine?

[cloud rivet]

there's a version number in the format header

but basically I don't need to worry about backwards compatibility right now

so I can just completely change it

[wraith urchin]

Well as soon as you start creating data you will lol

[cloud rivet]

it's strongly typed so if I change something it will break the app.

or be ub

😅

[wraith urchin]

Yeah but then you have to fix the data

[cloud rivet]

I just overwrite it

hrm

each section should have a header magic number to check

that would prevent ub

[wraith urchin]

Let's say you spend an hour working on a level, then you realize you need to add a struct field

Do you just throw away your whole level?

[cloud rivet]

right now the level is being designed in houdini, I just export from houdini, it runs my python script and I get a gltf

then I run my packager that will turn the gltf into .rsy

and overwrite what was there

[wraith urchin]

So all the data is sourced from outside your fomat then

[cloud rivet]

yes

[wraith urchin]

OK that's fine for now probably

[cloud rivet]

I mean I have a box and a floor right now

it's not a lot

[wraith urchin]

As long as you commit to houdini

[cloud rivet]

I don't have an epic boss waiting on the beach

[wraith urchin]

Oh man if I lost leviathan.json that would be a nightmare

[cloud rivet]

please don't

[wraith urchin]

It's all in git don't worry