tensorflow-models/pose-detection + Three JS

[eternal zodiac]

I've implemented this before. It's not as easy as you might think, because you don't have information about rotation of the bones on the long axis..

So you usually have to infer it from the structure of the skeleton...

for instance.. for a thigh bone, you can use the vector from hip, to knee, to form one axis of your matrix, then a vector from left hip, to right hip, to get another axis.. then cross those 2 vectors to get the 3rd axis, which you can then convert to a quaternion and plug into your bone (in the bones parent space).

You then apply similar techniques for all the bones. It's non trivial.

An alternate approach is to use some sort of IK (inverse kinematics) library, and treat the pose points as IK targets for each bone.
BUt in that case you still have to set up the IK rig to match whatever skeleton you are using, so.. it's not neccesarily easier.. just a different approach.

[eternal zodiac]

the forearm and the upper arm, form a coordinate space when they aren't fully extended.. when they are fully extended.. all bets are off.. tho you might could use the hand as the coordinate system for that case...

if im understanding

hard to say without knowing your whole setup... but sometimes yes..?

If you hit a wall with precedural approaches.. you might consider using an IK rig or a physics based rig with points constraints to the target points..

Different modellers and file formats do different things. FBX i think comes in pole swapped Z/Y .. and depending on the exporter, sometimes handedness changes

depends what the source of the GLTF was too tho.. if you import an FBX in blender.. the bones all point sideways, but it still "works" as an animated model... you export as GLTF .. the bones still point sideways.. but it works.. but could throw off any logic you have about bone directions

yeah. interesting...