How create and use VAT (Vertex Animation Texture)? | TiXL | Page 1

tawdry fjord Aug 22, 2024, 2:54 PM

#

Here I'll share my experiments about VAT.

I'm using blender with these addons:

https://github.com/JoshRBogart/unreal_tools : generates the uvmap and textures for vertex displacement.

https://github.com/matyalatte/Blender-DDS-Addon : allows to export textures as DDS, DXGI format: R16G16B16A16_FLOAT (because DDS supports negative color values)

tawdry fjord Aug 23, 2024, 8:33 PM

#

So the big issue is:
in order to be able to deform the mesh with a VAT, the mesh's uv map needs to look like this

#

but this kind of uv coordinates makes the mesh impossible to render properly (that's why I'm showing only a black wireframe in the video above)

tawdry fjord Aug 23, 2024, 10:36 PM

#

Yay! with a GLTF model i'm not stuck with black wireframe rendering

tawdry fjord Aug 27, 2024, 11:35 PM

#

@minor topaz please, I need help to get acces to the second uv coordinates set in a GTLF
if I can get it to work inside [UVsViewer] I think i'll be able to understand how to use it with my [DisplaceMeshVAT]

📎 VAT_REbaseRexpVAT.glb

tawdry fjord Sep 8, 2024, 9:57 PM

#

yay!! I'm making progress! TexCoord for rendering and TexCoord2 for VAT

tawdry fjord Sep 8, 2024, 10:12 PM

#

of course the normals are not following the deformation... 😑

#

ok 🙂 the blender addon is also generating a VAT for normals 🙂

tawdry fjord Sep 9, 2024, 2:25 PM

#

yay! I managed to deal with the normals

#

minor topaz Sep 9, 2024, 4:41 PM

#

Wow. That looks really cool!

tawdry fjord Sep 9, 2024, 4:45 PM

#

i need to make an example with a real wow effect 🙂

strong totem Sep 9, 2024, 4:55 PM

#

Yeah, I need a person running... 🙂

tawdry fjord Sep 10, 2024, 5:11 PM

#

ok it was supposed to be flat shaded... but that's fine.. of course I knew the face was going to be an issue... gonna have to fix that, model by me, animation from Mixamo

#

performance is pretty good

strong totem Sep 10, 2024, 5:44 PM

#

So tomorrow individual agent behaviour and on thursday the LOTR battle... 😉

#

Really impressive.

tawdry fjord Sep 11, 2024, 10:08 AM

#

Unreal Tools (blender addon for exporting Vertex Animation Texture) has limitation to 8192 vertices

strong totem Sep 11, 2024, 12:39 PM

#

Oops.
Well, we'll need GLTF animation support after all... 🙂

tawdry fjord Oct 16, 2024, 9:34 PM

#

parameters to export VAT from Blender with https://github.com/matyalatte/Blender-DDS-Addon

tawdry fjord Oct 16, 2024, 10:57 PM

#

I made a PR https://github.com/tooll3/t3/pull/479

GitHub

Feature/vat Vertex Animation Texture by newemka · Pull Request #47...

Add support of two UV texture coordinates to meshes (works for GLTF only)
Update many mesh operator to make the stride compatible with the new struct.
New operators: [MeshDisplaceVATBeta] and its e...

tawdry fjord Oct 17, 2024, 9:50 AM

#

it breaks [RepeatMeshAtPoints] and I don't understand how to fix it

tawdry fjord Oct 17, 2024, 6:42 PM

#

Fixed ^_^

tawdry fjord Oct 17, 2024, 9:11 PM

#

VAT + SetShadow + trying to add spot cone projection = ♥

minor topaz Oct 17, 2024, 9:12 PM

#

tawdry fjord VAT + SetShadow + trying to add spot cone projection = ♥

Wow.

#

Is the crashing a cube a VAT?

tawdry fjord Oct 17, 2024, 9:12 PM

#

😎

#

yes

minor topaz Oct 17, 2024, 9:12 PM

#

haha

#

Magic

tawdry fjord Oct 17, 2024, 9:13 PM

#

i'm so happy my idea to add that ellipse to the shadow is working

#

// Check if shadowCoord is within [0,1]
    if (shadowCoord.x >= 0 && shadowCoord.x <= 1 && shadowCoord.y >= 0 && shadowCoord.y <= 1)
    {
        shadowFactor = ComputeShadowFactor(shadowCoord, shadowMapTexelSize, ShadowBias);
    }
    float test = 0.1;
    float d = length(shadowCoord.xy/.8);
    float cone = 1-step(1.0,d);
    
    // Final fragment color.
    float4 litColor = float4(directLighting + ambientLighting, 1.0) * BaseColor * Color;
    litColor.rgb = lerp(litColor.rgb, ShadowColor.rgb, (1 - shadowFactor*cone) * (ShadowColor.a));
    litColor += float4(EmissiveColorMap.Sample(texSampler, pin.texCoord).rgb * EmissiveColor.rgb, 0);
    litColor.rgb = lerp(litColor.rgb, FogColor.rgb, pin.fog * FogColor.a);
    litColor.a *= albedo.a;
    return litColor;

#

here is a quick standalone with VAT and SetShadow merged https://www.swisstransfer.com/d/92c60b90-ae20-4b24-9b42-46228b370aa9

tawdry fjord Oct 17, 2024, 10:29 PM

#

gobo test

minor topaz Oct 17, 2024, 10:32 PM

#

tawdry fjord gobo test

Wow... does this work?

#

super cool.

tawdry fjord Oct 17, 2024, 10:40 PM

#

yes we can project an image, but right now it's a pattern I made inside the shader. It would be neat is to have an image input in thePointLight op

#

also maybe ShadowColor parameter shouldn't be in DrawMeshWithShadow

#

my first goal was to hide this

#

I need to find an other place to add the disc

minor topaz Oct 17, 2024, 10:54 PM

#

ahh

tawdry fjord Oct 17, 2024, 10:59 PM

#

#

i made this with affinity

minor topaz Oct 17, 2024, 11:00 PM

#

haha...

tawdry fjord Oct 17, 2024, 11:00 PM

#

to show the idea

minor topaz Oct 17, 2024, 11:01 PM

#

I already added properties for the pointLight falloff and prenumba to the PointLight structure.

#

But a texture would be really cool.

#

Should be easy enough, if you already got the pattern right.

tawdry fjord Oct 18, 2024, 7:00 PM

#

back to VAT and new mesh struct. CombineMeshes is broken and this time I don't find where to fix the stride from 64 to 72

#

False alarm, I found how to fix it

tawdry fjord Oct 26, 2024, 3:51 PM

#

... sigh ... [ExtrudeCurve] too...
Fixed! ^_^

tawdry fjord Oct 27, 2024, 6:52 PM

#

damn... I reseted my local Dev branch to the last commit on Dev, then I merged my VAT branch, now I get the error

#

this error message and log files are not a good help to find the issue 😦

#

I found which cs file was missing ! 🙂

minor topaz Oct 27, 2024, 7:27 PM

#

oh...

tawdry fjord Oct 27, 2024, 7:29 PM

#

it's all my fault, I added a t3 and t3ui files without the cs file that goes with them

#

I need to make a cleanup commit

tawdry fjord Oct 27, 2024, 8:09 PM

#

done !

minor topaz Oct 27, 2024, 11:45 PM

#

nice! I will look at it tomorrow.

tawdry fjord Nov 9, 2024, 11:16 PM

#

I heard that for houdini users this link is interesting https://sidefxlabs.artstation.com/projects/zOyke6

SideFX Labs

Labs Vertex Animation Textures 3.0 - Part 1

Vertex Animation Textures 3.0 ROP is part of the FREE and open-source toolset in Houdini called SideFXLabs. Vertex Animation Textures (VAT) is an efficient and powerful solution to recreate complex and non-traditional animations in real time. Because VAT only uses textures and shaders to achieve the visuals on the GPU, it has a much lighter perf...

vital bone Nov 9, 2024, 11:24 PM

#

tawdry fjord Nov 9, 2024, 11:25 PM

#

Awesome!

vital bone Nov 9, 2024, 11:26 PM

#

I can confirm the rot map doesnt work as a normal map hehe

#

after unticking the recompute normals setting in the displacemeshVAT node

tawdry fjord Nov 9, 2024, 11:27 PM

#

I'm also not sure about the Normal map exported by the Blender addon

vital bone Nov 9, 2024, 11:27 PM

#

it doesnt have a normals option even for export

#

#

it would be super cool to be able to vary the time offset

tawdry fjord Nov 9, 2024, 11:32 PM

#

it's a limitation of the technique

vital bone Nov 9, 2024, 11:36 PM

#

I think, but I havent actually tried it, in unreal engine the material graph has the vertex offset parameter, and when you instance objects the material shader can get a reference to the unique instance ID, so you can offset the VAT UV using that instance ID value to get variation

tawdry fjord Nov 9, 2024, 11:37 PM

#

ho nice !

vital bone Nov 9, 2024, 11:37 PM

#

so a vertical uv shift is like a time shift variation

tawdry fjord Nov 9, 2024, 11:38 PM

#

we need to investigate this

vital bone Nov 9, 2024, 11:40 PM

#

how does vertex displacement currently work in t3? i never tried it

#

because if it can happen at set material node stage, in theory then you could access the point id after if its been instanced maybe?

#

i dont know enough

strong totem Nov 11, 2024, 1:25 PM

#

I have a scene where I iterate over all points (not too many, around a hundred) with a loop and select a single point (filter out all others to be precise) via the loop counter, then change parameters via the same counter as a seed.
That may work here as well.

#

In my case it's butterflys that should not all beat their wings the same, so a similar issue.

#

It's slow but works.

vital bone Nov 13, 2024, 8:39 PM

#

strong totem In my case it's butterflys that should not all beat their wings the same, so a s...

that sounds like a cool setup

tawdry fjord Nov 15, 2024, 3:30 PM

#

here is a new version of T3, I've added the link about houdini in DisplaceMeshVAT 's description

tawdry fjord Nov 15, 2024, 6:15 PM

#

small update https://www.swisstransfer.com/d/2fab469f-ef73-4fa3-8798-e81f50911f0a

vital bone Nov 27, 2024, 5:05 PM

#

@tawdry fjord I found it

#

ok so, I wanted to try and track a point on the mesh after displacement, it seems to work when the point ID is 0, but if I change the ID it will go funky

#

is the way im getting the point info correct?

tawdry fjord Nov 27, 2024, 5:07 PM

#

don't use operators with _ in the name they might be obsolete

vital bone Nov 27, 2024, 5:07 PM

#

oo

tawdry fjord Nov 27, 2024, 5:07 PM

#

try with MeshVerticesTopoints instead of _MeshBuffer

vital bone Nov 27, 2024, 5:08 PM

#

hell yee

#

thats the ticket

#

thank you

#

its working

tawdry fjord Nov 27, 2024, 5:08 PM

#

😎

vital bone Nov 27, 2024, 5:11 PM

#

tawdry fjord Nov 27, 2024, 6:13 PM

#

vital bone

very nice with [PointTrail2], it could be fun with ParticleSystem. Also you can simplify the process, right after [MeshVerticesToPoints] use [FilterPoints] so you don't need [PointsToCPU]

vital bone Nov 27, 2024, 6:15 PM

#

ooh right nice!

#

thank you

#

I wanted to do like a single point, but colorize it with a velocity magnitude/direction vector to do a feedback system like the paint tool thing you made a while ago

#

at some point hehe

tawdry fjord Nov 27, 2024, 6:16 PM

#

ho yeah in that case you would need PointToCPU

vital bone Nov 27, 2024, 6:17 PM

#

ooh I see, to get the position data, otherwise filter point will just return a buffer of specific point?

tawdry fjord Nov 27, 2024, 6:17 PM

#

unless we make a custom version of FilterPoints

vital bone Nov 27, 2024, 6:17 PM

#

but in theory, if I filter point first, then use pointToCpu it will be faster right?

#

maybe

tawdry fjord Nov 27, 2024, 6:17 PM

#

yeah I do that often

#

not sure if it's faster, But I believe it's faster

minor topaz Nov 28, 2024, 11:24 AM

#

vital bone

That looks soo cool!

vital bone Nov 28, 2024, 1:01 PM

#

@tawdry fjord ive been messing about with this map editor for a Spring engine game recently, and I found a helper tool with a .bat file that will convert an image to .dds through drag and drop, i didnt test it with a vat texture though, but could be interesting from dev standpoint to check out this nvtt thing to allow conversion of textures to dds more easily for t3 https://github.com/Beherith/springrts_smf_compiler/tree/master/tools/win

GitHub

springrts_smf_compiler/tools/win at master · Beherith/springrts_smf...

This tool allows the compilation and decompilation of maps to springrts's binary smf map format. - Beherith/springrts_smf_compiler

tawdry fjord Nov 28, 2024, 2:26 PM

#

vital bone <@476129930027991081> ive been messing about with this map editor for a Spring e...

Since i'm generating the VAT textures in blender and i have the DDS exporter plugin in blender I don't really need this. What would be interesting and futur proof is to support openEXR images loading in T3

#How create and use VAT (Vertex Animation Texture)?