#vertex normal shader on .glb animated model

19 messages · Page 1 of 1 (latest)

halcyon bramble
#

i have a animated glb model that works great, i tried to swap in a shader for vertex normals and now my animations wont play

const vertexShader = `
  varying vec3 vNormal;

  void main() {
    vNormal = normal;
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
  }
`;

const fragmentShader = `
  varying vec3 vNormal;

  void main() {
    gl_FragColor = vec4(abs(vNormal), 1.0);
  }
`;

const normalMaterial = new THREE.ShaderMaterial({
  vertexShader,
  fragmentShader,
});

const loader = new GLTFLoader();

loader.load('myguy.glb', (gltf) => {
  gltf.scene.scale.set(2, 2, 2);
  gltf.scene.traverse((object) => {
    if (object.isMesh) {
      object.material = normalMaterial;
      object.castShadow = true;
    }
  });

  this._target = gltf.scene;
  this._params.scene.add(this._target);

```  my old working code without custom shader below```javascript

    const loader = new GLTFLoader();
    loader.load('myguy.glb', (gltf) => {
      gltf.scene.scale.set(2, 2, 2);
      gltf.scene.traverse(object => {
        if ( object.isMesh ) object.castShadow = true;
      });

        this._target = gltf.scene;
        this._params.scene.add(this._target);
Screenshot_2023-02-20_102709.png Screenshot_2023-02-17_224745.png
potent pine
#

Threejs uses vertex shaders to perform animation, if you want to write your own, you need to include the relevant parts inside your custom shaders.

#

Example with skinning includes above

halcyon bramble
halcyon bramble
#

had to remove ' gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);', how can i make it update the color every frame with the new normals ? ```javascript
const loader = new GLTFLoader();

loader.load('myguy.glb', (gltf) => {
gltf.scene.scale.set(2, 2, 2);
gltf.scene.traverse((object) => {
object.material = new THREE.ShaderMaterial( {
vertexShader: [
'#include <skinning_pars_vertex>',
'varying vec3 vNormal;',

                            'void main() {',

                            '#include <skinbase_vertex>',
                            '#include <begin_vertex>',
                            '#include <skinning_vertex>',
                            '#include <project_vertex>',

                            '       vNormal = normal;',

// ' gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);',
'}'
].join( '\n' ),
fragmentShader: [
'varying vec3 vNormal;',
'void main() {',
' gl_FragColor = vec4(abs(vNormal), 1.0);',
'}'
].join( '\n' ),
skinning: true
} );
});

this._target = gltf.scene;
this._params.scene.add(this._target);

potent pine
#

Heyo, I'm uncertain what you mean by "new normals", normals don't change, even if the mesh moves.

They are consistent with the vertices of the model, so they should show the same relative direction at the same vertices

#

The pictures you sent seem correct 🧑‍🚀

halcyon bramble
#

i thought "vertex normals" are the direction the vertex is pointing, at the start the feet are on the ground so the color is rgb/xyz(0, -1, 0) so 100% green, as the model lifts its foot the color changes to a "new normal" rgb/xyz(0.5, 0.5, 0) 50%red 50%green 0%blue. makes it hard to google things when i dont know the right words lol

potent pine
#

Ehm what are you trying to do? 😅

halcyon bramble
#

going to stream frames into controlnet and use stable diffusion to paint each frame, that way i can skip the normal map approximation that controlnet dose and have a more acurate normal map fed into stable diffusion

Screenshot_2023-02-20_143843.png Screenshot_2023-02-20_144316.png
potent pine
#

I would think the normal map you've displayed would be sufficient right? The normals shouldn't change, how they're drawn will not change as you're just taking an absolute value, not a relative one.

Is there anything wrong with the current shader that would need to be different?

halcyon bramble
#

i want to use normals relative to the camera, as the model lifts its foot the sole should turn from green to red or blue depending on the axis it is pointing down, currently its only accurate if the model is in the default T pose

Screenshot_2023-02-20_102709.png
#

if its still green when the arms are at the side as it is now, controlnet will have reflections as if those vertices are level with the x axis

potent pine
#

I think this is some translation to world matrix problem, sorry but I'm about to go 😅

Perhaps some other fine folk here could pitch in?

halcyon bramble
#

no worries 🙂 thanks for your help so far