animation transitions not so pretty

[molten kiln]

I set up a system to make my 3d character walk and blend between various clips. when i transition from walk forward to the left strafe animation, the character briefly turns into a ball of limbs pointing in random directions before settling down into the new strafe animation. everything works more or less, it's just clunky for some of the transitions. some pairs of clips also refuse to blend nicely, not just in the transition but trying to play both simultaneously. is this just the current state of animation or am i doing something wrong?

[cinder pine]

How are you doing the animations? Can you share your code?

[solid timber]

That seems like an issue in the way the animation got exported, or an incompatibility between the way bevy reads animations and however they got exported from the tool you use to make them.

at least i'm assuming you made this in some 3d software and didnt handwrite it in 100% bevy code

Ideally you'd want to check that the same animation transitions correctly in your 3d software (blender etc), and once you know it does you'll want to make sure its exported cleanly to gltf

[cinder pine]

to piggyback on laund here, you can put your GLTF file in here https://gltf-viewer.donmccurdy.com/ and visualize it and narrow down if it's the file itself

[molten kiln]

i am happy to share the code, but i have to admit that my setup is a bit compilicated, so i should provide some context, which is probably going to be the end of anyone wanting to help me. to be honest i was kind of hoping people were going to just say the animation systems just weren't production ready, but here goes...

I have ported makehuman into bevy. makehuman is a framework that allows creating custom humanoid shapes from a large number of morph targets, supports resizeable clothes etc..

problem 1) meshes are generated dynamically, so skeleton data is fit dynamically. sometimes the joint angles do not match up
solution 1) i had to do some awkward transform math to make sure all bones are rotated exactly as they are in the reference skeleton, adjusting positions to maintain proper alignment with the new shape

i want to share animation clips between these humans so i grabbed a few from mixamo, retargeted in blender to reference skeleton, export to bevy.

problem 2) mixamo has position tracks on all bones. this breaks retargeting because the bones are all in different positions on different meshes
solution 2) i manually process the gltf file, grabbing each clip and filtering out the translation tracks and creating new AnimationClip assets

in other words i am not using the standard AnimationClip asset import pipeline but doing something custom. it's complicated but it works. animation clips look fine shared among differently shaped humans.

so, we have dynamically generated meshes, dynamically skinned, and animation clips which are dynamically generated all at runtime. technically the clips come from a glb file but it is not so simple

now as to the problem, i have 5 clips total im testing out, an idle loop, walk forward, strafe right, strafe left, walk backward.

now im using bevy_enahanced_input to feed in wasd input as vec2 to drive animations. this is normalized so A is weight 1 on strafe left
but A + S is weight sqrt(2)/2 on strafe left and walk back.

i notice the following:

1. walk forward and strafe right work exactly as intended, blending and transitions are perfect, at any weight.
2. walk back and strafe left are weird. they work individually, but only at full weight (either 1 or sqrt(2) /2 blended). i cut the weight in half and the clips are broken (because they are blended with idle then)
3. at full weight every clip works individually, but i get these 2 sets of clips that work together nicely WD and AS input. transitions within these subsets are fine. But if i try to blend walk forward and strafe left (AW input) then it breaks.

i have no idea. could it be related to something in my import pipeline? sure. but i thought the fact that each clip works fine individually meant i was doing it right.

graph weights logic

pub(crate) fn player_animation_controller(
    movement: Single<&Action<Movement>>,
    run_input: Single<&Action<Run>>,
    mut graphs: ResMut<Assets<AnimationGraph>>,
    mut controller: Query<(&PlayerAnimationController, &mut AnimationPlayer)>,
) {
    if let Ok((controller, mut player)) = controller.single_mut() {
        for (_, clip) in player.playing_animations_mut() {
            clip.set_weight(clip.weight().lerp(0., LERP_SPEED));
        }
        // Test, reduce the weight on the movement clips
        let movement = ***movement;// / 2.;

        let speed = movement.length();
        let running = ***run_input;

        let graph = graphs.get_mut(&controller.graph).unwrap()
        let normal_idle = player.animation_mut(controller.normal_idle).unwrap();
        normal_idle.set_weight(normal_idle.weight().lerp(1.0 - speed, LERP_SPEED));

        if movement.y >= 0. {
            let normal_walk = player.animation_mut(controller.normal_walk).unwrap();
            normal_walk.set_weight(normal_walk.weight().lerp(movement.y, LERP_SPEED));
        } else { 
            let normal_walk_back = player.animation_mut(controller.normal_walk_backwards).unwrap();
            normal_walk_back.set_weight(normal_walk_back.weight().lerp(movement.y, LERP_SPEED));
        }

        if movement.x >= 0. {
            let normal_walk_strafe_right = player.animation_mut(controller.normal_walk_strafe_right).unwrap();
            normal_walk_strafe_right.set_weight(normal_walk_strafe_right.weight().lerp(movement.x, LERP_SPEED));
        } else {
            let normal_walk_strafe_left = player.animation_mut(controller.normal_walk_strafe_left).unwrap();
            normal_walk_strafe_left.set_weight(normal_walk_strafe_left.weight().lerp(movement.x, LERP_SPEED));
        }
    }
}

graph setup

    let normal_idle = clips.get("normal-idle").unwrap();
    let normal_walk = clips.get("normal-walk").unwrap();
    let normal_run = clips.get("normal-run").unwrap();
    let normal_walk_backwards = clips.get("normal-walk-backwards").unwrap();
    let normal_run_backwards = clips.get("normal-run-backwards").unwrap();
    //let normal_run_jump = clips.get("normal-run-jump").unwrap();
    let normal_walk_strafe_left = clips.get("normal-walk-strafe-left").unwrap();
    let normal_walk_strafe_right = clips.get("normal-walk-strafe-right").unwrap();
    let normal_run_strafe_left = clips.get("normal-run-strafe-left").unwrap();
    let normal_run_strafe_right = clips.get("normal-run-strafe-right").unwrap();


    let mut graph = AnimationGraph::new();
    let movement = graph.add_blend(1., graph.root);
    let normal_blend = graph.add_blend(1., movement);

    let normal_idle = graph.add_clip(normal_idle.clone(), 1., normal_blend);
    let normal_walk = graph.add_clip(normal_walk.clone(), 1., normal_blend);
    let normal_run = graph.add_clip(normal_run.clone(), 1., normal_blend);
    let normal_walk_backwards = graph.add_clip(normal_walk_backwards.clone(), 1., normal_blend);
    let normal_run_backwards = graph.add_clip(normal_run_backwards.clone(), 1., normal_blend);
    //let normal_run_jump = graph.add_clip(normal_run_jump.clone(), 0., normal_blend);
    let normal_walk_strafe_left = graph.add_clip(normal_walk_strafe_left.clone(), 1., normal_blend);
    let normal_walk_strafe_right = graph.add_clip(normal_walk_strafe_right.clone(), 1., normal_blend);
    let normal_run_strafe_left = graph.add_clip(normal_run_strafe_left.clone(), 1., normal_blend);
    let normal_run_strafe_right = graph.add_clip(normal_run_strafe_right.clone(), 1., normal_blend);


  

here is the animation clip import. not sure if anyone really wants to read this or how much it will help but here it is

pub(crate) fn get_animation_clips(
    path: impl AsRef<Path>,
    translation_tracks: TranslationTracks,
) -> Result<AHashMap<&'static str, AnimationClip>, BevyError> {
    let (document, buffers, _) = gltf::import(path)?;
    if document.skins().len() > 1 { return Err(BevyError::from("More than one skin present in file")) };
    let Some(skin) = document.skins().next() else { return Err(BevyError::from("No skins available")) };
    let mut transforms = AHashMap::<&'static str, Transform>::default();
    let mut node_indices = AHashMap::<&'static str, usize>::default();

    // Get joint local transforms
    for joint in skin.joints() {
        let name = joint.name().unwrap_or("");
        node_indices.insert(NAME_INTERNER.intern(name).leak(), joint.index());
        let (pos, rot, scale) = joint.transform().decomposed();
        let transform = Transform {
            translation: Vec3::from_array(pos),
            rotation: Quat::from_array(rot),
            scale: Vec3::from_array(scale),
        };
        transforms.insert(NAME_INTERNER.intern(name).leak(), transform);
    }

    // Convert to global transforms
    let mut global_transforms = AHashMap::default();
    let root = &find_root_joints(&skin);
    compute_global_transform(root, &transforms, &mut global_transforms, Transform::IDENTITY)?;

    // Get bone paths
    let joint_targets = build_joint_paths(&find_root_joints(&skin));

    // Output clips
    let mut new_clips = AHashMap::default();

    // Build new clips
    for animation in document.animations() {
        let mut clip = AnimationClip::default();
        let clip_name = animation.name()
            .ok_or(BevyError::from("Animation clip has no name"))?;

        for channel in animation.channels() {
            // Get input values (t1, t2, ...)
            let sampler = channel.sampler();
            let input_accessor = sampler.input();
            let input_view = input_accessor.view()
                .ok_or(BevyError::from("Failed to get input_view for animation"))?;
            let buffer = &buffers[input_view.buffer().index()];

            let start = input_accessor.offset() + input_view.offset();
            let end = start + input_accessor.count() * std::mem::size_of::<f32>();
            let data = &buffer[start..end];
            let times: &[f32] = bytemuck::cast_slice(data);
            let times = times.iter().map(|x| *x).collect::<Vec<_>>();

            // Get output values (pos1, pos2, ...) or (quat1, quat2, ...)
            let output_accessor = sampler.output();
            let output_view = output_accessor.view()
                .ok_or(BevyError::from("Missing output view"))?;
            let buffer = &buffers[output_view.buffer().index()];

            let target = channel.target();
            let target_property = target.property(); 
            let target_node = target.node();
            let target_name = target_node.name()
                .expect("Failed to match node name");
            let target_name = Name::new(NAME_INTERNER.intern(target_name).leak());
            let target_id = AnimationTargetId::from_names(joint_targets[&target_name].iter());

            let start = output_view.offset() + output_accessor.offset();
            let floats_per_element = match target_property {
                gltf::animation::Property::Translation | gltf::animation::Property::Scale => 3,
                gltf::animation::Property::Rotation => 4,
                _ => { continue } // Morph target weights
            };
            let end = start + output_accessor.count() * floats_per_element * std::mem::size_of::<f32>();
            let values = &buffer[start..end];
            let floats: &[f32] = bytemuck::cast_slice(values);

            // Add new curve to new clip
            match target_property {
                gltf::animation::Property::Translation => {
                    if matches!(translation_tracks, TranslationTracks::None) { continue };
                    if matches!(translation_tracks, TranslationTracks::Root) && target_name.as_str() != root.name().unwrap() { continue };

                    let values: Vec<Vec3> = floats
                        .chunks(floats_per_element)
                        .map(|chunk| Vec3::from_array([chunk[0], chunk[1], chunk[2]]))
                        .collect();
                    clip.add_curve_to_target(
                        target_id,
                        AnimatableCurve::new(
                            animated_field!(Transform::translation),
                            AnimatableKeyframeCurve::new(times.into_iter().zip(values.into_iter()))
                                .expect("Failed to construct curve")
                        )
                    );
                }

                gltf::animation::Property::Scale => {
                    let values: Vec<Vec3> = floats
                        .chunks(floats_per_element)
                        .map(|chunk| Vec3::from_array([chunk[0], chunk[1], chunk[2]]))
                        .collect();
                    clip.add_curve_to_target(
                        target_id,
                        AnimatableCurve::new(
                            animated_field!(Transform::scale),
                            AnimatableKeyframeCurve::new(times.into_iter().zip(values.into_iter()))
                                .expect("Failed to construct curve")
                        )
                    );
                }
                gltf::animation::Property::Rotation => {
                    let values: Vec<Quat> = floats
                        .chunks(floats_per_element)
                        .map(|chunk| Quat::from_array([chunk[0], chunk[1], chunk[2], chunk[3]]).normalize())
                        .collect();
                    clip.add_curve_to_target(
                        target_id,
                        AnimatableCurve::new(
                            animated_field!(Transform::rotation),
                            AnimatableKeyframeCurve::new(times.into_iter().zip(values.into_iter()))
                                .expect("Failed to construct curve")
                        )
                    );
                }
                _ => { continue }
            }
        }
        new_clips.insert(NAME_INTERNER.intern(clip_name).leak(), clip);
    }

    Ok(new_clips)
}

i was testing out rescaling the translation tracks and implementing root motion, but it is not as simple as i thought so right now i am filtering out all translation tracks

[solid timber]

My thinking is that each clip can work individually, but if between them a lot of things change the interpolation bevy does between clips is probably whats messing up

[molten kiln]

yes it makes the most sense to me.

but the walk backward clip is almost just the walk forward clip in reverse so i don't why it wouldn't work just as well blending with idle for example. the poses are nearly identical.

[cinder pine]

The poses are nearly identical but the timing that they are in the poses isn't so when it blends it might just be trying to blend values that don't make sense visually. I'm trying to think of a way to explain it

[cinder pine]

Wait am I reading this right? Your weight for walking backwards or left is negative?

[molten kiln]

😬 that sure was the problem. good eye. everything works perfectly now

not feeling too smart at the moment, but it's kind of an easy mistake to make. maybe there should be checks for this in the animation system.

kind of odd that it still worked as a single clip though

[cinder pine]

Its an easy mistake to make. I missed it the first time I read through