Weird Raycasting bug

[chilly moth]

It looks like your code is putting object inside of the collider using non-physics engine methods. When you pushout from the collider then falling physics takes over correctly.

Either use Character controller that handles colliders correctly or use physics to move objects.

[clever oasis]

I'm making a raycast based controller. The way it works now is it zeros out yvelocity and adjusts based on the hit object under the player.