Using a scope issues

[steady sleet]

When using the scope I find that my aim suddenly shifts left or right for no reason like it's been pushed.

How to stop it from happening?

[eternal umbra]

Your hand is probably too close to the headset and losing tracking. Unless you are using external tracking like an Index)

Either push your hands forward (not natural) or shake your hand for a second and tracking will work, be it temporarily though.