Totem minigame getting cancelled

[cinder yoke]

Have you completed the Orlick questline? Because you might just be triggering the auto charm you get from one of his quests