Pact ability cooldown can get stuck

[solar bison]

i think this is for #1438224825113120788