I'll refer the state where you have no safezone protection after killing someone as "red watch"
The red watch system is very broken and ppl can very easily abuse it, here's some ideas to try to make it a little better
Shooting arrows at other people in safezone doesn't give you red watch
Bomb hitting other people in safezone doesn't give you red watch
When someone leaves the server with red watch, the duration increases and transfers between servers
Check if the victims health is 0 or below THEN give red watch to the killer
(There's a bug where if you go in safezone while dying you survive and the person attacking you gets red watch)
Lastly, add a skill that reduces damage from other players specifically (and make it not work in arena)