The game provides us all a way to protect our Keeps by using Peace shields. A 24hr Peace shield is 1k gold - an amount that's easy to accumulate with daily activity.
This game isn't passive, it requires daily attention, growth planning, and collaboration. The fact that the game is 24/7 worldwide with many different timezones is a draw for many.
I would say since you were scouted several times your opponents took their time meaning you weren't bubbled for a long period of time - or you're always in the same place on the map so easy to track. Nothing unfair happened here. It's the game in a nutshell.