I also thought of this:
The server could spawn new scenes and send them over, but give the authority to the peer that logically owns the to do stuff that would get synced by rpc calls or a MultiplayerSynchronizer.
I don't know, there are really infinite ways to do a thing, it's so hard to come up with a solid architecture/plan.