@tame pumice
Ty for the response.
I've been doing complicated things like this for nearly two decades now, ranging from building physics engines from scratch to running scientific computations for aerospace companies.
I have a special need when it comes to performance :p
But yeah, this solution is the logical and practical way this should be solved. There are a few optimizations that I could've added in my initial message but discord's message limit hit me.
I personally run a self modded SCUM server where I modded the crap out of the server, I reverse engineered their C++ assembly code with the help of IDA and other tools, now I do the frick I want with it. :p
I even asked them if it was fine and they were pretty chill about it as well since they said they're about to roll mod support of their own sooner or later.
SCUM has a partial solution to what I said, its not perfect since they didn't want to follow all the instructions, but it works for their purposes I guess. (They cheated by just increasing the network sync distance to a larger one and turned off occ. culling(this check) after a few hundred meters :p
Garry on the other hand went full ham with rust, they almost had it, but their programmers are having issues so this progress is kind of stale right now, but you can still scroll the rust dev log and find the entries. It works exactly like I mentioned, but without server GPU support since they want it to be portable.