I know that patch 9 is underway and suggestions for it likely cannot be taken any more. However if you folks at EGS continue to optimize the game, I have an idea for how to optimize the main locations:
🟥 - The location is fully loaded - this will be directly around the location as well as inside of it (duh).
🟩 - The interior and exterior of the location have become low-poly, and a few small meshes and items will unload.
🟦 - The exterior stays the same as in the green section, but everything in the interior except for key items (Mayor's jukebox, Baker's oven, etc) and the AI have been unloaded.
All the land outside the three colored zones. - The exterior will become even more low-poly, and everything inside the location is unloaded. In order to prevent the player from noticing the unloaded interior, low-poly curtains that are color-coded to the location they're in (dark red with golden stripes for museum, blue with yellow stars for mayor's house, etc).
Here is an example of how this could work with the Raven Brooks Banner: