When having the same friends come to my Server for Co-Op they complained of loosing explored area that they have already done on my server. The same friend stated its happened x2 with some kind of reset of fog of war. I do toggle my main world from solo to hosted as needed for my friends to joining here and there.
One potential fix idea is to use a cartographer like table like in Valheim that shares players explored area after they interact with the table. I would recommend making it at least a tier 3 table though. Would solve the issue and make it more enjoyable when joining a friend's world.