If a foragable item grows in the spot you'd typically have your model teleport to - via the rafflesia flowers, your character WILL NOT teleport and just stay at the starting position after the fade-to-black.
I ran to the Savannah spot to see if I was able to teleport home when I couldn't teleport to the Savannah, and I saw this flower here and assumed it's the cause of the problem haha :)
After removing it, I am able to teleport back and forth just fine!