Cave of memories blocked

[oblique crater]

Cave of memories won't let me go in and gives me the message, "you shouldn't go in. You dont know whose room this is."

[pliant island]

Did you complete the altar to unlock the cave. Unlocking the Savannah does not unlock the cave