Right-clicking an unselected interactable (NPC, harvest node, etc) while having an object still selected, will often fail to select the new object that the mouse is hovering over (shown with the arrows on the nameplate) and instead send the interact request to the last (still) selected object .
To Reproduce: Garden. Place 5-6 objects, try to just use right-click to select and water/fertilize. It will fail within 5-10 attempts.