Can not comment on your first issue, I'm also not very experienced with authentik. But I'd assume you have a wrong redirect somewhere, I'd suggest you post some screenshots of your proxy and outpost config.
Passwordless login only works on the same domain, the Idea behind this authentication method is, that it verifies the domain and the identity of the one you are authenticating to. The only think you could do is register one passkey for the domain and one for the IP, but in generall I'd suggest you either use IP or name from now on. 😉