#Can't get Immich to work through Apache2 reverse proxy

1 messages · Page 1 of 1 (latest)

trail sage
#

I've been spent several evenings trying to get Immich working through an apache2 reverse proxy, but every time the browser ends up timing out with a 502 error.

The apache config is attached.
I am confident that this config and the Apache2 server are working, as if I change the port number from 2283 to another port associated with a different service available on <localhost> then by going to album.mydomain.com, that service is working correctly. Apache logs page requests to that service as expected. mod_proxy and mod_proxy_wstunnel are enabled.

When reverted to port 2283, apache2 just records one log entry in access.log with a 502 bad gateway error, even with logging set to verbose.

I can't find anything relevant in the Immich logs, even when the log level is cranked up to verbose.

The Immich server itself is working correctly when accessed directly on the local network.

Is there a way to get Immich to log failed web access attempts? Any other ideas as to what may be wrong?

📎 album.cfg
stoic urchinBOT
#

:wave: Hey @trail sage,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich.

References

#

Checklist

I have...

  1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: read applicable release notes.
  3. :ballot_box_with_check: reviewed the FAQs for known issues.
  4. :ballot_box_with_check: reviewed Github for known issues.
  5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: uploaded the relevant information (see below).
  7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

  • Your docker-compose.yml and .env files.
  • Logs from all the containers and their status (see above).
  • All the troubleshooting steps you've tried so far.
  • Any recent changes you've made to Immich or your system.
  • Details about your system (both software/OS and hardware).
  • Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
  • The version of the Immich server, mobile app, and other relevant pieces.
  • Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

stoic urchinBOT
stray thunder
wooden valley
#

Are your other services also running in docker? Does curl http://localhost:2283 work fine?

trail sage
#

There are some other services running in docker. However curl http://localhost:2283 does not work. From another machine on the local network, curl http://192.168.10.1:2283 does work.
I'm guessing this must be something to do with what docker is doing in iptables (which I have not touched since docker setup) as I don't see anything in immich to do with "permitting localhost" or similar. Clues appreciated.

trail sage
stray thunder
trail sage
#

It works from anywhere on 192.168.10.* except 192.168.10.1, which is the server's address.
It's weird because curl http://192.168.10.1:80 does work from 192.168.10.1 as Apache2 is also running (not in docker) on that server, proving that internal routing is working. curl http://localhost:80 also works.

stray thunder
#

What about http//127.0.0.1:2283

trail sage
#

No, http//127.0.0.1:2283 just hangs like the others. http://127.0.0.1 does work to the main web server

stray thunder
trail sage
#

Just noticed docker has added this to iptables:
-A DOCKER -d 172.18.0.2/32 ! -i br-9996ae0bd001 -o br-9996ae0bd001 -p tcp -m tcp --dport 2283 -j ACCEPT
-A DOCKER ! -i br-9996ae0bd001 -p tcp -m tcp --dport 2283 -j DNAT --to-destination 172.18.0.2:2283

wooden valley
#

That's the internal network, you could try curl'ing that as well

#

What's the output of ss -tulpn | grep 2283 @trail sage ?

trail sage
#

tcp LISTEN 0 4096 0.0.0.0:2283 0.0.0.0:* users:(("docker-proxy",pid=4003411,fd=8))
tcp LISTEN 0 4096 [::]:2283 [::]:* users:(("docker-proxy",pid=4003418,fd=8))

wooden valley
#

try from the IP shown by ip a show docker0

#

what is the host OS btw @trail sage

#

and any output with sudo sysctl -a | grep rp_filter | grep '1$'

trail sage
#

Debian Bookworm
No output from sudo sysctl -a | grep rp_filter | grep '1$'
There is however one relevant entry if I discard the last grep: net.ipv4.conf.br-9996ae0bd001.rp_filter = 0

wooden valley
#

yeah but it's the 1 we don't want, 0 or 2 is fine

trail sage
#

Seems to be a problem with the way docker/bridge handles localhost traffic.