I'm trying to deploy PayloadCMS on AWS using an application load balancer and ECS. Making any request without logging in works fine, and the backend admin panel works as well. The frontend is hosted at app.website.com and the backend is hosted at admin.website.com. In development, everything works as expected. I'm also using the same database for both dev and prod. But after logging in with the frontend, any future request (including those that don't have any access controls) from the frontend will time out and give me a 504 gateway timeout error.
For testing purposes I've enabled all traffic to and from the load balancer, ecs containers, and the database. The admin panel login works fine, and I can perform any operation within the backend. I've found that deleting the payload-token cookie on the frontend allows me make calls to the backend again as expected, but logging in will cause the error to return. Also, whenever the backend server restarts (such as during a new deployment), I need to delete the payload-token cookie if present, otherwise the 504 gateway timeout will happen.
Does anyone have any ideas why this could happen?
Edit: I'm using PostgreSQL in AWS RDS.
The error can be replicated easily in postman by calling the login endpoint, then calling any other endpoint.
Edit 2: it works when deploying the container to AWS LightSail, so this is probably an issue with my setup and not with Payload like I originally thought. This isn't true, it was authenticating but the cookie wasn't working so attempting to access authenticated endpoints wasn't working.
#504 Gateway timeout on every request after authenticating
shy zephyr
red craterBOT
red craterBOT
red craterBOT
shy zephyr
It looks like the error is actually with payload. According to its access logs, my application load balancer is forwarding the request correctly but the container is not responding. This is why it's working for logins. I'm unsure if payload is handling cookies incorrectly or most likely there's a setting I have wrong.
Could there be an issue with the fact the ECS containers are using http in the internal subnet? I'm unsure how payload handles authentication while inside a load balancer that terminates TLS.
shy zephyr
The issue was caused by a tokenizer system to obfuscate data at rest. It would tokenize emails in the database, and I was using beforeOperation(login)/beforeCreate/beforeRead hooks to tokenize/detokenize data before reads/writes. For some reason this was interfering with authentication.
red craterBOT