Slow to connect replica | Railway | Page 1

solemn otter Jun 8, 2024, 1:48 PM

#

I am using Uptime Kuma to detect down services. Our game server has a few replicas. And for some reason it seems like one of the replicas are 'slow' to connect to. Using Chrome developer tools I can see that it will randomly take a very long time (1s+) to connect to the replica. I am pinging the health-check endpoint which returns how long the check took. My server responds with <10-20ms, but the request does not resolve for over 1s+.

It only randomly happens. I can keep pinging the health-check end point and I get a lot of very quick responses. But every 4-5 attempts hits the 'slow' replica. (I assume).

Project ID: 4c3b4b0e-006a-407e-90c7-9c3031cd622f

Service: 2077a108-22c6-4367-951d-41afafbfd450

Health-check endpoint is {service url}/health-check

Maybe this will resolve on its own soon, I will keep an eye on it. For now I have set the timeout to 5s so we aren't being flodded with alerts.

sterile ventureBOT Jun 8, 2024, 1:48 PM

#

Project ID: 4c3b4b0e-006a-407e-90c7-9c3031cd622f,2077a108-22c6-4367-951d-41afafbfd450

tall inlet Jun 8, 2024, 1:52 PM

#

Are you using the edge proxy?

solemn otter Jun 8, 2024, 1:58 PM

#

whats the edge proxy

tall inlet Jun 8, 2024, 2:01 PM

#

a new home grown proxy that will replace railways current envoy proxy, it's meant as a faster (to say the least) replacement

#

and it's a simple toggle switch away

solemn otter Jun 8, 2024, 2:09 PM

#

i dont know anything about it

#

i can try it out?

tall inlet Jun 8, 2024, 2:10 PM

#

absolutely

#

please do

solemn otter Jun 8, 2024, 2:10 PM

#

this happened right after i pushed some changes and forced a new build, which i know would suggest its probably something i did.. but..! the pattern doesnt make sense. at all. 3HC_think

#

maybe i can log out the instance id on health check to see if its the same instnace that is responding slowly

tall inlet Jun 8, 2024, 2:11 PM

#

that would be a very good idea, do you know the env var for the replica id?

solemn otter Jun 8, 2024, 2:12 PM

#

yeah i think so

#

any link to info about this edge proxy thing?

tall inlet Jun 8, 2024, 2:18 PM

#

https://railway.app/changelog/2024-05-17-new-edge-proxy-beta#introducing-the-new-edge-proxy-(beta)
https://help.railway.app/feedback/introducing-railway-s-new-edge-proxy-e6c7d6cf

solemn otter Jun 8, 2024, 2:26 PM

#

thanks

#

all seems to be working now, i did enable edge proxy but i also redeployed, well, everything

#

#

so yeah uh we are back to normal

#

3HC_Shrug

tall inlet Jun 8, 2024, 2:35 PM

#

awsome, let me know if you see any more issues

solemn otter Jun 8, 2024, 2:39 PM

#

seems like the new prooxy kills socket io

#

so ive turned of the edge proxy beta on our socket io service

#

i wonder if thats worth reporting..?

tall inlet Jun 8, 2024, 2:46 PM

#

websockets should work just fine

#

are you getting any specfic error?

#

websockets do indeed work

solemn otter Jun 8, 2024, 2:53 PM

#

try with a couple of replicas?

#

i suspect the handshake/upgrade is failing

tall inlet Jun 8, 2024, 2:54 PM

#

are you on the legacy or v2 runtime?

solemn otter Jun 8, 2024, 3:37 PM

#

V2

tall inlet Jun 8, 2024, 3:43 PM

#

i cant reproduce any websocket errors with the v2 or the legacy runtime, or the edge proxy, please share your errors

solemn otter Jun 9, 2024, 1:35 AM

#

I'll investigate and open a new thread if needed!

#

Thanks

#Slow to connect replica