alpine stump
I'm trying to load test a Go microservice using Locust.
We've been exploring Railway's horizontal scaling since we are considering that a distributed approach might yield better results.
I tried a 4-replica setup with 2vCPU and 2GB RAM. The performance is definitely better than vertical scaling alone (as tested on 8vCPU, 8GB RAM).
However, I've noticed that based on the metrics, 2 replicas seem to not work at all. Their usage seem to be 0vCPU and just ~50MB RAM even under load.
Locust charts
Tried to bump it to 2000 VUs since I thought maybe it needed more traffic.
The two replicas seem to not work at all. Is this a load balancing issue?
Redeployed the service
replicas 1 and 3 seem to work now but replica 0 dropped in utilization
Still testing with 1000 VUs
opaque zephyr
Where is Locust making these requests from?
alpine stump
Inside the project
opaque zephyr
Aka, private networking?
alpine stump
Yes
opaque zephyr
Then I suspect Locust is not correctly balancing its requests across all returned IP addresses from the DNS lookup.
alpine stump
Is it also possible to encounter this in public networking?
opaque zephyr
No, on the public network, our L7 proxy will randomly distribute incoming requests to all replicas.
alpine stump
I see. I hope this issue is not present when using cloudflared since we are considering the use of CF tunnels to route in/outbound traffic of our services in Railway.
opaque zephyr
I am not sure if the Cloudflare Tunnel service will correctly distribute requests to all DNS results.
I feel like you may have told me previously, but why use a Cloudflare tunnel instead of our proxy?
alpine stump
For compliance reasons. We are aiming to be PCI-DSS compliant, among other things, and using a centralized approach for a WAF/Network stack is a business decision.
Also, using CF tunnels instead of other service mesh solutions is a better fit in terms of cost and maintainability in our case