Many users specifically from the State of Rio de Janeiro, coming from various ASNs, report latency to reach all of our sites that are routed through Cloudflare.
This problem has been occurring for 5 days. Solutions?
thanks in advance
#Users with slowness and latency coming from Brazil / Rio de Janeiro
frozen jackal
ancient ermine
Many users specifically from the State of Rio de Janeiro, coming from various AS...
It shows the round trip avg there as 13.5ms, no? Seems pretty ok to me. Packet loss on a hop doesn't matter if it doesn't continue, which that doesn't
frozen jackal
It shows the round trip avg there as 13.5ms, no? Seems pretty ok to me. Packet l...
Hi @ancient ermine thanks!
In general the problems is latency on the 5th hop. All the connections ends on our server but all them that come from Rio de Janeiro happens with slows.
Many users claim to open the website. Avg > 15secs to just open the website. In others cities the avg to open the first page is 1sec
ancient ermine
Hi <@905865275583365161> thanks!
In general the problems is latency on the 5th h...
The latency of a specific hop doesn't matter unless it increases there and stays until the end. To routers, responding to Ping/ICMP is lowest priority and has to happen on the CPU instead of normal traffic which goes over hardware accelerated parts. There's a huge difference between sending stuff to a router vs through a router
If you're interested in learning more about how to debug things with traceroute, https://archive.nanog.org/sites/default/files/10_Roisman_Traceroute.pdf is a good read.
Anyway there's nothing that stands out with that trace, no packet loss at destination and the average/jitter isn't very high.
I would try to get more info on the slowness. On what exact asset/resource is being slow? Is it cached? etc
frozen jackal
@ancient ermine thank you!
I dunno if this helps to understand.
We've 15 domains on Cloudflare, some of them in Pro account. We'd tested all of them using Any Desk in our clients and all had the same problem. It's so slow to open the homepages and browse in our systems. And all of then are hosted in Rio de Janeiro. The other 80% of our clients don't have any problems.
We temporarily shutdown the proxy from one new subdomain and the problem doesn't happen.
ancient ermine
<@905865275583365161> thank you!
I dunno if this helps to understand.
We've 15...
Can you show the headers tab? There's a few I would pay attention to: CF-Ray, the end bit of that header ex: -ORD is the Airport Code of the Cloudflare location, and CF-Cache-Status if you hit cache or not
frozen jackal
Hello @ancient ermine , sorry to no reply before.
I'll gonna check it
ache-control:
no-store, no-cache, must-revalidate
cf-cache-status:
DYNAMIC
cf-ray:
8ace86f93e26cae3-GIG
content-encoding:
br
content-type:
text/html; charset=UTF-8
date:
Fri, 02 Aug 2024 13:48:44 GMT
Request URL:
https://gestao.sistemacorban.com.br/index.php
Request Method:
GET
Status Code:
200 OK
Remote Address:
[2606:4700:20::681a:b21]:443
Referrer Policy:
strict-origin-when-cross-origin
alt-svc:
h3=":443"; ma=86400
cache-control:
no-store, no-cache, must-revalidate
cf-cache-status:
DYNAMIC
cf-ray:
8ace9be419477a78-GIG
content-encoding:
br
ancient ermine
Request URL:
https://gestao.sistemacorban.com.br/index.php
Request Method:
GET
S...
It's dynamic so it's going to your origin. If you go to one of those static (hopefully cached) assets like jquery.min.js or any of the .css's, that is cf-cache-status: HIT, what's the latency on those?
frozen jackal
@ancient ermine The latency of these files is at acceptable levels. The problem is already underway, before the package reaches Cloudflare. I believe that in DNS caching by the backbone mesh here in Brazil. We created a subdomain with a CNAME name and it performs very well in traceroute and pingplotter. We asked the customer community in Rio de Janeiro to use a new route and the issue was resolved. In the meantime we will wait for the main link to normalize.
@ancient ermine Thank you so much for your time, I'm appreciate
wintry epoch
Is this still occurring? If so, can you please get a traceroute/mtr to the destination from them, while they're experiencing the slowness?
wintry epoch
also the colo they see from https://gestao.sistemacorban.com.br/cdn-cgi/trace at the same time
frozen jackal
Thanks for reply @wintry epoch . We have not had any reports of new slowdowns since we created a new CNAME entry in our DNS and shared it with our impacted customers. I will ask my ground team to carry out new tests on our main links and collect the requested data and then return this in this post.
wintry epoch
Thanks @frozen jackal - only if it's still ongoing
You said 5 days - do you know exactly when the issue started?
frozen jackal
Hello @wintry epoch
I'm sorry for not responding quickly.
I contacted the user support team and they just got back to me now.
The problem of slowness and high latency remains in most requests coming from Rio de Janeiro/Brazil to Clouflare.
Our support started reporting this slowness last Monday (July 29th)
Below is information obtained from users who are currently experiencing slowdowns when accessing the gestao.sistemacorban.com.br domain
When they call an alias (crm.sistemacorban.com.br) this problem does not occur (this alias is just a CNAME entry pointing to gestao.sistemacorban.com.br)
fl=937f70
h=gestao.sistemacorban.com.br
ip=177.23.183.233
ts=1722886018.696
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=010-gig09
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=216f263
h=gestao.sistemacorban.com.br
ip=2a09:bac5:aea:878::d8:107
ts=1722886089.67
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=on
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=937f80
h=gestao.sistemacorban.com.br
ip=2804:16d8:b738💯6ccd:4a98:f67a:57b5
ts=1722886168.668
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=216f411
h=gestao.sistemacorban.com.br
ip=2804:14d:7285:8a8b:74b9:1ac8:a9cb:730c
ts=1722886334.224
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
frozen jackal
@wintry epoch hello!
So, some users from Rio de Janeiro who use the temporary URL (CNAME entry) also claim slowness and high latency.
We created a new one for now (no latency claimed yet)
Trace a slow user
fl=937f76
h=crm.sistemacorban.com.br
ip=191.242.124.222
ts=1722890198.972
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/2
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519
Trace for the same user to the new one Entry
fl=937f104
h=crmrj.sistemacorban.com.br
ip=191.242.124.222
ts=1722890968.121
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/2
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519