#andreas-munk_webhooks
marble nexusBOT
👋 Welcome to your new thread!
⏲️ We'll be here soon! Typically we respond in a few minutes, but sometimes we might take a bit longer if the server is busy or if you have a particularly tricky question.
⏱️ We close idle threads, which makes them read-only. Once a thread is closed it won't be reopened, but you can always start a new thread if you have another question.
🔗 This thread will always be available, even after it's closed. You can find it again using Discord's search, or you can save this link: https://discord.com/channels/841573134531821608/1464201531313361002
📝 Have more to share? Add more details, code, screenshots, videos, etc. below.
still trail
hey there 👋 taking a look into the example you shared
can you confirm if you've already tried the advice mentioned in this guide: https://support.stripe.com/questions/webhooks-how-to-investigate-and-fix-timed-out
ocean prawn
Yes - we reduces the response time in an update a few days ago... and we also have other customers with same response times and no timeouts/errors
Padeldome Vienna
Tennis Point Vienna
We also checked if there are differences in the used api version
We have about 1.300 customers, a lot of them using stripe and there are about 10 customers with these problems
still trail
thanks for this context! I'm still looking into the example you shared above
it does look like this was a true timeout, but looking into our logs to confirm
this specific example timed out after ~20 seconds, so it does appear to be a true timeout
one thing that sometimes causes this is Stripe's IP address may be blocked by firewall or security configurations - the IP addresses may need to be whitelisted
the full list of IP addresses we use to send webhook requests can be found here: https://stripe.com/docs/ips#webhook-notifications
I'd recommend ensuring that those IP addresses are allowlisted with the hosting provider of the webhook URL that is experiencing these errors
in most cases, that resolves issues like this, and if the issue persists after doing so, it should be possible to narrow down a more specific root cause
ocean prawn
thanks - whitelisting in progress - we keep our eyes on the logs...
still trail
great! if these errors persist after whitelisting then we'd be glad to help you dig deeper into this
ocean prawn
thanks!
But is it possible that there are some blocked ips just for some customers and all other customers don't have these blocked webhook ips
still trail
this is not likely to be relevant in this case, Stripe doesn't commonly block IPs in this context
there could be blocks on the endpoint host providers' sides, which is more likely
ocean prawn
yes i know but if there are some of the stripe webhook ips blocked, why are they used just at 10 of our 1000 customers?
mortal helm
jumping in for a helping hand here
I think what's missing from this conversation is a very crucial context of this being transient.
if your IPs were blocked we would see a different behavior where all conversations would be timing out instead of delivery retries being successful.
mortal helm
exactly my point. that being said, the way to prevent this from happening in the future, is to reduce the logic at the level of your webhook endpoint and treat the webhook endpoint as a dispatcher instead of a handler where the post should be handled as a fire-and-forget meaning that you would respond as fast as possible with a 200, and behind the scenes you can have a queue where you submit the event and have a handler there that consumes the events on that queue
ocean prawn
here another example from a customer with just a few transactions
mortal helm
this would ensure that any error wouldn't be caused by your logic but instead would really be a problem from your provider or your service being down for some reason
please read what I'm writing before just posting some random charts.
we already acknowledged it's a transient issue. now the real problem is what in your code is preventing a response before the timeout that Stripe sets for awaiting a 200 response
ocean prawn
I understand what you mean and we try to reduce the process as you can see on the chart where the response time is much lower since 19.1. - but we have the same logic at the chart of Padeldome Vienna with about 1000 transactions in 7 days without any errors
mortal helm
so you need to look at your logs, maybe you're calling our API, maybe you're calling your database, I can't know for sure... this is something that you need to investigate on your end
ocean prawn
what is the maximum response time after that your hook get in timeout?
mortal helm
we don't communicate an official timeout because this might change but the unofficial answer is 20seconds