#wiiim

1 messages · Page 1 of 1 (latest)

proven hawkBOT
copper raven
#

hello! how can I help?

nimble mesa
#

Hey 🙂

proven hawkBOT
nimble mesa
#

Let's say we have 100.000 subscriptions scheduled for the 1st of May at 00:00 o clock.
We listen to the subscription.created webhook to create some data in our database.
When processing the webhook, we need to retrieve the SubscriptionSchedule of which the subscription was created from. We then release the subscription schedule. In some cases (if the subscription schedule was canceled by the customer), we do another api call to write some metadata and a cancelAt property on the created subscription. This means we do 2-3 api requests, when processing the webhook.

Our concern is now, that on the 1st of may Stripe will completely block us from rate limiting if all 100,000 subscription.created events receive our webhook in a very short amount of time.

Can you explain to me a bit, how fast does Stripe sent the event to our webhook? We are processing the event async, as suggested by another Stripe Staff, so we return 200 OK instantly after parsing the event and verifying the signature.

austere scroll
#

we send the event as fast as we can and do not rate limit ourselves on how we actually send them

#

so yes you should respond to the webhook with a 200 and push the processing into a queue that you work through for example

nimble mesa
#

So we should put the received event into a queue and put some workers that process e.g. 20 events/s so we wont hit the rate limit of 100 requests/s?

austere scroll
#

well the rate limit is on concurrent requests(it's 100 per second, which is a lot) , so it depends how you build it, if you're making each request sequentially one after another it's hard to hit the limit. There's always a chance though and if you hit the limit you just have to slow down and wait for a little bit before retrying . Also worth closely checking how you integrate and eliminate any extra API calls that can be avoided or deferred until later

nimble mesa
#

Well we process the event async (through a Spring Boot Event Listener), so it's possible for multiple threads to make concurrent requests to stripe's api. I already tried to eliminate as many api calls as possible, e.g. if we only need the subscription id, don't call the retrieve endpoint to get the whole object. But for the subscription schedule, we need to get the object and update some data. Those two calls happen sequentially, so that wouldn't count as 2 requests/s as stripe's api obivously doesn't respond instantaneous?

austere scroll
#

well like if it takes our API 1 second to reply and you're waiting to get the response/read it and then make the next request it's hard to ever reach 100/s yes

#

if you're using multiple threads then you could hit the concurrent limit yep

nimble mesa
#

Okay. Can you estimate on how stripe will react when 100,000 subscription schedules are triggered to create a subscription? Especially the webhook, like I guess its not really possible to send 100,000 webhook events at once. Is there maybe some internal queue on stripe's side that'll sent all events over e.g. 10 minutes? For that large amount of events.

austere scroll
#

I can't say no

#

like I said, we don't make any attempt on our side to slow this down and we just send them as fast as our own event consumer is able to

#

it does happen that sometimes that causes excess load on merchants servers unfortunately, overall I'd suggest things like only setting up your endpoints to listen to specific event types you need to reduce the volume of HTTP traffic you'll get

#

without opening a can of worms, I don't understand what you're actually doing

#

like you can use end_behavior:"release" to have the schedule release itself, at the end of its phases, I'm not seeing why you need to call the API yourself

nimble mesa
#

Oh, im sorry. I'll try to explain: We are a company which will serve a country-wide ticket for public transportation. This launches on 1st of May and we have customers that pre-order this ticket. We are legally required so serve the tickets at 1st of May at 00:00 o clock.

nimble mesa
austere scroll
#

well it's never going to be possible to do everything within one minute at 00:00 if that's what you mean

nimble mesa
#

No, I completly understand. It's just said by the government, but everyone knows that's not going to work. I just don't want things to explode. I'm totally fine if the whole process takes 1 hour. I just want that every processing is succesful, even if it takes some amount of time because of re-trying some api calls or something like that.

austere scroll
#

what will happen is at 00:00 UTC we'll start processing things and generating events and creating subscriptions. I would not be surprised at all if some subscriptions don't get created and emit the .created events multiple minutes later; Midnight UTC on the first of a month is one of the busiest times for us in terms of system load so all the consumers that are handling things like schedules transitioning phases will be operating at a lot of load(we spin up capacity for this but still, it might take a while for things to process)

#

ultimately as long as you robustly build this nothing would fail and it would all be eventually consistent. You'd want to track each task(an instance of managing a new subscription and doing whatever it is you need) somewhere in your databases and have status booleans and flags and so on that let you know what state each thing is in and have ways to re-run the task idempotently if possible