#grb
mystic adder
tepid mason
I believe the rate limit for test-mode API keys is 25 requests per second. You're running multiple concurrent tests that are somehow generating more traffic than that?
short jasper
I think it varies by endpoint but yes, much more than that
Not sustained at that level, but the requests tend to come in big bursts. E.g., when all of the VMs first start up, they all hit the list prices endpoint to get the prices, and that endpoint seems to have a particularly low rate limit. The create customer endpoint is also a frequent problem
short jasper
It's just a web app. About 500 tests total. But high parallelism is what causes the problem
tepid mason
I see what you're saying. Is there any reason the requests need to be concurrent? Why can't you do them serially over a slightly longer period of time?
To answer your earlier question: there isn't a way to increase test-mode rate limits currently.
short jasper
Any given test will rarely (if ever) make concurrent Stripe requests. But it's the tests themselves that are concurrent. E.g., just the billing system tests alone would probably take something like 5-10 minutes if run serially, and that's not including the rest of the test suite. The billing tests currently run with 21-way parallelism, but some other tests in the system go as high as 140 simultaneous parallel tests in flight. I currently have our total CI run down to about 2.5 minutes total. Trying to get it even lower for faster feedback
In any case, if there isn't a way then I'll just run with multiple accounts. (Maybe this makes my last question make more sense... it's a lot easier to mix up keys when you have 4 CI keys, plus personal dev keys!)
tepid mason
That makes perfect sense now, thanks for the clarification. Real quick, do you have an account ID I can look at for the live-mode account that's handling your production server?
So you have it backwards. The API keys shouldn't be published anywhere public. IDs of accounts and other Stripe objects are totally fine to share. You can't do anything with them without the API keys
short jasper
That was the publishable key which is present in users' browsers when using Checkout/Elements/etc.
acct_1F4y9KDU21w5evhb
tepid mason
Right, but posting that to a public discord server full of developers is not a good idea.
Anyway, taking a look now. Will circle back in a second
short jasper
Fair enough
tepid mason
Not sure if this is helpful, but you could probably migrate some amount of the current load to Stripe Mock: https://github.com/stripe/stripe-mock
You can't simulate errors or anything that requires persistent state and there are a few other limitations (see the "limitations" list in README.md), but simulating some of the supported successful flows from within your own mock Stripe API could potentially reduce overall load for you and cut down on response times.
short jasper
Thanks, I might do that. We do have one particular suite that could probably be switch to that
tepid mason
Okay, awesome! I can't really think of a way to reduce the time for the tests to run aside from what you've already mentioned.
Hopefully Stripe Mock and help take some of the load off the top though
short jasper
Yeah, worth a try. Thanks, have a good day