#I_Be3X_I
timid fiber
sly iris
Hi. You could look into using auto pagination: https://stripe.com/docs/api/pagination/auto
sudden harness
Let me check reaaly quick 🙂
So, lets say that a user has 200 charges
How would we use this when making two separate calls in two separate threads
and prevent those two threads to call the same data?
sly iris
Why do you need multithreading?
sudden harness
we will just make 1 call with each thread.
Because when we are retrieving 300000 charges from a customer, it makes the process very slow
our app connects to the customer data by querying it in real time and displaying the calculations on the Stripe App
We are currently not storing their data anywhere
sly iris
Gotcha. Yeah doing that for 300,000 charges is going to be slow no matter what. You could try multithreading, but how that works will depend on your language. Also you wouldn't be able to use auto pagination if you require multithreading. You'd need to keep track of some state (like starting_after: https://stripe.com/docs/api/charges/list#list_charges-starting_after) for the queries and have the info communicated to multiple threads somehow (which will depend on your language)
Also keep in mind rate limits with threads
sudden harness
yup
those are exactly the problems we are having
if we use Starting_after, it is still a proble for multithreading
problem*
because we don't know how many charges are in a specific time frame. Let's say we have multiple threads and we define starting_after for them. We don't know if for a specific data range, the user had more than 100 charges (request limit). If that is the case, we will miss big chunks of data when querying it through multithreading
sly iris
Yeah. It's going to be difficult to do it this way. When you're dealing with that many charges and not storing any info on your end, it's going to be slow.
sudden harness
Even if we store it on our end, we won't be able to process the data live for a Stripe App
which kinda makes the Stripe App ecosystem redundant since users are looking for live and instant information there. If you find any work around for this, let us know 😄
Thank for your time
sly iris
What does your app do exactly? Why do you need all 300,000 charges at once?
sudden harness
We run forecasts
For example, we take last years data and forecast the revenue for the next month. We are also working on calculating other metrics (that also require at least 1 year of data from the user)
How do you use this : "created.gte" Using python?
it is one of the parameters, but I am not sure if it will work since it has a "."
sudden harness
no prob
wild lynx
created.gte expects a timestamp (numeric value)
Such as created.gte = 1639755688
in Python, I believe this field is a dictionary
so must be something like
"gte": 1639755688
}
sudden harness
so must be something like
```created = {
"gte": 1639755688
}
```
This makes more sense
Alright thanks!
If you know any way to query more than 100 charges per request let us know 🙂
wild lynx
Yeah as my colleague suggested above, you'd either use multithreading (which may run into rate limits) OR use memoization (store the data previous data/results & only run the algorithm taking in the new inputs)
sudden harness
Are there any plans to increase the limit per query?
wild lynx
None that I'm aware of
sudden harness
okay thanks both!
wild lynx
NP! 🙂 Good Luck