#raphal_api
trail valleyBOT
👋 Welcome to your new thread!
⏲️ We'll be here soon! Typically we respond in a few minutes, but sometimes we might take a bit longer if the server is busy or if you have a particularly tricky question.
⏱️ We close idle threads, which makes them read-only. Once a thread is closed it won't be reopened, but you can always start a new thread if you have another question.
🔗 This thread will always be available, even after it's closed. You can find it again using Discord's search, or you can save this link: https://discord.com/channels/841573134531821608/1435977361676963890
📝 Have more to share? Add more details, code, screenshots, videos, etc. below.
Below are links to other discussions we've had with you in the past week in case you want to review that information. If your question is related to one of these previous discussions, please provide a comprehensive summary of the current state and what you need help with now. We help many users simultaneously, so a summary allows us to resolve your issue as soon as possible.
- raphal_api, 2 hours ago, 3 messages
quiet rune
👋 Hi there, just taking a look now
I'm not sure of another way to do this in bulk. Can you give me a bit more info on the ETL approach you've tried?
quiet rune
I don't know a huge amount about it, but Sigma (https://dashboard.stripe.com/sigma/queries) may be an option. I see there's a connected_accounts_metadata table which looks like it should have the info you're looking for
rare jacinth
Yes of course, we tried with 2 ETL solutions: Hevo and Fivetran.
Hevo update is based on logs, they catch the creation of the account but the metadata is not updated, so is stays empty, unless we do a historical load on the account object which takes hours.
Fivetran update of metadata takes ~6h which is close to what we have with GET /v1/accounts API calls. So it seems they fetch all the accounts, every time.
The optimal solution would be to ingest logs of updated metadata so we don't have to fetch all the account every day.
quiet rune
Ah ok, I don't know anything about those systems unfortunately. Sigma mentioned above is the only other possibility I can think of
rare jacinth
Okay, thank you, I'll look into it
rare jacinth
We don't have sigma enabled so I can't test it right now, but how long would it take to query "select * from connected_accounts_metadata" and export it if it has ~100k rows ? And is it possible to run the query with API calls or an automation tool like n8n and export the result of the query with it ?
trail valleyBOT