#garebear_webhooks
drifting badgerBOT
👋 Welcome to your new thread!
⏲️ We'll be here soon! Typically we respond in a few minutes, but sometimes we might take a bit longer if the server is busy or if you have a particularly tricky question.
⏱️ We close idle threads, which makes them read-only. Once a thread is closed it won't be reopened, but you can always start a new thread if you have another question.
🔗 This thread will always be available, even after it's closed. You can find it again using Discord's search, or you can save this link: https://discord.com/channels/841573134531821608/1334641922865762447
📝 Have more to share? Add more details, code, screenshots, videos, etc. below.
round lodge
can see this user resulted in two webhooks with different titles. the one with "subscribed to" has a subscription id that doesn't exist. the one with "attempted to subscribe to" has the actual subscription id.
this is causing failures in our webhook processor since these subscriptions don't actually exist. no code on our end has changed here and this happened with our last two subscriptions but now one that happened a couple hours ago
i didn't see anything on the stripe status page so wanted to figure out if this is a known issue or what could be happening here
torpid turret
Hello, can you send me the ID of each of those events? (evt_1234)
Not immediately sure what this could be, but if I can look at a specific example that should be enough to debug
round lodge
just double checking it's safe to share those event ids publicly?
kindred geyser
@torpid turret @round lodge I don't want to hijack, but I wanted to pop in and say I came here to report the same issue, so far two instances within the last hour.
round lodge
i don't mind at all. more confirmation is great!
kindred geyser
Fwiw, the event IDs are safe to share, here are mine:
evt_1Qn5eO2nDm85BVWRHEfgqOHu
evt_1Qn54d2nDm85BVWRfXXXpFfm
Both of them for different customers, but both immediately preceded by a customer.subscription.created event for a Subscription that actually exists.
Right now this isn't creating issues re. billing for us since we're getting the Subscription that does exist from the preceding events, but it's firing our alerting on the nonexistent Subscriptions.
round lodge
the event with a subscription id that doesn't exist also has an invoice id that doesn't exist
torpid turret
Thanks for the IDs, looking into those and I am checking if we have any ongoing service degredations that may cause this.
Gotcha, those events can help me look into this issue as well. I may be able to find more around what those IDs are/were
round lodge
### User 1
evt_1Qn5KhJeU8q0Jt2jy04NLMjF - Success
evt_1Qn5KhJeU8q0Jt2jyBuOKd7r - Failed, subscription/invoice id doesn't exist
### User 2
evt_1Qn5ADJeU8q0Jt2jGQKKgRGh - Success
evt_1Qn5ADJeU8q0Jt2jYT5HQIST - Failed, subscription/invoice id doesn't exist
for each of the users those are the two customer.subscription.created events they received. the first one is fine and has the correct subscription id the second refers to some non-existent subscription and invoice id
torpid turret
Thanks for those events, the subscription IDs aren't turning up anything for me either. I do see that our logs show that one API request seems to have created both of those subscriptions. I will escalate this and keep you updated.
https://dashboard.stripe.com/logs/req_9rATtTwSsrFReA
kindred geyser
Want to echo what @round lodge said above - the Invoices in these events don't exist either. One other interesting tidbit that I might add is that the event for the non-existent Subscription also has "discount": null while the actual real Subscription does indeed have a discount / coupon applied to it (at least in my case).
Not sure if that information helps at all.
torpid turret
That is good to know, I will pass that on. Also do you know if this started at around the same time for you (~within the last hour)?
round lodge
my subscriptions also have discounts applied (though it doesn't show up in the event itself)
kindred geyser
That is good to know, I will pass that on. Also do you know if this started at around the same time for you (~within the last hour)?
Yes, as far as I can tell this started for us at 2025-01-30 21:16:07 UTC
torpid turret
Thanks, passing all of this on. We are still working to identify and fix the root cause. I will keep posting updates here
round lodge
thanks!
torpid turret
@kindred geyser @round lodge We are rolling out a fix now
torpid turret
Good question, looking in to that.
The only thing that I am finding to prevent the retries is to have the server acknowledge them with a 200
round lodge
alright i figured that was the case but was hoping to save on a deploy just to ack these heh. i'll take care of that once the fix if rolled out and we don't see further issues. I appreciate your help!
loud trout
Want to echo what @garebear said above
we are experinceing same type of issue
torpid turret
@round lodge update: we are working to address the retries from our side, though I don't have an ETA on it at the moment
round lodge
awesome thanks!
torpid turret
Yeah, I definitely understand that having to deploy special code for this isn't ideal. Apologies, I misread something so I thought the easiest way forward was that for a bit.
Also I don't think I said it earlier but thanks for the report!
loud trout
we received webhook that referencing non existing subs ..
round lodge
it's been identified and a fix is rolling out now so hopefully no more soonTM 🤞
loud trout
as example : evt_1Qn5KJGNN1T4yZi6SinKQUrZ
no pbm - just giving more info - just in case
torpid turret
Yep, thanks for the info!
loud trout
start around 3 pm eastern time zone