Api Function queue

[winter kiln]

Hey guys, so I have an issue with a webhook from an external service, the problem is that sometimes I get duplicated events that hit the endpoint at the exact same time, causing the updates that I need to be duplicated.

One possible fix I thought of is to create a queue, and do a first in first out (FIFO).

The problem is that when I deploy it to vercel it doesnt work, the approach I took is to create a variable that will hold the array with requests:

const queue: { req: NextApiRequest; res: NextApiResponse }[] = []
let isProcessing = false

export async function handleApiFunction (req: NextApiRequest, res: NextApiResponse) {
  queue.push({ req, res })

  if (!isProcessing) {
    console.log({ isProcessing })
    await processQueue()
  }
}

And for some reason this locally seemed to work, but not when deployed to Vercel.

The reason I suppose is that each call of that function is "stateless" in a way, and consecutive calls dont really have the same queue array.

The question then becomes, how can I solve that problem within the realm of Nextjs/Vercel, or do I have to look for other solutions?

[hasty folio]

Post created!

🔎 This post has been indexed in our web forum and will be seen by search engines so other users can find it outside Discord

      🕵️ Your user profile is private by default and won't be visible to users outside Discord, if you want to be visible in the web forum you can add the "Public Profile" role in <id:customize>

      ✅ You can mark a message as the answer for your post with `Right click -> Apps -> Mark Solution`
      (if you don't see the option, try refreshing Discord with Ctrl + R)

[river cradle]

@winter kiln yes. as you said, it is possible that duplicated events can be handled by different instances, i.e. AWS lambda Node.js runtimes, which Vercel platform employs behind the scenes, especially heavy traffic situation. Obviously only one runtime when getting it up and running locally on ur machine.

So to address ur issue, duplicate events, there are two options.

1. make the api idempotent with event_id

2. use Vercel KV, Redis, with rdb.SetNX(event_id), which is a kinda exclusive lock. I faced the similar issue with GCP Pubsub, emit duplicate events, so i implemented it in Golang. I can send it to you if you want.

[winter kiln]

I was going to use kv but the problem is kv is not available to enterprise clients, and the project is under enterprise account, so then the idea becomes using a db to store the queue there or completely move away from nextjs for that feature and do a normal backe d