Microservices - guarantee one after the other processing?

[odd igloo]

Hi,

We're trying to use the Nest microservices tools to enable IPC. Using Redis as our communication layer we are having an issue in which we emit events, and at the receiving end the events are not guaranteed to be processed on after the other. Essentially, the event processor is not awaiting the completion of an event before moving on to process the next one.

I imagine we could switch to using the message pattern to help with this, but that also isn't really the solution for our use case. The producer of these messages doesn't care about the response, just that it was successfully added to the queue.

BullMQ worked well for this but since BullMQ is only developed for js we wanted to use something more generic like Kafka so we could process and send messages across different languages.

The Nest microservices docs aren't super clear so I'm not really sure where to go from here. Does anyone have any thoughts?

[spring bay]

Events (emit + eventpattern) are by definition fire and forget. If you need to wait for response, then you need a synchronous handling (send + messagepattern)

You say you don't care about the response, but you do - "task complete" is also a response.

[orchid hazel]

Hi, i have some problems about NATS, can you help me ?

[spring bay]

I've never used NATS, but don't ask to ask.. just ask

[orchid hazel]

i have 2 services:

• service A (nestjs) is subscriber
• service B (nodejs) is publisher
• service B push a message to service A, service A got data and return value to service B

Problem: Service B cannot get response from service A

[odd igloo]

Thank you for the response. Let me clarify a bit more though.

Prior to playing with the Nest Microservices stuff, we were using BullMQ. The message producer would send a message and then basically forget about it. On the other end, the consumer would process just one at a time -- it wouldnt begin processing task b until task a had either been market failed or complete in the bullmq. So truly, the producer did not care about the response, but the queue did.

Given a bunch of nice things about using the microservices architecture we wanted to switch but still keep the option of that functionality. Basically, just wonder if its possible to mimic that functionality in nest microservices. There are other cases where we truly do want fire and forget events, but thats a different issue.

[spring bay]

Well, the question is whether you need a job queue or message queue, they all have different purposes. If you need to wait for task completion, you need to hold the connection open for the entire duration of the process, which tends to be problematic for broker based message passing. At that point just use direct REST or RPC calls.

I would avoid trying to bend asynchronous message passing into synchronous request-response pattern, even though the Nest docs suggest it's easy, I would call it an anti pattern

[odd igloo]

right, that makes sense

do you think i should just stick with BullMQ for these tasks then? Can use the nest microservices event stuff just where its needed

as far as i can understand the differences between task queues and message queues, we want a message queue

[spring bay]

I would say use what forks for you. There's no point in bending your project just for the sake of using the "microserivces" library. Microservices can be just a bunch of services communicating over REST.

You're the only ones who know the actual requirements, so I can only tell you what is out there, but the decision is on you.