#Distributed Job Scheduling

6 messages · Page 1 of 1 (latest)

somber aspen
#

Hey guys, what do you suggest for distributed job scheduling? right now everything is under a single repository. we plan to convert our back-end to mono-repo with microservices architecture but that is not milestoned yet. so, I found out that I can use miaowing/nest-schedule module and implement distributed locking on it using Redis, however I wanna know my options. for example, Bull looks promising as well but It will probably introduce complexity. anyway, I am waiting for your suggestions. thank you

nova flicker
#

for crons/timers I use @nest/schedule with redis distributed locking

for jobs I use bull (i.e. delayed, priority jobs, and ratelimit on throughput)

and for jobs where retrying and message delivery is critical, I use kafka/rabbit

#

I don't think there's a true answer for everyone's problems, it totally depends on the nature of the applications :)

slate kite
#

Why not use plain old crons that would launch your js scripts / a docker container ?

I use Kubernetes crons which have a retry mechanism, and when its critical I also use rabbit

nova flicker
#

i used to use upstash to hit an endpoint on a cron... but just switched to redis locks + cron in the apps

#

havent used kubernetes crons tbh