#Rate limiting API calls across multiple tasks?

1 messages · Page 1 of 1 (latest)

valid rune
#

I'm making Gemini API calls from my tasks and need to stay under their rate limit (1M tokens/minute).
Currently if multiple tasks run in parallel, they can exceed the limit and get 429s.
Is there a built-in way to rate limit across tasks? Ideally I want tasks to wait/queue automatically when approaching the limit rather than fail and retry.
This is different from concurrency limits since rate limiting also depends on how fast tasks complete, not just how many are running at once.

proven prairieBOT
#

Hi @valid rune. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

There isn’t a built‑in, global “API rate limiter across tasks” in Trigger.dev today. The docs only cover:

  • Concurrency limits per queue / environment (how many runs execute at once), not request/token rates. [Concurrency]
  • Retry helpers like retry.fetch that can back off on 429s, but they still react after the limit is hit rather than proactively queueing. [Retry helpers]
  • A community rate limiter package that uses Redis to rate limit Trigger.dev tasks. [Rate limiter]

The community rate limiter is the closest match to what you’re asking for: it’s specifically “a rate limiter for Trigger.dev” and is implemented as a separate package using Redis. [Rate limiter] You’d use that to coordinate limits across multiple tasks instead of relying only on queue concurrency.

Beyond that, the knowledge sources don’t describe any first‑party feature that:

  • Tracks external API tokens (like “1M tokens/minute”),
  • And then automatically delays/waits tasks before they hit the limit.

So to get the behavior you want (tasks waiting/queuing instead of failing):

#
  1. Use a shared mechanism (e.g. the community Redis rate limiter) across all tasks that call Gemini, so they all consult the same budget before making a call. [Rate limiter]
  2. Optionally combine that with:
    • Queue concurrency limits to cap parallelism. [Concurrency]
    • retry.fetch with backoff for any remaining 429s you still hit. [Retry helpers]

If you need something beyond what the community rate limiter provides, the docs don’t describe another built‑in solution, so you’d have to implement custom logic around a shared store (e.g. Redis) yourself.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: