Are workflows appropriate for client-initiated AI processing tasks given concurrency limits?

[quasi igloo]

Hi all! I have a question about workflow concurrency limits and whether workflows are the right fit for client-initiated AI processing use cases like mine.

My use case:

• User selects image(s)
• In parallel: Send base64 image for AI processing + upload image to CDN
• Validate AI response based on rules
• Write to database
• Send notification
• (Future step) Additional AI enrichment processing

Why I chose workflows initially:
I implemented this using workflows because I wanted:

• User status updates for each step
• Durable retries
• Clean abstraction

My concern
I'm now questioning this approach due to workflow parallelism limitations. Here's my current implementation for reference.

My question
Should I be chaining actions together instead and only use workflows for batching? Or are workflows actually intended for use cases like mine?

I noticed the docs now include this guidance:

You can specify how many workflows can run in parallel by setting the maxParallelism workpool option. It has a reasonable default. You should not exceed 50 across all your workflows. If you want to do a lot of work in parallel, you should employ batching, where each workflow operates on a batch of work, e.g. scraping a list of links instead of one link per workflow.

Any advice would be appreciated!

[trim spoke]

I too am worried about this - I could see 50 per project (a few users working together) in my application at any given time. Curious if you've found any strategies for this.

[deft kite]

If you're a pro user there are pretty high limits. But are you saying a few users would have 50 simultaneous actions that all need to run at the same time? parallelism is roughly duration * rate, so if each action is 5 seconds and you have 60 actions per minute, you'd have parallelism of 5. If you have users kicking of bursts of 50 things at once, the workpool / workflow help smooth out the load.
I'd start with a cap of 50 and if you start running too hot, there's alternative options, including asking us to bump up your limits (which we'll happily do, but don't want folks to fork bomb themselves and/or get an unexpected bill)

[quasi igloo]

Thanks, I think that makes sense. The missing piece of context from my original question is that my application allows for users to batch AI process images during onboarding, so one user could kick off 20 workflows in parallel, making it more conceivable I might hit the 50 limit, and it's key to my first time user experience they get them back as soon as possible. I understood how the workpool / workflow would spread out the load to make sure they all resolve, but I just wanted to make sure I was using the components as intended and would be able to scale up if I start getting a bunch of users onboarding at once (🙏).

If I'm understanding correctly, sounds like I'm using them as intended and there's a way to scale up if needed