#[Gunicorn] Is there a reason not to use an obscenely large number of threads?

8 messages · Page 1 of 1 (latest)

eternal pike
#

We have this application with a fairly large number of hits, in which some views definitely need optimizing.

From everything I see online, the recommendation is generally to have 2 * num_core + 1 workers, and up to 4 threads per worker... but I really can't find anything that explains why you should limit the number of threads to 4.

Most of the requests are fast enough, but less than 1% does take longer than 5 seconds or so, and during peak use, if multiple users call the slow views, the site generally seems to degrade.

Is it bad to say screw 4, launch 40 threads per worker, while we work on the actual optimizations?

All the settings page says for both workers and threads is:

A positive integer generally in the 2-4 x $(NUM_CORES) range. You’ll want to vary this a bit to find the best for your particular application’s work load.

The views are IO, not CPU bound.

Does anyone have any good references that explain why everyone says to use 4, or is it a "it was recommended once, so now we all repeat it because it's all we've ever known" kinda deal?

true lodge
#

no idea, I'm also curious what that recommendation is based on. I guess there's not much else than try different mixes of workers and threads and check how it turns out. it's scary doing that in production. maybe try to write some test views that simulate different workloads/request-response-times and use something like https://locust.io/ to bombard a local gunicorn and measure request throughput?

I'd love to hear other approaches, haven't done much on this front yet.

frail narwhal
#

Experience shows me that 2n+1 is pretty good baseline

#

Too much can make things slower

heady rivet
#

I wish there was a definitive answer as well. Hell it'd even merit a "why2plus1.com" 😂

hushed mica
#

The 2 * num_core + 1 recommendation, AFAIK, comes from the (probably almost always false) assumption that you want one worker using the CPU and another one waiting for I/O. No idea where the 4 threads recommendation comes from.

#

In my experience so far, a single uvicorn worker can run hundreds of threads for IO-bound tasks with good performance. I still have to test this a bit more though.

eternal pike
#

I think we're some way away from moving to ASGI 😁