Multi GPU problem

[hexed marsh]

Hi, how can I evenly distribute workers across multiple GPU? I am trying to get the Stable Diffusion model up, however I am getting an out of memory error as gunicorn is trying to run them on one GPU. How can I solve this problem, given that I need to run all the workers on the same port. Either how can I configure proxying requests inside the pod.

[quiet spear]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution