runic spire
I try it with my existing fastapi application and found the worker is unable to go back from running -> idle state. My application has asyncio processes running in the background. Is that the reason?
Also is the new loadbalancing endpoint support socket-based API?
thorny harnessBOT
rose dagger
I try it with my existing fastapi application and found the worker is unable to ...
Maybe because of still running process, or the request isn't returned?
im not sure with the socket-based support, maybe you want to ask staff via support
modest ivy
sockets should not work, i say should not because we use a proxy to forward the request, SSE will work, but doubt sockets will
idle > running is determined by the health check, read into that, make sure the port is exposed properly in your container
rose dagger
idle > running is determined by the health check, read into that, make sure the ...
i think they're asking about running back to idle state* like when the job's done
modest ivy
running > idle is whenever the request is done and based on your idle timeout and min workers, e.g. if you have min worker to 1 then at least 1 worker will never idle
rose dagger
just curious, so if a function that handles the request starts a subprocess, it will be killed after returning the response + idle timeout right?
modest ivy
the serverless control plane doesn't care what's running in the worker, all it cares about for load balancing is that a request is sent to worker using http proxy and its handling it, once the request is done, it knows that no other requests are pending, then it follows idle timeout + min workers logic
so if you create a httpserver that takes a requests and runs a background job but returns the request and completes it, well the serverless doesn't not know of any background jobs, it considers it complete and then will stop the worker
long running requests beyond 10 mins are not a good use case for load balancer
rose dagger
alright thanks for the explanation!, and i just read the docs, it says that lb limits to 5.5~ mins only
modest ivy
oh yes its that, we can increase it, 10 mins is likely the max we can get to in future
we may figure out a better pattern for long running jobs in future but queue serverless is the best solution for it
runic spire
thanks @rose dagger @modest ivy well noted. I might experiment with a simple socket app to test.