We are sending jobs off to our whisper serverless functions and experiencing massive delay times sometimes and sometimes it just goes through quickly? At the moment we are just testing so we are using a single 16GB GPU? Has anyone got any advice on this?
#is anyone experiencing a massive delay time when sending jobs to GPUs on serverless?
weak sorrel
neat rapids
Don't use 1 single GPU, its probably throttled. Leave max workers alone, don't change it from the default of 3.
jolly lance
We are sending jobs off to our whisper serverless functions and experiencing mas...
Flashboot on? Are you using the default template?
neat rapids
Problem is always going to be 1 single worker, people keep doing that then complaining about delay times, when they MUST NOT set max workers to 1 🤦♂️
Its set to 3 by default FOR A REASON, DON'T CHANGE IT.
I think RunPod should just disallow setting it below 3, because people keep making this mistake time and again.