dry violet
We deployed a LLaVA-v1.6-34B model on 2xA100SXM infra as a serverless endpoint. When we send a request, we don't get a response. And the request is indefinitely in the IN_QUEUE status. Any suggestions for what we should we look at to start debugging this?
We've previously been successful deploying LLaVA-v1.5-13b. But again grateful for suggestions
next chasm
Click on the worker to view the logs, something is most likely erroring out.
dry violet
Not seeing anything obviously wrong in the logs
next chasm
You are trying to use an image created for a pod in serverless
For serverless, use this instead: https://github.com/ashleykleynhans/runpod-worker-llava
