Hello. I've been trying to serve ollama on RunPod using ollama Docker image (https://hub.docker.com/r/ollama/ollama) but haven't found a way to run it.
I tried using the docker run ... command in the Container Start Command input but I encountered an error: unknown command "docker" for "ollama". Does anyone know the correct method to use ollama on RunPod?
#How can I use ollama Docker image?
ancient bear
valid sealBOT
bleak rapids
You can't use docker in docker on RunPod. You have to create a template for it.
ancient bear
Hello, thank you for the response. If I'm unable to use ollama as shown in the screenshot, could you please tell me if there's another way to use ollama on RunPod?
bleak rapids
Looks fine, but you may need to add
bash -c 'sleep infinity'
As the Container Start Command to ensure that your container keeps running and doesn't keep restarting.
ancient bear
I think I'm missing something here. I thought the template is for pulling and running a Docker image. If that's the case, shouldn't I need to add the run commands from the ollama documentation (docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama) into the Container Start Command input?
bleak rapids
I think I'm missing something here. I thought the template is for pulling and ru...
No, you can't run docker in docker that won't work
ancient bear
Aha now I get that the Docker container is already running in this case.. In that case could you please tell me how I can set configurations such as port before it starts running?
bleak rapids
Not familiar with that image sorry.
ancient bear
Okay. Thank you for your help 😀
bleak rapids
Maybe @jaunty topaz has some time to look into it for you