Download Mixtral from HuggingFace | Runpod | Page 1

hazy plover Apr 10, 2024, 8:25 PM

#

How can I download this model in my pod ?

karmic zenithBOT Apr 10, 2024, 8:25 PM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

solar walrus Apr 10, 2024, 8:43 PM

#

If you use a template with oobabooga installed, it can download it for you.

hazy plover Apr 10, 2024, 8:56 PM

#

Yeah but I want to run it with ollama via a modelfile

safe cedar Apr 11, 2024, 3:41 PM

#

U can run ollama normally

#

run the installation script for linux and the start ollama server

#

and will work fine

#

use a pytorch template as a starting point to launch a pod

#

and then just use the terminal

#

thro the jupyter notebook server

hazy plover Apr 11, 2024, 4:08 PM

#

But what do I need to write on the model file ?
Normally it's FROM XXX but here there is multiple files

safe cedar Apr 11, 2024, 4:09 PM

#

No clue, i dont use custom models, but im sure there are yt videos on it / docs on it

#

just giving a complete shot in the dark - i havent watched this

YouTube

Matt Williams

Adding Custom Models to Ollama

I bet you have always wanted to have an emoji model. Even if you haven't, this video will show you how to make your own Ollama models.

Here is the docker command I mentioned.
docker run --rm -v .:/model ollama/quantize -q q4_0 /model

▶ Play video

#

but i see other similar videos

#Download Mixtral from HuggingFace