Anyone here has tried using Mojo with local Whisper from OpenAI? | Modular | Page 1

mellow bloom Jul 16, 2024, 4:50 PM

#

Hi! I'm trying to use Whisper from OpenAI locally, but python in general is pretty slow. Anyone has tried running any kind of Whisper from OpenAI locally?

summer nicheBOT Jul 16, 2024, 4:50 PM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

paper ember Jul 16, 2024, 5:24 PM

#

You might want to try https://github.com/ggerganov/whisper.cpp , runs very well on my Mac (it uses Metal) I thought of looking into porting it to Mojo but without GPU support it won't lead to anything impressive I am afraid...

GitHub

GitHub - ggerganov/whisper.cpp: Port of OpenAI's Whisper model in C...

Port of OpenAI's Whisper model in C/C++. Contribute to ggerganov/whisper.cpp development by creating an account on GitHub.

#

Do you have a Nvidia GPU device or Apple Silicon? If not I am afraid you need to use the smaller models to have acceptable run times.

mellow bloom Jul 16, 2024, 8:29 PM

#

I've a small Nvidia GPU (Geforce GTX 1650). When you say "without GPU support", you mean Mojo doesn't support GPU? (I'm novice at understanding the the hardware/computing relationships with machine learning models)

#

@paper ember it seems I should be using MAX instead of MOJO?

#Anyone here has tried using Mojo with local Whisper from OpenAI?