Hi All, I am wondering if there is way to utilize the integrated Mac M1 / M1 Pro / M2 /M2 pro GPUs for running Wyoming-whisper and Wyoming-piper ?? This is albeit similar to running Wyoming whisper on a external
Server with CUDA / GPUs support in case did a Nvidia GPU. But can we use Mac’s integrated GPU instead if used an apple silicon hardware ?
#Run Wyoming-Whisper and Wyoming-Piper on Mac M1 GPU
exotic silo
tired pivot
The question is simple: is Docker on Mac able to pass through GPU to the containers.
exotic silo
Well the answer is definitely no as docker on Mac doesn’t pass through GPU as of yet ! I am saying if we install Wyoming-whisper locally as per instructions at https://github.com/rhasspy/wyoming-faster-whisper , will then use integrated GPUs on Mac ?
maiden kelp
I think even if your docker runtime was passing through the GPU, I don't think wyoming faster whisper has any libraries or support for Metal on Mac
it technically doesn't even have support for cuda completely yet, you have to do some container manipulation to get that working
exotic silo
Well PyTorch has MPS that basically enables GPU support on Mac. So container is out of the question, since again docker container on Mac is based on Linux that doesn’t support MPS ! So trying to find out if Wyoming-whisper can be run natively on Mac with GPU support ! And yes whisper is based on PyTorch
maiden kelp
ah, you mean installing it outside of docker. I'd say best way to find out is to try 😁
celest dagger
Well PyTorch has MPS that basically enables GPU support on Mac. So container is ...
https://github.com/raelix/wyoming-faster-whisper
wip or just for their own use, mlx-whisper, but it works, just need to change the hardcode language in handler.py handle_event
warped kelp
Late to the party, but I just wrote a guide to run Whisper on a Mac M1 natively, without Docker: https://community.home-assistant.io/t/remote-voice-assist-pipeline-whisper/841968
mystic panther
on a mac, i'm guessing cuda doesn't matter. Have an nvidia gpu and i'd like to run whisper on my server w/ gpu acceleration, while keeping home assistant separate
quick crest
On my Mac Mini M4 i use whisper.cpp using the Metal support. I have instant Responses. https://github.com/ggml-org/whisper.cpp?tab=readme-ov-file#core-ml-support
„On Apple Silicon devices, the Encoder inference can be executed on the Apple Neural Engine (ANE) via Core ML. This can result in significant speed-up - more than x3 faster compared with CPU-only execution.“
Edit: I use the large-3-turbo model
quick crest
Forgot to say with whisper.cpp you will need that to run as well. Whisper.cpp do not have Wyoming compatibility out of the box https://github.com/ser/wyoming-whisper-api-client
