Does GPU can increase Piper/Whisper (Assist) overall performances of Voice PE ?

[stoic wave]

Did a quick research but cannot find anything directly related to Voice PE.
I am wondering whether or not a GPU could increase performance ?

[crimson dew]

whisper set up with gpu acceleration for makes a big difference to STT speed.

[stoic wave]

Is there a guide on how to set up whisper with GPU ?

[crimson dew]

i have had some luck with the linuxserver container running on my server that has the gpu in it

heres a cat of my docker-compose i use for it

version: '3'

services:
  faster-whisper:
    image: lscr.io/linuxserver/faster-whisper:gpu
    container_name: faster-whisper
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      #- WHISPER_MODEL=tiny-int8
      #- WHISPER_MODEL=base.en
      #- WHISPER_MODEL=medium-int8
      - WHISPER_MODEL=cstr/whisper-large-v3-turbo-int8_float32
      - WHISPER_BEAM=5 #optional
      - WHISPER_LANG=en #optional
    volumes:
      - ./config/:/config
    ports:
      - 10300:10300
    restart: unless-stopped
    devices:
      - /dev/dri:/dev/dri
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

can adjust as needed but its a starting point

[stoic wave]

I am running HA in a VM running HAOS.
I can easily allocate the GPU to the VM but wondering if there are any particular parameters that I should be aware of for whisper to use it... or does it work "out of the box" once the device is made available ?

[torpid carbon]

Note that version is deprecated and devices should not be needed in your case.

[crimson dew]

yeah, some of my container stuff is old. i realise i should update it but the whole...
"if it works don't touch it" applys in this case 🙂

[crimson dew]

i dont think the haos addon for whisper supports gpu acceleration (I am not sure though) so I think you would need to run the whisper container on your host OS or another VM.

[torpid carbon]

Sometimes but this can have unintended consequences and docker compose complains about the deprecation on every invocation.

[crimson dew]

that system may or may not still be using docker-compose and whilst I am not entirely happy about it... it probably wont get "fixed" until I fix a lot of other stuff and rework that server entirely 😛 whilst i keep alot of stuff up to date theres a few things that i havent really touched...

[stoic wave]

had quite the same setup but i recently moved to Incus for the awesome API, VM and OCI capabilities 🙂
And decided to give a chance at HAOS at the same time since it seems to be the prefered way and the most supported way of doing things HA side

Speaking of containes, can someone point me the path where the Docker Containers configuration file are ?
How is it manage ?

[torpid carbon]

What files? You can check the supervisor source code. Are you sure HAOS is the right choice?