silent thunder
I recently setup a gpu server for HA to offload assist to. I've spun up piper and faster whisperer but there seems to be some weird issues.
Mainly PIper shows connected, its selected in my pipeline but HA never sends the requests and runs TTS locally even though the local Piper is disabled.
Another small but insignificant issue is Whisper shows unknown in the gui but I can see the requests in the logs.
Any idea whats going on?
silent thunder
Anyone?
west sleet
You have not provided enough information about your configuration and installation method.
Using HAOS, you are unlikely to be able to use a GPU.
If you are running piper and whisper on a separate machine, you must ensure that all components for using a GPU are installed.
silent thunder
My post states its running on a sperate server. What other information do you need? The pipeline is setup per the docs and I get no error logs.
west sleet
Your GPU, piper installation method, and piper version—please state this first.
silent thunder
P100 and A4000 (tried both) installed via docker on ubutu, piper latest version
strong umbra
Share your compose file.
silent thunder
services:
wyoming-piper:
container_name: piper
image: rhasspy/wyoming-piper:latest
ports:
- "0.0.0.0:10200:10200"
volumes:
- "./piper-data:/data"
command: ["--voice", "en_US-lessac-medium"]
environment:
- NVIDIA_VISIBLE_DEVICES=0
restart: unless-stopped
faster-whisper:
image: lscr.io/linuxserver/faster-whisper:gpu-legacy
container_name: faster-whisper
runtime: nvidia
environment:
- PUID=1000
- PGID=1000
- TZ=Etc/UTC
- WHISPER_MODEL=large-v3
- WHISPER_DEVICE=cuda
- NVIDIA_VISIBLE_DEVICES=0
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- BIND_ADDRESS=0.0.0.0
deploy:
resources:
reservations:
devices:
- driver: nvidia
capabilities: [gpu]
device_ids: ["0"]
volumes:
- ./faster-whisper/data:/config
ports:
- "10300:10300"
restart: unless-stopped
@strong umbra
strong umbra
Sorry, looked at the wrong place. There's a lot of invalid or needless things in your file. Mine works with this
services:
whisper:
container_name: whisper
image: lscr.io/linuxserver/faster-whisper:gpu
volumes:
- ./whisper-data:/config
environment:
- PUID=1000
- PGID=1000
# # https://github.com/home-assistant/addons/blob/master/whisper/DOCS.md#option-model
- WHISPER_MODEL=large-v2
- WHISPER_BEAM=13
- WHISPER_LANG=...
- TZ=...
restart: unless-stopped
ports:
- 10301:10300
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
What does this look like?
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
| NVIDIA-SMI 535.274.02 Driver Version: 535.274.02 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Tesla P100-PCIE-16GB Off | 00000000:01:00.0 Off | 0 |
| N/A 47C P0 34W / 250W | 6254MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA RTX A4000 Off | 00000000:02:00.0 Off | Off |
|100% 40C P8 15W / 140W | 1MiB / 16376MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
Whisper works just fine. Its just piper
strong umbra
Oh. Right. Sorry again. The piper image has a --use-cuda flag you might be able to overwrite via command: https://github.com/rhasspy/wyoming-piper/commit/a9bedf7947b6813807caa9eba22c745cad68e5c1
I stopped using my GPU for it because it really gave no measurable benefit.
I think I used this: https://github.com/slackr31337/wyoming-piper-gpu
This has that flag already set: https://github.com/slackr31337/wyoming-piper-gpu/blob/main/run.sh#L27
silent thunder
I mainly just want it to stop using HA's CPU. I dont much care if it does use the CPU on the AI server lol. But it doesnt seem to even be sending anyhting to the remote server
HA shows it connected, but docker logs for piper are blank and my CPU spikes to 100% when it starts reading text to me
@strong umbra
strong umbra
Unfortunately I can't tell you much about that part.