I got a Home Assistant Voice PE a few days ago and initial setup worked fine. I played around a bit with Ollama on a VM and setting up an assistant with that. In the Ollama process I had to remove all exposed entities for it to even return a response, but that took 30 seconds for a simple prompt, so ultimately that proved to be too slow on my NAS without a GPU.
Now I'm trying to put it back to use my default HA assistant with whisper STT / Piper TTS, but the VPE no longer does anything. It responds to the wake word (even when I change it), but then immediately goes idle again, looking at the ESPHome device logbook. After it does that, I have to say the wake word twice for it to play the wake sound, but then immediately goes idle again, so I can't give it any commands.
Any clue on what I could have done to break things? Thanks!