Wyoming Protocol Question

[tame imp]

Is it possible to Multiplex the Wyoming protocol similar to how a web proxy can run in multiple other Web servers?

[unreal flicker]

What are you actually trying to do?

[tame imp]

I run my voice pipeline locally using AMD lemonade-sdk. It wraps many projects like llama.cpp, kokoro, and whisper and acts as a singular point to manage them. I had an idea of adding in wyoming endpoints so you no longer had to have custom components for sst,tts, and intents. I however thought before I trusted an AI answer and to save myself from reading through the source...I would ask if that was possible first.

[unreal flicker]

People have made Wyoming wrappers for various things before so it is likely possible. The Wyoming protocol stuff is in THIS repo mostly but if you look around for Wyoming wrappers you will find some for other stuff and perhaps get some inspirations.

[tame imp]

That’s a much simpler route, but goes against the idea of a one deployment server. I will have to port it to cpp. Only question is if I am running 3 ports or can I multiplex them on one port…

[unreal flicker]

I don't think so, at least not directly... Is your goal to only use a single port or to just have a integration setup show multiple things?

[tame imp]

The service already multiplexes multiple things on a single port. It would be cool to continue to do so. lemonade is different as its mostly concentrated on user experience when it comes to the llm stack. It has ui components as well as other items that make it more user friendly in a single stack. I was trying to stick within those bounds. This is not meant to be simple to create, but simple to use once implemented. It sounds like I would need a port per service...

[unreal flicker]

Whilst wyoming needs 1 port per service to integrate to it. you might be able to make some kind of middleware that takes the connections and then pipes them to the right place over the common single connection to lemonade.

[unreal flicker]

how that middleware would look, I am not sure. perhaps a custom "virtual integration" that depends on wyoming. I am not even sure if that is the right tool for the job.
you could run an app (addon) container on HAOS which is the middleware then exposes connections locally to HA which could autodetect them. but this doesnt feel right for some reason.

[tame imp]

Intuitively I think you are correct. If I wanted to replace 3 custom components with 1 python app that could serve up tts,sst, and intent running a python app using the wyoming libs would be a much easier route. The entire stack is openAI API compatible. I would really love nothing more than a virtual integration that uses the standard HA openai component. But from the github issues around that, it seems it has been rejected multiple times. Lemon-sdk has openai, ollama, and anthropic endpoints for max compatibility. (The ollama component could replace one of the 3 I am using, but its seems to be a bit limited).

[tame imp]

It will not be simple, but if I upstream it...hopefully it will be maintained.

[unreal flicker]

There are some custom integrations that do openai endpoint integration but I don't know much about them.

[unreal flicker]

it will be if you do it 😛