Ollama integration

[main elbow]

Hello, im new to home assistant and i just set it up on an old laptop. Im trying to integrate ollama to my home assistant’s voice assistant feature but when i make a prompt it doesn’t respond, just displaying the … bubble indicating that it’s writing. When i make the same prompt from the command prompt it responds immediately. How can this be fixed?

I followed the tutorial from network chuck
https://youtu.be/XvbVePuP7NY?si=jRzvsCyllgiU-22D

[drifting ridge]

Install FasterWhisper, OLLAMA, and Piper using docker-compose.yml

services:
  faster-whisper:
    image: lscr.io/linuxserver/faster-whisper:latest
    container_name: faster-whisper
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Chicago
      - WHISPER_MODEL=tiny-int8
    volumes:
      - //opt/appdata/faster-whisper/data:/config
    ports:
      - 10300:10300
    restart: unless-stopped
  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    devices:
      - /dev/dri
      - /dev/kfd
    environment:
      - HSA_OVERRIDE_GFX_VERSION=11.0.0
      - OLLAMA_DEBUG=0
      - OLLAMA_FLASH_ATTENTION=1
      - OLLAMA_MAX_LOADED_MODELS=2
      - OLLAMA_MAX_QUEUE=256
      - OLLAMA_NUM_PARALLEL=1
    ports:
      - 11434:11434
    volumes:
      - /opt/appdata/ollama:/root/.ollama
    restart: always
  piper:
    image: lscr.io/linuxserver/piper:latest
    container_name: piper
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Chicago
      - PIPER_VOICE=en_US-lessac-medium
    volumes:
      - /opt/appdata/piper/data:/config
    ports:
      - 10200:10200
    restart: unless-stopped

Allow the system running HomeAssistant access to those services. Eg. 172.16.40.24:10200,10300, & 11434

In HomeAssistant navigate to, Submenu, Settings > Devices & Integrations.
Click the Add Integration, search for Ollama.
Eg:
Host: 172.16.40.24
Model: llama3.2
Next and Finish/Submit.
Click the Add Integration, search for Wyoming Satellite and add those services from above.
Eg:
Host: 172.16.40.24
Piper, 10200
FasterWhisper, 10300
Next and Finish/Submit.

Now, navigate back to, Settings > Voice Assistants.
Set the new integrations from the dropdowns and change voice to lessac-medium followed by clicking Update

Now try going to Submenu > Overview and click on Assistant.
To have Voice input you'll need https

[main elbow]

Oh thanks

[drifting ridge]

That OLLAMA is using an AMD GPU, You may need to do latest for Intel or have NVidia tools installed on the host system to get OLLAMA to work with your GPU.

[main elbow]

Yes its an amd integrated gpu

A radeon smth

[drifting ridge]

Then that should work

[main elbow]

Ok ok

Thanks so much

[drifting ridge]

You may need to change the - HSA_OVERRIDE_GFX_VERSION=11.0.0

[main elbow]

I have whisper already installed should I delete that and replace it with faster whisper

[drifting ridge]

Depends on your setup, did you use the HA Addon for whisper?

Also I personally would.

[main elbow]

Yes those are all my plugins

[drifting ridge]

If you do it via docker, I'd say remove them all and just add the integration but you could try to go into Settings > Voice Assistants > HomeAssistant and see if piper, and whisper are set?

[main elbow]

They are all set

[drifting ridge]

Hmmm. does the logbook say anyhting?

[drifting ridge]

Do Update then go to the popout menu and say Start a conversation

How much data do you have on your HA?

[main elbow]

Its been thinking for 4 minutes

At this point i can look outside

[main elbow]

You mean how much stuff is connected to it?

Its 2 wiz lamps a Minecraft server and a pihole server

[drifting ridge]

Yeah, I'm having issues getting the weather to connect to mine... I have exposed it but it doesn't seem to do it?

Do you have Docker installed?

You might just try doing the docker compose up -d on that docker-compose.yml and then remove the current addons then setup as above ^^

[main elbow]

No i have it on a vm to try it out

[drifting ridge]

Ah ok

[main elbow]

Doesn’t do anything with lights ether

[drifting ridge]

This is my complete configuration for HA:

I have a .env with C11_TUNNEL_TOKEN I got from Cloudflare Zero Trust so that I can have easy HTTPS.
Then here is the docker-compose.yml for HA:

services:
  c11-cloudflared:
    image: cloudflare/cloudflared:latest
    command: tunnel --no-autoupdate run
    container_name: c11-cloudflared
    depends_on:
      - homeassistant
    environment:
      - TUNNEL_TOKEN=${C11_TUNNEL_TOKEN}
    restart: always
  homeassistant:
    image: lscr.io/linuxserver/homeassistant:latest
    container_name: homeassistant
    devices:
      - /run/dbus:/run/dbus:ro
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Chicago
    network_mode: host
    privileged: true
    volumes:
      - /opt/appdata/homeassistant/data:/config
    restart: unless-stopped

[main elbow]

Ok

I will run it on docker

Im just waiting for an ethernet cable to come

[drifting ridge]

Seems legit

Also if you are doing CF Tunnels don't forget to add the trusted proxy:

# /opt/appdata/homeassistant/data/configuration.yml
# allow reverse proxy for cloudflared
http:
  use_x_forwarded_for: true
  trusted_proxies:
    - 172.18.0.2/32

[main elbow]

Whats a cf tunnel

[drifting ridge]

I was beating my head against a wall on why when I point has.example.tld to http 127.0.0.1:8123
It has giving me a 400 error when hit from https://has.example.tld
It shows in the homeassistant.log in /opt/appdata/homeassistant/data/homeassistant.log that this IP was trying to connect. 172.18.0.2 so I added it wit hthe above, and everything started working!

[main elbow]

Nice 👍

[drifting ridge]

Cloudflare Tunnels is a free service from Cloudflare a global CDN, & Security provider.
They allow you to no have to open ports through your firewall to access something like HA from outside the network with a domain. If you need a domain you can get a free one from freenom but it only lasts for one year. Not sure what happens after that lol

[main elbow]

Ye ik cloudflare

Oh so its like a port forward?

I already own papasot.net

[drifting ridge]

Why I like them is that it's free, it makes things secure, and your mostly protected by the CF network.

• Added benefit, the voice stuff works perfect from the domain if any browser!
  Free HTTPS 😛

Also if you want to test the TTS and STT you can download the companion app and still have it work with the mic but just not outside the network unless you have a VPN server setup

For clarification, You can use the Voice stuff from the companion app with no problems inside your home network but once you leave your wifi, network you will have issues.
Hooking it up with your domain would be easy and it would allow mic access from your main PC web browser

Sorry for the fanboy and ramble lol

And I just got this working today as well so I was like OOOH I can try to help! lol

[main elbow]

Ye thanks a lot

I really appreciate it

[main elbow]

Iv been using Playit.gg to access some services from outside

[drifting ridge]

Ah, nice! how does it work for you?
Also I was considering Talescale but I may have to check out PlayIt, what services can you run through it?

[main elbow]

Ye iv been using it for a ling time for my mc server and for dns

Long

[drifting ridge]

Is it free? Also does it offer like a proxy connection or like a dynamic DNS address?

[main elbow]

Its free for up to 3 server with 3 connections each

So you can use one account on 3 computers with 3 tcp or udp connections each

[drifting ridge]

Of that is BA!

Sorry about basically having you tear down like your whole setup 😦 I just knew what worked for me lol

[main elbow]

Its ok