HASSOS load 30 | Home Assistant | Page 1

ebon garnet May 21, 2023, 7:36 PM

#

top - 19:30:19 up 2 days, 26 min, 0 users, load average: 26.23, 27.41, 24.42
Tasks: 295 total, 3 running, 291 sleeping, 0 stopped, 1 zombie
%Cpu0 : 0.7/8.1 9[||||| ] %Cpu1 : 1.3/4.7 6[|||| ]
%Cpu2 : 21.3/5.3 27[||||||||||||||||||| ] %Cpu3 : 42.1/3.3 45[||||||||||||||||||||||||||||||| ]
%Cpu4 : 0.0/100.0 100[||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||] %Cpu5 : 32.0/2.7 35[|||||||||||||||||||||||| ]
GiB Mem : 90.8/3.7 [ ]
GiB Swap:100.0/1.2 [ ]

PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND

487322 root 20 0 3868.0m 2.2g 107.9 59.4 169:04.31 R python3 -m homeassistant --config /config
928435 root 20 0 2317.9m 531.4m 100.0 14.0 0:14.93 R python3 -m wyoming_faster_whisper --uri tcp://0.0.0.0:10300 --model small --beam-size 3 --language nl --data-dir /data --download-+
64 root 20 0 0.0m 0.0m 1.3 0.0 275:47.34 S [kswapd0]
389262 root 20 0 374.1m 88.4m 0.7 2.3 50:55.47 S node server/bin/www
1 root 20 0 158.3m 1.8m 0.0 0.0 31:33.84 S /sbin/init

#

This was a couple of minutes ago.

#

Some time before that, I had a much longer list of processes, but I cut off before the headers line...

#

    489 root      20   0 1671.0m  16.6m  61.4   0.4 118:24.82 S /usr/bin/containerd                                                                                                        
    529 root      20   0 2121.2m  27.0m  52.3   0.7 139:15.33 S /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock                                                    
   2358 root      20   0  703.7m   6.0m  23.4   0.2  55:08.54 D coredns -conf /etc/corefile                                                                                                
 926111 root      20   0   25.8m   0.6m  14.9   0.0   0:10.72 D /usr/lib/systemd/systemd-journald                                                                                          
    455 root      20   0 1094.2m   1.6m  11.6   0.0  38:10.70 D /usr/sbin/NetworkManager --no-daemon                                                                                      
 837251 root      20   0    5.9m   1.7m  10.6   0.0   3:14.31 R top                                                                                                                        
 486594 root       0 -20    0.0m   0.0m   8.8   0.0   0:47.45 I [kworker/0:0H-kblockd]                                                                                                    
   3282 root      20   0 1588.5m   5.2m   8.5   0.1  67:38.89 S mosquitto -c /etc/mosquitto/mosquitto.conf                                                                                
 487229 root      20   0 1497.9m   7.7m   6.7   0.2   0:21.48 S /usr/bin/containerd-shim-runc-v2 -namespace moby -id d4d1dbf016c44e0e6bd096b45cfa9119e6510be2215df98a2b694e114f7c3ed8 -ad+
 389262 root      20   0  374.4m  81.5m   6.4   2.2  49:46.64 D node server/bin/www                ```

#

 487150 root      20   0 3116.3m 537.5m   4.9  14.2   2:22.39 D python3 -m wyoming_faster_whisper --uri tcp://0.0.0.0:10300 --model small --beam-size 3 --language nl --data-dir /data --+
   5496 root      20   0  324.2m  11.2m   4.0   0.3  12:39.53 S /usr/bin/python3 /usr/local/bin/esphome dashboard /config/esphome --socket /var/run/esphome.sock --ha-addon                
 487322 root      20   0 2879.5m   1.8g   4.0  49.4 160:34.52 S python3 -m homeassistant --config /config                                                                                  
   5289 root      20   0   96.9m   3.9m   3.3   0.1   4:39.49 D python3 -m wyoming_piper --piper /usr/share/piper/piper --uri tcp://0.0.0.0:10200 --length-scale 1 --noise-scale 0.667 --+
 486633 root       0 -20    0.0m   0.0m   3.3   0.0   0:08.37 I [kworker/3:2H-kblockd]                                                                                                    
 384906 root      20   0  200.7m  10.5m   3.0   0.3   6:50.42 D /usr/bin/python3 /usr/local/bin/esphome run huiskamer-ir.yaml --device OTA                                                
    453 systemd+  20   0   13.1m   0.8m   2.7   0.0  21:54.01 S /usr/lib/systemd/systemd-resolved                                                                                          
   2927 root       9 -11   92.8m   0.5m   2.7   0.0   8:26.50 S pulseaudio --system --disallow-exit --exit-idle-time=-1 --disable-shm                                                      
 926192 root      20   0    2.8m   0.1m   1.5   0.0   0:00.82 D nc -w 1 localhost 10200                                                                                                    
 926193 root      20   0    4.7m   0.1m   1.5   0.0   0:01.04 D grep -q faster-whisper                       ```

#

...now the load has dropped to 1.3, but (if the pattern from the last few days remains...) it will oscillate up and down a few times again, and finally settle after midnight (local time), so in about 2 hours time.

buoyant zinc May 21, 2023, 8:00 PM

#

Please paste this to a paste site or something for better/cohesive formatting and maybe order by RES. Also try docker stats or glances

ebon garnet May 21, 2023, 8:45 PM

#

Ok, I will. I typed in docker stats and will report when it gets through 😦

ebon garnet May 22, 2023, 6:12 PM

#

https://pastebin.com/HvEV3TxK

#

https://pastebin.com/WkRMyvjn

#

https://pastebin.com/EnxnBaf1

buoyant zinc May 22, 2023, 6:28 PM

#

Huh. Why is docker stats so empty? The memory usage of homeassistant itself seems quite high. My docker install averages about 471 MiB/24h.
The profiler might be useful to figure out why: https://www.home-assistant.io/integrations/profiler/. Not sure how much is normal for whisper as I don't use it.

ebon garnet May 22, 2023, 6:37 PM

#

I think docker stats is so empty because it doesn't have enough CPU time to run any containers...

#

I think I'm going to kick out whisper for now, to see if that stabilises the system...

#

...once I can get back in!

buoyant zinc May 22, 2023, 6:39 PM

#

docker stats --no-stream might help for that

#

It's probably thrashing and you might have to wait forever

ebon garnet May 22, 2023, 6:42 PM

#

https://pastebin.com/0fuqgazF

buoyant zinc May 22, 2023, 6:47 PM

#

Yeah probably a good idea to docker stop addon_core_whisper for now. Not sure what HA is doing with all that memory. Maybe it'll decrease once whisper is stopped

ebon garnet May 23, 2023, 4:40 PM

#

I've pushed the Stop button in the addon, that'll do the same thing, I think... Now I'll just watch to see what it does tonight! It's faily predictable when it starts. I thought it had to do with, somehow, whisper having to go through a whole day of audio data (because of user error on my side)...?

#HASSOS load 30