mint raptor
So, I'm wanting to optimise the storage of my data - and keep energy generation stats forever.
I've got the following entity filters in the config to push my data to influxdb:
include:
entity_globs:
- sensor.fronius*
- sensor.smart_meter_63*
- sensor.solcast_pv*
I can see these entities within InfluxDB, but now I can't select any of these to be used within the Energy dashboard.
How would I go about doing this?
left trail
for an entity to be selectable in the Energy Dasboard it has to meet the following
recorder (so the data can be transferred to the built in LTS tables)unit_of_measurement (eg kWh)device_class (either energy, gas or water)state_class (usually total_increasing or total)Not sure how this is related to InluxDB though, as the Energy Dashbaord uses the data from it's own Long Term Statistics database
mint raptor
Wait, it isn't in the MySQL DB anyway?
left trail
By default HA uses an sqlite database. How did you set up the recorder integration?
mint raptor
recorder:
db_url: !secret mysql_url
purge_keep_days: 14
commit_interval: 60
include:
entity_globs:
- sensor.uptime*
And installed MariaDB as an addon.
I'm re-thinking my logging / recording - as the DB was well over 4Gb in MariaDB - and 99% of the stuff stored, I really don't care about.
so I want to start with doing an 'exclude by default' type arrangement.
the idea being that I can purge all the things, but keep long term data in influxdb - which is apparently better for time series stuff over the long haul.
left trail
If you only have an include, that will be the only entity for which it stores data
mint raptor
yeah - but the other includes are in the influxdb stuff..... or is it just that HA cannot reference influxdb for energy values?
left trail
No, it only takes the data from it's own database
mint raptor
ie I have to keep them forever in the mariadb ... DB...?
left trail
yes
mint raptor
that's kind of annoying 😐
left trail
but it will covert them to Long Term Statistics, so only the delta per hour
it will not store all state changes indefinately
mint raptor
ohhhh - so it won't keep every reading.
left trail
no, only for the first 14 days (as per your purge_keep_days setting)
mint raptor
it is possible that before, my setup was logging just about everything..... and I really don't care for that if I'm honest....
So, the long term statistics, will they also be purged after 14 days?
mint raptor
but that's ONLY stuff thats in the energy dashboard? Or is there some kind of selector as to what gets LTS and what doesn't?
left trail
it is possible that before, my setup was logging just about everything..... and ...
if you didn't have any includes or excludes it would log everythign
left trail
but that's ONLY stuff thats in the energy dashboard? Or is there some kind of se...
If the sensor is numeric, and has a state_class (either measurement, total or total_increasing) and is included in the recorder, it will be stored in LTS
you could add the same entity_globs you have for InfluxDB to recorder to be able to select those entities
mint raptor
gotcha - that makes sense.
I wasn't even aware there was an LTS setup - which is pretty much what I was trying to replicate....
Good to know now I was going down the wrong path
left trail
more info here, from when it was introduced https://www.home-assistant.io/blog/2021/08/04/release-20218/#long-term-statistics
and here for the technical stuff
mint raptor
ok, cool - thanks..... that gives me something to go off.
hopefully some agressing pruning of data and only including what I want to keep will make the DB a bit more managable...
I was also under the impression that the power generation stuff would just disappear after 14 days - so good to know that isn't the case too