"History Proxy" helper

[ruby sage]

Hi everyone. 🙂

after the discussions on Git issues, Git discussion, the forum and here about the new power dashboard and the problem with house-battery entities providing their values inverted compared to what HA expects,
I remembered about something I used in a couple of other smart home solutions before I switched to HA:

A helper like proxy device, that allows calculations like a template sensor, but doing the same thing for history.
So it doesn't save it's own history, but fetches the history for the devices used in the template and does the same calculation for each value it receives.

This wouldn't only allow to do things like inverting an value. You can also do scaling, translations, calucalate differences or sums of other sensors on the fly.
Benefits:

• No additional history saved that bloats the database
• You can use it on your existing entities and therefore don't loose your existing history
• It's calculated on the backend, so your maybe weak end-devices don't get additional load
• You would be able to use these like any other entity in every (also already existing ones) dashboard card, without the need to implement special code for transforming values over and over again

My question as someone who only wrote custom cards so far, to the community with more experience:
Would it be possible to implement something like this as an custom integration, or is this something that can only be done in Core?
I'm afraid integrations aren't involved in fetching history at all, so proxy can't be created that way. Is this correct?

The only other way I see would be a combination with a custom Recorder integration acting as the proxy, which might also not be possible at all.
Otherwise this has to become a feature request I'm afraid, which might decrease the chances to see this implemented rather soon ...

[shell rapids]

Custom integrations can do this in principal. They are just code in core, so they don't have many restrictions

[ruby sage]

But are they involved in history fetching, not only writing?
Or is HA going to the DB directly for this, so you don't have a chance to place kind of a callback for transforming the values

[shell rapids]

Integrations can't mess with existing entities but they create a new entity that gets its state from an existing one. They can also copy and transform the existing history from the old entity to the new one

[ruby sage]

Ok, thanks. The part about copying and transforming old history over to a new one is already very interesting.
Didn't know that.

But the idea was to not save new history here at all.
The existing entities would stay like they are and kept to write their history.

The "proxy" entity on the other side would not write any own history.
It just fetches the data from other entities when history is requested and calculates the result "on-the-fly".

This was a very popular feature on other smart home solutions, as you can do almost every calculation with your existing history/entities in any charting card without the need for specific features to be implemented there.

So it's a more central approach for data transformation to be used in the UI.

[shell rapids]

Altering history is not as simple as it sounds because long term history is saved as statistics like min/max/mean/change/sum, not a single value.

[ruby sage]

Sure, but as it wouldn't change written history, just alter requested/fetched data on-the-fly before returning, it shouldn't be too complicated.

You have to route the history request the "proxy" received to all "child" entites with the same settings (timespan, raw or aggregation, and in this case also the aggregation type) to get the correct values and then the calculation has to be done with each of these value time points.

Then it has to be returned.

Thanks for your replies.

[fair lagoon]

There is a feature request to select the "polarity" of the power sensors in the energy dashboard.

I don't think HA supports entities without it's own history / proxy history.

[ruby sage]

Yes, that’s also my conclusion now.

Still think this would be a cool feature, looking at the plus points above.

But as it doesn’t seem to be possible, that this can be implemented through a custom integration,
I will post that as feature request.

Maybe it gets some attention at a later time.

But thanks for your replies. 🙂

[charred blade]

@ruby sage What you'd like to do is exactly what template helpers are for. You can use these to take value of source entity and do some calculations / transformation on it which will result in a new entity.
The original sensor will still be there and collect history as before.
The new template sensor you only use for the "on-the-fly" data you just exclude from the recorder and it won't collect history / statistics for it.

When you want to do even more complicated stuff you can create a custom component and utilize the async_track_state_change_event callback which allows you to listen and act on changes of the source sensor. I use this in the custom component I maintain Powercalc as well.

[ruby sage]

And when I turn off history collection by excluding from recorder on the template sensor and use a get-history-call on it, it will calculate these historical values life "on-the-fly" from the history values of the sensors used in the template?
Just to be sure, because this is what this thread is all about.

Because everyone in the HA forums said that you won't have history from before creating the template sensor when e.g. using it to invert an existing sensor in the energy dashboard for battery power.

[paper oasis]

No templates don't generate retroactive history.

What you're looking to do doesn't seem possible, unless you replace the recorder with your own component.