Value of Power (and others) lags behind real SolarEdge power with multiple minutes.

[remco-k]

Hi Addie,

I'm new to Domoticz and i've been using this plugin for a couple of days now.
I am having the same type of issue as @FritsPh described in #21 (comment). Being a software developer (with some Python experience) myself I started to investigate. I had seen and processed my SolarEdge modbus before by myself and that data was spot-on, so I know it must be possible to get better corresponding values as it is now.

First let me describe what I am experiencing, in short: Power is lagging behind the power that my inverter is showing on its display and returning via modbus. Its not the 5 second time interval causing it, because it takes a way longer time for the number to creep up (or down) to the 'real' power delivered at that time: 5 minutes if the real power doesn't change in the meantime.
The same goes for other values like DC Current and DC Voltage and presumably others. Didn't check them all.

Knowing that the modbus data is spot on, I was first turning my attention to Domoticz. Maybe something got cached or averaged out somewhere between your plugin and my screen (being some process in Domoticz). It didn't look like that. So I started to have a look at your plugin.py file.

There is an Average class. That class is being used by almost all (all but 3) items in both of the SINGLE_PHASE_INVERTER and THREE_PHASE_INVERTER lookup tables.

A snippet:

[Unit.POWER_AC,        "Power",             0xF8,  0x01,     0x00,       {},                     "power_ac",        "power_ac_scale",       "{:.2f}",  None,           None,                                  Average() ],

Its the Average() that is to be discussed. I think this is the cause of lagging behind the real numbers. Since the Average class needs a max_samples value, this is set this way:

self.max_samples = 300 / int(Parameters["Mode2"])

and

unit[Column.MATH].set_max_samples(self.max_samples)

Which makes the output of your plugin creep up or down to the current real value in 5 minutes time.

Since the darkness and the night has set in I currently have no way of checking all of the above facts, so many of it comes from the top of my head, but it seems to me the usage of the Average class is the culprit here.
My proposed solution would be: Setting MATH for most (or all?) items in your lookup tables to None, to resolve the issue.

But since I'm new to Domoticz and your code looks well and organized I might be missing out on why you choose to use the Average class in all these items. I'd like to hear from you what that reason would be. A comment in your code gives the reason partly:

# Domoticz shows graphs with intervals of 5 minutes.
# When collecting information from the inverter more frequently than that, then it makes no sense to only show the last value.

But I'm looking at my Domoticz screen and there I want to see the realtime values (Like I see when I look at my P1 meter Power value, that is realtime), not 5 minute average processed ones. Next to that this also causes the Domoticz graph to lag behind as well.
Being a Domoticz newbie, I could say something really stupid here: Isn't calculating averages what Domoticz should be doing by itself to store the values for the graph?

Next to this I have a small feature reqeust: Please add a 1 second interval option:

                <option label="1  second"  value="1" />                

I already added this myself and works like a charm. With the above described issue, that is. :)

Regards, Remco.

[addiejanssen]

Hi Remco,

You are right; most values returned by the inverter are averaged out by the plugin.

Regarding your question: Isn't calculating averages what Domoticz should be doing by itself to store the values for the graph?
Domoticz does not know what these values mean and cannot do any calculation on them.
For some you might want to have an average, whereas for others it would be interesting to see the maximum or minimum value over that timeframe.
As a result, Domoticz shows a graph that is based on the last stored value in a 5-minute (300 seconds) timeframe.

With an interval of 5 seconds over a period of 300 seconds, we receive 60 values from the inverter.
This would result in a graph based on a single random chosen value out of those 60 values; this resulted in some strange graphs that made no sense at all.
So, I decided to calculate the average over that 5-minute period.
As a result, the value you see doesn’t represent the real-time value returned by the inverter, but the graphs are more realistic this way.

The only exception to this is the “Total Energy” value; this is a counter that only goes up (unless you do a factory reset of your inverter).
Therefore, the last value stored by Domoticz is always the highest one received.

You can see the real-time values in the Setup -> Devices list; this is what the plugin receives from the inverter.
But I get your point: you don’t see the real-time information on the dashboards.

Since you already played with the code: experiment with it.
Replace the Average() with a None in the Math column and see what it does to your graphs.

Regarding the 1 second interval: more than happy to add it.
Be aware: this will have an impact on CPU usage of the box running Domoticz.
That may not be an issue for a nice computer, but it might be loading the CPU of a Raspberry…

Kind regards,

Addie

[remco-k]

With an interval of 5 seconds over a period of 300 seconds, we receive 60 values from the inverter.
This would result in a graph based on a single random chosen value out of those 60 values; this resulted in some strange graphs that made no sense at all.
So, I decided to calculate the average over that 5-minute period.
As a result, the value you see doesn’t represent the real-time value returned by the inverter, but the graphs are more realistic this way.

Yep, I agree that for displaying the graphs in Domoticz this average would make some sense, with the remark that no other plugins do this. That makes this plugin somewhat different from others.
But, furthermore, Domoticz has options to push data to external systems such as InfluxDB, having high resolution storage options. Then this averaging (or maxing) of values becomes a real problem and those graphs are unusable because of it.

So, I had a play with it and decided that you have a point regarding the graphs in Domoticz, but also that averaging needs to be optional for the user. So I just sent you a pull request with these changes in it:

• Added a 1 second poll interval (for: Value of Power (and others) lags behind real SolarEdge power with multiple minutes. #35)
• Added a "Auto Avg/Max math" switch that can enable the previously always enabled avg and max calculations. Now disabled by default. (for Value of Power (and others) lags behind real SolarEdge power with multiple minutes. #33)
• Removed the separate "Debug" True/False log level switch and added that level ("Debug") option to the "Log Level" level options, now having: Normal|Extra|Debug

While I was at it I added the 1 second interval and merged 2 logging option parameters to one.
Its this pull request: #36

Something completely different: I also have a GoodWe inverter, I didn't manage to find a working plugin for Domoticz that communicates via Modbus UDP to that inverter. Currently I'm in the process of creating a new plugin that does exactly that and it is working in a primitive state/proof of concept at the moment. I used your plugin as a reference point to get things up and running quite quickly. :) So thank you for the good example.

Hope you will merge it, or discuss about it if needed,
Regards,
Remco

[addiejanssen]

Hi Remco,

Thanks for the contribution; I have merged your updates into the codebase.
I have set the default for the "math" switch to "on" to remain compatible with the previous version of the plugin.

Kind regards,

Addie

[remco-k]

Bedankt en graag gedaan Addie. :)