Computing stuff tied to the physical world

In search of a good graph

In Software on Sep 24, 2013 at 00:01

I’ve been using the DyGraphs package for some time now in HouseMon, but it’s not perfect. Here is a generated graph for this morning, comparing two different ways to measure the same solar power generation:

Screen Shot 2013-09-23 at 09.20.52

The green series comes from the a pulse counter which generates a pulse every 0.5 Wh. These values then get sent to HouseMon every 3 seconds. A fairly detailed set of readings, as you can see – this is a typical morning in autumn on a cloudy day.

The blue series comes from the SMA inverter, which I’m reading every 5..6 minutes. It might be more accurate, but there is far less data coming in.

The key is that the area (i.e. Wh) under these two lines should be identical, if both devices are accurately calibrated. But this doesn’t quite seem to be the case here. (see update)

One known problem is that DyGraphs is plotting these “step charts” incorrectly: the value of the step should be at the end of the step, but DyGraphs can only plot steps given a starting value. So the way to read these blue steps, I think, is to keep the points, but to think of the step lines as being drawn horizontally preceding that point, not trailing it. I’m not fully convinced that this explains all the differences in the above examples, but it would seem to be a better match.

I’ve been looking at Highcharts and Highstocks as an alternative graphing library to use. They seem to support steps-with-end-values. But this sort of use really has quite a few other requirements as well, such as: being able to draw large datasets efficiently, support for multiple independent time series (DyGraphs was a bit limited here as well), interactive and intuitive zooming, good automatic choice of axis scales and fonts.

It would also be nice to be able to efficiently redraw a graph when new data points come in. Graphs used to be live in HouseMon, but at some point I ran into a snag and never bothered to fix it. For me, right now, live updating is less important than being able to generate accurate graphs. What’s the point of showing results on screen after all, if those results are not really represented truthfully…

I’ll stay on the lookout for better graphing solution. All suggestions welcome!

Update – See @ward’s comments below for the proper explanation of what’s going on here. It’s not a graphing problem after all (or rather: a different one…).

  1. There’s a lot to be said of not wasting time re-inventing the wheel but there comes a point when you need to think about doing your own thing rather than thrashing around with lots of other libraries which each do a different 90% of the job. Your own thing might be a nice patch to the library, of course.

  2. Lately I have been using d3.js. I thought I learned about it from you?

    • D3 remains an interesting option. Pretty massive, but still. There’s also cubism.js – built on top of D3. I’ve been using Raphaël for another experiment lately. None of these will do everything needed for time series graphing, at least not out of the box. But yeah, I may have to go this way…

  3. javascript to insert points in a poly line of an svg (possibly through d3.js?)? The graphing solution you want is a one-trick pony anyway, so why not keep it lean and have a dedicated [set of] svg[s] for it…

  4. You may want to have a look at At least as a user I like them.

  5. Hi, I’m using highstock for drawings all my graphs : home automation : -> ‘Graphiques’ menu, some graphs can be quite long to appears because of size data about 2Mo. The API allows a lot of customization! This interface is also compatible with touchscreen interface that is great for using a website as home automation interface through a smartphone or tablet. Some of my graphs use asynchronous data with multiple axis. I’m not using dynamic graph, but i know that highstock can update graphs ‘in live’, there are a lot of examples in their website (i’m sure you have already seen them ;-) ) regards,

    • Hi RzBo,

      I really like your Website. Did you build it yourself or is it open source? I am busy creating my own home automation website and really like the ideas on your site.

  6. From your description it is unclear to me if both your pulse counter AND the SMA inverter report energy (i.e., Wh). Your pulse counter does, as you say it generates a pulse every 0.5Wh.

    However, based on your chart, I would say the SMA reports instantaneous power (i.e. Watts), as the blue curve intersects almost exactly with the green curve at each ‘step’. In that case it would be logical that the areas under both curves don’t match.

    Are you sure the SMA reports either energy consumption (Wh) or the averaged power (W) over the given time period, and NOT instantaneous power (W) at that specific point in time?

    • BTW: the reason why I am suggesting this is because to me it seems that shifted step values (that you attribute to DyGraph) don’t seem to be the issue here. If I look at the blue data point at approx 8h54 (305 W), it doesn’t correspond neither to the previous green curve 8h49-8h54 (that would average at something around 250 W), nor to the next green curve 8h54-8h59 (that would average at something like 280 W.

    • Hey, you’re absolutely right – thanks for the insight! That explains everything – and it also means I should not be using step lines for the SMA data.

    • And to follow up on myself here as well: the difference is quite consistent: the SMA reports ≈ 10..15W higher instantaneous values than the pulse counters, which is logical because this value is the incoming solar energy, from which a bit gets used by the inverter itself.

      Great – puzzle solved!

  7. Thanks for all the suggestions – quite a range to choose from, and quite a range of effort on my part for these different suggestions. I’ll keep it all in mind as good advice.

  8. Perhaps it would be interesting to plot the efficiency of the inverter? Is the difference between green and blue always 10W (or close to that); or is it more like a percentage from the power? If it is fixed, this would mean the the invert efficiency degrades for lower power output (which I would expect). Judging from the graphs (but it is hard to see), it seems to be around 10 W regardless of the power output.

    • Yep – will have to wait until I implement better ad-hoc graphing options, but definitely interesting. Note that these are low power levels. Full sunlight will push towards 3500..4000 W, and I remember seeing somewhat higher power losses there.

  9. “One known problem is that DyGraphs is plotting these “step charts” incorrectly: the value of the step should be at the end of the step”

    For integrated data, such as electricity counters that is of course correct. In a dedicated electricity graphing it is easy to work around. What if you just shift the y-values one index to the x-values? Then it displays as intended.

    However, the issue is with the generic graphing solution as in housemon. Then you would need some additional attribute indicating the graphing style with the property (Energy). When you would add that next to say “unit”, you can also be flexible in display.

    To make it even ‘better’, the semantics should not be graphing style, but should describe the nature of the measurment. For example integrated vs instanteneous.

  10. Just had my first little play with Angularjs. It seems to be as happy to update SVG as HTML (well, I can set the radius of a circle to match the calculated cost in one of their first demos) so if you do your own graphing it should be easy to get it to update automatically: lines with ng-repeat or something.

  11. @jcw: “but the cursor values will end up wrong”.

    That is indeed a bit clumsy. That is why I use bar charts for daily consumption. At least the cursor is right. I use just line charts for power consumption, which is measured equidistantial. Works well for me.

    I also see those power consumption graphs more as power graphs. What I read from them is how much Watt my solar panels are producing, even though it is derived from some kWh measurment performed in my case every 10 seconds. Did you try graphing the 5-6 minute data as a interpolated or spline line? May just look more appealing and it is not really wrong.

    I can think for hours on the best way to display power production and consumption. Maybe I shouldn’t.

  12. @Papsnoek

    Thanks :-) I have built it ‘from scratch’. It is not a ‘true’ open source, but I can share all source code (website part and all my jeenodes module source code) from Assembla site (svn deposit). send an email ridezebigone @ gmail dot com and i will invite you. regards,

    PS : Jcw, we cannot reply to a reply ?

Comments are closed.