Computing stuff tied to the physical world

Archive for November 2012

Picking up the P1 bus

In Hardware on Nov 30, 2012 at 00:01

With the smart meter installed, I just couldn’t resist a quick readout check of the serial data of the public “P1 port”. It’s an (inverted) TTL serial signal @ 9600 baud, even parity, 7 bits with data coming out the moment you put some voltage of the “request” line.

That request line is probably nothing other than the power feed of the optocoupler inside the unit, since the output voltage more or less matches the voltage I feed it. So this could probably be operated at 3.3V as well as 5V.

A good test case for the Hameg HMO2024 scope, which has serial bus decoding built-in:


The green bars indicate correct parity. Not only is decoding a breeze this way, the latest scope firware update also added a “Bus Table” so that you can view the decoded data as a list and even dump it to a USB stick. Here’s the first part of what came out – as a CSV file:

  "Bus Table: BUS1 (UART: CH1, L, 9600,7,E,1)"
  Frame,Mark,Start time[s],Length,Data,State

Pretty trivial to decypher, so I whipped up a Tcl script for my trusty JeeMon swiss army knife (whatever works, right?):

Screen Shot 2012 11 29 at 18 08 52

And here’s the result of one scope capture (serial numbers were altered a bit):



(probably took some committee years of work to come up with this sort of gibberish)

Four electricity counter totals (night/day and consumed/produced, respectively), then two actual power consumption/production levels, then the gas meter readout. Note that the meter does not know the separate consumption and production levels – it only sees the total, but it can detect whether the flow is positive or negative.

Easy stuff. Access to the values used for our electricity and gas bill at last!

PS. This will also allow comparing and calibrating the results obtained by other means: three 2000 pulse/kWh counters attached to a JeeNode and three current transformers attached to the Flukso Meter. They each measure different things, but it’s all hooked up in such a way that the total consumption or production can be calculated with each setup.

Smart metering

In Hardware on Nov 29, 2012 at 00:01

JeeLabs has entered the 21st century…

The electricity company just installed a new “smart meter” – because they want to track consumed and produced electricity separately, something the total count on the old Ferraris-wheel meter cannot provide:

DSC 4279

See that antenna symbol on there? Its green LED is blinking all the time.

At the bottom on the right-hand side is an RJ11 jack with a “P1” connection. This is a user-accessible port which allows you to get readings out once every 10 seconds. It’s opto-coupled with inverted TTL logic, generating a 9600 baud serial stream from what I’ve read. Clearly something to hook up one of these days.

The gas meter hanging just beneath it was also replaced:

DSC 4280

Why? Because it sends its values out periodically over wireless to the smart meter, which then in turn sends it out via GPRS to the utilities company.

Apparently these gas counter values are only reported once an hour. Makes sense, in a way: gas consumption is more or less driven by heating demands, and aggregated over many households these probably vary fairly slowly – depending on outside temperature, wind, humidity, and how much the sun is shining. Not nearly as hard to manage as the electricity net, you just have to keep the gas pressure within a reasonable range.

Electricity is another matter. And now it’s all being monitored and reported. Not sure how often, though – every 2 months, 15 minutes, 10 seconds? How closely will big brother be watching me? First internet & phone tracking, and now this – I don’t like it one bit…

Welcome to the 21st century. Everything you do is being recorded. For all future generations to come.

Snooping with an Optocoupler

In Hardware on Nov 28, 2012 at 00:01

The thing with OpenTherm, is that the amount of current going through the wire is used by the boiler to send messages to the thermostat.

The reverse path, i.e. from thermostat to boiler, is signalled by voltage changes, which are considerably easier to detect. So let’s save that for later.

There is a small complication, in that the polarity of the wiring between boiler and thermostat is not defined, so either one of them could be “”+” and “-“, respectively. Of course once you’ve hooked things up, that polarity never changes again.

Can we measure the current going through a wire, without knowing in which direction it is flowing? We know it’ll be 5..7 mA for one signalling state and 20..25 mA for the other.

Here’s what I came up with:

JC s Grid page 47

There are two optocouplers in there, with the diodes connected in opposite ways. Depending on the direction of the current, one of them will block and the other one will light up – once the voltage over the 82 Ω resistor exceeds about 1V, i.e. at ≈ 12 mA.

For documentation purposes, the actual build:

DSC 4272

The voltage drop over this circuit is at most just over 1V, which may or may not interfere with proper operation of the boiler and thermostat. Testing will be needed to find out.

The first advantage of this circuit is that it works with either polarity without needing a bridge rectifier (which would introduce yet another voltage drop). In addition, the output signal is galvanically isolated from the OpenTherm loop, i.e. “floating”. That means it can be connected in whatever way is needed.

The second part of an “OpenTherm snooper” – if it ever materialises – will be to measure the voltage between the wires and hopefully also to self-power the rest of the circuit. Note that the optocoupler LED lights up when a high current is passing through, and this is also the state where the photo transistor is drawing more current through the 10 kΩ resistor.

Here’s the diode voltage (yellow) and output (blue), using the same ± 10V @ 50 Hz signal as yesterday. The vertical zero axis is one division down from the centre, for both traces:


Note how the output triggers on both positive and negative excursions of the input signal due to the anti-parallel LEDs, which is why it ends up having twice as many pulses. So the first half is one LED turning on and off, and the second half is the other LED – both lead to the common OUT pin being pulled down. For OpenTherm use, there’d never be both polarities – only one LED would be active, depending on how the circuit is connected.

The pulse-width asymmetry you see is an artefact of the way the sine wave is applied (using a 150 Ω resistor). This will not happen with a 7..25 mA current toggle and 82 Ω. And while the MCT62 is not one of the fastest optocouplers, especially with a 10 kΩ collector pull-up, I expect that the resulting pulses will still be accurate enough.

So far so good. I haven’t built the rest yet – just doodling and trying to figure it all out.

Sensing with an Optocoupler

In Hardware on Nov 27, 2012 at 00:01

The OpenTherm setup keeps me thinking…

I haven’t given up on the OpenTherm Gateway yet, but I’ve also been toying with related ideas for some time to try and just listen in on that current/voltage conversation using a self-powered JeeNode, which then reports what it sees as wireless packets.

It’s all based on Optocouplers, so here’s a first circuit to try things out:

JC s Grid page 47

A very simple test setup, which I’m going to feed a ±10V sine wave @ 50 Hz, just because the component tester on my oscilloscope happens to generate exactly such a signal. The 1 kΩ resistor is internal to the component tester, in fact. Here’s what comes out:


The yellow trace is the voltage over the IR LED inside the optocoupler, the blue trace is the voltage on the OUT pin. VCC is a 3x AA Eneloop battery pack @ 3.75V – what you can see is that the LED starts to conduct at ≈ 0.8V, and generates just enough light at 0.975V for the photo transistor to start conducting as well, pulling down the output voltage. With 1.01V over the LED, it already generates enough light for the output to drop to almost 0V.

In other words: within a range of just 41 mV at about 1V, the optocoupler “switches on”.

So much for the first part of this experiment. My hope is that this behavior will be just right to turn this MCT62 optocoupler into a little OpenTherm current “snooper” – stay tuned…

Watts, Amps, Coulombs

In Hardware on Nov 26, 2012 at 00:01

Sometimes I see some confusion on the web regarding the units to measure power with.

Here’s a little summary, in case you ever find yourself scratching your head with this stuff:

  • Electric potential is sort of a “pressure level” when using the water analogy, expressed in Volts (V)
  • Current is the flow of electrons, and is expressed in Amperes (A)
  • Charge is the “amount of electricity”, and is expressed in Coulombs (C)
  • Power is the product of volts and amperes, and is expressed in Watts (W)

Another measure of power is Volt-Amperes, this is not the same as Watts in the case of alternating current with reactive loads, but let’s not go there for now…

To summarise with the water analogy:

  • Volts = how high has the water been pumped up
  • Amps = how much water is flowing
  • Coulombs = the amount of water
  • Watts = how much energy is being used (or generated)

You can probably guess from this list that pumping water up twice as high (V) takes twice as much energy, and that pumping up twice as much (A) also takes twice as much energy. Hence the formula:

    Watt = Volt * Ampere

Other equations can also help clarify things. They all add time into the mix (in seconds).

Current is “charge per second”:

    Ampere = Coulomb / second

This is also the way I estimate average current consumption when diving into ultra-low power JeeNode stuff: using the oscilloscope to integrate (sum up) all the instantaneous current consumptions over time, I get a certain Coulomb (or micro-coulomb) value. If that’s a periodic peak and the system is powered-down the rest of the time, then the estimate becomes: X µC used per Y sec, hence the average current consumption is X / Y µA. The advantage of working with Coulombs in this way, is that you can add up all the estimates for the different states the system is in and still arrive at an average current level.

Another one: power consumption is the amount of energy consumed over time. This is often expressed in Watt-hour (Wh) or kilowatt-hour (kWh):

  • two 100 W lightbulbs running for 5 hours = 2 x 100 x 5 = 1000 Wh = 1 kWh
  • one LED strip drawing 2 A at 12 V for 3 hours = 2 x 12 x 3 = 72 Wh

And then there’s the “mAh” unit used with batteries, i.e. milli-ampere-hour. Quite simple again, once you get used to this metric system, and realise that you also need the voltage:

  • 2 AA batteries of 1.5V @ 2000 mAh each provide 2 x 1.5 x 2000 = 6000 mWh = 6 Wh
  • a 5 mA load on batteries of 2000 mAh will run for 2000 / 5 = 400 hours

Battery capacities are roughly as follows for the most common types:

  • an AA cell has 2500 mAh @ 1.5V = 3.75 Wh
  • an AA rechargeable cell has 2000 mAh @ 1.2V = 2.4 Wh
  • an AAA cell has 1000 mAh @ 1.5V = 1.5 Wh
  • an AAA rechargeable cell has 800 mAh @ 1.2V = 0.96 Wh
  • a CR2032 coin cell has 200 mAh @ 3V = 0.6 Wh

Wanna be able to run for a week on a coin cell? Better make sure your circuit draws no more than 200 / (24 x 7) = 1.2 mA on average under optimal conditions.

Wanna make it run a year on that same coin cell? Stay under 22 µA average, and it will.

With 2 or 3 AA batteries, you get an order of magnitude more to consume, so if you can get the average under 200..220 µA, those batteries should also last a year (ignoring the fact that batteries always have some self-discharge, that is).

The difference between 2, 3, or 4 AA batteries in series only affects the voltage you get out of them. Chips do not run more efficiently on more voltage – on the contrary, in fact!

For low-power use: run your circuit on as low a voltage as possible, but no lower (wink).

Bi-directional signalling and power

In Hardware on Nov 25, 2012 at 00:01

It looks like the OpenTherm gateway is sensitive to noise and wiring lengths. All my attempts to move the gateway upstairs, next to the boiler/heater, failed. Somehow, this:

  THERMOSTAT  <=>  GATEWAY  <=>  10 m wire  <=>  HEATER

… is not the same as this!

  THERMOSTAT  <=>  10 m wire  <=>  GATEWAY  <=>  HEATER

The OpenTherm documentation (PDF) specifically allows up to 50 meters of untwisted wiring, but I’m clearly running into some issue here.

Time to drag the scope downstairs and hook it up between gateway and heater:


The yellow trace is the voltage between the two wires, while the blue trace is the current through those wires. I used a 1 Ω resistor and measured the voltage drop, but had to switch to the most sensitive scale (since I’m using the standard x10 probe), hence all that noise.

Still, you can see the magic of the way the OpenTherm protocol works:

  • in rest, there’s 6V between the wires and about 6 mA of current flowing (a 1 kΩ load)
  • this is used by the thermostat to power itself (by keeping a capacitor charged)
  • when the thermostat sends data, it briefly reduces its current draw
  • since the boiler (or gateway) is feeding a constant current, this makes voltage go up
  • that voltage change is then detected and decoded by the boiler / gateway
  • about 40 ms later, the boiler / gateway then sends a reply
  • it does this by briefly forcing more current down the wire
  • this in turn can be detected by the thermostat, which then decodes that reply
  • there’s a small residual ripple, as the thermostat tries to maintain its 7V idle level

I was going to perform the same measurement on the other side of the gateway, i.e. the connection to the thermostat, but for some reason the gateway really doesn’t like me touching anything or connecting any wires to it (let alone a grounded scope probe). Maybe some noise is picked up and feeding back into one of the PIC’s I/O pins, and completely throwing it off. Luckily, the whole gateway always resets properly when left alone again.

I also sometimes see the thermostat indicating a fault (even just by touching the wire with the scope probe) – so it seems to be getting some power, but it’s definitely not happy.

Maybe the gateway’s output circuit is too sensitive, due to some high-impedance parts in the circuit? That would explain why even just using some long wires two floors down prevents the gateway from working.

Hm, not good – especially since I only wish to monitor the wire, not control it…

Update – these problems are caused by a floating ground. More on this once I get it all sorted out. With many thanks to Schelte Bron for dropping by and helping analyse this!

It’s a current source!

In Hardware on Nov 24, 2012 at 00:01

Let me answer yesterday’s question first: “Is this circuit actually useful for anything?”

You bet: this is called a Current source.

The circuit will deliver a constant current by varying the voltage drop, even when the load varies. You can see this in the fairly flat curve on the Component Tester screenshot included yesterday: no matter what level positive voltage you apply to this thing, it’ll draw about 2 mA (just ignore the negative end of the scale).

Actually, I cheated a bit. The real two-transistor current source circuit looks like this:

Curr source

By moving that 10 kΩ resistor away from the load, and tying it directly to “+” the circuit works even better. I’ve simulated it with an external power supply to drive that resistor separately, and get this CT screen:


Totally flat! – And that 2 mA current level is set by the 330 Ω resistor, by the way.

One use for this could be a constant-current LED driver (although its efficiency would be very low – you really need a switching circuit with an inductor to get good efficiency).

So how does this mind-bending circuit actually work?

The key point to note, is that the emitter-to-base junction is essentially a diode (which is probably why transistors are drawn the way they are!). And it has a fixed forward-drop voltage of about 0.65V. As long as the base is less than 0.65V above the emitter voltage, the transistor will be switched off. As soon as the base is raised higher, current will flow through that forward diode and the transistor will start to conduct.

This is also why you always need a current limiting resistor: the base voltage cannot rise above 0.65V, it’ll simply conduct more current. Until the current limits are exceeded and the transistor is destroyed, that is…

First, imagine that the leftmost transistor is absent: then the 10 kΩ will pull up the base of the rightmost transistor and cause it to fully conduct. The circuit now essentially acts as the load in series with the 330 Ω resistor. With a maximum load (a short-circuit), the whole supply voltage will end up across that 330 Ω resistor.


With the leftmost transistor in place, something special happens: as soon as the voltage over the 330 Ω resistor rises above 0.65V, the leftmost transistor will start to conduct, pulling the base of the rightmost transistor down. It will continue to do so until the voltage over the 330 Ω resistor has dropped to 0.65V again. Because at some point the base of the rightmost transistor will be pulled so low that it no longer fully conducts – thus reducing the current through the 330 Ω, and thus lowering the voltage drop across it.

You’re seeing a neat little negative feedback loop in action. These two transistors are going to balance each other out to the point where the 330 Ω resistor ends up having a voltage drop of exactly 0.65V – regardless of what the load is doing!

To get 0.65V over 330 Ω, we need a 0.65/330 = 1.97 mA current.

And so that’s what this circuit will feed to the load. As you can see in that last scope capture, the regulation is extremely good between 0.65 and 9V.

By simply varying the 330 Ω value, we can set any desired fixed current level.

The reason I’m bringing this up, is that this circuit is in fact used in the OpenTherm gateway – see this schematic (look for the upside-down PNP version). With some extra circuitry to set the resistor to either 100 Ω or 28 Ω (100 Ω in parallel with 39 Ω). So the gateway is driving either 7 mA or 25 mA through the thermostat.

Welcome to the magical world of electronics – it’s full of clever little tricks like this!

Mystery circuit

In Hardware on Nov 23, 2012 at 00:01

Here’s a little puzzle for you – what does this circuit do?

DSC 4257

That’s two BC549 NPN transistors and two resistors. Let me draw the schematic:


Better still, here’s what it shows when hooked up to my Component Tester:


Horizontal scale: -10 V .. +10 V, vertical: -10 mA .. +10 mA.

Bonus question: is this circuit actually useful for anything?

Reducing the packet rate

In Software on Nov 22, 2012 at 00:01

One last optimisation, now that the OpenTherm relay sends nice and small 3-byte packets, is to reduce the number of packets sent.

The most obvious reduction would be to only send changed values:

Screen Shot 2012 11 11 at 14 40 15

This is a trivial change, but it has a major flaw: if packets are lost – which they will, once in a while – then the receiving node will never find out the new value until it changes again.

There are several ways to solve this. I opted for a very simple mechanism: in addition to sending all changes, also send out unchanged values every few minutes anyway. That way, if a packet gets lost, at least it will be resent within a few minutes later, allowing the receiver to resynchronise its state to the sender.

Here’s the main code, which was rewritten a bit to better match this new algorithm:

Screen Shot 2012 11 11 at 14 35 52

This still keeps the last value for each id in a “history” array, but now also adds a “resend” counter. The reason for this is that I only want to re-send id’s which have been set at least once, and not all 256 of them (of which many are never used). Also, I don’t really want to keep sending id’s for which nothing has been received for a long time. So I’m setting the re-send counter to 10 every time a new value is stored, and then counting them down for each actual re-send.

The final piece of the puzzle is to periodically perform those re-sends:

Screen Shot 2012 11 11 at 14 36 10

And in the main loop, we add this:

Screen Shot 2012 11 11 at 14 36 24

Here’s how it works: every second, we go to the next ID slot. Since there are 256 of them, this will repeat roughly every 4 minutes before starting over (resendCursor is 8 bits, so it’ll wrap from 255 back to 0).

When examining the id, we check whether its resend counter is non-zero, meaning it has to be sent (or re-sent). Then the counter is decremented, and the value is sent out. This means that each id value will be sent out at most 10 times, over a period of about 42 minutes. But only if it was ever set.

To summarise, as long as id values are coming in:

  • if the value changed, it will be sent out immediately
  • if it didn’t, it’ll be re-sent anyway, once every 4 minutes or so
  • … but not more than 10 times, if it’s never received again

And indeed, this reduces the packet rate, yet sends and re-sends everything as intended:

  L 13:48:48.073 usb-A40117UK OK 14 24 18 197
  L 13:48:49.080 usb-A40117UK OK 14 25 51 0
  L 13:48:50.072 usb-A40117UK OK 14 26 43 0
  L 13:48:52.072 usb-A40117UK OK 14 28 51 0
  L 13:49:20.075 usb-A40117UK OK 14 56 60 0
  L 13:49:36.548 usb-A40117UK OK 14 20 238 51
  L 13:50:37.549 usb-A40117UK OK 14 20 238 52
  L 13:51:33.532 usb-A40117UK OK 14 24 18 186
  L 13:51:37.563 usb-A40117UK OK 14 20 238 53
  L 13:52:34.537 usb-A40117UK OK 14 24 18 188
  L 13:52:38.553 usb-A40117UK OK 14 20 238 54
  L 13:52:40.088 usb-A40117UK OK 14 0 2 0
  L 13:52:41.096 usb-A40117UK OK 14 1 10 0
  L 13:52:46.087 usb-A40117UK OK 14 6 3 1
  L 13:52:54.101 usb-A40117UK OK 14 14 100 0
  L 13:52:56.101 usb-A40117UK OK 14 16 18 0
  L 13:53:00.100 usb-A40117UK OK 14 20 238 54
  L 13:53:04.099 usb-A40117UK OK 14 24 18 188
  L 13:53:05.090 usb-A40117UK OK 14 25 51 0
  L 13:53:06.098 usb-A40117UK OK 14 26 43 0
  L 13:53:08.098 usb-A40117UK OK 14 28 51 0
  L 13:53:34.538 usb-A40117UK OK 14 24 18 192

Time to close that OpenTherm Gateway box and move it next to the heater. Onwards!

Reducing the payload size

In Software on Nov 21, 2012 at 00:01

The OpenTherm relay sketch presented yesterday sends out 9-byte packets containing the raw ASCII text received from the gateway PIC. That’s a bit wasteful of bandwidth, so let’s reduce that to a 3-byte payload instead. Here is some extra code which does just that:

Screen Shot 2012 11 11 at 12 37 33

I’m using a very hacky way to convert hex to binary, and it doesn’t even check for anything. This should be ok, because the packet has already been verified to be of a certain kind:

  • marked as coming from either the thermostat or the heater
  • the packet type is either Write-Data or Read-Ack
  • every other type of incoming packet will be ignored

Note the shouldSend() implementation “stub”, to be filled in later to send fewer packets.

Here are the results of this change, as received by the central node:

L 11:29:26.651 usb-A40117UK OK 14 16 18 0
L 11:29:27.547 usb-A40117UK OK 14 20 236 30
L 11:29:28.635 usb-A40117UK OK 14 28 45 0
L 11:29:29.642 usb-A40117UK OK 14 0 2 0
L 11:29:30.650 usb-A40117UK OK 14 25 46 0
L 11:29:31.562 usb-A40117UK OK 14 1 10 0
L 11:29:31.658 usb-A40117UK OK 14 1 10 0
L 11:29:34.649 usb-A40117UK OK 14 25 46 0

Much better, although still a lot of duplication and far too many packets to send.

I’ll fix this tomorrow, in the final version of this otRelay.ino sketch.

OpenTherm relay

In Software on Nov 20, 2012 at 00:01

Now that the OpenTherm Gateway has been verified to work, it’s time to think about a more permanent setup. My plan is to send things over wireless via an RFM12B on 868 MHz. And like the SMA solar inverter relay, the main task is to capture the incoming serial data and then send this out as wireless packets.

First, a little adapter – with 10 kΩ resistors in series as 5V -> 3.3V “level converters”:

DSC 4253

(that’s an old JeeNode v2 – might as well re-use this stuff, eh?)

And here’s the first version of the otRelay.ino sketch I came up with:

Screen Shot 2012 11 11 at 00 26 19

The only tricky bit in here is how to identify each message coming in over the serial port. That’s fairly easy in this case, because all valid messages are known to consist of exactly one letter, then 8 hex digits, then a carriage return. We can simply ignore anything else:

  • if there is a valid numeric or uppercase character, and there is room: store it
  • if a carriage returns arrives at the end of the buffer: bingo, a complete packet!
  • everything else causes the buffer to be cleared

This isn’t the packet format I intend to use in the final setup, but it’s a simple way to figure out what’s coming in in the first place.

It worked on first try. Some results from this node, as logged by the central JeeLink:

  L 10:38:07.582 usb-A40117UK OK 14 84 56 48 49 65 48 48 48 48
  L 10:38:07.678 usb-A40117UK OK 14 66 52 48 49 65 50 66 48 48
  L 10:38:08.558 usb-A40117UK OK 14 84 56 48 49 57 48 48 48 48
  L 10:38:08.654 usb-A40117UK OK 14 66 52 48 49 57 51 53 48 48
  L 10:38:09.566 usb-A40117UK OK 14 84 49 48 48 49 48 65 48 48
  L 10:38:09.678 usb-A40117UK OK 14 66 68 48 48 49 48 65 48 48
  L 10:38:10.574 usb-A40117UK OK 14 84 48 48 49 66 48 48 48 48
  L 10:38:10.686 usb-A40117UK OK 14 66 54 48 49 66 48 48 48 48
  L 10:38:11.550 usb-A40117UK OK 14 84 48 48 48 70 48 48 48 48
  L 10:38:11.646 usb-A40117UK OK 14 66 70 48 48 70 48 48 48 48
  L 10:38:12.557 usb-A40117UK OK 14 84 48 48 49 50 48 48 48 48

One of the problems with just relaying everything, apart from the fact that it’s wasteful to send it all as hex characters, is that there’s quite a bit of info coming out of the gateway:

Screen Shot 2012 11 10 at 22 44 51

Not only that – a lot of it is in fact redundant. There’s really no need to send the request as well as the reply in each exchange. All I care about are the “Read-Ack” and “Write-Data” packets, which contain actual meaningful results.

Some smarts in this relay may reduce RF traffic without losing any vital information.

OpenTherm data processing

In Software on Nov 19, 2012 at 00:01

Before going into processing the data from Schelte Bron’s OpenTherm Gateway, I’d like to point to OpenTherm Monitor, a multi-platform application he built and also makes freely available from his website.

It’s not provided for Mac OSX, but as it so happens, this software is written in Tcl and based on Tclkit, by yours truly. Since JeeMon is nothing but an extended version of Tclkit, I was able to extract the software and run it with my Mac version of JeeMon:

  sdx unwrap otmonitor.exe
  jeemon otmonitor.vfs/main.tcl
Heh – nothing beats “re-using” one’s own code in new and mysterious ways, eh?

Here’s the user interface which pops up, after setting up the serial port (it needed some hacking in the otmonitor.tcl script):

Screen Shot 2012 11 10 at 22 35 47

I left this app running for an hour (vertical lines are drawn every 5 minutes), while raising the room temperature in the beginning, and running the hot water tap a bit later on.

Note the high error count: looks like the loose wires are highly susceptible to noise and electrostatic fields. Even just moving my hand near the laptop (connected to the gateway via the USB cable) could cause the Gateway to reset (through its watchdog, no doubt).

Still, it looks like the whole setup works very nicely! There’s a lot of OpenTherm knowledge built into the otmonitor code, allowing it to extract and even control various parameters in both heater and thermostat. As the above window shows, all essential values are properly picked up, even though this heater is from a different vendor. That’s probably the point of OpenTherm: to allow a couple of vendors to make their products inter-operable.

But here’s the thing: neither the heater nor the thermostat are near any serial or USB ports over here, so for me it would be much more convenient to transmit this info wirelessly.

Using a JeeNode of course! (is there any other way?) – stay tuned…

PS. Control would be another matter, since then the issue of authentication will need to be addressed, but as I said: that’s not on the table here at the moment.

Tackling OpenTherm

In Hardware on Nov 18, 2012 at 00:01

Another project I’ve been meaning to tackle for a very long time is to monitor the central heating and warm water system. Maybe – just as with electricity – knowing more about what’s going on will help us reduce our fairly substantial natural gas bill here at JeeLabs.

The gas heater is from Vaillant and it’s connected to a Honeywell ChronoTherm – this is a “modulating” thermostat which automatically chooses its set-points based on the time of day and the day of the week. It all works really well.

The heater upstairs and the thermostat in the living room are connected by a two-wire low-voltage connection, using the OpenTherm protocol. There’s not that much “open” about this protocol, but people have hacked their way in and have discovered all the basic information being exchanged between these units.

A while back, I got a free PCB (thx, Lennart!) of a circuit by Schelte Bron, called the OpenTherm Gateway, and since all the required components were listed and easily available from Conrad, I decided to give it a go. Here’s the whole thing assembled:

DSC 4244

The documentation is very well done: schematics, parts list, troubleshooting, and more.

This is a “gateway” in that it sits between the heater and the thermostat, so it can not only listen in on the conversation but actually take over. Things you can do with it is to adjust the set-point (i.e. desired room temperature), feed in the temperature from an outside sensor, set the ChronoTherm’s clock, and probably more. I’m only interested in monitoring this stuff for now, i.e. reading what is being exchanged.

The gateway is based on an 8-bit PIC controller, and has some funky electronics to do its thing – because the way these signals are encoded is pretty clever: there are only two wires, yet the heater actually powers the thermostat through them, and supports bidirectional I/O (hint: it uses voltage and current modulation).

One little gotcha is that this gateway brings out its interface as an RS232-compatible serial port. And to my surprise, I found out that I don’t even have any laptops to read out these +/- 12V level signals anymore!

So the next task was to get things back into “normal” logic levels. Simple, although it’s a bit of a hack: remove the on-board MAX232 level converter chip, and insert wires to bring out the original 5V logic levels instead:

DSC 4250

(bare wire + green clip = GND, read wire + yellow clip = RX, white wire + white clip = TX)

Step two: hook it up to a USB-BUB, set to handle 5V logic levels:

DSC 4251

Step three: plug the USB cable into my laptop to pick up the data coming in at 9600 baud.

As you can see, this has wires dangling all over – just to check that the gateway works:

  OpenTherm Gateway 3.2

Yeay, looks like we’re getting something! Coming up next: making sense of this data…

Ultra-low power op-amp

In Hardware on Nov 17, 2012 at 00:01

Op-amps are one of the building blocks of the modern analog electronics industry.

Here’s an interesting one, the MAX4470 .. MAX4474 series:

Screen Shot 2012 11 14 at 13 09 51

Simple layout, again in a tiny SMD package:

DSC 4269

The other members of this family are dual and quad versions, if you need more op-amps.

This chip is nice because of its phenominally low current consumption: 750 nA at 5V. It gets even better: at 3.3V, I measured a ridiculously low 190 nA!

Here are some more specs from the Maxim datasheet:

Screen Shot 2012 11 14 at 13 14 17

Might not be the highest-performance op-amp out there, but still – this thing could be quite handy to implement comparators, voltage followers, oscillators, amplifiers, filters, and more. Especially when the “power budget” is really really low.

PS. I’m assuming this chip isn’t oscillating with the above test setup, but in normal use you really need to tie the input pins to something to avoid that.

Nanoampere regulator

In Hardware on Nov 16, 2012 at 00:01

The TPS78233 from Texas Instruments looks like a standard LDO linear voltage regulator:

Screen Shot 2012 11 14 at 00 57 17

It takes an input voltage up to 5.5V and regulates it down to 3.3V (the above image from the datasheet is the 2.7V regulator). Not a spectacular voltage range, but it has a very nifty trick up its sleeve:

This regulator only draw 450 nA, i.e. 0.45 µA, when unloaded!

That’s about a quarter of the current consumption of the already-spectacular MCP1702 and MCP1703 used in JeeNodes – a ridiculously low 2.5 microwatts.

Here’s a little test setup (yep, those SMD’s are small – can you see the two 10 µF caps?):

DSC 4258

To get a sense of this level of current consumption: 3x AA batteries of 2000 mAh would last 5 centuries (ehm, well, except for their pesky self-discharge) – which is a bit silly, of course.

To get another idea: when I measure the output voltage with a multi-meter, the current consumption “jumps” to about 750 nA. Why this relatively big change? Because most multi-meters have a 10..11 MΩ input impedance, and 3.3V over 11 MΩ is… 300 nA!

In case you’re wondering: measuring minute currents is easy with Dave Jones’ µCurrent.

The fascinating thing about the TPX82xx series is that it achieves this extremely low idle current while still being able to regulate and supply up to 150 mA. Furthermore, that enable pin might come in very handy for certain ultra-low energy harvesting scenario’s.

But I’m not going to replace the regulator on JeeNodes for a number of reasons:

  1. The MCP1702 can handle input voltages up to 13V (vs only 5.5V for the TPS78233).

  2. There’s no through-hole version, so this would not work for standard JeeNode kits.

  3. Those extra savings only kick in when you get everything into the few-µA range, and so far, things like Room Nodes still draw a few dozen µA’s.

  4. In many cases, when the max is 5.5V anyway, no regulator will be needed at all (note that running only the RFM12B on it may create a problem with signal levels).

But hey, it’s good too know that these chips exist. A few microwatts… wow!

Microwave interference?

In Hardware on Nov 15, 2012 at 00:01

To get more options with remote nodes, I wanted to get this DCF77 clock working again, broadcasting the current time of day once a minute:

3311819588 20f3b58d12 o

Last time I tried (long ago), it didn’t work – but I’ve been running the OOK Relay for ages now, and since it still had the DCF77 receiver hooked up, I decided to have another go.

Sure enough – all it was is a broken wire… doh!

Now there are time-of-day packets in the air again, here at JeeLabs – sent by ookRelay2:

Screen Shot 2012 11 13 at 12 24 29

It works really well – at last – and will come in handy soon enough, I expect.

But the curious thing is those 3 missed packets at 12:13, 12:16, and 12:19.

Could it be the microwave? It was just around the time I was heating up something…

More investigation needed.

Watts up?

In Hardware on Nov 14, 2012 at 00:01

Ok, so all the solar panels are in place and doing their thing (as much as this season allows, anyway). But seeing that live power usage on my desk all day long kept tempting me to try and optimise the baseline consumption just a tad more…

Previous readings have always hovered around 115 Watts, lately. Since the JeeLabs server + router + internet modem use about 30 W together, that leaves roughly 85 W unaccounted for. Note that this is without fridges, boilers, heat circulation pumps, gas heaters, or other intermittent consumers running. This baseline is what we end up consuming here no matter what – vampire power from devices in “standby” and other basic devices you want to keep running at all times, such as the phone and internet connection.

It’s not excessive, but hey: 100 W day-in-day-out is still over 850 kWh on a yearly basis.

Well, today I managed to get the baseline down waaay further:

DSC 4242

That’s including the JeeLabs server + router + modem. So the rest of the house at JeeLabs is consuming under 40 W. Perfect: I’ve reached my secret goal of a baseline under 50 W!

Here’s how that “idle” power consumption was reduced this far:

  • I turned off an old & forgotten laptop and Ethernet switch, upstairs – whoops!
  • I removed another gigabit Ethernet switch under my desk (more on that later)
  • the 10-year old Mac Mini + EyeTV + satellite dish setup has been dismantled and replaced by a small all-in-one TV drawing 0.5W in standby (the monitor is re-used)
  • I’m switching to DVB-C (i.e. coax-based) reception, available from the internet modem by upgrading to the cheapest triple-play subscription with “analog + digital” channels
  • that means: no settop box, just the internet modem (already on anyway) and a new low-end but modern Sharp 22″ TV / DVB-C / DVD-player / USB-recorder

As it turns out, the Mac Mini (about 10 years old) plus the master-slave AC mains switch controlling everything else were drawing some 20 W – day in day out. Bit silly, and far too much unnecessary technology strung together (though working, most of the time).

The other biggie: no more always-on Ethernet switches, just the WRT320N wireless router in front of the server, with a second wired gigabit connection to my desk. That’s two really fast connections where it matters, everything else uses perfectly-fine WiFi.

The main reason for having an Ethernet switch near my desk was to allow experimenting with JeeNode-based EtherCards, Raspberry Pi’s, and so on. But… 1) that switch was really in the wrong place, it would be far more convenient to have Ethernet in the electronics corner at JeeLabs, and 2) why keep that stuff on all the time, anyway?

So instead, I’m now re-using a spare Airport Express as wireless-to-wired Ethernet extension router. Plug it in, wait a minute for it to settle down, and voilá – instant wired Ethernet anywhere there is an AC mains socket:

DSC 4243

And if I need more connections, I can route everything through that spare Ethernet switch.

It’s not the smallest solution out there, but who cares. Why didn’t I think of all this before?

The difference between 2 and 3

In AVR, Software on Nov 13, 2012 at 00:01

One. Ok, next post :)

I was curious about the difference between Power-down and Standby in the ATmega328p. Power-down turns off the main clock (a 16 MHz resonator in the case of JeeNodes), whereas standy keeps it on. And quite surprised by the outcome… read on.

There’s an easy way to measure this, as far as software goes, because the rf12_sendWait() function has an argument to choose between the two (careful: 2 = Standby, 3 = Power-down – unrelated to the values in the SMCR register!).

I tweaked radioBlip.ino a bit, and came up with this simple test sketch:

Screen Shot 2012 11 05 at 18 13 23

With this code, it’s just a matter of hooking up the oscilloscope again in current measurement mode (a 10 Ω resistor in the power line), and comparing the two.

Here’s the standby version (arg 2):


… and here’s the power-down version (arg 3), keeping the display the same:


I’ve zoomed in on the second byte transmission, and have dropped the baseline by 21 mA to properly zoom in, so what you’re seeing is the ATmega waking up to grab store a single byte out of into the RFM12B’s reception FIFO, and then going back to sleep again.

The thing to watch is the baseline level of these two scope captures. In the first case, it’s at about 0.5 mA above the first division, and the processor quickly wakes up, does it’s thing, and goes into power-save again.

In the second case, there’s a 40 to 50 µs delay and “stutter” as the system starts its clock (the 16 MHz resonator), does its thing, and then goes back to that ultra-low power level.

There are bytes coming in going out once every 200 µs, so fast wakeup is essential. The difference is keeping the processor drawing 0.5 mA more, versus a more sophisticated clock startup and then dropping back to total shutdown.

What can also be gleaned from these pictures, is that the RF12 transmit interrupt code takes under 40 µs @ 16 MHz. This explains why the RF12 driver can work with a clock frequency down to 4 MHz.

The thing about power-down mode (arg 3), is that it requires a fuse setting different from what the standard Arduino uses. We have to select fast low-power crystal startup, in 258 cycles, otherwise the ATmega will require too much time to start up. This is about 16 µs, and looks very much like that second little hump in the last screen shot.

Does all this matter, in terms of power savings? Keep in mind that this is running while the RFM12B transmitter is on, drawing 21 mA. This case was about the ATmega powering down between each byte transmission. Using my scope’s math integral function, I measured 52.8 µC for standby vs 60.0 µC for power-down – so we’re talking about a 12 % increase in power consumption during transmit!

The explanation for this seemingly contradictory result is that the power-down version adds a delay before it actually sends out the first byte to transmit. In itself that wouldn’t really make a difference, but because of the delay, the transmitter stays on noticeably longer – wiping out the gains of shutting down the main clock. Check this out:


Standby is the saved white reference trace, power-down is yellow.

See? Ya’ can’t always predict this stuff – better measure before jumping to conclusions!

PS. While I’m at it – enabling the RF12’s “OPTIMIZE_SPI” mode lowers charge usage only a little: from 52.8 to 51.6 µC, i.e. using fast SPI @ 8 MHz i.s.o. 2 MHz for sending to the RFM12B. Hardly worth the trouble.

High-side switching

In Hardware on Nov 12, 2012 at 00:01

If low-side switching is so troublesome, then why not just flip everything around, right?

High side partial

Not so fast. The I/O pin is tied to a microcontroller running at 3.3 or 5V, so its voltage level will vary between 0 and a few volts. Whereas “+” is more likely to be 5V, 12V, or even 24V.

This means that to keep the PNP transistor switched off, we need to keep the base voltage at nearly the same level as that “+” line. Unfortunately, this is impossible – not only could high voltages on I/O pins of a µC damage them, there is also some protection circuitry on each pin to protect against electrostatic discharge (ESD). If you were to look inside the µC chip, you’d find something like this on each I/O pin:


What that means is that if you try to pull an I/O pin up to over VCC+0.7V, then that topmost diode will start to conduct. This is no problem as long as the current stays under 1 mA or so, but it does mean that the actual voltage of an I/O pin will never be more than 4V (when running on 3.3V). Which means that PNP transistor shown in the first image will always be on, regardless of the I/O pin state.

We’ll need a more complex circuit to implement a practical high-side power-on switch:

Npn pnp

The workhorse, i.e. the real switch, is still the PNP transistor on the right. But now there’s an an extra “stage” in front to isolate the I/O pin from the higher voltages on the base of that PNP transistor. There’s now essentially a low-side switch in front of the PNP.

When I/O is “0”, no current flows into the base of the NPN transistor, which means it won’t conduct, and hence no current flows into the base of the PNP transistor either.

When I/O is “1”, the NPN transistor will conduct and pull its collector towards ground. That leaves a 10 kΩ resistor between almost ground (0.4V) and almost high (“+” – 0.7V), since the base-to-emitter junction of a transistor is more or less a forward-conducting diode. So the base of the PNP transistor is pulled down, and the PNP transistor is switched on. The resistor values are not too critical here – making them both 10 kΩ would also work fine. But they have to be present to limit both base currents.

A similar circuit can be created with two MOSFETs. With the proper choice of MOSFETs, this will in fact be a better option, because it can handle more current and will have less power loss (i.e. heat). The resistors will need to be placed differently.

Note that all circuits can be analysed & explained in the same way, as long as there are no feedback loops: step-by-step, reasoning about the effect of each stage on the next.

Low-side switching

In Hardware on Nov 11, 2012 at 00:01

Yesterday’s post brought up some good comments, which I’d like to expand on a bit.

To summarise, this is about how to switch power to an electric circuit using an I/O pin.

Yesterday’s solution worked for me, but would fail if the voltage range is not as nicely predictable, i.e. trying to control say between 2 and 12V with an I/O pin which supplies 1.8 to 3.3V. In this case, the 0.7V diode drop of the base-to-emitter junction of a transistor won’t always be of much help.

Let’s examine some approaches. First, what is perhaps the most obvious way:

Low side

With a “normal” (BJT) NPN transistor, you feed it some current by making an I/O pin high, and it’ll conduct. There needs to be a resistor in series, large enough to limit the current, but small enough to drive the transistor into saturation (10 kΩ should work for loads up to say 25 mA, you can reduce it to switch more current).

With an N-MOSFET, you pull the gate up, again by setting an I/O pin high. In this case there should be a resistor to pull the gate down until the I/O pin is set up as an output, to prevent power-up glitches. This resistor can be much larger, 1 MΩ or more. MOSFETs need almost no current (“flow”), they just need voltage (“pressure”) to function.

The benefit of these circuits is that you can easily switch 5V, 12V, or even 24V – with an I/O pin which remains at very low voltage levels (say 1.8 to 3.3V)

In a perfect world, these would both be fine, and be very convenient: “1” is on, “0” is off.

Unfortunately, a transistor is not a perfect switch, so there will be some residual voltage drop over it (0.2..0.4V for the BJT, under 0.1V for the MOSFET). Also, the selected MOSFET has to switch on at low voltages – many types need 4V or more to fully switch on.

One problem with these “low-side” switches (i.e. in the ground wire), is that the circuits will start to float: with a small voltage drop over the transistor, all signal levels to this circuit will be raised slightly, and sometimes unpredictably. So if the circuit has any other connections to the microcontroller (or anything else, for that matter), then these levels will vary somewhat. It’s like shaking hands with someone while standing on a treadmill :)

What’s even worse: when the power is switched off, the circuit ends up being tied to its power supply “+” side, but disconnected from ground – this can cause all sorts of nasty problems with electricty finding its way through other connected pins.

Having said that: if the circuit to be switched has no other outside connections, then either of these setups will work just fine. One example is LEDs and LED strips – which is why the MOSFET Plug uses N-MOSFETs exactly as outlined here. All you need to do is stick with “Common Anode” type RGB LED’s, i.e. tie all the “+” pins (anodes) together to the power supply, and let the MOSFETs do the switching between the “-” pins (cathodes) and GND.

For anything more elaborate, we need “high-side switching” – coming up tomorrow!

Switching with a lower voltage

In Hardware on Nov 10, 2012 at 00:01

The SMA Bluetooth relay described yesterday has to switch the power to the RN-42 module using an I/O pin on the ATmega. Currents are fairly small: up to about 50 mA.

I tried directly powering the RN42 from two I/O pins in parallel, but it looks like they don’t have enough current drive capacity for this purpose. So the task is to find a simple way to switch on power somehow.

The simplest solution would seem to be a P-MOSFET in the “high side” supply, i.e. between PWR and the RN-42’s supply pin, but there is a problem: PWR will be somewhere between 3.3 and 5V (actually it’s more like between 3.6 and 4.0V with the 3xAA Eneloop batteries I’m using), but the I/O pin on the ATmega won’t be more than 3.3V – since the ATmega sits behind a 3.3V voltage regulator. I tried the P-MOSFET, before realising that it’d always be driven on – the I/O pin voltage is sufficiently low to switch the MOSFET on, even with a logic “1” – not good!

MOSFETs are driven by voltage whereas transistors are driven by current, so an obvious thing to try next is to use a PNP transistor in more or less the same configuration. Voltage differences wouldn’t be so critical, if no current flows. Also, there’s the extra base-to-emitter voltage drop or so that each normal transistor has. Still, a simple PNP transistor might switch on if the difference in voltage is large enough – this can be overcome with a PNP Darlington transistor, which is simply two PNP transistors, cascaded in a certain way. The property of these things – apart from their high amplification (hFE) – is that you need to drive the base with a slightly larger voltage. A lower voltage in this case, with PNP types. Could also have used two discrete PNP transistors.

Here’s the circuit:

Screen Shot 2012 11 07 at 15 15 47

And sure enough, it works. I happened to have an SMD “BCV 28” lying around:

DSC 4238

The 10 kΩ resistor in series with the base limits the drive current to under 1 mA – more than enough to drive the Darlington into saturation, i.e. the state where the collector-to-emitter voltage drop is at its lowest.

That’s it. Every 5 minutes, a reading arrives on the central JeeLink, as shown by JeeMon:

Screen Shot 2012 11 07 at 15 21 56

Drat, cloudy again! Oh well, just the usual gray November day…

Using watchdog resets

In AVR, Hardware, Software on Nov 9, 2012 at 00:01

The Bluetooth readout node running the “smaRelay” code is ready to go:

DSC 4237

As mentioned yesterday, I’ve decided to take the easy way out and completely power down the Bluetooth module and go through a full powerup/connect/readout cycle about once every 5 minutes.

The advantage for me of a battery-powered unit, is that I don’t have to locate this thing near a power outlet – it can be placed out of sight, unobtrusively doing its work.

I seem to have developed an allergy for power cables and wall warts all over the place…

The power on/off logic held a little surprise, about which I’ll report tomorrow.

Here’s is the new part of the updated smaRelay.ino sketch:

Screen Shot 2012 11 07 at 14 27 33

Quite a different use of the watchdog here:

  • on powerup, go into low-power mode and wait for 5 minutes
  • prepare the watchdog to perform a reset in 8 seconds
  • power up Bluetooth, connect to the SMA, and read out some values
  • then power down Bluetooth and power up the RFM12 radio
  • send out a data packet over RF12
  • lastly, turn the radio off and power down
  • let the watchdog do the reset and make this sketch start over

This approach has as “benefit” that it’ll fail gracefully: even if anything goes wrong and things hang, the hardware watchdog will pull the ATmega out of that hole and restart it, which then starts off again by entering an ultra-low power mode for 5 minutes. So even if the SMA is turned off, this sketch won’t be on more than about 1% of the time.

Here’s the energy consumption measurement of an actual readout cycle:


The readings are a bit noisy – perhaps because I had to use 1 mV/div over a 1 Ω resistor (the 10 Ω used before interfered too much with this new power-switching code).

As you can see, the whole operation takes about 4 seconds. IOW, this node is consuming 153 milli-Coulombs every 300 seconds. That’s 0.5 mC/sec, or 0.5 mA on average. So the estimated lifetime on 3x AA 1900 mAh Eneloops is 3800 hours – some 5 months.

Update – The first set of batteries lasted until March 18th, 2013 – i.e. over 4 months.

Good enough for now. Deploy and forget. Onwards!

Bluetooth power consumption

In Hardware on Nov 8, 2012 at 00:01

The Bluetooth module in yesterday’s setup has a nasty power consumption profile:


The yellow line is total power consumption, which to over 60 mA at times, and the smaRelay.ino sketch is querying the SMA inverter roughly every 10.5 seconds. The drop in baseline is the ATmega going to sleep as it waits for the next period, so you can clearly see what the Bluetooth module is doing – while kept on and connected to the SMA in fact.

I’m not sure that the Hameg’s math integral function is up to summing such fast-changing values, but it’s the best I’ve got to measure power consumption here at JeeLabs at the moment (well, either this or tracking the discharge on a hefty electrolytic capacitor).

Note the baseline consumption of about 5 mA, and the frequent but highly irregular brief power consumption pulses. That’s BT doing its frequency hopping thing, I assume.

Anyway, my goal was to get an estimate of the average power consumption, so here we go:

  • two cursors were used to peg the integral (summed) value over one cycle
  • charge usage over one 10.5 second period turns out to be 134 millicoulombs
  • that’s 134 / 10.5 ≈ 12.75 mC per second, i.e. 12.75 mA average

Whoa… not much of a candidate for battery-power this way!

That leaves a couple of options:

  • just power it via a USB adapter and move on
  • explore the RN-42’s low-power mode, which is claimed to go as low as 300 µA
  • completely turn off power to the RN-42

I’m inclined to go for the latter. I don’t really need solar PV readings that often, since the SMA accumulates its daily and total generated power anyway. And during the night, all this reading and reporting activity is also quite useless.

That would also solve – or rather: work around – the intermittent problems present in the current code, in which the sketch stops relaying after a few minutes. It always seems to get stuck after a while, waiting for incoming data from the Bluetooth module.

One readout every 10 minutes would probably be plenty for me, and since the SMA has a time-of-day clock which can be read out over BT, I can stop readouts during the night (or even simpler: add an LDR and switch off when it’s dark).

It looks like powering up, establishing a connection, and reading out one set of values can be done in under 6 seconds, so that leads to a 1% duty cycle. Let’s say 200 µA on average – this ought to run a year on 3x AA Eneloops.

Sometimes, sophisticated solutions are just distractions – whatever works, right?

Relaying SMA data as RF12 packets

In Hardware, Software on Nov 7, 2012 at 00:01

Yesterday’s post shows how to read out the SMA solar PV inverter via Bluetooth. The idea was to install this on the Mac Mini JeeLabs server, which happens to be in range of the SMA inverter (one floor below). But that brings up a little dilemma.

Install a potentially kernel-panic-generating utility on the main JeeLabs server? Nah…

I don’t really care whether this issue gets fixed. I don’t want to have the web server go down for something as silly as this, and since it’s a kernel panic, there’s no point trying to move the logic into a Linux VM – the problem is more likely in Apple’s Bluetooth / WiFi stack, which will get used no matter how I access things.

The alternative is to implement a little “SMA Relay” using a JeeNode with a Bluetooth module attached to it, which drives the whole protocol and then broadcasts results periodically over RF12. That way I can figure out and control it.

I tried to use the SoftwareSerial library built into the newer Arduino IDE releases, but ran into problems with lost bytes – even with the software UART speed down to 19200 baud.

So I ended up first debugging the code on an Arduino Mega, which has multiple hardware UARTs and allows good ol’ debugging-with-print-statements, sending out that debug info over USB, while a separate hardware UART deals with all communication to and from the Bluetooth module.

Once that worked, all debugging statements were removed and the serial Bluetooth was switched to the main (and only) UART of the JeeNode. The extra 10 kΩ R’s in the RX and TX lines allow hooking up a USB BUB for re-flashing. The BUB will simply overrule, but needs to be removed to try things out:

DSC 4236

(the Bluetooth module I used in this setup is Sparkfun’s BlueSMiRF Silver)

Next step was to add a little driver to JeeMon again, the aging-but-still-working Tcl-based home monitoring setup at JeeLabs. Fairly straightforward, since it merely needs to extract a couple of 16-bit unsigned ints from incoming packets:

Screen Shot 2012 11 06 at 10 39 10

And sure enough, data is coming in (time in UTC):

Screen Shot 2012 11 06 at 10 32 35

… and properly decoded:

Screen Shot 2012 11 06 at 10 33 08

The ATmega code has been added as example to JeeLib on GitHub, see smaRelay.ino.

I’m still debugging some details, as the Arduino sketch often stops working after a few minutes. I suspect that some sort of timeout and retry is needed, in case Bluetooth comms get lost occasionally. Bluetooth range is only a few meters, especially with the re-inforced concrete floors and walls here at JeeLabs.

Anyhow, it’s a start. Suggestions and improvements are always welcome!

Accessing the SMA inverter via Bluetooth

In Software on Nov 6, 2012 at 00:01

As pointed out in recent comments, the SMA solar PV inverter can be accessed over Bluetooth. This offers various goodies, such as reading out the daily yield and the voltage / power generation per MPP tracker. Since the SB5000TL has two of them, and my panels are split between 12 east and 10 west, I am definitely interested in seeing how they perform.

Besides, it’s fun and fairly easy to do. How hard could reading out a Bluetooth stream be?

Well, before embarking on the JeeNode/Arduino readout, I decided to first try the built-in Bluetooth of my Mac laptop, which is used by the keyboard and mouse anyway.

I looked at a number of examples out there, but didn’t really like any of ’em – they looked far too complex and elaborate for the task at hand. This looked like a wheel yearning to be re-invented… heh ;)

The trouble is that the protocol is fully packetized, checksummed, etc. The way it was set up, this seems to also allow managing multiple inverters in a solar farm. Nothing I care about, but I can see the value and applicability of such an approach.

So what it comes down to is to send a bunch of hex bytes in just the right order and with just the right checksums, and then pulling out a few values from what comes back by only decoding what is relevant. Fortunately, the Nanode SMA PV Monitor project on GitHub by Stuart Pittaway already did much of this (and a lot more).

I used some templating techniques (in good old C) which are probably worth a separate post, to generate the proper data packets to connect, initialise, login, and ask for specific results. And here’s what I got – after a lot of head-scratching and peering at hex dumps:

    $ make
    cc -o bluesman main.cpp
    logged in
    daily yield: 2886 Wh @ Sun Jun 29 15:38:03     394803
    total generated power: 75516 W
    AC power: 432 W
    451f: DC voltage = 181.00 V
    451f: DC voltage = 142.62 V
    DC voltage
    251e: DC power = 252 W
    251e: DC power = 177 W
    DC power

The clock was junk at the time, but as you can see there are some nice bits of info in there.

One major inconvenience was that my 11″ MacBook Air tended to crash every once in a while. And in the worst possible way: hard kernel panic -> total reboot needed -> all unsaved data lost. Yikes! Hey Apple, get your stuff solid, will ya – this is awful!

The workaround appears to be to disable wireless and not exit the app while data is coming in. Sounds awfully similar to the kernel panics I can generate by disconnecting an FTDI USB cable or BUB, BTW. Needless to say, these disruptions are extremely irritating while trying to debug new code.

Next step… getting this stuff to work on an ATmega – stay tuned!

Verifying synchronisation over time

In AVR, Software on Nov 5, 2012 at 00:01

(Perhaps this post should be called “Debugging with a scope, revisited” …)

The syncRecv.ino sketch developed over the last few days is shaping up nicely. I’ve been testing it with the homePower transmitter, which periodically sends out electricity measurements over wireless.

Packets are sent out every 3 seconds, except when there have been no new pulses from any of the three 2000 pulse/kWh counters I’m using. So normally, a packet is expected once every second, but at night when power consumption drops to around 100 Watt, only every third or fourth measurement will actually lead to a transmission.

The logic I’m using was specifically chosen to deal with this particular case, and the result is a pretty simple sketch (under 200 LOC) which seems to work out surprisingly well.

How well? Time to fire up that oscilloscope again:


This is a current measurement, collected over about half an hour, i.e. over 500 reception attempts. The screen was set in 10s trace persistence mode (with “false colors” and “background” enabled to highlight the most recent traces and keep showing each one, so all the triggers are superimposed on one another.

These samples were taken with about 300 W consumption (i.e. 600 pulses per hour, one per 6s on average), so the transmitter was indeed skipping packets fairly regularly.

Here’s a typical single trigger, giving a bit more detail for one reception:


Lots of things one can deduce from these images:

  • the mid-level current consumption is ≈ 8 mA, that’s the ATmega running
  • the high-level current increases by another 11 mA for the RFM12B radio
  • almost all receptions are within 8..12 ms
  • most missing packets cause the receiver to stay on for up to some 18 ms
  • on a few occasions, the reception window is doubled
  • when that happens, the receiver can be on, but still no more than 40 ms
  • the 5 ms after proper reception are used to send out info over serial
  • the ATmega is on for less than 20 ms most of the time (and never over 50 ms)
  • it looks like the longer receptions happened no more than 5 times

If you ignore the outliers, you can see that the receiver stays on well under 15 ms on average, and the ATmega well under 20 ms.

This translates to a 0.5% duty cycle with 3s transmissions, or a 200-fold reduction in power over leaving the ATmega and RFM12B on all the time. To put that in perspective: on average, this setup will draw about 0.1 mA (instead of 20 mA), while still receiving those packets coming in every 3 seconds or so. Not bad, eh?

There’s always room for improvement: the ATmega could be put to sleep while the radio is receiving (it’s going to be idling most of that time anyway). And of course the serial port debugging output should be turned off for real use. Such optimisations might halve the remaining power consumption – diminishing returns, clearly!

But hey, enough is enough. I’m going to integrate this mechanism into the homeGraph.ino sketch – and expect to achieve at least 3 months of run time on 3x AA (i.e. an average current consumption of under 1 mA total, including the GLCD).

Plenty for me – better than both my wireless keyboard and mouse, in fact.

Predicting the next packet time

In AVR, Software on Nov 4, 2012 at 00:01

The problem described in the last post was caused by the receiver losing track of what the transmitter is doing when packets are missing.

Here are the key parts of an improved syncRecv.ino, with several refinements:

  • on powerup, an estimate is chosen as before, using a few iterations
  • we save the best estimate in EEPROM and reuse it after a restart
  • the loop then switches to predicting the next arrivals each time
  • failure to capture within 20 packets restarts a full estimation run
  • every 4, 8, 12, etc lost packets, the window size is doubled (up to a maximum)
  • on each successful capture, the window size is halved (down to a minimum)
  • the cycle time estimate is averaged with a 6-fold smoothing filter

The estimation logic is roughly the same as before:

Screen Shot 2012 10 31 at 15 15 10

And here’s the main loop, with the rest of the logic described above:

Screen Shot 2012 10 31 at 15 15 25

The code will lock onto packets even when some of them are missing. It does so by simply ignoring the last miss, and predicting another packet again exactly one cycle later.

Here are some first results, illustrating synchronisation loss:

Screen Shot 2012 10 31 at 15 20 55

Still some rough spots, but this algorithm should receive properly with a < 1% on/off cycle.

Losing sync is bad

In AVR, Software on Nov 3, 2012 at 00:01

Yesterday’s code has a nasty side-effect of getting into a corner out of which it almost can’t recover at times. Here’s a somewhat longer run, edited for brevity (note the indented lines):

Screen Shot 2012 10 30 at 23 52 35

On one occasion, it took 238 retries (12 minutes) to get back into sync!

Note how the code is sailing slightly too close to the edge, by always being at 0 or 1 ms from the expected time, just because this watchdog timer “happens” to run at a particular rate.

One fix is to gradually widen the window again when too many receptions in a row fail:

Screen Shot 2012 10 30 at 23 56 44

The other step is to always leave a bit more time when calculating a new estimate – it’s better to err on the side of receiving too early than to miss the start of the packet:

Screen Shot 2012 10 30 at 23 55 53

Whoops, looks like this doesn’t get it right – but it illustrates the widening:

Screen Shot 2012 10 31 at 00 02 29

This change works a lot better (i.e. plus 2, instead of minus 2):

Screen Shot 2012 10 31 at 00 07 05

Here’s the sample output:

Screen Shot 2012 10 31 at 00 05 35

The syncRecv code on GitHub has been updated with this tweak. More to come!

Synchronised reception

In AVR, Software on Nov 2, 2012 at 00:01

That homeGraph setup brought out the need to somehow synchronise a receiver to the transmitter, as illustrated in a recent post. See also this forum discussion, which made me dive in a little deeper.

Let’s take a step back and summarise what this is this all about…

The basic idea is that if the transmitter is transmitting in a fixed cycle, then the receiver need not be active more than a small time window before and after the expected transmission. This might make it possible to reduce power consumption by two orders of magnitude, or more.

Here’s the initial syncRecv.ino sketch I came up with, now also added to JeeLib:

Screen Shot 2012 10 30 at 22 55 17

The idea is to turn the radio off for T – W milliseconds after a packet comes in, where T is the current cycle time estimate, and W the current +/- window size, and then wait for a packet to come in, but no later than T + W.

Each time we succeed, we get a better estimate, and can reduce the window size. Down to the minimum 16 ms, which is the granularity of the watchdog timer. For that same reason, time T is truncated to multiples of 16 ms as well. We simply cannot deal with time more accurately if all we have is the watchdog.

Here are some results from three different JeeNodes:

Screen Shot 2012 10 30 at 22 48 04

Screen Shot 2012 10 30 at 22 42 44

Screen Shot 2012 10 30 at 22 39 44

Screen Shot 2012 10 30 at 22 37 04

(whoops – the offset sign is the wrong way around, because I messed up the subtraction)

Note that for timing purposes, this sketch first waits indefinitely for a packet to come in. Since the transmitter doesn’t always send out a sketch, some of these measurement attempts fail – as indicated by multiple R’s.

That last example is a bit surprising: it was modified to run without powering down the radio in between reception attempts. What it seems to indicate is that reception works better when the radio is not put to sleep – not sure why. Or maybe the change in current consumption affects things?

As you can see, those watchdog timers vary quite a lot across different ATmega chips. These tests were done in about 15 minutes, with the sending sketch (and home power consumption levels) about the same during the entire period.

Still, these results look promising. Seems like we could get the estimates down to a few milliseconds and run the watchdog at its full 16 ms resolution. With the radio on less than 10 ms per 3000 ms, we’d get a 300-fold reduction in power consumption. Battery powered reception may be feasible after all!

New Flukso meter at JeeLabs

In Hardware on Nov 1, 2012 at 00:01

The Flukso is a little open-source box which can read out a couple of CT current clamps and/or pulse counters to provides electricity / gas / water consumption details – via a JSON/REST interface, either locally or on the Flukso site (private or shared, your call):

Flm02a top

The design is based on the Dragino, and includes an ATmega piggy-back board with extra circuitry specifically for reading out current clamps. It’s not every day that you see designs which can actually deal with power outages in such a way that the last readings get saved to EEPROM in the last few milliseconds – as the system is going down! – but that’s exactly what the Flukso does, showing its great attention to detail.

The Flukso meter’s designer, Bart Van Der Meerssche, is also the driving force behind the Electro:camp meetings, so we had some opportunities to chat and dream about the future these past few days. Lots of interesting options and wild ideas floating around. With Linux in the equation, a lot more sophistication becomes feasible.

To have a better test situation, I’ve decided to add a Flukso setup to JeeLabs, which is in fact trivial since it can connect over WiFi. Power consumption is under 3 W:

DSC 4231

I added the three current clamps as follows (consumption only, no PV solar yield for now):

  1. One 50 Amp CT clamp for the RCD-protected groups 1..3
  2. One 50 Amp CT clamp for the non-RCD-protected groups 4..7
  3. One 50 Amp CT clamp for the induction cooker, group 9

The total should match what I’m measuring with my other 2 pulse counters.

The daughterboard is a prototype with on-board RFM12B (software is work-in-progress).

The other news is that the pulse counter wiring has been fixed, so this is now correct:

DSC 4227

That’s an early morning with heavy clouds. Hey, where’s that sun when you need it!