Computing stuff tied to the physical world

Parallel WSN choices

In Hardware on May 20, 2009 at 00:01

Just came across the eKo range of commercial products by Crossbow. Interesting how the eKo and Jee-Node/-Hub/-Mon designs almost exactly overlap – well, seen from a large distance anyway.

Their “JeeNode”:

Picture 2.png

Their “JeeHub”:

Picture 3.png

Their Linux server hardware and software:

Picture 4.png

Of course the eKo products are a million times more mature and advanced than the Jee-stuff which isn’t even a product, but still…

▶ View 2 Comments

JeeHub Lite

In AVR, Hardware on May 4, 2009 at 00:01

The current JeeHub includes a MMnet1001 Linux module, which is great for running JeeMon. But for an even more basic always-on system, it’s possible to forego the Linux part – here’s the idea:

The whole thing is powered from USB, which must remain on permanently. There is no need for the attached computer to be on all the time, because incoming data can be saved into the on-board DataFlash memory. Depending on data volume and memory size, it ought to be possible to have a few days of autonomy.

The 433 and 868 MHz OOK receivers make this thing suitable for receiving weather data from off-the-shelf components, as well as responding to KAKU and FS20 remotes (and others like them). And since the RFM12B is so versatile, it can also send out 433 and 868 MHz OOK commands to those systems.

One thing I forgot to add in the above diagram, is a DCF77 receiver – or perhaps using a crystal or RTC. As has been mentioned a few times, the JeeHub needs to have a fairly accurate sense of time to do its main job of collecting real-time readings.

Only a minimal number of signals/ports are needed for the main tasks, there are several pins left to connect a few extra sensors.

The main bottleneck will probably be the amount of code needed to make such a JeeHub Lite perform basic home automation tasks. An Atmega328 might help a bit.

The normal monitoring and reporting work is still done with JeeMon, but now running on the attached PC (which can be Windows, Mac OS X, Linux, whatever). In such a configuration JeeMon must catch-up and extract saved data from the DataFlash in the JeeNode whenever it is launched.

RF woes – solved!

In Software on Apr 28, 2009 at 00:01

Turns out my troubles with using the 868 MHz OOK radio next to the RFM12B module were caused by a silly software mistake (I mixed up the port assignments). No hardware or RF issues after all.

Latest sample output:

Picture 2.png

(the VOLT and BARO readings are bogus because the hardware is not connected)

As you can see, it’s receiving both packet types now. Still some trickiness with allocating the port signals properly, since some lines do require specific pins: the OOK receiver uses the analog comparator, but the ADC gets used as well, so I’ll need to adjust things a bit to use a pin change interrupt instead (used a crude workaround for now).

So now the basics are there to receive all types of signals with a single JeeHub: packets from other JeeNodes using the RFM12B, an 868 MHz OOK receiver for weather data and the FS20 remotes, and a 433 MHz OOK receiver for picking up KAKU remote commands (and possibly some other cheap weather sensors later).

▶ View 2 Comments

RF woes

In Hardware on Apr 22, 2009 at 00:01

I’ve been trying to set up a JeeNode as forwarding gateway from the various 868 MHz OOK signals into the RFM12B packet format. The lower left block in the following diagram, that is:

RF woes.png

But I can’t seem to get it to work…

The 868 MHz OOk receiver and the RFM12B transceiver both work fine. In isolation. But when I run code which enables both receivers, the OOK module stops receiving proper data.

There was a mention somewhere about having to provide a really clean power level to the OOK radio, so I’ll probably try again with a separate regulator for it. Or maybe move the two radio modules further apart, since they are both to the same 868 MHz frequency band.

Can’t think of much else. Hm, again.

JeeHub modularity

In Hardware on Apr 13, 2009 at 00:01

The current JeeHub setup is a bit of a mish-mash:

Picture 1.png

It includes the RFM12B radio of course, to receive data from all JeeNodes. But it also includes a 433 MHz OOK receiver, a BMP085 pressure / temperature sensor, and a DCF77 clock signal receiver.

Trouble is, these things interfere with each other …

The 433 MHz OOK receiver seems to severely degrade RFM12B reception – so I yanked it out. Didn’t have an immediate need to decode KAKU signals anyway.

But worse, the DCF77 receiver also fails to work inside the current JeeHub enclosure. I suspect that either radiated RF power or some messy interference on the power supply levels is preventing it from detecting the 77 kHz signal. I really want DCF77 as reference clock, but I haven’t been able to make any use of it so far.

It looks like there’s too much going on in the current JeeHub prototype, electrically or electro-magnetically.

This is silly – and there’s a very simple way out: move some data sources / sensors to a separate JeeNode. After all, the JeeHub has been doing one thing really well for some time now, i.e. collecting data from remote JeeNodes via the RFM12B:

Picture 2.png

All I need to do is add support for OOK receivers plus BMP085 to the pulse software, and then attach them to one of the remote JeeNodes. Would also make it possible to connect both 433 and 868 MHz OOK receivers.

Haven’t made up my mind about the DCF77 receiver though. Having it tightly integrated with the JeeHub still makes sense – it’s where all the data comes through and gets time-stamped. Maybe the DCF radio will start to work once I disconnect all the other cruft. If not, I’ll either move it physically away from the rest of the JeeHub or reassign it to a remote JeeNode as well.

Surprising, how physical / electrical aspects are starting to impose themselves…

▶ View 3 Comments

New "jeelab" mailing list

In News on Apr 8, 2009 at 00:01

For those who want to share their interest / questions / ideas w.r.t. any of the Jee Labs projects, I’ve set up a mailing list at Google Groups:

Picture 2.png

See http://groups.google.com/group/jeelab/about – if you don’t want to sign in to Google to join this list, let me know and I’ll add your email address manually.

Update – the mailing list has been replaced by the Jee Labs discussion forums at http://talk.jeelabs.net/.

Let's pick a few more names

In Hardware, Software on Mar 20, 2009 at 00:01

Not to worry: I’m not going to go crazy on the Jee<blah> naming used so far…

But I do need to give the different pieces some name. To be able to refer to them in these posts, but more importantly as names for the software for all this. I also need to introduce a basic structure (and some limits).

Picture 1.png

Here goes:

A JeeNode is this ATmega-with-4-ports-and-RFM12B thing. There may be lots of JeeNodes for various purposes. They can communicate with each other via wireless.
I use the Ports and RF12 libraries with JeeNodes, although this is not a hard requirement – both the JeeNodes and the libraries can be used in numerous other ways.
Each JeeNode has a letter ‘A’ to ‘Z’ assigned to it as node ID. Nodes are usually given a unique ID to avoid mixups, but this is not strictly required.
There can be up to 250 separate groups of JeeNodes. Nodes can only communicate with other nodes in the same group. Gateways between groups could be implemented later, if needed.
Most nodes will be pulse JeeNodes, i.e. running a specific piece of software called – you guessed it – “pulse”, which continuously monitors some attached sensors and reports the readings via wireless.
The JeeHub is either a JeeNode by itself connected to a Mac, Windows, or Linux PC via USB, or a JeeNode connected to Ethernet via a small dedicated Linux module.
This JeeNode-as-part-of-a-JeeHub is called the central node from now on. It runs a specific software configuration, also called “central”. There should always be exactly one central node.
The software running on the Mac, Windows, or Linux machine(s) is called the server from now on (how original, eh?). It consists of a system-dependent executable runtime called JeeMon plus the code and data for the application itself.

It probably doesn’t hurt to reiterate that “JeeNode” and “JeeHub” are hardware, whereas “JeeMon” and the Ports / RF12 libraries are software.

The above names are also used in the source tree repository now.

There are no doubt still infinitely many ways to lead you astray, but I hope that these definitions will help me place everything into an increasingly coherent context.

Energy monitoring

In Hardware, Software on Mar 19, 2009 at 00:01

This setup has been running here for a few months:

Picture 1.png

A light reflectance sensor detects the revolutions of the electricity meter. It is hooked up to a JeeNode, which counts the revolutions and sends its counter value to the central JeeHub node every 10 seconds.

For basic tracking of total electricity consumption, that’s all you need. Occasional packet loss has no influence on aggregated results since the counts are tracked near the sensor. The same works for gas and water consumption.

However, I also want to get a good estimate of the consumption rates in real time. This can be done by timing the rotation period, at millisecond resolution preferably. So the first improvement a while back was to let the sensor JeeNode send packets the moment a revolution is detected, and not just every 10 seconds. That way the central node can do millisecond-resolution timing, which is easy with a stable clock and Linux.

This works reasonably well, but only if reception is nearly perfect. Any lost packet messes up the preceding and the following timing estimate. Only with two packets sent right after a counter change can the period be measured. I kept the periodic 10-second sends enabled as fail-safe, but these late packets are useless for real-time rate calculations.

The next improvement was to let the sensor node do the millisecond timing. It wasn’t obvious how, though, as sensor nodes do not have a very accurate clock: a ceramic resonator has around 0.5% accuracy and probably also some temperature sensitivity. Turns out that it doesn’t really matter: as long as these real-time estimates are not summed together, errors won’t propagate any further.

So now the sensor node sends both the counter value and the last rotation period in milliseconds. The counter value is perfect for long term energy consumption monitoring – after all, it’s exactly the same as what the power company will use to charge me later. The millisecond period is a good estimate of current electricity use, and probably fairly stable from one measurement to the next. Absolute accuracy and drift are not so important here, because the main use of the real-time estimate is to see small changes as they occur.

Here’s an example of the entries logged on the JeeHub:

Picture 2.png

All log entries have the format “T [time] [millis] [type] …”. Where “time” is the time on the Linux module of the JeeHub in YYYYMMDD.HHMMSS format and “millis” is the current millisecond counter on the JeeNode inside the JeeHub. This might seem redundant, but keep in mind that the Linux module is not always on – it can pull data off the JeeNode when started up again. Additional timing data will come from the on-board DCF77 receiver once that is fully operational. So there are three sources of timing information inside the JeeHub: Linux, JeeNode, and the DCF77 atomic-clock reception. Each have different levels of accuracy and drift, but because all of them are logged, this can be fully determined later.

The raw text log files grow by around 0.5 Mbyte/day right now. Gzip compresses them to roughly 20% of that, so they can easily be kept around: the MMnet1001’s flash memory will hold several years of these full-detail logs.

Back to the actual log entries now.

The “HM2” packets are from the electricity & gas metering node. Packets 56, 57, 58, 60, 61, 62, 63, 0, and 1 were lost (more on that later). The electricity revolutions went from 132 to 136 in this time frame, and the gas counter stayed at 139. The rotation time for the electricity meter in packet 54 was 74*256+209 = 19,153 ms. There is a third slot of four 0’s in each packet for measuring water consumption, currently waiting for a new sensor-enabled flow meter to be installed.

Looking at some calculated results, I think this setup will be able to track real-time power consumption with a resolution of under 1 Watt. The resolution actually improves with lower energy consumption rates – precisely what you’d want.

The VOLT and BARO entries are from sensors within the JeeHub. The ALT entry is weather data from a remote sensor and comes from another system for now.

A further improvement with the energy data was to switch to the latest RF12 library code and to use acknowledgements. Every time a packet has been properly acknowledged, the sensor node stops sending data until one of the counters changes. When no ack is received, data is resent every 3 seconds. With good reception most of the time, the number of packets sent (and logged) willl be quite small. And with packet loss, the data will usually arrive 3 or 6 seconds later anyway. Note how in the above log file excerpt each counter value from 132 through 136 was recorded, but retransmits were needed to get counts 134 and 135 to the JeeHub.

Final point: the wireless communication speed was raised from 38400 to 57600 baud – and reception appears to be at least as good as before.

Packet loss – I’ve been seeing long stretches of fairly bad reception at times. Until I realized that these coincided with our wireless headphone set being turned on. I don’t know what that thing is sending out into the ether, but it’s clearly interfering with the 868 MHz communication between JeeNodes. Oh well – it’s a good test for dealing with missed packets…

▶ View 5 Comments

JeeHub enclosure

In Hardware on Mar 18, 2009 at 00:01

Ok, so I wanted to wrap the JeeHub prototype into some sort of enclosure…

JeeHub enclosure

Yeah, why not, a cardboard box.

Got a couple of these from SparkFun, from whom I’ve ordered a few times. The fit is very tight:

JeeHub enclosure

And all connectors come out on one side:

JeeHub enclosure

From left to right:

Access to the FTDI connector of the JeeNode
The ethernet port of the MMnet1001 Linux module
A mini USB plug, only used to supply power
DC power, has to be between 5 and 6 volt

In normal use, only the Ethernet and USB ports are hooked up. But even without anything connected, the JeeNode will continue to receive and log data packets (once I finish the dataflash code, that is).

Collecting data with JeeMon

In Software on Mar 17, 2009 at 00:01

This is the information I’m currently tracking real-time:

Picture 1.png

(it’s in Dutch, but you can probably guess most of these)

And this is the setup I’m using for it:

Picture 2.png

The NAS in there is used as gateway to pass 868 MHz OOK data from a CUL receiver, which decodes signals from my KS300 weather station plus a couple of S300 temperature/humidity sensors. This will one day be replaced by on-board reception on the JeeHub, so that only the JeeHub needs to stay powered up at all times. All other sensors are hooked up to a couple of JeeNodes which transmit the readings wirelessly.

The Mac, Windows, Linux, and the JeeHub all run identical copies of the software, which is called “JeeMon”. It has a built-in web server, an embedded database, and a flexible set of network functions.

Each JeeMon instance will automatically self-update to the latest version on startup. During development on the Mac the JeeHub acts as transparent proxy, as if the different sensors were connected directly to the Mac (through a little Tcl-based system called “Tequila”). Once ready, the latest JeeMon release is wrapped into one file and placed on the internet. Finally, a restart of the JeeHub completes the upgrade.

That first screen dump above is a small test app on my Mac which bypasses JeeMon and connects directly to the JeeHub as Tequila client. I keep it open to check that data is coming in and gets saved on the JeeHub.

So this is the big picture for collecting energy/gas/water and environmental data in the house. The software can run on practically anything, can be accessed with a browser anywhere, and with proper security in place the various pieces can be connected and used across any network topology.

All of the above is working this very moment. The major task ahead is the full-scale processing, presentation, and interaction of it all. But that can now conveniently be done on my development machine, with HTML, CSS, JavaScript, etc.

Anyway, IMO this is a very flexible foundation for a 1-watt home monitoring server.

JeeHub software

In AVR, Software on Mar 13, 2009 at 00:01

Well, I’m quite pleased with how the software on the AVR side of the JeeHub is working out. When you look at the collapsed version of the current 270 lines of C++ code, you get this:

Picture 1.png

You can see the full source code here.

Everything becomes a DataSource, i.e. an object which is polled periodically and which can then submit readings for further storage or processing. It’s trivial to add more data sources without affecting any existing code.

On startup each data source is created and registers itself automatically. For example, the “power” object is set up to get polled/measured every 10 seconds. When polled, it takes a few ADC voltage readings, puts these into a struct, and hands it to the DataSource::newReading() method.

Currently, newReading() does nothing more than turn around and call the show() method of that same data source, which then sends a plain text version of the data out over the serial line. For unattended operation, readings will need to be saved in dataflash memory for retrieval at some later moment.

The above code compiles to 8104 bytes on an ATmega168 so far. Lots of room remaining!

JeeHub prototype

In AVR, Software on Mar 12, 2009 at 00:01

Voilá – the first JeeHub prototype exists!

JeeHub prototype

(click on the image to see the annotated version)

It’s a pretty simple setup so far, as can be seen on the backside:

All the pieces work, at least in isolation. But the Linux board and the JeeNode haven’t been connected together yet (I need to figure out which serial port to use on the MMnet1001) and there is some strange interaction between the different interrupt-driven pieces on the JeeNode. Looks like both hardware and software troubles at this stage.

Here’s some output:

Picture 1.png

So it’s picking up RFM12B packets, reading external power and battery voltage levels, reading out temperature and barometric pressure, and decoding both the KlikAanKlikUit remote controls and the DCF77 clock signal. But there’s some stuff here which is definitely not kosher – could be interrupt handling issues.

Update – looks like gcc 4.3.0 has trouble with interrupt code. When I changed to 4.3.2, everything started working as expected.

There are also a few hardware glitches in this early prototype, such as the switching regulator interfering with the DCF77 receiver (will change to a linear one) and the 433 MHz radio interfering with the RFM12B (could add some decoupling or shielding).

This first version of the code has been added to the subversion repository.

JeeHub physical layout

In Hardware on Mar 11, 2009 at 00:01

I’m going to try and fit all the components of the prototype JeeHub onto an 8.5 x 11.5 cm board to match a little enclosure that’s been lying around here:

Picture 1.png

It looks like everything fits – though it’s a bit tight:

JeeHub proto

There are two subsystems: the MMnet1001 Linux module and the rest, based on a JeeNode. The “rest” is actually the core of the system since it will run day-in day-out, whereas Linux could be shut down when no ethernet access or fancy reporting is needed.

The remaining components on this set-up are:

a DCF77 receiver for accurate time tracking
a BMP085 barometric pressure sensor
a 433 MHz OOK receiver to pick up remote control commands
a 8 Mbit dataflash memory as piggy-back on the JeeNode
a status LED

Furthermore, the RFM12B wireless on the JeeNode can control standard 433 & 868 MHz radio-frequency units such as power switches.

Everything can be powered from a USB port or a 5..6V DC wall plug. The automatic battery back-up allows running the JeeNode core off-the-grid. It’s not a priority but I want to see to what extent a JeeHub can function on its own power. The Linux module will be powered by a 5 -> 3.3V regulator, with a shutdown-pin controlled from the JeeNode.

Much of the basic software for the JeeNode core has already been written. But that’s of course just the tip of this “home-monitoring and -control” iceberg…

Don't mess with the boot flash

In Software on Mar 9, 2009 at 00:01

Ok, so I tried to get the on-board 4 Mb dataflash of the MMnet1001 working, just out of curiosity…

That was a bad idea. Nothing went wrong for a while, but then I rebooted:

And then nothing. Whoops, I messed with the boot partition instead of the dataflash chip! Thankfully, tech support at Propox helped me out quickly. It’s actually quite easy to re-flash the MMnet1001: connect a USB cable to the USB device (not host) port pins, and then simply jump through a few basic software hoops. Neat – well done.

Ah, that looks better:

Picture 1.png

And so on.

So the 4 Mb dataflash remains a mystery for now. No problem, the 1 Gb NAND flash is the main deal, and it works just fine. I’ve added about 7 Mb of my own so far, but that still leaves plenty of free space:

Picture 4.png

The hardware clock works fine, but it needs to be set once before reading it out to get rid of an error message. IOW, set the time zone in “/etc/config/system” (I used “CET-1”), then set the proper date and time with “date”, then finish with “busybox hwclock -w -u”. Note that the “/sbin/hwclock” version doesn’t work for setting the clock (it seems to be ok for readout).

On the software side, I now have a Tclkit build cross-compiled via the OpenWrt SDK and working on the MMnet1001, so the rest of the work can be done on my development system and then copied to this amazing little Linux, eh… box? plug? chip?

Anyway, all is well.

Thank you, Andrzey, for helping out an AT91 newbie ;)

Tiny embedded Linux

In Hardware on Mar 8, 2009 at 00:01

The MMnet1001 is an interesting little setup, based on the AT91 ARM chip. Here’s a close-up image, as provided by Propox:

The ethernet connector is about the only hint you get on how small this thing actually is. It’s shorter than a JeeNode (and it has a lot more hardware!).

I did not order the development board with this, given that I only need minimal hardware interfacing. So my first try was to simply hook up 3.3V power to J2-1 and J2-2 and plug in an Ethernet cable.

Hmm, nothing happens. No new device shows up on the LAN. It turns out that the bootstrap uses a debug serial line, and unfortunately in this revision the internal pull-up on the RX pin is not enabled. So the bootstrap picks up something on the line, and then waits for a complete command.

A 10 KΩ pull-up resistor on J1-13 fixes it. The user LED is flashing!

Now I can see a network device, but it won’t let me in via SSH and I don’t see an HTTPD server on port 80 either.

Ok, time to hook up the serial port @ 115200 baud via an FTDI cable. Aha!

Picture 3.png

(actually, this is the text I get after editing out some cruft from “/etc/issue”, not a first-time hookup)

Great. I’m in. And there’s a “dropbear” SSH. Ok, turns out the root password hasn’t been set – that’s easy to fix.

Yes! Now I can connect over ethernet at last:

Picture 5.png

Perfect. The serial line is no longer needed. This thing now boots up and lets me in via SSH without further tweaking.

Power consumption @ 3.3V is about 250 mA idle and 350 mA when running the CPU at full load. Here are some more system specs:

Picture 6.png

All is well. Onwards!

JeeHub mock-up

In Hardware on Mar 7, 2009 at 00:01

Meet the JeeHub, a self-contained server and gateway between networked JeeNodes and an Ethernet network:

JeeHub prototype

On the right is a Propox MMnet1001 module with an AT91 ARM processor, 64 Mb RAM, and 1 Gb flash, running Linux. On the left is a JeeNode, to be used as receiver and central node in the 868 MHz wireless connection with all other JeeNodes. It hasn’t been hooked up yet – this was just a mock-up to make a pretty picture.

The software on the JeeHub is portable, the same code also runs on Windows, Mac OS X, and Linux x86. The benefit of a JeeHub is that you don’t have to leave a PC on to collect / process / present all measurement data, or to control devices around the house. The JeeHub draws less than 1 watt of power and can be left on to provide all these functions around the clock. It has a built-in web server and database, and can connect to other systems on the local network as well as the internet to perform both interactive and unattended tasks.

The basic JeeHub hardware and software is working here now, with automatic self-updates for completely maintenance-free use, but things are still changing far too quickly to document them and turn this loose.

So yeah, this post is a bit of a teaser for now…

Home control shield

In AVR on Jan 22, 2009 at 16:37

The “home control” unit I mentioned recently was intended to have two shields, stacked as follows:

Unfortunately, the top shield with the I²C display turns out to either shield the underlying DCF77 receiver too much, or to generate too much interference, or both. So I’m going to leave it off:

As a result, this is just an Arduino Duemilanove with a “peripherals” proto-shield by Adafruit on top, containing:

an RFM12B 868 MHz radio module from HopeRF
a µSD socket with a 2 Gb card from Kingston
a DCF77 receiver from Pollin (pollin.de)

The resistors are used as level converters, since the SPI bus has to run at 3.3V. There’s also 3.3V regulator (look between the resistors and the pin header), because the µSD programming current can run up to 100 mA – more than the Arduino’s FTDI chip can supply.

All peripherals have been verified to work, separately as well as in combination.

So this unit now has all the pieces needed to log (and accurately timestamp) virtually unlimited amounts of data flying through RFM12B airspace… fully unattended, with just a USB or +9V power source.

Shield progress

In AVR, Hardware on Jan 14, 2009 at 15:57

The Arduino shield stack is shaping up. Here’s the peripheral shield, with all modules installed:

This now contains the following peripherals:

a µSD card reader with 2 Gb card (top right)
an RFM12B 868 MHz transceiver (bottom right)
an 433 MHz AM receiver (top left, hidden)
an 868 MHz AM transmitter (bottom left)
a DCF77 time code receiver (middle, with ferrite antenna)

The header pins support a status shield on top with a small 3×12 LCD display (connected via I²C) and some leds & buttons:

The whole setup, when stacked on an Arduino Duemilanove, now becomes this:

What it does will depend on the software written for it, obviously.

Shield stack

In AVR, Hardware on Jan 12, 2009 at 16:11

Here is the start of a new set of shields for the Arduino, based on Adafruit’s Proto Shield:

This little thing might one day become my “home monitoring & control server”, but that’s still a long way off. You can see a µSD card slot with 2 Gb of flash memory on the middle board, which will become the main peripheral shield. The top shield is for basic access and status info, and is optional. Additional shields could be inserted into this stack later.