Computing stuff tied to the physical world

Posts Tagged ‘Network’

Time-controlled transmissions

In Hardware, Software on Jun 27, 2011 at 00:01

Receiving data on remote battery-powered JeeNodes is a bit of a dilemma: you can’t just leave the receiver on, because it’ll quickly drain the battery. Compare this to sending, where nodes can easily run for months on end.

The difference is that with a remote node initiating all transmissions, it really only has to enable the RFM12B wireless module very briefly. With a 12..23 mA current drain, brevity matters!

So how do you get data from a central node, to remote and power-starved nodes?

One way is to poll: let the remote node ask for data, and return that data in an ACK packet as soon as asked. This will indeed work, and is probably the easiest way to implement that return data path towards remote nodes. One drawback is that if all nodes start polling a lot, the band may become overloaded and there will be more collisions.

Another approach is to agree on when to communicate. So now, the receiver again “polls” the airwaves, but now it tracks real time and knows when transmissions for it might occur. This is more complex, because it requires the transmitter(s) and receiver(s) to be in sync, and to stay in sync over time.

Note that both approaches imply a difficult trade-off between power consumption and responsiveness. Maximum responsiveness requires leaving the receiver on at all times – which just isn’t an option. But suppose we were able to stay in sync within 1 ms on both sides. The receiver would then only have to start 1 ms early, and wait up to 2 ms for a packet to come in. If it does this once a second, then it would still be on just 0.2% of the time, i.e. a 500-fold power saving.

Let’s try this out. Here’s the timedRecv.pde sketch (now in the RF12 library):

Screen Shot 2011 06 24 at 20.54.37

It listens for incoming packets, then goes into low-power mode for 2 seconds, then it starts listening again. The essential trick is to report two values as ACK to the sender: the time the receiver started listening (relative to that receiver’s notion of time), and the number of milliseconds it had to wait for the packet to arrive.

There’s no actual data processing – I’m just interested in the syncing bit here.

The sender side is in the timedSend.pde sketch:

Screen Shot 2011 06 24 at 20.58.17

This one tries to send a new packet each time the receiver is listening. If done right, the receiver will wake up at just the right time, and then go to sleep again. The ACK we get back in the sender contains valuable information, because it lets us see how accurate our timing was.

Here’s what I get when sending a new packet exactly 2,096 milliseconds after an ACK comes in:

Screen Shot 2011 06 24 at 20.39.33

Not bad, one ack for each packet sent out, and the receiver only has to wait about 6 milliseconds with its wireless receiver powered up. I’ve let it run for 15 minutes, and it didn’t miss a beat.

For some reason, send times need to be ≈ 2.1 s instead of the expected 2.0 s.

Now let’s try with 2,095 milliseconds:

Screen Shot 2011 06 24 at 20.37.17

Something strange is happening: there’s consistently 1 missed packet for each 5 successful ones!

My hunch is that there’s an interaction with the watchdog timer on the receiver end, which is used to power down for 2000 milliseconds. I suspect that when you ask it to run for 16 ms (the miminum), then it won’t actually synchronize its timer, but will fit the request into what is essentially a free-running counter.

There may also be some unforeseen interaction due to the ACKs which get sent back, i.e. there’s a complete round-trip involved in the above mechanism

Hmm… this will need further analysis.

I’m using a standard JeeNode on the receiving end, i.e. running at 16 MHz with a ceramic resonator (specs say it’s 0.5% accurate). On the sender side, where timing is much more important, I’m using a JeeLink which conveniently has an accurate 16 MHz crystal (specs say it’s 10 ppm, i.e. 0.001% accurate).

But still – even this simple example illustrates how a remote can receive data while keeping its wireless module off more than 99% of the time.

EtherCard library API

In Software on Jun 19, 2011 at 00:01

As you may have noticed in the last few weblog posts, the API of the EtherCard library has changed quite a bit lately. I’m not doing this to be different, but as part of my never-ending quest to try and simplify the calling interface and to reduce the code size of the library (these changes shaved several Kb off the compiled code).

The main change was to switch to a single global buffer for storing an outgoing Ethernet packet and for receiving the next packet from the controller. This removes the need to pass a buffer pointer to almost each of the many functions in the library.

Buffer space is scarce on an ATmega, so you have to be careful not to run out of memory, while still having a sufficiently large buffer to do meaningful things. The way it works now is that you have to allocate the global buffer in your main sketch:

Screen Shot 2011 06 15 at 09.32.45

This particular style was chosen because it allows the library to access the buffer easily, and more importantly: without requiring an intermediate pointer.

To make this work, you have to initialize the EtherCard library in the proper way. This is now done by calling the begin() function as part of your setup() code:

Screen Shot 2011 06 15 at 09.34.46

The ether variable is defined globally in the EtherCard.h header file. The begin() call also needs the MAC address to use for this unit. The simplest way to provide that is to define a static array at the top of the sketch with a suitable value (it has to be unique on your LAN):

Screen Shot 2011 06 15 at 09.37.18

Next, you can use DHCP to obtain an IP address and locate the gateway and DNS server:

Screen Shot 2011 06 15 at 09.39.27

The printIp() utility function can optionally be used to print some info on the Serial port.

If you are going to set up a server, then a fixed IP address might be preferable. There’s a new staticSetup() function you can use when not doing DHCP:

Screen Shot 2011 06 15 at 09.42.55

The gateway IP address is only needed if you’re going to access an IP address outside of your LAN, and the DNS IP addres is also optional (it’ll default to Google’s “8.8.8.8” DNS server if you do a DNS lookup). To omit values, pass a null pointer or leave the arguments off altogether:

Screen Shot 2011 06 15 at 09.45.55

Just remember to call either dhcpSetup() or staticSetup() after the begin() call.

DNS lookups are also very simple:

Screen Shot 2011 06 15 at 09.47.32

The one thing to keep in mind here, is that the website argument needs to be a flash-based string, which must be defined as follows:

Screen Shot 2011 06 15 at 09.51.10

Note the “PROGMEM” modifier. See the Saving RAM space weblog post for more info about this technique.

This concludes the intialization part of the EtherCard library. Next, we need to keep things going by frequently polling for new incoming packets and responding to low-level ARP and ICMP requests. The easiest way to do so is to use the following template for loop():

Screen Shot 2011 06 15 at 09.59.01

The packetReceive() function polls for new incoming data and copies it into the global buffer. The return value is the size of this packet (or zero if there is none).

The packetLoop() function looks at the incoming data and takes care of low-level responses. The return value is the offset in the global packet buffer where incoming TCP data can be found (or zero if there is none).

As to what to do next: it really all depends on what you’re after. Check out the examples in the Ethercard library for how to build web servers and web clients on top of this functionality.

To get an idea of the code overhead of the EtherCard library: a simple web client using DHCP and DNS is around 10 Kb, while an even simpler one using static IP addresses (no DHCP and no DNS) is under 7 Kb. The fairly elaborate EtherNode sample sketch, which includes DHCP and the RF12 library is now ≈ 13 Kb.

IOW, lots of room for adding your own app logic!

Ping example

In AVR, Software on Jun 18, 2011 at 00:01

Another example of how to use the EtherCard library:

Screen Shot 2011 06 15 at 09.08.33

This example illustrates doing DHCP, DNS, and then periodically pinging a remote server to see if it’s alive.

Sample output:

Screen Shot 2011 06 15 at 09.10.11

As you can see, it not only pings others, but also reports when it’s being pinged by others.

Add a LED and you could make a trivial server status display. Or add a web page fetch to verify that the web server is also still alive. Or add a graphics display to show a graph of the ping times. Or add a relay and power-cycle the server when it’s not responding … endless possibilities, and very simple to implement!

Complete web client demo

In AVR, Software on Jun 17, 2011 at 00:01

After yesterday’s addition of DHCP to the EtherCard library, it’s only a small step to create a sketch which does everything needed for convenient stand-alone use on a local LAN.

Here’s a webClient.pde demo sketch which sets itself up via DHCP, then does a DNS lookup to find a server by name, then does a web request every 5 seconds and displays the first part of the result:

Screen Shot 2011 06 15 at 09.04.57

Sample output:

Screen Shot 2011 06 15 at 09.05.51

The total sketch is under 10 Kb, so there’s still lots of room to add the RF12 wireless driver, as well as a fair amount of application logic.

Who says a little 8-bit processor can’t be part of today’s internet revolution, eh?

DHCP support

In AVR, Software on Jun 16, 2011 at 00:01

Not long ago, Andrew Lindsay added support for DHCP to his EtherShield library, which is derived from the same “Guido Socher & Pascal Stang” code base as the EtherCard library. Fantastic work, hats off!

I’ve reworked the code to simplify things, and more importantly: reduce its code size. Many of these changes made it into Andy’s Github code base, but I’ve decided to take it quite a bit further and completely redo the way DHCP gets called in sketches.

It’s now based on just one call: dhcpSetup(). For completeness, I must mention that lease expiry and re-lease have been swept under the rug for now. Here’s the new testDHCP.pde demo I’ve come up with:

Screen Shot 2011 06 16 at 00.10.07

Sample output:

Screen Shot 2011 06 08 at 13.43.27

The entire demo sketch is currently 7732 bytes (6420 without the serial I/O). The DHCP part appears to require about 1200 bytes in all this. Note that the actual code to perform or serve web requests will add several more Kb to the total size, this merely illustrates the minimal code for doing DHCP.

There’s a “#define SELECT_BIT” in the enc28j60.cpp source which can be changed for boards which don’t use B0 (i.e. arduino digital I/O pin 8) as chip select. There’s also a “#define FULL_SPEED” setting which switches the SPI bus to 8 MHz during block transfers – it’s enabled by default and has a noticeable impact on performance.

So now any ENC28J60-based EtherCard with an ATmega328 can be connected to a local LAN and become usable as web server and/or client without having to configure an IP address, DNS server, or gateway!

RF12 broadcasts and ACKs

In Software on Jun 10, 2011 at 00:01

In yesterday’s post, the general design of the RF12 driver was presented, and the format of the packets it supports.

The driver also support broadcasting, i.e. sending packets to all interested nodes, and ACKs, i.e. sending a short “acknowledge” packet from receiver to transmitter to let the latter know that the packet was properly received.

Broadcasting and ACKs can be combined, with some care: only one node should send back the ACK, so the usefulness of ACKs with broadcasts is limited if the goal was to reliably get a packet across to multiple listeners.

Broadcasts and ACKs use the HDR byte in each packet:

Rf12 Packets

There are three bits: C = CTL, D = DST, and A = ACK, and there is a 5-bit node ID. Node ID 0 and 31 are special, so there can be 30 different nodes in the same net group.

The A bit (ACK) indicates whether this packet wants to get an ACK back. The C bit needs to be zero in this case (the name is somewhat confusing).

The D bit (DST) indicates whether the node ID specifies the destination node or the source node. For packets sent to a specific node, DST = 1. For broadcasts, DST = 0, in which case the node ID refers to the originating node.

The C bit (CTL) is used to send ACKs, and in turn must be combined with the A bit set to zero.

To summarize, the following combinations are used:

  • normal packet, no ACK requested: CTL = 0, ACK = 0
  • normal packet, wants ACK: CTL = 0, ACK = 1
  • ACK reply packet: CTL = 1, ACK = 0
  • the CTL = 1, ACK = 1 combination is not currently used

In each of these cases, the DST bit can be either 0 or 1. When packets are received with DST set to 1, then the receiving node has no other way to send ACKs back than using broadcasts. This is not really a problem, because the node receiving the ACK can check that it was sent by the proper node. Also, since ACKs are always sent immediately, each node can easily ignore an incoming ACK if it didn’t send a packet shortly before.

Note that both outgoing packets and ACKs can contain payload data, although ACKs are often sent without any further data. Another point to make, is that broadcasts are essentially free: every node will get every packet (in the same group) anyway – it’s just that the driver filters out the ones not intended for it. A recent RF12 driver change: node 31 is now special, in that it will see packets sent to any node ID’s, not just its own.

It turns out that for Wireless Sensor Networks, broadcasts are quite useful. You just kick packets into the air, in the hope that someone will pick them up. Often, the remote nodes don’t really care who picked them up. For important events, a remote node can choose to request an ACK. In that case, one central node should always be listening and send ACKs back when requested. An older design of the Room Node sketch failed to deal with the case where the central node would be missing, off, or out of range, and would retry very often – quickly draining its own battery as a result. The latest code reduces the rate at which it resends an ACK, and stops asking for ACKs after 8 attempts. The next time an important event needs to be sent again, this process then repeats.

RF12 packet format and design

In Software on Jun 9, 2011 at 00:01

The RF12 library contains the code to support the RFM12B wireless module in an Arduino-like environment. It’s used for JeeNodes but also in several projects by others.

Here’s the general structure of a packet, as supported by the RF12 driver:

I made quite a few design decisions in the RF12 driver. One of the goals was to make the communication work in the background, so the driver is fully interrupt-driven. An important choice was to limit the packet size to 66 bytes of payload, to keep RAM use low and still allow just over 64 bytes of payload, enough to send data in 64-byte chunks with a teeny bit of additional info.

Another major design decision was to support absolutely minimal packet sizes. This directly affects the power consumption, because longer packets take more time, and the longer the receiver and transmitter are on, the more precious battery power they will consume. For this same reason, the transmission bit rate is set fairly high (about 50 kbits/sec) – a higher rate means the same message can be sent in less time. A higher rate also makes it harder for the receiver to still pick up a good packet, so I didn’t want to push this to the limit.

This is how the RF12 driver can support really short packets:

  • the network group is one byte and also doubles as second SYN byte
  • the node ID is small enough (5 bits) to allow a few more header bits in the same byte
  • there are only three header bits, as described in more detail in tomorrow’s post
  • there is only room for either the source node ID or the destination node ID

That last decision is a bit unusual. It means an incoming packet can only inform the receiver where it came from, or define which receiver the packet is intended for – not both.

This may seem like a severe limitation, but it really isn’t: just add the missing info in the payload and agree on a convention so that the receiver can pick it up. All the RF12 does is enforce a truly minimal design, you can add any info you like as payload.

As a result, a minimal packet has the following format:

That’s 9 bytes, i.e. 72 bits – which means that a complete (but empty) packet can be sent out in less than 1.5 ms.

Tomorrow, I’ll describe the exact logic used in the HDR byte, and how to use broadcasts and ACKs.

Server upgrade

In Hardware on Jun 3, 2011 at 00:01

The JeeLabs server is running out of steam…

No, not CPU load:

Screen Shot 2011 05 30 at 12.41.31

It’s idling at under 10%, and drawing 10..15W, exactly as planned.

It’s running out of RAM, all friggin’ 4 Gbytes of ’em!

There are 4 VM’s running so far, of which Drupal gets 768 Mb, and WordPress / Redmine each get 512 Mb. With Parallels, VM’s take up a bit more memory than the raw amount, I suppose there’s some caching going on.

It’s bordering on the ridiculous. The entire WordPress MySQL database is probably not more than a few dozen Mb by now (which in itself is silly, but that’s another story). Yet it needs more than an order of magnitude more RAM to serve it all up as web pages?

I can’t help but think that a fully RAM-based database (with disk image backup) could dramatically reduce the memory needs. But hey, I’m not in the business of redoing the database world – well, not these days anyway ;)

The shining exception is the nginx reverse proxy I use: it runs in 128 Mb, and would probably run just as fine with 64 Mb. It serves lots of static pages, and handles all the re-routing for the rest of the traffic. Brilliant software.

So I bought a fresh pair of 4 Gb memory sticks off eBay (thx, BWired):

Dsc 2541

With the latest Mac Mini models, memory upgrading is – at last – trivial. You might have seen the backSoon server for a few minutes, which is just a JeeNode USB with EtherCard presenting this (dynamic!) web page:

Screen Shot 2011 05 30 at 18.56.17

So there you go. All the JeeLabs .org and .net sites (and a few more) now run with a bit more breathing space. We’re solidly back in the green again:

Screen Shot 2011 05 30 at 19.01.55

Onwards!

Update – Looks like software always expands to consume all available memory: this time it is Apache, btw. I probably need to tweak it, it just grows and grows and grows! (the Drupal VM is now 1.5 Gb)

Screen Shot 2011 06 03 at 11.15.42

RF bootstrap design

In Software on May 24, 2011 at 00:01

After some discussion on the forum, I’d like to present a draft design for an over-the-air bootstrap mechanism, IOW: being able to upload a sketch to a remote JeeNode over wireless.

Warning: there is no release date. It’ll be announced when I get it working (unless someone else gets there first). This is just to get some thoughts down, and have a first mental design to think about and shoot at.

The basic idea is that each remote node contacts a boot server after power up, or when requested to do so by the currently running sketch.

Each node has a built-in unique 2-byte remote ID, and is configured to contact a specific boot server (i.e. RF12 band, group, and node ID).

STEP 1

First we must find out what sketch should be running on this node. This is done by sending out a wireless packet to the boot server and waiting for a reply packet:

  • remote -> server: intial request w/ my remote ID
  • server -> remote: reply with 12 bytes of data

These 12 bytes are encrypted using a pre-shared secret key (PSK), which is unique for each node and known only to that node and the boot server. No one but the boot server can send a valid reply, and no one but the remote node can decode that reply properly.

The reply contains 6 values:

  1. remote ID
  2. sketch ID
  3. sketch length in bytes
  4. sketch checksum
  5. extra sketch check
  6. checksum over the above values 1..5

After decoding this info, the remote knows:

  • that the reply is valid and came from a trusted boot server
  • what sketch should be present in flash memory
  • how to verify that the stored sketch is complete and correct
  • how to verify the next upload, if we decide to start one

The remote has a sketch ID, length and checksum stored in EEPROM. If they match with the reply and the sketch in memory has the correct checksum, then we move forward to step 3.

If no reply comes in within a reasonable amount of time, we also jump to step 3.

STEP 2

Now we need to update the sketch in flash memory. We know the sketch ID to get, we know how to contact the boot server, and we know how to verify the sketch once it has been completely transferred to us.

So this is where most of the work happens: send out a request for some bytes, and wait for a reply containing those bytes – then rinse and repeat for all bytes:

  • remote -> server: request data for block X, sketch Y
  • server -> remote: reply with a check value (X ^ Y) and 64 bytes of data

The remote node gets data 64 bytes at a time, and burns them to flash memory. The process repeats until all data has been transferred. Timeouts and bad packets lead to repeated requests.

The last reply contains 0..63 bytes of data, indicating that it is the final packet. The remote node saves this to flash memory, and goes to step 3.

STEP 3

Now we have the proper sketch, unless something went wrong earlier.

The final step is to verify that the sketch in flash memory is correct, by calculating its checksum and comparing it with the value in EEPROM.

If the checksum is bad, we set a watchdog timer to reset us in a few seconds, and … power down. All our efforts were in vain, so we will retry later.

Else we have the proper sketch and it’s available in flash memory, so we leave bootstrap mode and launch it.

That’s all!

ROBUSTNESS

This scheme requires a working boot server. If none is found or in range, then the bootstrap will not find out about a new sketch to load, and will either launch the current sketch (if valid), or hit a reset and try booting again a few seconds later.

Not only do we need a working boot server, that server must also have an entry for our remote ID (and our PSK) to be able to generate a properly encrypted reply. The remote ID of a node can be recovered if lost, by resetting the node and listening for the first request it sends out.

If the sketch hangs, then the node will hang. But even then a hard reset or power cycle of the node will again start the boot sequence, and allows us to get a better sketch loaded into the node. The only drawback is that it needs a hard reset, which can’t be triggered remotely (unless the crashing sketch happens to trigger the reset, through the watchdog or otherwise).

Errors during reception lead to a failed checksum at the end, which then leads to a reset and a new boot loading attempt. There is no resume mechanism, so such a case does mean we have to fetch all the data blocks again.

SECURITY

This is the hard part. Nodes which end up running some arbitrary sketch have the potential to cause a lot of damage if they also control real devices (lights are fairly harmless, but thermostats and door locks aren’t!).

The first line of defense comes from the fact that it is the remote node which decides when to fetch an update. You can’t simply send packets and make remote nodes reflash themselves if they don’t want to.

You could interrupt AC mains and force a reset in mains-powered nodes, but I’m not going to address that. Nor am I going to address the case of physically grabbing hold of a node or the boot server and messing with it.

The entire protection is based on that initial reply packet, which tells each remote node what sketch it should be running. Only a boot server which knows the remote node’s PSK is able to send out a reply which the remote node will accept.

It seems to me that the actual sketch data need not be protected, since these packets are only sent out in response to requests from a remote node (which asks for a specific sketch ID). Bad packets of any kind will cause the final checksums to fail, and prevent such a sketch from ever being started.

As for packets flying around in a fully operational home network: that level of security is a completely separate issue. Sketches can implement whatever encryption they like, to secure day-to-day operation. In fact, the RF12 library includes an encryption mechanism based on XTEA for just that purpose – see this weblog post.

But for a bootstrap mechanism, which has to fit in 4 Kb including the entire RF12 wireless packet driver, we don’t have that luxury. Which is why I hope that the above will be enough to make it practical – and safe!

RF12 skeleton sketch

In Software on May 7, 2011 at 00:01

The RF12 library has all the code to drive an RFM12B wireless module, and supports full interrupt-drive sending and receiving of arbitrary packets up to 66 bytes in length.

Interrupt drivers are fiendishly hard to debug and get 100% right, but often well worth the effort. The result is code which behaves almost as if it’s running in the background, i.e. it makes the ATmega appear to support multi-tasking, with all I/O happening all by itself.

In the case of the RFM12B, this is quite important, because there are some very strict timing requirements as to how and when to exchange data with the hardware. Once a driver is interrupt-driven, the rest of the code doesn’t have to be as strict – all critical timing requirements are dealt with, even if you don’t poll the driver regularly.

But the logic of all this stuff can be a bit overwhelming at first. So, to help out, and prompted by a recent discussion on the forum, I’ve set up an example of how to write a sketch which can read and send packets:

#include <Ports.h>
#include <RF12.h>

MilliTimer sendTimer;
typedef struct { ... } Payload;
Payload inData, outData;
byte pendingOutput;

void setup () {
    ...
    // call rf12_initialize() or rf12_config()
}

static void consumeInData () {
    ...
}

static byte produceOutData () {
    ...
    return 1;
}

void loop () {
    if (rf12_recvDone() && rf12_crc == 0 && rf12_len == sizeof inData) {
        memcpy(&inData, (byte*) rf12_data, sizeof inData);
        // optional: rf12_recvDone(); // re-enable reception right away
        consumeInData();
    }

    if (sendTimer.poll(100))
        pendingOutput = produceOutData();

    if (pendingOutput && rf12_canSend()) {
        rf12_sendStart(0, &outData, sizeof outData, 2);
        // optional: rf12_sendWait(2); // wait for send to finish
        pendingOutput = 0;
    }
}

You’ll need to do a few things to get this going, which are all common sense really:

  • define a proper struct for the Payload contents you want to send/receive
  • set up the RF12 driver with the proper configuration settings
  • fill in the code to handle incoming data in inData
  • fill in the code to save new outgoing data to outData

This sketch will also work with an RFM12B Board and an Arduino.

One crucial detail is that you can’t just send data whenever you feel like it – you have to throttle the outgoing sends a bit using sendTimer , and you have to ask the RF12 driver for permission to send using rf12_canSend(). Failure to do this will “mess up the air waves” and severely interfere with RF communication between any nodes, even those that play nice.

To write a sketch which only sends, leave consumeInData() empty – don’t throw out the first “if”, because those rf12_recvDone() calls are still essential.

To write a sketch which only receives, simply make produceOutData() return 0. Removing the last two if’s is also ok, in this case.

Once you have your sketch working, you can start adding tricks to reduce power consumption: turning the RFM12B on and off, running at lower clock speeds, putting the ATmega into a low-power sleep state, etc.

OOK fix

In Hardware on Feb 10, 2011 at 00:01

Here’s a great suggestion from Stefan Schulze (“kami” on the forum) for getting that wrong 433 MHz transmitter working with the OOK 433 Plug:

Img 5951

Sideways:

Img 5950

It requires some pin-bending, and you can attach an optional antenna, as he did.

I’m currently discussing option with the supplier to find a more “official” solution, but if you want to use the OOK plug right now, the above is definitely an option.

Thanks, Stefan!

Update – For everyone who received the OOK 433 Plug in its current form: I will send a solution out to everyone once it has been worked out. If you don’t need the TX side immediately, it might be an option to postpone soldering it until that solution is available.

Update #2 – Problem has been resolved, the OOK Plug is now supplied with the correct 3-pin transmitter.

OOK Murphy

In Hardware on Feb 8, 2011 at 00:01

Looks like there is a problem with the OOK 433 Plug

Here’s the transmitter I started out with:

Dsc 2455 2

Here’s the print layout I designed, based on that:

Screen Shot 2011 02 07 at 20.36.25

Everything worked fine. So I ordered larger quantities, to be ready for production. Here’s what I got in that new batch order:

Dsc 2455 3

Whoops… different unit!

Worse still, the original board had +, Gnd, Data as pins (as seen on the board layout), whereas the new boards have Data, Gnd, +, Antenna. It would be possible to cut the antenna pin off, but that’s not enough since the pinout is in fact reversed!

I’ve contacted the supplier. Let’s see how this goes. For the time being, I’m forced to take the OOK 433 Plug “off the shelf”. Sorry about that – please hang in there, also if you’ve already ordered this plug. Note that this only affects the 433 MHz transmitter – the 433 MHz receiver should work just fine.

OOK relay, revisited

In Software on Feb 3, 2011 at 00:01

With the modded RFM12B receiving 868 MHz signals, and the new OOK 433 Plug doing the same for the 433 MHz band, the new OOK relay is coming in sight.

Just a lousy bit of code. Elementary – I thought…

Except it wasn’t. Software always seems to take a lot more time (and concentration) than hardware. Silly!

Still, I think I managed to collect all the pieces lying around here from earlier experiments in that area, and combine them into a new ookRelay2.pde sketch.

It’s fairly elaborate and too long to show here, but I’ll pick out some pieces:

  • all the decoders live in the decoders.h file
  • since they all share common logic, each is derived from a common “DecodeOOK” class
  • the protocol for each decoder is the same: feed puse widths to nextPulse(), and it will return true whenever a valid packet has been decoded, then call getData() to get a pointer and byte count
  • the ookRelay2 sketch includes a variety of decoders, I hope we can improve/extend/add-more over time
  • there are two pulse sources: the 868 MHz receiver and the 433 MHz receiver
  • for each, a “DecoderInfo” table is defined with decoders to use for them
  • the runPulseDecoders() function does what the name says: evaluate each of the decoders in turn
  • when a decoder succeeds, data is added to an outgoing buffer (and optionally, printed to serial)
  • in this example, I send the accumulated data off to the RF12 wireless network, but Ethernet or any other transport mechanism could be used as well

With this out of the way, you can probably, eh… decode the following lines at the top op the ookrelay2 sketch:

Screen Shot 2011 02 02 at 23.30.36

And here’s the main loop, which is keeping things going:

Screen Shot 2011 02 02 at 23.31.24

The hard part is doing this efficiently with accurate timings, even though a lot of stuff is happening. That’s why there are two interrupt routines, which trigger on changes in 868 MHz and 433 MHz signals, respectively:

Screen Shot 2011 02 02 at 23.33.22

I’m still debugging, and I need to analyze just how much leeway there is to run all the decoders in parallel. Earlier today I had the 433 MHz reception going, but right now it seems this code is only picking up 868 MHz signals:

Screen Shot 2011 02 02 at 23.34.46

Oh well, it’s a start. Feel free to check out the code, which lives as example in the RF12 library.

Update – Bug fixed, now 433 MHz decoding works.

Meet the RFM12B Board

In Hardware on Feb 2, 2011 at 00:01

With the RFM12B becoming a nice low-cost option for low-volume wireless communication, and the RF12 library proving to be a solid software driver for it, it’s time to generalize a bit further…

Say hello to the new RFM12B Board:

Dsc 2448

This board adds a voltage regulator and 3.3V/5V level conversions, to be able to use the RFM12B on 5V systems such as the various Arduino’s out there, the RBBB, … anything you want, really.

There are 8 pins on this board, of which the 8th is a regulated 3.3V supply which can be used in other parts of the circuit – the voltage regulator will be able to supply at least 100 mA extra on that supply pin.

The other 7 pins are:

  • +5V
  • Ground
  • SPI clock (SCK) – Arduino digital 13
  • SPI data out (SDO) – Arduino digital 12
  • SPI data in (SDI) – Arduino digital 11
  • SPI select (SEL) – Arduino digital 10
  • IRQ – Arduino digital 2

Just hook each of those up to an Arduino, and you can use the RF12 library as is!

Sample output:

Screen Shot 2011 02 01 at 22.08.14

Look ma, just like a JeeNode or JeeLink!

With an 8-pin stacking header and a bit of bending, cutting, and soldering two wires (I used a jumper wire, cut in half), you can even stick this thing right onto an Arduino:

Dsc 2449

But of course using a normal solderless breadboard and some wire jumpers will work just as well.

Note that this board can also be tied to 3.3V systems – just use the bare PCB (and short out three solder jumpers), which then becomes a breakout board for the RFM12B. No need to mess with the 2.0 mm pin/pad distance on the RFM12B module itself.

Docs can be found in the Café, and the kit/pcb is now available in the shop, as usual.

Back-soon server

In Hardware, Software on Jan 31, 2011 at 00:01

Soon, I’m going to move the JeeLabs server to a new spot in the house. Out of sight, now that the setup is stable.

But to do so requires rerouting an ethernet cable to the internet modem downstairs.

To do it right, I’d like to have a “we will be back soon” surrogate server plugged into the internet modem while transitioning, so that the status is reported on-line:

Screen Shot 2011 01 30 at 16.36.21

I could plug in a temporary Linux box, of course, or a laptop. But I want to keep this option available at all times, so a dedicated solution would be more practical. That way I can easily take the server off-line at any moment.

Ah, but that’s easy, with an Ether Card and an RBBB:

Dsc 2434

This combination just needs a 5..6V power supply, and 6 wires between the RBBB and the Ether Card.

Here’s the backSoon.pde sketch, which I’ve added to the EtherCard library:

Screen Shot 2011 01 30 at 17.24.05

In this case, being able to configure the MAC address of the interface as well as the IP address is in fact quite convenient, because this way the modem needn’t notice the hardware switch.

Only needs about 6 Kb. Actually, I’ll probably add a wireless option and use a JeeNode instead, to report total hits every once in a while. But either way, such a “back-soon server” really doesn’t come any simpler than that!

So if one of these days you see that message while surfing at JeeLabs, you know where it’s coming from :)

PS. I’ve put the back-soon server on-line as a test, it can be reached at https://jeelabs.org:8080/.

EtherNode on a breadboard

In AVR, Hardware, Software on Jan 30, 2011 at 00:01

After some recent comments on the forum about the etherNode.pde sketch, I wanted to make sure that the code is still working properly on this hassle-free setup (as well as with the Carrier Board):

Dsc 2432

And sure enough, it works as intended:

Screen Shot 2011 01 29 at 21.12.06

The output shows some packets from node 19 – which is the modded RFM12B running the RFM12B_OOK sketch, and it’s working nicely as you can see.

This sketch takes about 12 Kb of code (web server + EtherCard + RF12 driver), so there is ample room to add more features in there. You could use a modded RFM12B module, for example, and have everything you need to create a gateway between 868 MHz OOK devices and the LAN. In both directions in fact, since the RFM12B can also be tricked into sending out OOK packets.

Note that the Ether Card uses a few extra I/O pins from the JeeNode, so be sure to connect the 2×4 headers on the SPI/PSI pins between JeeNode and Bridge Board.

And the best part is that even with this Ether Card attached, all 4 ports on the JeeNode are still available for other uses. They are all brought out on the 8 leftmost pins, clearly labeled with both port numbers and Arduino pin numbers. Also available from the bridge board, as bonus: 2 LED’s and a push button.

As you can see, there’s a lot of empty real-estate on that breadboard – yearning to be put to good use…

PS. In case you’ve been waiting for one of those wooden bases, as included with the JeeNode Experimenter’s Pack: not to worry, I haven’t forgotten about you. The recent batch of packs were sent out without the base – they will be shipped separately once back in stock.

New OOK and DCF relay

In Hardware on Jan 29, 2011 at 00:01

With all the pieces finally in place, and now that I’m getting a little bit more time to hack around again, this seemed like a good time to reconsider the OOKrelay.

So I combined a JeeNode USB, the new OOK 433 Plug, a Carrier Board with box, and half a Carrier Card:

Dsc 2429

In the top left I also added a DCF77 receiver from Conrad, attached to the Carrier Card prototyping board. It’s a bit hard to see, because the little receiver board is actually mounted upright. Here’s a better view:

Dsc 2430

A JeeNode USB was selected, because this thing will be powered permanently, and I chose to hide the connector inside the box to make it more robust. So all this needs is a little USB charger. The LiPo charge option might be useful if I decide to make this thing more autonomous one day (i.e. to record accurate power outage times).

Note that this is a modded JeeNode, as described here, to be able to receive 868 MHz OOK signals.

So what this thing can do – as far as the hardware goes – is listen for both 433 MHz and 868 MHz OOK signals at the same time, as well as pick up the DCF77 atomic clock signals from Frankfurt. Sending out 433/868 MHz OOK is possible too, but since the unit isn’t constantly listening for OOK packets, it’ll have to poll for such commands, which will introduce a small delay.

That’s the hardware, i.e. the easy part…

The software will be a lot more work. I’m going to adapt / re-implement the functionality from the OOKrelay sketch, i.e. this unit will decode and re-transmit all incoming data as RF12 packets, so that they can be picked up by a JeeLink hooked up to my PC/Mac. The clock signal will be useful to accurately time-stamp all receptions, and is really of more general use.

So far, the following I/O pins have been used:

  • one port for the OOK 433 Plug, i.e. one DIO and one AIO pin
  • one input pin for the modded JeeNode, to receive 868 MHz OOK signals
  • one input pin for the DCF77 signal

There is still lots of room left for expansion. A Pressure Plug perhaps, to track barometric pressure. Or a Memory Plug to save up the last data while the central receiver is unavailable. Or both, since these can combined on a single I2C port.

Absent from all this, is a display. First of all, squeezing a 2×16 LCD in there would have been very tight, but more importantly, now that there is a JeePU, there really is no need. I’m already sending the info out by wireless, so a remote graphical display is definitely an option – without PC – or I could use a central server to get this info to the right place(s). This box is intended to be hidden out of sight, somewhere centrally in the house.

Only thing I might consider is a small LED near the USB cable, to indicate that all is well. Maybe… I’m not too fond of blinking LEDs everywhere in the house :)

Meet the OOK 433 Plug

In Hardware on Jan 28, 2011 at 00:01

After yesterday’s post about modding the RFM12B to receive OOK RF data, here’s a new option which goes the other way: add-on hardware for 433 MHz.

Meet the OOK 433 Plug:

This plug can probably be used without antenna, but adding one is bound to increase the maximum reception range (I haven’t done any range tests yet).

The receiver output is tied to the AIO pin, which must be configured as input on the JeeNode. The transmitter is tied to the DIO pin, to be used as output pin.

Both receiver and transmitter can be operated from 3..12V (I’ve only tested with 3.3V and 5V, so far). For the transmitter, I would expect the power output and range to increase with voltage, but even at 3.3V things seem to work quite well in both directions. There is a jumper to select between PWR and +3V as power supply, and there is an optional series resistor for the receiver output, which may be needed at higher voltages (normally, it can be omitted by closing that second jumper).

The reason for creating a separate plug is that it allows a single JeeNode to operate simultaneously on the 433 MHz and 868 MHz frequency bands. As suggested in the comments on yesterday’s post, a modded 868 MHz RFM12B can probably be made to receive 433 MHz OOK signals, but the receiver sensitivity is bound to be fairly low since the RF circuitry is really dimensioned for 868 MHz. Haven’t tried this, though.

There are several sketches which can be used with the OOK 433 Plug. To use the receiver on this board with little or no adjustment, try recv433_test.pde or ookRelay.pde. For tranmission, there is the send433_test.pde sketch.

I expect to combine and integrate code from the different OOK and relaying sketches in the near future, now that this plug has been created, but for now you’ll have to tinker with what’s available.

If you have suggestions (of better still: code or docs) for new protocol handlers to add and implement on a JeeNode, please let me know or share it on the discussion forum. It will interesting to see how many OOK protocols on 433 and 868 MHz we can figure out and implement, to make this all truly versatile for lots of different RF devices out there.

All code and docs are in the Café, with the OOK 433 Plug available from the shop, as usual.

Enjoy!

Note – This server was down on Jan 28, from about 1:00 to 10:00 (CET). I’m investigating what happened.

OOK reception with RFM12B

In Hardware, Software on Jan 27, 2011 at 00:01

A while back, JGJ Veken (Joop on the forum) added a page on the wiki on how the RFM12B can receive OOK.

I never got around to trying it … until now. In short: if you’re not afraid of replacing an SMD capacitor on the RFM12B wireless module, then it’s trivial!

Here’s what needs to be done – the capacitor on the left is 4.7 nF:

Screen Shot 2011 01 25 at 14.16.36

Unsolder it and replace it with a cap in the range 150..330 pF (I used 220 pF).

This cap appears to determine the time constant w.r.t. how fast the RSSI signal adapts to varying RF carrier signal strengths. With 4.7 nF, it’s a bit too sluggish to detect an OOK signal – which is nothing other than a carrier being switched on and off (OOK stands for: On / Off Keying).

The next trick is to connect the FSK/DATA/nFSS pin of the RFM12B via a 100 Ω resistor to AIO1 (a.k.a. analog 0, a.k.a. PC0, a.k.a. ATmega pin 23 – phew!):

Dsc 2427

As far as I can tell, this is a digital signal, so connecting it to AIO0 is really not a requirement. It might be more practical to connect it to one of the B0/B1 pins on the SPI/ISP header. Perhaps I should add a jumper in a future revision of the JeeNode PCB?

And lastly, the RFM12B must be placed in a special mode to get the RSSI signal onto that pin – i.e. compared to the RSSI threshold, also configured into the RFM12B (97 dBm).

All the pieces were there, and all I had to do was to follow the steps mentioned on the wiki page.

I made some changes to the code and added it as RF12MB_OOK.pde example sketch. Here is the main logic:

Screen Shot 2011 01 25 at 16.42.14

As you can see, all incoming data is forwarded using the normal RF12 mode packet driver.

Sample output:

Screen Shot 2011 01 25 at 16.56.39

It’s happily picking up FS20, EM10, S300, and KS300 packets, and the overall sensitivity seems to be excellent. And since it forwards all data as packets into the rest of the JeeNode network, I now have all the data coming in over a single JeeLink.

Sooo… with this “mod”, no separate OOK receiver is needed anymore for the 868 MHz frequency band!

PS. Haven’t done too many tests with this yet. Transmission is unaffected, as far as I can tell. Reception of packets with the RF12 driver still seems to work – it may be more susceptible to RF variations, but then again a “normal” packet uses FSK which is a constant carrier, so in principle this modification should not affect the ability of the RFM12B to receive standard FSK packets.

Poor Man’s Mesh Network

In Software on Jan 16, 2011 at 00:01

The recent packet relay and Extended node ID posts set the stage for a very simple way to create a fairly long-range (and very low-cost) little Wireless Sensor Network around the house.

Say hello to the Poor Man’s Mesh Network :)

Screen Shot 2011 01 14 at 13.54.50

(the main computer doesn’t have to be a PC/Mac, an embedded Linux box would also work fine)

Well, that’s a bit presumptuous. It’s not really “mesh” in the sense that there is no dynamic or adaptive routing or re-configuration involved at all. All the packet routes are static, so failure in one of the relays for example, will make everything “behind” it unreachable. In the world of wireless, that matters, because there are always unexpected sources of interference, and occasionally they completely wipe out the ability of a network to communicate on a given frequency band. For example, a couple of times a year, the data I’m collecting from my electricity and gas meters here at JeeLabs just stops getting in. Not sure what’s going on, but I’m not touching that setup in any way, and it’s unrelated to power outages. It might not be an RF issue, but who knows.

So what I’m referring to is not a super-duper, full-fledged, kitchen-sink-included type of wireless network. But given the extremely low cost of the nodes, and the fact that the software needs less than 4 kb of code, I think it’s a good example of how far you can get when using simplicity as guiding principle. Even an 8-bit MPU with 8 Kb of flash and 512 bytes of RAM would probably be sufficient for sensor nodes – so with the ATmega328’s used in JeeNodes, we really have oodles of spare capacity to implement pretty nifty applications on top.

Most of the details of how this works have already been presented in previous posts.

The one extra insight, is that the packet relay mechanism can be stacked. We can get data across N relays if we’re willing to limit our data packets to 66-N bytes. So by sacrificing a slight reduction in payload length we can extend the number of relays, and hence the total maximum range, as much as we like. Wanna get 10 times as far? No problem, just place a bunch of relays in the right spots along the whole path. Note that ACK round-trip delays will increase in such a setup.

The big design trade-off here is that all packet routing is static, i.e. it has to be set up manually. Each sensor node (the stars in that diagram) needs to have a netgroup which matches a relay or central node nearby, and within each netgroup each sensor node has to have a unique ID.

It’s not as bad as it may seem though. First of all, the range of the RFM12B on 433, 868, and 915 MHz is pretty good, because sub-GHz radio waves are far less attenuated by walls and concrete floors than units operating at 2.4 GHz. This means that in a small home, you wouldn’t even need a relay at all. I get almost full coverage from one centrally-placed node here at JeeLabs, even though the house is full of stone walls and reinforced concrete floors. As I mentioned before, I expect to get to curb and to the far end of our (small) garden with one or two relay hops.

Second, this star topology is very easy to adjust when you need to extend it or make changes – especially if all packet relays are one “hop” away from the central node, i.e. directly talking to it. You can turn one relay off, make changes to the nodes behind it, and then turn it back on, and the rest of the network will continue to work just fine during this change.

I’ve extended the groupRelay.pde sketch a bit further, to be able to configure all the parameters in it from the serial/USB side. These settings are saved in EEPROM, and will continue to work across power loss. This means that a relay node can now be as simple as this:

Dsc 2412

IOW, a JeeLink, plugged into a tiny cheapo USB power adapter. All you need to do is pre-load the groupRelay sketch on it (once), adjust its settings (as often as you like), and plug it in where you need it. How’s that as maintenance-free solution? And you can add/drop/alter the netgroup structure of the entire network at any time, as long as you’re willing to re-configure the affected nodes. If some of them turn out to be hard to reach because they are at the limit of the range, just insert an extra relay and tell the central software about the topology change.

It doesn’t have to be a JeeLink of course. A JeeNode, or some home-brew solution would work just as well.

Now that this design has become a reality, I intend to sprinkle a lot more sensors around the house. There have been lots of little projects waiting for this level of connectivity, from some nodes outside near the entrance, to a node to replace one of the first projects I worked on at JeeLabs!

So there you go. Who needs complexity?

Extended node IDs

In Software on Jan 15, 2011 at 00:01

The packet relay implementation shown a few days ago has some properties which somewhat limit its usability.

It all has to do with node IDs. As a relay it all works fine, i.e. packets will get relayed as intended. But the problem is that with a relay, all the packets relayed into another netgroup appear to come from that single relay node.

To go into this, let’s first set up a relay with the following configuration:

  • the main netgroup is #1, the relay listens on netgroup #2
  • the central node in netgroup 1 is node 31
  • the relay node listens on netgroup 2 as node 31
  • the relay sends packets out to netgroup 1 as node 30

Here’s that setup:

Screen Shot 2011 01 14 at 12.05.01

So – in principle – we could have up to 59 sensor nodes for getting actual work done. But there’s a problem: all packets sent by nodes 1..30 in netgroup 2 look like packets coming from node 30 once they reach the central node in netgroup 1!

Note that this is no problem if the relay is only used to extend the range of a single node: all we have to do is give the relay the same number of the node in netgroup 1 as what is assigned to that single node in netgroup 2.

But with more nodes behind the relay, we’re losing all their node ID’s. This is not very practical if we need to know exactly where the reported sensor data was generated, for example. We’re hitting the limitation that there are only 31 different ways to identify all incoming packets!

The solution is to insert the node ID of the original node as first byte of the payload data. So a packet coming from say node 7 in netgroup 2, will come in as a packet from node 30 (the relay), with an extra byte inserted in front of the packet, containing the value 7. The groupRelay.pde sketch has been extended to support this “multi-node relaying” capability.

On the receiving end, the software needs to know that node 30 in netgroup 1 is a relay. It then knows that the first byte is not data but an “extended” node ID, and it can use that to re-identify the packet as coming from a node in netgroup 2.

The extra data byte means that the maximum payload length across a relay is one byte less than what the nodes in netgroup 1 can send to the central node, i.e. 65 bytes instead of the 66 bytes supported by the RF12 driver.

Let’s use some conventions for node ID’s to support this mechanism. Node ID’s will be assigned as follows:

  • node ID’s 1..26 are available for “real” end points, e.g. sensor nodes
  • node ID’s 27..30 are available for relays and other “management” nodes
  • node ID 31 is to be used as standard ID for the central receiver
  • node ID 31 is also used by relays on the listening side
  • lastly, node 31 is normally the node which always listens, and which sends out ACKs

Note that there is nothing in the RF12 driver implementation which enforces such a numbering scheme. It’s just a convention to simplify configuration, and to simplify the software later on.

This scheme has the following implications:

  • the maximum number of end point sensor nodes is 26 x 250 = 6,500 nodes
  • each netgroup can have at most 4 group relays, i.e. the “fanout” is up to 4
  • each relay introduces a “hop” and may reduce the max payload size by one byte

Tomorrow, I’ll describe how to use this in a larger context.

Nodes, Addresses, and Interference

In Software on Jan 14, 2011 at 00:01

The RF12 driver used for the RFM12B module on JeeNodes makes a bunch of assumptions and has a number of fixed design decisions built-in.

Here are a couple of obvious ones:

  • nodes can only talk to each other if they use the same “net group” (1..250)
  • nodes normally each have a unique ID in that netgroup (1..31)
  • packets must be 0..66 bytes long
  • packets need an extra 9 bytes of overhead, including the preamble
  • data is sent at approximately 50,000 baud
  • each byte takes ≈ 160 µs, i.e. a max-size packet can be sent in 12 milliseconds

So in the limiting case you could have up to 7,500 different nodes, as long as you keep in mind that they have to share the same frequency and therefore should never transmit at the same time.

For simple signaling purposes that’s plenty, but it’s obvious that you can’t keep a serious high-speed datastream going this way, let alone multiple data streams, audio, or video.

On the 433 or 868 MHz bands, the situation is often worse than that – sometimes much worse, because simple OOK (which is a simple version of ASK) transmitters tend to completely monopolize those same frequency bands, and more often than not, they don’t even wait for their turn so they also disturb transmissions which are already in progress! Add to that the fact that OOK transmitters often operate at 1000 baud or less, and tend to repeat their packets a number of times, and you can see how that “cheap” sensor you just installed could mess up everything!

So if you’ve got a bunch of wireless weather sensors, alarm sensors, or remotely controlled switches, chances are that your RF12-based transmissions will frequently fail to reach their intended destination.

Which is why “ACKs” are so important. These make it possible to detect when packets get damaged or fail to arrive altogehter. An ACK is just what the name says: an acknowledgement that the receiver got a proper packet. No more no less. And the implementation is equally simple, at least in concept: an ACK is nothing but a little packet, sent the other way, i.e. back from the receiver to the original transmitter.

With ACKs, transmitters have a way to find out whether their packet arrived properly. What they do is send out the packet, and then wait for a valid reply packet. Such an “ACK packet” need not contain any payload data – it just needs to be verifiably correct (using a checksum), and the transmitter must somehow be able to tell that the ACK indeed refers to its original packet.

And this is where the RF12 driver starts to make a number of not-so-obvious (and in some cases even unconventional) design decisions.

I have to point out that wireless communication is a bit different from its wired counterpart. For one, everyone can listen in. Radio waves don’t aim, they reach all nodes (unless the nodes are at the limit of the RF range). So in fact, each transmission is a broadcast. Whether a receiver picks up a transmitted packet is only a matter of whether it decides to let it through.

This is reflected in the design of the RF12 driver. At the time, I was trying to address both cases: broadcasts, aimed at anyone who cares to listen, and directed transmissions which target a specific node. The former is accomplished by sending to pseudo node ID zero, the latter requires passing the “destination” node ID as first argument to rf12_sendStart().

For the ACK, we need to send a packet the other way. The usual way to do this, is to include both source and destination node ID’s in the packet. The receiver then swaps those fields and voilá… a packet ready to go the other way!

But that’s in fact overkill. All we really need is a single bit, saying the packet is an ACK packet. And in the simplest case, we could avoid even that one bit by using the convention that data packets must have one or more bytes of data, whereas ACKs may not contain any data.

This is a bit restrictive though, so instead I chose to re-use a single field for either source or destination ID, plus a bit indicating which of those it is, plus a bit indicating that the packet is an ACK.

With node ID’s in the range 1..31, we can encode the address as 5 bits. Plus the src-vs-dest bit, plus the ACK bit. Makes seven bits.

Why this extreme frugality and trying to save bits? Well, keep in mind that the main use of these nodes is for battery-powered Wireless Sensor Networks (WSN), so reducing power usage is normally one of the most important design goals. It may not seem like much, but one byte less to send in an (empty) ACK packet reduces the packet length by 10%. Since the transmitter is a power hog, that translates to 10% less power needed to send an ACK. Yes, every little bit helps – literally!

That leaves one unused bit in the header, BTW. Whee! :)

I’m not using that spare bit right now, but it will become important in the future to help filter out duplicate packets (a 1-bit sequence “number”).

So here is the format of the “header byte” included in each RF12 packet:

Screen Shot 2011 01 13 at 23.35.02

And for completeness, here is the complete set of bytes sent out:

Screen Shot 2011 01 13 at 23.34.14

So what are the implications of not having both source and destination address in each packet?

One advantage of using a broadcast model, is that you don’t have to know where to send your packet to. This can be pretty convenient for sensor nodes which don’t really care who picks up their readings. In some cases, you don’t even care whether the data arrived, because new readings are periodically being sent anyway. This is the case for the Room Nodes, when they send out temperature / humidity / light-level readings. Lost one? Who cares, another one will come in soon enough.

With the PIR motion detector on Room Nodes, we do want to get immediate reporting, especially if it’s the first time that motion is being detected. So in this case, the Room Node code is set up to send out a packet and request an ACK. If one doesn’t come in very soon, the packet is sent again, and so on. This repeats a few times, so that motion detection packets reach their destination as quickly as possible. Of course, this being wireless, there are no guarantees: someone could be jamming the RF frequency band, for example. But at least we now have a node which tries very hard to quickly overcome an occasional lost packet.

All we need for broadcasts to work with ACKs, is that exactly one node in the same netgroup acts as receiver and sends out an ACK when it gets a packet which asks to get an ACK back. We do not want more than one node doing so, because then ACKs would come from different nodes at the same time and interfere with each other.

So normally, a WSN based on RFM12B’s looks like this:

Screen Shot 2011 01 13 at 23.35.09

The central node is the one sending back ACKs when requested. The other nodes should just ignore everything not intended for them, including broadcasts.

Note that it is possible to use more than one receiving node. The trick is to still use only a single one to produce the ACKs. If you’re using the RF12demo sketch as central receiver, then there is a convenient (but badly-named) “collect” option to disable ACK replies. Just give “1c” as command to the second node, and it’ll stop automatically sending out ACKs (“0c” re-enables normal ACK behavior). In such a “lurking” mode, you can have as many extra nodes listening in on the same netgroup as you like.

To get back to netgroups: these really act as a way to partition the network into different groups of nodes. Nodes only communicate with other nodes in the same netgroup. Nodes in other netgroups are unreachable, and data from those other nodes cannot be received (unless you set up a relay, as described a few days ago). If you want to have say hundreds of nodes all reporting to one central server, then one way to do it with RF12 is to set up a number of separate netgroups, each with one central receiving node (taking care of ACKs for that netgroup), and then collect the data coming from all the “central nodes”, either via USB, Ethernet, or whatever other mechanism you choose. This ought to provide plenty of leeway for home-based WSN’s and home-automation, which is what the RF12 was designed for.

So there you have it. There is a lot more to say about ACKs, payloads, and addressing… some other time.

Another topic worth a separate post, is using (slightly) different frequencies to allow multiple transmissions to take place at the same time. Lots of things still left to explore, yummie!

Packet relaying vs. storage

In Software on Jan 13, 2011 at 00:01

In yesterday’s post I introduced a groupRelay.pde sketch, which implements a packet relay.

This can be used to (approximately) double the range between sensor nodes and the central data-collecting node. I’ve got two uses for this myself:

  • To try and get through two layers of reinforced concrete here at JeeLabs, i.e. from the garage to the living room to the office floor where my central data-collecting node is. I can get through one floor just fine (easily, even with a few extra walls), but two is giving me trouble.

  • To have a simple way to work with multiple groups of JeeNodes around here for testing and experimentation, while still allowing me to “merge” one of the test groups with the main, eh, “production” group. This can easily be accomplished by turning a suitably-configured relay on or off.

Note that all traffic takes place in the same 868 MHz frequency band. This isn’t a way to double the amount of bandwidth – all the packets flying around here have to compete for the same RF air space. All it does is separate the available space into distinct logical groups, i.e. net groups, which can be used together.

To summarize from yesterday’s post, this is how the relay code works right now:

Screen Shot 2011 01 12 at 18.07.18

If you think of time as advancing from top to bottom in this diagram, then you can see how the packet comes in, then gets sent out, then the ACK comes in, and finally the ACK gets sent back to the originating node. Let’s call this the Packet pass-through (PPT) approach.

This is very similar to how web requests work across the internet. There is an “end-to-end” communication path, with replies creating one long “round trip”.

But that’s not the only way to do things. The other way is to use a Store-and-forward (SAF) mechanism:

Screen Shot 2011 01 12 at 18.07.30

In this case, the relay accepts the packet, stores it, and immediately sends back an ACK to the originating node. Then it turns around and tries to get the stored packet to its destination.

This is how email works, BTW. The SMTP servers on which email is built can all store emails, and then re-send those emails one step closer to their intended destination.

There are several differences between PPT and SAF:

  • with PPT, it takes longer for the originating node to get back an ACK
  • with SAF, you get an ACK right away, even before the destination has the data
  • with PPT, all failures look the same: no proper ACK is ever received
  • with SAF, you might have gotten an ACK, even though the destination never got the data
  • with PPT, the logic of the code is very simple, and little RAM is needed
  • with SAF, you need to store packets and implement timeouts and re-transmission

But perhaps most importantly for our purposes, PPT allows us to place payload data in the ACK packet, i.e. ACKs can contain replies, whereas with SAF, you can’t put anything in an ACK, because the originating node already got an empty ACK from the relay.

Since SAF is harder to implement, needs more storage, and can’t handle ACK reply data, it just an inferior solution compared to PPT, right?

Not so fast. The main benefit of SAF, is that it can deal with nodes which don’t have to be available at the same time. If the relay is always on, then it will always accept requests from originating nodes. But the destination nodes need not be available at that time. In fact, the destination node might use polling, and ask the intermediate relay node whether there is data waiting to be sent out to it. In effect, the SAF relay now becomes sort of a PO box which collects all incoming mail until someone picks it up.

The implications for battery-powered wireless networks are quite important. With an always-on relay node in the middle, all the other nodes can now go to sleep whenever they want, while still allowing any node to get data to any other node. The basic mechanism for this is that the low-power nodes sleep most of the time (yeay, micro power!) and then periodically contact the relay node in one of two ways:

  • sending out a packet they want to get to some other place
  • polling the relay to get data waiting for them back as ACK reply data

The “sleep most of the time” bit is an essential aspect of low-power wireless networks. They can’t afford to keep a node awake and listening for incoming wireless packets all the time. An RFM12B draws about 15 mA while in receive mode (more than an ATmega!), and keeping it on would quickly deplete any battery.

So if we want to create an ultra low-power wireless network, we will neeed a central relay node which is always on, and then all the other nodes can take control over when they want to send out things and ask for data from that central node whenever they choose to. Which means they could sleep 99.5% of the time and wake up for only a few milliseconds every second, for example. Which is of course great for battery life.

BTW, in case you hadn’t noticed: we’re now entering the world of mesh-networking…

But the drawbacks of SAF remain: more complex logic, and the need to be able to queue up a lot of packets. So we need one node which is always on, and has plenty of memory. Hmmm, ponder, ponder… I remember having seen something suitable.

Of course: the JeeLink! It draws power via USB and has a large DataFlash memory buffer. Whee, nice! :)

Relaying RF12 packets

In Software on Jan 12, 2011 at 00:01

Since the RF12 driver does not implement a full OSI network “stack”, there are no such things as routers and mesh networks in Jee-land. This means you’re basically limited to the range of a single 868/915 MHz point-to-point packet connection.

There are a number of ways to increase the range. One is to use a directional antenna (I’ve never tried this, but it has been mentioned on the discussion forum). Another option is to lower the transmission baud rate inside the radio and the bandwidth settings, so that the power is “beamed” into a narrower frequency range. Both of these are based on RF properties.

A third option if you can’t get from here to there in one go is to take multiple “hops”. That’s what a mesh network tries to do fully automatically, adapting on the fly to varying reception conditions and network topologies.

I’m more interested in much simpler approaches, which can easily be implemented in a little 8-bit ATmega with limited code storage. There’s a lot we can do, even within these pretty hard constraints – and it’s fun to push those little boundaries!

So here’s another option: a dedicated “packet relay” node, which listens to a specific RF12 net group most of the time, and when a packet comes in, it turns around and sends the packet out over another net group. Apart from picking two net groups for this, the mechanism should be fairly transparent.

Here’s what happens when you install such a relay:

Screen Shot 2011 01 11 at 23.44.59

That only covers packets going one way. A refinement is to also deal with ACK packets. In that case, the relay should wait a little while to see whether an ACK comes in, and if it does, send that ACK back out to the original packet source:

Screen Shot 2011 01 11 at 23.45.05

Here’s a groupRelay.pde sketch, which implements this:

Screen Shot 2011 01 11 at 23.58.18

It’s pretty straightforward. But it’s only made for broadcast scenarios, i.e. the originating node must be using a broadcast packet (i.e. send to special node “0”). The relay can then simply re-broadcast the information.

In the case of ACKs, the originating node id will be in the header. The relay saves this, sends out a packet with an ACK-type request of its own, waits briefly for such an ACK to come in, and when it does, sends an ACK back to the originating node.

Note how the loss of packets will have the same effect as without a relay. The only difference being that there are 2 hops (or 4 w/ ACK) where packets can be lost or get damaged.

Tomorrow, I’ll explain a little bit what’s going on and what sorts of trade-offs this leads to.

GLCD library

In Hardware, Software on Jan 5, 2011 at 00:01

There’s a new GLCD library to drive the 128×64 graphics LCD display on the Graphics Board. The library is called, wait for it… GLCDlib – with a wiki page and a web interface to the source code in subversion. There’s also a ZIP archive snapshot, but it probably won’t get updated with each future subversion change. For some notes about using subversion (“svn”), see this post.

The main class is “GLCD_ST7565”, it has the following members:

Screen Shot 2011 01 04 at 19.52.45

(some longer entries above were truncated, see the website for the full version)

The settings in this library have been hard-coded for use with the Graphics Board, which uses ports 1 and 4 to drive this display. If you want to use this with other I/O connections, you’ll need to change the #define’s at the top of the “GLCD_ST7565.cpp” source file in the library.

Here is the demo used in an earlier post, now included as “glcd_demo.pde” example sketch in the library:

Screen Shot 2011 01 04 at 19.49.49

This produces an output screen similar to this image. Note the use of flash-based string storage with “PSTR” to reduce RAM usage. It not an issue in this example, but more strings tend to rapidly consume RAM, leading to strange and hard-to-find bugs.

The nice thing about GLCDlib, is that you can also use it over wireless. There is a “GLCD_proxy” class, which sends all graphics commands out to another node. Each command is sent as a packet, complete with ACKs, retries, and resends to deal with lost packets.

The “JeePU.pde” example sketch implements the “host”, i.e. a JeeNode with Graphics Board, listening to incoming wireless requests. The “JeePU_demo.pde” sketch shows how to talk to such a remote JeePU node.

Because the transport layer (i.e. wireless or other comms mechanism) is separated out from the main graphics primitives, it is very easy to switch between a locally-connected GLCD and a remote one on a JeePU node. The magic is contained mostly in these lines:

Screen Shot 2011 01 04 at 20.02.40

The only other change needed to use a remote GLCD is to add these lines at the start of setup():

Screen Shot 2011 01 04 at 20.04.57

See the JeePU_demo.pde sketch for an example of how this can be used.

The JeePU node should be running in its own RF12 net group, because clients use broadcasts to send out the graphics commands. They do not need to know the node ID of the JeePU, just its net-group. This also means that multiple GLCD proxy clients can run at the same time, and each could be adjusting a specific part of the same JeePU display … whee, a multi-node status display!

One of the advantages of running the Graphics Board as a JeePU node, is that the other nodes don’t need to load the entire GLCDlib code, in particular they avoid the 1 Kb RAM buffer needed to drive the display.

The graphics code is based on what used to be the ST7565 library by Limor Fried at AdaFruit, which was in turn derived from public domain code attributed to “cstone@pobox.com”.

Several great extensions (and a couple of bug fixes) for the core graphics routines were written by Steve Evans (aka tankslappa on the wiki and forum). Steve also implemented the remote/proxy code and the JeePU “host” and JeePU_demo “client”.

I just hacked around a bit on all that, including renaming things and adding an ACK mechanism to the RF12 wireless layer.

This code is likely to change and extend further, as we come up with more things to do with the current implementation. But for now, enjoy :)

Update – all the code is now at https://git.jeelabs.org/glcdlib.

RF12 acknowledgements

In Software on Dec 11, 2010 at 00:01

The RFM12B wireless module is a transceiver, i.e. able to send and receive packets over wireless. This is an important advantage over simple sensor units which just send out what they measure, and things like RF-controlled power switches which only listen to incoming data but are not able to report their current state.

The only thing is… it’s a bit more work.

This is reflected in how the RF12 library works:

  • simple reception is a matter of regularly polling with rf12_recvDone()
  • simple transmission means you also have to call rf12_canSend() and rf12_sendStart()
  • the above are both essentially uni-directional, so packets can get lost

The second mechanism added to RF12 was a set of “easy transmission” functions, i.e. rf12_easyPoll() and rf12_easySend(). These look similar, but they send out data packets asking for an ACK (acknowledge) packet from the receiver to confirm that the packet was correctly received. If nothing comes in, they will re-send the packet (and repeat a few times, if needed). This mechanism greatly improves the chance of a message arriving properly at the destination. Losing an occasional packet is one thing, losing all retries is a lot less likely!

Note that packets can be damaged or get lost at any time. It may well be that the original packet arrived just fine, but the ACK got lost instead. The sender will resend, and then (probably) get the ACK which stops this retry cycle.

So with the easy transmission functions, note that very occasionally a packet might be received twice. If it is crucial to weed these out, you can include a counter in your data packets to help detect and ignore duplicates.

With RF12demo as receiver, ACK handling is automatic. It knows when the originating node wants to get an ACK, and will send it out as soon as possible. This is reported in the output as the text “-> ack”.

The code for this in RF12demo is horrendous:

Screen Shot 2010 12 10 at 18.27.08

This is silly, and overkill for simple cases. So let’s improve on it.

I’ve added two utility definitions to the RF12.h header, which can simplify the above code to:

Screen Shot 2010 12 10 at 19.58.02

That’s better, eh?

The rest is just there to deal with a special configuration setting in RF12demo.

So if all you want is to add logic in your own sketch to send back an empty ACK packet when requested, the above can be simplified even further to:

Screen Shot 2010 12 10 at 19.59.09

For completeness, here’s a complete processing loop for a receiving sketch which supports nodes using the easy transmission mechanism:

Screen Shot 2010 12 10 at 19.59.45

You have to send out the ACK after processing the packet, because the rf12_sendStart() call will re-use the same packet buffer and overwrite the incoming packet.

Also, RF12_WANTS_ACK and RF12_ACK_REPLY are defined as macros which access the global rf12_hdr variable, as set by rf12_recvDone(). IOW, the convenience comes for free, but it does depend on some fixed assumptions. I can’t think of a situation where this would lead to problems, given that RF12-based sketches are probably all structured in the same way, and that globals are part of the RF12 driver.

For another example, see the blink_recv.pde sketch, which has also been simplified with these two macros.

Ethernet web client

In Hardware, Software on Nov 29, 2010 at 00:01

The Ether Card and the associated EtherCard library offer an easy way to connect a JeeNode to a LAN and internet. I consider this option useful enough to even want it within reach on a breadboard, so I designed the Bridge Board to support it as well. Here’s the setup, based on a JeeNode Experimenter’s Pack:

Dsc 2365

This is all it takes for an Ethernet-connected setup, and all JeeNode ports are still freely available for project use.

However, the EtherCard library only includes some examples for the web server approach, such as the EtherNode. What about using this setup to send requests out, and POSTing results to a remote web server?

Well, encouraged by a few great DNS-related patches to the code by Gérard Chevalier, who was also so kind to show me how he does DNS lookups and web requests, I’ve come up with two new example sketches:

  • The getStaticIP.pde sketch does just a HTTP get to a fixed IP address.
  • The getViaDNS.pde sketch looks up my server’s IP address by name, via a DNS lookup on startup.

Here is the code of getViaDNS:

Screen Shot 2010 11 27 at 15.38.35

Note that to make any kind of outgoing request outside your LAN, the IP address of the local LAN gateway has to be specified at setup time.

The dependency on the Ports library is due to the use of the MilliTimer class in this code. And because of that, the RF12 library also needs to be mentioned (it won’t increase the code size). Welcome to dependency hell…

Here is some sample output:

Screen Shot 2010 11 27 at 15.29.28

Even though this example is relatively short, I still find it quite awkward. The EtherCard library code really needs a major overhaul – far too much of the underlying mess is still being exposed for no good reason, IMO.

Furthermore, a massive code cleanup will be needed to overcome the current limitations of the EtherCard library: the need for a large RAM buffer, and the limitation that TCP requests and replies may only use a single ethernet frame (approx 1500 bytes).

But getting it working first was more important, and with these new examples you can now see how to implement outgoing TCP/IP requests (not just web/http, btw). Pachube, anyone?

Room Node display

In Software on Nov 17, 2010 at 00:01

Now that there’s a Graphics Board, I thought I’d make a little display with the last few readings from a couple of room nodes around here. Ironically, it’s just a 8×21 character text display for now – no graphics in sight:

Dsc 2281

The information consists of:

  • a packet sequence number (only 4-byte packets are treated as room nodes)
  • the node ID
  • the temperature in °C
  • the relative humidity in %
  • the measured light intensity (0..255)

New readings get added at the bottom, with older readings scrolling upwards.

Unfortunately, the ST7565 library doesn’t have a normal print() & println() API, so the first thing I did was to create a new wrapper class:

Screen Shot 2010 11 16 at 12.42.57

One quirk about this code is that since we’re using a RAM buffer, the ST7565 screen contents needs to be explicitly updated. I solved it by adding a poll() method which you need to call in the main loop. It’ll make sure that the display gets refreshed shortly after anything new has been “printed” (default is within 0.1 s).

Another thing this class does is to scroll the contents of the display one line up when the bottom is reached. It does this in a slightly lazy manner, i.e. the display is not scrolled immediately when a newline is sent to it but when the first character on the next line falls outside the display area – a subtle but important difference, because it lets you use println() calls and the display won’t constantly leave an empty line at the bottom.

Scroll support does require one change to the “ST7565.cpp” source code. This:

    static byte gLCDbuf[1024];

Has to be changed as follows, to make the RAM buffer accessible from other source files:

    byte gLCDbuf[1024];

(should be around line 42 in ST7565.cpp)

With that out of the way, here’s the glcdNode.pde sketch, which has been added to the RF12 library:

Screen Shot 2010 11 16 at 12.53.36

For debugging purposes, the same information shown on the display is also sent to the serial port:

Screen Shot 2010 11 15 at 02.24.28

Note that gldeNode is hard-coded to receive packets from net group 5 @ 868 MHz, as you can see in the call to rf12_initialize().

So now I have a battery-powered wireless gadget which lets me track what our house is trying to tell us!

Update – the ST7565 library needs to be changed a bit more, I now realize. Perhaps the easiest way to do so is to simply insert the following line somewhere near the top of ST7565.c:

    #define buffer gLCDbuf

That way, the buffer will have a more meaningful name when made global by the above-mentioned patch.

Update #2 – no more need for the ST7565 library, use the new GLCDlib instead. The glcdNode demo sketch has been adapted and moved over to it.

More RF12 driver notes

In Software on Sep 15, 2010 at 00:01

Yesterday’s RF12 driver changes added a new feature to capture packets from different net groups, and to easily send out packets to different netgroups (don’t hold your breath for a “group broadcast” to every node out there).

As reported on the forum, it’s still not perfect due to the high number of false packet triggers.

One effect of this is that correct packets are missed occasionally. Here is the output from yesterday again:

OKG 15 4 248 48
OKG 15 4 250 48
OKG 5 61 7 23 83 97 7 0 155 79
OKG 15 4 251 48
OKG 15 4 252 48
OKG 15 4 7 49
OKG 15 4 255 48

Node 4 in net group 15 is a little test node which sends out a packet once a second (it’s the radipBlip.pde sketch, in fact). And as you can see, 3 packets already got lost in that brief time window.

My hunch is that the radio syncs on far too many bit patterns. This starts the radio off, collecting bytes, until either the proper packet length has been reached or the maximum of 68 bytes (hdr + length + data).

The problem is that a correct preamble and sync pattern right after such a false start will be interpreted as data. It would be possible to solve this in software, by passing a set of valid groups to the RF12 driver, and have it abort the reception immediately if something comes in. Then the driver can immediately restart a fresh sync search, thus hopefulyl capturing a subsequent real packet.

More work will be needed in this direction, but I wanted to get the change in so others can try it out and have a go at coming up with improvements for this code.

Two more small changes were added to the RF12 driver: a way to use it in “buffer-less transmit” mode, and a slightly more flexible way to wait for the completion of packet sending.

Buffer-less transmit mode is a way to avoid having to use extra memory for sending out packets. An example:

    if (rf12_canSend()) {
        struct { ... } *buf = (void*) rf12_buf;
        rf12_len = sizeof *buf;
        // fill in values here, i.e. "buf->temp = 123", etc.
        rf12_sendStart(0);
    }

So instead of passing an area of memory to copy into the rf12_buf area, as done with a traditional call to the 3-argument rf12_sendStart(), you set up a pointer directly into that area and store results directly into them.

The crucial point is that you can only do this between the time when rf12_canSend() returns 1 and a subsequent call to rf12_sendStart(). That is the only time when the data in rf12_buf is guaranteed not to get changed by the RF12 driver itself.

In many cases, you don’t need all this trickery. This is just a way to reduce the amount of (scarce) ram needed to work with packets and the RF12 driver.

The other change in the RF12 driver concerns the following call, which uses the optional “sync” parameter to specify how to wait for send-completion:

    rf12_sendStart(hdr, buf, len, sync);

This recent addition to the RF12 driver is useful for reducing power consumption, because it lets you send out a packet through the RFM12B while the ATmega itself enters a low power mode.

I have added a new function called rf12_sendWait() which now does that waiting side separately. The above call is still available for backward compatibility, but for future use, it should be written as two calls:

    rf12_sendStart(hdr, buf, len);
    rf12_sendWait(sync);

The reason for this change, is that this now also supports the buffer-less transmit mode described above:

    if (rf12_canSend()) {
        struct { ... } *buf = (void*) rf12_buf;
        rf12_len = sizeof *buf;
        // fill in values here, i.e. "buf->temp = 123", etc.
        rf12_sendStart(0);
        rf12_sendWait(2);
    }

Small changes, all in all. This will of course be documented on the cafe/docs site, but since that site is going to be replaced by a more advanced wiki-based setup in the near future, this post will have to do for now. I don’t want to deal with two sets of documentation… maintaining one set is hard enough!

New RF12 driver mode

In Software on Sep 14, 2010 at 00:01

The RF12 driver has been extended with the ability to send and receive packets from any net group. This feature was long overdue – it allows a node to listen to packets coming from different net groups, and could be used to implement a packet forwarding “relay” or a mesh network.

The way to activate this “any net-group” mode, is to initialize the RF12 driver with net group zero:

    rf12_initialize(12, RFM_868MHZ, 0);

This puts the RFM12B in a special 1-byte sync mode. The new rf12_grp variable will contain the net group whenever a packet is received, i.e. whenever rf12_recDone() returns true.

To send a packet to a specific netgroup, say 34, you can now use the following code:

    if (rf12_canSend()) {
        ...
        rf12_grp = 34;
        rf12_sendStart(...);
    }

The RF12demo.pde has been extended to support this “any net-group” mode – simply set the group as “0g”.

When use in this special mode, the format of the reported lines changes slightly: “OK” becomes “OKG” plus the net group number – followed by the header byte + data bytes, as usual.

Here is some sample output, first normal mode, then any net-group mode:

Current configuration:
 A i1* g5 @ 868 MHz 
OK 3 215 66 0 0
OK 61 7 23 83 97 7 0 155 79
OK 61 52 239 0 218 39
OK 61 9 2 8 68 161 75 15 0 16 0
OK 3 217 66 0 0
OK 61 52 240 0 222 39
> 0g
 ?G 45 25 37 209 155 76 99 138 66 247 140 29 251 47 157 163 51 158
 ?G 7 133
 ?G 144 133 255 220 119 254 229 249 92 225 94 213 221 102 160 1 233 59 251
 ?G 255 185 205 23 165 55 175 172 161 229 207 108 141 152 56 127 208 134
> 1q
OKG 15 4 248 48
OKG 15 4 250 48
OKG 5 61 7 23 83 97 7 0 155 79
OKG 15 4 251 48
OKG 15 4 252 48
OKG 15 4 7 49
OKG 15 4 255 48

There is however a serious problem with the promiscuous / any net-group mode: the RFM12B module will report a lot of garbage. I added a “quiet” option to RF12demo to be able to suppress the reporting of bad packets, i.e. dropping “?…” lines. That’s why no “?…” lines were reported once the “1q” command was entered.

As you can see in the output, packets from both net-groups 5 and 15 are picked up and reported. Yippie!

The new version of the RF12 library has been checked into subversion and is also available as a ZIP file, see the software page for details.

There is a bit more to say about these changes… so stay tuned!

Pulling data from an EtherNode

In Software on Aug 17, 2010 at 00:01

Last month’s EtherNode sketch was an example of a simple web server which allows viewing incoming packets received by the RFM12B. Here’s a sample web page again:

Screen Shot 2010 07 13 at 231929

If JeeMon could access and pick up that data without requiring an extra JeeLink or JeeNode, then you could place the EtherNode wherever reception is best while running JeeMon on your desktop machine, or anywhere else.

In response to a request on the forum for just that, I started writing a little demo “application.tcl” for JeeMon to do this sort of web-scraping. Here’s what I came up with (code):

Screen Shot 2010 08 16 at 10.35.49

Sample console output:

Screen Shot 2010 08 16 at 10.42.48

The point here, is that it needs to periodically poll the EtherNode, get a web page from it, and skip the readings it has already seen before. That’s what most of the code in “EtherNodePull” does. Each packet that remains will be sent to the “GotPacket” proc, which just logs it on the console.

But that’s just one half of the required solution…

The bigger challenge is to also make JeeMon decode these packets, as if they came in through a serial USB link. There is quite a bit of logic in sketches/central/host.tcl to do that for a JeeNode or JeeLink running the “central” sketch (which is almost identical to RF12demo).

The reason this is more complicated, is that I want to be able to decode each packet in different ways, depending on the sketch running on the remote (sending) node. My network has more than just room nodes, and will be extended with many more node types in the future.

One workaround would be to collect all nodes of the same type in their own group, i.e. net group 1 for room nodes, net group 2 for the ookRelay, etc. And yes, that would work – but it’s not very convenient, and I’d need separate etherNodes to pick up the packets from each net group. Messy.

The approach I have used so far, is to maintain a config section for JeeMon, with information about the type of each node, organized by frequency band, net group, and node id:

Screen Shot 2010 08 16 at 10.52.23

It’s not automatic, but this way I just need to adjust one list whenever a new wireless node is brought online.

The current code in sketches/central/host.tcl is all about picking up packets, and mapping them thtough this configuration section to know what is what. It does this by setting up a pseudo “connection” whenever packets come in for the first time and includes logic to tear down this connection again when no new packets are received within a certain amount of time.

To use this approach with an EtherNode as data collection node, I need to re-factor the exisiting code and make the core mechanism independent of the Serial implementation. I also need to bring more of the code from central/host.tcl into the JeeMon code, so it can be re-used for EtherNodes.

Re-factoring is my middle name – I’ll update this post when the code changes are complete.

Improved EtherNode

In Software on Jul 14, 2010 at 00:01

The EtherNode.pde sketch described in this post not so long ago has been extended a bit with some of the functionality of the RF12demo.pde sketch.

That way, an EtherNode can now also be used as sort of a replacement for RF12demo – through Ethernet i.s.o. USB: i.e. to collect data (by polling the server and scraping all new data from the home page) and to send out packets (via a GET request with a properly-formatted HTTP query string).

First of all, the configuration settings are now saved to EEPROM, so that the webserver will come back with the same settings after a restart:

Screen Shot 2010 07 13 at 23.13.11

As you can see, the “collect” mode has also been added. When enabled, the EtherNode will not reply with an ACK when receiving a packet. The issue here is that at most one node should respond with an ACK to broadcast packets. IOW, for each net group you use, you should normally have a single node running RF12demo or EtherNode with collect mode turned off. The others will be “lurking” (with collect mode enabled), i.e. they will be able to see all packets but they won’t reply.

Note: there is one other way to deal with collect mode and ACKs: if you have software such as JeeMon (or your own code) set up to explicitly generate an ACK, then again you should set collect mode on, so that the ATmega itself doesn’t generate the ACK as well. This is why this feature was called “collect mode” btw: the ATmega collects packets and passes it hrough, but it does not act as a full receiver which sends out ACKs.

Also new is that for debugging, you can now connect a serial console to see the full incoming requests:

Screen Shot 2010 07 13 at 23.12.22

The main screen hasn’t changed very much:

Screen Shot 2010 07 13 at 23.19.29

The only changes are a new link to a “send packet” page, and the fact that packets are now properly numbered from 0000 to 9999 (and then wrapping back to 0000).

The send packet page is new: it lets you send out a packet with up to 66 bytes of arbitrary data, to either a specific node, or as broadcast:

Screen Shot 2010 07 13 at 23.43.52

There is currently no way to request an ACK or send out ACKs via this mechanism. It’s just for sending out a data packet via a web request.

This is just the beginning of what’s possible with a JeeNode hooked up to Ethernet, of course. The sky is the limit, since everything else is a matter of uploading new software. Long live open source :)

EtheRBBBreadboard

In AVR, Hardware, Software on Jun 7, 2010 at 00:01

Please excuse the crazy title. This picture probably tells you more:

Dsc 1509

You’re looking at an RBBB from Modern Device, hooked up to the Ether Card from Jee Labs. The hookup is trivial, it need 6 wires: 2 for power and 4 for SPI:

Screen Shot 2010 05 29 at 04.18.26

And that’s all you need to create a webserver!

Here’s a sample screen (yep, it’s been running for almost 12 hours):

Screen Shot 2010 05 29 at 00.03.48

And here’s the code for it:

Screen Shot 2010 05 28 at 12.05.03

It presents a self-refreshing page with the “uptime”, i.e. how long the server has been running since power-up.

The code is available as “rbbb_server.pde” example sketch in the EtherCard library and is a simplified adaptation of the etherNode sketch, omitting the RF12 driver and calls, of course. It compiles to 6 Kb of code.

As with the etherNode sketch, the MAC address and IP address have to be set to suitable values in the source code before uploading it to the RBBB.

BTW… did you notice anything?

The RBBB is running at 5V. And it works. Because the Ether Card is compatible with 3.3V and 5V signals!

Which also means that the Ether Card can be used with any Arduino board. It’s not specific to JeeNodes and it’s not limited to being used with the Carrier Board, either.

The generic software for the Ether Card is contained in the “EtherCard” library, listed on the software page in the Café. It doesn’t depend on Ports or RF12 libraries, so this thing really is a completely independent product.

It just “happens” to fit gorgeously into a box alongside a JeeNode :)

So there you have it – the Ether Card can be used with just about any microcontroller setup. All it needs is a power supply of 3.6V or more, 4 SPI I/O pins, and the proper software to make it sing of course…

Onwards!

JeeNode as web server

In AVR, Software on May 31, 2010 at 00:01

The new Ether Card add-on makes a lot of new fun stuff possible. Even with a measly 8-bit ATmega chip, it’s possible to create some pretty neat things. Like a little webserver for displaying the last received RF12 packets.

I’ve added an EtherCard library to the subversion repository, next to the Ports and RF12 libraries, etc. It’s now listed on the software page in the Café.

This is mostly the Ethernet code by Guido Socher and Pascal Stang, but I’ve moved some stuff around a bit, and kept the changes added by Andras Tucsni to make this work together with the RF12 driver.

A new “BufferFiller” class has been added, to help create web pages and such:

Screen Shot 2010 05 22 at 00.04.55

What it does is expand a PSTR given to it, by replacing $S, $F, and $E with strings copied from RAM, flash, or EEPROM, respectively. A bit like “printf” in standard C. It’s easiest to just show an example of use:

Screen Shot 2010 05 22 at 01.42.00

The $D code is expanded as integer, for convenience. And since BufferFiller derives from the Arduino’s “Print” class, all the usual “print()” and “println()” members are also available.

The above code will generate the following web page (and yes, it’s been up and running more than 3 days):

Screen Shot 2010 05 25 at 09.55.06

It’s part of the etherNode.pde sample sketch in the EtherCard library – a little web server which shows the last few incoming RF12 packets and continuously self-refreshes to present the latest info.

There’s also a configuration page:

Screen Shot 2010 05 21 at 18.46.32

The whole etherNode demo sketch is under 10 Kbyte, including both EtherCard and RF12 drivers, leaving plenty of room to implement more features.

Assembling the Ether Card

In Hardware on May 30, 2010 at 00:01

(This is going to be a long post, so I’ve split it up a bit)

Here are the steps needed to assemble the Ether Card kit, starting from this PCB:

Dsc 1433

This board is very easy to build, since it uses only through-hole parts and has relatively few components. The basic idea is to build from the flattest to the highest components. That way, when you turn the card over for soldering, you can push on it to press the component against the board.

So let’s start with the 7 resistors. Don’t mix them up, they have three different color-coded values:

Dsc 1435

Turn over the board, don’t use too much solder, make clean solder joints, and snip the wires off when done:

Dsc 1436

The important thing is to get the solder flowing into the plated-through holes.

Next, the ferrite bead (a small inductor which blocks high frequencies):

Dsc 1439

More…

Read the rest of this entry »

Meet the Ether Card

In AVR, Hardware, Software on May 24, 2010 at 00:01

Yesterday’s post was a sneaky way to show you a glimpse of an exciting new addition to the always-evolving Jee Labs product range – meet the Ether Card !

Dsc 1454

It’s a low-end Ethernet extension card, with a form-factor specifically made for the Carrier Board + Box:

Dsc 1467

It is based on the good ol’ trusty ENC28J60 chip, with all the components needed to hook it up to a JeeNode (or any other unit running the SPI bus at 3.3V).

I’ve started working a bit on the software side. The Ether Card is pretty standard in every respect, with the GPL2 code by Pascal Stang and Guido Socher working just fine with it. All it needs is a different chip select pin (PB0, Arduino pin 8) and proper interrupt guards to prevent the RF12 driver from interfering.

The card has been running smoothly for days on end here. It gets slightly warm, I’d estimate some 20°C above ambient. The regulator stays cool when powered from 5V. Total current draw is ≈ 150 mA, incl. JeeNode.

The Ether Card only uses through-hole parts, no SMDs. It has been added to the shop as kit and as PCB-only version. Here’s the PCB – in glorious blue-and-gold:

Dsc 1433

Here’s a sample web server requiring under 10 Kb of flash and showing a page with the last 25 RF12 packets:

Screen Shot 2010 05 19 at 11.33.18

So there you have it. JeeNodes can now handle wireless and wired networking.

It’s going to be oodles of fun to develop software for / on / with this Ether Card!

Credits: I would like to thank Andras Tucsni, who started the ball rolling by prototyping a complete working system and implementing the chip-select and interrupt changes needed for inter-operation with the RF12 driver on SPI. Andras also wrote the demo sketch which generated the above output. Knowing that it can be done and having working sample code makes a huge difference!