Computing stuff tied to the physical world

Binary packet decoding – part 2

In AVR, Software on Dec 8, 2010 at 00:01

Yesterday’s post showed how to get a 2-byte integer back out of a packet when reported as separate bytes:

Unfortunately, all is not well yet. Without going into details, the above may fail on 32-bit and 64-bit machines when sending a negative value such as -12345. And it’s not so convenient with other types of data. For example, here’s how you would have to reconstruct a 4-byte long containing 123456789, reported as 4 bytes:

Screen Shot 2010 12 07 at 09.56.08

And what about floating point values and C structs? The trouble with these, is that the receiving party doing the conversion needs to know exactly what the internal byte representation of the ATmega is.

Here is an even more complex example, as used in the roomNode.pde sketch:

Screen Shot 2010 12 07 at 08.44.28

This combines different measurement values into a 4-byte C struct using bit fields. Note how the “temp” value crosses two bytes, but only uses specific bits in them.

Fortunately, there is a fairly simple way to deal with all this. The trick is to decode the values back into meaningful values by the receiving ATmega instead of an attached PC. When doing so, we can re-use the same definition of the information. By using the same hardware and the same C/C++ compiler on both sides, i.e. the Arduino IDE, all internal byte representation details can be left to the compiler.

Let’s start with this 2-byte example again:

I’m going to rewrite it slightly, as:

Screen Shot 2010 12 07 at 08.57.23

No big deal. This sends out exactly the same packet. But now, we can rewrite the receiving sketch as follows:

Screen Shot 2010 12 07 at 09.00.14

The effect will be to send the following line to the serial / USB connection:

    MEAS 12345

The magic incantation is this line:

Screen Shot 2010 12 07 at 09.01.45

It uses a C typecast to force the interpretation of the bytes in the receive buffer into the “Payload” type. Which happens be the same as the one used by the sending node.

The benefit of doing it this way, is that the same approach can be used to transfer any type of data as a packet. Here is an example how a Room Node code sends out a 4-byte struct with various measurement results:

Screen Shot 2010 12 07 at 09.07.07

And here’s how the receiving node can convert the bytes in the packet back to the proper values:

Screen Shot 2010 12 07 at 09.10.55

The output will look like:

    ROOM 123 1 78 -15 0

Nice and tidy. Exactly the values we were after!

It looks like a lot of work, but it’s all very straightforward to implement. Most importantly, the correspondence between what happens in the sender and the receiver should now be obvious. It would be trivial to include more data. Or to change some field into a long or a float, or to use more or fewer bits for any of the bit fields. Note also that we don’t even need to know how large the packet is that gets sent, nor what all the individual bytes contain. Whatever the sender does to map values into a packet, will be reversed by the receiver.

This works, as long as the two struct definitions match. One way to make sure they match, is to place the payload definition in a separate header file, say “payload.h” and then include that file in both sketches using this line:

Screen Shot 2010 12 07 at 09.16.47

The price to pay for this flexibility and “representation independence”, is that you have to write your own receiving sketch. The generic RF12demo sketch cannot be used as is, since it does not have knowledge of the packet structures used by the sending nodes.

This can become a problem if different nodes use different packets sizes and structures. One way to simplify this, is to place all nodes using the same packet layout into a single net group, and then have one receiver per net group, each implemented in the way described above. Another option is to have a single receiver which knows about the different types of packets, and which switches into the proper decoding mode depending on who sent the packet.

Enough for now. Hopefully this will help you implement your own custom WSN to match exactly what you need.

Update – Silly mistake: the “rf12_sendData()” call doesn’t exist – it should be “rf12_sendStart()”.

  1. I have to say I disagree, and I’m surprised to see you support this plan. Your whole reason on the previous page for not sending data as strings was to save memory and overhead, and here you’re doing it all over again. (I know you’re saving wireless bytes, but whether it’s wired or wireless, you still have to do it)

    I completely agree with binary wire packets, no doubt there, but I think it’s vastly preferable to do the binary decoding back on the host pc, rather than on the wireless receiver. For all the same reasons, memory constraints, the obtuseness of C vs [fun language]

    • You have a point. My reason for suggesting this approach, is that it will be easier for those who don’t want to deal with the way binary data is represented. On a PC, you’re going to have to create a mechanism to perform the proper unpacking, with all the intricacies of bit fields, sign-extension, and floats (but not alignment, since the ATmega has none).

      The scenario I’m addressing is one with one or more (clever/special) custom remote nodes, and one central node which is only used as gateway for wireless packets into and out of the PC (e.g. a JeeLink). In that case, RAM will probably not be scarce since the receiver is not doing anything complicated, and strings can probably be used just fine (flash-based if need be, but that’s another topic).

      Having said this – I’m actually going to do exactly what you propose: I intend to keep the generic RF12demo sketch on the central node (or something derived from it), and define a little notation in JeeMon to easily allow it to unpack the binary data back into meaningful values. But not everyone is going to use JeeMon, I expect.

      This post was written because people keep running into this, even with just a single int. The above is general enough to handle lots of individual cases, IMO.

      I think there’s room for both approaches.

  2. Thanks for this post. I’ve always struggled with pointers. I completely agree that the way to decode the wireless payload is to use the same struct and processor on both ends. Très élégant!

  3. Thanks for posting this example, it was exactly what I needed for my setup and it is laid out very clearly (took me a moment before I remembered about bitfields.) Having dealt with little- and big-endian compatibility issues before, I appreciate something simple like this.

  4. by the way, where is rf12_sendData() defined? is it similar to rf12_sendStart?

  5. My thanks also – great set of crystal clear posts for those like me who are easily confused.

  6. Nice write-up. Very helpful to someone like me who is a relative C novice. Thanks Dan

  7. This post really solved some of the troubles i’m having :) Being used to higher order languages really becomes a problem now and then.

    Thanks Jean

  8. (256 * a) is better as shift of 8 (a << 8). It better expresses the semantics of what you are doing.

    It might also be more efficient than multiplying.

  9. As a complete newbie, I’m going to have to re-read this a couple of times to get close to understanding it. Not becuase your post was anything less than excellent, but becuase some of these principles are brand new to me.

    Where you say “The generic RF12demo sketch cannot be used as is, since it does not have knowledge of the packet structures used by the sending nodes.”, does this mean that data that is received by the skecth but is different to what is expected gets deemed a bad packet and ignored or is that the purspoe of the ‘q’ command?

    • What I meant to say is that RF12demo does not know how to decode packets, since that depends on the sketches used to send them. So all it can do is pass the raw data through as bytes.

      The bad packets are those for which the checksum is incorrect. Those are always rejected. The “1q” command causes them to not even get reported.

  10. I’m trying to send a struct composed of floats. That goes OK. What about hot to detach individual values while using JeeMon sketches? Are there any examples of that?

Comments are closed.