Yesterday’s post was about the desire to automate node discovery a bit more, i.e. having nodes announce and describe themselves on the wireless network, so that central receivers know something about them without having to depend on manual setup.
The key is to make a distinction between in-band (IB) and out-of-band (OOB) data: we need a way to send information about ourselves, and we want to send it by the same means, i.e. wirelessly, but recognisable in some way as not being normal packet data.
One way would be to use a flag bit or byte in each packet, announcing whether the rest of the packet is normal or special descriptive data. But that invalidates all the nodes currently out there, and worse: you lose the 100% control over packet content that we have today.
Another approach would be to send the data on a different net group, but that requires a second receiver permanently listening to that alternate net group. Not very convenient.
Luckily, there are two more alternatives (there always are!): one extends the use of the header bits already present in each packet, the other plays tricks with the node ID header:
There are three header combinations in use, as described yesterday: normal data, requesting an ACK, and sending an ACK. That leaves one bit pattern unused in the header. We could define this as a special “management packet”.
The out-of-band data we want to send to describe the node and the packet format is probably always sent out as broadcast. After all, most nodes just want to report their sensor readings (and a few may listen for incoming commands in the returned ACK). Since node ID zero is special, there is one combination which is never used: sending out a broadcast and filling in a zero node ID as origin field. Again, this combination could be used to mark a “management packet”.
I’m leaning toward the second because it’s probably compatible with all existing code.
So what’s a “management packet”, eh?
Well, this needs to be defined, but the main point here is that the format of management packets can be fully specified and standardised across all nodes. We’re not telling them to send/receive data that way, we’re only offering them a new capability to broadcast their node/packet properties through these new special packets.
So here’s the idea, which future sketches can then choose to implement:
- every node runs a specific sketch, and we set up a simple registry of sketch type ID’s
- each sketch type ID would define at least the author and name (e.g. “roomNode”)
- the purpose of the registry is to issue unique ID’s to anyone who wants a bunch of ’em
- once a minute, for the first five minutes after power-up, the sketch sends out a management packet, containing a version number, a sequence number, the sketch type ID, and optionally, a description of its packet payload format (if it is simple and consistent enough)
- after that, this management packet gets sent out again about once an hour
- management packets are sent out in addition to regular packets, not instead of
This means that any receiver listening to the proper net group will be able to pick up these management packets and know a bit more about the sketches running on the different node ID’s. And within an hour, the central node(s) will have learned what each node is.
Nodes which never send out anything (or only send out ACKs to remote nodes, such as a central JeeLink), probably don’t need this mechanism, although there too it could be used.
So the only remaining issue is how to describe packets. Note that this is entirely optional. We could just as easily put that description in the sketch type ID registry, and even add actual decoders there to automate not only node type discovery, but even packet decoding.
Defining a concise notation to describe packet formats can be very simple or very complex, depending on the amount of complexity and variation used in these data packets. Here’s a very simple notation, which I used in JeeMon – in this case for roomNode packets:
light 8 motion 1 rhum 7 temp -10 lobat 1
These are pairs of name + number-of-bits, with a negative value indicating that sign extension is to be applied (so the temp field ranges from -512..+511, i.e. -51.2..+51.1°C).
Note that we need to fit a complete packet description in a few dozen bytes, to be able to send it out as management data, so probably the field names will have to be omitted.
This leads to the following first draft for a little-endian management packet format:
- 3 bits – management packet version, currently “1”
- 5 bits – sender node ID (since it isn’t in the header)
- 8 bits – management packet number, incremented with each transmission
- 16 bits – unique sketch type ID, assigned by JeeLabs (see below)
- optional data items, in repeating groups of the form:
- 4 bits – type of item
- 4 bits – N-1 = length of following item data (1..16 bytes)
- N bytes – item data
Item types and formats will need to be defined. Some ideas for item types to support:
- type 0 – sketch version number (a single byte should normally be enough)
- type 1 – globally unique node ID (4..16 bytes, fetched from EEPROM, could be UUID)
- type 2 – the above basic bit-field description, i.e. for a room node: 8, 1, 7, -10, 1
- type 3 – tiny URL of an on-line sketch / packet definition (or GitHub user/project?)
Space is limited – the total size of all item descriptions can only be up to 62 bytes.
Sketch type ID’s could be assigned as follows:
- 0..99 – free for local use, unregistered
- 100..65535 – assigned through a central registry site provided by JeeLabs
That way, anyone can start implementing and using this stuff without waiting for such a central registry to be put in place and become operational.
Tomorrow, a first draft implementation…