Ok, so we have JeeNodes and JeePlugs, and it’s now possible to sense and hook up all sorts of fun stuff. In theory, it’s all trivial to use and easy to integrate with what you already have, right? Well… in practice there’s a lot of duplication involved – literally, in fact: for my experiments, I often take an existing sketch, make a fresh copy and start tweaking it. Shudder…

• New plug. New bit of code to include in the sketch for that plug. New sketch.

• New device connected. New bit of code to talk to that device. New sketch.

• New idea. New logic to implement that idea. New sketch.

Yawn. Some of this WSN stuff sure is starting to become real tedious…

There are a couple of ways to deal with this. The traditional way is to modularize the source code as much as possible: create a separate header and implementation source file for each new plug, device, sensor, and function which might need to be re-used at some point. Then all you have to do is create a new sketch (again!) and include the bits and pieces you want to use in this sketch.

I have a big problem with that. You end up with dozens – if not hundreds – of tiny little files, all with virtually no code in them, since most interfaces definitions and interface implementations are trivial. My problem is not strictly the number of little files, but the loss of overview, and the inability to re-factor such code collections across the board. It just becomes harder and harder to simplify common patterns, which only show after you’ve got a certain amount of code. The noise of the C/C++ programming itself starts to drown out the essence of all these (small & similar) bits of interface code.

The other serious problem with too fine-grained modularization of the source code, is that you end up with a dependency nightmare. Some of my sketches need the RF12 driver, other need the PortsI2C class, yet others use the MilliTimer.

At the opposite end of the spectrum is the copy-and-paste until you drop approach, whereby you take the code (i.e. sketches) you have, and make copies of it all, picking the pieces you want to re-use, and dropping everything else. I’ve been doing that a bit lately, because most of this code is so trivial, but it’s a recipe for disaster – not only do I end up with more and more slightly different versions of everything over time, it also becomes virtually impossible to manage bug fixes and fold them into all the affected sources.

A version control system such as subversion can help (I couldn’t live without it), but it just masks the underlying issues, really. Being that some parts of the code deal with the essence of the interface, and other parts exists just to make the code into a compilable unit.

There is another alternative: go all out with C++ and OO, i.e. create some class hierarchies and make lots of members virtual. So that slight variations of existing code can be implemented as derived classes in C++, with only a bit of code for the pieces which differ from the previous implementation. This is not very practical on embedded microcontrollers such as the ATmega, however. V-tables (the technique used internally in C++ to implement such abstractions) tend to eat up memory when used for elaborate class hierarchies, and worse still, much of that memory will have to be in RAM.

There is a solution for this too, BTW: C++ templates. But I fear that the introduction of template programming (and meta-programming) is going to make the code virtually impenetrable for everyone except hard-core and professional C++ programmers. Already, my use of C++ in sketches is scaring some people off, from what I hear…

Is there a way to deal with a growing variety of little interface code snippets, in such a way that we don’t have to bring in a huge amount of machinery? Is there some way to plug in the required code in the same way as JeePlugs can be plugged in and used? Can we somehow re-use bits and pieces without having to copy and paste sketches together all the time?

I think there is…

The approach I’d like to introduce here is “code generation”. This technique has been around for ages, and it has been used (and abused) in a wide range of tasks.

The idea is to define a notation (a related buzzword is “DSL“) which is better suited for the specific requirements of Physical Computing, Wireless Sensor Nodes, and Home Automation. And then to generate real C/C++/sketch code from a specification which uses this notation to describe the bits and pieces involved:

To create a sketch for a JeeNode with the Room Board on it and using say an EtherCard as interface to the outside world, one could write something like the following specification:

The key point to make here is that this is not really a new language. The code you add is the same code you’d write if you had to create the sketch from scratch. But the repetitive stuff is gone. In a way, this is copy-and-paste taken to extremes: it is automated to the point that you no longer have to think of it as copying: all the pieces are simply there for immediate re-use.

Problems will not be gone simply by switching to a code generator approach. There will still be dependencies involved, for example. The “RoomBoard” device might well need the MilliTimer class to function properly. But it is no longer part of the code you write. It doesn’t show up in the source file, there’s no #include line as there would be in C/C++ or in a sketch. Which means it also no longer matters at this level whether the RoomBoard driver uses a MilliTimer class or not.

Code generation in itself also doesn’t solve the issue of having lots of little snippets of code. But what you can do, is combine lots of them together one source file, and then have the generator pick what it needs each time it is used:

Define RoomBoard {
    ...
}
Define EtherCard {
    ...
}

The technique of code generation has many implications. For one, you have to go through one more step before the real code can be compiled and then uploaded – you have to first produce an acceptable sketch. And with mistakes, the system has to be able to point you to the error in the original specification file, not some very obscure C/C++ statement in the generated source code.

And of course it’s a whole new mechanism on top of what already exists. One more piece of the puzzle which has to be implemented and maintained (by someone – not necessarily you). And documented – because even if the specification files can reduce a large amount of boilerplate code to a single line, that one line still needs to be clearly documented.

So far, these notes are just a thought-experiment. I’ll no doubt keep on muddling along with new sketches and plugs and nodes for some time to come.

But wouldn’t it be nice if one day, something like this were a reality?