Computing stuff tied to the physical world

Posts Tagged ‘JeeMon’

Goodbye JeeMon

In Software, Musings on Jun 6, 2012 at 00:01

As long-time readers will know, I’ve been working on and off on a project called JeeMon, which bills itself as:

JeeMon is a portable runtime for Physical Computing and Home Automation.

This also includes a couple of related projects, called JeeRev and JeeBus.

JeeMon packs a lot of functionality: first of all a programming language (Tcl) with built-in networking, event framework, internationalization, unlimited precision arithmetic, thread support, regular expressions, state triggers, introspection, coroutines, and more. But also a full GUI (Tk) and database (Metakit). It’s cross-platform, and it requires no installation, due to the fact that it’s based on a mechanism called Starkits.

Screen Shot 2012 05 23 at 19 26 10

I’ve built several version of this thing over the years, also for small ARM Linux boards, and due to its size, this thing really can go where most other scripting languages simply don’t fit – well under 1 Mb if you leave out Tk.

One of (many) things which never escaped into the wild, a complete Mac application which runs out of the box:


JeeMon was designed to be the substrate of a fairly generic event-based / networked “switchboard”. Middleware that sits between, well… everything really. With the platform-independent JeeRev being the collection of code to make the platform-dependent JeeMon core fly.

Many man-years have gone into this project, which included a group of students working together to create a first iteration of what is now called JeeBus 2010.

And now, I’m pulling the plug – development of JeeMon, JeeRev, and JeeBus has ended.

There are two reasons, both related to the Tcl programming language on which these projects were based:

  • Tcl is not keeping up with what’s happening in the software world
  • the general perception of what Tcl is about simply doesn’t match reality

The first issue is shared with a language such as Lisp, e.g. SBCL: brilliant concepts, implemented incredibly well, but so far ahead of the curve at the time that somehow, somewhere along the line, its curators stopped looking out the window to see the sweeping changes taking place out there. Things started off really well, at the cutting edge of what software was about – and then the center of the universe moved. To mobile and embedded systems, for one.

The second issue is that to this day, many people with programming experience have essentially no clue what Tcl is about. Some say it has no datatypes, has no standard OO system, is inefficient, is hard to read, and is not being used anymore. All of it is refutable, but it’s clearly a lost battle when the debate is about lack of drawbacks instead of advantages and trade-offs. The mix of functional programming with side-effects, automatic copy-on-write data sharing, cycle-free reference counting, implicit dual internal data representations, integrated event handling and async I/O, threads without race conditions, the Lisp’ish code-is-data equivalence… it all works together to hide a huge amount of detail from the programmer, yet I doubt that many people have ever heard about any of this. See also Paul Graham’s essay, in particular about what he calls the “Blub paradox”.

I don’t want to elaborate much further on all this, because it would frustrate me even more than it already does after my agonizing decision to move away from JeeMon. And I’d probably just step on other people’s toes anyway.

Because of all this, JeeMon never did get much traction, let alone evolve much via contributions from others.

Note that this isn’t about popularity but about momentum and relevance. And JeeMon now has neither.

If I had the time, I’d again try to design a new programming environment from scratch and have yet another go at databases. I’d really love to spend another decade on that – these topics are fascinating, and so far from “done”. Rattling the cage, combining existing ideas and adding new ones into the mix is such an addictive game to play.

But I don’t. You can’t build a Physical Computing house if you keep redesigning the hammer (or the nails!).

So, adieu JeeMon – you’ve been a fantastic learning experience (which I get to keep). I’ll fondly remember you.

JeeMon for early birds

In Software on Nov 25, 2011 at 00:01

Time to dive in. Let’s create a development setup for JeeMon on Mac OSX, in a new folder called “jee”:

    cd ~/Desktop
    git clone git:// jee
    cd jee

That gives me the following files:

Screen Shot 2011 11 23 at 23 21 47

There is more here than strictly needed for production use – just ignore most of this for now. The main bits are “jeemon” and the “kit/” sub-folder with all the JeeRev code in it.

On Linux, the commands will be almost the same, but you’ll need to get a different JeeMon zip file.

Since I don’t use Windows myself, I’ll have to rely on help / support from others (yes, you!) to get the details right. Thanks to @tankslappa, here’s a first report of an install on XP SP3:

cd %homepath%\desktop
“C:\Program Files\Git\bin\git” clone git:// jee
Cloning into jee…
remote: Counting objects: 1810, done.
remote: Compressing objects: 100% (670/670), done.
remote: Total 1810 (delta 1187), reused 1742 (delta 1119)
Receiving objects: 100% (1810/1810), 1.45 MiB | 87 KiB/s, done.
Resolving deltas: 100% (1187/1187), done.
cd jee

Then unzip using your favorite zip manager, and you should be good to go (I’ll optimize further, one day).

Note: on Mac OSX and Linux, if “.” is not in your path, you’ll need to add it or type “./jeemon” i.s.o. “jeemon” everywhere it is mentioned below.

At this point, JeeMon is ready for use. There are a few built-in commands – here’s a quick sanity check:

    jeemon env general

The output provides some general details about the current runtime environment:

           JeeMon = v1.5
          Library = /Users/jcw/Desktop/jee/kit
         Encoding = utf-8
        Directory = /Users/jcw/Desktop/jee
       Executable = /Users/jcw/Desktop/jee/jeemon
      Tcl version = 8.6b1.1

If you got this far, then everything is working as intended. If not: you’ve hit a bug – please get in touch.

But the normal procedure is to simply launch it:


If this is the first time, you’ll get something like this:

    No application startup code found.
    21:22:29.157      app is now running in first-time configuration mode
    21:22:29.189      web server starting on

Where “first-time configuration mode” means that JeeMon didn’t find a “main.tcl” rig, which is normally used to start up. To get past this step, you need to point your web browser to the indicated URL, which’ll show this page:

Screen Shot 2011 11 23 at 21 28 54

There’s not much point in selecting anything else but “YES”at this stage. This creates a 3-line “main.tcl” file:

    # default JeeMon startup file
    Log main.tcl {in [pwd]}
    Jm needs HomeApp

From now on, JeeMon will start up using the built-in “HomeApp” rig when there are no command-line args.

The next steps depend on what you’re after – you can either dive into Tcl programming and explore how JeeRev (i.e. the kit/ area) is structured, or you can try out some examples and get a more top-down impression of it all.

To explore Tcl, the thing to keep in mind is that JeeMon will act as a standard no-frills Tcl 8.6 programming environment when launched with a source file as argument (just like tclsh and tclkit). Here’s how to make a hello-world demo – create a file called “hello.tcl” with your favorite text editor and put the following code in it:

    puts "hello, world"

Then run that code using the command “jeemon hello.tcl“. You can probably guess what it does…

If you want to use a convenient interactive shell with scrollback, history, and debugging support, download the TkCon script and launch it using “jeemon tkcon.tcl” – this uses Tk to create a mini IDE.

For Tcl manuals, books, and demos, see To really dive in, check out this tutorial or this.

But chances are that you just want to get an idea of what JeeMon does in the context of Physical Computing, House Monitoring, or Home Automation. I’ll describe just two simple examples here, and will point to the JeeMon homepage for the rest. Note that everything described below is really part of JeeRev, i.e. the standard library used by JeeMon – or to put it differently, by the source files located inside the “kit/” folder.

First, let’s launch a trivial web server (you can see the code at examples/hello-web/main.tcl):

    jeemon examples/hello-web/

You’ll see a log output line, similar to this:

    18:23:46.744      web server starting on

JeeMon is now running in async event mode, and keeps running until you force it to stop (with ^C, kill, whatever). Leave it running and go to the indicated URL in your browser. You’ll be greeted with a web page generated by JeeMon. Hit refresh to convince yourself that it really works.

Now quit JeeMon and check out the files in the “examples/hello-rf12/” folder. This example connects to a JeeNode/JeeLink running the RF12demo sketch, and displays incoming packets via its web server. But before launching JeeMon, you have to tell it what serial interface (COM or tty port) to use: copy “config-sample.txt” to “config.txt” and edit it to match your setup. Make sure the JeeNode/JeeLink is plugged in, then start JeeMon:

    jeemon examples/hello-rf12/

The output will look something like this:

    18:37:57.166      web server starting on
    18:37:58.154    rf12? [RF12demo.8] A i1* g5 @ 868 MHz 

And two dozen more lines which you can ignore. Now go to that same URL again with your web browser. If your config file defines the proper net group and you have some JeeNodes sending out packets in that net group, you’ll see them in your browser after a while. This is what I got within a few minutes, here at JeeLabs:

    #1 18:42:21 - OK 19 186 200 8 1 226 2 32 0 0 213 81 33
    #2 18:42:31 - OK 19 186 200 8 1 226 2 32 0 0 213 81 33
    #3 18:42:44 - OK 3 132 43 9 0
    #4 18:42:51 - OK 19 186 200 8 1 226 2 32 0 0 213 81 33
    #5 18:43:11 - OK 19 186 200 8 1 226 2 32 0 0 213 81 33
    #6 18:43:21 - OK 19 186 200 8 1 226 2 32 0 0 213 81 33
    #7 18:43:29 - OK 6 1 95 212 0

Now might be a good time to look at the code in examples/hello-rf12/main.tcl – to see how it all works.

Wanna experiment? Easy: quit JeeMon, create a new directory “blah“, and copy the files from the example to it. Edit them as you like, then start JeeMon using “jeemon blah“. Go ahead, try it! – the worst that can happen is that you get error messages. Usually, error tracebacks will refer to JeeRev files in the “kit/” folder.

This concludes my intro for those who want to get their feet wet with JeeMon. We’ve only scratched the surface!

I’ll work out more examples on the JeeRev page as I start building up my own setup here at JeeLabs. If you want to try things and run into trouble (there will be rough edges!) – here are two places to post problems and bugs:

Feel free to ask and post anything else on the forum. If it gets out of hand, I’ll set up a separate area for JeeMon.

A quick note about the Shop: I’m running into some shortages across the board (such as wire jumpers), which also affect the JX and WSP packs. All the missing bits are on order, but some of this might not get in before early December. My apologies for the delay and inconvenience! -jcw

JeeRev sits under the hood

In Software on Nov 24, 2011 at 00:01

Here’s another post about the JeeMonBusRev trilogy, i.e. the JeeMon project I started a few years ago.

First, let me go into why there are still two pieces and two names left in this project:

  • JeeMon is two things at once: the name of the entire system, and the name of the executable file to start it up. This exe file contains a general-purpose set of tools – it has no clue about Physical Computing or Home Automation. It could as well be used to build an accounting system or a CAD system… if you wanted to.

  • JeeRev is the name of the software which extends the JeeMon executable for domain specific use, i.e. its run time library. This is where the code lives which does know very specifically about Physical Computing and Home Automation. This is also where all the (software development) action is.

Do these names stand for anything? Yeah, probably – but I’ve stopped agonizing about cool acronyms.

JeeRev is used by JeeMon in one of two ways:

  • Development mode – as a “kit/” folder full of source files, which JeeMon will automatically detect and use. This is what you get when you grab the code from GitHub. Easy to browse, tweak, and keep up-to-date.

  • Production mode – as a single file called “jeemon-rev“. Normally, JeeMon will automatically download the latest official version of that file if it doesn’t exist yet (if there’s no “kit/” folder to put it in development mode). End users (i.e. normal people!) don’t care about JeeRev. They just download and launch JeeMon.

The precise startup process is fully configurable – it’s documented in JeeMon’s README file on GitHub.

I suggest forgetting about production mode for now. It’s not going to matter for quite some time, even though that aspect has had a major effect on the “structural design” of JeeMon. For now, everything will be happening in development mode, with JeeRev fully exposed – like a car kept in the garage with the hood open, so to speak.

To expose what’s in JeeRev, I have to describe one more mechanism – which is the way the different pieces of an application work together in JeeMon. Since I don’t know how familiar you are with recent implementations of Tcl, I’ll start from scratch and go through the basics – it’ll be easy to follow if you know some programming language:

Commands – everything in Tcl is based on commands. The line < puts "hello, world" > (without the <>’s) is treated as a call to the command puts with one argument. The difference with older versions of Tcl, is that commands can also be namespaces (it’s actually the other way around, and they’re called “ensembles”, but let’s not be picky for now). That means that a command such as the following can mean different things:

    Interfaces serial connect $device 57600

The simplest case would be that there is a command called “Interfaces” which gets called with 4 arguments. So you might expect to see a command definition somewhere in the code, which looks like this:

    proc Interfaces {type action device baudrate} {

But there is another interpretation in Tcl 8.6 (and 8.5): it can also be used to call a “serial” command defined in the “Interfaces” namespace. And in this particular case, it’s in fact nested: a command called “connect” defined in the “serial” namespace, which in turn is defined in the “Interfaces” namespace. Perhaps like this:

    namespace eval Interfaces {
      namespace eval serial {
        proc connect {device baudrate} {

Or, more succinctly:

    proc Interfaces::serial::connect {device baudrate} {

This is like writing “Interfaces.serial.connect(…)” in Python, Ruby, or Lua. But not quite – it’s slightly more general, in that callers don’t need to know how code is structured to be able to perform certain operations. It also allows you to alter the internal organization of the code later, without having to change all the calls themselves. But it can also be a bit more tricky, because you can’t tell from the appearance of a call where the actual code resides – you have to follow (or know) the chain in the code.

This is a fairly important concept in Tcl. The command “a b c d e” can lead to very different code paths (even in the same app, if things are added or re-defined at run-time), depending on whether and how “a”, “b”, etc are defined. At some point, the command name lookup “stops” and the arguments “begin”.

Apologies for the lengthy exposé. I really had to do this to introduce the next concept, which is…

Rigs – this is a mechanism I’ve been refining for the past few years now, which unifies the above nesting mechanism with the way code is arranged in source files. To continue the Interfaces example: there’s a file in JeeRev called serial.tcl, and inside is – as you would expect – a “proc connect …” definition.

The point is: that “serial.tcl” source file is located in a folder called “Interfaces”. So the nesting of command detail is reflected in the way files and folders are organized inside the “kit/” area, i.e. JeeRev.

If you’re familiar with Python, you’ll recognize the module mechanism. Yep – “rigs” are similar (not identical).

Some rig conventions: 1) definitions starting with lowercase are public (i.e. “serial” and “connect”), whereas everything else is considered private to that rig, 2) “child” rigs can access the definitions in their “parent” rigs, and 3) if you don’t like the way a built-in rig works, you can supply your own and completely override it.

If you look at this in objected-oriented terms: rigs are essentially singletons, like Python / Ruby / Lua modules. Speaking of objects: the result of the “Interfaces serial connect …” call is a connection object – Tcl 8.6 finally includes a single OO framework (there used to be tons!), so things are becoming a lot more uniform in Tcl-land.

Thanks for reading this far, even though I haven’t said anything about what’s inside JeeRev yet.

But with this out of the way, that’s easy: JeeRev consists of a collection of rigs, implementing all sorts of features, such as serial communication (serial), a web server (Webserver), logging (Log), config files (Config), and the very important “state manager” (State) which provides an event-based publish / subscribe mechanism for representing the state of sensors and actuators around the house. There is a rig which manages automatic rig loading for JeeMon (Jm), and one which has some utility code I didn’t know where else to put (Ju).

There’s a rig called “app”, which orchestrates startup, general flow of events, and application-wide event “hooks”. This hook mechanism is similar to the Drupal CMS (written in PHP) – because good ideas deserve to prosper.

And lastly, if you supply a rig called “main”, then this will become the first point where your code can take control.

I can’t go into much more detail in this post, but I hope it gives an idea of the basic design of JeeMon + JeeRev.

Tomorrow, I’ll close this series with details about getting started with JeeMon – for those who care and dare!

What’s in the yellow box?

In Software on Nov 23, 2011 at 00:01

So much for the big picture – but what’s this JeeMon?

JeeMon yellow

JeeMon ties stuff together – it can talk to serial and network interfaces using event-driven asynchronous I/O. This means that it can talk to lots of interfaces at the same time (like what the Twisted framework does in Python, but more deeply integrated into the system).

JeeMon collects data – it includes a column-oriented embedded and transactional database. This means that it can store lots of measurement data in a very compact format, using variable int sizes from 1 to 64 bits (usually an order of magnitude smaller than SQL databases, which add extra overhead due to their generality and are less suitable for repetitive time-series data).

JeeMon talks to browsers – it has an efficient embedded co-routine based web-server. This means that it can be used with web browsers, delivering rich user interfaces based on JavaScript, HTML 5, CSS 3, Ajax, and SSE. By pushing the logic to the browser, even a low-end JeeMon server can create a snappy real-time UI.

JeeMon likes to network – with the built-in event-based background network support, it’s easy to connect to (or provide) a range of network services. This lets you do remote debugging and create distributed systems based on JeeMon, as well as extract or feed information to other network-based systems.

JeeMon has a pretty face – on desktop machines running Windows, Mac OSX, or Linux, the “Tk” toolkit is included to quickly create a local user interface which does not require a browser. This means that you can create distinctive front panels showing real-time data or perhaps extensive debugging information.

JeeMon is dynamic – it is based on the 100% open source Tcl scripted programming language, which is used by several Fortune 100 companies (if that matters to you). This means you can work in a mature and actively evolving programming environment, which gets away from the usual edit-compile-run cycle. Making changes in a running installation means you don’t have to always stop / restart the system during development.

JeeMon is general-purpose – the structure of JeeMon is such that it can be used for a wide range of tasks, from quick command-line one-offs to elaborate long-running systems running in the background. This is possible because all functionality is provided as modules (“rigs”) which are loaded and activated on-demand.

Technically, JeeMon consists of the following parts:

  • the Tcl programming language implementation
  • (on desktop versions) the Tk graphical user interface
  • support for serial interfaces, sockets, and threads
  • the Metakit embedded database (by yours truly)
  • co-routine based wibble web server, by Andy Goth
  • upgrade mechanism using Starkits (by yours truly)
  • domain-specific code, collectively called JeeRev

That last item is the only part in this whole equation which is in flux. It creates a context aimed specifically at environmental monitoring and home automation. This is the part where I keep saying: Wait, it’s not ready yet!

Tomorrow, I’ll describe what’s inside this “JeeRev” …

JeeMon? JeeBus? JeeRev?

In Software on Nov 22, 2011 at 00:01

For several years now, I’ve been working on a software project on the side, called JeeMon. Then something called JeeBus was created, and then JeeRev came along.

That’s a lotta names for a pile of unfinished code!

I’d like to explain what it’s all about, and where I want to take this (looking back is not so interesting or useful).

The Big Picture ™

With monitoring and control in the home and <cough> Home Automation </cough>, the goal is quite simple, really: to figure out what’s going on in the house and to make things happen based on events and rules.

In a nutshell: I want energy consumption graphs, and curtains which close automatically when it gets dark.

There are no doubt roughly one million different ways to do this, and ten million implementations out there already. JeeMon adds one more to the mix. Because I’ve looked at them all, and none of the existing ones suit me. Call it “Not Invented At JeeLabs” if you like, but I think I can help design / build a better system (please remind me to define “better” in a future weblog post, some day).

I’ve pondered for some time on how to present the big picture. Threw out all my initial attempts. This says it all:

Jeemon context

Let’s call the yellow box… JeeMon ! Forget about JeeBus. Forget JeeRev (it’s still there, under the hood).

JeeMon is the always-on software “switchboard” which talks to all this hardware. It can run on a dedicated little server box or on the router. The devices can be connected via serial, USB, Ethernet, WiFi, FSK, OOK, whatever.

Functionality & Features

The primary tasks of JeeMon are to …

  1. track the state of sensors around the house
  2. display this real-time state via a built-in web server
  3. interface to remotely operated devices around the house
  4. present one or more web pages to control these devices
  5. act as a framework which is easy to install, configure, use, and extend

Everything is highly modular, JeeMon is based on drivers and other extensions – collectively called “features”.

Some other tasks which are a natural fit for JeeMon:

  • configuration of each sensor, device, and interface involved
  • collecting and saving the full history of all sensor data and control actions
  • presenting a drill-down interface to all that historical data
  • managing simple rules to automatically make things happen
  • access control, e.g. to prevent “junior” from wreaking havoc

That’s about it. It’s that simple, and it needs to be built. JeeMon has been “work in progress” for far too long.

Reflow Timer software

In Software on Nov 4, 2010 at 00:01

Another episode in the reflow controller story…

Here is yesterday’s graph again, but manually annotated this time:

Annotated Reflow

Actually, I went ahead and extended the code to add those axis labels in there. I was concerned that they would overlap and distract from the graph data itself, but after seeing this… it clearly improves readability.

The trick is to get the PID control factors right, and these will be different for each setup. Right now, I just picked a couple of values which seem to be working ok on my particular grill. I’ve extended the JeeNode sketch to allow adjusting these values via a serial USB connection:

    <N> P       P factor (x1000)
    <N> I       I factor (x1000)
    <N> D       D factor (x1000)
    <N> L       I limit (x1000)

The PID calculation is:

    (Pfactor*Pval + Ifactor*Ival - Dfactor*Dval) / 1000

In other words, these factors are specified as a multiple of 0.001.

The result is brought into a range of 0..100. This in turn is used to determine when, how often, and how long to turn on the heater in the grill/oven/skillet.

The reflow profile parameters are also adjustable from the serial link:

    <N> o       ON temperature (°C), default 70
    <N> w       minimum time in WARMUP phase (sec), default 60
    <N> p       temperature at end of PREHEAT phase (°C), default 140
    <N> s       temperature at end of SOAK phase (°C), default 170
    <N> m       maximum REFLOW temperature (°C), default 250
    <N> r       minimum time in REFLOW phase (sec), default 15
    <N> c       temperature at end of COOL phase (°C), default 150
    <N> f       temperature at end of FINAL phase (°C), default 50

Once the FINAL phase ends, the JeeNode will power itself down.

A few more parameters:

    <N> l       lower calibration temperature limit (°C), default 40
    <N> u       upper calibration temperature limit (°C), default 120
    <N> d       stable duration (sec), default 5
    <N> t       stable trigger gap (°C), default 25
    <N> a       number of temperature averages to take, default 250

Some parameters for reporting, which happens once per second:

    <N> i       wireless node ID (sending disabled if 0), default 8
    <N> b       wireless frequency (4=433, 8=868, 9=915), default 8
    <N> g       wireless net group, default 5
    <N> e       enable (1) or disable (0) serial reports, default 0

And finally, the parameters which control the FS20 remote switch:

    <N> H       house code to use for FS20, default 4660
    <N> h       device ID to use for FS20, default 1

All PID factors and other parameters are stored in EEPROM, so they will remain in effect until changed.

To get a summary of all the current settings, type a question mark: “?”.

To reset all parameters to their “factory” defaults, type an exclamation mark: “!”.

The code for the “reflowTimer.pde” sketch is here The current code size is ≈ 14 Kb. I’ll probably be tweaking it a bit further in the coming days.

One thing I’d like to try adding to the current sketch is an easy way to self-calibrate and come up with a workable set of P/I/D factors, so that it can be used with a variety of electrical grills, toasters, skillets, ovens, barbecues, whatever – under the motto: if it can melt solder, we should try it!

The JeeMon script is here and is about 150 lines of code. If you save it as “application.tcl” next to JeeMon, it will automatically be picked up when JeeMon is launched. The code is still work-in-progress at this point: you will have to manually edit the “device” variable to refer to your attached JeeNode/JeeLink running RF12demo – you can also set it to a COM port (Windows) or tty device (Linux/Mac). Likewise, the “nodeID” variable should be set to match the current setting in the Reflow Timer sketch (“i” parameter):

    variable device   usb-A900ad5m    ;# which JeeNode/JeeLink to attach to
    variable nodeID   8               ;# which node ID to listen to

The frequency band + netgroup of the JeeNode/JeeLink are assumed to have been previously set in RF12demo.

Note that the script is an optional GUI front-end – you can launch it anytime, or you can ignore this whole JeeMon thing, since the sketch does not depend on it. It’ll drive the reflow process with or without the GUI.

If you try this out, or have suggestions about how to improve things, please let me know.

Update – I’ve adjusted the info above to match the latest code changes.

Meet the Reflow Timer

In Hardware, Software on Nov 3, 2010 at 00:01

Now we’re cookin’ – here’s the complete reflow configuration I am setting up for use at Jee Labs:

Dsc 2201

Yes, it’s a Project On Foam again!

As before, I’m using a 700 Watt low-end toaster/grill. It can heat about the area of a 10×16 cm pcb and it’s really small and practical for me. I removed the teflon-coated hot plates, and placed a thin aluminum sheet in there, to respond more quickly to heat changes. A small oven or a skillet could probably also be used.

The power is controlled by an FS20 remote switch (available from Conrad or ELV, both in Europe). This is very convenient, since JeeNodes can control this thing through the RFM12B without any further hardware. The big advantage: no need to mess around with 220V AC mains – it’s RF-isolated!

The LCD display makes this thing independent of a PC/Mac. And the battery pack makes it a fully stand-alone solution. The JeeNode (and LCD / radio) will shut off once the temperature drops below 50 °C. This whole setup draws about 30 mA, so with a run time of 10-minutes, four AA batteries will last hundreds of runs, i.e. plenty!

The Thermo Plug and Blink Plug have both been extended in the shop as pre-assembled unit and kit, respectively, including a thermouple which can be used up to 350 °C. I’ve also added a 4-cell battery holder.

Here’s how to operate this thing:

  • set up everything, place the board inside the grill, and close the lid
  • press the GREEN button, the green LED goes on
  • wait for the BEEP, then carefully open the lid
  • wait until the green LED turns off, i.e. the temperature drops under 150 °C
  • done!

This is an example of what happens during a run:

Screen Shot 2010 11 02 at 13.10.19

Tomorrow, I’ll comment on this graph and the JeeMon app that produced it.

JeeMon device discovery

In Software on Oct 12, 2010 at 00:01

Ok, I must admit that JeeMon has been a bit too ambitious in its original inception. It works quite nicely here at Jee Labs, but there are just too many hoops you have to jump through to make it happen on your own.

I’ll first explain why things are the way they are, and how that is supposed to work, and then I’ll present a different setup for the OOK Scope which jettisons all that machinery and lets you use the OOK Scope with nothing but a single source file and JeeMon.

The idea behind the “Serial” and “JeeSketch” rigs (code modules) in JeeMon, is that it should be possible to respond to changes in interfaces dynamically. So there’s a way to scan for USB interfaces periodically:

Serial periodicScan <cmd>

This will compare the list of USB devices it sees with the list it saw last time (once every 5 seconds by default). And then call the specified <cmd> whenever an interface is added or has gone away. Only FTDI interfaces are detected, by the way.

The next step is to decide what to do when a new USB device is attached. I’ve been using a convention for some time now, whereby every sketch starts by sending out its own name, with optional version and configuration details. For example, RF12demo will send out a string like this to the USB connection when it starts up:

[RF12demo.6] A i1 g5 @ 868 MHz

The trick is to wait for such a string when JeeMon detects the device and opens it. This is handled by the “JeeSketch” rig. Once the sketch type is known, JeeMon then tries to locate a “host.tcl” driver for it. It does this by looking in a directory, as configured at start up. I’ve been placing all my drivers in a directory called “sketches”, so my startup includes the following line:

JeeSketch register ./sketches

When RF12demo connects, JeeMon checks whether “./sketches/RF12demo/host.tcl” exists, and runs it.

Similarly, when plugging in a JeeNode running the OOK Scope sketch, it announces itself as:


The script “./sketches/ookScope/host.tcl” then gets started, it creates a GUI window, takes over the communication link to process all incoming (binary) data bytes, and does its pulse histogram thing.

So far so good. It’s a nicely modular mechanism. I can add a new sketch, i.e. a “blah.pde” file for use in a JeeNode, and add a matching “host.tcl” script alongside to deal with the communication in any way it likes. Then, all I need to do is plug the device in and everything starts up. With a bit of care, everything is also shut down and cleaned up again when the device is removed.

Unfortunately, that’s not the end of the story. One of the important devices I want to support is a JeeNode or JeeLink running RF12demo. But how can JeeMon deal with remote nodes? After all, all they do is send packets to the central device. Each of these nodes will be running a sketch, and not all of them are necessarily the same. So we either need some sort of auto-discovery or some central configuration file. For a first implementation, I decided to use a configuration file to try and keep things, eh, simple. Which is why my startup code also contains these two commands:

set appDir [file dir [dict get [info frame 0] file]]
Config setup $appDir/config.txt

That’s a tricky way of making sure that the “config.txt” file will be picked up from the same directory as the source code, i.e. the “application.tcl” file.

I’ll refrain from describing the config.txt file in full detail. Let me just include an example which I’ve been using around here:

Screen Shot 2010 10 11 at 22.52.15

As with any such type of “registry”, you can see lots of little config details, for use in different modules and parts of the code. Even some obsolete stuff, in fact.

Does it work? Oh, sure, it works great and it’s very easy to extend for new modules and usage scenarios. Even node discovery works nicely, both coming on-line and dropping off-line, as seen in the voltmeter demo.

But there’s a problem with what I’ve described so far…

… there’s too much rope – to hang yourself. It’s brittle, it needs lots of documentation to use this stuff (unless you’re willing to dive into the JeeMon Tcl code), and it’s just too much trouble if you want to do something simple with JeeMon, like run the OOK Scope and nothing else. The entry curve is way too steep.

I can’t say I’ve figured out an alternative. There is a lot of ground to cover, and it is fairly hard to implement a system which dynamically adapts to interfaces getting plugged in and nodes coming (wirelessly) online.

But at the end of the day, all that extra bagage is unnecessary for simple cases.

Fortunately, JeeMon is flexible enough to adapt in any way I want. I don’t have to use any of the above machinery. So here’s the OOK Scope logic as a single “application.tcl” file. No scanning, no config, nothing:

Screen Shot 2010 10 11 at 23.05.06

The full code is available here. To run this version of the OOK Scope, download that file, make sure it is called “application.tcl”, and place it next to your JeeMon executable. Then launch JeeMon. Just make sure to have the JeeNode running ookScope.pde plugged in.

If more than one FTDI interface is found, you will be asked to pick one:

Screen Shot 2010 10 11 at 23.22.45

That’s it: ookScope.pde, application.tcl, plus JeeMon – should work on Windows, Mac OS X, and Linux.

IR pulse pickup

In Hardware, Software on Oct 11, 2010 at 00:01

With the sending of IR pulses out of the way, the next step is picking them up. That’s a lot simpler since there are various sensors which do this.

But unfortunately I only have a QSC113 IR photo transistor at hand right now. It doesn’t have any sort of 38 kHz filtering, it just senses IR light. Then again, it still seems to work – I don’t understand why, to be honest:

Dsc 2123

Anyway, this seemed like a good test case to throw at the OOK Scope. The name is a bit confusing actually – it was used to analyze OOK-encoded radio signals a couple of months ago, but it’s really more general than that. What the OOK Scope does, is create a histogram of incoming pulse widths (20 .. 4604 µs with the current code).

As a refresher, here’s some OOK Scope output with a 433 MHz radio attached, after pressing buttons on my KAKU remote control:

Screen Shot 2010 10 10 at 22.06.54

The histogram runs sideways: at the top are the short pulses, at the bottom the long pulses. The longer the bar, the more pulses of that length have been picked up.

Now the fun part is that this same setup can also be used for IR pulses:

Dsc 2122

I simply replaced the OOK radio in port 2 by my IR photo transistor, with the short lead to GND and the long lead to AIO2. Here’s what comes out when using the Apple remote:

Screen Shot 2010 10 10 at 22.47.36

The pulses are a lot sharper, which is not surprising since it’s picking up direct light pulses now, not radiomagnetic waves of a much lower frequency and intensity that has to be detected and converted into an electrical signal by the radio.

Reading these values is a bit more work, because the ookScope.pde does some trickery to increase the range of pulse widths it can report in a single byte:

if (width >= 128) {
    width = (width >> 1) + 64;
    if (width >= 192) {
        width = (width >> 1) + 96;
        if (width >= 224) {
            width = (width >> 1) + 112;
            if (width >= 240) {
                width = (width >> 1) + 120;
                if (width >= 248) {
                    width = (width >> 1) + 124;
                    if (width >= 252) {
                        width = (width >> 1) + 126;
                        if (width > 255)
                            width = 255;

See the OOK Scope post for details on how this works.

Back to the readings. The values reported are two strong bands centered around ≈ 105 and 158, respectively, with another set of pulses at 224. The pulse widths corresponding to these are ≈ 420 µs, 652 µs, and 1536 µs if I got my calculations right. That last one might be the time between retransmissions.

I’ll have to revisit this with a real 38 kHz filtering IR sensor when I get them in – but that sort of looks right to me. Another remote sending out RC5 codes ended up with very similar pulse widths, but with more variation.

Tomorrow, I’ll describe the OOK Scope setup in more detail, since it doesn’t seem to be working for everyone…

Update – I just figured out the pinout of a 5V sensor with IR filtering (TSOP1138) – here’s what comes out:

Screen Shot 2010 10 10 at 23.32.07

Hm, more variation now. Then again, also a lot more sensitive, it seems. And it looks like it should still be possible to discriminate between the two pulse widths at position 135, i.e. around 568 µs.

The pulse trains are now more consistent on the scope:

Dsc 2124

Note that I’m driving the JeeNode I/O pin to nearly 5V, which isn’t really such a good idea…

PS. That’ a DSO Nano scope, in case you’re wondering…

Save data to a logfile with JeeMon

In Software on Oct 1, 2010 at 00:01

I’ve added a little demo application called “SaveToFile” to JeeMon which saves data from an attached JeeNode or JeeLink to file, complete with timestamps.

When started, you’ll need to specify which device you want to log (unless there is only one):

Screen Shot 2010 09 30 at 21.01.18

Click on one of the lines, and you’ll get this dialog on Mac OS X (Windows and Linux look a little different):

Screen Shot 2010 09 30 at 21.01.31

Once you click “Save”, the main window changes to show the last received line:

Screen Shot 2010 09 30 at 21.02.07

Here’s the contents of the log file at this point:

Screen Shot 2010 09 30 at 21.02.55

You can just leave it running to collect more data.

The following code implements all this and shows a bit of Tcl and Tk scripting in action:

Screen Shot 2010 09 30 at 21.22.40

This code is now included in the JeeMon on-line demos, so all you need to do is get JeeMon, launch it, and then click on “Run web-based demo”:

Screen Shot 2010 09 30 at 21.04.34

(make sure you don’t have a file called “application.tcl” next to JeeMon)

A window will open, just select the “SaveToFile” link, and it’ll be downloaded to the “JeeMon-demos” folder and then launched.

To run this code from the downloaded copy, launch JeeMon as:

    ./JeeMon JeeMon-demos/

To view the code, unpack that ZIP file and you’ll get what is shown above. You can adapt it to your needs, of course. It’s all open source software!

Pulling data from an EtherNode

In Software on Aug 17, 2010 at 00:01

Last month’s EtherNode sketch was an example of a simple web server which allows viewing incoming packets received by the RFM12B. Here’s a sample web page again:

Screen Shot 2010 07 13 at 231929

If JeeMon could access and pick up that data without requiring an extra JeeLink or JeeNode, then you could place the EtherNode wherever reception is best while running JeeMon on your desktop machine, or anywhere else.

In response to a request on the forum for just that, I started writing a little demo “application.tcl” for JeeMon to do this sort of web-scraping. Here’s what I came up with (code):

Screen Shot 2010 08 16 at 10.35.49

Sample console output:

Screen Shot 2010 08 16 at 10.42.48

The point here, is that it needs to periodically poll the EtherNode, get a web page from it, and skip the readings it has already seen before. That’s what most of the code in “EtherNodePull” does. Each packet that remains will be sent to the “GotPacket” proc, which just logs it on the console.

But that’s just one half of the required solution…

The bigger challenge is to also make JeeMon decode these packets, as if they came in through a serial USB link. There is quite a bit of logic in sketches/central/host.tcl to do that for a JeeNode or JeeLink running the “central” sketch (which is almost identical to RF12demo).

The reason this is more complicated, is that I want to be able to decode each packet in different ways, depending on the sketch running on the remote (sending) node. My network has more than just room nodes, and will be extended with many more node types in the future.

One workaround would be to collect all nodes of the same type in their own group, i.e. net group 1 for room nodes, net group 2 for the ookRelay, etc. And yes, that would work – but it’s not very convenient, and I’d need separate etherNodes to pick up the packets from each net group. Messy.

The approach I have used so far, is to maintain a config section for JeeMon, with information about the type of each node, organized by frequency band, net group, and node id:

Screen Shot 2010 08 16 at 10.52.23

It’s not automatic, but this way I just need to adjust one list whenever a new wireless node is brought online.

The current code in sketches/central/host.tcl is all about picking up packets, and mapping them thtough this configuration section to know what is what. It does this by setting up a pseudo “connection” whenever packets come in for the first time and includes logic to tear down this connection again when no new packets are received within a certain amount of time.

To use this approach with an EtherNode as data collection node, I need to re-factor the exisiting code and make the core mechanism independent of the Serial implementation. I also need to bring more of the code from central/host.tcl into the JeeMon code, so it can be re-used for EtherNodes.

Re-factoring is my middle name – I’ll update this post when the code changes are complete.

A little GUI for the RGB LED strip

In Software on Jun 16, 2010 at 00:01

After yesterday’s post, this RGB strip control project is essentially done. Keep in mind that I’m doing this only to adjust what comes out as “white”. I’ll leave light levels and color variations up to nature, since all I need to do is look out the window…

But since wireless control is present anyway, I might as well create a little demo using the JeeMon software.

So here goes. The setup I’m using is as follows:

Screen Shot 2010 06 14 at 16.31.37

The JeeNode is configured as node 30 and the JeeLink as node 1 (in the same group). The manual way to send out a color command via serial USB is to type:


Where each of the four values are in the range 0..255 (0 = off, 255 = full on).

Now the GUI version (Mac OS X):

Screen Shot 2010 06 14 at 18.21.51

The code for this is as follows:

Screen Shot 2010 06 14 at 18.16.26

Like everything in JeeMon, it’s cross platform since it uses Tcl/Tk.

Here are the screen shots for Linux (Ubuntu) and Windows (W7, XP, 2K), respectively:

Screen Shot 2010 06 14 at 18.01.42 Screen Shot 2010 06 14 at 18.35.25

Screen Shot 2010 06 14 at 18.14.21 Screen Shot 2010 06 14 at 18.09.26

As you can see, the light gray background color was adjusted for the Mac – on some of the other platforms it needs to be darkened a bit to match the default label background.

Anyway. I’m not a GUI wizard, so this’ll have to do for now…

Reflow profiles

In Software on May 19, 2010 at 00:01

This is part 7 of my reflow controller series.

Reflow soldering is a pretty simple process. Take a PCB, add solder paste, place components, and then let the whole thing go through a controlled reflow termperature profile. As I’ve described before.

In my (limited) experience, these temperature profiles are not nearly as critical as one might expect. Just preheat the thing while staying under the melting point of solder, then ramp up and keep it all well over the melting point for a while. Not too high and not too long, so nothing gets damaged. Then let it cool down. That’s basically it.

I’m going to use the following profile to start with:

  • do nothing for 10 seconds
  • heat up to 140°C
  • stay there for at least 30 seconds
  • heat up to 170°C
  • stay there for another 20 seconds
  • heat up to 250°C
  • stay there for 15 seconds
  • turn off and open up the grill for fast cool down
  • the finished board can be removed when under 150°C or so

In code:

Screen Shot 2010 05 12 at 23.00.15

I’m staying a bit longer at the high temperature because my grill is a bit uneven in its temperature distribution. I want the cooler spots to work properly as well. Hopefully that won’t damage anything.

So how do you go from a thermostat to a reflow controller? Simple: implement the profile. I added a RunProfile proc, which keeps calling itself over and over again, so it behaves like a background process. Its task is to adjust the target variable over time to match the requested reflow profile. When a step has been completed, it will be removed from the front of the profile list:

Screen Shot 2010 05 12 at 23.02.44

RunProfile is called in start, right after calling InitPlot.

Ok, let’s try it:

Screen Shot 2010 05 13 at 00.10.50

Hey, this is starting to look like something!

I changed the colors a bit and am now also plotting the target temperature for reference.

Latest source code available here.

But all is not well. While trying this out, I noticed that the current setup hangs once in a while. No more incoming data, so the plot and the control stops. I suspect that there’s an occasional conflict between sending out OOK commands and handling incoming packets – perhaps a bug in RF12demo. I worked around this by omitting all the redundant OOK commands.

There was also another case where the last temperature target was set, but the heater wouldn’t get turned on, i.e. HeaterControl returning 0.

So… progress, but not finished yet!

Temperature control

In Software on May 16, 2010 at 00:01

This is part 6 of my reflow controller series. Let’s see if we can get the reflow grill to a stable 150°C.

It’s not trivial: this grill has a slow startup time and a very noticeable lag in its response curve. Turning on the grill and turning it off at 150° is not the way to do it, since the stored heat will lead to a huge overshoot.

The proper way to do this is to use a PID control algorithm. PID stands for Proportional Integral Derivative. It’ll be interesting to try that, but first I want to try something simpler (actually, it is still PID, but just P and D).

The idea is to try and predict where the temperature will end up when we turn the heater off. I’ve got two relevant pieces of information for my particular grill, obtained from the last few trials:

  • The grill will heat up at about 2.5°C/sec once it gets up to speed.
  • When turned off at that rate, it appears to overshoot by about 40°.

Let’s try something easy first. Let’s find out what the grill does when turned off at 110°. I’ve added this line to the GotData proc to do so automatically:

if {$value >= 110} { set heat 0 }


Screen Shot 2010 05 12 at 17.29.53

Not bad – still some overshoot, so I’m going to assume an overshoot of 50° from now on. BTW, this is not the same as keeping the oven at a preset temperature, but it’s a start. In fact… I’m going to keep this line as a permanent safety valve:

if {$value >= 265} { set heat 0 }

It’s not essential, since the grill has a mechanical temperature cut-off as well, but that way I can be sure that this code will never try to push its heater beyond 265°C. It would have avoided yesterday’s runaway failure.

To improve on this, let’s assume that the overshoot is proportional to the heat-up rate. So if we turn off the heater while it’s heating up at 1.25°C/sec, it will overshoot by 25° – it seems plausible, since that probably means the heater hasn’t been on that long yet, so there is less “stored excess heat” in the system.

Next thing to do is to track the rate of change and base heater decisions on that. I’ve added a new HeaterControl proc which decides what to do for a given target temperature:

Screen Shot 2010 05 12 at 18.35.46

Note that heater control is simply a matter of setting the heat variable. It controls both the remote switch and the GUI checkbox, courtesy of Tcl’s built-in variable tracing facilities. This does the actual control, in GotData:

set heat [HeaterControl $value $lastv $x $lastx]

And in the start proc, this extra code will get the ball rolling:

variable target 0
after 10000 set [namespace which -var target] 150

IOW, after 10 seconds, JeeMon will attempt to maintain the grill temperature at 150°C. Let’s try it:

Screen Shot 2010 05 12 at 18.38.52

Woohoo, it works! A thermostat!

The “application.tcl” source code is available here. Next step is to add a reflow temperature profile.

Controlling the oven

In Software on May 15, 2010 at 00:01

This is part 5 of my reflow controller series.

Today, I’d like to be able to remotely turn the grill on and off. To avoid having to deal directly with high voltages (220V is scary!), I’m going to use an RF controlled switch – i.e. this FS20 module:

Dsc 1428

It’s perfect here, because it operates @ 868 MHz and can be controlled directly from a JeeLink or JeeNode, and because it has an on/off button right on the unit itself (unlike these). Which is great as emergency stop – we’re going to play with serious levels of electricity, current, and heat after all.

As it so happens, the RF12demo sketch I’ve been using to receive packets from the thermocouple node also supports sending FS20 commands out of the box.

So all that’s needed is to extend the GUI a bit with a control element, and hooking that up to send the proper FS20 command out.

This requires a few extra lines in the initPlot proc:

variable heat
pack [checkbutton .h -text Heater -variable [namespace which -var heat]]
trace add variable heat write [namespace which HeatChange]

And a proc called HeatChange, for which I’ll use a bit of test code for now:

proc HeatChange {args} {
  variable heat
  puts "heat = $heat"

The result is a window with an extra checkbox at the bottom:

Screen Shot 2010 05 12 at 024300

Clicking that button simply generates some test output:

heat = 1
heat = 0
heat = 1
heat = 0

Great. The GUI side is working. Here’s an updated version of HeatChange which actually sends out the proper FS20 commands:

proc HeatChange {} {
  variables heat conn
  if {$heat} {
    $conn send 54,32,1,17f
  } else {
    $conn send 54,32,1,0f

The first 3 values are the house code and address bytes. They can be anything, because FS20 modules are configured by putting them in a special listening mode (press the button until the LED starts blinking). The next RF command sent to them will then be remembered, so that it will respond to that specific code from then on. Code 17 means ON, code 0 is OFF – that’s part of the standard FS20 protocol (see this German info page). The trailing “f” tells RF12demo to send everything out as an FS20 command.

IOW, to respond to these RF signals, put the FS20 unit in that special mode and then send one of the above commands by clicking on the checkbox in the GUI. You should now be able to manually control the remote switch.

Note: make sure you have the latest RF12demo. A nasty OOK bug was fixed a few days ago. If your JeeLink hangs: unplug, reconnect, then upload the latest code.

One more thing I’d like to do is include the heater status in the plot. That requires a few more changes. Here’s the latest “application.tcl” (I’ve collapsed the start and HeatChange code, since they are the same as before):

Screen Shot 2010 05 12 at 03.57.04

Let’s try this new setup, i.e. measuring and controlling 100% by wireless.

What I wanted to do is hook it up to my Ersa I-Con Nano temperature-controlled soldering station (with the soldering tip removed), because that would have been a great demo of how real temperature control works:

Dsc 1418

Unfortunately, that didn’t work – and drove home that there’s a real risk of fire involved in these experiments. Here’s what happened:

Screen Shot 2010 05 12 at 15.48.26

The temperature shot up to 450°C in seconds! – I think there’s a sensor in the very tip of the iron, and it wasn’t touching anything, so this heater went full blast – charring the thermocouple insulation on its way up. I switched the iron off manually, and then everything coooled off.

Second try, this time replicating yesterday’s setup:

Screen Shot 2010 05 12 at 16.02.57

Perfect. A step pulse and the response curve (grill was opened @ 175°C, like yesterday).

Warning: if you try these experiments, make sure you unplug your oven / grill / whatever when you’re done. Starting a fire while you’re tinkering with something else, or out of the house, or asleep is not a good idea…

Tomorrow, I’m going to create a feedback-control loop.

Oven temperature plot

In Software on May 14, 2010 at 00:01

This is part 4 of my reflow controller series.

Data is coming in over the serial USB connection. Quick, let’s visualize it!

There are two ways to do this: with some external tool such as Excel, or with the GUI facilities built into JeeMon.

Let’s do Excel first, because it requires less coding. Change yesterday’s “application.tcl” example to this:

proc start {} {
  set conn [Serial connect usb-A900adav 57600]
  oo::objdefine $conn forward onReceive [namespace which GotData]

  variable fd [open logfile.csv w]
  chan configure $fd -buffering line
  puts $fd "time,temperature"

  vwait forever

proc GotData {msg} {
  variable fd
  if {[Serial cmdParser $msg OK -node id -int1 temp]} {
    puts $fd "[Log now],$temp"

The cmdParser function in Serial helps with decoding the “OK …” lines. It takes type arguments and variable names. In this case the node id header and an integer, to be stored in a variable called temp. The “-int1” notation means: treat the int as having one decimal, i.e. convert 123 to 12.3, etc.

Run JeeMon and it will create a “logfile.csv” file with readings (in “Comma Separated Values” format). You may have to stop JeeMon before you can open the logfile with another application.

Using “Numbers”, a Mac OS X application similar to Excel, this is what I get:

Screen Shot 2010 05 10 at 20.11.19

You can see the sensor at room temperature, heating up as I touched the thermocouple, and then cooling off again gradually.

The other approach is to create the plot with Tk, the GUI toolkit which is built into JeeMon, as I did with the OOK Scope. This is more work, but you get a plot which updates in real time.

So now let’s make a graph. I turned the grill on until it reached about 175°C, then let it overshoot and cool back down to 175°C again, and then I opened the lid. This is the result over a period of some 8 minutes:

Screen Shot 2010 05 10 at 23.16.59

Looks like this little grill will overshoot by some 40°C, and that it can heat up about 2.5°C per second. It’s only 700 Watt, which probably explains it. Should be fine for reflow, though.

This is the code I used, i.e. “application.tcl” (source here):

Screen Shot 2010 05 12 at 01.04.40

All this is standard Tcl/Tk code, as documented here if you want to explore how it works. With some elbow grease, I hope to add such basic plotting facilities to JeeMon as utility code, hiding most of the distracting details.

Tomorrow, I’m going to add remote switching to control the oven.

Setting up JeeMon

In Software on May 13, 2010 at 00:01

This is part 3 of my reflow controller series. I’ve got a remote node sending out temperature readings once a second, and now I want to do something with that data.

First step is simply to get it into JeeMon and display values as they come in.

Warning: JeeMon is not Emacs, Eclipse, or Processing. For several reasons:

  • JeeMon is portable across a far wider range of platforms. The core also works on tiny embedded Linux boards such as this one, for example.
  • I want a system which can be wrapped up, shipped, and used elsewhere without installation hassles – even cross-platform. JeeMon can do that.
  • I prefer to use the same editing environment for everything I do because I work with lots of different environments, so I use “TextMate” on Mac and fall back to “vi” on Linux.
  • JeeMon can be grown into a “big” system, but it doesn’t need to. It can also be used as bridge for numerous other tools.
  • There’s a lot to like about big environments which take care of everything, but I prefer lower-level tools which let me get under the hood and tinker.

Does that make JeeMon primitive? I don’t think so. Infancy: yes. Limited scope: yes (so far). One-man activity: yes (so far). Perfect: nope (nothing ever is). Fit for its intended purpose: you bet!

All good things come in three, so to work with JeeMon, you need three things:

  • The JeeMon runtime, a single executable file.
  • A programmer’s editor. Pick your favorite. Get a good one. It’ll change your life, as developer.
  • Willingness to figure out how to glue things together using the Tcl programming language.

Let’s get started.

1. Set up the tools

There are JeeMon binaries for Windows, Mac OS X, and Linux as ZIP files, 2..3 Mb each:

Download the one you need, unpack, and you should end up with an executable called “JeeMon”. Feel free to rename it to “jeemon” (lowercase) or even “jm”. These files are 100% open source, but they’ve been wrapped up into single-file executables to get going fast. See the JeeMon page for more background info on the technology used inside JeeMon.

As for which programmer’s editor to use, you’ve probably already got a preference. It doesn’t matter what it is – stick with it and learn it well, is all I can say. A good editor lets you find references and definitions, colorize your source code, compare file versions, lookup documentation, create boilerplate templates, interface with a version control system, and much much more.

2. Get organized

This is going to be a moving target. I’m still exploring the best way to manage code and data, so that it is easy to use in the editor, in the Arduino IDE, and with JeeMon.

The Arduino IDE already sort of imposes a structure to use for its libraries and sketches. Fine.

We just need a good place for JeeMon. I suggest creating a new folder called “JeeMon”, right next to your “Arduino” sketches folder. On my Mac, that happens to be the Documents folder. This is where the above JeeMon executable should be placed.

Here’s a mock-up of the folder structure I have, when following the above guidelines:

Screen Shot 2010 05 09 at 17.42.47

It should be fairly similar on Windows and Linux, I expect.

3. Tie everything together

This is where the real work starts. We need to tell JeeMon how to hook up to the JeeLink, and what to do with incoming packets once connected.

Everything in JeeMon is always driven from a file called “application.tcl”. For this first trial, we can just create that file next to the JeeMon executable itself. Create a file called “application.tcl” with the following contents:

proc start {} {
    array set ports [SysDep listSerialPorts]
    parray ports
    vwait forever

In prose: call the listSerialPorts function in the SysDep module, and store the results in an array called ports. Then print that array. Then just stop and wait (but don’t exit, because then the output would be gone too).

In JeeMon, modules are called “rigs” btw – SysDep is a built-in rig. The above “application.tcl” file is also a rig. Once initialized, the system executes “application start” as its very last step. Which is how the “start” procedure in the above “application.tcl” file gets control. There’s no magic and very little syntax. Just some conventions.

Launch JeeMon. Sample output here, with three JeeNodes / JeeLinks hooked up:

ports(usb-A8007UsI) = /dev/tty.usbserial-A8007UsI
ports(usb-A900ad5m) = /dev/tty.usbserial-A900ad5m
ports(usb-A900adav) = /dev/tty.usbserial-A900adav

Your output will be different. The info you need to extract from the output is the connection name of your JeeLink. In my case it is “usb-A900adav”, so that’s what I’ll use in the following examples.

Stop JeeMon.

Replace the contents of “application.tcl” with the following code, but use the name of your interface in that second line, of course:

proc start {} {
    Serial connect usb-A900adav 57600
    vwait forever

In prose: call connect in the Serial rig, then wait forever. Communication takes place in the background.

When you start JeeMon again, you should see some output similar to this:

22:04:30.328        . (adav) [RF12demo] _ i31 g6 @ 868 MHz 
22:04:30.342        . (adav) DF I 42 4
22:04:30.343        . (adav) Available commands:

That’s output from the JeeLink. When the thermocouple node is turned on, you should see its packets being reported. Sample output:

22:04:34.348        . (adav) OK 33 209 0
22:04:34.350        . (adav)  -> ack
22:04:35.346        . (adav) OK 33 212 0
22:04:35.348        . (adav)  -> ack
22:04:36.344        . (adav) OK 33 206 0
22:04:36.346        . (adav)  -> ack
22:04:37.341        . (adav) OK 33 209 0
22:04:37.343        . (adav)  -> ack

Not much to write home about. But now you’ve got all the pieces in place to start doing more interesting stuff. GUI, networking, webserver, database, it’s all there in JeeMon, waiting to be activated and combined as needed.

There will be some minor differences between Windows, Mac OS X, and Linux, but not much really. If you’re following along and things don’t work as expected, please let me know. I’ll be happy to adjust these notes to cover as many possible details as needed to get going.

Let’s get back to the reflow side of things. We need to figure out how our thermocouple and our oven behave, and after that we need to find a way to control the oven temperature. No worries – one step at a time.

Tomorrow, I’ll set up a temperature graph. Two, in fact.

Note – as of mid 2011, this info is no longer valid. JeeMon has evolved to version 1.5.

A mini scope

In Hardware, Software on Apr 22, 2010 at 00:01

Triggered by a discussion on the forums, here’s a little demo of how to build your own “Poor Man’s Scope”:

Screen Shot 2010 04 21 at 20.36.10

I connected an LDR between AIO1 and ground, and played around with light levels during this screen shot. The last samples were all over the map because I inadvertently touched the LDR leads.

All you need to reproduce this experiment is: an LDR, a JeeNode, and the JeeMon runtime. Oh, and the Arduino IDE to compile and upload a sketch to the JeeNode.

The sketch is adapted from the arduino-arduinoscope.pde demo. This is not nearly as fancy (or even flashy) as that Processing-based version, but it’s very easy to set up and get started with, IMO.

Anyway, here is the sketch running on the JeeNode:

Screen Shot 2010 04 21 at 20.44.32

It does a lot more than I’ll be using here, since I’m only picking up and displaying the first analog value, i.e. AIO1 on the JeeNode.

To get started with JeeMon, you could go read this page to get some background, but all you need really is the executable, so you can just download the appropriate one: for Windows, Mac, or Linux.


  • unpack the ZIP file, so you end up with a “JeeMon” executable
  • create a directory called “JeeMon-rigs”
  • in that directory, create a file called “application.tcl” (contents shown below)
  • adjust the serial port of the JeeNode to your setup (look at the top of application.tcl)
  • launch JeeMon, and you should get the real-time scope display shown above

Here is the application.tcl file, written in Tcl and using the built-in Tk GUI subsystem for visual display:

Screen Shot 2010 04 21 at 20.53.36

I’ve sprinkled some comments in the code. There are a fair number of pesky little details (as in all programming languages), but even without knowledge of Tcl/Tk you should be able to more or less see what’s going on.

There’s a lot one could improve or add to this thing. There always is. This is just a starting point.

But there you go: a Poor Man’s Scope in less than 100 lines of JeeNode sketch + JeeMon rig code. Fun!

Turning the protocol around

In Software on Apr 8, 2010 at 00:01

Yesterday‘s post made it possible to send a message to a remote LCD screen. The remote node is set to constantly receive packets, and once a valid packet comes in, it displays the text.

There is no acknowledgement, so the sender does not know whether the remote node actually did receive the message, nor even whether it is turned on.

This can be solved by “turning the protocol around”: instead of sending packets to a remote receiver, the remote node periodically polls a central node to see if there is new data. The communication becomes bi-directional.

Let’s first change the code in the remote node to do this periodic polling (code here):

Screen Shot 2010 04 07 at 19.26.48

I’m using the “easy transmission” calls in the RF12 driver. They are used in a somewhat unusual way: by sending out empty packets, and then waiting for a valid acknowledgement packet to come in. This is done by checking the return value of rf12_easyPoll() – it will be 1 when an ack has been received.

But now there’s a problem. The JeeLink will immediately send an ACK back when it gets the data:

Screen Shot 2010 04 07 at 19.28.25

That’s not what we want, because the acknowledgement will contain no message text.

Instead, the RF12demo needs to be configured to use “collect” mode instead of “respond” mode. That will stop it from sending out the ACK:


Now something else happens. Lots of packets will start coming in, in quick succession, because the remote node’s easy transmission setup is not getting any ACKs. So it keeps retrying:

Screen Shot 2010 04 07 at 19.31.40

We’ll need to change the JeeMon “application.tcl” script to use ACKs instead of direct packets (code here):

Screen Shot 2010 04 07 at 19.52.21

This change is fairly subtle. The SendToLCD code hasn’t changed. But it’s used in a completely different way now: instead of just sending out one packet, we set up a connection and keep it going – and therefore listening…

The trick is now to check what’s coming in, and when an empty packet from node 9 is coming in (“OK 41”, i.e. 32+9), then we send an ACK (128+9) with the message text. And since we’re done, we also quit the application.

There’s one small GOTCHA… it doesn’t work!

The problem is a bug in the RF12demo code, which prevents it from sending out packets that look like an ACK. I’ve fixed it now, so you’ll need to update your JeeLink or JeeNode – whichever you use as central node.

Once you’ve done that, it should work: power up both nodes and then launch JeeMon in the same way as before. If all is well, a message will again appear on the LCD screen:

Dsc 1327

Note that I’ve extended the remoteLCD sketch to support 2 lines of output: everything after a newline ends up on the second line of the display.

Sooo – similar result, but using a completely different mechanism.

This approach has as benefit that the remote node will stubbornly keep trying to obtain a message to display, even when packets get lost. But in its current form, that’s also a drawback: when JeeMon is not running, the LCD node will keep on retrying forever. With the easy transmission mechanism, it normally retries 8 times, once a second, but since we restart the packet every 5 seconds, it ends up constantly retrying, each second.

There are some other advantages with this remotely-initiated protocol. One of them is less important in this particular case: since the remote node is in control, it doesn’t have to keep the receiver on all the time, and can go into low-power sleep mode any time to save on battery consumption.

The second advantage is that the remote node doesn’t need to know where to get its data from. The easy transmission mechanism uses broadcasts, so whichever node decides to respond to it, gets its message displayed. As long as exactly one node always does, to prevent constant retries.

A third advantage is that the central node can detect when a remote node is up and when not, and act accordingly. This is a topic for another weblog post…

P.S. The Dimmer, Gravity, and Lux plugs have been added to the shop.

Remote LCD from JeeMon

In Software on Apr 7, 2010 at 00:01

Yesterday‘s post got a message to an LCD display, using a JeeLink with RF12demo to send the packet over the air. It works, but the packet had to be manually encoded as decimal bytes. Not very convenient.

Today, I’ll use JeeMon to streamline this somewhat. There will be several iterations of this example, so that it can also be used as an introduction to JeeMon and the Tcl programming language it uses.

To that end: here is a list of Tcl links which may be useful if you want to learn more about Tcl. Having said that – the code below should be fairly clear, even if you don’t want to dive in further.

This first example is mostly to get started with JeeMon.

Read the Getting Started section to obtain a suitable version of JeeMon (Windows, Mac, several Linux’es).

Got that? Ok, make sure it launches and no errors appear. Most likely result of doing so should be a message with “Can’t start (yet), …“. Good. It works.

Let’s set up JeeMon to send a message to the remote LCD.

  • Create a directory called “JeeMon-lcd1”.

  • Inside, create a file “application.tcl” with a text editor, and place the following Tcl program code in that text file (you can download it here):

Screen Shot 2010 04 06 at 10.14.32

  • Change the “usb-A600dVNz” in the above program to match the USB device you’re using to send the message with (can be either a JeeLink or a JeeNode). You can use the Arduino IDE to find out the proper name (probably COMn on Windows). Do not use the port of the remote LCD node, since that JeeNode is listening to incoming packets, not its serial port.

  • Launch JeeMon from the command line, by typing this command: “JeeMon lcd1” – JeeMon will now load the code contained in the JeeMon-lcd1 directory.

If all went well, you should see two things happen:

  • Some debug output will be printed:

    Sending: 72,101,108,108,111,44,32,87,111,114,108,100,33,9s
  • the text “Hello, World!” appears on the LCD!

Dsc 1307

To be continued…

Update – only moments ago, a bug in the RF12 driver was fixed. Make sure to get the latest one and re-build / upload the remoteLCD.pde sketch again if needed.

JeeMon on Bifferboard

In Hardware, Software on Mar 16, 2010 at 00:01

The Bifferboard is an interesting computer which was mentioned on the forum a while back:


It has an ethernet jack, and all it needs is 5V power and a USB stick with Linux on it. You can get an impression of its diminutive size with the JeeLink next to it.

So let’s try it out!

The Bifferboard will work with several different Linux distributions. I decided to use Debian because it’s stable, easy to manage, and has lots of packages ready to install. And because I’m very familiar with it.

Setting up Linux is fairly easy, using these instructions. The hardest part has been done and is fully automated with two scripts: formatting and creating a USB stick to contain a complete Debian 5 (lenny) system. This needs to be done from Linux on your desktop machine, I used Ubuntu.

The only tricky part is that a matching kernel needs to be flashed onto the Bifferboard itself. My older unit has 1 Mb flash, newer units have 8 Mb flash. There are two ways to flash the unit: via a special serial console cable and via a direct ethernet cross-over connection. I tried the ethernet thing, but couldn’t get it to work, and since I had already hooked up a serial line for debugging, I ended up using that method.

The serial port is available as 3.3V I/O signals, but it requires soldering an extra header to the Bifferboard:


(only 3 lines are needed: RX / TX / GND, but I’m swimming in 6-pin connectors around here…)

Those pins are then tied to a USB-BUB, which in turn is hooked up to my Linux setup. The connection is 115200 baud, and shows the kernel boot log on startup.

Once the reflashing is done and the USB stick has been inserted, you end up with a standard Debian setup, which can be accessed over the network via SSH:

Screen shot 2010-03-15 at 12.19.06.png

If there is interest, I can set up a wiki page with more details about all this.

Now the fun part: the Bifferboard is x86 compatible, so it will run the standard JeeMon build for 32-bit Linux!

Screen shot 2010-03-15 at 12.16.03.png

There’s about 25 Mb of usable RAM (no swap space), which should be plenty to run just about anything in JeeMon with some other processes alongside it.

The one thing to keep in mind, is that this is a low-power board. It’s equivalent to a 486SX, i.e no hardware FPU, and it looks like it runs about 100 times slower than my (modern) Mac. But hey, you do get something in return: an very small unit and a power consumption of only 1.5 .. 2.5 watt!

FritzBox call log

In Software on Mar 12, 2010 at 00:01

I’ve been using the FRITZ!Box 7170 as ADSL modem and telephone interface for some time now:

Screen shot 2010-03-08 at 12.50.41.png

It has a nifty feature: when you enable the built-in call monitor by typing in “#965“, you can connect to TCP/IP port 1012 to get real-time messages about all phone calls.

This is easy to hook up to JeeMon. I added a new “FritzBox.tcl” file with the following code:

Screen shot 2010-03-08 at 13.15.39.png

(Some experimental code was omitted from the above listing)

To enable this, the following line was added to application.tcl:

Screen shot 2010-03-08 at 13.14.42.png

Here’s an example of an incoming call, as shown in the JeeMon log:

Screen shot 2010-03-09 at 13.51.12.png

And here’s an outgoing call:

Screen shot 2010-03-08 at 12.55.09.png

Neat! I’ll hook this up as events once the JeeMon event system is ready.

These experiments are very useful to see how the current design is working out. The above code depends on the “MessageStream” class in the “Serial” rig. It doesn’t really add much value in this case, but some class hierarchies seem to be emerging. This socket connection is an example of a similar-but-not-quite-the same case as serial connections, requiring its own class. A socket connection is very similar to a serial connection once initialized – but this doesn’t show as clearly in the code as I would like and there’s already a bit of code duplication.

Will probably need some re-shuffling later, to better match concepts / design and implementation.

Outbound X10

In Software on Mar 11, 2010 at 00:01

I’ve added support for controlling X10 devices via the CTX15 module described in the past two days:

Screen shot 2010-03-09 at 20.31.20.png

First thing was to extend the ookRemote GUI, by adding these definitions to ookRemote.tcl:

Screen shot 2010-03-09 at 20.34.10.png

With this change, the configuration section in “config.txt” supports CTX15 buttons:

Screen shot 2010-03-09 at 20.34.56.png

These new buttons will send X10 commands to the CTX15 module. But this needs a bit of wrapping to turn them into frames, including the proper checksums. So I added two methods to “sketches/ctx15try/host.tcl”:

Screen shot 2010-03-09 at 20.37.16.png

The “send” method overrides the existing one and sets up a frame before calling the original method with it.

That’s it. I can now make the Marmitek AM12 switch “clunk” with my mouse – audible feedback! :)

JeeMon interface for X10

In Software on Mar 10, 2010 at 00:01

Now that the CTX15 has been connected via a JeeNode, it’s fairly straightforward to tie it into JeeMon.

First I created a new folder called “sketches/ctx15try/” with a file “host.tcl” in it, using this code as quick test:

Screen shot 2010-03-08 at 14.37.36.png

The reason there is nothing more, is that I changed the ctx15try.pde sketch a little to return lines starting with “CTX15” – since these are the easiest to tie into JeeMon. The change affects these lines in yesterday’s demo:

Screen shot 2010-03-08 at 14.33.08.png

I’ve also wrapped the received frame in {}’s, because that simplifies processing in JeeMon (Tcl interprets $’s and []’s unless escaped).

To activate everything, I added an extra line to the config.txt configuration file, so that the output from the JeeNode USB (serial# A900adwo) automatically gets picked up as sketch:

Screen shot 2010-03-08 at 15.01.22.png

Here is an example of the above code running:

Screen shot 2010-03-08 at 14.36.50.png

Good. CTX15 events are coming in. They are easily recognizable as (unit#, command) pairs.

But as you can see, these pairs don’t always arrive in the same frame. So we have to add a bit of logic to get things right. Here’s an updated version of host.tcl:

Screen shot 2010-03-08 at 14.51.50.png

The solution used above is as follows: if it looks like a unit code (A..P + 1..16), then save that as last one seen. If it’s anything else, use the last saved location. I’m not sure this covers all possible scenarios, but for this very simple test it seems to work. Here’s a screen shot of the real-time status window:

Screen shot 2010-03-08 at 14.50.48.png

As you can see, the CTX15 event was turned into an “AOFF” command for the “ctx_A01” device, which was created on-the-fly by this code.

Hmm, looking further, I think the “A” prefix needs to be stripped from “AOFF”. Oh well, it’s all scripted, I’ll adjust that when I get to using this stuff for real.

So that’s it. A few dozen lines of C code in the JeeNode to act as pass-through and do the polling, and a few dozen lines of Tcl code to tell JeeMon how to interpret the incoming data. Now we get notifications whenever any X10 activity is detected on the mains power line.


JeeMon goes embedded

In Hardware, Software on Mar 5, 2010 at 00:01

Testing out a new setup … I started by launching the JeeMon runtime executable as follows:

Screen shot 2010-03-02 at 15.41.09.png

Then I went back to my Mac and opened a browser window:

Screen shot 2010-03-02 at 15.56.12.png

It works! You’re looking at a little web server running on a tiny Technologic TS-7500 board:


ARM based, 64 Mb RAM, µSD card socket on board, running Debian (lenny). It’s not very high powered, but in return it only uses 400 mA @ 5V, i.e. it can run off USB power.

I haven’t yet figured out how to get the FTDI USB serial driver on there, but once that’s solved this little unit will be able to act as host for USB-connected JeeNodes or JeeLinks.

The RAM and storage is more than sufficient to run even a very elaborate JeeMon setup, I expect.

This might be an interesting low-end always-on home sensor & automation system!

Pushing data as web client

In Software on Mar 3, 2010 at 00:01

Another way to get data to another place is via push HTTP requests, i.e. acting as a client for another web server somewhere on internet.

The Pachube site is one place to send measurement results. I’ve set up a test feed to use with JeeMon, with two data streams. One of the nice features is that you can easily produce and embed graphs from that system:

Screen shot 2010-02-26 at 02.45.13.png

Here’s how I implemented it in JeeMon. First, I created a configuration section:

Screen shot 2010-02-25 at 22.37.12.png

(I’ve omitted most of my private API key, but anyone can sign up and get their own …)

Next, the code, in the form of a “pachube.tcl” file / module / rig:

Screen shot 2010-03-02 at 03.11.12.png

Note how the configuration file in fact contains “settings” which are simply evaluated as a script: the two lines starting with “Fetch …” are handled by the Fetch proc in pachube.tcl, using the JeeMon notification mechanism to get called back whenever either of the “meter1” or “meter2” readings changes. The Update proc then updates the “latest” variable accordingly. Lastly, SendToWebSite periodically does an HTTP PUT request in the background, supplying all the info needed to successfully submit to the Pachube website.

Btw, this simple example is flawed, in that it does not calculate averages – it just sends the last reading. But things like averaging require a sense of history, and persistence. Haven’t added that to JeeMon yet…

The missing link, as usual, is a line in the application file to start the ball rolling:

Screen shot 2010-03-02 at 03.03.46.png

There’s a lot of flexibility at the protocol and network levels. Such as creating an XML request via templates, if the remote server needs XML. This isn’t limited to HTTP requests, or to using port 80 – send emails, FTP files, etc.

There are quite a few details in the above code which I won’t go into. Again, I’m doing this mostly to show how little code it takes to initiate periodic HTTP client requests to an outside website.

Apart from sites such as Pachube, this could also be used from a tiny embedded Linux running JeeMon, to submit incoming data to a different setup elsewhere on the LAN, for example. IOW, JeeMon can be a front-end for other systems. It’s not about lock-in. It’s a switchboard. It can glue systems together. It bridges gaps.

JeeMon as web server

In Software on Mar 2, 2010 at 00:01

JeeMon comes with a nifty built-in HTTP/1.1 web server (using coroutines to handle requests in parallel).

So let’s create another real-time status display, but as web application this time:

Screen shot 2010-02-25 at 17.56.25.png

As before, I’m leaving out the lipstick – just showing the core functionality.

Here’s what I did to generate this page. First of all, I’m re-using some functionality which is already in the GUI version, to dynamically construct a matrix with the proper columns and rows, so the “statusWindow” code has to be running for the above to work. For real use, that code should probably be reorganized into its own rig.

The main task was to create a HTML template in a file called “statusPage.tmpl”:

Screen shot 2010-02-26 at 00.41.23.png

This uses a simple templating mechanism based on Tcl’s “subst” command. The meta tag sets up the page to self-refresh every 5 seconds. The last line tells the Wibble web server to deliver the page as HTML.

To launch the server, this line was added to the main application file (port 8080, current dir is doc root):

Screen shot 2010-02-25 at 18.00.15.png

That’s it – there’s very little to activating a web server in JeeMon, as you can see. Which is important, because on tiny embedded Linux systems, an HTTP server will probably be the only option to present information.

Creating a full-blown site with CSS, JavaScript, and Ajax is a matter of adding more files – the usual stuff…

Did I mention that it’s all 100% open source, so you can browse / extend / change all of this? – I did? Oh, ok ;)

Node discovery

In Software on Mar 1, 2010 at 00:01

This post isn’t about GUIs, but about the dynamic behavior behind them.

And about wireless. Let’s take the voltmeter setup I described recently, and make it send its readings over the air – it’s a JeeNode after all, so it has wireless connectivity built in.

I extended the original sketch to send readings out roughly once per second:

Screen shot 2010-02-25 at 15.57.03.png

Extending the host side to also work from received packets is trivial:

Screen shot 2010-02-25 at 16.49.26.png

The problem now, is that this node will start sending out packets, but JeeMon has no idea what to do with them. One day, automatic node type registration would be great to add, but for now I’ll just use the configuration file to associate nodes with sketches. Here’s an extract of my current “config.txt” file:

Screen shot 2010-02-25 at 16.00.30.png

For this example, I added the “3/ { type analogPlug }” line to the “868:5/” entry (i.e. 868 MHz, netgroup 5). And sure enough, JeeMon will now launch the GUI for this node, displaying results coming in over the air in real-time:

Screen shot 2010-02-25 at 16.03.00.png

Here’s the relevant part of the log output:

Screen shot 2010-02-25 at 16.04.34.png

The fun part is that a timeout mechanism has also been implemented. If no packets come in from the analog node for 5 seconds, the connection is torn down and the window is closed again. The timeout value is sketch-specific and is included in the above configuration info.

In other words: node is ON => window appears, node is OFF => window goes away. Look ma, no hands!

Note: it’s not good UI practice to open and close windows like this. In a more refined setup, these windows would be panels in a larger window, or they could just be grayed-out to signal their disconnected state, for example.

The analog node is surprisingly convenient this way. I can use it anywhere, turn it on, look on the screen, and turn it off when I’m done. It wouldn’t be hard to add another wireless node to show the results on an LCD screen, btw. If this is extended with a way to assign node IDs ad-hoc, and a way for a sketch to identify itself over the air, then a whole family of wireless “instruments” could be created – using nothing but a JeeNode and some plugs.

This illustrates a guiding principle in JeeMon: don’t ask, just do it! – JeeMon tries to be as dynamic as possible, adapting to changes in status and configuration by adjusting everything it does accordingly. In the case of nodes, new ones are discovered and stale ones are weeded out, in real-time. With objects getting created and destroyed automatically. What these objects do, as side-effect, is up to each one of them,

Right now, this dynamism in JeeMon is still fairly messy. There’s too much state in too many places which needs to be managed carefully. But with a bit of luck, this can eventually be simplified further to create a very consistent and predictable behavior.

A real-time status GUI

In Software on Feb 28, 2010 at 00:01

One more post about setting up a GUI with JeeMon…

I wanted to have a basic real-time display of all the readings coming in, sorted by source. Like this:

Screen shot 2010-02-25 at 15.48.08.png

(the size of this screen shot was reduced to fit this post)

Everything in this window is dynamic: rows get added when new sources start reporting their data, and columns get added when new parameter types are reported. I’m not storing any information persistently yet, which is why this status window looks as follows right after starting up JeeMon:

Screen shot 2010-02-23 at 21.31.15.png

Not very exciting, but all you have to do is wait …

Let’s see what it took to create this GUI. First, I added a generic “Notifier” rig to JeeMon, and a “Reading” method for all sketches to use. The rooms sketch, for example, now reports its incoming results as follows:

Screen shot 2010-02-23 at 22.06.04.png

Similar changes were made to the ookRelay and RF12demo “host.tcl” sources.

The rest of the GUI code is in “statusWindow.tcl”:

Screen shot 2010-02-24 at 01.16.42.png

The tabular display is based on a very nice Tcl/Tk open source package called Tablelist by Csaba Nemethi, which has been added to the JeeMon core library.

The coupling with the rest of the system is accomplished by the “Notifier attach readings * …” call at the end of the setup code. This registers a callback which will be invoked when anything in the “readings” notification group changes. Notifiers are going to be used a lot in JeeMon because they provide such a manageable way to glue independent parts of an application together.

And lastly, adding the following line to the main application starts the ball rolling:

Screen shot 2010-02-23 at 21.42.16.png

There’s quite a bit going on here which I won’t go into. I just wanted to illustrate how little code it takes to create a basic, fully dynamic, cross-platform, self-updating status display window. There – how’s that for adjectives!

Voltmeter with GUI

In Software on Feb 27, 2010 at 00:01

Let’s make a 5-digit voltmeter with a JeeNode, an Analog plug, and JeeMon. But instead of a readout on an LCD screen, I’m going to show the results on the desktop. To start with that last part, here’s my little display on a Mac:

Screen shot 2010-02-23 at 20.23.25.png

You’re looking at my feeble attempt to create a good-looking / souped-up GUI, btw :)

Anyway. Here’s the sketch I’m running on the JeeNode, with an Analog plug inserted into port 3:

Screen shot 2010-02-23 at 11.01.16.png

Trivial stuff. It sends out lines with “VOLT <value>” readings twice per second.

The important part is that this sketch starts up with “[analogPlug]” as greeting. To hook up to this sketch, all we need to do is create a directory called “analogPlug”, with a file called “host.tcl” in it.

This new “analogPlug” directory is a sub-directory of “JeeMon-sketches”. This directory, in turn, has been registered as being used for sketches – with this command in my main application code:

Screen shot 2010-02-23 at 12.21.51.png

Back to the “JeeMon-sketches/analogPlug/host.tcl” file. Here’s its contents:

Screen shot 2010-02-23 at 11.40.34.png

That’s the whole kaboodle.

Now, whenever I plug in a JeeNode running the above “analogPlug.pde” sketch, a window pops up showing readings in real-time. When I unplug the JeeNode, the window is closed again. It’s all automatic, as long as JeeMon is kept running.

This mechanism is not limited to JeeNodes or Analog plugs, of course.

OOK remote control

In Software on Feb 26, 2010 at 00:01

The JeeMon story continues. Here’s a fun little panel for controlling some switches around the house:

Screen shot 2010-02-22 at 21.42.14.png Screen shot 2010-02-22 at 23.31.35.png

Screen shot 2010-02-23 at 20.19.28.png Screen shot 2010-02-22 at 21.51.23.png

These screen shots show Windows (7 and 2K), Mac OS X, and Linux Ubuntu, respectively – long live diversity!

This GUI was created with these configuration settings:

Screen shot 2010-02-26 at 02.09.33.png

Those settings were loaded by adding this line to the main application code:

Screen shot 2010-02-26 at 02.09.16.png

Which in turn loaded the following “ookRemote.tcl” source file I wrote, as module / rig:

Screen shot 2010-02-26 at 02.08.48.png

That’s the entire app. Adding lipstick (i.e. a fancier visual design) is left as exercise for the reader.

Note how the design is split in a configuration section and a source code file – I’m probably going to use this approach often in JeeMon. The idea is that people who don’t care about the code behind all this stuff can just adjust the configuration file (using a visual interface, hopefully, one day). While those who like to tinker can view the full source code, and explore and extend it at will, and more importantly: ad infinitum.

It’s called open source for a reason!

PS. There’s another major reason for the split between configuration settings and source code files: turnkey deployment. You can put all the source code files in a ZIP archive called “”, so that all you need for a new installation with JeeMon, is: 1) your archive, 2) the proper JeeMon runtime, and 3) a config file to match the target setup. The config file is optional, btw – more on that another time…

Declarative configuration

In Software on Feb 22, 2010 at 00:01

Things are starting to come together quite nicely with JeeMon – all in less than 1000 lines of Tcl code so far. That’s not to say that it’s perfect yet – I’m still massively rearranging some of these code structures – but there is already some functionality to play with, which helps expose the strengths and weaknesses so far.

The basic approach is declarative in nature: specifying what should happen, but not necessarily in the same place as when or how all the work should be done.

Here’s my first tentative configuration file:

Screen shot 2010-02-21 at 14.31.44.png

For each serial interface, an action is specified when that interface comes on-line. For example I can now plug in my A8007Up6 device (a USB-BUB with a JeeNode) and JeeMon will open a terminal window showing the output stream. Unplugging closes the window again (and automatically cleans everything up).

Right now, I’m editing this config file by hand, but at some point that could be augmented with a set of GUI or web-based configuration panels (even remotely).

There are a lot of challenges ahead. One of them is decoupling things so that code and data end up organized in the most convenient, logical, and flexible way. I’m trying very hard to avoid any calls to the “Config” module from generic JeeMon code. This allows you to set up the configuration file structure and hierarchy in any way you like.

My application file currently looks as follows:

Screen shot 2010-02-21 at 15.42.58.png

As you can see, I’m setting up a periodic port scanner and supplying a “SerialEvent” callback to decide what to do when serial ports get added or removed. In those decisions, most of the logic is driven from the “config.txt” configuration file. I’m about to implement a similar callback for radio events, i.e. RF12 nodes coming online and dropping off again.

In a way, the above two files are the entire application.

The rest is utility code to make the above convenient and concise. That rest might well be 99% of the code, but it’s all structured as loosely-coupled “rigs” (modules), which can be used / overridden / ignored at will.

The way I see it, there can be three sources for such utility code: your own code, plug-ins from others, and a couple of rigs included in the JeeMon core itself. The above two files require just the core, using the built-in rigs (Config, Gui, JeeSketch, Log, Serial, and SysDep).

Serial USB devices

In Software on Feb 21, 2010 at 00:01

For JeeMon, I’d like to be able to auto-detect USB device insertions and to identify FTDI serial numbers (with names like “FTE54GKC” and “A9007CNE”).

Cross-platform… i.e. on Windows, Mac, and Linux!

It looks like it can be done.

On Windows, we can browse the good ol’ registry, as follows:

Screen shot 2010-02-20 at 15.24.16.png

Sample output:

Screen shot 2010-02-20 at 15.24.41.png

This was tested on Win2000 and Windows 7, in the hope that everything in between works the same.

On Mac OS X, it’s easiest (for a change):

Screen shot 2010-02-20 at 15.26.02.png

Sample output:

Screen shot 2010-02-20 at 15.26.37.png

On Linux, it’s a bit messy because there are so many different distributions:

Screen shot 2010-02-20 at 15.27.43.png

Sample output:

Screen shot 2010-02-20 at 15.28.01.png

Tested on Debian 5 (Lenny), Ubuntu 9.10 (Karmic), and Gentoo.

The above code can be found in the subversion repository, i.e. here.

I’m testing this all at once on a single machine btw, courtesy of VMware Fusion. And using JeeMon’s built-in self-update mechanism to quickly get new versions across while debugging and tweaking things.

By calling “SysDep listSerialPorts” periodically, we can automatically detect a change and see which plug was inserted (FTDI only for now). Without depending on any external libs or executables.


RF transport independence

In Software on Feb 20, 2010 at 00:01

With the basic serial interface access and dispatch in place in JeeMon, it’s time to move on to JeeNode / JeeLink packet processing.

What I want is to be able to forget all about how readings got to JeeMon. It shouldn’t make a difference whether I’m using a directly connected Room Node and grabbing the readings off its serial USB connection, or whether they came in through the air via the RF12 wireless driver – or by any other means. Take a snapshot with your cell phone, send the picture to EverNote, have it OCR’d, and then grab the readings off the web … whatever!

The way I’ve tied this into JeeMon, is to let the interface to RF12DEMO act as de-multiplexer. This is purely a decision at the RF12DEMO listener level. Each incoming packet is examined to determine which node it came from. Then we need a way to map from nodes to listener class – i.e. find out what sketch is running on the remote node. This is hard-coded for now:

Screen shot 2010-02-18 at 22.57.41.png

What this does is actually a bit more elaborate: a RF12DEMO listener will set up and manage a bunch of extra “RF12_PacketStream” listeners, one for each node. When packets come in, they will simply be dispatched to the corresponding stream. Each packet stream can process its packets in a different way.

The fun part is that these packet streams can use the same listener classes as with direct serial interfaces. All they need is an extra “decode_RF12” method:

Screen shot 2010-02-18 at 23.02.04.png

The task of decode_RF12 is to re-cast the incoming packet as messages of the same structure as what would come in over a serial connection.

Here’s the “rooms” listener as example:

Screen shot 2010-02-18 at 23.04.34.png

This one class encapsulates all the protocol details of room nodes, both serial and via RF12. When a 4-byte data packet comes in via RF12 (as $a..$d), the bits are decoded and an “onMessage” call is generated with a “ROOM” message id and the 5 decoded readings.

Here is a log of this code in action, one message per line:

Screen shot 2010-02-18 at 22.37.15.png

The way to read these messages is as key-value pairs, i.e. id = OK, type = RF12, name = usb-A900ad5m, etc.

The first two lines show an incoming OK message from node 21 (53=21+32), which is then turned into a ROOM message, tagged as coming from the “rf12-868.5.23” packet listener.

The next 3 lines are more involved: first an EM10 message came in over USB, then an OK message came in which got dispatched again, as the same EM10 message. That’s because I’m running JeeMon with a direct connection to the ookRelay board, even though it transmits all its information over RF12. So everything from the ookRelay comes in twice (great for debugging).

The point is that the two EM10 messages have the same info. It no longer matters how the message got here (but it is traceable, using the remaining details). And all the code to accomplish this is in a single source file, right next to the sketch running on the ookRelay board.

This design makes it possible to develop an application using only the serial USB connection, and then later add logic to send the information via RF12 (or not). Infrared, XBee, Twitter, anything: transport independence!

Note that nothing is done with these decoded messages at this stage. This messaging framework is independent of higher-level application decisions, such as where to store / send / show msgs, or even when to process them.

Serial port encapsulation

In Software on Feb 19, 2010 at 00:01

This post continues to look a bit into the new JeeMon design.

Let’s focus on serial interfaces first, mostly USB. There’s a “Serial” module which does all the basics. On the Mac, if I want to open device /dev/tty.usbserial-A900ad5l, then the following call with do everything:

Serial connect usb-A900ad5l 57600
The name of the device would be different on Windows and Linux (COM5, or USB1), but that’s all.

By default, this creates a new Serial object, which logs all incoming text to stdout. To send a command out, we need to keep a handle to this object:

set conn [Serial connect usb-A900ad5l 57600]
$conn send "some text"
Nice, but not very exciting…

Let’s take it one step further. The “JeeSketch” module does the automation described in the previous post, i.e. detect the running sketch, associate it with a class, instantiate an object for it, and call the methods of that object whenever text lines come in over that serial port. Here’s a complete JeeMon custom “application.tcl” program:

Screen shot 2010-02-18 at 14.44.59.png

First, all the sketch drivers are made available with one or more “register” calls. This lets the appropriate classes take over for each new serial connection – depending on what sketch is running. That’s all it takes. Servicing such serial ports now becomes an event-driven background activity.

The listeners are defined in separate files, one for each type of sketch:

Screen shot 2010-02-18 at 15.05.58.png

The ookRelay/host.tcl file looks like this, for example:

Screen shot 2010-02-18 at 14.51.58.png

This structure makes it easy to manage stuff that belongs together. Projects can be exchanged (or archived, or revision-controlled), with all the pieces needed to use them in one place. And as far as I’m concerned, it won’t be limited to JeeNodes etc, or Tcl, or a specific platform. This has to remain general enough to hook up to any hardware and use with any language (via networking, files, and direct launching of executables/scripts). My goal for JeeMon is not to limit anyone’s options, but to create a simple switchboard between whatever is needed.

(I’m still mucking around with the organization and naming of code and files, as you can see)

Nice, but still not very practical…

The problem with the above is that it doesn’t deal with devices getting plugged in or unplugged. Well, unplugging is the easy bit – the above code will automatically clean up after itself on connection loss, so that part is covered.

Wouldn’t it be nice if we could just plug in new devices and get them to automatically start doing something?

I implemented such a mechanism in a recent revision of the code, but I’m hesitant to add it again – because it was Mac OS X specific, where USB devices connected via the FTDI driver include a unique code. On Windows and Linux, you just get COM<n> and USB<n> devices, where “n” seems to be related to the order and number of device insertions.

I haven’t looked into “libusb” yet. Should I? Will it help for Windows too? Do I need to start learning about USB enumeration? What OSS-compatible tools and libs are there?

Update – on Linux, it looks like /sys/bus/usb/devices/* has all the info needed to identify USB devices. So that only leaves Windows – good: at most one lib or dll to deal with.

In the kitchen

In Software on Feb 18, 2010 at 00:01

Ok, so maybe it’s time to start describing some of the new stuff cooking in the kitchen lab.

I’ve been exploring software designs to use as basis for JeeMon, that new switchboard-type application for physical computing devices. The idea is to treat the system as one big message-passing machine – a bit like Message-Oriented-Middleware (MOM), which has been around for ages, but without getting sucked into any heavy-weight conventions.

Messages can be passed around, queued, stored, duplicated, filtered, transmitted, returned, ignored, sorted, etc. After all, living organisms do nothing but send chemical messages around, so why not do the same for an infra-structure focused on physical computing (sensors, displays, actuators) and home automation?

If you’ve been following this weblog for a while, you’ll have noticed that almost all the output I generate in sketches is of the form “identifier arg1 arg2 …”. So for example, packets received by RF12demo look like:

OK 61 9 2 8 79 243 87 13 0 15 0
A barometric reading from the BMP085 sensor on the pressure plug may get reported as:
BMP 233 10019
And so on. One or more upper case letters, optionally followed by digits. Then the payload (which may also include floating point and string values, not just ints).

Another convention I’ve been sticking with is to report the name of the sketch on the serial port during startup:

Or an identifier plus the current configuration settings:
[RF12DEMO] W i23 g5 @ 868 MHz
These two conventions can be used for an object-oriented design. With a bit of preparation, the name of such sketches can be automatically associated with a class, and each line treated as a method call.

Here’s the skeleton of the first level of code I use for decoding RF12demo.pde output:

Screen shot 2010-02-16 at 00.15.27.png

Another class definition, for the ookRelay.pde sketch:

Screen shot 2010-02-16 at 00.16.11.png

Or to put it differently: with JeeMon running, you can hook up a device (JeeLink, JeeNode, Arduino, etc) containing some sketch and the matching class will automatically be associated with that serial port, once the sketch has been identified. A new object is then created, and its methods will be called whenever the device sends new output (messages?) over the serial line.


What I would like to do, is manage the sketch (C/C++) and the class (Tcl) together, perhaps as two files in a common development directory for that project. That way the interface between the two pieces of hardware essentially vanishes into the background. The point being that each class can then do all sorts of things, such as storing results in a database, sending it to another system over the network, updating web server pages, popping up a GUI window to show incoming data in real-time, etc.

This mechanism is very simple, even under the hood. This matters, because it has to work even when JeeMon is running on low-end embedded Linux hardware. But at the same time, such a MOM + OO design will allow considerable flexibility for abstract / high-end use later.

PS. If you’re familiar with Tcl, you might be surprised to see all this “oo” stuff in here. That’s because JeeMon uses Tcl 8.6 with built-in object system. Multiple inheritance, mixins, filters, dynamic class change support, delegation, prototypes / per-object methods, it’s all in there. I’m also using ensembles and there’s an interesting coroutine-based web server waiting in the wings (called wibble).

For the record: nothing ever gets added just to be buzz-word compliant. If a feature simplifies application-level concepts and leads to a cleaner implementation in JeeMon, it’ll be used, else I’ll pass. Life’s too short to jump on bandwagons.

Temperature plots

In Software on Jan 22, 2010 at 00:01

With over a dozen nodes now active, JeeMon is finally starting to collect a bit more useful data to work with:

Screen shot 2010-01-22 at 12.03.23.png

Looks like some temperatures have a granularity of 1°C – might be a few cheap DS1820 units I had…

The Mason-like templating code in “try-temps.html” to generate this plot is still far from obvious:

Screen shot 2010-01-20 at 01.52.42.png

Once I get to spend some real time on all this, it’ll look totally different. JeeMon as a whole is going to change completely in fact – until then, you can find some technical background information in an older post.

Let’s just say that it gets the job done right now…

JeeMon – one year later

In Software on Jan 4, 2010 at 00:01

JeeMon is a little web server / database / reporting tool I wrote a while back. It has been in use at Jee Labs for over a year now. Here is the electricity use for 2009:

Screen shot 2009-12-31 at 17.27.15.png

Couple of glitches, such as incorrect readouts when power failed and the sending node got reset to zero counts.

Here’s the gas consumption for 2009:

Screen shot 2009-12-31 at 17.28.06.png

Gas consumption (heating and hot water) is relatively high for this house, which is an open split-level design with lots of windows. Well isolated, but there’s simply no way to keep heat from rising up through the open stairways.

Here are the temperature and humidity readings for the past month:

Screen shot 2009-12-31 at 17.38.39.png

These are commercial S300 and KS300 sensors, and they seem to have frequent glitches. It looks like those could easily be filtered out, though.

Still, we managed to get €600 back on the 2009 energy bill, and with about 3000 kWh and 2000 m3 we’re 30% below the average consumption in this residential neighborhood, for both electricity and gas.

Being aware of what’s going on makes a difference, IMO. It has become a habit to turn off all the lights when I leave a room, and closing curtains right away when it gets dark. When we go out, we turn down the thermostat. Who needs high tech, when common sense is all you need? It’s so obvious, yet so effective …

The JeeMon database for all of 2009 is 26 Mb, i.e. tiny when considering that this includes every reading and some aggregated series as well. In fact, it contains a lot more data than what’s shown in the above graphs.

I’ve got several ideas and plans for JeeMon in 2010. I want to make it far more modular, so that nearly all its current functionality becomes available as easy-to-extend plug-ins. And it needs to be fullly configurable – as it is, JeeMon is still little more than a one-off implementation. But it has served its purpose very nicely already.

More efficient computing

In Hardware on Nov 4, 2009 at 00:01

A few days ago, I replaced my (3y old) MacBook Pro + (8y old) Cinema Display with a brand new 27″ iMac. Here’s the result:

Screen shot 2009-10-31 at 17.59.38.png

That’s the total energy consumption of my work desk – i.e. computer, lights, and a some USB-powered stuff. The scale is in Watt-per-5-minutes, so if you multiply all figures by 12 you’ll get the normal Watt ratings.

There are many different operating modes here – night-time near off, full sleep, display sleep, display on, a few computing peaks, and in the evening also the lights on.

On Oct 30 both machines were active, while all the data from the old setup was being migrated to the new one.

The interesting parts are the low periods at night which more or less dropped to zero, even though the iMac is simply sleeping (with BlueTooth and USB still on), not turned off.

It’s also nice to see that even with this new dual-core 3.3 GHz monster, energy consumption is noticeably lower most of the time – not just in idle mode!

These readings were obtained with the ES-1 sensor described a while back + a central JeeNode + JeeMon.

Monitoring failure?

In Hardware, Software on Oct 23, 2009 at 00:01

Yesterday, the Jee Labs energy monitoring system stopped working:

Picture 2.png

A few hours went by before I found out, because I’m not constantly checking these graphs nowadays.

Couldn’t figure out why, so I reset the sending node next to the electricty/gas meters, and shrugged it off as a fluke because it all started working again.

A few hours later, after dinner, we find out that there’s water dripping down right next to that JeeNode, but more importantly the whole meter closet is soaking wet in some places. Whoops… not good!

Turns out that there was a leak in the kitchen drain right above that meter closet. Fortunately, the problem was easy to identify. And didn’t lead to fireworks.

That was around 8 PM. Only this time the JeeNode is also soaking wet – free-hanging constructions are not such a good idea after all, I guess. Unplug. Wipe off the water. Let it dry for a few hours. Plug it back in.

OK, the energy monitoring setup is working again. Tomorrow, I’ll fix the drain pipe – properly.

The drawback of tinkering is that you always blame your own stuff when there is a failure. But in this case, it was trying to tell me about a problem coming from elsewhere…

OOK packet decoding

In Software on Oct 22, 2009 at 00:01

The OOK packets are now collected and stored in the JeeMon database.

Here are the first few readings from the barometric pressure sensor inside the OOK relay, as received by the central node and graphed by JeeMon:

Picture 3.png

Here is the outside temperature from a WS300 weather station on the roof:

Picture 1.png

JeeMon is written in Tcl – the following new code was added to process packets from the OOK relay and get the different parameters stored:

Picture 2.png

The RELAY decoder processes incoming data, separating the bytes into one or more messages and calling the corresponding type-specifc decoders. The RF868 decoder merely reformats incoming data to the format used by another decoder called “ALT” and which was already present in JeeMon. The RF434 decoder is just a stub for now. Note also that the DCF decoder logs the current time – it may seem odd, but storing the received time along with the current system time can be used later to detect local clock drift and compensate for it.

It’s perhaps all mumbo-jumbo for now, I just wanted to illustrate how messages get routed inside JeeMon, and that it takes relatively little logic to deal with new data sources.

This would really benefit from a flexible plug-in system, so that this sort of logic does not have to be added in JeeMon itself but can be picked up on-the-fly. Then it would be easy to add new nodes such as the OOK relay and have JeeMon automatically incorporate the corresponding logic to decode, process, and store their data packets.

Parallel WSN choices

In Hardware on May 20, 2009 at 00:01

Just came across the eKo range of commercial products by Crossbow. Interesting how the eKo and Jee-Node/-Hub/-Mon designs almost exactly overlap – well, seen from a large distance anyway.

Their “JeeNode”:

Picture 2.png

Their “JeeHub”:

Picture 3.png

Their Linux server hardware and software:

Picture 4.png

Of course the eKo products are a million times more mature and advanced than the Jee-stuff which isn’t even a product, but still…

Bubba JeeMon back up

In Hardware, Software on May 12, 2009 at 00:01

The title of this post can be interpreted in two ways, both accurate: 1) the Bubba Two NAS from Excito was my back-up server for the MMnet1001 module, and 2) that Bubba box is now my main JeeMon server and back in the air with all the electricity / gas / weather monitoring working again, as well as the Pachube and Bwired feeds.

I will probably stick to the Bubba from now on. It’s running Debian Etch Linux, and there’s a JeeMon runtime for it so the switch-over was trivial.


This NAS box consumes about 4 watt, which is less than the specs because it has been modded to use a 2.5″ SATA disk instead of the standard 3.5″ ones they normally ship. It’s also a 300+ Gb file server which has been running various periodic tasks for almost a year now. There’s in fact much more disk storage in the form of 3 additional USB disks, but these are normally powered down (via a KAKU switch).

Added data feed

In Software on Apr 27, 2009 at 00:01

The website has been promoting domotics “by example” for a long time now. Pieter Knuvers, the guy behind it all, added a new service to submit and display data from other fellow energy geeks like me…

The bwired upload service was announced (in Dutch) on Pieter’s weblog.

So I added the bwired.tcl Tcl code to JeeMon to generate the XML input he wants:

Picture 2.png

It’s trivial but way too messy stuff, mainly because I don’t have the proper abstractions in place yet to easily generate all sorts of aggregated data. In this case hourly averages and daily totals.

Results can be viewed here, name “jcw”.

Oh well, good exercise. Pretty much the same logic as for the Pachube feed.

JeeMon self-update mechanism

In Software on Apr 26, 2009 at 00:01

As I’ve mentioned before, JeeMon is a self-contained executable which requires no installation and runs on Windows, Mac OS X, and various Linux systems (including the ARM chip on the MMnet1001 modules used in the JeeHub). There are no special requirements or dependencies, no Java runtime, no .NET, no nothing (note: on Linux you have to check that the standard C++ runtime libraries are present).

The JeeMon builds are here – you only need to fetch one and launch it to use it.

This is based on a generic runtime called Tclkit which contains a full implementation of the Tcl programming language as well as the Metakit transaction-based embedded database. It’s hard to overestimate the amount of functionality included in these 1 .. 2 Mb executables.

The only specific code in JeeMon is the startup logic to turn it into a web-server with home monitoring functionality. It’s all inside one main.tcl file, the essence of which is shown here:

Picture 3.png

The code starts off with a few definitions and then loads the rest of the application. From file if present, else from a file loaded from the internet. The “fetchLatestLibrary” code by default also updates to the latest version, this can be disabled by deleting the “Jee-library.update” file.

By changing one URL shown above, JeeMon can be turned into any other type of Tcl application, because everything the application actually does comes from the downloaded file.

For those who don’t like this automatic mode, there is always the option to download all the pieces and run them as is. Both JeeMon and the Jee-library are based 100% on open source software. JeeMon is implemented in C/C++ and can be compiled from scratch using the standard gcc/automake tools, and Jee-library can be unwrapped to inspect every single bit of it – it is nothing but a packaged collection of plain text files and images. I’d be happy to document this if needed.

I suspect that most people won’t quite appreciate the implications of the way JeeMon is built and packaged until they try it out. And the reality is that most people won’t ever try out this stuff – fine with me. The technology underneath JeeMon is just a set of choices. In the end, the degree to which these choices become invisible in day-to-day use of this system is all that matters. And for a platform-independent modern programming environment, it doesn’t get much more general purpose and deployable than this…

Tracking JeeMon status

In Software on Apr 25, 2009 at 00:01

This is what I see on my screen these days:

Picture 1.png

This text is not inside a window but on the desktop, i.e. it gets covered up by all regular windows (there is always Exposé’s F11 shortcut to uncover it).

The text is barely readable, not only using a tiny font but also using 50% transparency. This is intentional: the text is not there to distract but to offer me a quick way to check on the status of my permanently-running setup on the JeeHub with embedded MMnet1001 Linux module.

It’s all done through a little utility called GeekTool, there are no doubt similar ones for Windows and Linux. Geektool periodically executes some shell commands and shows the output on the desktop.

The first listing displays the last 5 lines of the log output:

ssh propie tail -5 jee_out

The second listing is plain-text output generated by a new web page in JeeMon:

wget -q -O- http://propie/data/current

Adding that page to JeeMon was trivial (data collection details omitted):

Picture 2.png

Now, I can always see the latest readings and the status of the JeeMon server.

P.S. I will be away until May 3rd and won’t be able to respond to emails and comments until then. This weblog will be on auto-pilot for a week, enjoy!

Nine times faster

In Software on Apr 17, 2009 at 00:01

The graph pages in JeeMon are a lot faster now. Here are the “before” …

Picture 1.png

… and “after” timing statistics:

Picture 2.png

A nine-fold speed increase. Great.

The change is that hourly and daily values have been re-calculated and stored in the database, instead of constantly recalculating these on-the-fly. The last value is not stored, as it might be incomplete and could change when more readings come in.

Keep in mind that these measurements are made with the MMnet100 Linux module, which is a very low-cost / low-power ARM-based system. On a modern PC, the graph pages usually come out instantly, even without storing any condensed hourly or daily datasets.

The other thing to keep in mind is that all JeeMon processing takes place in scripted languages: Tcl on the server and JavaScript in the browser. And that this is handling non-trivial amounts of data, since JeeMon is basing all these graphs on 5-minute values collected for up to an entire year.

In short: 9x faster is good enough, the graph pages now finish loading within two seconds.

It’s nice to see hunches work out in reality…

Update – on the MMnet1001, the UBIFS flash filesystem has compression turned on as default for all files. It would seem to make more sense to keep Metakit database files uncompressed, so I did a “chattr -c Jee-database” and made sure to rewrite the file from scratch with decompressed data. However, it looks like this only shaves another 5% off the graph page times, bringing times down to from 2.0 to 1.9 seconds. Oh well, it was worth a try.

Inside JeeMon

In Software on Apr 14, 2009 at 00:01

I thought I’d share a bit here how JeeMon works and what it’s made of.

One of the main goals for JeeMon, is that it should support two modes of operation:

  • running on a personal computer as one of many processes, launched and stopped at will
  • running on a dedicated system which is always running, possibly with quite limited resources

There is not that much difference between these two modes, but it does mean JeeMon should present itself as a webserver and that its processing demands have to be low. Being portable to different platforms also helps.

JeeMon is essentially a webserver plus database, both embedded and combined into a single process. It also contains all the logic to collect readings from – and send commands to – a JeeHub via a USB or serial port.

I picked the technology I know well, some of which I have helped develop or developed myself over the past years:

  • the code is written in two languages: Tcl on the server and JavaScript in the browser
  • the Tcl runtime uses a highly portable and self-contained system called Tclkit
  • the web server is my own design, using a highly modular Mason-like system called Mavrig
  • the embedded database is Metakit, for very compact and efficient storage of data
  • the JavaScript code uses jQuery as library to simplify many tasks
  • interactive plots are generated in the browser using the Flot package
  • the JeeMon application is packaged as a Starpack into a single installation-free runtime
  • similarly, all Tcl code, Javascript code, and website content is wrapped up into a Starkit

[more details below…]

Read the rest of this entry »

Graph navigation

In Software on Apr 10, 2009 at 00:01

JeeMon lets you zoom in (“drill down”) on the electricity / gas / water consumption data. Here’s how:

Picture 1.png

That’s 4 graphs: 1-day details with 5-minute resolution, 2 weekly summaries, and an entire year.

Picture 2.png

The weekly graphs only differ in the level of detail: on the left is the daily change each hour, whereas the graph on the right shows daily totals.

Clicking on a day in either of the weekly graphs causes the daily graph to display that day. And a click on any week in the year overview switches both weekly graphs to that week.

With two mouse clicks on this page you can display the details of any day in the past year. To easily go back to the initial display, a “Go to Today” button appears once a different graph has been selected (same as a refresh).

Note also that selecting a time range in the daily graph calculates the total consumption in that period, and hovering over a bar in the bar graphs displays the exact value as a tooltip.

No other zooming or panning is available, just these fixed choices. Simple and effective, IMO.

JeeMon demo (alpha)

In News, Software on Apr 9, 2009 at 00:01

Today I’m releasing a first version of the open source JeeMon home monitoring application, along with a demo dataset. Warning: this is an early alpha version – all features and bugs are still likely to change, a lot!

There are builds for Windows (x86), Mac OS X (x86 and ppc), and Linux (x86 + x86_64 + some embedded systems, such as the JeeHub’s MMnet1001 module).

It takes 3 steps to run this demo – there is no installation:

  • Get a runtime for your computer from this area
  • On Unix-like systems, run this: chmod +x JeeMon-*
  • Launch JeeMon

Here is the debug output this generates on my laptop:

Picture 1.png

And here’s a Windows screenshot with similar output:

Picture 1.png

Now point your browser to http://localhost:8888/ to see what JeeMon has to offer.

When started for the first time, JeeMon downloads a few extra files from the internet: “Jee-demodata”, “Jee-library”, and “Jee-library.update” (under 2 Mb total). These files are obtained from public servers at Dropbox, I do not track downloads or statistics.

On every subsequent startup, JeeMon looks for updates and refreshes the “Jee-library” file if there is a new version, so by simply starting JeeMon again a few days later you can track its development progress. To disable automatic updates, delete the “Jee-library.update” file.

I’m releasing this code to give you an early glimpse into the Jee Labs kitchen – and to gauge the interest and figure out how to improve JeeMon. If the demo works for you, great. If it doesn’t, you can either try to figure out what the problem is, or delete all the Jee-* files and try again at a later date. The new mailing list announced yesterday is open to anyone wishing to comment and make suggestions.

Let me reiterate that this JeeMon alpha release is for infinitely curious and technically interested people, not for those looking for a finished home energy monitoring solution. Running this JeeMon demo is bound to raise more questions than I can deal with – but it’s all open source, so feel free to explore this as much as you like.

Whatever. Enjoy …

Update – the demo dataset now contains data since 2009-01-01, i.e. a total of over 3 months of readings.

New "jeelab" mailing list

In News on Apr 8, 2009 at 00:01

For those who want to share their interest / questions / ideas w.r.t. any of the Jee Labs projects, I’ve set up a mailing list at Google Groups:

Picture 2.png

See – if you don’t want to sign in to Google to join this list, let me know and I’ll add your email address manually.

Update – the mailing list has been replaced by the Jee Labs discussion forums at

It gets a lot worse…

In Software on Apr 7, 2009 at 00:01

Yesterday’s post was about web server performance. Here’s the latest build of JeeMon with the same page:

Picture 1.png

That’s nearly 17 seconds to render the page now, versus 5 yesterday… whoops!

Here’s why I call this major progress anyway:

  • The “electricity.html” page no longer contains plot data values and is half its previous size.
  • All plot values are obtained via Ajax calls, running (mostly) in parallel.
  • This opens up the path to unobtrusive refreshes and automatic plot updates.
  • All plots now contain meaningful data, the previous version used some bogus values.

As you can see in the above graph, over 15 seconds is spent in waiting for the server to return all plot data.

And the reason this now takes so long is very simple: all values are calculated on-the-fly by the server from stored 5-minute values. That’s over 100,000 aggregated values to generate a simple 52-week bar graph!

All this should sort itself out once I start caching hourly and daily results in the database. For now, only the raw readings and the 5-minute aggregated counts get stored. The finecky bit is to make all aggregations / calculations consistent across arbitrary JeeMon restarts.

Here’s the latest version of the generated page:

Picture 2.png

Website debugging tool

In Software on Apr 6, 2009 at 00:01

Just to prove that JeeMon is far from ready for real-world use…

The Safari web browser has excellent tools to figure out where bottlenecks are. Here is an overview of where the loading time is going for the main electricity consumption page, as produced by JeeMon running on the MMnet1001 Linux module. That web page has several detailed graphs on it – here are the stats:

Picture 1.png

As you can see, all the time is consumed by the initial page generation on the server. That’s 4.64 seconds to generate the page – which is totally unacceptable. JeeMon serves that same page 50 times faster when running locally on my Mac laptop (!).

Here are the amounts of data transferred to serve that page:

Picture 2.png

The main page including the plot data is about 5 % of the total size – the other files are cached as static content, which is why they do not increase the transfer time.

There is a lot more information to be gleaned from these measurements, btw.

I have a hunch on where the bottlenecks in JeeMon are, but haven’t looked into it yet. Which is exactly how it should be: aim for solid / correct behavior first and use the proper instruments to measure performance and track the effects of optimizations later (I happen to use Safari, but similar tools exist for other browsers).

Stay tuned for updates, once I get to tweaking this…

JeeMon sneak preview

In Software on Apr 5, 2009 at 00:01

I’ve started work on a basic website design for JeeMon:


Here’s the gas consumption page:


The same JeeMon code is currently running on the “production” (ahem) JeeHub and on my development Mac.

These are real live webpages, with real (but not yet 100% correct) datasets. Still, this is just a tiny tip of a massive iceberg. Lots and lots of things left to do.

Well, at least it’s a pretty iceberg IMO ;)

Real-time tracking

In Software on Apr 3, 2009 at 00:01

Here’s a real-time graph of data collected by JeeMon:

And a snapshot, in case the above real-time graph isn’t working right now:

Picture 2.png

It’s generated via a new site called pachube – I’ve extended JeeMon to feed that site through simple REST-style HTTP requests. Since the graph only has 15-minute resolution, that’s how often it gets updated.

Here’s another real-time feed, showing our gas consumption in l/hr:

Pachube is still in beta. I’ve only just started looking into it. You can define any number of feeds, each having any number of data values. Not sure how it deals with missing values, nor what to do about data which is collected at different rates. The graphs are somewhat configurable, as you can see – this will no doubt still evolve further.

Pachube can poll a server you designate (“automatic” mode), or you can feed it new data at your own pace (“manual” mode), as is being done here. The advantage of the latter is that it gets through firewalls and avoids the security issues of running a public server.

It’s an interesting approach, apparently pachube aims to become some sort of free exchange for producers and consumers of location-specific real-time data.

I’ll leave this setup running for now, but don’t be surprised if this post stops showing an up-to-date graph, since I might stop this feed one day.

The logic of data logging

In Software on Mar 22, 2009 at 00:01

The point of several projects here is to monitor energy consumption, and probably also some weather data because it’s so easy to add. Where “monitoring” means logging as well as keeping measurement data in a database for quick access. But how do you make sure that all “readings” get logged, yet still retain the ability to adjust and extend the underlying database and software? It would be awful if each non-trivial change required conversion.

Here are a few basic design decisions I took for JeeMon:

  • Readings come in as text from the attached JeeNode, one line per reading.
  • All readings are time-stamped and appended to a logfile, in text format.
  • To keep things manageable, logfiles roll over to a new one on 0:00 GMT each day.
  • The first entry in a new logfile is the name and size of the previous one.
  • The database is treated as a cache: it can be reconstructed from the logs at any time.

And here’s a requirement I want to meet:

  • It must be possible to restart JeeMon at any time without losing readings.

The idea then is to “catch up” with the logs on each restart where possible, and to clear the cache db and reload it from scratch when the software changes are more substantial.

So what does it take to not lose any readings? Keeping in mind that JeeMon will run either on a little embedded Linux module right next to the central JeeNode, or on a personal Mac, Windows, or Linux computer. Well, first of all, this is why the central JeeNode has a backup battery and dataflash memory: it stays on at all times, and continues to receive and collect packets from all the other JeeNodes even when there is no JeeMon running. The duration of this autonomous operation will be fairly modest: a few hours or perhaps one day. Enough to handle brief power loss, to mess around with configurations, and to re-connect things occasionally.

This means there are now three places where data gets added and must be kept in sync with the rest:

Picture 1.png

(Four places if you also count the final destination: dynamically updating browser windows)

  • The dataflash memory gets a copy of each reading collected by the central JeeNode.
  • All readings must end up in the log files when JeeMon is running.
  • The cache database stores decoded values in an efficient internal format.
  • And lastly, the browser(s) present more or less detailed results, summaries, and derived info.

JeeMon will start up in logfile “catch up” mode: the current content of the dataflash memory will be checked against the last logfile entry. The first task is then to request old data from the JeeNode to bring the log files up to date. During this time, the JeeNode continues to add new incoming data to the dataflash.

Once all missed data has been transferred, the JeeNode switches to “real-time” mode, saving new readings to dataflash and sending them out to JeeMon at the same time. In real-time mode, the log files will track all readings as soon as they come in.

That’s just part of the story, though. Now JeeMon enters “db sync” mode. With information from the cache db, new entries and new log files are scanned and processed, with new readings added to the database. Once that is done, JeeMon switches to normal real-time operation.

All log files are kept online. Rebuilding a database from scratch is as simple as deleting the current one and restarting JeeMon. Since a full rebuild might take quite some time, the internal JeeMon webserver always starts up in a “please-wait” mode and switches to the real server after real-time operation has been enabled.

Part of the above logic has now been implemented. JeeMon resumes its logs, syncs / rebuilds its database as needed, and processes new readings while running. The firmware in the JeeNode to save and replay readings to and from dataflash is still work-in-progress, however.

So while the JeeHub / JeeNode needs to stay powered up, I can now restart JeeMon at will. Which is great, because that streamlines development.

Let's pick a few more names

In Hardware, Software on Mar 20, 2009 at 00:01

Not to worry: I’m not going to go crazy on the Jee<blah> naming used so far…

But I do need to give the different pieces some name. To be able to refer to them in these posts, but more importantly as names for the software for all this. I also need to introduce a basic structure (and some limits).

Picture 1.png

Here goes:

  • A JeeNode is this ATmega-with-4-ports-and-RFM12B thing. There may be lots of JeeNodes for various purposes. They can communicate with each other via wireless.
  • I use the Ports and RF12 libraries with JeeNodes, although this is not a hard requirement – both the JeeNodes and the libraries can be used in numerous other ways.
  • Each JeeNode has a letter ‘A’ to ‘Z’ assigned to it as node ID. Nodes are usually given a unique ID to avoid mixups, but this is not strictly required.
  • There can be up to 250 separate groups of JeeNodes. Nodes can only communicate with other nodes in the same group. Gateways between groups could be implemented later, if needed.
  • Most nodes will be pulse JeeNodes, i.e. running a specific piece of software called – you guessed it – “pulse”, which continuously monitors some attached sensors and reports the readings via wireless.
  • The JeeHub is either a JeeNode by itself connected to a Mac, Windows, or Linux PC via USB, or a JeeNode connected to Ethernet via a small dedicated Linux module.
  • This JeeNode-as-part-of-a-JeeHub is called the central node from now on. It runs a specific software configuration, also called “central”. There should always be exactly one central node.
  • The software running on the Mac, Windows, or Linux machine(s) is called the server from now on (how original, eh?). It consists of a system-dependent executable runtime called JeeMon plus the code and data for the application itself.

It probably doesn’t hurt to reiterate that “JeeNode” and “JeeHub” are hardware, whereas “JeeMon” and the Ports / RF12 libraries are software.

The above names are also used in the source tree repository now.

There are no doubt still infinitely many ways to lead you astray, but I hope that these definitions will help me place everything into an increasingly coherent context.

Collecting data with JeeMon

In Software on Mar 17, 2009 at 00:01

This is the information I’m currently tracking real-time:

Picture 1.png

(it’s in Dutch, but you can probably guess most of these)

And this is the setup I’m using for it:

Picture 2.png

The NAS in there is used as gateway to pass 868 MHz OOK data from a CUL receiver, which decodes signals from my KS300 weather station plus a couple of S300 temperature/humidity sensors. This will one day be replaced by on-board reception on the JeeHub, so that only the JeeHub needs to stay powered up at all times. All other sensors are hooked up to a couple of JeeNodes which transmit the readings wirelessly.

The Mac, Windows, Linux, and the JeeHub all run identical copies of the software, which is called “JeeMon”. It has a built-in web server, an embedded database, and a flexible set of network functions.

Each JeeMon instance will automatically self-update to the latest version on startup. During development on the Mac the JeeHub acts as transparent proxy, as if the different sensors were connected directly to the Mac (through a little Tcl-based system called “Tequila”). Once ready, the latest JeeMon release is wrapped into one file and placed on the internet. Finally, a restart of the JeeHub completes the upgrade.

That first screen dump above is a small test app on my Mac which bypasses JeeMon and connects directly to the JeeHub as Tequila client. I keep it open to check that data is coming in and gets saved on the JeeHub.

So this is the big picture for collecting energy/gas/water and environmental data in the house. The software can run on practically anything, can be accessed with a browser anywhere, and with proper security in place the various pieces can be connected and used across any network topology.

All of the above is working this very moment. The major task ahead is the full-scale processing, presentation, and interaction of it all. But that can now conveniently be done on my development machine, with HTML, CSS, JavaScript, etc.

Anyway, IMO this is a very flexible foundation for a 1-watt home monitoring server.

Ignoring imprecise data

In Software on Feb 9, 2009 at 12:05

For some time, I’ve had a second set of calculations running which produces much more accurate and regular data than some of what’s on these past charts:

Picture 1.png

The difference with the graphs is that only readings which come in exactly at the time the counter changes are considered here. All retransmission (i.e. readings tagged with a late time stamp) are simply ignored. Only one retransmission happens to be in this example (the one marked “6+2”), but it still produces consistent results.

It looks like the gas measurements could be improved by tagging all readings in the sensor node instead of at the receiving end of the packets. That way retransmissions would still be tagged properly, even when readings arrive a bit late.

Hmm, this requires running a clock with milli-second accuracy on the sensor board for best results… not entirely trivial over extended periods of time. The sensor board will need to be synchronized to some other clock – either an on-board DCF77 radio or some data received from another node. And I’ll need to use a crystal instead of a resonator (0.005% vs 0.5% drift).

Still not right

In Software on Feb 8, 2009 at 23:55

The gas measurements are still giving me trouble:

Picture 3.png

I’ve changed scales and plotted all values to better see what’s going on over a small time frame. The electricity consumption looks ok now (these are night-time values). But the gas measurements keep jumping up and down by a factor of around 5 all the time!

Maybe there’s a problem with the sensor readout – there are about 4 readings coming in per minute. Or perhaps the automatic idle transmission of packets every 10 seconds is interfering with this cycle?

I don’t know what’s going on… needs more work!

Measurement anomalies

In Software on Feb 7, 2009 at 02:38

Here’s a more detailed view of the measurement quirks related to gas consumption:

Picture 1.png

The yellowish line is electricity usage, which hovers around 200 watt, while the blue line is the calculated gas consumption. What happens is that the heater cycles on and off approximately once every 15 minutes. When off, no gas is consumed so the meter just stops providing pulses. Then, after roughly 10 minutes it starts again at our heater’s standard rate of around 1.5 m3/hr. If you look very closely, you can see that the “zero” values are actually just above zero. That’s because whatever gas flowed until the rotation stops and whatever gas flows right after it starts all gets averaged-out over the entire period.

What needs to be done, I think, is to re-interpret the long averaged-out period as a period of continued flow at the original full rate, followed by zero flow:

Picture 3.png

I.e. the incoming data was treated as the single-hatched “B”, whereas it should be treated as the cross-hatched bar “A”. Both areas are the same.

Actually, this is not 100% accurate since we don’t know whether the meter stopped near the beginning or near the end of its revolution. But there is simply not enough information in the gas measurement data to disambiguate these cases.

However… as you can see, the electricity consumption also has some surprisingly regular small changes. Those measurements suffer from the same problem, but they have a much smaller impact due the the higher measurement rate. My hunch is that these small wattage changes are actually the electricity consumption of the heater, as it turns on and off!

But hey, who cares for now – these are just instant-by-instant consumption rates. The integral of these rates is the total consumption, and those are by definition “accurate” since I’m measuring the very same metering revolutions which the electricity and gas company use to charge me later on.

Update – Here are the results with the above adjustments:

Picture 1.png

Energy plots

In Software on Feb 6, 2009 at 00:03

And here’s the electricity and gas consumption for (part of) last Wednesday:

Picture 1.png

There are some quirks due to the way things are measured. The meter cannot accurately detect when gas consumption drops to zero, since that simply produces no readings. This can be resolved by adding fake readings when a very low consumption is reported, estimating the times when the gas flow stopped and then started again.

The electricity peaks at 19:45 are from the induction stove, the one around 20:15 is probably the close-in boiler.

This Tcl code generates the necessary HTML/JavaScript snippet to produce the plot:

Picture 3.png

All measurement data selection is handled by the “dataset loop” command, which takes a measurement parameter name, the selection range, and the names of the time and value variables to set during the loop.

Temperature graphs

In Software on Feb 5, 2009 at 13:30

Some weather & energy data has been coming in over here for over a month now, logged to text files, and patiently awaiting further processing. Here are the results of a few days of hacking around on the software side of things:

Picture 1.png

This is a test page served by a custom web server, with all data now stored in a database. I’ve used several bits of software which are dear and near to me:

  • the database used is my own trusty Metakit
  • the custom software is written in Tcl
  • the web server is based on my Rig modules
  • the plots are generated in the browser by Flot
  • it’s all in two files using using Tclkit / Starkits

The code is portable and works on Mac OS X, Windows, and Linux. The system requirements are minimal, so it runs just fine on my NSLU2 and Bubba NAS boxes.

Things are starting to come together nicely…

More readings

In Software on Dec 19, 2008 at 15:55

Here is the latest display showing on my screen:


Updated in real time, with new readings coming in every few seconds for electricity and gas and every few minutes from the weather station (still sitting here in the office for now, so the values are bogus).