Home monitoring and home automation have some very obvious properties:
- a bunch of sensors around the house are sending out readings
- with actuators to control lights and appliances, driven by secure commands
- all of this within and around the home, i.e. in a fairly confined space
- we’d like to see past history, usually in the form of graphs
- we want to be able to control the actuators remotely, through control panels
- and lastly, we’d like to automate things a bit, using configurable rules
In information processing terms, this stuff is real-time, but only barely so: it’s enough if things happen within say a tenth of a second. The amount of information we have to deal with is also quite low: the entire state of a home at any point in time is probably no more than perhaps a kilobyte (although collected history will end up being a lot more).
The challenge is not the processing side of things, but the architecture: centralised or distributed, network topology for these readings and commands, how to deal with a plethora of physical interfaces and devices, and how to specify and manage the automation rules. Oh, and the user interface. The setup should also be organic, in that it allows us to grow and evolve all the aspects of our system over time.
It’s all about state and messages: the state of the home, current, sensed, and desired, and the events which change that state, in the form of incoming and outgoing messages.
What we need is MOM, i.e. Message-oriented middleware: a core which represents that state and interfaces through messages – both incoming and generated. One very clean model is to have a core process which allows some processes to “publish” messages to it and other to “subscribe” to specific changes. This mechanism is called pubsub.
Ideally, the core process should be launched once and then kept running forever, with all the features and functions added (at least initially) as separate processes, so that we can develop, add, fix, refine, and even tear down the different functions as needed without literally “bringing down the house” at every turn.
There are a couple of ways to do this, and as you may recall, I’ve been exploring the option of using ZeroMQ as the core foundation for all message exchanges. ZeroMQ bills itself as “the intelligent transport layer” and it supports pubsub as well as several other application interconnect topologies. Now, half a year later, I’m not so sure it’s really what I want. While ZeroMQ is definitely more than flexible and scalable enough, it also is fairly low-level in many ways. A lot will need to be built on top, even just to create that central core process.
Another contender which seems to be getting a lot of traction in home automation these days is MQTT, with an open source implementation of the central core called Mosquitto. In MOM terms, this is called a “broker”: a process which manages incoming message traffic from publishers by re-routing it to the proper subscribers. The model is very clean and simple: there are “channels” with hierarchical names such as perhaps “/kitchen/roomnode/temperature” to which a sensor publishes its temperature readings, and then others can subscribe to say “/+/+/temperature” to get notified of each temperature report around the house, the moment it comes in.
MQTT adds a lot of useful functionality, and optionally supports a quality-of-service (QoS) level as a way to handle messages that need reliable delivery (QoS level 0 messages use best-effort delivery, but may occasionally get dropped). The “retain” feature can hold on to the last message sent on each channel, so that when the system shuts down and comes back up or when a connection has been interrupted, a subscriber immediately learns about the last value. The “last will and testament” lets a publisher prepare a message to be sent out to a channel (not necessarily the same one) when it drops out for any reason.
All very useful, but I’m not convinced this is a good fit. In my perception, state is more central than messages in this context. State is what we model with a home monitoring and automation system, whereas messages come and go in various ways. When I look at the system, I’m first of all interested in the state of the house, and only in the second place interested in how things have changed until now or will change in the future. I’d much rather have a database as the centre of this universe. With excellent support for messages and pubsub, of course.
I’ve been looking at Redis lately, a “key-value store” which is not only small and efficient, but which also has explicit support for pubsub built in. So the model remains the same: publishers and subscribers can find each other through Redis, with wildcards to support the same concept of channels as in MQTT. But the key difference is that the central setup is now based on state: even without any publishers active, I can inspect the current temperature, switch setting, etc. – just like MQTT’s “retain”.
Furthermore, with a database-centric core, we automatically also have a place to store configuration settings and even logic, in the form of scripts, if needed. This approach can greatly simplify publishers and subscribers, as they no longer need local storage for configuration. Not a big deal when everything lives on a single machine, but with a central general-purpose store that is no longer a necessity. Logic can run anywhere, yet operate off the same central configuration.
The good news is that with any of the above three options, programming language choice is irrelevant: they all have numerous bindings and interfaces. In fact, because interconnections take place via sockets, there is not even a need to use C-based interface code: even the language itself can be used to handle properly-formatted packets.
I’ve set up a basic installation on the Mac, using Homebrew. The following steps are not 100% precise, but this is more or less all that’s needed on Windows, MacOSX, or Linux:
brew install node redis npm install -g express redis-server (starts the database server) express (creates a demo app) npm install connect-redis node app.js (starts the demo app on port 3000)
There are many examples around the net on how to get started, such as this one, which is already a bit dated.
Let’s see where this leads to…