• Software – a virtual world, artificially constructed, and limited only by imagination
• Hardware – a real world, where electrons and atoms set the rules and the constraints

I’ve long been pondering about the difference between the two, and how I enjoy both, but in very different ways. And now I think I’ve figured out, at last, what makes each so much fun and why the mix is so interesting.

I’ve spent most of my professional life in the software world. This is the place which you can create and shape in whatever way you like. You set up your working environment, you pick and extend your tools, and you get to play with essentially super-natural powers where nearly everything goes.

No wonder people like me get hooked to it – this entire software world is one big addictive game!

The hardware world is very different. You don’t set the rules, you have to discover and obey them. Failure to do so leads to non-functional circuits, or even damage and disaster. You’re at the mercy of real constraints, and your powers are severly limited – by lack of knowledge, lack of instruments, lack of … power.

Get stuff working in either world can be exhilarating and deeply satisfying. Yes! I got it right! It works!

All of this appeals to an introvert technical geek like me, and all of this requires little human interaction, with all its complex / ambiguous / emotional aspects. It’s a competition between the mind and the software / hardware. There are infinitely many paths, careers, and explorations lying ahead. This is the domain of engineers and architects. This is where puzzles meet minds. I love it.

The key difference between software and hardware, when you approach it from this angle, is how things evolve over time: with software, there is no center of gravity – everything you do can become irrelevant or obsolete later on, when a different approach or design is selected. With hardware, no matter how elaborate or ingenious your design, it will have to deal with the realities of the real world.

So while after decades of software we still move from concept to concept, and from programming language to programming language, the hardware side more and more becomes a stable domain with fixed rules which we understand better and better, and take more and more advantage of.

In a nutshell: software drifts, hardware solidifies.

Old software becomes useless. Old hardware becomes used less. A very subtle difference!

The software I’ve built in the past becomes irrelevant as it gets superceded by new code and things are simply no longer done they way they used to be. There’s no way to keep using it, beyond a certain point.

Hardware might become too bulky or slow or power-consuming to keep using it, or it might mechanically wear out. But I can still hook up a 40-year old scope and it’ll serve me amazingly well. Even when measuring the latest chips or MOSFETs or LCDs or any other stuff that didn’t exist at the time.

Software suffers from bit rot – this happens especially when not used much. Hardware wears out, but only when used. If you put it away, it can essentially survive for decades and still work.

In practice, this has a huge impact on how things feel when you fool around – eh, I mean experiment – to try and to learn new things.

Software needs to be accompanied by documentation about its internals and it needs to be frequently used and revisited to keep it alive. Writing software is always about adding new cards to an existing house of cards – assuming I can remember what those cards were before. It’s all virtual, and it tends to fade and become stale if not actively managed.

Hardware, on the other hand, lives in a world which exists even when you don’t explore it. Each time I sit down at my electronics bench, I think “hm, what aspect of the real world shall I dive into this time?”.

I love ‘em both, even though working on software feels totally different from working on hardware.

I think that the fact that things are nevertheless workable is mostly because the “plug and play” side of things still tends to work – for most people and in most cases, anyway. You assemble the kits, solder the header, hook things up, plug it into USB, get the latest code, upload an example sketch, and yippie… success!

But many things can and do go wrong – electrically (soldering / breadboarding mistakes), mechanically (bad connections), and especially on the software side of things. Software on the host, but most often the problems are about the software “sketch” running on the JeeNode. You upload and nothing happens, or weird results come out.

Ok, so it doesn’t work. Now what?

There’s a chasm, and sooner or later everyone will have to cross it. That’s when you switch from following steps described on some web page or in some PDF document, to taking charge and making things do what you want, as opposed to replicating a pre-existing system.

To be honest, following instructions is boring – unless they describe steps which are new to you. Soldering for the first time, heck even just connecting something for the first time can be an exhilarating experience. Because it lets you explore new grounds. And because it lets you grow!

As far as I’m concerned, JeeLabs is all about personal growth. Yours, mine, anyone’s, anywhere. Within a very specific domain (Physical Computing), but still as a very broad goal. The somewhat worn-out phrase applies more than ever here: it’s better to teach someone how to fish (which can feed them for a lifetime) than to give them a fish (which only feeds them for a day).

IMO, this should also drive how documentation is set up: to get you going (quick start instructions) and to keep you going, hopefully forever (reference material and pointers to other relevant information). A small part of the documentation has to be about getting a first success experience (“don’t ask why, just do it!”), but the main focus should be on opening up the doors to infinitely many options and adventures. Concise and precise knowledge. Easy to find, to the point, and up to date.

Unfortunately, that’s where things start to become complicated.

I’m a fast reader. I tend to devour books (well, “skimming” is probably a more accurate description). But I don’t really think that thick books are what we need. Sure, they are convenient to cover a large field from A to Z. But they are reducing our options, and discourage creative patterns – What if I try X? What if I combine Y and Z? What if I don’t want to go a certain way, or don’t have exactly the right parts for that?

This weblog on the other hand, is mostly a stream-of-conscience – describing my adventures as I hop from one topic to the next, occasionally sticking to it for a while, and at times diving in to really try and push the envelope. But while it may be entertaining to follow along, that approach has led to over 1000 articles which are quite awkward as documentation – neither “getting started” nor “finding reference details” is very convenient. Worse still, older weblog posts are bound to be obsolete or even plain wrong by now – since a weblog is not (and should not be) about going back and changing them after publication.

I’ve been pondering for some time now about how to improve the documentation side of things. There is so much information out there, and there is so much JeeLabs-specific stuff to write about.

Write a book? Nah, too static, as I’ve tried to explain above.

Write an eBook? How would you track changes if it gets updated regularly? Re-read it all?

A website? That’s what I’ve been doing with the Café, which is really a wiki. While it has sections about software and hardware, I still find it quite tedious (and sluggish) for frequent use.

I’ve been wanting to invest a serious amount of time into a good approach, but unfortunately, that means deciding on such an approach first, and then putting in the blood, sweat, and tears.

My hunch is that a proper solution is not so far away. The weblog can remain the avant garde of what’s going on at JeeLabs, including announcing new stuff happening on the documentation side of things. Some form of web-based system may well be suited for all documentation and reference material. And the forum is excellent in its present role of asking around and being pointed to various resources.

Note that “reference material” is not just about text and images. There is so much information out there that pointers to other web pages are at least as important. Especially if the links are combined with a bit of info so you can decide whether to follow a link before being forced to surf around like a madman.

The trick is to decide on the right system for a live and growing knowledge base. The web is perfect for up-to-date info, and if there’s a way to generate decent PDFs from (parts of) it, then you can still take it off-line and read it all from A to Z on the couch. All I want, is a system which is effective – over a period of several years, preferably. I’m willing to invest quite a bit of energy in this. I love writing, after all.

Suggestions would be welcome – especially with examples of how other sites are doing this successfully.

The “Blue Mosque”, and lots more fascinating / touristy things. A humbling experience for a Westerner like me.

With apologies for not responding immediately to all emails – I’ll catch up on this in the next few days.

Onwards!

It all started on October 25th in 2008, with a weblog post about – quite appropriately – the Arduino.

Then it took a few more months to evolve into a daily habit, and yet another few months to set up a shop, but apart from that it has all remained more or less the same ever since.

You might have been following this from the start, and you might even have been going through the long list of daily posts later, but there you have it – a personal account of my adventures in the world of Physical Computing. If anything, these years have been the source of immense inspiration and delight. I’ve been able to re-connect to my inner geek, or rather: my inner ever-curious and joyful child. And to so many like-minded souls – thank you.

“Standing on the shoulder of giants” is a bit over-used as a phrase, but it really does apply when it comes to technology and engineering. What we can do today is only possible because many generations of tinkerers, inventors, and researchers before us have created the foundations and the tools on which we can build today. It feels silly even to try and list them – such a list would be virtually endless.

I’m not a technocrat. I think our IT world has done its share to rob people of numerous meaningful and competence-building jobs, and to introduce new mind-numbing and RSI-inducing repetitive tasks. Our (Western) societies have become de-humanized as more and more screens take over in the most unexpected workplaces, and our car trips and train rides are turning us into very selectively-social beings, reserving our emotions but even our respect and courtesy for our families and the people we choose as our friends. Technology’s impact on daily life is a pretty horrible mess, if you ask me.

But what drives me, are the passion and the creativity and the excitement in the field of technology. Not for the sake of technology, but because that’s one of the major domains where cognition and rationality have free reign. You can learn (and reason) all about history, medicine, psychology, or you can invent (and reason about) things which do new things, be it electrical, mechanical, biological, informational, or otherwise. Technology as a source of boundless evolution and innovation is breath-taking, we “merely” have to tap it and put it to good use.

And what thrills me most is not what I can do in that direction, but what others have done in the past and are still doing every day. Learning about all that existing technology around us is like looking into the minds of the persons who came up with all that stuff, feeling their struggles, their puzzles, and ultimately the solutions they came up with. I’m in awe of all the cleverness that has emerged before us, and even more in awe of the thought that this will no doubt go on forever.

It’s really all about nurturing curiosity, asking questions, and solving the puzzles they bring to the surface:

I have no special talents. I am only passionately curious. — Albert Einstein

Here’s the good news: we all have that ability. We all came into the world the same way. We can all be explorers.

If you start doing this early on in life and hold onto it, you’ll never be hungry and you’ll never get bored. And if you didn’t have that opportunity back then: nothing of substance prevents you from starting today!

We live in amazing times. Ubiquitous internet and access to knowledge. Open source Physical Computing. Online communities with a common language. This weblog is simply my way of reciprocating all these incredible gifts.

First there was the 7th HackersNL meeting in Utrecht. The name of the event is unfortunate, IMO (this whole “hacker” monicker doesn’t sit well with normal people, i.e. 99.9% of humanity), but the presentations were both absolutely fantastic. A wide scale of design topics by David Menting, including his “linear clock” for which he designed custom hardware based on a standard tiny Linux + WiFi board, and then a talk about turning a cheap laser cutter into a pretty amazing unit by ripping out the driver board and software, and replacing it with their own custom hardware with an MBED module plus software (wiki) – by Jaap Vermaas and Peter Brier. Both cutting edge, if you pardon the pun, and above all a pressure cooker where two dozen people get to talk about “stuff”, mostly related to Physical Computing really. Everything is open source.

If you live in the neighborhood of Utrecht, I can highly recommend this recurring meeting, scheduled for the last Thursday of each month – so take note, hope to see you there, one day!

The other event was the Air Quality Egg Workshop, by Joe Saavedra. Basic idea: a sensor unit, to measure air quality in some way, plus an “egg” base station which can tie into Pachube (both ways), relays the sensor data, and includes an RGB color light plus push-button.

Except that it doesn’t exist yet. We built a wired prototype based on a Nanode with SparkFun protoshield, a CO sensor, an NO2 sensor, and a DHT22 temperature/humidity sensor.

Here’s my concoction (three of the sensors were mounted away from the heat generated by the Nanode):

It’s now sitting next to the JeeLabs server, feeding Pachube periodically. We’ll see how it goes, since apparently these sensors need 24..48 hours to stabilize. Here are some of the readings so far:

What I took away from this, is:

1. Whee, there sure is a lot more fun stuff waiting to be explored!
2. When you put a fantastic bunch of creative people together, you get magic!
3. Not enough time! Would it help to keep flying westwards to cram more hours into a day?

The above paragraph is a mix of reality and fiction, BTW. Because I’m talking about two things at once – the Mac I work on, and… my brain. Both have changed :)

The past month has been extremely chaotic for me. I’ve been trying to figure out what I really want to do, and how to make it happen. The outcome surprised me: I absolutely want to keep doing what I’ve been doing these past few years, with JeeLabs. So the good news, if you been following along, is that I will. But there will be changes, because the intensity of it all is not sustainable for me, not at the previous energy level anyway. I will spread out stories over more weblog posts – thus also making it easier for you to keep up and follow along.

In this day and age of instant gratification, mass consumption, and immediate mail-order fulfillment, I’m going to go against the grain and buck the trend – by reducing short term the frequency of JeeLabs shop fulfillments, dealing with shop-related tasks less often. The shop will become even more of a secondary activity here, but fulfillment improvements are in the pipeline. The product range will grow further, but the pace and scale of commerce most likely not. It gives me pleasure to send out packages and to stay in contact with the people who are going to use these products. The shop isn’t about volume and turnover, but about allowing others to reproduce and extend some projects I’m coming up with and working on. Because making stuff is fun.

My passion, my energy, and my time will remain focused on the weblog, or rather on the projects that drive it all. Whether the frequency can stay as is, time will tell. I hope it can – with occasional breaks in the year – because the daily cycle is great fun, keeps me focused, and is clearly being appreciated.

As Seth Godin describes in his manifesto, the schooling system has taken our dreams away. I’ve been lucky to keep (or rather, rediscover) mine, and want to help as much as I can to make sure others will be able to latch onto their dreams as well, with curiosity and creativity as the driving forces – in the context of Physical Computing, that is.

The internet, at least the part I care about, is evolving into an extra-ordinary global learning powerhouse. It started with Wikipedia and led to the inspiring TED presentations, MIT’s Open Courseware, and the Khan Academy (an absolutely astounding initiative which is turning the way education works on its head). There is no excuse anymore for not knowing what you’d like to know, it’s all there.

And as I’m finding out, there is no excuse anymore for not sharing what you know, either.

Onwards!

It’s a bit radical and unexpected, but there is no way around it. This weblog is “driven by passion”, as you will probably know, and the crazy bit is that there’s just too much going on here to keep things going smoothly. I’ve been running behind on shop fulfillment again, and I’ve been running behind even more on answering emails and with helping out on the forum. First thing I hope this will do, is to let me catch up and regain my footing.

In sharp contrast to last year’s emergency stop, this time it’s not so much lack of ideas or lack of energy, but lack of clear focus and direction. The stories I would love to tell need more time – diving into various aspects of physical computing in considerably more depth and detail than what’s been happening on the weblog lately. And it’s not happening because the daily bite-sized cycle is chopping up my attention (even at times when I have enough weblog posts queued up for many days on end – go figure!). And maybe it’s also a hill climbing issue.

For an interesting insight about attention, see Paul Graham’s essay titled Maker’s Schedule, Manager’s Schedule.

I’ve updated the alphabetical and chronological indexes to all the posts on this weblog, to give you something to go through for the coming weeks. It’s a stopgap measure, but it’ll just have to do – and there should be enough to keep you interested and hopefully also pique your interest and keep you excited in the month ahead.

The difference with last year, is that I’m putting a precise cap on the duration of this “outage”: 30 days from now. That’s when this weblog will resume, probably with some announcements and adjustments to its style and format.

Talk to you one month from now!

PS. If you want to learn about electricity, then there are numerous resources on the web. Let me single out one: a 50-minute video by Walter Lewin at MIT about batteries and power (lecture 10 on this page). You can get a deep understanding of what a battery is, why its internal resistance matters, what power is, how heat comes out, what shorting a battery does, and even sparks. It’s a fantastic presentation, and the video was just picked at random!

The benefit of doing everything yourself, is that you can make things work exactly as you want them.

The drawback of doing everything yourself, is that you have to do everything yourself…

Having become pretty independent in my work areas, my hobbies, and my income streams over the years, I know all about those trade-offs. Or at least I think I know about most aspects of this DIY-vs-outsourcing range.

It’s a bit like trying to stay on your feet with a floor covered with marbles…

Example: I used to rent a web server (a real physical one, with full root access and Linux on it). No worries about hardware outages or connectivity details. Being housed at an ISP with thousands of servers, means they’ll have round-the-clock watchdogs and support staff, and will jump into action the minute something is seriously wrong.

At the same time, I had total control over the web server software and operating system configuration. With a Linux distribution such as Debian, maintenance was delightfully simple (“apt-get update && apt-get upgrade”).

The flip side is that I had to choose and configure a web server (“lighty” / lighttpd at the time), and technologies to create dynamic database-driven websites (I built my own back then, based on Metakit – my own database).

Did it work? Sure. Did it evolve? Nope. Too busy. Didn’t want to risk breaking anything.

Only thing that setup did was track security updates (automatically). I had two break-ins over the 10 years that this went on. Learned more about rootkits than I care about (they’re evolving to amazingly sophisticated levels).

Did I learn a lot? You bet. And some of that knowledge is priceless and timeless. Big, big benefit.

But I also had to learn lots of stuff I really care very little about. For me, network routing, package installation dependencies, mail server configuration, and lighttpd configuration were a waste of time. The latter because lighttpd wasn’t really kept up to date very actively by its developer(s). Other options became more practical, meaning that all that lighttpd-specific knowledge is now useless to me.

The story is repeating itself right now. Redmine, which I use on http://jeelabs.net/ is not up to date, because I haven’t found a simple upgrade path. The difference is that it’s not just me not updating my stuff, I now have the same stagnant state with stuff from others. So what’s the point of Redmine? As far as I’m concerned, it’s a dead end (luckily, everything in there is stored in Markdown format – a solid long-term standard which I also use for the forum and the weblog).

With the forum, running on Drupal, it’s different again. Module updates are automated more or less, so I tend to track them from time to time. But Drupal itself is a little harder to update. And sure enough, it’s falling behind… With Drupal, I’m also running into not being knowledgeable enough to put it to really good use.

But the reason for writing this post is a different one – see the message at the top.

For the web shop, I use the Shopify web store service. They have the servers (at Rackspace – very good ops, I’ve used them for a couple of years). And Shopify develop and run the web store software (using Ruby on Rails).

They take care of dealing with nasty things such as possible DoS attacks, heavy data security, financial gateway interfaces – lots of important issues I no longer need to worry about. So far so good.

But they have their own agenda:

• some things don’t change, and that’s good: it works, the shop is operational
• some things don’t change, but that’s bad: years have gone by, and they still haven’t got a clue about VAT
• some things change, and that’s good: improvements to the service, new features for customers
• some things change, but that’s bad: they change their API and their XML data structures

That last one is what bites me now. I created a little scripted setup whereby I always pull information about orders from their shop database, to fill my database here with all the details, so I can generate paper invoices, and do the fulfillment of orders here. Doing this here was necessary to be able to do the Value Added Tax thing properly, as required by law and as my accountant wants it, of course.

So to summarize, the choices are:

1. do everything yourself (and pay in time)
2. outsource everything (and pay in money)
3. choose a mix (and deal with the interface changes)

Everything is a trade-off, of course. In my case, I’m moving more and more to #1 as far as operational choices are concerned (own server, own fiber connection), and #2 as far as day-to-day software use is concerned (solid, but actively developed open source software, and Apple hardware + Mac OSX for my main workplaces). These choices are optimal for me, in terms of cost and stability.

The choice to host my own servers was made a lot simpler because I’m running VM’s for the different sites, built from ready-to-run images from TurnKey Linux. What makes them (and others, like Bitnami) different, is that all VMs are automatically backed up to the cloud (Amazon S3 in my case). The way TKL does this is really clever, reducing the amount of data in incremental backups, even for all the records stored in MySQL. So not only are my VM’s pre-configured and run out of the box, they automatically self-update and they automatically self-backup – if anything goes completely wrong, I can switch to cloud-based instances and be up and running again in no time.

TurnKey Linux is an example of using third-party stuff to side-step (and in fact avoid) a massive amount of effort, while retaining maximum operational flexibiity. My Amazon S3 bill is a whopping $1.01 per month…

But the web shop setup at Shopify is far from optimal. It was supposed to be choice #2, but ended up being #3 due to the mismatch between what I need (a European shop with correct VAT handling) and what they offer (flashy stuff, aimed at the masses). In hindsight, it was a bad choice, but I really don’t want to do this myself.

Oh well, I’ll suffer the consequences – will fix my scripts and get everything going again by next Monday!

PS. My little presentation yesterday at HackersNL #5 can be found here (PDF) – for those who read Dutch.

This is a German site which supports direct bank-to-bank transfers. Looks like it’s working in 5 countries:

• Austria
• Belgium
• Germany
• Netherlands
• Switzerland

I can’t find a trace of UK or Italy in the setup, even though it’s mentioned on their web site. My impression is that this service is still very young – the “Payment Network AG” company behind this was registered last October. But the good news is that their support is responsive and effective, by email as well as by phone.

One benefit for customers is speed: I get immediate notification, avoiding the usual 1..3 day delay normally involved with bank transfers. The other benefit is convenience, since you can complete the payment as part of the order, instead of having to switch to your online bank account and manually copy all the relevant info.

The benefit for me is lower cost: a third of what PayPal charges (it does add up: VAT, payment/bank/shop fees).

The thing with this sort of service, is that it’s very hard for me to get an impression of how well it works in practice. I did a “test payment” while setting things up, but that’s a weak approximation of the whole process when using it for real, and I can only do a more realistic test with my own country and my own bank account.

So if you ever feel an uncontrollable urge to order something from the JeeLabs web shop (yeah, I know, it’s unlikely) and live in one of the above-mentioned countries, then please feel free to give this a go:

The name is in German (Sofort Überweisung), but the page will be in English by default (all pages are available in multiple languages by clicking on the flag – top right).

Please feel free to email me with anything odd (or neat) which comes up, especially if it doesn’t work as expected of course. I can easily cancel an entire transaction if things get really out of hand.

But with a little luck, life will simply have become one notch simpler with this new option – for everyone!

Reading has changed a lot these past few decades. I used to devour books in the library and subscribe to lots of magazines. As a kid, when visiting New York one summer, I spent weeks on the floors of the the New York Public Library – because they had all the back issues of Scientific American and you could read as much as you want!

The thing with SciAm, is that it had a column every month, called The Amateur Scientist – which, in hindsight, was really the ultimate “maker” breeding ground. I don’t think I ever built anything described in it (’cause teenagers don’t have any money), but that did not diminish the fun and learning experience one bit.

A side-effect of all this was that my environment filled itself with books, papers, magazines, and articles.

And although the human mind is incredibly good at remembering where things are, by association, and particularly by how it looks and its location, there comes a point when ya’ can’t find that one friggin’ article back. With computers, things quickly got (much!) worse… no more clues as to which book (file!) is large, which one looks worn-out, what the books (files!) around it look like, or to leaf through it quickly to locate a section (bits!) by its visual appearance.

Besides, most magazines and books are really just meant to be read once. You digest the info, learn from it, and never look back. It seems silly to buy them in dead-tree form, and continuously add more bookshelves for them.

So I started to get more and more books, articles, and magazines in PDF form. They were easy to store, could be browsed as well as searched via keywords. I bought – and still buy – lots of books that way. My favorite PDF shop (for programming-related books) is probably the Pragmatic Programmers – nice collection, well-written and good-looking books, and you get update notifications when books get revised (a key benefit of the electronic format).

My collection of PDFs is growing fast. Purchased as well as downloaded. And now also lots of electronics datasheets.

This reached a point where I decided that I wanted to get rid of the paper stuff, at least for normal technical books to which I have no particular emotional attachment. So I got one of these a couple of years ago:

That’s a Fuijistu ScanSnap S510M document scanner. There are newer models now, for Mac and PC. The thing about this scanner is that it’s surprisingly effective. It scans quickly, and does both sides of the page at the same time. But the real gem is the supporting software. It knows what’s color and what’s black and white, it knows what’s up and what’s down, it knows what’s portrait and what’s landscape, and it it knows how to start up the software when you press the big button on the front. Best of all, it comes with OCR software which places the recognized text inside the PDF, and puts it there invisibly – behind the scanned images, so to speak. That sounds crazy, but the result is that the pages you look at are complete photographic reproductions, and yet the document is fully searchable!

To be honest, the OCR process is so time-consuming that I don’t enable it for books & magazines. But for invoices and loose sheets of paper, this is incredibly useful. I do not need to organize it – text search does it all!

I’ve cut up some 10 meters of books already, and turned them into PDFs. Yeah, it hurts a little at first, but hey.

For reading PDFs, I use the Mac’s built-in Preview, which is a lot better (and faster) than Adobe’s, eh… junk.

For locating documents, by file name or by content, there is Spotlight in the Mac, which also works with a server. This search technology is fast enough to instantly locate documents in many dozens of gigabytes of data. And since it’s available to all applications, there are some great front ends for it such as Yep, Leap, and Papers. I’ve been using DEVONthink Pro Office for all my docs and notes, because of its integration with the ScanSnap.

The above is all for the Mac, but there are probably similar offerings for Windows.

But the real revolution is much more recent…

There’s an “app” for the iPad, called GoodReader. This little bit of software lets me put over a thousand documents on the iPad and actually be able to find stuff, read stuff, and manage stuff. About 25 GB so far. Offline.

Which means I can now manage my entire collection as a folder on the server, add books, reorganize as needed, add tags and quickly access it from multiple Macs through Yep, as well as have the entire set on an iPad.

The Ultimate Bookshelf, no less, if you ask me. Alan Kay’s DynaBook has become an affordable reality.

To put it differently: food for thought – especially slow food for slow (off-line) thought, as far as I’m concerned!