Computing stuff tied to the physical world

Archive for the ‘Linux’ Category

Flashback – Dive Into JeeNodes

In AVR, Hardware, Linux, Software on Oct 4, 2013 at 00:01

Dive Into JeeNodes (DIJN) is a twelve-part series, describing how to turn one or more remote JeeNodes, a central JeeLink, and a Raspberry Pi into a complete home monitoring setup. Well, ok, not quite: only a first remote setup is described with an LDR as light sensor, but all the steps to make the pieces work together are described.

More visually, DIJN describes how to get from here:

dijn01-essence.png   dijn01-diagram

.. to here:

Screen-Shot-2013-02-09-at-12.22.13

This covers a huge range of technologies, from embedded Arduino stuff on an ATmega-based JeeNode, to setting up Node.js and the HouseMon software on a Raspberry Pi embedded Linux board. The total cost of a complete but minimal setup should be around €100. Less than an Xbox and far, far more educational and entertaining, if you ask me!

It’s all about two things really: 1) describing the whole range of technologies and getting things working, and 2) setting up a context which you can explore, learn, and hack on tinker with in numerous ways.

If you’re an experienced Linux developer but want to learn about embedded hardware, wireless sensors, physical computing and such, then this offers a way to hook up all sorts of things on the JeeNode / Arduino side of things.

If you’re familiar with hardware development or have some experience with the Arduino world, then this same setup lets you get familiar with setting up a self-contained low-power Linux server and try out the command line, and many shell commands and programming languages available on Linux.

If you’ve set up a home automation system for yourself in the past, with PHP as web server and MySQL as back end, then this same setup will give you an opportunity to try out rich client-side internet application development based on AngularJS and Node.js – or perhaps simply hook things together so you can take advantage of both approaches.

With the Dive Into JeeNode series, I wanted to single out a specific range of technologies as an example of what can be accomplished today with open source hardware and software, while still covering a huge range of the technology spectrum – from C/C++ running on a chip to fairly advanced client / server programming using JavaScript, HTML, and CSS (or actually: dialects of these, called CoffeeScript, Jade, and Stylus, respectively).

Note that this is all meant to be altered and ripped apart – it’s only a starting point!

FanBot Registration System

In Linux, Software on Jun 29, 2013 at 00:01

Nearly 400 FanBots have already been built – see this page and the KekBot weblog.

To finish this little series, I wanted to describe the little setup I built and how it was done. Nothing spectacular, more an example of how you can create a software cocktail / mashup to get a one-off short-lived project like this going fast.

First the basic task at hand: allow kids to register their FanBot and print out a “passport” with a picture of their creation and some information about when and how to get their FanBot back after the RoboCup 2013 event. At the same time, it would be nice to have a monitor display the current count and show images of the last few FanBot pictures taken.

passport

I decided to use 3 Raspberry Pi’s as “slave”, each with a webcam, an LCD screen (thanks David, for the laser-cut frame!), and Ethernet + power. The code running on them is on github. It would be nice if each slave could continue to accept registrations even with the network connection temporarily down, so I went for the following approach:

  • collect registrations locally, using rsync to push new ones to the master
  • everything is based on a few simple shell scripts for quick adjustment
  • two C/C++ programs were needed: to read the FanBot data and to drive the LCD
  • do a git pull on power-up and re-compile as needed before starting operation
  • periodic webcam readout and periodic rsync, using loops and sleep calls

For the master, I thought about using the Odroid/U2, but decided against it as there would be too much setup involved in the last minute. Next idea was to use an Acer Inspire One Linux netbook, but it turned out to be way too slow. For some reason, Node.js was consuming 100% CPU (I suspect a bad nodemon disk change polling error on the slow SSD). No time to figure that out, so I ended up using my second Mac laptop instead.

The master logic started out as follows:

  • an rsync server was set up to collect files from all the slaves in one spot
  • the display is presented via a browser in full-screen mode and Node.js
  • the original plan was to also allow outside access, but we never got to that
  • for live updating, I decided to use SocketStream (same as in HouseMon)
  • a PDF was generated for each image by convert, which is part of ImageMagick
  • an AppleScript then prints each PDF to the USB-attached laser-printer

These choices turned out to be very convenient: by having the rsync folder inside the Node.js application area, and using nodemon, refresh became automatic (nodemon + socketstream will force a refresh on each client whenever a file system change is detected). So all I had to do was write a static Node.js app to display the count and the latest images.

So far so good. A couple of glitches, such as an erratic printer (new firmware fixed it), an overheating RPi (fixed by clocking at 700 MHz again), and all the slave rsyncs fighting when the same registration was present on more than one (fixed by adding the “-u” flag).

During installation, we found out that there was no wired ethernet w/ DHCP, as I had assumed. Oops. So the Mac laptop was turned into a WiFi -> wired bridge, since WiFi was available most of the time. Whoops: “most” is not good enough, the RPi’s have no clock, they really need an NTP server – especially since the whole system was based on rsync and file modification times! We hacked a fixed DNS – but internet still has to be there.

And then the fun started – various little wishes came up after the fact:

  • can we get images + passports off-site during the event? sure: rsync them to Dropbox
  • could we have a page with all images? sure: on my server at JeeLabs, I made an rsync loop from one Dropbox area to another, and made a little photo-album using Dropbox
  • could we show a live count, even though the monitor website was not accessible? sure: I simply added another shell loop on my server again, to generate an image with the count in it, and made it show up as first one in the photo album.

Here is the shell script for that last feature:

cd ~/Dropbox/FanBots
while :
do
  N=$(echo `ls *.jpg | wc -l`)
  if [ "$N" != "$P" ]
  then
    echo $N - `date`
    convert -background black -fill red -size 480x640 \
      -gravity center "label:$N\n\nFanBots" 0-teller.png
    P=$N
  fi
  sleep 10
done

This can run anywhere, since Dropbox is taking care of all updates in both directions!

The point of all this is not which languages I used (your choice will definitely be superior), nor what hardware was selected for this (it changed several times). The point is that:

  1. it all got done in a very limited amount of time
  2. the basic rsync approach made things resilient and self-recovering
  3. github pulls made it possible to apply last-minute tweaks
  4. some nice changes were possible very late in the game
  5. dropbox integration made it possible to add features remotely

Happy ending: all the kids are having a good time … including me :)

PS. Tomorrow (Sunday) is the RoboCup 2013 finals. See you there?

Packaged but not virtualised

In Hardware, Linux, Software on Jun 26, 2013 at 00:01

Since VMware started virtualising x86 hardware many years ago, many uses have been found for this approach of creating fully self-contained software environments. I’ve used VMware many years, and was pleased when Apple switched the Macs to “Intel inside”.

Nowadays, I can run numerous variants of Windows as well as Linux on a single laptop. I switched to Parallels as VM core, but the principle remains the same: a large file is used as virtual hard disk, with an operating system installed, and from then on the whole environment simply “runs” inside the Mac host (in my case).

Nowadays, we all take this for granted, and use virtual environments as ultra-low cost web servers without even giving it any thought. I have a number of virtual machines running on a Mac Mini for all the JeeLabs sites in fact (and some more). One piece of hardware, able to suspend and resume N completely independent systems at the click of a mouse.

But there’s a drawback: you end up duplicating a lot of the operating system and main services in each VM, and as a result this stuff not only consumes disk space but also leads to a lot of redundant backup data. I keep the VM’s very lean and mean here, but they still all eat up between 3 and 10 gigbytes of disk space. Doesn’t sound like much, but I’m pretty sure my own data uses less than 10% of that space…

Both Parallels and VMware (and probably also VirtualBox) support snapshotting, which means that in principle you could set up a full Linux environment with a LAMP stack, and then use snapshots to only store what’s different for each VM. But that’s not useful in the long run, because as soon as you get system updates, they’ll end up being duplicated again.

It’s a bit like git “forks”: set up a state, create a fork to create a variant, and then someone decides to add some features (or fix some bugs) to the original which you’d also like to adopt. For git, the solution is to merge the original changes into your fork, and use “fast-forwarding” (if I understand it correctly – it’s all still pretty confusing stuff).

So all in all, I think VM’s are really brilliant… but not always convenient. Before you know it, you end up maintaining and upgrading N virtual machines every once in a while.

I’ve been using TurnkeyLinux as basis of all my VM’s because they not only maintain a terrific set of ready-to-run VM images, but because they also make them self-updating out of the box, and because they’ve added a very convenient incremental daily backup mechanism which supports restarting from the back-up. You can even restart in the cloud, using Amazon’s EC2 VM “instances”.

And then someone recently told me about Docker – “The Linux container engine”.

This is a very interesting new development. Instead of full virtualisation, it uses “chroot” and overlay file systems to completely isolate code from the rest of the 64-bit Linux O/S it is running on. This lets you create a “box” which keeps every change inside.

My view on this, from what I understand so far, is that Docker lets you play git at the operating system level: you take a system state, make changes to it, and then commit those changes. Not once, but repeatedly, and nested to any degree.

The key is the “making changes” bit: you could take a standard Linux setup as starting point (Docker provides several base boxes), install the LAMP stack (if someone hasn’t done this already), commit it, and then launch that “box” as web server. Whereby launching could include copying some custom settings in, running a few commands, and then starting all the services.

The way to talk to these Docker boxes is via TCP/IP connections. There’s also a mechanism to hook into stdin/stdout, so that you can see logs and such, or talk to an interactive shell inside the box, for example.

The difference with a VM, is that these boxes are very lightweight. You can start and stop them as quickly as any other process. Lots of ‘em could be running at the same time, all within a single Linux O/S environment (which may well be a VM, by the way…).

Another example would be to set up a cross compiler for AVR’s/Arduino’s (or ARMs) as a box in Docker. Then a compile would be: start the box, feed it the source files, get the results out, and then throw the box away. Sounds a bit drastic, but it’s really the same as quitting an application – which throws all the state in RAM away. The point is that now you’re also throwing away all the uninteresting side-effects accumulated inside the box!

I haven’t tried out much so far. Docker is very new, and I need to wrap my mind around it to really understand the implications, but my hunch is that it’s going to be a game changer. With the same impact on computation and deployment as VMware had at the time.

As with VM’s, Docker boxes can be saved, shared, moved around, and launched on different machines and even different environments.

For more info, see this page which draws an analogy with freight containers, and how one standard container design has revolutionised the world of shipping and freight / cargo handling. I have this feeling that the analogy is a good one… a fascinating concept!

Update – Here’s a nice example of running a Node.js app in Docker.

FanBots at RoboCup 2013

In ARM, Hardware, Linux, Software on Jun 25, 2013 at 00:01

There’s an event coming up in a few days in Eindhoven, called the RoboCup 2013:

teun

This is the world championship of robotic football. For all ages, and with various levels of sophistication. Some robots are tiny cars, trying to work together to play a match, some robots are little human-like machines consisting of lots of servos to make them walk on two feet, and some robots are very sophisticated little “towers” on three wheels, zipping along at impressive speeds and able to catch, hold, and kick a standard-size football.

I’m not directly involved in this, but I saw some of the preparations and it really promises to be a fantastic event. Lots of geeks with laptops, soldering irons, and mechanical stuff. The kick-off is next Wednesday and the event lasts for five days, with the finals on Sunday.

On the side, there is a gorgeous (not-so) little project going on, whereby kids of ages 8 to 12 can visit a workshop to build their own FanBot (for free). The FanBot can blink its eyes and mouth, and wave its arms – part of the exercise is to create a little “program” of things it does at the press of a button, so this will also be a first encounter with programming!

There’s a Dutch page about the project and a general page in English mentioning FanBots.

But the really exciting goal will be to create a huge stand for all the FanBots and make them cheer the football-playing robots in the main event hall. The aim is to get up to 1000 of them, all blinking and waving – and all orchestrated and controlled from a central system. It’s all being designed and built by Peter Brier and by Marieke Peelen of KekkeTek.

One task in the project is to register all the FanBots and print out a little “passport” as last step – which the kids can then use to pick up their own FanBot once the RoboCup 2013 event is over. This is the part I volunteered to help with.

I’ll describe the setup in more detail soon, but that’s where the Raspberry Pi’s w/ LCD screen and webcams will be used. It’ll be fun to see whether we can get 1000 kids to build and sign up their little robots during three days – there’s quite a bit of logistics involved!

Admission is free, so if you’re in the neighbourhood: don’t miss it – it’s going to be fun!

Using eMMC as root disk

In Hardware, Linux on Jun 24, 2013 at 00:01

The Odroid/U2 mentioned yesterday also has the option to add an eMMC card, right next to the µSD card slot – I went for a whopping 64 GB “SSD” in this case:

DSC_4489

The reason to use this, is that it’s indeed nearly three times as fast:

# hdparm -tT /dev/mmcblk0

/dev/mmcblk0:
 Timing cached reads:   1604 MB in  2.00 seconds = 802.41 MB/sec
 Timing buffered disk reads: 146 MB in  3.02 seconds =  48.40 MB/sec

That’s faster than the maximum transfer rate of USB 2.0, by the way.

But how do you set these things up? The bottom view shows yet another connector, yuck:

DSC_4490

It turns out that you can get an adapter from the Odroid/U2 shop:

DSC_4491

That turns it into a µSD card, but unfortunately it wasn’t recognised in my Mac laptop. Luckily, this concoction did seem to do the trick:

DSC_4492

Through this, I was able to burn the Debian disk image onto it, and through some clever logic in the bootstrap loader, the eMMC card simply takes precedence when present and starts up fine with the same device name as the initial µSD card (which now ends up on /dev/mmcblk1 if also present).

One last step is needed to resize the root partition to take full advantage of the entire eMMC card. Google found this article which explains the whole (nasty) procedure.

Final result:

# free
             total       used       free     shared    buffers     cached
Mem:       2031632      77660    1953972          0       9540      37420
-/+ buffers/cache:      30700    2000932
Swap:            0          0          0

And… tada!

# df -H
Filesystem      Size  Used Avail Use% Mounted on
rootfs           62G  1.2G   58G   2% /
udev            1.1G     0  1.1G   0% /dev
tmpfs           209M  222k  208M   1% /run
/dev/mmcblk0p2   62G  1.2G   58G   2% /
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs           417M     0  417M   0% /run/shm
/dev/mmcblk0p1  133M  8.5M  124M   7% /boot

Neat, neat, neat – although the total system cost is close to $200 at this point, due to the expensive eMMC memory card – five times the cost of a Raspberry Pi, but with four times the RAM, a comfy 60 GB of fast SSD storage, and probably 5..10 times the performance.

It’s not for everyone and it has no GPIO pins to hook anything up, but I like it!

More about the Odroid/U2

In ARM, Hardware, Linux on Jun 23, 2013 at 00:01

As mentioned recently, there are many alternatives to the Raspberry Pi. I’m looking mostly at ARM-based systems these days, because Node.js support for MIPS is not really there (yet?), and you really want a JIT-type system to get the most out of a Node.js based setup.

Well, having received the Odroid/U2, first thing I had to do is get a micro-HDMI to HDMI adapter (always some new connector/cable type, yawn…). Also had to get a 16 GB µSD card, because the Debian image I wanted to try was for cards of 8 GB or more (yawn!).

But now that everything is in, I’m finally able to take that little black cube for a spin:

article_img

First impression: it’s fast, much snappier than a Raspberry Pi. Try launching aptitude, for example – which takes a long time on RPi’s and other low-end ARM & MIPS systems.

Both are running class-10 SD cards, as far as I can tell. The RPi is running at 700 MHz:

# cat /proc/cpuinfo 
Processor   : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS    : 697.95
Features    : swp half thumb fastmult vfp edsp java tls 
[...]
Hardware    : BCM2708

The OU2 has four cores running at 1.7 GHz:

# cat /proc/cpuinfo 
Processor   : ARMv7 Processor rev 0 (v7l)
processor   : 0
BogoMIPS    : 1992.29
processor   : 1
BogoMIPS    : 1992.29
processor   : 2
BogoMIPS    : 1992.29
processor   : 3
BogoMIPS    : 1992.29
Features    : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls 
[...]
Hardware    : ODROIDU2

Disk speeds are interesting, first RPi then OU2:

# hdparm -tT /dev/mmcblk0
/dev/mmcblk0:
 Timing cached reads:   340 MB in  2.00 seconds = 169.91 MB/sec
 Timing buffered disk reads:  62 MB in  3.00 seconds =  20.66 MB/sec

# hdparm -tT /dev/mmcblk0
/dev/mmcblk0:
 Timing cached reads:   1616 MB in  2.00 seconds = 808.23 MB/sec
 Timing buffered disk reads:  52 MB in  3.05 seconds =  17.03 MB/sec

I haven’t set up the eMMC card yet, which is said to be several times faster than µSD.

Power consumption of the OU2 is 2.9 W on the AC mains side of a little 5V adapter I’m using. Not bad. I’m tempted to try and set this up as my only always-on server here at JeeLabs, including all the web server stuff even. There’s 2 GB of RAM in this thing, should be enough to run some serious stuff. Perhaps even the (Java-based) CrashPlan backup system I’ve been using for almost a year now.

I don’t really care that much about file storage – a 64 GB eMMC disk would be plenty for everything that needs to be online here at all times, perhaps with an external 2.5″ USB hard disk for secondary / archival backup.

Oscilloscope sampling rate

In Hardware, Linux on Jun 16, 2013 at 00:01

Just to finish the series on Raspberry Pi I/O – here’s proof of its GPIO pulsing speed:

SCR57

The trouble is the irregularity caused by Linux’s multi-tasking properties, and I wanted to show you some examples of that. After a few dozen captures, I came up with this one:

SCR58

Looks pretty bad, right? Tons of hickups in what was meant to be a 5.2 MHz pulse train.

Not so fast…

The sample rate used by the oscilloscope for this capture was 10 MHz, so it’s sampling the signal at 100 ns intervals. That’s extremely risky, because when the signal is also pulsing at this rate, you might end up missing each peak for a substantial amount of time. This would make it appear as if the signal was low or high all the time, despite its rapid changes.

This effect is called “aliasing”, and the solution is simple:

Do Not Sample At A Rate Close To The Frequencies In Your Signal!

I’m inclined to discard the above snapshot – it’s probably a measurement artefact.

Here’s another screen, this time at 50 MHz sample rate, i.e. 20 ns between captures:

SCR59

At that rate, it’s impossible to miss pulses which are all in the range of 100 ns or longer. As you can see, there are still glitches (almost on every snapshot there is one), but it’s not as bad as the previous screen shot was suggesting.

Anyway. Enjoy your RPi and all the Linux high-level stuff it can do (like, ehm, run the Arduino IDE and cross-compile stuff for ATmega’s), but beware of treating it like a CPU which is exclusively at your service.

A Raspberry Pi is fast most of the time, but it’s (occasionally) a bit occupied with itself…

Fast I/O on a Raspberry Pi

In Hardware, Linux, Software on Jun 15, 2013 at 00:01

To follow up on yesterday’s post about WiringPi and how it was used to display an image on a colour LCD screen attached to the RPi via its GPIO pins.

A quick calculation indicated that the WiringPi library is able to toggle the GPIO pins millions of times per second. Now this might not seem so special if you are used to an ATmega328 on a JeeNode or other Arduino-like board, but actually it is

Keep in mind that the RPi code is running under Linux, a multi-tasking 32-bit operating system which not only does all sorts of things at (nearly) the same time, but which also has very solid memory and application protection mechanisms in place:

  • each application runs in its own address space
  • the time each application runs is controlled by Linux, not the app itself
  • applications can crash, but they cannot cause the entire system to crash
  • dozens, hundreds, even thousands of applications can run next to each other
  • when memory is low, Linux can swap some parts to disk (not recommended with SD’s)

So while the LCD code works well, and much faster than one might expect for all the GPIO toggling it’s doing, this isn’t such a trivial outcome. The simplest way to deal with the RPi’s GPIO pins, is to manipulate the files and directories under the /sys/class/gpio/ “device tree”. This is incredibly useful because you can even manipulate it via plain shell script, using nothing but the “echo”, “cat”, and “ls” commands. Part of the convenience is the fact that these manipulations can take place entirely in ASCII, e.g. writing the string “1″ or “0″ to set/reset GPIO pins.

But the convenience of the /sys/class/gpio/ virtual file system access comes at a price: it’s not very fast. There is too much involved to deal with individual GPIO pins as files!

WiringPi uses a different approach, called “memory mapped files”.

Is it fast? You bet. I timed the processing time of this snippet of C code:

int i;
for (i = 0; i < 100000000; ++i) {
  digitalWrite(1, 1);
  digitalWrite(1, 0);
}
return 0;

Here’s the result:

    real    0m19.592s
    user    0m19.490s
    sys     0m0.030s

That’s over 5 million pulses (two toggles) per second.

This magic is possible because the I/O address space (which is normally completely inaccessible to user programs) has been mapped into a section of the user program address space. This means that there is no operating system call overhead involved in toggling I/O bits. The mapping is probably virtualised, i.e. the kernel will kick in on each access, but this is an interrupt straight into protected kernel code, so overhead is minimal.

How minimal? Well, it takes less than 90 ns per call to digitalWrite(), so even when running at the RPi’s maximum 1 GHz rate, that’s less than 90 machine cycles.

Note how the RPi can almost toggle an I/O pin as fast as an ATmega running at 16 MHz. But the devil is in the details: “can” is the keyword here. Being a multi-tasking operating system, there is no guarantee whatsoever that the GPIO pin will always toggle at this rate. You may see occasional hick-ups in the order of milliseconds, in fact.

Is the RPi fast? Definitely! Is it guaranteed to be fast at all times? Nope!

What it took to generate that LCD image

In Hardware, Linux, Software on Jun 14, 2013 at 00:01

As shown yesterday, it’s relatively easy to get a bitmap onto an LCD screen connected to a few I/O pins of the Raspberry Pi.

But there were some gotcha’s:

  • the image I wanted to use was 640×480, but the LCD is only 320×240 pixels
  • the WiringPi library has code to put a bitmap on the screen, but not JPEGs
  • this is easy to fix using the ImageMagick “convert” command
  • but the result has 24-bit colour depth, whereas the LCD wants 16-bit (5-6-5)

The “convert” command makes things very easy, but first I had to install ImageMagick:

    sudo apt-get install ImageMagick

Then we can run “convert” from the command-line (long live the Unix toolkit approach!):

    convert snap.jpg -geometry 320x240 -normalize snap.rgb

This handles both the rescaling and the transformation to a file with (R,G,B) types, each colour is one byte. As expected, the resulting image file is 320 x 240 x 3 = 230,400 bytes.

I didn’t see a quick way to convert the format to the desired 5-6-5-bit coding needed for the LCD, and since the code to write to the LCD is written in C anyway (to link to WiringPi), I ended up doing it all in a straightforward loop:

#define PIXELS (320*240)
uint8_t rgbIn24 [PIXELS][3];
unsigned rgbOut15 [PIXELS];

[...]

for (x = 0; x < PIXELS; ++x) {
  uint8_t r = rgbIn24[x][0] >> 3;
  uint8_t g = rgbIn24[x][1] >> 2;
  uint8_t b = rgbIn24[x][2] >> 3;
  rgbOut15[x] = (r << 11) | (g << 5) | b;
}

Note that this is a C program compiled and running under Linux on the RPi, and that it can therefore easily allocate some huge arrays. This isn’t some tiny embedded 8-bit µC with a few kilobytes – it’s a full-blown O/S running on a 32-bit CPU with virtual memory!

The other gotcha was that the bitmap to be supplied to the LCD library on top of WiringPi was expected to store each pixel in a 32-bit int. Total waste of space, and probably the result of a rushed port from the UTFT code (written to assume 16-bit ints) to the RPi. Which is why rgbOut15 is declared as “unsigned” (int, i.e. uint32_t) array, although the image really only needs an uint16_t. Oh well, what’s an extra 150 kB, eh?

Anyway, it works. Perfection can wait.

Note that this sends out all the data pixel by pixel to the display, and that each pixel takes the bit-banging of 2 bytes (the interface to the LCD is 8-bit wide), so each pixel takes at least 20 calls to digitalWrite(): 8 to set the data bits, and 2 two to toggle a clock pin – times two for the 2 bytes per pixel. That’s at least 4.6 million GPIO pin settings in half a second!

Tomorrow, I’ll show how WiringPi can make GPIO pins change in under 100 nanoseconds.

Getting an image on the LCD

In Hardware, Linux on Jun 13, 2013 at 00:01

To follow up on yesterday’s setup, I’ve managed to drive the display from Linux, using the WiringPi library by Gordon Henderson.

It supports driving the Raspberry Pi’s GPIO pins from C using Arduino- / Wiring-like calls to digitalWrite(). That, in combination with the UTFT library by Henning Karlsen, and some work done by iTead to combine everything was enough to get this self-portrait:

DSC_4482

The display is a bit shiny, and the contrast and brightness are a bit low, unfortunately :(

Drawing the image takes about half a second, but considering that it’s all done through bit-banging of a few I/O pins from a “user space” program, that’s pretty good actually!

There are a couple of interesting (and sneaky) details, which I’ll go into in the next post.

Hurray for open source and the remixing / sharing it enables!

LCD screen for the RPi

In Hardware, Linux on Jun 12, 2013 at 00:01

For a little side-project I’m involved in (more on that in a future post), I wanted to try adding a little LCD screen with touch sensor to a Linux setup. Found this one at iTead:

DSC_4480

There’s a little ribbon cable and adapter (just re-wiring) available for it, which allows plugging this thing into a Raspberry Pi. The nice thing about this screen, apart from being 320×240 pixels and 16-bit color TFT, is its physical size – just right for a stacked setup:

DSC_4481

Stay tuned, as I explore how to hook this thing up and make it show stuff…

Another new kid on the block

In ARM, Linux on Jun 10, 2013 at 00:01

Besides the Odroid U2 quad-core Linux board mentioned a few days ago, there’s another option in the same price range as the Raspberry Pi – the Beaglebone Black, a.k.a. “BBB”:

Screen Shot 2013-06-06 at 23.15.47

Yeah, ok, it’s slightly more expensive, but it comes with 2 GB of speedy eMMC memory built-in, whereas you need to add an SD card to make the RPi start up.

This is a bigger deal than you might think, because the BBB comes with Angström Linux pre-installed, and you can get going by simply plugging the BBB into a USB port:

  • the moment you do, a memory disk appears, and it gets auto-mounted
  • there’s a “start.htm” file you can double-click to launch your web browser
  • then just follow instructions and you’ll be up and running

This is very similar to the way the MBED ARM module works, and it shows that streamlining the first encounter really can help people to get started – fast!

From a general point of view, the Beaglebone black is actually quite different from the RPi, despite their obvious similarities of both being low-cost, low-power, yet fairly performant little Linux stand-alone boards with with keyboard-monitor-and-mouse capability and built-in Ethernet. Here’s where they differ, at least as I see it:

  • The RPi is aimed at the educational market, i.e. for learning about Linux and doing cool stuff with it once you’ve mastered that first step. Get a board, hook into a huge community, get your hands dirty, and start learning and hacking!

  • The BBB is aimed more at the embedded Linux market, i.e. for building all sorts of advanced projects with Linux and a powerful computing platform inside. Plug it in, put your Linux and physical computing skills to work, and make neat things happen!

The difference? The RPi is the most affordable solution out there, and does really well at being a complete Linux system with fairly good video and audio capabilities, but somewhat limited I/O expandability (even just thinking about how to hook up and size an expansion board on top is a puzzle). The BBB is packed with faster and more powerful hardware (but less focused on video and audio) and is loaded with expansion connectors and pins, ready to add a board, or even a whole stack of them. Even without boards on top (called capes), you have tons of I/O pins to play with – whether you need digital, analog, PWM, or all sorts of bussed interfaces, it’s often all there.

Some details about the BBB running at 300 MHz idle (it scales to 1000 MHz on demand):

# cat /proc/cpuinfo 
processor       : 0
model name      : ARMv7 Processor rev 2 (v7l)
BogoMIPS        : 297.40
Features        : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc08
CPU revision    : 2

Hardware        : Generic AM33XX (Flattened Device Tree)
Revision        : 0000
Serial          : 0000000000000000

For a nice 20-min story about a project where RPi’s and BBB’s were used, see this video.

There’s a project called BoneScript, which aims to make the Beaglebone’s I/O capabilities as easy to use from JavaScript (i.e. Node.js) as what the Arduino IDE did for C/C++ and the ATmega’s I/O pins. Sounds like an excellent step in the right direction to me.

Either way, I see a bright future ahead for Node.js running on these little boards!

Move over, raspberry

In Hardware, Linux on Jun 4, 2013 at 00:01

The Raspberry_Pi is a great little board at an amazing little price. But as mentioned yesterday, it’s not very fast as a server. I suspect that a lot of it has to do with the SD card interface and the SD card itself, so I started looking around for alternatives.

And boy, there sure are lots of ‘em – still well below €100. I’ll single out one system, the Odroid U2 – knowing full well that there must be over a dozen others out there:

article_img

It’s smaller than a Raspberry Pi, but it comes mounted on a “big” (6x6x6 cm) heat sink.

The specs are pretty impressive:

Screen Shot 2013-06-01 at 11.45.13

And the board is neat – the result of a huge mobile phone market driving size down:

201301251344193656

201301251345520979

Could this be used as central web / file / home automation server?

Speedy Raspberry

In Hardware, Linux on Jun 3, 2013 at 00:01

The Raspberry Pi is an amazing little board, with an amazing amount of power and functionality at an incredible price – it’s probably fair to say that our technology geek’s world will never be the same again, now that we have the RPi.

But it’s no speed monster…

Sometimes during development, it really gets in the way – especially when you’re used to working on a modern fast laptop, which is some 20 times faster (and that’s per-core).

So I decided to overclock one of my RPi’s, and see what it does. Overclocking in Raspbian is trivial with the standard raspi-config utility:

Screen Shot 2013-05-28 at 16.52.58

Just go to the Overclock section, and pick one:

Screen Shot 2013-05-28 at 16.46.48

I went for the fastest there is, but decided to run a 2-hour test to make sure it really works. Turns out that someone has already done all the work, and even created such a stress test:

root@raspberrypi:~# ./overclock-test.sh 
Testing overclock stability...
reading: 1
945+1 records in
945+1 records out
3965190144 bytes (4.0 GB) copied, 352.455 s, 11.3 MB/s
reading: 2
[...]
reading: 10
945+1 records in
945+1 records out
3965190144 bytes (4.0 GB) copied, 358.522 s, 11.1 MB/s
writing: 1
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 95.2065 s, 5.6 MB/s
writing: 2
[...]
writing: 10
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 83.4848 s, 6.4 MB/s
./overclock-test.sh: line 18:  2414 Terminated
              nice yes > /dev/null
CPU freq: 1000000
CPU temp: 58376
[    5.217175] Registered led device: led0
[    8.943589] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[    9.418819] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[   19.181046] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
[   19.431053] bcm2835-cpufreq: switching to governor ondemand
[   19.431082] bcm2835-cpufreq: switching to governor ondemand
[   21.736829] Adding 102396k swap on /var/swap.  Priority:-1 extents:1 across:102396k SS
[   22.134893] ip_tables: (C) 2000-2006 Netfilter Core Team
[   22.174573] nf_conntrack version 0.5.0 (7774 buckets, 31096 max)
[  750.951153] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
Not crashed yet, probably stable.

Ya gotta love that last message, nicely semi re-assuring ;)

I did fit a set of cooling fins from Watterott (some nice IR heat pictures there):

DSC_4475

As checked with vcgencmd measure_temp, the temperature never rose above 58.4°C.

Onwards!

Carambola 2 power consumption

In Hardware, Linux on May 7, 2013 at 00:01

The Carambola 2 mentioned yesterday is based on a SoC design which uses amazingly little power – considering that it’s running a full Linux-based OpenWrt setup.

There are a couple of ways to measure power consumption. If all you’re after is the average power on idle, then all you need to do is insert a multimeter in the power supply line and set it in the appropriate milliamp range. Wait a minute or so for the system to start up, and you’ll see that the Carambola 2 draws about 72 mA @ 5V, i.e. roughly a third of a watt.

If you have a lab power supply, you can simply read the power consumption on its display.

But given an oscilloscope, it’s actually much more informative to see what the power consumption graph is, i.e. over time. This will show the startup power use and also allows seeing more detail, since these systems often periodically cycle through different activities.

The setup for “seeing” power consumption is always the same: just insert a small resistor in series with the “Device Under Test”, and measure the voltage drop over that resistor:

JC's Grid, page 51

Except that in this case, we need to use a smaller resistor to keep the voltage drop within bounds. Given that the expected currents will be over 100 mA, a 100 Ω resistor would completely mess up the setup. I found a 0.1 Ω SMD resistor in my lab supplies, so that’s what I used – mounting it on a 2-pin header for convenience:

DSC_4448

With 0.1 Ω, a 100 mA current produces a voltage of 10 mV. This should have a negligible effect on the power supplied to the Carambola 2 (a 1 Ω resistor should also work fine).

Here’s the result on the scope – white is the default setup, yellow is with WiFi enabled:

SCR11

Sure takes all the guesswork out of what the power consumption is doing on startup, eh?

Embedded Linux – Carambola 2

In Hardware, Linux on May 6, 2013 at 00:01

This has got to be one of the lowest-cost and simplest embedded Linux boards out there:

DSC_4447

It’s the Carambola 2 by 8devices.com:

Screen Shot 2013-05-04 at 11.02.06

The 28 x 38 mm (!) bare board is €19 excl VAT and shipping, and the development bundle (as shown above) is €33. The latter has a Carambola2 permanently soldered onto it, with 2 Ethernet ports, a slave USB / console / power port, a USB host port, a WiFi chip antenna (which is no longer on the base board, unlike the original Carambola), and a switching power supply to generate 3.3V from the USB’s 5V.

The processor is a MIPS-based Atheros chip, and with 64 MB ram and 11 MB of available flash space, there is ample room to pre-populate this board with a lot of files and software.

The convenience of the development setup, is that it includes an FTDI chip, so it comes up as a USB serial connection – you just need to find out what port it’s on, connect to it at 115200 baud via a terminal utility such as “screen” on Mac or Linux, and you’ll be hacking around in OpenWrt Linux in no time.

Note that this setup is very different from a Raspberry Pi: MIPS ≠ ARM, for one. The RPi has a lot more performance and RAM, has hardware floating point, and is more like a complete (portable) computer with its HDMI video out port. The benefit of the Carambola 2 is its built-in WiFi, built-in flash, and its low power – more on that tomorrow.

HouseMon resources

In AVR, Hardware, Linux, Musings, Software on Feb 6, 2013 at 00:01

As promised, a long list of resources I’ve found useful while starting off with HouseMon:

JavaScript – The core of what I’m building now is centered entirely around “JS”, the language behind many sites on the web nowadays. There’s no way around it: you have to get to grips with JS first. I spent several hours watching most of the videos on Douglas Crockford’s site. The big drawback is the time it takes…

Best book on the subject, IMO, if you know the basics of JavaScript, is “JavaScript: The Good Parts” by the same author, ISBN 0596517742. Understanding what the essence of a language is, is the fastest way to mastery, and his book does exactly that.

CoffeeScript – It’s just a dialect of JS, really – and the way HouseMon uses it, “CS” automatically gets compiled (more like “expanded”, if you ask me) to JS on the server, by SocketStream.

The most obvious resource, http://coffeescript.org/, is also one of the best ways to understand it. Make sure you are comfortable with JS, even if not in practice, before reading that home page top to bottom. For an intruiging glimpse of how CS code can be documented, see this example from the CS compiler itself (pretty advanced stuff!).

But the impact of CS goes considerably deeper. To understand how Scheme-like functional programming plays a role in CS, there is an entertaining (but fairly advanced) book called CoffeeScript Ristretto by Reginald Braithwaite. I’ve read it front-to-back, and intend to re-read it in its entirety in the coming days. IMO, this is the book that cuts to the core of how functions and objects work together, and how CS lets you write on a high conceptual level. It’s a delightful read, but be prepared to scratch your head at times…

For a much simpler introduction, see The Little Book on CoffeeScript by Alex MacCaw, ISBN 1449321046. Also available on GitHub.

Node.js – I found the Node.js in Action book by Mike Cantelon, TJ Holowaychuk and Nathan Rajlich to be immensely useful, because of how it puts everything in context and introduces all the main concepts and libraries one tends to use in combination with “Node”. It doesn’t hurt that one of the most prolific Node programmers also happens to be one of the authors…

Another useful resource is the API documentation of Node itself.

SocketStream – This is what takes care of client-server communication, deployment, and it comes with many development conveniences and conventions. It’s also the least mature of the bunch, although I’ve not really encountered any problems with it. I expect “SS” to evolve a bit more than the rest, over time.

There’s a “what it is and what it does” type of demo tour, and there is a collection on what I’d call tech notes, describing a wide range of design docs. As with the code, these pages are bound to change and get extended further over time.

Redis – This a little database package which handles a few tasks for HouseMon. I haven’t had to do much to get it going, so the README plus Command Summary were all I’ve needed, for now.

AngularJS – This is the most framework-like component used in HouseMon, by far. It does a lot, but the challenge is to understand how it wants you to do things, and altough “NG” is not really an opinionated piece of software, there is simply no other way to get to grips with it, than to take the dive and learn, learn, learn… Let me just add that I really think it’s worth it – NG can be magic on the client side, and once you get the hang of it, it’s in fact an extremely expressive way to create a responsive app in the browser, IMO.

There’s an elaborate tutorial on the NG site. It covers a lot of ground, and left me a bit overwhelmed – probably because I was trying to learn too much as quickly as possible…

There’s also a video, which gives a very clear idea of NG, what it is, how it is used, etc. Only downside is that it’s over an hour long. Oh, and BTW, the NG FAQ is excellent.

For a broader background on this sort of JS frameworks, see Rich JavaScript Applications by Steven Sanderson. An eye opener, if you’ve not looked into RIA’s before.

Arduino – Does this need any introduction on this weblog? Let me just link to the Reference and the Tutorial here.

JeeNode – Again, not really much point in listing much here, given that this entire weblog is more or less dedicated to that topic. Here’s a big picture and the link to the hardware page, just for completeness.

RF12 – This is the driver used for HopeRF’s wireless radio modules, I’ll just mention the internals weblog posts, and the reference documentation page.

Vim – My editor of choice, lately. After many years of using TextMate (which I still use as code browser), I’ve decided to go back to MacVim, because of the way it can be off-loaded to your spine, so to speak.

There’s a lot of personal preference involved in this type of choice, and there are dozens of blog posts and debates on the web about the pro’s and con’s. This one by Steve Losh sort of matches the process I am going through, in case you’re interested.

Best way to get into vim? Install it, and type “vimtutor“. Best way to learn more? Type “:h<CR>” in vim. Seriously. And don’t try to learn it all at once – the goal is to gradually migrate vim knowledge into your muscle memory. Just learn the base concepts, and if you’re serious about it: learn a few new commands each week. See what sticks.

To get an idea of what’s possible, watch some videos – such as the vim entries on the DAS site by Gary Bernhardt (paid subscription). And while you’re at it: take the opportunity to see what Behaviour Driven Design is like, he has many fascinating videos on the subject.

For a book, I very much recommend Practical Vim by Drew Neil. He covers a wide range of topics, and suggests reading up on them in whatever order is most useful to you.

While learning, this cheatsheet and wallpaper may come in handy.

Raspberry Pi – The little “RPi” Linux board is getting a lot of attention lately. It makes a nice setup for HouseMon. Here are some links for the hardware and the software.

Linux – Getting around on the command line in Linux is also something I get asked about from time to time. This is important when running Linux as a server - the RPi, for example.

I found the linuxcommand.org resource which appears to do a good job of explaining all the basic and intermediate concepts. It’s also available as a book, called “The Linux Command Line” by William E. Shotts, Jr. (PDF).

There… the above list ought to get you a long way with all the technologies I’m currently messing around with. Please feel free to add pointers and tips in the comments, if you think of other resource which can be of use to fellow readers in this context.

Dive Into JeeNodes

In AVR, Hardware, Linux, Software on Feb 1, 2013 at 00:01

Welcome to a new series of limited-edition posts from JeeLabs! Read ‘em while they last!

Heh… just kidding. They’ll last forever of course, as does everything on this thing called internet. But what I’m going to describe in probably a dozen posts or so is the following:

dijn-essence

Hm, that doesn’t quite explain it, I guess. Let me try again:

JC's Grid, page 63

So this is to announce a new “DIJN” series of weblog posts, describing how to set up your own Wireless Sensor Network with JeeNodes, as well as the infrastructure to report a measured light-level somewhere in your house, in real time. The end result will be fully automated and autonomous – you could take your mobile phone, point it to your web server via WiFi, and see the light level as it is that very moment, adjusting as it changes.

This is a far cry from a full-fledged home monitoring or home automation system, clearly – but on the other hand, it’ll have all the key pieces in place to explore whatever direction interests you: ready-made sensors, DIY sensors, your own firmware on the remote nodes, your own web pages and automation logic on the central server… it’s up to you!

Everything is open source, which in this context matters a lot, because that also means that you can really dive into any aspect of this to learn and explore the truly magical world of Physical Computing, Wireless Sensor Networks, environmental sensing and control, as well as state-of-the art web technologies.

The focus will be on describing every step needed to implement this from scratch. I’ll cover setting up all the necessary software and hardware, in such a way that if you know next to nothing about any of the domains involved, you can still follow along and try it out – whether your background is in software, electronics, wireless, or none of these.

If technology interests you, and if I can bring across even a small fraction of the fun there is in tinkering with this stuff and making new things up as you g(r)o(w) along, then that would be a very nice reward for everyone involved, as far as I’m concerned.

PS. “Dijn” is also old-Dutch for “your” (thy, to be precise). Quite a fitting name in my opinion, as this sort of knowledge is really yours for the taking – if you want it…

PPS. For reference: here is the first post in the series, and here is the overview.

Arduino sketches on RPi

In AVR, Linux, Software on Jan 17, 2013 at 00:01

There’s an arduino-mk package which makes it simple to build and upload sketches on a Raspberry Pi. Here’s how to set it up and use it with a JeeLink, for example:

Install the package:

  sudo apt-get install arduino-mk

That gets all the required software and files, but it needs a tiny bit of setup.

First, create the library area with a demo sketch in it:

  mkdir ~/sketchbook
  cd ~/sketchbook
  ln -s /usr/share/arduino/Arduino.mk
  mkdir blink
  cd blink

Then create two files – the blink.ino sketch for a JeeLink or JeeNode SMD / USB:

// Blink - adapted for a JeeNode / JeeLink

void setup() {
  pinMode(9, OUTPUT);
}

void loop() {
  digitalWrite(9, LOW);
  delay(1000);
  digitalWrite(9, HIGH);
  delay(1000);
}

… and a Makefile which ties it all together:

BOARD_TAG    = uno
ARDUINO_PORT = /dev/ttyUSB0
ARDUINO_LIBS =
ARDUINO_DIR  = /usr/share/arduino

include ../Arduino.mk

(that Arduino.mk file in the sketchbook/ dir is well-commented and worth a read)

That’s it. You now have the following commands to perform various tasks:

  make             - no upload
  make upload      - compile and upload
  make clean       - remove all our dependencies
  make depends     - update dependencies
  make reset       - reset the Arduino by tickling DTR on the serial port
  make raw_upload  - upload without first resetting
  make show_boards - list all the boards defined in boards.txt

The first build is going to take a few minutes, because it compiles the entire runtime code. After that, compilation should be fairly snappy. Be sure to do a make clean whenever you change the board type, to re-compile for a different board. The make depends command should be used when you change any #include lines in your code or add / remove source files to the project.

This setup makes things run smoothly, without requiring the Arduino IDE + Java runtime.

Flukso with RFM12B

In Hardware, Linux on Jan 12, 2013 at 00:01

Some exciting new developments going on…

DSC_4347

You’re looking at the final prototype of the latest Flukso meter, which can be connected to AC current sensors, pulse counters, and the Dutch smart metering “P1″ port. Here’s the brief description from that website:

Flukso is a web-based community metering application. Install a Fluksometer near your fuse box and you will be able to monitor, share and reduce your electricity consumption through this website.

The interesting bit is that it’s all based on a Linux board with wired and wireless Ethernet, plus a small ATmega-based add-on board which does all the real-time processing.

But the most exciting news is that the new version, now entering production, will include an RFM12B module with the JeeNode-compatible protocol. A perfect home automation workstation. Yet another interesting aspect of this, is that Bart Van Der Meersche, the mastermind behind Flusko, is working on getting the Mosquitto MQTT broker running permanently on that same Flukso meter.

Here’s the basic layout (probably slightly different from the actual production units):

Screen Shot 2013-01-11 at 21.10.20

Flukso runs OpenWRT, and everything in it is based on the Lua programming language, which is really an excellent fit for such environments. But even if Lua is not something you want to dive into, the open-endedness of PubSub means this little box drawing just a few Watt can interface to a huge range of devices – from RF12 to WiFi to LAN, and everything flowing in and out of that little box becomes easily accessible via MQTT.

PS. I have no affiliation with Flukso whatsoever – I just like it, and Bart is a nice fellow :)

Collecting serial packets

In Linux, Software on Oct 17, 2012 at 00:01

As promised after yesterday’s “rf12cmd” sketch, here’s a Lua script for Linux which picks up characters from the serial port and turns them into Lua data structures:

Screen Shot 2012 10 14 at 17 31 37

Ah, if only things were that simple! On Mac OSX, the serial port hangs when opened normally, so we need to play non-blocking tricks and use the lower-level nixio package:

Screen Shot 2012 10 14 at 17 28 40

(note also the subtle “-f” vs “-F” flag for stty… welcome to the world of portability!)

Here’s some sample output:

  $ ./rf12show.lua 
  (table) 
    [1] = (string) hi
    [2] = (string) rf12cmd
    [3] = (number) 1
    [4] = (number) 100
  (table) 
    [1] = (string) rx
    [2] = (number) 868
    [3] = (number) 5
    [4] = (number) 19
    [5] = (12 bytes) BAC80801E702200000DA5121
  (table) 
    [1] = (string) rx
    [2] = (number) 868
    [3] = (number) 5
    [4] = (number) 3
    [5] = (4 bytes) 21DE0F00
  (table) 
    [1] = (string) rx
    [2] = (number) 868
    [3] = (number) 5
    [4] = (number) 19
    [5] = (8 bytes) 7331D61AE7C43901

That’s the startup greeting plus three incoming packets.

I’m using a couple of Lua utility scripts – haven’t published them yet, but at least you’ll get an idea of how the decoding process can be implemented:

  • dbg.lua – this is the vardump script, extended to show binary data in hex format
  • benstream.lua – a little script I wrote which does “push-parsing” of Bencoded data

Note that this code is far too simplistic for real-world use. The most glaring limitation is that it is blocking, i.e. we wait for each next character from the serial port, while being completely unresponsive to anything else.

Taking things further will require going into processes, threads, events, asynchronous I/O, polling, or some mix thereof – which will have to wait for now. To be honest, I’ve become a bit lazy because the Tcl language solves all that out of the box, but hey… ya’ can’t have everything!

Bencode in Lua, Python, and Tcl

In Linux, Software on Oct 4, 2012 at 00:01

Ok, let’s get on with this Bencode thing. Here is how it can be used on Linux, assuming you have Lua, Python, or Tcl installed on your system.

Let me show you around in the wonderful land of rocks, eggs, and packages (which, if I had my way, would have been called rigs … but, hey, whatever).

Lua

Lua’s Bencode implementation by Moritz Wilhelmy is on GitHub. To install, do:

    luarocks install bencode

(actually, I had to use “sudo”, this appears to be solved in newer versions of LuaRocks)

LuaRocks – a clever play on words if you think about it – is Lua’s way of installing well-known packages. It can be installed on Debian using “sudo apt-get install luarocks”.

The package ends up as as a single file: /usr/local/share/lua/5.1/bencode.lua

Encoding example, using Lua interactively:

    > require 'bencode'
    > s = {1,2,'abc',{234},{a=1,b=2},321}
    > print(bencode.encode(s))
    li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee

To try it out, we need a little extra utility code to show us the decoded data structure. I simply copied table_print() and to_string() from this page into the interpreter, and did:

    > t = 'li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee'
    > print(to_string(bencode.decode(t)))
    "1"
    "2"
    "abc"
    {
      "234"
    }
    {
      a = "1"
      b = "2"
    }
    "321"

(hmmm… ints are shown as strings, I’ll assume that’s a flaw in to_string)

Python

Python’s Bencode implementation by Thomas Rampelberg is on PyPi. Install it as follows:

    PKGS=http://pypi.python.org/packages/2.7
    easy_install $PKGS/b/bencode/bencode-1.0-py2.7.egg

It ends up as “/usr/local/lib/python2.7/dist-packages/bencode-1.0-py2.7.egg” – this is a ZIP archive, since eggs can be used without unpacking.

This depends on easy_install. I installed it first, using this magic incantation:

    wget http://peak.telecommunity.com/dist/ez_setup.py
    python ez_setup.py

And it ended up in /usr/local/bin/ – the standard spot for user-installed code on Linux.

Now let’s encode the same thing in Python, interactively:

    >>> import bencode
    >>> bencode.bencode([1,2,'abc',[234],{'a':1,'b':2},321])                        
    'li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee'

And now let’s decode that string back into a data structure:

    >>> bencode.bdecode('li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee')                  
    [1, 2, 'abc', [234], {'a': 1, 'b': 2}, 321]

Note: for another particularly easy to read Python decoder, see Mathieu Weber‘s version.

Tcl

Tcl’s Bencode implementation by Andreas Kupries is called Bee and it part of Tcllib.

Tcllib is a Debian package, so it can be installed using “sudo apt-get install tcllib”.

Ok, so installation is trivial, but here we run into an important difference: Tcl’s data structures are not “intrinsically typed”. The type (and performance) depends on how you use the data, following Tcl’s “everything is a string” mantra.

Let’s start with decoding instead, because that’s very similar to the previous examples:

    % package require bee    
    0.1
    % bee::decode li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee
    1 2 abc 234 {a 1 b 2} 321

Decoding works fine, but as you can see, type information vanishes in the Tcl context. We’ll need to explicitly construct the data structure with the types needed for Bencoding.

I’m going to use some Tcl trickery by first defining shorthand commands S, N, L, and D to abbreviate the calls, and then construct the data step by step as a nested set of calls:

    % foreach {x y} {S String N Number L ListArgs D DictArgs} {
        interp alias {} $x {} bee::encode$y
    }
    % L [N 1] [N 2] [S abc] [L [N 234]] [D a [N 1] b [N 2]] [N 321]
    li1ei2e3:abcli234eed1:ai1e1:bi2eei321ee

So we got what we wanted, but as you can see: not all the roads to Rome are the same!

ARMs on Foam

In ARM, Hardware, Linux on Sep 29, 2012 at 00:01

One of the things I’d really like to do is hack on that Laser Cutter I described recently.

The electronics is based on a LaOS Board, but I’d like to see what you can do with an embedded Linux board such as a Raspberry Pi in this context – driving that LaOS board, for example. Because adding Linux to the mix opens up all sorts of neatness.

So here’s my new prototype development setup:

DSC 4163

That oh-so-neat foam board acts as base, with two PCB’s fastened to it using, heh … remember these?

DSC 4164

They’re called “splitpennen” in Dutch. Long live foam board and splitpennen!

This is a pretty nifty setup, in my opinion. Tons and tons of ways to implement features on this combo, and there’s plenty of power and storage on both boards to perform some pretty neat tricks, I expect.

Anyway – this is more a big project for cold winter days, really. It’ll take a long time before anything can come out of this, but isn’t it incredible how the prices of these things have reached a point where one can now dedicate such hardware to a project?

ARM to ARM serial connection

In ARM, Hardware, Linux on Sep 26, 2012 at 00:01

After connecting a JeeNode to a Raspberry Pi, let’s do the same with the LPCXpresso I mentioned a while back. Let’s do it in such a way that we can upload new code into it.

With an LPC1769, this is even simpler than with an ATmega328, because the serial boot loader is built-into the chip out of the box. No need to install anything. So you can always force the chip back into serial boot loader mode – it can’t be “bricked” !

But the process is slightly different: you have to pull down a specific “ISP” pin while resetting the chip, to make it enter serial boot loader mode. So we’ll need one more GPIO pin, for which I’ll use the RPi’s GPIO 23. The wiring is even cleaner than before, because this time I planned a bit more ahead of time:

DSC 4156

Except that this runs into the same problem as before. The LPCXpresso does not have the essential 3.3V regulator next to the ARM chip, it’s on the part that has been cut off (doh!). So again, I’m going to have to add an SMD MCP1703 regulator:

DSC 4157

(that little critter is between pins 1 and 2, and upside down, to get the pinout right)

Here’s the complete hookup:

DSC 4159

First step is again to make sure that Linux isn’t doing anything with the serial port:

  • remove the two args referring to /dev/ttyAMA0 in the file /boot/cmdline.txt

  • add a hash (“#”) in front of this line in /etc/inittab to comment it out:

    T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100
    
  • make the system aware of the changes in inittab with: kill -1 1 (or reboot)

Tomorrow, I’ll go into actually uploading to that LPC1769 ARM board. Stay tuned!

Note – As of October 1st, VAT prices in the Netherlands will increase from 19% to 21%. As a consequence some adjustments will also have to be made in the JeeLabs Shop. Up or down depends on many obscure factors, including exchange rates, current costs, stock levels, and the wish to stick to decently rounded values – so if you want to avoid any surprises and were planning to order stuff, please keep that switch-over date in mind.

Upload to JeeNode fixed

In Hardware, Linux on Sep 23, 2012 at 00:01

Ok, I finally figured out what caused yesterday’s upload problem. It’s indeed timing.

But not quite as I thought, and I missed a hint in the scope signal capture yesterday (at 322 ms after the reset). My premature conclusion was that resetting the GPIO pin and then starting up avrdude was taking too long, so the boot loader would give up before receiving the proper starting character over the serial line.

But that doesn’t quite make sense. Looking at the screen shot again, we can see that the RF12demo greeting (blue line) starts about 800 ms after the RESET pin gets pulled low. Even though OptiBoot is supposed to wait a full second after power-up. And there are two handshake attempts well within that period.

The other subtle hint was in the not-quite-equidistant characters being sent out (yellow line). Why would handshake chars be sent out in such a repeatable yet irregular pattern?

Having written an avrdude replacement in Tcl for some experiments I did with JeeMon a while back, I decided to try and extend it a bit to toggle the reset pin right before sending out the data. That way no delay would interfere, so the reset would happen moments (probably mere milliseconds) before starting the boot loader serial handshake. I didn’t really want to start hacking on avrdude for such a “simple” task as toggling a GPIO pin on the Raspberry Pi. Besides, that replacement code is only 70 lines of Tcl, if all you have to deal with is the basic stk500 protocol understood by the boot loader.

Controlling the GPIO pin also turned out to be pretty straightforward:

Screen Shot 2012 09 20 at 23 59 50

But no matter what I tried, and no matter what timing delays I inserted, the darn thing just wouldn’t upload!

Then, just for the heck of it, I tried this variation – reversing the open and reset order:

Screen Shot 2012 09 21 at 00 01 18

And lo and behold – now it works: repeatedly and reliably!

Here’s the scope’s serial protocol analyser, showing a proper upload in progress:

SCR32

You can see data being sent to the JeeNode, and the 0x14h reply sent back to the RPi.

So was this all about timing? Yes and no. Let’s revisit yesterday’s list of pulses again:

Screen Shot 2012 09 21 at 00 06 02

A single pulse 322 ms after reset, and then several pulses at 673 ms (presumably the first boot loader protocol handshake character). The problem is really that first pulse – it’s not a valid character but a glitch!

What I think is happening is that the JeeNode resets, the glitch at 322 ms causes the boot loader to give up and launch the sketch, and then all subsequent boot handshake characters get ignored. Looks like opening the serial port produces a glitch on the transmit output pin.

By first opening the serial port and then doing the GPIO18 reset, the problem is avoided, and then it all works.

Thank you JeeMon – I hadn’t expected to fire you up again, but you’ve saved my day!