Computing stuff tied to the physical world

Summary of ISP options

In AVR, Hardware, Software on May 29, 2011 at 00:01

First of all, see this weblog post for an overview of what ISP is, why you’d want to know, and when you need it.

There have been quite a few posts about ISP on this weblog over time – all tagged, so they can be reached via this link: https://jeelabs.org/tag/isp/.

I’ll summarize here, since particularly the different sketches floating around by now might be a bit confusing.

The ZIF socket is a way to connect to chips before they have been soldered into a circuit. There’s one for through-hole DIP chips, and one for 32-TQFP SMD chips:

Once soldered into a circuit, you can use an I(C)SP programmer, which not everyone has. So I created a couple of different options based on a JeeNode (can also be an RBBB or Arduino) to perform the same function.

My first trial was a hack with the isp_flash.pde sketch, which then led to the Flash Board (info). The nice thing about this board is that it has 128 Kbyte of EEPROM on board – enough to hold the complete programming instructions for an ATmega168 or ATmega328.

One way to use the Flash Board is as capturing ISP programmer: you send a sketch to it as if it were an ISP programmer, and then you can disconnect it and use it repeatedly to upload that sketch to other boards via ISP:

I used this to build a 4-fold Multi-ISP programmer to program all those gazillion ATmega’s needed for JeeNodes:

Took a few attempts to get all the bugs worked out, but now it works perfectly.

But there’s a much simpler way to do this, if all you want is to fix a JeeNode, occasionally:

… or use an Arduino:

These two ISP setups use an isp_repair.pde sketch, which has as novelty that it includes the sketch and bootstrap its needs for programming – as data array in flash memory. So all that’s needed is: connect the two boards and power up the master. No need to use the Arduino IDE’s “Burn Bootloader” command or anything.

Then came OptiBoot, which is compatible with the Arduino Uno, and smaller/faster. This is a relatively new boot loader so you might want to update your current JeeNodes – even though an ISP programmer isn’t normally needed for development.

I updated the isp_repair.pde sketch, and then fixed a few remaining problems with it:

And finally, I added a LiPo battery, and 4 config switches, to end up with this tool:

On the software side, the most recent design is called isp_repair2.pde. It supports two different sketches, up to four different bootstrap loaders, and setting the fuses either as 16 MHz crystal or optimized specifically for ceramic resonators.

The include files with the bootstraps included with these sketches are generated from HEX files created by the Arduino IDE. This is done with a small Tcl script called hex2c.tcl, which is located in a slightly older isp_prepare example sketch folder. It’s not needed for normal use, only when one of the boot loaders changes.

Phew. Still there?

In a nutshell: if you need to re-flash your ATmega, pick one of the above options to hook things up, and use the isp_repair2 sketch in the Ports library. By default, it’ll upload a blink demo sketch and the OptiBoot loader.

Arduino, ISP

Fixing the isp_repair sketch

In AVR, Hardware, Software on May 26, 2011 at 00:01

A few days ago I posted a new sketch to reprogram an ATmega with the OptiBoot loader when you don’t have an ISP programmer. Worked for me, so I thought… kick it into the world!

Whoops…

First of all, there should have been a warning that if it didn’t work, this would leave you with an unusable ATmega. Never occurred to me, since I have an ISP programmer within reach to recover from such mistakes.

Fortunately, someone on the forum reported that the ATmega can be brought back to life with an old version of isp_repair.pde (which can be found here, BTW).

That’s odd… can’t program with the sketch, but can recover with the same sketch and different data bytes?

Time to dig in. First, I wanted to make sure that the timing was slow enough to work in all cases. Time to fire that logic analyzer up again:

Screen Shot 2011 05 25 at 11.33.22

Looks good – since I’m using standard digitalWrite() calls, the pins aren’t toggling very fast at all:

Screen Shot 2011 05 25 at 11.34.29

Then it dawned on me:

The lock bits don’t look right: 0xFF – should have been 0xCF (top 2 bits are always 1, i.e. same as 0x0F).

Maybe everything was working, except the setting of the fuse bits? That would explain everything: a new boot loader gets loaded in the top 512 bytes, overwriting parts of the old boot loader, but the fuse bits perhaps wouldn’t get adjusted to just to the new boot address!

I changed a couple of things:

do the full chip erase before setting the fuse bits
set the lock bits to 0x0F at the end, i.s.o. 0x3F
included both bootstraps in the updated sketch
tri-state the ISP programming pins when done

The erase is needed to recover from a locked fuse state. The programming always took place after the erase, so it went well, but the fuse bits themselves would still be locked while trying to adjust them.

The second step should have been there all along, the way I was doing it the boot section was not protected from overwriting itself. This might explain the occasional report I got of people damaging boot loaders during use.

You can now also adjust the #define OPTIBOOT at the top of the sketch to 0 to revert to the original bootstrap code and fuse settings. So if OptiBoot is not what you want, recompile and restore as needed.

And lastly, the SPI programming pins are now reset to high-impedance after programming, so that the programming connections can be left in place without interfering with the target board.

Here’s the serial output from the updated sketch:

Screen Shot 2011 05 25 at 11.41.31

And here’s why it would sometimes have worked: if your ATmega had the lock bits unset (0x3F i.s.o. 0x0F), then the fuse settings would work as intended even with the chip erase in the wrong order. But with a locked setup, not everything would get set to the proper state.

Which goes to show: bugs can bite at any time!

Update – still some issues to iron out (see forum), but it looks like these are more related to OptiBoot than to this bootstrap replacement sketch.

Update #2 – OptiBoot issue solved.

▶ View 4 Comments

Arduino, JeeLink, JeeNode, ISP

Self-powered Opti-rebooter

In AVR, Hardware on May 18, 2011 at 00:01

After yesterday’s post about switching to OptiBoot, I decided to go one step further and go for a self-powered solution, a bit like this unit which has been serving me well for many months now.

So here’s the “Opti-rebooter” with a 400 mAh LiPo backpack:

Dsc 2521

Hookup is trivial, given that the JeeNode USB has an on-board LiPo charger. I picked a battery which matches the width of the board and fits quite well:

Dsc 2522

The two connectiosn of the LiPo battery are wrapped up in heat-shrink tubing to prevent accidental short circuits – the currents involved could easily cause trouble. Note that this is a “raw” LiPo cell, without any protection circuitry. That’s fine in this case, because the on-board circuitry takes care of charging.

To make this complete, the isp_repair.pde was extended to completely power down when done. So there’s no need for an on-off switch, the idle consumption is probably lower than the battery’s own self-discharge rate. And it blinks the on-board led twice: once on power up and once when properly programmed. Very convenient!

The way to use this thing, is: connect to a target JeeNode / JeeLink / Arduino, press the RESET button, wait three seconds, disconnect, done.

I also made changes to the very similar but slightly more elaborate isp_prepare.pde sketch, which was presented in this weblog post.

The updated isp_prepare.pde sketch supports a number of variations. These can be configured by adjusting a few values at the start of the sketch:

Screen Shot 2011 05 16 at 15.00.32

You can choose which type of boot loader to install, which sketch to load initially (just to start off, this can be overwritten later), and there’s a third option to adjust some “fuse bits” in the ATmega to select how fast the ATmega can start up after resuming from a power down.

That third option is particularly useful with battery-powered JeeNodes using the RF12 driver. With USE_FAST_WAKEUP set to 1, the power-up sequence is fast enough for an ATmega to wake up when the first data byte is received. This means you can completely power down the ATmega (while leaving the receiver running), and it’ll still be able to pick up incoming packets and respond to them. There is one gotcha: USE_FAST_WAKEUP can only be set to 1 on units running with a resonator – JeeLinks and Arduino’s running with a crystal must be not use the fast wakeup mechanism, as the clock may not start up properly in this case.

Haven’t tried it though – for now, that LiPo powered unit shown above is really the most convenient way for me to upgrade all the JeeNodes, JeeLinks, and Arduino’s floating around the house at JeeLabs.

Arduino, Network

Meet the RFM12B Board

In Hardware on Feb 2, 2011 at 00:01

With the RFM12B becoming a nice low-cost option for low-volume wireless communication, and the RF12 library proving to be a solid software driver for it, it’s time to generalize a bit further…

Say hello to the new RFM12B Board:

Dsc 2448

This board adds a voltage regulator and 3.3V/5V level conversions, to be able to use the RFM12B on 5V systems such as the various Arduino’s out there, the RBBB, … anything you want, really.

There are 8 pins on this board, of which the 8th is a regulated 3.3V supply which can be used in other parts of the circuit – the voltage regulator will be able to supply at least 100 mA extra on that supply pin.

The other 7 pins are:

+5V
Ground
SPI clock (SCK) – Arduino digital 13
SPI data out (SDO) – Arduino digital 12
SPI data in (SDI) – Arduino digital 11
SPI select (SEL) – Arduino digital 10
IRQ – Arduino digital 2

Just hook each of those up to an Arduino, and you can use the RF12 library as is!

Sample output:

Screen Shot 2011 02 01 at 22.08.14

Look ma, just like a JeeNode or JeeLink!

With an 8-pin stacking header and a bit of bending, cutting, and soldering two wires (I used a jumper wire, cut in half), you can even stick this thing right onto an Arduino:

Dsc 2449

But of course using a normal solderless breadboard and some wire jumpers will work just as well.

Note that this board can also be tied to 3.3V systems – just use the bare PCB (and short out three solder jumpers), which then becomes a breakout board for the RFM12B. No need to mess with the 2.0 mm pin/pad distance on the RFM12B module itself.

Docs can be found in the Café, and the kit/pcb is now available in the shop, as usual.

▶ View 10 Comments

Uploading? ISP? FTDI? Huh?

In AVR, Hardware, Software on Jul 4, 2010 at 00:01

There seems to be a fair bit of confusion in- and outside the Arduino world, and it’s spilling over to JeeNodes …

I’d like to go through some terms and buzzwords to try and clarify how to get your Arduino or JeeNode to do the thing you want it to do. I’m going to assume that you are familiar with the process of writing software (“code”), compiling it, and running it – at least on a Windows, Mac, or Linux computer (let’s call them all PC’s for now). With software for the ATmega, there are two approaches, depending on whether you use the Arduino IDE or not:

Screen Shot 2010 06 28 at 23.50.22

Both lead to the same result: a “hex” file with code that needs to be transferred from the PC to the ATmega.

The step which can be puzzling when starting out with physical computing and embedded hardware is how to get things across from your PC to that little Arduino or Arduino-like system you’re holding in your hand. And vice versa, since we often want to get results back or see some confirmation that things are working properly.

The confusion comes from the different conceptual levels involved, and things like:

do you mean how to connnect? – plug in a USB cable
do you mean how does the ATmega change its own code? – through a boot loader
do you mean via ISP (In System Programming)? – no, that’s normally not needed
do you mean via FTDI? – yes, that’s the name of the chip which is hooked up to USB
isn’t FTDI a connector? – yeah, that too, sort of… i.e. a 6-pin convention
isn’t a power + serial cable enough? – no, resets also play a key role
do I have to use the Arduino IDE? – no, you can also use a program called “avrdude”
what’s avrdude? – a program which can upload to a boot loader or an ISP programmer
do I need an ISP progranmmer? – nope, the boot loader does essentially the same thing
so why not just get rid of ISP? – because you need ISP to install the boot loader

Confused? Welcome to the club…

In case you’re wondering… the process is called “uploading” because the PC initiates this as outbound transfer:

Screen Shot 2010 06 29 at 01.51.50

To get your code (“sketch” in Arduino-speak) into an ATmega, you need three things, working together to make uploading happen under all possible conditions:

An electrical connection – to power the ATmega, to communicate with it, and to restart the ATmega when it is stuck or busy doing something else.
A common language / protocol – the communication must be understood by both sides, i.e. PC and ATmega.
Software on both sides of the connection – Sending something when the other side isn’t listening, or listening while no-one is sending will not have the desired outcome.

Let’s go through each of these separately.

1. An electrical connection

There are several ways to make the connection. With an Arduino, or any similar board which has a USB connector, you can simply plug in the USB cable:

Screen Shot 2010 06 28 at 23.52.41

Some boards use a separate USB interface (“FTDI adapter”), allowing reuse of that interface for multiple boards:

Screen Shot 2010 06 28 at 23.54.08

The end effect is the same: a connection which powers the ATmega and allows communicating with it using a simple serial protocol. There’s also a trick in this hookup to let the PC reset the ATmega whenever it wants.

2. A common language / protocol

Ah, now it gets interesting. First thing to note is that there is no single common language / protocol!

That’s right. It all depends on what you want to do. Here we want to upload code from the PC to the ATmega. That requires exchanging “ISP commands” over the connection. But once uploading is done, we really want to reuse the connection as a regular two-way serial link.

The way it works is that the PC will reset the ATmega just before uploading new code. This activates a “boot loader” on the ATmega. Now both sides will be in sync (briefly) so they can exchange the necessary information to make an upload happen. IOW, right after reset the protocol is “ISP commands”. Once the upload is done, the connection can be reused for any protocol you like – as determined by the code that was uploaded.

3. Software on both sides of the connection

Back to the software now. We need to send ISP commands over the connection.

As you may have guessed, that’s exactly what the Arduino boot loader on the ATmega understands. When reset, the boot loader in the ATmega gets control. It waits and listens for incoming STK500 ISP commands. If none come in within a second or so, it relinquishes control to whatever was previously uploaded to the ATmega.

On the PC side, we need software which resets the ATmega and then immediately sends all the ISP commands needed to transfer and program the contents of a hex file.

This is what “avrdude” does. You can either use it implicitly from the Arduino IDE by starting an “Upload” from the menu, or you can launch it manually from the command line – avrdude needs a few options to tell it where the USB port is, what baudrate to use, the type of ATmega, the protocol to use (i.e. STK500), etc.

There’s more…

The above describes the different pieces and concepts involved in getting code into an ATmega. The beauty of it is that once it works, it really works well. Supplying power, uploading, communication, control, debugging … all with one simple USB cable. You only need to go back a few years to appreciate just how much this approach simplifies embedded development.

But there’s one important detail: the ATmega has to have a functioning boot loader. Placing a boot loader into an ATmega is a bit more complicated (and involves other things such as “fuses”). It’s a chicken-and-egg problem.

This is where the ISP programmer comes in. An ISP programmer is a fairly simple piece of hardware. In fact, you can make your own, as I’ve described in several posts on this weblog. See this and this post for some quick solutions which require nothing more than a working Arduino or JeeNode.

The good news, is that you usually don’t have to worry about installing the boot loader – it’s all done for you. Once. For the mechanism described above, you’ll never need an ISP programmer.

Some people actually prefer to use the ISP technique for uploading their sketches. In fact, sometimes you have no choice, i.e. when you need the serial port at all times, or when you can’t spare the 1..4 Kb required by the boot loader code, or when working with ATtiny chips which don’t support bootloaders.

In thoses cases, you’ll need a setup with an ISP programmer. But for most people doodling around with ATmega’s and the Arduino IDE, the above boot loader mechanism is usually very convenient and the easiest to get going.

Either way, it helps to understand the process. I hope the above was helpful in that direction.

Fixing a faulty ATmega (Arduino)

In AVR, Hardware, Software on Jul 2, 2010 at 00:01

After a recent post on how to repair an ATmega with a faulty (or even missing) bootloader and helping someone out with it, it occurred to me that this mechanism will work for any Arduino – not just JeeNodes.

Any 3.3V or 5V Arduino’ish system which lets you upload the isp_repair.pde sketch can be used to program any board with an ISP connector on it. The code is for ATmega328’s, because that’s all I use around here these days.

The trick is to hook up a few power and I/O lines in a specific way:

Screen Shot 2010 06 28 at 16.59.11

I’m using an Arduino Pro as example, but that’s just one of many you could use. Now connect these six wires:

ISP pin 1 = BLUE = Digital pin 4
ISP pin 2 = RED = VCC (+5V)
ISP pin 3 = GREEN = Analog pin 0
ISP pin 4 = PURPLE = Analog pin 3
ISP pin 5 = ORANGE = Digital pin 7
ISP pin 6 = BLACK = GND

Note: if your working board operates at 3.3V, then you should connect the RED wire to +3.3V, not +5V (that way signal levels and power supply will match). Also: the target board should not be connected to anything, since it will be powered through the ISP connector.

The only thing left to do is to upload the isp_repair.pde sketch to your working board, and open up a console window. You should see something like this:

That’s it – disconnect all the wires. The ATmega on the target board now has a boot loader and the standard pin 13 blink sketch. Ready again to accept whatever sketch you upload to it!

▶ View 1 Comment

Arduino, Network

EtheRBBBreadboard

In AVR, Hardware, Software on Jun 7, 2010 at 00:01

Please excuse the crazy title. This picture probably tells you more:

Dsc 1509

You’re looking at an RBBB from Modern Device, hooked up to the Ether Card from Jee Labs. The hookup is trivial, it need 6 wires: 2 for power and 4 for SPI:

Screen Shot 2010 05 29 at 04.18.26

And that’s all you need to create a webserver!

Here’s a sample screen (yep, it’s been running for almost 12 hours):

Screen Shot 2010 05 29 at 00.03.48

And here’s the code for it:

Screen Shot 2010 05 28 at 12.05.03

It presents a self-refreshing page with the “uptime”, i.e. how long the server has been running since power-up.

The code is available as “rbbb_server.pde” example sketch in the EtherCard library and is a simplified adaptation of the etherNode sketch, omitting the RF12 driver and calls, of course. It compiles to 6 Kb of code.

As with the etherNode sketch, the MAC address and IP address have to be set to suitable values in the source code before uploading it to the RBBB.

BTW… did you notice anything?

The RBBB is running at 5V. And it works. Because the Ether Card is compatible with 3.3V and 5V signals!

Which also means that the Ether Card can be used with any Arduino board. It’s not specific to JeeNodes and it’s not limited to being used with the Carrier Board, either.

The generic software for the Ether Card is contained in the “EtherCard” library, listed on the software page in the Café. It doesn’t depend on Ports or RF12 libraries, so this thing really is a completely independent product.

It just “happens” to fit gorgeously into a box alongside a JeeNode :)

So there you have it – the Ether Card can be used with just about any microcontroller setup. All it needs is a power supply of 3.6V or more, 4 SPI I/O pins, and the proper software to make it sing of course…

Onwards!

▶ View 7 Comments

Arduino, Reflow

Setting up the thermocouple node

In AVR, Hardware, Software on May 10, 2010 at 00:01

This is part 2 of my reflow controller series. Unlike what was announced yesterday, I’m going to first describe how to set up the temperature sensing wireless node. JeeMon hookup to follow soon.

Our very first step could be to connect the thermocouple via a JeeNode and USB to the PC, but I’m going to do something more interesting and go straight for a wireless hookup. One reason for this is due to a problem with direct connections, but since this is going to be used in un-tethered mode anyway, it’s a good excuse to use a wireless configuration right from the start.

Here is the “thermoSend.pde” sketch I’m going to use (code here):

Screen Shot 2010 05 09 at 14.29.35

This contains all the ingredients needed for a simple basic sensor node: all we do is set up wireless, and then read out the thermocouple value and send it off once a second.

The easy transmissions code works with broadcast packets, so we don’t need to define a destination for the wireless packets, just this sensor node’s ID (1), the choice of frequency band (868 MHz), and a group ID (6). That’s done with the rf12_initialize() call.

The readings are converted to an integer in the range 0 .. 3300, representing a temperature range of 0 .. 330.0°C. Then we send that 2-byte integer in binary mode over wireless. That’s what the code inside loop() does. The map function is part of the Arduino library code.

The easy transmit system will take care of re-transmission if a packet is lost. All the transmission details are handled by rf12_easyPoll(), which needs to be called often.

One detail of the easy transmit system to keep in mind, is that only changed values are sent. No packets will go out if the reading is the same as the last one. In this scenario, that’s fine – there’s always some jitter in the readings, so we should see new packets at least a few times a minute, even when the sensor temperature is constant.

Ok, let’s get going. If you haven’t installed the Arduino IDE yet, go do it now.

Create the above sketch named “thermoSend” in the Arduino IDE. On my Mac, it ends up as a folder named “Documents/Arduino/thermoSend/” in my home directory. Plug in the JeeNode via an FTDI adapter such as the USB-BUB (or use a JeeNode USB) and check your Arduino IDE / Tools / Serial Port settings to make sure you’re hooked up to it. With multiple nodes on USB, it’s easy to mess up the wrong one – happens to me all the time…

Now the node is ready. Unplug, hook it up to a battery, and power it up.

You’re sending packets into the ether now. Quick, let’s try and collect those packets before they falll on the floor!

To do that, hook up a JeeLink or a second JeeNode. It needs to be running the RF12demo sketch, which is what it does out of the box if you got it from Jee Labs or Modern Device.

We need to connect to the JeeLink. Easiest way is to use the Arduino IDE. Make sure you are connected to the proper USB port. So again, check the Arduino IDE / Tools / Serial Port menu and select the JeeLink serial port.

Now open the Arduino IDE’s serial console. You should see something like this:

[RF12demo] A i1 g212 @ 433 MHz
Available commands:
[etc]

You are now in direct communication with the JeeLink. We need to set it up to listen to the proper transmissions. Sort of like tuning to the proper radio station. So type the following line and press “Send”:

2i 8b 6g

This sets node ID = 2, frequency band = 868 MHz, and net group= 6.

If all is well, and the wireless node is powered up, you should start to see packets come in, something like this:

OK 33 212 0
 -> ack
OK 33 209 0
 -> ack
OK 33 212 0
 -> ack
OK 33 209 0
 -> ack
OK 33 212 0
 -> ack

That’s a header byte (i.e. 33, associated with node ID 1) and 2 data bytes.

There won’t be packets every second, because only changed readings are sent. But there will be packets, and if you touch the thermocouple end, you should immediately see the rise in temperature:

OK 33 212 0
 -> ack
OK 33 226 0
 -> ack
OK 33 24 1
 -> ack
OK 33 44 1
 -> ack

No, I’m not suffering from sudden hypthermia: that’s not 4.4°C :) – what you’re seeing is binary overflow of the first data byte. A byte can only hold values 0..255. Anything higher and it will wrap around. That’s why two bytes are sent. The second byte contains the reading divided by 256, i.e. the number of wraparounds. You’re looking at the binary representation of an “int” on Atmel AVR chips.

So the above readings are: 21.2°C, 22.6°C, 28.0°C, and 30.0°C.

IOW, the actual temperature is: (byte1 + 256 * byte2) * 0.1°C

The next episode will be about hooking up JeeMon to the Jeelink and using it to read out the data and do something more meaningful with it. Stay tuned.

Plug shield on Arduino Mega

In AVR, Hardware on Feb 7, 2010 at 00:01

The Plug Shield can also be used on an Arduino Mega:

Note the two extra wire jumpers, since the I2C interface is on pins 20 and 21 on the Mega board.

The above has an RTC plug and an LCD plug hooked up, so let’s to set up a simple clock with this – and use it to demonstrate the brand-new RTClib along the way:

Screen shot 2010-02-05 at 17.22.01.png

I’ve omitted the details of the Wire coding, but you can get the full sketch here.

Fascinating concurrency

In AVR, Software on Feb 4, 2010 at 00:01

There is a new language for the Arduino / JeeNode / ATmega328, called Occam-π.

I found out about it yesterday, at http://concurrency.cc/ – it’s high level, and it supports parallel programming. The current development environment release is for Mac OS X, with Windows and Linux coming soon.

Here is a complete program with 4 blinking LED’s, one on each DIO pin of the JeeNode ports:

Screen shot 2010-02-03 at 01.13.19.png

That’s it. Compiles to roughly 2 Kb. Each extra blink adds just 20 bytes, btw.

And yes, it really makes four LEDs blink at an independent rate:

There is slightly more to it than that, but this is mind-blowing stuff. The “parallelism” is simulated, of course. Looks like the ATmega can do around 6000 context switches per second (i.e. parallel task switches).

There is a roughly 20 Kb interpreter part that needs to be uploaded once (which is why this requires at least an ATmega328). After that, the IDE will upload just the bytecode for your program, i.e. 2 Kb in the above case.

B R I L L I A N T .

Imagine hooking up the RF12 driver to this – there’s plenty of room for the extra 3 Kb or so. And for doing all sorts of things… in parallel! My earlier complaint post about how awful it is to do several things at once on an ATmega board might just have been wiped off the table.

Looks like I’ve got some very serious learning ahead of me to try and get to grips with all this.

▶ View 21 Comments

Pin I/O performance

In AVR, Hardware, Software on Jan 6, 2010 at 00:01

There was a discussion on the Arduino developer’s mailing list about the impact of a small change to the digitalWrite() function, and for some time I’ve been hearing that digitalWrite() has a huge amount of overhead.

Time to find out.

Here is the sketch I used to measure how often a pin I/O command can be issued using various mechanisms:

Screen shot 2010-01-05 at 11.42.53.png

The logic is that I’m counting how often the same command can be called between timer overflows, i.e. every 1024 µs (one byte, incrementing @ 16 MHz / 64), before the timer tick count changes again.

And here’s the sample output:

Screen shot 2010-01-05 at 11.42.14.png

There’s a small amount of jitter, which tells me the loops are syncing up almost exactly on the timer ticks. Interrupts have not been disabled, so the timer interrupt is indeed being serviced – once for each loop.

What these values tell me, is that we can do about:

10 analog 10-bit readings per millisecond with analogRead()
128 pwm settings per millisecond with analogWrite()
220 pin reads per millisecond with digitalRead()
224 pin writes per millisecond with digitalWrite()
1056 pin reads per millisecond with direct port reads
1059 pin writes per millisecond with direct port writes

(I’ve corrected the counts by 1000/1024 to arrive at these millisecond values)

So the Arduino’s digital I/O in IDE version 0017 can do roughly 1/5th the speed of direct port access on a 16 MHz ATmega328.

But WAIT! – There’s a large systematic error in the above calculations, due to the loop overhead. It looks like the loop takes 1024000/1251 = 819 ns overhead, so the actual values are quite different: digitalRead() -> 3712 ns, direct port read -> 151 ns. Now the values are more like 1/25th!

So let’s redo this with more I/O in each loop iteration (all 4 ports):

Screen shot 2010-01-05 at 11.55.31.png

The sample output now becomes:

Screen shot 2010-01-05 at 11.56.59.png

With these results we get: one digitalRead() takes 4134 ns, one direct port read takes 83 ns (again correcting for 819 ns loop overhead). The conclusion being that digitalRead() is 50x as slow as direct port reads.

Which one is correct? I don’t know for sure. I retried the direct port read with 16 entries per loop, and got 67 ns, which seems to indicate that a direct port read takes one processor cycle (62.5 ns), as I would indeed expect.

Conclusion: if performance is the goal, then we may need to ditch the Arduino approach.

Update – Based on JimS’s timing code (see comments): digitalRead() = 58 cycles and direct pin read = 1 cycle.

Update #2 – The “1 cycle” mentioned above is indeed what I measured, but incorrect. The bit extraction was probably optimized away. So it looks like direct pin access can’t be more than 29x faster than digitalRead(). As pointed out by WestfW in the comments, digitalRead() and digitalWrite() have predictable performance across all use cases, including when the pin number is variable. In some cases that may matter more than raw speed.

Update #3 – Another caveat – Lies, damn lies, and statistics! – is that the register allocations for the above loops make it extremely difficult to draw exact conclusions. Let me just conclude with: there are order-of-magnitude performance implications, depending on how you do things. As long as you keep that in mind, you’ll be fine.

▶ View 15 Comments

Arduino Duemilanove

In AVR, Hardware on Dec 30, 2009 at 00:01

Not everything is about wireless. Nor about nodes, ports, or plugs. Of course not – that would be boring :)

To offer more choices, I’ve added the Arduino Duemilanove to the web shop:

Duemilanove_HI.JPG.jpeg

Jee Labs is now an official distributor for these famous boards. Woohoo!

Arduino and complexity

In Hardware, Software on Dec 28, 2009 at 00:01

This is – in a nutshell – the Arduino world:

Screen shot 2009-12-27 at 18.53.04.png

In prose: an ATmega board, some hardware peripherals, an environment for embedded software development, a front-end for visualization, and a website to bring all these facets together for a growing worldwide community.

For each of these, there are alternatives and variations. These add variety and increase the number of choices for everyone interested in this low-cost physical computing world. The “Arduino” name is conquering the world, and it sticks … even as “duino” suffix. Names are a great way to create a brand.

But what is Arduino, really? Is it an ATmega? Is it a board that fits in the palm of your hand? Is it a standard for connecting other boards? Is it a standard way of structuring software? Is it defined by a set of standard libraries? Is it the dual aspect of the IDE, i.e. Wiring vs. Processing? Is it the main web site or the discussion forum?

The answer depends no doubt on who you ask. And if this is a healthy ecosystem, then all of these will evolve and improve over time. In my opinion, the current state of an ecosystem is far less important than its ability to evolve.

That’s where complexity comes in.

It’s not hard to construct a great system, even an elaborate one. But what is extremely hard, is to come up with a system which supports evolution of all the pieces involved. Because with multiple pieces, you have to make decisions. You have to interconnect the pieces, and that requires making choices. And once you do, you reduce the number of future paths, including many you don’t even know about yet.

Making choices can be good. Hooking up an Arduino to a PC via a serial connection is a great low-cost solution for uploading, debugging, and interacting. The transition from an RS232 connection + a separate power supply to a USB connection with “built-in” 5V power is an excellent example of how evolution can lead to substantial progress.

Technological evolution can take years. And as in life, that’s where the really interesting stuff happens.

Unfortunately in the Arduino world, I’m hitting some nasty edges which tell me we need more generality or flexibility. I’ll describe a few, in terms of interfaces between the different pieces shown above:

Voltage levels – the most widely used Arduinos are based on 5V, meaning that all their I/O pins are also 5V-based. More and more sensors can only be used with a supply voltage up to 3.6V or so. Interfacing them to a standard Arduino requires the use of level converters. Often, resistors will do – but you can’t simply ignore the issue. This is an area where I2C can help.

Hardware modules – the Arduino “shield” concept doesn’t extend very far, because each shield determines which pins it needs. Pins are a scarce resource, and each pin location on the connector is fixed. It’s hard to design shields which will actually stack with others. Even more so now that there is a new Arduino Mega with slightly different pin allocations.

Software modules – libraries are a great concept: code which is there when you need it, but doesn’t get in the way (i.e. included) if you don’t. Right now, adding a library which depends on say the “Wire” library means that you have to include the Wire library in your sketch even if you don’t ever use it. This is almost a show-stopper.

C/C++ code – on the surface, programming for the Arduino board is like programming in C and C++, with a bunch of very common issues resolved and a wide variety of useful functions available, ready to use. Newbies can get going fast and old hands can find their way around in no time. Except… the IDE does some funky things with headers and declarations. And it breaks a couple of fairly basic C/C++ conventions.

Automation – the IDE works with a specific scenario in mind. As of the latest 0017 release, multiple sketches and consoles are supported. This solves some of the issues, but it’s hard to automate this further: after uploading, I still have to manually launch the serial console. If I have some other app talking to the hardware, there is no mechanism to upload a new sketch and then resume that app because the serial USB connection is dedicated.

Visualization – it’s interesting to see how the Arduino / Wiring / Processing approach is bridging a huge range of hardware in an integrated manner. But there are several choices involved which reduce flexibility towards entirely different approaches. Some visualizations (as well as other types of data processing) would benefit from using a more internet-/web-centric approach. Lower-end hardware cannot support the Java environment used by Processing, so that code can’t be leveraged. There are alternatives. Scripting language support would be useful.

Don’t get me wrong: all these issues are tough ones! It’s probably impossible to solve them all and to make everyone happy. These issues are definitely not things open source software is obliged to address from day one. But my point is that this is not about where we are today, but about having the mechanisms to evolve to where we want to be tomorrow.

If you’ve followed this weblog for a while then you can probably guess that I’ve got some ideas and suggestions on how to move forward on all this. I would really like to help take the existing Arduino platform further – but I’m not sure at this point that it can be done within the current constraints. I don’t even know whether anyone else considers the issues listed above to be important. All I know is that as I continue to add more software and design new hardware, the “push-back” from the current Arduino design choices is increasing… a lot.

Do I want to change the Arduino world (i.e. make a case, convince people, find compromises) – or do I want to sidestep some of its decisions by starting all the way from scratch? I haven’t decided.

▶ View 3 Comments

Read the rest of this entry »

Re-flashing and ISP

In AVR, Hardware on Dec 11, 2009 at 00:01

This is the second of two posts about everything related to uploading, re-flashing, bootstraps, FTDI, and ISP.

Yesterday’s post described the process of uploading new sketches via USB or RS232 using a boot loader.

But how did that boot loader get into the ATmega328?

That’s a bit of a chicken-and-egg problem. Ya’ can’t use a boot loader to get the boot loader into flash memory!

This is where a lower-level hardware-based mechanism called In System Programming (ISP) comes in. ISP is a clever trick in the ATmega chip which in effect activates a special boot loader built into the hardware. This is done by holding the RESET line of the ATmega low, at which point the chip becomes essentially useless, since the reset prevents the chip from starting to run its firmware. The trick is that in this mode, some of the pins on the ATmega become an interface to that special hardware-based built-in boot loader.

So what we need to do, is to manipulate those pins to send commands which will store data into the flash memory. What we’re going to send in most cases, is the data for the boot loader in upper memory. With that boot loader in place we can then switch to “normal” uploading via the serial interface and USB.

ISP programming is also used to change “fuse bits” on an ATmega. There are a few configuration settings for the ATmega – what type of clock it uses, how it boots up, enabling a watchdog timer, etc. These are stored in “fuses” which can only be adjusted via ISP programming.

There are 3 I/O pins involved in ISP programming, plus the reset line and power. On the Arduino as well as on the JeeNode and JeeLink boards, these lines are available via a 2×3-pin ISP connector with the following layout:

ISP pins.png

The interesting thing is that you don’t even need an Arduino or JeeNode to set up flash memory and fuses. ISP is so low-level that it works directly on the pins of an ATmega chip. This is very useful to pre-load the boot loader (or test program – anything you like, really) onto a chip before soldering it permanently onto a board.

(continued…)

▶ View 4 Comments

Read the rest of this entry »

Uploading and FTDI

In AVR, Hardware on Dec 10, 2009 at 00:01

This is the first of two posts about everything related to uploading, re-flashing, bootstraps, FTDI, and ISP.

First, our goal: what we want to do is get our software (“sketch”) from the PC/Mac into the AVR ATmega328 microcontroller on our Arduino or JeeNode. This is a fairly simple process once all the pieces are in place…

The effect is that the flash memory built into the ATmega is re-flashed. Being flash memory, your software will remain in the microcontroller even without power – ready to start again once power is applied.

There are two ways to get your software into the ATmega: In System Programming (ISP) and uploading. ISP will be described tomorrow, here we’re going to focus on uploading.

Uploading works with a “boot loader”. This is a small piece of software started after power-up (or a reset), which usually listens on the RX and TX pins of the serial interface for commands. These commands then tell it what data to store in which part of the flash memory:

Screen shot 2009-12-09 at 12.25.31.png

The surprising thing is that the boot loader itself is also stored in flash memory. It’s stored in a small (1..4Kbyte) area in upper memory, and the data it writes always goes to the lower part of memory. That’s why it’s called a bootstrap or boot loader – imagine lifting yourself by pulling on the straps of your own boots. In this case the ATmega is lifting its own functionality up by reprogramming its flash memory via its own boot loader.

(continued…)

Plug Shield clock

In Hardware, Software on Nov 30, 2009 at 00:01

As a reminder that not everything here at Jee Labs is about JeeNodes, here’s a clock for the Arduino (which keeps track of time, even when not plugged in):

This was built with an Arduino Duemilanove, a Plug Shield, an RTC Plug, and an LCD Plug piggy-backed onto a 2×16 character display.

Here’s the sketch:

Screen shot 2009-11-29 at 13.12.46.png

And here are links to the PortsLCD.h and PortsLCD.cpp source files.

But there could be some serious trouble ahead…

This code depends on an extended version of the LiquidCrystal library that comes with the Arduino IDE version 0017. Since I don’t want to modify that code, I had to use different names, so the PortsLCD.h/.cpp files use the following class names:

LiquidCrystalBase – an abstract class containing all the generic LCD code from the original class
LiquidCrystalPins – this is for use with plain pin connections, as before
LiquidCrystalPort – this uses bit-banged I2C with one of 4 JeeNode ports
LiquidCrystalI2C – this is for use with hardware I2C, using the Wire library

These names are slightly different from the previous ports-only version, btw.

So PortsLCD is a library which does everything the original did, and more. The flexibility is that you can write your sketches with this and then use any type of LCD you like with it, by just changing a single line of code. And if you’ve got a different hardware hookup, a new class can be added for it based on this same code, so that again your sketch only needs to change a single line of code to use it.

So far, so good. This is the benefit of object-oriented code and polymorphism.

But there is a price, due to the way the Arduino IDE does things: even if you don’t use the Wire library, you’ll need to include it in your sketch! For similar reasons, I’ve been forced to include the RF12 driver in all my demo sketches, even those that don’t use it. Leaving it out leads to build errors and prevents uploading.

This is not a C/C++ issue, the gcc compiler is actually quite smart about leaving out things which are never used. No – this complication is caused by the way in which the Arduino IDE tries to do some clever things to simplify naive use of libraries. I’ll go into this in another post – it is not a show stopper yet, but it may become one as I start combining more features and code, since memory on an ATmega is quite limited.

To put it bluntly: the way the Arduino IDE currently deals with libraries is a ticking time bomb…

▶ View 1 Comment

Updated Plug Shield

In Hardware on Nov 28, 2009 at 00:01

The updated Plug Shield has come in, here pre-assembled and with all the headers soldered on:

This fixes the problem with D9, and adds one extra feature: there is now a pull-up for the IRQ line, with a solder jumper to connect it to D3. See the label “JD3” at the middle right in the above picture. This makes the IRQ pin on the port headers compatible with the JeeNodes when an IRQ pin is needed.

This shield is still called “v1” due to an oversight, but that should not be too much of a problem because only 3 prototypes have been sent out. If you got such an “old” prototype shield and want the final one, get in touch and I’ll send it (pcb-only).

On a related note, the Memory Plug has also been updated (to “v2”, correctly this time). This fixes an addressing limitation with 128 Kbyte EEPROMs.

Onwards!

Arduino?

In AVR, Hardware, Software on Nov 23, 2009 at 00:01

What is it? Hype? Hobby? Hacker stuff? Here is the best overall introduction I’ve seen so far – by Dave Jones. It’s 19 minutes – plenty of time to get used to his accent :)

(view directly on YouTube)

I think he really touches on all the important aspects and inevitable trade-offs.

Me, I use the Arduino bootloader for JeeNodes and JeeLinks all the time, and the Arduino IDE to compile and upload stuff to them, as well as the serial console in many cases. I do use my own editor environment – which is easy to do once you disable Arduino’s built-in one (this is not well documented, Google for “Arduino external editor”). So for me the Arduino is really the bootloader, plus the IDE just as compile/upload system.

As Dave points out, the Arduino is a wrapper around the avr-gcc compiler + avrdude in combination with a convenient USB-based upload/console/power hookup. The rest is libraries, conventions, a Java based IDE (based on Wiring), and optionally a Java-based PC-side front end called Processing.

On the embedded software side, it’s really standard full-scale C and C++.

Which is great, IMO. I can keep going with the Arduino-compatible JeeNodes and JeeLinks, and their built-in wireless, port conventions, 3.3V operation, and all the add-on plugs – knowing that much of this will work fine with as well as without all the stuff going on in the Arduino ecosystem right now.

Arduino plug stack

In AVR, Hardware on Nov 13, 2009 at 00:01

You could combine all these …

… and create an Arduino sandwich like this – with full access to all the Arduino pins:

That’s a Plug Shield with, from left to right / top to bottom:

UART Plug – with a hardware serial port
RTC Plug – with a battery-backed real time clock
Expander Plug – with 8 extra general I/O lines
Memory Plug – with extra EEPROM memory
Pressure Plug – with pressure + temp sensor
LCD Plug – with a 2×16 char LCD display on top

Each of the plugs can be accessed via I2C, i.e. using the Arduino’s “Wire” library.

Would all this work? Yes, I’m pretty certain it would.

Would it be useful? Probably not in this combination…

I just wanted to show off the I2C plugs and shield ;)

Build and pinout errors on older posts

In AVR, Software on Nov 8, 2009 at 00:01

There has been an unfortunate dependency between the Ports library and the RF12 library for some time now, causing the Arduino IDE to generate errors such as these:

(etc…)

The workaround right now is to include both in your sketch, even if you only need one:

I’ve been planning to do a major overhaul of the library and software in general, but until then this is the way to avoid those pesky errors. It probably affects quite a few sample sketches on this weblog.

Another thing to watch out for (thanks, Ian!) is that some older examples on this weblog use the JeeNode v2 or even v1, which have a different pinout. To uses those examples with the latest JeeNodes, you have to swap pins 4 and 5, i.e. +3V and AIO.

Please let me know when examples from earlier weblog posts don’t work as described. A small fix and note added to these posts might be all that is needed!

Meet the Plug Shield

In Hardware on Nov 5, 2009 at 00:01

There are a couple of Arduino’s lying around idly here at the Jee Labs, so I thought it’d be nice to be able to attach some JeePlugs to them. Not all interface tasks require a wireless JeeNode, after all…

The main issue is how to deal with different voltage levels, i.e. 5V logic levels on the Arduino vs 3.3V on all the Jee stuff. Luckily, this is not a problem for an I2C bus, since there are chips which automatically convert between two different I2C bus levels.

So here’s the Plug Shield, carrying an RTC plug:

Check out the gold lettering!

There is room for up to 5 port connectors, all connected in parallel, with right-angle 6-pin female headers (I take straight headers and just bend their pins). This supports two plugs lying flat on the board and three pointing outwards, although many more can be attached through daisy-chaining.

The PWR pin is connected to 5V. The IRQ signal is currently not connected to any pin but could be tied to PD3 to be compatible with JeeNodes.

Note that the Plug Shield will only work with I2C-type JeePlugs – things like the Room Board and the Thermo Plug cannot be used.

There is a regulator to supply 3.3V to the port connectors on-board, as well as an I2C bus level converter. To maximize the space available to plugs, these are all SMD’s. The reset button and ISP header have been brought out as well, since the original ones on the underlying Arduino board cannot be reached.

This shield allows stacking and can be sandwiched between an Arduino and any other shields. The only requirement is that analog pins 4 + 5 must be used for hardware I2C – these are not available as I/O pins.

Here’s a demo, reading out the RTC:

Screen shot 2009-11-03 at 00.32.07.png

Uses the standard “Wire” library that’s included with the Arduino IDE.

There is a wiring mistake in the shield, causing PB1 to be shorted to ground, so as a result digital pin 9 can’t be used with this shield. I placed one of the headers the wrong way around. Doh!

I’ve added this shield in the shop anyway. Got only a handful of ’em for those who don’t care too much about that PB1/D9 issue. New shields will be made soon.

Apart from that, it’s working great. All I2C-based JeePlugs can now be used with standard Arduinos, without plug-stacking conflicts or voltage translation hassles. Want an RTC? LCD? UARTs? More I/O lines? You got it.

Update – The pcb was updated soon after this post, the above mistake only applies to the first few shields.

Arduino Mega + shield

In AVR, Hardware on Jun 28, 2009 at 00:01

Got myself an Arduino Mega – an Arduino on steroids:

Arduino Mega + shield

Compatible with current Arduino shields, but extended to support a lot more pins.

Ordered it from NKCelectronics, along with their new extended shield for it:

Arduino Mega + shield

There’s a lot of them pins on there…

I decided to attach a solder-less breadboard to it, to be able to experiment with projects which need more I/O lines or comms than a standard Arduino:

Arduino Mega + shield

It fits, but just barely. Had to cut off the little hooks on the plastic. Using a breadboard in this way covers up all the identifying pin texts on the silkscreen, alas.

Actually, it doesn’t quite fit – couldn’t get all headers on:

Arduino Mega + shield

I’m using a mix of 4- and 6-pin headers, since I had them lying around. As you can see, the last 4-pin header makes it impossible to fit that other header in, so I left off pins 22 and 24. It might have worked with an 8-pin header i.s.o. 2x 4-pin.

And here’s the final “stack”:

Arduino Mega + shield

Conclusion so far? I’m not sure this is the way to go. I think it’s an abomination, to be honest. There are so many pins to connect that the two boards are very hard to separate once stacked up. You don’t want to pull too hard on one side and end up with bent pins when the thing finally comes apart.

And how often do you need that many I/O pins one one fixed board setup? Sure, the ATmega1280 has 128 Kb of flash memory, which is plenty to get fairly complex sketches going. But it also builds on that trend of using stacked shields – and let’s face it: a single shield is often great, but with 2 or 3 you end up with pin allocation nightmares (for designers) or conflicts (for users). Multi-stacking is tricky, requiring special stacking headers.

Nah, I’d much rather go with ports, and extensions for that, and connecting two or three independent units over a bus if need be. But then of course I would, that’s why I went for such an approach with JeeNodes after all.

Better delays

In AVR, Software on Mar 31, 2009 at 00:01

As described in this post on an Arduino forum, I’ve suggested some small changes to the delay() and delayMicroseconds() code in the Arduino/Wiring libraries:

Picture 3.png

(rest of delayMicroseconds unchanged)

Note that both changes only add new code, no lines have been removed or changed (the comments need to be adjusted, though).

The above changes only affect microsecond delays > 500 us and millisecond delays < 50 ms. In that range, a loop is used which should be accurate in the couple-of-microsecond scale and which does not lock out interrupts.

It should lead to a more accurate millis() value since interrupts can be serviced more often, and to a far more accurate delay for low-millisecond values. I hope something like this makes it into a future release of the Arduino IDE – timing capabilities are still a bit limited on the Arduino, IMO.

Arduino, MuxShield

Multiplexing code for Arduino

In AVR, Software on Mar 25, 2009 at 00:01

Here is an extract of the code to multiplex 5 serial ports in software on an Arduino Duemilanove:

Picture 3.png

It’s a big bag of tricks, really. There is a timer running at 3x the baudrate, which detects start bits and then picks out data bits every three ticks. The crucial issue for the receivers is that the work is done in parallel for 5 input streams, but that these events are not happening at the same time. There are in fact 5 little state machine, each with their own independent state.

For the output, it’s slightly simpler: a buffer with all the 5 transmit bit states is scanned and sent out. The trick here is to fill that buffer with the proper bit patterns. When nothing is sent on a channel, its corresponding bit remains high in all values in the buffer. The transmit buffer has 30 entries, 3 per bit plus the start and stop bits. A simplification here is that all bytes are sent at the same time, i.e. start and stop bits occur at the same time on all active transmit channels.

The idea of multiplexing is that all received data is collected and sent out to the main (hardware) serial port, running at 57600 baud. Extra character codes 0x01 .. 0x05 are inserted into the multiplexed data stream to identify from which channel the data is coming. On the transmit / de-multiplexing side, the character codes 0x01 .. 0x05 are filtered out and used to indicate to which channel to send out the next characters.

If you connect two of these multiplexing shields back-to-back, you can send 5 independent bi-directional serial streams over a single 3-wire cable. Come to think of it, the following setup would make a great test for these multiplexing shields:

Picture 2.png

The full source code for the multiplexing sketch can be found here. It has been tested at 9600 baud (all receivers must run at the same speed). It probably works at higher speeds even, but this hasn’t been tested. At some point, the Atmega chip is going to get swamped while handling the barrage of timer 1 interrupts. I haven’t really tested the transmit part yet, i.e. the de-multiplexing side of things.

Arduino, Sensors

Ports on a standard Arduino

In AVR, Software on Mar 1, 2009 at 00:01

The Ports library described in recent posts works fine with any type of Arduino, Freeduino, etc – not just JeeNodes. It merely uses some conventions for the “DIO” and “AIO” pins of each of the 4 ports.

Here is an example using a HM55B compass module from Parallax, using an Arduino Duemilanove with a Proto Shield from AdaFruit:

Ports on a standard Arduino

There are actually two sensors in the above setup – the bigger sensor is a Parallax H48C 3-axis accelerometer (more on that below). Both need +5V to operate, so they won’t work with simple 4-pin ports on a JeeNode.

Ports are mapped to the Arduino pins as follows:

Port 1 DIO = Arduino digital pin 4 (PD4)
Port 1 AIO = Arduino analog pin 0 (PC0)
Port 2 DIO = Arduino digital pin 5 (PD5)
Port 2 AIO = Arduino analog pin 1 (PC1)
Port 3 DIO = Arduino digital pin 6 (PD6)
Port 3 AIO = Arduino analog pin 2 (PC2)
Port 4 DIO = Arduino digital pin 7 (PD7)
Port 4 AIO = Arduino analog pin 3 (PC3)

The ATmega register bits are listed in parentheses.

Here is the full code of the HM55B driver plus demo:

Picture 2.png

Sample output:

Ports on a standard Arduino

The H48C 3-axis accelerometer demo is very similar, source code can be found here. Sample output:

Ports on a standard Arduino

These two hookups both use some new utility code in the Ports library to shift a specified number of bits in and out of the DIO line while clocking the AIO line. Note also that these sensors needs two ports each, since they both use 3 IO lines for their SPI-like interface.