Black Magic Probe

The Black Magic Probe (BMP) is a great debugging tool for ARM Cortex µCs. This is a description of how to create one using a low-cost “Blue Pill” board.

The BMP can upload firmware to a wide range of µCs over JTAG or SWD, as well as connect to a serial port for simple communication. It also supports gdb debugging, e.g. single-stepping, breakpoints, and examining memory.

The BMP is supported by PlatformIo, my go-to tool for all embedded µC development (see example at end): BMP + PIO = an amazing combo!

Here is a target Blue Pill (left), being prepared for use with a BMP (right):

This simple 4-pin female header “adapter” allows uploads and debugging. For serial, 6 pins are needed. For RESET / SWO, 2 more pins must be connected.

The 8-pin header layout is compatible with the 6-pin header on a “HyTiny” board, another low-cost STM32F103 board (from Haoyu):

It’s a somewhat arbitrary choice, but as good as any - most boards differ in their pin choices anyway. One solution will be described further down below.

There is no 5V power on the header, to avoid accidental damage. Only 3.3V, with limited current capability (around 50 mA). Boards which need more current or other voltages have to be powered in other ways.

Chicken-and-egg

The problem with turning a Blue Pill (BP) into a Black Magic Probe (BMP) is that there’s no convenient way to program that BP until you have a BMP! The solution chosen here, is to get an (equally cheap) “ST-Link” clone, which has far less functionality than a BMP but makes a very useful fallback option:

If all else fails, that ST-Link can also help restore a broken or damaged BMP.

The plan

Here are the steps needed to reach that magical setup which the BMP offers:

  • get a Blue Pull and an ST-Link (from Aliexpress, for example)
  • install the “stlink” command-line utility
  • download the BMP firmware (or build it from scratch)
  • upload the BMP bootstrap using stlink
  • upload the BMP main firmware using stlink
  • solder an 8-pin female header to the proper pins

And for completeness, since BMP’s µC support continues to evolve:

  • install the “dfu-util” command-line utility
  • this allows upgrading a BMP to a newer version without any other tools

The stlink utility by user “texane” on GitHub, is an open source tool for STM’s “ST-Link” hardware, of which there are several generations by now.

ST-Link V2 is based on the common STM32F103 µC and has been widely cloned for quite some time now. It’s very easy to find “ST-Link sticks” on eBay and on Chinese sites (they all come from China anyway).

Unfortunately, these ST-Links don’t expose their serial port UART pins, so they can’t be turned into a BMP without losing some of its most useful features. But they can be used to re-program a Blue Pill, which is the next best option.

See the README for Win/Mac/Linux installation instructions. It’s very easy.

BMP firmware

There are two approaches to getting the BMP firmware: lazy and long-term. The lazy option is to download the build files I’ve collected here:

This is a perfectly usable snapshot, but it may not be the very latest version.

The long-term way is to clone the BMP repository and build the firmware yourself. This allows building updates with a simple git pull. This approach is easy if you have the “arm-gcc-embedded” toolchain installed and in your PATH.

Click here to see a transcript for MacOS

Upload to the Blue Pill

The firware consists of two pieces:

  • blackmagic_dfu.bin is the DFU bootstrap, loaded at 0x08000000
  • blackmagic.bin is the BMP code itself, loaded at 0x08002000

Both need to be uploaded, and that’s where the stlink utility comes in:

st-flash --reset write blackmagic_dfu.bin 0x8000000
st-flash --flash=128k write blackmagic.bin 0x8002000

After the first upload, one LED on the BP will be blinking. When the second is done, both LEDs will be on. At this point, the ST-Link can be disconnected, and the Blue Pill is now running as a new Black Magic Probe.

The BMP code extends past 64 kB, which conflicts with the STM32F103C8 µCs on Blue Pills, as they reportedly only contain 64 kB of flash memory. In reality, all BPs have 128 kB flash (maybe not quite in spec, who knows). The solution is to force the upload anyway, hence the --flash flag.

Click here for some more MacOS details ...

Solder the header

The last step is to create some sort of mechanical structure to connect an 8-pin header (or whatever other convention you want to adopt). Here’s an idea, which takes advantage of the fact that most connections are on one side:

This mounts a small protoype board (a JeePlug in this case) on top.

The connections are as follows (see also this README):

# Function Pin Blue Pill Notes
1 SRST PB4 P4 / 9
2 GROUND G P4 / 2 always needed
3 JTCK/SWCLK PA14 P2 / 3
4 JTMS/SWDIO PA13 P2 / 2
5 UART1 TX PB6 P4 / 7
6 UART1 RX PB7 P4 / 6
7 3.3 V 3.3 P4 / 1 if powering target
8 SWO/RX2 PA3 P3 / 7
JTDI PA15 P4 / 11 not needed for SWD
JTDO PB3 P4 / 10 not needed for SWD

Where P2 is the 4-pin SWD header, and P3/P4 are the two rows of 20-pins.

Upgrade a BMP via DFU

As mentioned earlier, the firmware in the BP consists of two parts: a DFU boot loader, and the BMP software itself.

DFU stands for Device Firmware Upgrade and is a standard mechanism for uploading new code to a µC over USB. Many more modern µCs include this mechanism in ROM, but on older STM32F103 it’s part of the BMP firmware.

The BP can be put in DFU mode by changing a jumper and then plugging it in:

Click here for some more MacOS details ...

The board wil power up with a rapidly-blinking LED to indicate DFU mode. Go to the directory with upgraded firmware and run dfu-util as follows:

dfu-util -v -R -d 0483:df11 -s 0x08002000 -D blackmagic.bin

This utility uses -D (download?) to upload to the BP - very confusing!

The homepage for the open-source dfu-util is on SourceForge. On MacOS, it can be installed using HomeBrew, i.e. brew install dfu-util.

After the upgrade, the jumper must be moved back to its default “0” position.

The DFU mechanism can be tricky if more DFU-enabled devices are currently present, in which case an easier solution may be to revert to using an ST-Link and simply reflashing the firware as described before.

Pluggable variations

Since each board from different vendors seems to use another convention, no single header choice is going to suffice. One way of addressing this, is with a “socket + plug” system, with - yet another! - interconnect convention. What we need is a connector with 8 .. 12 pins, and various short cables which plug in and end in the proper board connector:

Here with a small selection of home-fabricated / hot-glued cable plugs:

The “connector” is based on “machined-pin IC sockets”. These have round pins, with the special property that the pins coming out will easily fit in another socket. By soldering wires to a second socket, we create a plug. But there are more ways to combine these:

The left one can be used to add pins to the bottom of a protoboard PCB, and the right one allows using only once side, when dual-in-line is not needed.

The pinout of the main socket attached to the Blue Pill supports a range of plugs using just 8 of the pins. The extra pins are useful to support full JTAG, for example. In the example above, a 16-pin socket was used, but 14 pins would have been more than enough.

Here is the selected pinout so far, with 5 pins still unused on a 16-DIP:

16P 14P 8P Function Pin Blue Pill
1 1 1 GROUND G P4 / 2
2 2 2 JTCK/SWCLK PA14 P2 / 3
3 3 3 JTMS/SWDIO PA13 P2 / 2
4 4 4 3.3 V 3.3 P4 / 1
5 5 - SWO/RX2 PA3 P3 / 7
6 6 -
7 7 -
8 - -
9 - -
10 8 -
11 9 - JTDI PA15 P4 / 11
12 10 - JTDO PB3 P4 / 10
13 11 5 5.0 V 5V P4 / 2
14 12 6 UART1 TX PB6 P4 / 7
15 13 7 UART1 RX PB7 P4 / 6
16 14 8 SRST PB4 P4 / 9

The 8-DIP pinout is sufficient for SWD, RESET, serial, and 3.3V or 5V power.

With good-quality machined-pin headers, these home-made connectors are surprisingly robust (with the help of hot glue) and easy to insert and remove. Some custom PCBs would be nice … maybe some day.

Example project with BMP

Here is a complete example of how to use this Black Magic Probe to upload firmware to a HyTiny, and view output sent over the serial port. I’m using MacOS, PlatformIO, and the JeeH library, but apart from USB device names (COM, /dev/ttyUSB, etc), this should carry over to any platform & dev setup.

1. PlatformIO

PlatformIO can be used on the cmd-line or as IDE plugin. Pick your poison. It’s a Python script which wraps up everything, including getting all the tools and toolchains needed. I’ll use the command-line mode as example here.

On MacOS, the command-line “core” version of PlatformIO can be installed with a one-liner, using Homebrew: brew install platformio.

2. Project setup

We need to create a folder with two files in it: a project file and a source file:

$ mkdir blink blink/src
$ cd blink
$ cat >platformio.ini
[env:blink]
platform = ststm32
board = hy_tinystm103tb
build_flags = -DSTM32F1
framework = stm32cube
upload_protocol = blackmagic
upload_port = /dev/cu.usbmodemDEE8C3AD1
monitor_port = /dev/cu.usbmodemDEE8C3AD3
monitor_speed = 115200
lib_deps = jeeh
^D
$ cat >src/main.cpp
#include <jee.h>

UartDev< PinA<9>, PinA<10> > console;
PinC<13> led;

int main() {
    console.init();
    enableSysTick();
    led.mode(Pinmode::out);

    while (true) {
        led.toggle();
        console.putc('.');
        wait_ms(500);
    }
}
^D
$

That’s it, but the project file needs some clarification if you’re new to PIO:

  • The board identifies the exact µC type, in this case an STM32F103TBU6.
  • The framework can be stm32cube, but also arduino, or libopencm3. It determines what runtime gets used.
  • The BMP connects as two sub-devices: the one ending in “1” (on MacOS) is the JTAG port, and the one ending in “3” is the serial port. On Windows, these will be two different COMx ports, on Linux two different /dev/ttyUSBx devices.
  • It’s not essential to name the ports in the project file, but then PIO will ask during each upload, which gets boring very quickly.
  • This example uses the JeeH library, which is automatically retrieved via the PIO “registry”, in this case https://platformio.org/lib/show/3082/JeeH.

3. Build

Build with pio run (if using the command line, else in the IDE). The first time, a lot of output will be generated, something like:

$ pio run
Processing blink (platform: ststm32; board: hy_tinystm103tb; framework: stm32cube)
-----------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/ststm32/hy_tinystm103tb.html
PLATFORM: ST STM32 6.0.0 > Tiny STM103T
HARDWARE: STM32F103TBU6 72MHz, 20KB RAM, 128KB Flash
DEBUG: Current (blackmagic) External (blackmagic, jlink, stlink)
PACKAGES:
 - framework-stm32cube 2.0.181130
 - toolchain-gccarmnoneeabi 1.70201.0 (7.2.1)
[...]
Linking .pio/build/blink/firmware.elf
Building .pio/build/blink/firmware.bin
Checking size .pio/build/blink/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [          ]   4.2% (used 852 bytes from 20480 bytes)
Flash: [          ]   0.7% (used 952 bytes from 131072 bytes)
=============== [SUCCESS] Took 2.44 seconds ===============
$

The next time around, fetching tools and libraries will be skipped.

4. Upload

This is done with pio run -t upload (which also rebuilds, if needed):

$ pio run -t upload
Processing blink (platform: ststm32; board: hy_tinystm103tb; framework: stm32cube)
[...]
Configuring upload protocol...
AVAILABLE: blackmagic, dfu, jlink, serial, stlink
CURRENT: upload_protocol = blackmagic
Looking for BlackMagic port...
Use manually specified: /dev/cu.usbmodemE2C2B4DF1
Uploading .pio/build/blink/firmware.elf
Target voltage: ABSENT!
Available Targets:
No. Att Driver
 1      STM32F1 medium density M3/M4
0x080001ac in main ()
Loading section .isr_vector, size 0x10c lma 0x8000000
Loading section .text, size 0x3b8 lma 0x800010c
Loading section .init_array, size 0xc lma 0x80004c4
Loading section .fini_array, size 0x4 lma 0x80004d0
Start address 0x80003b4, load size 1236
Transfer rate: 4 KB/sec, 309 bytes/write.
Section .isr_vector, range 0x8000000 -- 0x800010c: matched.
Section .text, range 0x800010c -- 0x80004c4: matched.
Section .init_array, range 0x80004c4 -- 0x80004d0: matched.
Section .fini_array, range 0x80004d0 -- 0x80004d4: matched.
Kill the program being debugged? (y or n) [answered Y; input not from terminal]
=============== [SUCCESS] Took 2.24 seconds ===============
$

5. Serial I/O

The serial communication takes place on a second port:

$ pio device monitor
--- Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at http://bit.ly/pio-monitor-filters
--- Miniterm on /dev/cu.usbmodemE2C2B4DF3  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
.............

One dot every half second, as expected.

The fact that the BMP uses separate ports for JTAG and serial can be very convenient, because that serial connection can be left running in a separate window, while we build and upload code changes as needed.