Test Driven Development (TDD) with the mbed

One of the most useful fallout’s from the acceptance of Agile techniques is the use of Test-Driven-Development (TDD) and the growth of associated test frameworks, such as GoogleTest and CppUTest, etc.

I won’t get into the details of TDD here as they are well covered elsewhere (I recommend James Grenning’s book “Test Driven Development for Embedded C” for a good coverage of the subject area), but the principle is

1. Write a test
2. Develop enough code to compile and build (but will fail the test)
3. Write the application code to pass the test
4. repeat until done

Obviously that is massively simplifying the process, but that’s the gist. The key to it all is automation, in that you want to write a new test and then the build-deploy-test-report (BDTR) cycle is automated.

To build a TDD environment with the mbed I needed to solve the following obstacles:

1. Build – Using a TDD framework and building the project
2. Deploy –  Download to the mbed
3. Test – auto-executing the test code on the mbed
4. Report –  Getting test reports back from the mbed to the host

Build

A couple of years ago we came across an excellent TDD framework for embedded C call Unity. I have written about it before and how to get it up and working with an embedded target. With the mbed now capable of native development, it was quite easy to set up a Unity based project using the ARMCC compiler (GCC ARM coming).

Unity uses Ruby scripts to auto-generate the application main file and function. The ruby script is run against your test file, e.g.

C:\unity\auto\generate_test_runner.rb myTest.c

Which creates the file myTest_Runner.c that contains the main function.

Next is to build the project including the unity files, the test code and the application code. This involved knocking together a rudimentary Makefile, using the ARMCC compiler and the appropriate flags and include paths, e.g.

AS = C:\Keil\ARM\ARMCC\bin\armasm.exe
AFLAGS = –cpu Cortex-M3 -g –apcs=interwork -I C:\Keil\ARM\RV31\INC -I C:\Keil\ARM\CMSIS\Include -I C:\Keil\ARM\Inc\NXP\LPC17xx –xref

CC = C:\Keil\ARM\ARMCC\bin\armcc.exe
CFLAGS=-c –cpu Cortex-M3 -g -O0 –apcs=interwork –split_ldm
IFLAGS=-IC:\unity\src -I C:\Keil\ARM\RV31\INC -I C:\Keil\ARM\CMSIS\Include -I C:\Keil\ARM\Inc\NXP\LPC17xx

LD = C:\Keil\ARM\ARMCC\bin\armlink.exe
LFLAGS = –strict –scatter “TDD.sct” –summary_stderr –info summarysizes –map –xref –callgraph –symbols –info sizes –info totals –info unused –info veneers

The output from the build is a file tdd.axf, where AXF stands for “ARM Executable Format”. This is the generic term for the formatted (not pure binary image) output files produced by the ARM Linker. However, the mbed requires plain binary (bin) format. Luckily this is very simple as there is converted as part of the ARMCC toolchain:

C:\Keil\ARM\ARMCC\bin\fromelf –bin -o build\tdd.bin tdd.axf

So now we have our binary executable, tdd.bin.

Deploy

Deployment is the easiest step as the mbed (on a Windows machine) is mounted as a drive [in my case F:/], so it is simply a case of copying tdd.bin to F:/.

Test

This where I was initially stumped; normally to get a new program to run on the mbed you have to physically press the reset button, at which time the mbed picks up the latest .bin image and executes it. Hummm, what to do…

While looking through some code from the guys on the mbed team (thanks to Emilio) it looked as if you could prompt the mbed to reset over the serial line, so I popped off an email to Emilio:

> what causes the mbed to run the copied bin file when running tests?

> Is it the serial.sendBreak()?

 Yes, exactly, it is treated as a “reset target” command from the interface chip.

Eureka –  I feel a (quick & dirty) Python script coming on :

 bin = 'build\\tdd.bin'
 disk = 'F:\\'
 copy(bin, disk)
 sleep(1.5)
 serial = Serial('COM8', timeout = 1)
 serial.flushInput()
 serial.flushOutput()
 serial.sendBreak()

Report

Unity reports the test result back, by default, using putchar (unity_internals.h):

but as I’d already rehosted the standard I/O library on the mbed, then this didn’t need modifying.

So now I can write the tests and report the output back over the mbed USB serial port. Adding to the Python I could simply echo the test reults to the console (but opens up many other possibilities):

while True:
    c = serial.readline()
    sys.stdout.write(c)
    sys.stdout.flush()

Finally I needed to exit the loop once the tests had either passed or failed. Unity finishes with either a “OK” or “FAIL” message at the end of all the tests:

so, adding the following to break out of the loop:

 while True:
    c = serial.readline()
    sys.stdout.write(c)
    sys.stdout.flush()
    if c.find('OK') != -1:
       break
    if c.find('FAIL') != -1:
       break

Unfortunately this didn’t work as expected; if an individual test passes it reports “PASS”, but if it fails it reports “FAIL”, causing my Python script to exit early. However, one of the nice things about Unity is you have the source (there are only two .h files and one .c file for Unity), and I could modify the final message from “FAIL” to “FAILED”, so the script only exits once all tests (pass or fail) have been executed.

Putting it all together

I ended up using a batch file to pull this all together, not pretty but just to prove it all works (test.bat):

 C:\unity\auto\generate_test_runner.rb myTest.c
 make
 C:\Keil\ARM\ARMCC\bin\fromelf --bin -o build\tdd.bin tdd.axf
 python build\copy.py

where copy.py did the deploy-run-report part.

So now the TDD cycle is:

1. Write a new test in myTest.c
2. Write the framework code for the test (e.g. empty application functions)
3. run test.bat and see test fail
4. Write application code to pass test
5. run test.bat
6. and see test pass
7. repeat until done

So what?

Good question. So what does this give me? As I’ve always stated I think the mbed is a great little platform for rapid prototyping, especially IoT based systems.

TDD is very much a state of mind and can be quite difficult to really grasp, but using Unity in the context of the mbed, I believe, will help you see what TDD can (and cannot do).

Automation of the BDTR (build-deploy-test-report) cycle means you can move away from using IDEs and rely less on target based debugging. You’ll find this can significantly improve productivity and also force you to plan you design –  not a bad thing.

Finally it shows that Python is fast becoming a useful tool, even for the embedded C programmer.

Plans

There are a number of ways I’m planning to take this forward:

1. GCC ARM –  this involves rehosting GCC stdio
2. Using Python subprocess calls rather than a batch file for the full BDTR cycle
3. Using CMake
4. Using XML to configure the Python script (e.g. get away from COM8 and F:\ in the file)
5. Use Jenkins open source continuous integration server to drive the tests

but any other suggestions welcome. Once I’ve cleaned up the project I’ll host it on github as before so keep an eye on https://github.com/organizations/feabhas

• About
• Latest Posts

Niall Cooling

Director at Feabhas Limited

Co-Founder and Director of Feabhas since 1995.
Niall has been designing and programming embedded systems for over 30 years. He has worked in different sectors, including aerospace, telecomms, government and banking.
His current interest lie in IoT Security and Agile for Embedded Systems.

Latest posts by Niall Cooling (see all)

• Navigating Memory in C++: A Guide to Using std::uintptr_t for Address Handling - February 22, 2024
• Embedded Expertise: Beyond Fixed-Size Integers; Exploring Fast and Least Types - January 15, 2024
• Disassembling a Cortex-M raw binary file with Ghidra - December 20, 2022

Niall Cooling

Website | + posts

Co-Founder and Director of Feabhas since 1995.
Niall has been designing and programming embedded systems for over 30 years. He has worked in different sectors, including aerospace, telecomms, government and banking.
His current interest lie in IoT Security and Agile for Embedded Systems.

• Niall Cooling

  https://blog.feabhas.com/author/feabhas/

  Navigating Memory in C++: A Guide to Using std::uintptr_t for Address Handling
• Niall Cooling

  https://blog.feabhas.com/author/feabhas/

  Embedded Expertise: Beyond Fixed-Size Integers; Exploring Fast and Least Types
• Niall Cooling

  https://blog.feabhas.com/author/feabhas/

  Disassembling a Cortex-M raw binary file with Ghidra
• Niall Cooling

  https://blog.feabhas.com/author/feabhas/

  Using final in C++ to improve performance