One of the most useful fallout’s from the acceptance of Agile techniques is the use of Test-Driven-Development (TDD) and the growth of associated test frameworks, such as GoogleTest and CppUTest, etc.
I won’t get into the details of TDD here as they are well covered elsewhere (I recommend James Grenning’s book “Test Driven Development for Embedded C” for a good coverage of the subject area), but the principle is
- Write a test
- Develop enough code to compile and build (but will fail the test)
- Write the application code to pass the test
- repeat until done
Obviously that is massively simplifying the process, but that’s the gist. The key to it all is automation, in that you want to write a new test and then the build-deploy-test-report (BDTR) cycle is automated.
To build a TDD environment with the mbed I needed to solve the following obstacles:
- Build – Using a TDD framework and building the project
- Deploy – Download to the mbed
- Test – auto-executing the test code on the mbed
- Report – Getting test reports back from the mbed to the host
A couple of years ago we came across an excellent TDD framework for embedded C call Unity. I have written about it before and how to get it up and working with an embedded target. With the mbed now capable of native development, it was quite easy to set up a Unity based project using the ARMCC compiler (GCC ARM coming).
Unity uses Ruby scripts to auto-generate the application main file and function. The ruby script is run against your test file, e.g.
Which creates the file myTest_Runner.c that contains the main function.
Next is to build the project including the unity files, the test code and the application code. This involved knocking together a rudimentary Makefile, using the ARMCC compiler and the appropriate flags and include paths, e.g.
AS = C:\Keil\ARM\ARMCC\bin\armasm.exe
AFLAGS = –cpu Cortex-M3 -g –apcs=interwork -I C:\Keil\ARM\RV31\INC -I C:\Keil\ARM\CMSIS\Include -I C:\Keil\ARM\Inc\NXP\LPC17xx –xref
CC = C:\Keil\ARM\ARMCC\bin\armcc.exe
CFLAGS=-c –cpu Cortex-M3 -g -O0 –apcs=interwork –split_ldm
IFLAGS=-IC:\unity\src -I C:\Keil\ARM\RV31\INC -I C:\Keil\ARM\CMSIS\Include -I C:\Keil\ARM\Inc\NXP\LPC17xx
LD = C:\Keil\ARM\ARMCC\bin\armlink.exe
LFLAGS = –strict –scatter “TDD.sct” –summary_stderr –info summarysizes –map –xref –callgraph –symbols –info sizes –info totals –info unused –info veneers
The output from the build is a file tdd.axf, where AXF stands for “ARM Executable Format”. This is the generic term for the formatted (not pure binary image) output files produced by the ARM Linker. However, the mbed requires plain binary (bin) format. Luckily this is very simple as there is converted as part of the ARMCC toolchain:
C:\Keil\ARM\ARMCC\bin\fromelf –bin -o build\tdd.bin tdd.axf
So now we have our binary executable, tdd.bin.
Deployment is the easiest step as the mbed (on a Windows machine) is mounted as a drive [in my case F:/], so it is simply a case of copying tdd.bin to F:/.
This where I was initially stumped; normally to get a new program to run on the mbed you have to physically press the reset button, at which time the mbed picks up the latest .bin image and executes it. Hummm, what to do…
While looking through some code from the guys on the mbed team (thanks to Emilio) it looked as if you could prompt the mbed to reset over the serial line, so I popped off an email to Emilio:
> what causes the mbed to run the copied bin file when running tests?
> Is it the serial.sendBreak()?
Yes, exactly, it is treated as a “reset target” command from the interface chip.
Eureka – I feel a (quick & dirty) Python script coming on :
bin = 'build\\tdd.bin' disk = 'F:\\' copy(bin, disk) sleep(1.5) serial = Serial('COM8', timeout = 1) serial.flushInput() serial.flushOutput() serial.sendBreak()
Unity reports the test result back, by default, using putchar (unity_internals.h):
but as I’d already rehosted the standard I/O library on the mbed, then this didn’t need modifying.
So now I can write the tests and report the output back over the mbed USB serial port. Adding to the Python I could simply echo the test reults to the console (but opens up many other possibilities):
while True: c = serial.readline() sys.stdout.write(c) sys.stdout.flush()
Finally I needed to exit the loop once the tests had either passed or failed. Unity finishes with either a “OK” or “FAIL” message at the end of all the tests:
so, adding the following to break out of the loop:
while True: c = serial.readline() sys.stdout.write(c) sys.stdout.flush() if c.find('OK') != -1: break if c.find('FAIL') != -1: break
Unfortunately this didn’t work as expected; if an individual test passes it reports “PASS”, but if it fails it reports “FAIL”, causing my Python script to exit early. However, one of the nice things about Unity is you have the source (there are only two .h files and one .c file for Unity), and I could modify the final message from “FAIL” to “FAILED”, so the script only exits once all tests (pass or fail) have been executed.
Putting it all together
I ended up using a batch file to pull this all together, not pretty but just to prove it all works (test.bat):
C:\unity\auto\generate_test_runner.rb myTest.c make C:\Keil\ARM\ARMCC\bin\fromelf --bin -o build\tdd.bin tdd.axf python build\copy.py
where copy.py did the deploy-run-report part.
So now the TDD cycle is:
Write a new test in myTest.c
Write the framework code for the test (e.g. empty application functions)
run test.bat and see test fail
Write application code to pass test
and see test pass
repeat until done
Good question. So what does this give me? As I’ve always stated I think the mbed is a great little platform for rapid prototyping, especially IoT based systems.
TDD is very much a state of mind and can be quite difficult to really grasp, but using Unity in the context of the mbed, I believe, will help you see what TDD can (and cannot do).
Automation of the BDTR (build-deploy-test-report) cycle means you can move away from using IDEs and rely less on target based debugging. You’ll find this can significantly improve productivity and also force you to plan you design – not a bad thing.
Finally it shows that Python is fast becoming a useful tool, even for the embedded C programmer.
There are a number of ways I’m planning to take this forward:
GCC ARM – this involves rehosting GCC stdio
Using Python subprocess calls rather than a batch file for the full BDTR cycle
Using XML to configure the Python script (e.g. get away from COM8 and F:\ in the file)
Use Jenkins open source continuous integration server to drive the tests
but any other suggestions welcome. Once I’ve cleaned up the project I’ll host it on github as before so keep an eye on https://github.com/organizations/feabhas
- TDD with Compiler Explorer - August 13, 2020
- Side effects and sequence points; why volatile matters - April 16, 2020
- Running the eclipse-mosquitto MQTT Broker in a docker container - February 20, 2020