Register for our webinar – ‘Introduction to Docker”Introduction to Docker’

Posted on October 24, 2018 by Shelley Robertson

Dec 5, 2018 at 10am BST & 4pm BST

The introduction to Docker series is proving popular with our Blog readers, so we have decided to make it the subject for our next webinar.

Docker is a relatively new technology, only appearing just over five years ago. It has become integral to modern continuous integration (CI) and continuous delivery in an Agile world.

In this 45 minute webinar, presented by Niall Cooling, he will introduce Docker and how it can be used in an embedded development workflow. There will also be time for questions.

If you’d like to submit an advance Docker-related question for Niall to include in the webinar, please let us know. You can submit your question when you register or by emailing us info@feabhas.com. We hope you can join us.

Click here to register and reserve a free place for the 10am BST webinar

Click here to register and reserve a free place for the 4pm BST webinar

Posted in Agile, training, webinar | Leave a comment

An Introduction to Docker for Embedded Developers – Part 5 Multi-Stage Builds

Posted on October 11, 2018 by Niall Cooling

Following on from the previous post, where we spent time reducing the docker image size, in this post I’d like to cover a couple of useful practices to further improve our docker image:

  1. Copying local files rather than pulling from the web
  2. Simplifying builds using a multi-stage build

Copying in Local Files

So far, when installing the GCC-Arm compiler, we have pulled it from the web using wget. This technique can suffer from two issues:

  1. Web links are notoriously fragile
  2. https adds complexity to the packages required with smaller base images such as Alpine-linux

An alternative approach, especially if you are managing your Dockerfiles in a git repository, is to pull the required file (e.g. gcc-arm-none-eabi-6-2017-q2-update-linux.tar.bz2) to your local file system and then copy this file into the docker image during the build process.

First we need to download to our local filesystem the version of GCC-Arm we want to use. The latest version can be found at: https://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads

As of today, the latest version is 7-2018-q2-update.

I happen to be working on a Mac, but as our image is Linux based, I want to download the Linux 64-bit image gcc-arm-none-eabi-7-2018-q2-update-linux.tar.bz2.

Once downloaded, the local (build) directory contains two files:

.
├── Dockerfile
└── gcc-arm-none-eabi-7-2018-q2-update-linux.tar.bz2

We now modify the Dockerfile to copy from the local file system into our base image using the following command:

COPY <local file> <destination>

So the command (the trailing ‘.’ is to the current container working directory):

COPY gcc-arm-none-eabi-7-2018-q2-update-linux.tar.bz2 .

will copy the zip file from our local file system into the container. We can now go ahead and un-tar it and configure it as before, e.g. Continue reading →

Posted in Agile, ARM, C/C++ Programming, Testing | Tagged , | 5 Comments

Updated: Developing a Generic Hard Fault handler for ARM Cortex-M3/Cortex-M4 using GCC

Posted on September 21, 2018 by Niall Cooling

The original article was first posted back in 2013. Since posting I have been contacted many times regarding the article. One re-occuring question has been “How do I do this using GCC?”. So I thought it was about time I updated the article using GCC.

GNU Tools for ARM Embedded Processors

The original article used the Keil toolchain, here I am using arm-none-eabi-gcc. One of the major benefits of CMSIS is that almost all the code from the original posting will compile unchanged as CMSIS uses conditionals to replace instructions where necessary.

However, note that some of the file names have changed since that original article, e.g.

#include "ARMCM3.h"

as a file no longer exists. Its contents have been split across a number of headers in the latest CMSIS. In addition, typically for a build, you will be building against a specific platform. In my case I’m targetting an STM32F4xx core.

In my project “ARMCM3.h” has been replaced with “cmsis_device.h” which maps on the the STM32F411.

From Keil to GCC

The code changes only occur when we use assembler to help dump the processor registers as part of the Hard Fault handling. As expected, inline assembler is specific to a toolchain.

The original Keil code was:

void Hard_Fault_Handler(uint32_t stack[]);

__asm void HardFault_Handler(void) 
{
  MRS r0, MSP
  B __cpp(Hard_Fault_Handler) 
}

The same code for GCC is:

Update
Thanks to @raz3l for helpfully commenting on the need for the use of the GCC naked attribute when using optimisation setting. The attribute allows the compiler to construct the requisite function declaration, while allowing the body of the function to be assembly code.

void Hard_Fault_Handler(uint32_t stack[]);

__attribute__((naked)) void HardFault_Handler (void)
{
  asm volatile(
      " mrs r0,msp    \n"
      " b Hard_Fault_Handler \n"
  );
}

Register Dump Analysis

Continue reading →

Posted in ARM, C/C++ Programming, CMSIS, Cortex | Tagged , , | 4 Comments

Technical debt

Posted on August 22, 2018 by Glennan Carnie

What is it & how does it affect software engineering management?

The ‘Golden Triangle’ of project management

The ‘golden triangle’ of project management uses the following constraints:

The rule is: you can pick any two of three; you can’t have them all.

When it comes to software development projects, it’s not uncommon to have a fixed time to market and budget, which means that, under pressure, the constraint that’s affected is quality.

Commonly, when the project management refers to ‘quality’ it implicitly means Intrinsic Quality.

Intrinsic quality and technical debt

Intrinsic Quality is the inherent ‘goodness’ of a system. That is, not what the product is/does, but how well it has been designed and constructed. If you like, Intrinsic quality is a measure of the engineering rigour that has been put into the product.

In the case of software-oriented products Intrinsic quality tends to manifest itself in architectural robustness and resilience to change.

Intrinsic quality is closely allied to the idea of Technical Debt.

Technical debt is a term created by Ward Cunningham in 1992, which describes “the obligation that a software organisation incurs when it chooses a design or construction approach that’s expedient in the short-term but that increases complexity and is more costly in the long-term.”(1)

A company will put effort into the design and architecture of their systems to give them greater flexibility in the future. Engineering rigour, and designing for change and maintainability reduces (but cannot complete eliminate, unfortunately) the impact of technical debt. That is, the higher the Intrinsic quality of a product, the less it will cost to maintain, modify and extend it during its life.

Note Intrinsic quality benefits the development organisation, and is largely invisible to the customer; thus very few customers are willing to pay for such work. Intrinsic quality is therefore an upfront cost to the development organisation, which has to be balanced against the reduced future costs of product maintenance.

If time-to-market and cost are fixed constraints in a project it is compelling to sacrifice the costs of engineering intrinsic quality.

Sacrificing intrinsic quality for short-term expediency must come at a (future) price. There’s no such thing as a free lunch! The challenge becomes calculating what the future cost will be.

The cost of technical debt

You can think of Technical Debt as a compound interest charge: it’s not only the length of time the ‘debt’ is held that’s a factor, but also the ‘interest rate’. This ‘interest rate’ isn’t fixed; and varies depending on where the compromises are made.

Technical debt affects all aspects of the software engineering process: including requirements and deployment to the user base, writing the code and the tools used to analyse code and modify it.(2)

Problem domain technical debts – that is, customer-facing omissions, compromises, failures, etc. – will (obviously) have the highest ‘interest rates’.

Architectural debts will have the largest effect on product extensibility, flexibility, maintainability, and so incur a high ‘interest rate’.

Coding issues – semantic errors, unit test failures, algorithmic inefficiencies – are the easiest to measure and categorise, so these areas tend to get the most attention. However, the ‘interest rate’ of such technical debts is relatively low, meaning issues can persist for long periods without significant impact on the overall technical debt of the product.

The ‘unknown-unknowns’

However, it’s not just the quality aspects or features that we know have been compromised in order to meet the cost/time constraints that must be counted as technical debt, The ‘unknown unknowns’ – that is, the things we don’t know we don’t know (made famous by former Secretary of Defence Donald Rumsfeld) – becomes a factor here too. The more unknown-unknowns there are in a domain, the easier it is to not factor them in. As a result, anything not-factored-in early enough also becomes a technical debt.

Take the following statistics by Tom de Marco (3). The chart shows the root cause of bugs in a typical software project

A couple of points worth noting here:

The smallest number of bugs can be traced to coding errors. Technical debts in this area have the lowest ‘interest rates’. Contrastingly, the largest number of bugs can be traced to requirements issues. These problem domain issues have the largest ‘interest rates’ for technical debt. Thus a typical software project accumulates its debts at the very highest rates of interest!

Evidence suggests that as developers move away from their ‘core’ skill – that is, the one they practice the most (writing code), the more unknown unknowns they are subject to. The chart then is also a pretty good indicator of ‘unknown unknowns’ in a project. The more ‘unknown unknowns’ the more likely it is the developer will make mistakes (and introduce bugs).

How much is my technical debt?

In 2012, researchers conservatively estimated that for every 100 KLOC (thousand of lines of code), an average software application had approximately US$361,000 of technical debt – that is, the cost to eliminate the structural-quality problems that seriously threatened the application’s business viability. (4)

5 steps to managing technical debt

1. Identify the technical debt – for example, applying the Swiss Cheese model to your system verification and validation (see below)

2. Measure the technical debt in terms of benefit and cost – Thinking of technical debt as compound interest, then the benefit is the amount of money you save by paying off the ‘loan’ early. The tricky bit is establishing what the ‘interest rate’ is for your organisation.

3. Prioritise the technical debt – identify the items that identify the items that have the highest payoff and repay them first. Of course, you can only prioritise technical debts that you can see. The dichotomy here is that the aspects most likely to have the highest technical debts are the ones you can’t currently see (the unknown unknowns)!

4. Repay the technical debt through refactoring – you can only refactor code successfully if you have adequate testing in place. That is, every restructuring change you make can have no impact on the (measurable) functionality of the system. Establishing and automating (where possible) verification and validation regimes for your project is an intrinsic quality exercise. And remember: the sacrificial lamb of project management is intrinsic quality! Companies with rampant technical debt tend to lack these regimes, thus exacerbating the problem by raising the ‘interest rate’ of their technical debt.

5. Monitor items that aren’t repaid – because their cost or value might change over time (certain technical-debt items can escalate to the point of becoming unmanageable). Once again, we can only monitor things we know are (currently) wrong. It is difficult to monitor unknown unknowns!

The Swiss Cheese Model and Technical Debt

There is no one, perfect, way to identify technical debts. Using multiple, independent techniques (each playing to their own strengths) is far more effective.

The “Swiss Cheese” approach to identifying Technical Debt uses multiple techniques, each with a different focus. The techniques are applied with the clear knowledge that no technique is perfect (nor should it be) but the flaws in any one technique do not overlap (much!) with the flaws of another layer.

  • The Static Analysis layer in the model identifies ambiguity and mistakes in codification. These are things that are difficult for engineers to spot, but easy to fix. Static Analysis tools are readily available and have a low cost to apply regularly on a code base. However, Static Analysis cannot identify incorrect algorithms or missing code and the debts it resolves are relatively tiny.
  • The Testing layer verifies system correctness. Since it focuses on failures (deviations from customer specification), technical debts are visible and obvious to the organisation.
  • The Review layer validates requirements and designs. It asks the questions: “Are we solving the right problem?”; “Are we solving it in the right way?” As review is a human-centric activity, tools typically help very little, beyond some metrics such as: cyclomatic complexity; or compile-time coupling, for example. As a result, the technical debts established by reviews are generally larger-scale, more ‘expensive’ and require far more effort (and money) to resolve.

Summary

Understanding Technical Debt is a critical part of software development project management. Sacrificing project intrinsic quality to expediate project delivery has to be very carefully balanced against the long-term costs of maintenance, extensibility, flexibility and re-use. Since the lifetime of a system could potentially extend into decades the costs of not removing Technical Debts could become untenable to the viability of the system.

Code-level restructuring / refactoring, whilst always beneficial, have the smallest beneficial impact. The higher ‘interest rates’ of Technical Debts associated with architectural problems typically far out-shadows the benefits gained from code-level fixes.
As a result, in order to be effective, engineers should be trained in software architecture, software design and even requirements analysis (5). All these topics are far more sophisticated that writing code and it takes time and effort to develop appropriate skills.

References

  • [1] https://www.construx.com/10x_Software_Development/Technical_Debt/
  • [2] Reducing Friction in Software Development – Paris Avgeriou, University of Groningen, Philippe Kruchten, University of British Columbia, Robert L. Nord and Ipek Ozkaya, Software Engineering Institute, Carolyn Seaman, University of Maryland, Baltimore County. Published by the IEEE Computer Society, 2016. https://ieeexplore.ieee.org/document/7367977/
  • [3] Structured Analysis and System Specification, De Marco T, Yourdon Press ISBN 0-138-54380-1, 1978
  • [4] B. Curtis, J. Sappidi, and A. Szynkarski, “Estimating the Principal of an Application’s Technical Debt,” IEEE Software, vol. 29, no. 6, 2012, pp. 34–42.
  • [5] A. Martini, J. Bosch, and M. Chaudron, “Investigating Architectural Technical Debt Accumulation and Refactoring over Time: A Multiple-Case Study,” Information and Software Technology, Nov. 2015, pp. 237—253.
Posted in General | Tagged , , | Leave a comment

Code Quality – Cyclomatic Complexity

Posted on July 27, 2018 by Niall Cooling

In the standard ISO 26262-6:2011 [1] the term “complexity” appears a number of times, generally in the context of reducing or lowering said complexity.

There are many different ways of defining “complexity”, for example, Fred Brooks, in his 1986 landmark paper, “No Silver Bullet — Essence and Accidents of Software Engineering” asserts that there are two types of complexity; Essential and Accidental. [2]

Rather than getting into esoteric discussion about design complexity, I’d like to focus on code complexity.

Over the years, I have found one metric to be the simplest and most consistent indicator of code quality – Cyclomatic Complexity. This is often also referred to as the “McCabe Metric”, after Tom McCabe’s original paper in 1976 “A Complexity Measure” [3]. 

It’s not perfect, it can be fooled and the complexity measure can differ slightly from tool to tool (as the original work was analysing Fortran for the PDP-10). But, given that caveat, it has major value to act as “raising a red flag” if code goes beyond certain thresholds. In addition, it is easily incorporated as a “quality gate” into a modern CI/CD (Continuous Integration/Continuous Delivery) build system.

Cyclomatic Complexity (CC)

Very simply, the metric is calculated by building a directed graph from the source code. It is done on a function-by-function basis and is not accumulative (i.e. the reported value is just for that function).

Given the following code:

void ef1(void);
void ef2(void);
void ef3(void);
   
void func(int a, int b)
{
  ef1();
  if(a < 0) {
    ef2();
  }
  ef3();
}

A directed graph of the code would look thus:

The Complexity is measured using the very simple formula (based on graph theory) of:

v(G) = e – n + 2p

where:

 e = the number of edges of the graph.
n = the number of nodes of the graph.
p = the number of connected components

However, as we are analysing just functions and not a collection of connected graphs, then the formula can be simplified to

v(G) = e – n + 2

as P = 1 representing the entry and exit nodes of a function.

In this simple example v(G) is equal to two. Running Lizard, an open-source Cyclomatic Complexity Analyzer, the result (shown as CCN) is as expected:

$ docker run --rm -v $(pwd):/usr/project feabhas/alpine-lizard lizard
================================================
  NLOC    CCN   token  PARAM  length  location  
------------------------------------------------
       8      2     30      2       8 func@6-13@./main.c

In these examples I’m running Lizard from a docker container (as this fits with our Jenkins build system). However Lizard can be installed locally using pip install lizard if you have both Python and pip installed.

Continue reading →

Posted in Agile, C/C++ Programming, General, Testing | Tagged , , | Leave a comment

Your handy cut-out-and-keep guide to std::forward and std::move

Posted on April 26, 2018 by Glennan Carnie

I love a good ‘quadrant’ diagram.  It brings me immense joy if I can encapsulate some wisdom, guideline or rule-of-thumb in a simple four-quadrant picture.

This time it’s the when-and-where of std::move and std::forward.  In my experience, when programmers are first introduced to move semantics, their biggest struggle is to know when (or when not) to apply std::move or std::forward.  Usually, it’s a case of “keep apply std::move until it compiles”.  I’ve been there myself.

To that end I’ve put together a couple of a simple overview quadrant graphics to help out the neophyte ‘mover-forwarder’.  The aim is to capture some simple rules-of-thumb in an easy-to-digest format.

Disclaimer:  these diagrams don’t address every move/forwarding use.   They’re not intended to.  That’s why we have books, presentations and long rambling articles on the topic.

Continue reading →

Posted in C/C++ Programming | Tagged , , , , , , , | 2 Comments

Setting up Sublime Text to build your project

Posted on April 12, 2018 by Glennan Carnie

For many years embedded development was dominated by complex integrated development environments (IDEs) that hide away all the nasty, messy details of a typical embedded software project.

Recently, with the rapidly accelerating adoption of agile techniques in embedded systems, there has been a move away from integrated development environments towards smaller, simpler, individual tools.  Tools like CMake, Rake and SCons are used to manage build configurations.  Container facilities like Docker provide lightweight environments for build and test.  And developers are free to use their code editor of choice (and let’s face it: the “best editor” is as close to a developer’s heart as the “one-true-brace-style”)

And on.  And on.

As the title suggests in this article we’re going at how to integrate the Sublime Text editor with the SCons build tool, to make developing a little more elegant and seamless.

Continue reading →

Posted in Agile, C/C++ Programming, General | Tagged , , , , | 1 Comment

“May Not Meet Developer Expectations” #77

Posted on February 15, 2018 by Glennan Carnie

Question:  Does the following compile?

int func()
{
  int (func);
  return func;
}

Continue reading →

Posted in C/C++ Programming | Tagged , , , , , | 3 Comments

Bugs do matter…(unsurprisingly)

Posted on January 18, 2018 by Niall Cooling

Welcome to 2018! How did that happen?

Thank you to everyone who attended last week’s webinar on “Measuring Software Quality“, and thank you for the positive feedback, it really does help us shape our future webinars/blogs.

During the talk, I discussed a suggestion by Sally Globe,  along the lines of “Bugs don’t matter”/”Perfect software is the enemy of rapid deployment” as long as you are “Not wrong long”, which came from an initial exchange on Twitter back last year:The caveat was this didn’t apply to safety-critical (which I think we definitely all agree with!). As I wasn’t at the talk, then some of the intent may be lost in translation.

However, as I see it, one of the major problems is that it assumes not only can we “fix it fast” (the goal of Continuous Delivery), which should be applauded (especially in these times of IoT and security vulnerabilities), but that someone notices before it is a major problem (think zero day vulnerabilities).

As a simple example; I happen to be a keen cyclist (yes a fully paid-up member of the MAMIL clan and #bloodycyclist). This week is the start of the 2018 cycling calendar with the Tour Down Under (TDU).  In 2018, the televised highlights in the UK are on a relatively new channel: FreeSports (Freeview/BT/TalkTalk 95, Sky 424 and FreeSat 252) as opposed to the usual EuroSportUK. Woohoo…(if that’s your bag of course)But guess what? Come 1AM, no cycling, so anyone who recorded it got one hour of random sport (none of it cycling), and then a further hour of the TDU. One minor problem, when the transmission ended there was still 59km to go, so no one saw the finish (usually pretty important)! Guess FreeSport’s excuse? software of course!

Okay, so not a major catastrophe in the big scheme of things, but if it was your project, would you be happy telling the powers-that-be “We weren’t wrong long”?

So the concept of “Not wrong long”; I guess it can work if:

  1. You can react quickly AND someone notices QUICKLY, or
  2. You have a monopoly, you say sorry and can blame the software, and
  3. You’re not controlling elements in the “real” world

I am a big fan of automation of the Test/Integration cycle using tools such a Jenkins and Docker (see previous posts). Assessing the potential for Continuous Delivery is a foundation of becoming more agile; but at the same time, we must always be wary of generalised and sweeping statements that have come from people not working in the embedded space.

Posted in Agile, Industry Analysis, Testing | Tagged , | Leave a comment

Exceptional fun!

Posted on December 21, 2017 by Glennan Carnie

In this article I want to look at some applications for one of C++’s more obscure mechanisms, the function try-block.

Continue reading →

Posted in C/C++ Programming | Tagged , , , , , , | 3 Comments