micro:bit C toolchain explained, part 1. ninja & cmake.

I use the offline build tools from Lancaster University for compiling and building C-code for the BBC Micro:bit board. I used the toolset without really understanding what was going on under the hood. I spent some happy time fixing this. Understanding our tools gives us a better chance of fixing things when they fail. I hope to pass on what I learned here.

So you have written some C code to go onto your micro:bit. Great. Now how does your beautiful prose get converted into the executable (often called a binary) file that can be copied onto the micro:bit?

The components of the toolchain that are unleashed upon your unsuspecting C code are shown below. Let’s start at the bottom.

Offline tool chain for building an executable from C code for the micro:bit

The ‘file to go to micro:bit’ is the end product of the chain of build tools and the Gnu C compiler (called gcc) shown above. This is the file that you copy to your micro:bit and is automatically executed when you power up the micro:bit. Let’s talk about gcc.

gcc

gcc stands for the Gnu C Compiler. If we write a simple ‘hello world’ C program contained in a single file to run on your workstation, telling your C compiler (in our case gcc) how to build the executable can be done in a single command. For example, the command gcc -o hello hello_world.c creates an executable file called ‘hello’ from a C program in a single file called hello_world.c, using gcc to do the compiling and linking. It gets more complicated when there are multiple files in the project. As we are using the micro:bit, there will be multiple files as many C files need to be brought into your project to enable the code to be built to run on the micro:bit hardware.

gcc is used to do the compiling and linking of your C code and the supporting libraries that are needed to enable the code to run on the micro:bit. gcc comes as standard with most distributions of Linux. But something has to tell gcc what files how to assemble the general mish-mash of interdependent files and libraries. This is what ninja does. ninja is a ‘build tool’. Let’s discuss what build tools are.

build tools

To understand the rest of the blocks in the toolchain, we need to have a little chat about build tools. A build tool looks for instructions in a text file, which is often called a build file on how to build ‘something’. That something could be how to use a compiler to compile and link files written in C, or for indeed another language. That something could also be how to build the instruction text file for a different build tool. This , which will then go on to do ‘something else’.

The build file contains a list of rules, such as the flags and controls to run the compiler with. It also contains a list of build statements which say how to build each of the files necessary to produce the output.

There are many different build tools. The micro:bit toolchain uses several. ninja, cmake, yotta and make are all build tools. Each of these tools looks for a text file specific to that tool. Here’s a little table for each build tool command and the corresponding build file that it works with.

build toolbuild file
yottayotta_config.json
cmakeCMakeLists.txt
ninjabuild.ninja

A build tool can often be used for building more than one type of language or thing. Let’s make build tools clearer by looking at how we use the ninja build tool to build a simple C project.

ninja

In the micro:bit toolchain, the ninja build tool uses a build file called build.ninja to instruct gcc on how to compile and link all of the project’s C files. Entering ‘ninja’ at the command line causes the contents of this file to be processed by the ninja build system. The build.ninja file contains details of the interdependencies of the files that are used to create the binary that we want to load onto the micro:bit.

Let’s look at how ninja could be used with a simple three file C project outside of the micro:bit system. Consider a simple example project, split across three files. The three files are called array.c, display.c and display.h. This example runs on your workstation, not on the micro:bit. I’m trying to keep it simple to show what ninja does and what a simple build.ninja file looks like. We will use this example to help explain some of the other build tools as well.

array.c

#include <stdio.h>
#include "display.h"

int main() {
    int v[5] = {1,2,3,4,5};
    display_arr(v, 5);
    return 0;
}

display.h

#ifndef DISPLAY_H
#define DISPLAY_H

void display_arr(int *arr, int len);

#endif

display.c

#include <stdio.h>
#include "display.h"

void display_arr(int *arr, int len) {
    for (int count=0; count<len; count++) {
        printf("arr[%i] = %i\n", count, arr[count]);
    }
}

We can see that the files depend on each other. To build these three files into a single executable we need to tell gcc how each file depends on other files. An example build.ninja file that will compile and link the three example files to create an executable called ‘array’ is shown below.

build.ninja

cc  = gcc
cflags = -Wall -g
rule compile
    command = $cc $cflags -c $in -o $out

rule link
    command = $cc $in -o $out

build array.o: compile array.c
build display.o: compile display.c
build array: link array.o display.o

default array

Putting all of these files into the same directory and typing ninja will create the following output:

[3/3] gcc array.o display.o -o array

You will find a new file called ‘array’ in the same folder. This is an executable file. Typing array at the command line gives this unexciting output:

arr[0] = 1
arr[1] = 2
arr[2] = 3
arr[3] = 4
arr[4] = 5

ninja is a relatively recent build tool. Traditionally the make build tool is used to build C programs. I will cover this tool in part 2 and show how this tool can be used to automate loading your code onto the micro:bit. However, with the micro:bit, ninja is used here. I’ve read that ninja offers about a 10x increase in speed of building the executable for large projects compared with make.

In our example above, I wrote the build.ninja file by hand. But how do we create this build.ninja file for our project? As I mentioned, building an executable for the micro:bit requires a bunch of C files and libraries to be compiled and linked with the source code that we write.

At the top of the ninja website, it states that ninja ‘is designed to have its input files generated by a higher-level build system’. In our case, this is done using the CMake build tool. So CMake produces the build.ninja file. Let’s have a look at CMake.

CMake

CMake is used to create the build.ninja file. That’s right – we are using one build tool to create the build file for another build tool to do something. There is a lot of this going on in the micro:bit build system. CMake can generate the build instruction files for a variety of build tools. Type cmake --help and look at the list of generators at the bottom. CMake uses the Ninja generator to generate the build.ninja file.

Let’s use CMake to create a build.ninja file for our previous example C code. In the directory with the C files, create a file called CMakeLists.txt with the content below.

CMakeLists.txt

project(array) 
add_executable(array array.c display.c)

Two lines only! Create a directory called build_ninja and cd into this. Then type CMake .. -G "Ninja". The “..” part will look for the CMakeLists.txt file in the directory above where you are now, then use the contents to create the build.ninja file. You will see a lot of new files are created including a build.ninja file.

The build.ninja file I wrote by hand earlier contains 10 non-blank lines. The new build.ninja file created by CMake contains 95 non-blank lines! Granted, a lot of them are comments, but still…. One of the first lines in the new build.ninja file is include rules.ninja. This file contains an extra 44 non-blank lines. So we have moved from 10 lines to a total of 95 + 44 = 139 lines. I included rules.ninja as a build file in the diagram at the top of this page as this file becomes part of the build.ninja file.

The next obvious question is ‘how is the CMakeLists.txt file created for the micro:bit build system’. This is done by yotta.

End dribble

Right, this is enough for one post. In the next part which I will link to here once it is ready, we will discuss yotta and the directory structure of the micro:bit C projects.

Additional reading

One good website that helped me figure out the programming toolchain, about 2/3 of the way down the article is https://hackernoon.com/the-first-video-game-on-the-bbc-micro-bit-probably-4175fab44da8.

Eclipse, yotta, C/C++ and the BBC Micro:bit

With the help of the excellent instructions at the link below I set up Eclipse with yotta to compile C code for the BBC Micro:bit under Linux:
http://flames-of-code.netlify.com/blog/microbit-cpp-3/
I get the debugger window to come up, but have not yet used this feature in anger.
The writer, achary, clearly knows more about Eclipse and embedded programming than I do. I got a little stuck at a couple of stages so created this page to pass on my solutions.

The offline C compiler for the BBC Micro:bit is developed at Lancaster University. Installing the yotta compiler and downloading example files is explained here.

The rest of this article assumes you followed the instructions on this installation guide and have cloned the microbit-samples directory. I assume that you have Eclipse installed, either the C/C++ installation or you have installed the C/C++ development environment.

Installing yotta

Instructions for installing yotta can be found in the yotta documentation here. Note that yotta is designed for Python2.7 only. I used pip to install yotta to my user directory, using the –user flag. Then I started getting errors:

‘module’ object has no attribute ‘X509_up_ref’

I faffed around upgrading my cryptography library as mentioned in the yotta documentation. Long story short, the recently installed ‘yotta’ and ‘yt’ commands in ~/.local/bin/yt and ~/.local/bin/yotta both referred to python 3. I probably did something sometime to cause this. To fix the error I changed ~/.local/bin/yt and ~/.local/bin/yotta from:

#!/usr/bin/python3.6

import yotta
yotta.main()

To:

#!/usr/bin/python2.7

import yotta
yotta.main()

Get yotta to work in debug mode

Using yotta with the –debug-build flag allows for easier debugging. By default, the code is compiled in an optimised mode which makes debugging harder. I need all of the help that I can get, so would like to use this flag. The command to run yotta with the –debug-build flag  is:

yotta build --debug-build

However, this will throw an error and the last line of the build will give the error:

ninja: build stopped: subcommand failed.

I am not sure what a ninja is doing in my system. If I could see him, I would probably already be dead.

To fix this error, the ‘-fomit-frame’ flag needs adding at two places in the file yotta_targets/mbed-gcc/CMake/Platform/mbedOS-GNU-C.cmake in your microbit-samples directory.

The two changes to mbedOS-GNU-C.cmake are:

line 21 from:
set(CMAKE_C_FLAGS_DEBUG_INIT “-g -gdwarf-3”)
to:
set(CMAKE_C_FLAGS_DEBUG_INIT “-g -gdwarf-3 -fomit-frame-pointer”)
line 28 from:
set(CMAKE_ASM_FLAGS_DEBUG_INIT “-g -gdwarf-3”)
to:
set(CMAKE_ASM_FLAGS_DEBUG_INIT “-g -gdwarf-3 -fomit-frame-pointer”)

Then remove the build directory in the microbit-samples directory and rebuild using:

yotta build --debug-build

If you don’t remove the old build directory directory, then the command will still fail. I know this.

Install pyOCD

We use the pyOCD tool to help debug and program the microbit. Details of this tool are on its github page.

‘pyOCD is an Open Source python 2.7 based library for programming and debugging ARM Cortex-M microcontrollers using CMSIS-DAP. Linux, OSX and Windows are supported.’

Note the ‘python 2.7’ bit. Initially I pip installed it, which defaulted to a python 3 install. The install worked, but when I came to try running pyocd, I got a  bunch of assertion errors. So I read the instructions…

To ensure that pyocd is installed using python2.7, I used:

pip2 install --pre -U --user pyocd

–pre # ‘Include pre-release and development versions. By default, pip only finds stable versions.’ from https://pip.pypa.io/en/stable/reference/pip_install/#install-pre
-U # same as –upgrade ‘Upgrade all specified packages to the newest available version.’
–user # libraries go to the user directory. In linux, this removes the need for sudo to install, which is a security issue.

Plugin your microbit.

Now we fire up a gdbserver using the newly installed pyocd. What is a gdbserver? This wikipedia page explains that ‘gdbserver is a computer program that makes it possible to remotely debug other programs.’

sudo ~/.local/bin/pyocd-gdbserver -t nrf51 -bh -r

-t # target (nrf51 is the chipset used on the microbit).
-bh # replace software breakpoints with hardware breakpoints.
-r # halt the target when reset.

I get this output:

INFO:root:DAP SWD MODE initialised
INFO:root:ROM table #0 @ 0xf0000000 cidr=b105100d pidr=2007c4001
INFO:root:[0]&amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;e00ff000: cidr=b105100d, pidr=4000bb471, class=1&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;
INFO:root:ROM table #1 @ 0xe00ff000 cidr=b105100d pidr=4000bb471
INFO:root:[0]&amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;e000e000:SCS-M0+ cidr=b105e00d, pidr=4000bb008, class=14&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;
INFO:root:[1]&amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;e0001000:DWT-M0+ cidr=b105e00d, pidr=4000bb00a, class=14&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;
INFO:root:[2]&amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;e0002000:BPU cidr=b105e00d, pidr=4000bb00b, class=14&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;
INFO:root:[1]&amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;f0002000: cidr=b105900d, pidr=4000bb9a3, class=9, devtype=13, devid=0&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;
INFO:root:CPU core is Cortex-M0
INFO:root:4 hardware breakpoints, 0 literal comparators
INFO:root:2 hardware watchpoints
INFO:root:Telnet: server started on port 4444
INFO:root:GDB server started at port:3333

There is a way to set up a file in udev to remove the need to use sudo to run pyocd. Please see this link on how to do this.

The guide I read recommended using the –persist flag with the pyocd-gdbserver command. I found my serial port communication with the Micro:bit stopped working. Instead of there being a single serial port connection to the Micro:bit, there were several. I suspect the –persist flag kept ‘zombie’ connections alive, causing my code to connect to a dead connection that only existed in the OS’ imagination.

–persist # keep GDB server running even after remote has detached.

As awac explains, this shows that we have access to 4 hardware breakpoints. The server is started at port 3333. This port will be entered into the Eclipse debugger setup, which is explained below.

Set up Eclipse

http://flames-of-code.netlify.com/blog/microbit-cpp-3/ covers setting up a C/C++ Eclipse project with the microbit-samples code downloaded from the Lancaster University github. I am using Eclipse Oxygen at the time of writing this post.

A new project is set up by using File, New, Makefile Project with Existing Code.

Alter the default build command from ‘make’ to ‘yotta build –debug-build’. Get to this by right clicking on the microbit-samples project and selecting ‘Properties’.

Use control-B to build the code. I get a bunch of depreciation warnings that can be ignored.

dynamic exception specifications are deprecated in C++11 [-Wdeprecated]

Configuring the Eclipse C/C++ debugger for use with yotta

The pyOCD site mentions that the plugin ‘Eclipse Embedded Systems Register View’ should be installed. This took me a little while to figure out how to install, so I created a separate post on how to do this here. This plugin is not yet of use for the microbit. I hope to get CMSIS-SVD configuration files from Nordic for the microbit’s microcontroller to be able to make use of the plugin.

I came a little unstuck when setting up the debug session as I could not find the ‘GDB Hardware Debugging’ option when editing my debug configuration. To get that option we need to install the GNU MCU Eclipse plugin from the Eclipse Marketplace. Go to the Help menu and click on ‘Eclipse Marketplace’. Put ‘gnu mcu’ as the search term to find the plugin. This plugin adds the ‘GDB Hardware Debugging’ option to your run configurations which we use when setting up Eclipse.

This allows me set up for the GDB debugger as shown in the screen shots below. Go to Run, Debug configurations.

Double-clicking on the heading ‘GDB Hardware Debugging’ and set up the Main, Debugger and Startup tabs as shown below. The C/C++ application is found in the ‘build’ subdirectory, in the microbit-samples directory at:

microbit-samples/build/bbc-microbit-classic-gcc/source/microbit-samples

The Debugger tab specifies the path to the GDB :

/usr/bin/arm-none-eabi-gdb

Enter port ‘3333’, which we noted earlier when starting the gdb server.

In the Startup tab, click on ‘Load image’ and ‘Use File’. Enter

${workspace_loc:/microbit-samples/build/bbc-microbit-classic-gcc/source/microbit-samples-combined.hex}

Initially I had some errors:

Reset command not defined for device 'Generic TCP/IP'

Looking at this Stackoverflow question, I fixed this by also unchecking Reset & Delay and Halt options in the debugger configuration:

Running the Eclipse debugger

Run the command ‘yt clean’ from the command line in your microbit-samples directory to clean out the last build. Then ‘control-b’ in Eclipse to create a fresh build. I feel I should be able to click on the little bug icon on the menu bar to get to the debugger, but this gives me a ‘launch failed. Binary not found.’ window. So I right click on the project and select ‘Debug As’,’Debug Configurations’, ‘microbit-samples Default’ then click on the ‘Debug’ button on the bottom right. I get offered the chance to go to the debug screen, which is a result.

Getting the BBC Micro:bit radio to work with the mbed online C/C++ compiler

This blog explains how to get the example programs for working with the non-Bluetooth radio on the BBC Micro:bit to compile correctly using the Mbed online C/C++ compiler.

Short story

Two options:

1 Place the line:

#define MICROBIT_BLE_ENABLED 0

in the MicroBit.h library and forget about the config.json file.

Or

2 Create an mbed_app.json file instead of the config.json file with this content:

{
"macros": [ "MICROBIT_BLE_ENABLED=0" ]
}

Long story

 
The Mbed online compiler and the yotta offline compiler for the BBC Micro:bit are explained at the Lancaster University github site here:

https://lancaster-university.github.io/microbit-docs/

I couldn’t get the example radio programs supplied with the online Mbed C/C++ compiler to work with the BBC Micro:bit. These programs did work with the yotta offline compiler. It took a while to figure out that the config.json file supplied with the examples is being ignored by the Mbed online compiler. The BBC Micro:bit has a custom radio setup which does not work when Bluetooth is enabled. The compiler needs to be told that Bluetooth is disabled. In the examples supplied for both the yotta offline compiler and for the mbed online compiler this is done using a config.json file containing:

{ 
    microbit-dal:{
        bluetooth:{
            enabled: 0 
        } 
     } 
}

The example programs are called simple-radio-rx and simple-radio-tx. For the Mbed online compiler, these can be found at:

https://os.mbed.com/teams/microbit/code/microbit-simple-radio-rx/

https://os.mbed.com/teams/microbit/code/microbit-simple-radio-tx/

For the offline yotta compiler the same programs and config.json files can be found at:

https://github.com/lancaster-university/microbit-samples/tree/master/source/examples

The hex files created using the Mbed online compiler resolutely refused to do anything when I loaded them onto the microbits. I figured out that the the config.json file was being ignored by the Mbed online compiler. To disable the Bluetooth through code, place this line:

#define MICROBIT_BLE_ENABLED 0

in the MicroBit.h library. After doing this, the hex files produced by compiling the example programs using the Mbed online compiler ran correctly.

I posted this on the Mbed questions site and an Mbed moderator said that the issue will be fixed:

https://os.mbed.com/questions/79592/BBC-Microbit-how-to-use-the-radio/?compage=1#c29069

A while later, a helpful guy on Stackoverflow advised me to use an mbed_app.json file instead of the config.json file with this content:

{
    "macros": [ "MICROBIT_BLE_ENABLED=0" ]
}

Setting up yotta and C with the BBC Micro:bit by modifying the examples directory

I set up to program the BBC Micro:bit (which I’ll call the microbit from now on) in C/C++ under Linux. I’ve been using micropython to program the boards up to now, but want to leverage the increased performance that using C can give and some of the C libraries that are available for e.g. encryption.

There is good documentation on the Lancaster University microbit github page:

https://lancaster-university.github.io/microbit-docs/

I installed the offline tools as I spend a lot of time working at sea where you can’t always rely on having an internet connection. I got a demo compiled and loaded by following the instructionso n the Lancaster University github site. Now it was time to write my own code. I had a little trouble getting this going, so here’s what I had to do to get my first program compiled and loaded. I did this by modifying the structure of the examples directory that was created by following the github instructions.

I downloaded the github repository linked above into

~/git/microbit-samples

Initially I made a new directory under:

~/git/microbit-samples/source/my_code

Whenever I ran ‘yt build’, I got a weird error referring to one of the bluetooth example programs. Plus yt build took longer than I thought it ought to. So…..

Move

~/git/microbit-samples/source/examples

out of:

~/git/microbit-samples/source/

Put these folders one level above this folder. Only put the main.cpp file into

~/git/microbit-samples/source/

Then run:

yt clean

and

yt build

I copied the file:

~/git/microbit-samples/build//bbc-microbit-classic-gcc/source/microbit-c-combined.hex

to the microbit. After waiting for the flashing LEDs to stop, it worked!

The microbit unmounts itself each time that you load a new hex file. So using the bash script that I detailed in an earlier blog to quickly mount the microbit is a time saver.

This is a quick way of getting started by building on top of the examples directory that most folk will start with.

disclaimer: I loiter in the same department as the authors of the C/C++ BBC Micro:bit repository.

Enabling the analog to digital converter (ADC) on the BBC Micro:bit using C/C++

To get the example ADC code to work on the Lancaster Github site, change the line:

MicroBitPin P0(MICROBIT_ID_IO_P0, MICROBIT_PIN_P0, PIN_CAPABILITY_BOTH);

To:

MicroBitPin P0(MICROBIT_ID_IO_P0, MICROBIT_PIN_P0, PIN_CAPABILITY_ANALOG);

I tested this using both the online Mbed compiler and the yotta compiler.