Give Me a Minute set up and use instructions

This page details how to set up and use Give Me a Minute.

Author: Matthew Oppenheim

email: matt.oppenheim@gmail.com

Github site: https://github.com/hardwaremonkey/microbit/tree/master/activity_indicator

Project page: Give Me a Minute.

  • v1.1 June 2017
  • v1.2 Oct 2017 Change detctor counts black pixels to detect a change in a Grid window
  • v1.3 Windows executable created
  • v1.4 Added information on installing drivers July 2018. Tested with Tobii Communictator and Windows 10.
  • v1.5 Moved to odt format.
  • v1.6 Added command line options.
  • v1.7 Moved from odt document to this webpage.

Introduction

It can be hard to know when an eye tracker user is composing speech on their communication device. The ‘Give me a minute’ project solves this by using a BBC Micro:bit to indicate when communication software, such as Sensory Software’s Grid 3 or Tobii’s Communicator is being used to prepare speech. See the page: Give Me a Minute for more details, photos and a video of the system in action.

The ‘Give me a minute’ software works by looking for a change at the top of the communication software window twice a second. A BBC Micro:bit (microbit) with custom software is attached to the communication device. When the text in the communication software is changed the microbit LEDs flash.

The system has been tested with Windows 8 and Windows 10 on Sensory Software’s Grid 2 and Grid 3 and Tobii’s Communicator software. Additional hardware required for the system is one microbit. This can be obtained for around £12. eBay is a good place to buy one. A micro USB cable is needed to connect the microbit board to the laptop running the eye tracker software. An optional case can be bought for the microbit.

Summary – first time use

Plug in the microbit using a micro USB cable from the microbit to
your PC or laptop.

Install the driver for the microbit.

Flash the file ubit_flash.py to the microbit.

Download activity_indicator.exe.

Run Grid 2 or Grid 3 or Tobii Communicator.

Run activity_indicator.exe. Double click on the activity_indicator.exe file or run it from a command prompt.

Summary – regular use

Plug in the microbit using a micro USB cable from the microbit to
your PC or laptop.

Run Grid 2 or Grid 3 or Tobii Communicator.

Run the Windows software.

The software will run without a microbit being plugged in. In this case, a message is printed on the screen telling you that no microbit has been found. If a microbit is plugged in and you still get the message that there is no microbit connected, then the microbit driver is not installed. Look at the section ‘Installing the microbit driver’.

If the Windows software does not run, you can download the Python script from the Github site and run that using Python. Instructions on how to download and install the necessary software to do this are included in this page in ‘Running the Windows software from the Python script’.

What happens during normal operation

When the microbit is plugged in, it should show a vertical bar on the LEDs. When the text at the top of a Grid window changes, the bar should spin around. Pressing button A on the microbit causes this to happen as well. This feature is included so you can test the microbit is working. The picture below shows the vertical bar pattern on the LEDs.

Microbit in standby mode for ‘give me a minute’.

How the system works

The amount of black text in the message window at the top of the communication software window is measured twice a second. When there is a change in the number of black pixels above a threshold, a trigger is sent to the microbit to flash.

Installing the microbit driver

Summary

Download the microbit driver installer.

Plug in your microbit.

Run the microbit driver installer.

Detailed installation

Unless Windows 10 is being used, a driver needs to be installed for
the microbit to be seen by the Windows operating system. In theory
Windows 10 will recognise the microbit, but in testing this was not
found to always be the case. The driver can be manually installed
using the instructions in this section.

I found that Windows 7 would find and download the
driver for the microbit on its own when left for about 5 minutes. The
microbit needs to be plugged in first.

If for any reason the driver is not found and
installed by Windows automatically, then the driver can be downloaded
and installed manually.

The name of the driver installation file is:

mbedWinSerial_16466.exe

A newer version may be available.

Get this file by clicking on the ‘Download current driver’ link on this page: https://os.mbed.com/handbook/Windows-serial-configuration

The microbit needs to be plugged in before the
serial driver installer is run.

Follow the instructions on the webpage to install it.

Installing code on to the microbit

The code for the microbit is called ubit_flash.py. This is a script written in micropython. To flash this on to the microbit use the mu program which can be downloaded from: https://codewith.mu/

Load the file ubit_flash.py into mu using the load button. Then flash
it to the microbit using the flash button. Help on using mu can be
found online.

As the microbits are being distributed to all 11-12 year olds in the UK, if you get stuck installing code on to the microbit and live in the UK, find a friendly 11-12 year old to help you out.

Running the Windows software

The script to monitor the communications software and detect any
changes is written in Python 3. One of the weaknesses of Python is
that making a stand alone Windows executable is not that straight
forwards. The cx_freeze toolkit is used to do this. The python script
is only about 1k long, but the directory with the necessary
supporting files to enable it to run is around 25 Mb. This directory
needs to be installed on to the local drive of the communications
device.

From the Github site download the directory:

build/exe.win-amd64-3.6/

To your local drive. Run the file:

build/exe.win-amd64-3.6/activity_indicator.exe

If a ‘VCRUNTIME140.dll missing error’ pops up instead don’t panic! Go to section ‘VCRUNTIME140.dll missing error’ below.

A window like the one shown below should appear. Once you are happy that the software is running, this should be minimised.

Example output from ‘Give me a minute’ terminal.

The software will look for an attached microbit. If a microbit is not found, the software will tell you this. Currently the software looks for either Grid or Tobii Eyetracker communications software. If neither communications software is running, then the software will tell you this. The software will then print a message each time that a change is detected and send a signal to an attached microbit.

Command line options

New for 2019! If you type ‘activity_indicator.exe –help’ (that is <two dashes>help) you will see a couple of command line parameters that can be used to adjust how the software runs. e.g.

.\activity_indicator.exe –help

Usage: activity_indicator.exe [OPTIONS]

Options:

–limit INTEGER Number of changed pixels to
trigger event. Default is 3.

–fraction FLOAT Fraction of screen, from the
top, to monitor. Default is

0.2.

–help Show this message and exit.

The ‘–limit’ parameter allows you to tune how
sensitive the watched area of the AAC software is to change. This is
useful as different panels have different sizes.

The ‘–fraction’ parameter allows you to
change the fraction of the AAC screen that is monitored for a change.
Usually we only need to watch the top 20% or so, as this is where the
text appears.

VCRUNTIME140.dll missing error

If a ‘VCRUNTIME140.dll missing error’ pops up instead don’t panic! The code relies on some Microsoft Visual Studio libraries being installed. You should put the error you get into the Google machine to find the correct Visual Studio download as this changes as new versions are released. At the time of writing, this link was: https://www.microsoft.com/en-us/download/details.aspx?id=52685

Running the Windows software from the Python script

If the Windows executable does not work, then the Python script can be run instead. To do this, Python needs to be installed along with a few libraries that are not part of the standard installation.

Python software installations

We need to install python3.6 and a few Python libraries. The python libraries required are listed in the file ‘requirements.txt’ in the Github site so that those of you who know how to use virtual environments can use this to quickly set up a compatible work space. Otherwise, follow the instructions below to install the software needed to run activity_flasher.py.

Download and install the 64-bit python3.6 installation from: https://www.python.org/downloads/windows/

Click on the ‘custom installation’ button so that the software
can be installed to c:\python36 so that all users can access it. By
default, the installer will install Python into the user directory
for whoever is logged on, which means that python will not be
available to other users.

There are a few libraries that need to be installed that the python script uses. These are pyserial and pyhook. After installation these libraries will be in c:\python36\lib\site-packages. Instructions on how to install these libraries are given below.

Installing pyserial library

Pyserial allows python to use the serial port to communicate with the
microbit. This can be installed using the pip utility. Pip is a tool
that downloads and installs python libraries from the pypi
repository. It automagically gets the correct version for the version
of python you are using.

From a terminal type:

pip install pyserial

If this fails, you may need the full path to the ‘pip’ command:

c:\python36\Scripts\pip3.6.exe
install pyserial

This should install the pyserial library. If it fails, which can be due to automatic library installation being blocked by Draconian IT
measures, then we can install it directly from a .whl file called:

pyserial-3.4-py2.py3-none-any.whl

‘This file is either bundled with this documentation, or can be downloaded from the site referred to in the section ‘Additional Python libraries needed’.

To install pyserial from the .whl file, from a terminal type:

c:\python36\Scripts\pip3.6.exe install <path to the .whl file>

Installing pywin32

The pywin32 library collection includes the library and dynamic
linked library (dll) for pythoncom which are required by the python
script. To install this, double click on the file:

pywin32-223.win-amd64-py3.6.exe

This file is either bundled with these instructions or can be downloaded from the site referred to in the section ‘Additional Python libraries needed’.

Installing the click library

From a terminal type:

pip install click

If this fails, you may need to type the complete path to the pip command:

c:\python36\Scripts\pip3.6.exe install click

This library allows parameters to be passed to the script from the command line, such as –limit and –fraction.

Summary of additional Python libraries needed

pyserial-3.4-py2.py3-none-any.whl, installed using ‘pip install pyserial’ or downloaded from: https://pypi.python.org/pypi/pyserial

pywin32-223.win-amd64-py3.6.exe downloaded from: https://github.com/mhammond/pywin32/releases/download/b223/pywin32-223.win-amd64-py3.6.exe

Pillow-4.2.1-cp36-cp36m-win_amd64.whl, installed using ‘pip install pillow’ or downloaded from: https://pypi.python.org/pypi/Pillow/4.2.1

Running the python script

The python script is called activity_indicator.py. Download this from
the github site and save it in a directory on your local hard drive.
Run the script using:

c:\python36\python.exe activity_indicator.py

Automating Give me a minute at startup

‘Give me a minute’ cane be set to run when Windows starts by
adding a one line .cmd file to the Windows startup directory. This
directory can be found by typing:

shell:startup into an explorer window at the top.

Create a command file in this directory called:

start_give_me_a_minute.cmd

You can give the file any name that you like.
Anything in this directory will run automatically when Windows is
started.

Put the path to the file activity_indicator.exe in cx_freeze directory into this file. If you are running the software using python with activity_indicator.py, then put the command that use to run this file into the .cmd file.

Board cover

A case or cover can be bought for the microbit for around £5. The kittenbot silicone cases make the boards look cheerful. Have a look at the photo on the project  page to see what one looks like. The only disadvantage of the silicone cases is that it is hard to stick them to the back of the communications device. Sticking the USB cable or cable connector down instead is one solution to fixing the board and cover to the back of the communication device.

Problems

Microbit is connected, but the software tells you that no microbit is found

If a microbit is plugged in and you still get the message that there is no microbit connected, then the microbit driver is not installed. Look at ‘Installing the microbit driver’.

Microbit is continually flashing

Any window on top of the white message bar at the top of the Grid or Communicator window will be read. So if you put the program output window shown above on top of the Grid or Communicator window, any updates will register, forming a feed back loop.

Microbit doesn’t flash for single letters being added or deleted

Different size displays will have different numbers of pixels used for each letter, so getting an ideal threshold that detects a change in the text while not triggering for moving the mouse cursor into the text window for all displays is difficult. Adjust the trigger threshold as detailed in ‘Command line options’.

No response after moving the Grid or Communicator window to a different screen

If you drag the communication software window from the screen in which it was started to a different monitor, the software stops tracking it. This is only an issue if you have more than one monitor.

Getting help

Let me know of your issues and I’ll try to fix them. I work at sea for half of the year and the bandwidth is not so good with the satellite communication system at sea, but I’ll do what I can. Please use the email address at the top of this document.

Background

The need for this software was identified at Beaumont College, Lancaster, UK by Fil McIntyre, the Head Technologist. Prototypes were successfully tested at Beaumont with student volunteers from the College. More project details can be found on the page: Give Me a Minute.

If you want to download and improve the code from the Github site, please go ahead. The link is at the head of this document. Let me know of any enhancements and I will add them to the code on the Github site.

Matthew Oppenheim, InfoLab21, Lancaster University
2019