Running handShake in administrator mode to operate Grid 3

Sensory Software’s Grid 3 is a popular communication software package, running in Windows. Naturally, I would like handShake to be able to operate this software through the software keystrokes that handShake generates.  To get Grid 3 to respond to a software keystroke, I have to ‘elevate’ the base.py script which runs on the communication device to run as an Administrator.

There is a second solution. I can use a Freetronics Leostick USB dongle as a pretend keyboard and have this generate keystrokes that appear as coming from a physical keyboard. I did this for a while, but this adds a layer of complexity and expense to the project. The simplest solution is to run handShake as an Administrator when using Grid 3, or other software that requires software keystrokes to come from an elevated source.

I tested out adding the functionality for handShake to detect when Grid3 was running, then automatically try to elevate the base.py script to run in Administrator mode. I got this running. Then removed the functionality. Why? Security. Software running as Administrator can damage your system if incorrectly or maliciously written.

Now handShake detects if Grid 3 is running and advises that this requires the software to be restarted as an Administrator, but does not try to automate this restart. The decision is left to the user.

The Grid 3 software is detected using the code shown below. The code looks through the titles of the windows for any that match the ones in the list called ADMIN_SOFTWARE. At the moment there is only one title to check for – ‘grid’. This is the title for a window running Grid3. As I find other packages that demand that my script runs as Administrator to have the ability to interact with it, then I will add their titles to the ADMIN_SOFTWARE list.

ADMIN_SOFTWARE = ['grid']

def target_admin_sware(software=ADMIN_SOFTWARE):
    ''' Check if target software requires this script to run as Administrator. '''
    toplist, winlist = [], []
    logging.info('Looking for software that requires elevation to Aministrator: {}'
        .format(ADMIN_SOFTWARE))
    def _enum_cb(hwnd, results):
        winlist.append((hwnd, win32gui.GetWindowText(hwnd)))
    win32gui.EnumWindows(_enum_cb, toplist)
    for sware in software:
        # winlist is a list of tuples (window_id, window title)
        for hwnd, title in winlist:
            if sware in title.lower():
                logging.info('found software requiring Administrator mode {}'
                    .format(title))
                return True
    return False

Running handShake as Administrator is a choice that the user makes and implements if he or she deems necessary.  As all of the code is on the project GitHub site, the code can be reviewed to check that it is safe to run as an Administrator.

One of the advantages of open source projects is that they are open to this kind of scrutiny to find security flaws.

For interest, I detail the code I added and then removed from the base.py script to enable it to detect if it is running as an administrator and if not, request to be restarted as an Administrator.

I added the option to run the script as an Administrator from the command line using the click library.

click.command()

@click.option('-a', '--admin', default=False, 
    help='Run as administrator. Required for Grid 3.')

@click.option('-k', '--keystroke', default='F1',
     help='Keystroke to send. Default is F1.')

def main(admin, keystroke):
    logging.info('software keystroke is {}'.format(keystroke))
    logging.debug('admin flag is {}'.format(admin))
    if is_admin():
        logging.info('running as administrator')
    else:
        logging.info('not running as administrator')
        if admin:
            logging.info("restarting as administrator")
            elevate()
    service_microbit(keystroke) 

To test if the script is running as an administrator I added this method:

def is_admin():
    ''' Is the script running as an Administrator? '''
    try:
        return windll.shell32.IsUserAnAdmin()
    except:
        return False

Using Python to detect activity in Sensory Software’s Grid 2

Update: March 2018. This work is being submitted to the Communications Matters conference.

Following on from the eyeBlink post, with the help of Fil at Beaumont, I modified the algorithm I’m using to detect when the Grid 2 or Grid 3 software is being used. The image below shows Sensory Software’s Grid 2 software being used to construct a sentence. The new text appears in the white area at the top of the window. Fil suggested that I change the Python script to just monitor this area at the top of the window. The script now looks for a change in the amount of black text in this area. After the usual software wrangling I think I got it working. The Python script looks at the top 20% of the window and counts the number of black pixels in this area. Every half second it recounts the number of black pixels. If there is a change in the number of black pixels above a threshold, then a trigger is sent to indicate that the Grid software is being actively used. I’m using a threshold of 20 pixels, so there needs to be an increase or decrease of 20 or more black pixels for a change to be detected. This allows you to move your mouse cursor around in the text area at the top of the Grid window without triggering that there has been a change. The activity detection script needs more testing, but preliminary results seem to show it works. Prior to this, I was monitoring the entire Grid window and looking for a change in the whole window above a threshold. This led to false triggers when cells were selected, but not activated. When a cell is selected, the colour of the cell changes, even when it is not activated to produce text. This change in colour was being detected.

Each time we test the script, we find new ways to break it, leading to some ‘try except’ exception handling clauses. The script is designed to run on Windows as Grid 2 and Grid 3 only work on this operating system. I use the win32gui library to interact with Windows and the python imaging library, PIL (known as pillow), to do the image processing.
 
Sensory Software’s Grid 2 Chatterbox grid being used to construct a sentence:
 

Grid 2 communications software by Sensory Software used to create speech.

The latest code and installation details on how to get this running using the BBC microbit to give a flash when the Grid software is being actively used can be found on my github site at:
 
If you have any questions, please ask.