I said I would produce a description of my final year project. Sorry
its a little longer than I hoped!
 
Simon Coles
[log in to unmask]
 
---------------------------------------------------------------------
 
                                  CANDi
                                  =====
            A communications aid for the neurologically disabled
                         Copyright Simon Coles 1994
 
Abstract:
=========
 
This document describes CANDi (Communications Aid for the Neurologically
Disabled), a software package under development for Apple's Newton range
of Personal Digital Assistants.
 
CANDi hopes to improve on existing communications aids for people with
severe disabilities reducing the key presses required to form a message.
 
 
Introduction:
=============
 
This is the first time CANDi has been described to a wide audience. I
welcome (need!) frank and constructive criticism. All correspondence
should be directed to "[log in to unmask]".
 
The document is split into several sections:
    - Background.
    - A brief introduction to Newton Technology.
    - The aims of CANDi.
    - Main functions of CANDi.
    - Functional enhancements to be added at a later date.
    - A discussion of the feasibility of CANDi's aims.
    - Why Apple's Newton, an "off the shelf" hardware platform, has been
      chosen, and the advantages this brings.
    - A discussion on the possible future directions that CANDi could
      progress over the longer term.
    - An estimate of progress so far, and possible dates when the
      software will be available.
    - Some definitions.
 
Please feel free to pass this document onto whoever you feel might be
interested.
 
 
Background:
===========
 
Target Audience:
----------------
 
CANDi, as the full name suggests, is aimed at people with severe
neurological disabilities & associated conditions. These people are
generally not well served by existing communications aids.
 
The initial release is aimed at the most severe Parkinsonians, as I have
personal experience in caring for these people, but there is no reason
why it cannot be useful for other conditions.
 
 
Problems:
---------
 
Severe Parkinson's Disease (PD) sufferers often have reduced motor
control, and badly impaired speech. Whilst conventional communications
aids are helpful, they have the following problems:
    - use of a keyboard is difficult, slow, and prone to errors
      (hitting the wrong key, or hitting it more than once).
    - co-ordinating their movements is often difficult, making "grid
      scanning" systems impractical.
 
Some Parkinsonians will have these problems only during a bad "off"
period; very severe cases may have them continuously.
 
PD is a very individual condition; each person has specific symptoms and
needs. This makes the task of "occasional" carers difficult - you cannot
go on a course and be taught how to care effectively for all
Parkinsonians. Communication with the sufferer is essential for
effective care, and often this communication with the sufferer itself
requires extensive background knowledge of the individual.
 
The occasional carer's lack of detailed background information tends to
cause problems with any kind of respite care:
   - they lack the detailed experience and techniques needed to
     communicate with the sufferer.
   - they lack an understanding of the sufferer's specific requirements,
     which has a detrimental effect on the sufferer (sufferers often
     return worse from respite care).
   - principal carers feel guilty for putting the sufferer in for
     respite care because they know they are condemning their charge to
     a period of well-meaning incompetence.
 
It is difficult to over-emphasise the dependence the sufferer has on the
principal and secondary carers, and the additional stress this causes on
both sides. CANDi hopes to reduce this dependence by allowing sufferers
to communicate effectively even with inexperienced carers. In addition,
it is hoped it can provide the means for them to communicate easily with
other people and so enhance the quality of their life.
 
History:
--------
 
My Mother suffered from a condition known as Multiple System Atrophy,
which presents itself as a very severe case of Parkinson's.
 
CANDi originated from my observations of how I and other carers
communicated with her. Some people were obviously better "listeners"
than others; we began to realise this was due to:
    - techniques to make the best use of the sufferer's limited
      communications abilities.
    - background knowledge to reduce the amount the sufferer had to say.
 
Given a suitable (portable) computer, the process of communication, even
with inexperienced people, could be made much easier. Just before I
started my final year project, Apple released the Newton; a suitable
hardware platform.
 
During my background research, it became apparent there were some
theoretical reasons for believing communication could be made easier.
 
Later, more functions and ideas came along - there are a lot of things
you can do when you put a large amount of computing power in someone's
hand. The CANDi concept sort of migrated from a simple communications
device into an expert system/carer's aid.
 
 
Aims:
=====
CANDi has two principal  functions:
   - to help the sufferer to communicate effectively, even when they
     have reduced motor function.
   - to provide occasional carers with some of the background
     information they require to care effectively for the individual.
 
It is hoped this will therefore improve the quality of life for all
concerned.
 
 
Newton Technology:
==================
 
"Newton" is a line of Personal Digital Assistants produced by Apple
Computer. They are small, portable computers which recognise your
handwriting when you write on their touch sensitive display - they have
no keyboard.
 
Eventually there will be many different devices implementing this
technology, from many manufacturers. The software should work unchanged
on all models, so users may pick the device most suited to their needs.
 
The first Newton is the MessagePad, a small device the size of a VHS
video cassette. The touch screen is 3" wide and 4" high. I am using the
MessagePad to develop CANDi, and to show the software's potential.
Initial indications are that CANDi will be better suited to larger
devices.
 
To give you an idea of price, apparently these are MacWarehouse's latest prices
 for the MessagePad - treat these only as a ballpark figure!
 
        Newton MessagePad            $799
        2 meg storage card           $279
        Apple fax modem              $139
        ext. battery pack            $ 25
 
Which gives you a CANDi capable system for $1,242 (painful!). This will
hopefully drop to more reasonable levels soon.
 
UK users will probably have to pay around 813 Pounds Sterling at today's
prices.
 
 
Main Functions:
===============
CANDi has two main communications modes:
   - "Typing" (the user types and the system fills in the word)
   - "Phrase" (the system tries to predict the phrase)
 
The distinction between the two is somewhat arbitrary, and they will
probably be merged in the future; at the moment they are separate
because there isn't enough room on the MessagePad screen.
 
There is an additional "Question and Answer" mode which helps
inexperienced carers diagnose problems.
 
All modes are based on the techniques carers use to communicate with
sufferers.
 
The prediction mechanism, used in all modes, constantly attempts to tune
itself to the individual user's habits, presenting only the most likely
choices.
 
 
Typing Mode:
------------
 
This is the simplest mode, and is a replacement for the conventional
typewriter based communication aid. The intention is to support as full
a range of communication as possible, without the user having to do any
unnecessary key presses.
 
A keyboard is displayed on screen, together with a list of words. As the
user taps on the keyboard, the machine searches an 8,000 word dictionary
and presents the user with a list of words which they might be typing.
 
For example, if the user typed "a", 16 of the most likely words
beginning with "a" would be presented. The user may make a selection
from this list, or continue typing. If the next letter was "t", then the
most likely words beginning with "at" would be shown, and so on.
 
With this mode, the user will probably be able to type any message with
around 2 or 3 key presses per word, as opposed to an average of 5 with a
conventional typewriter.
 
The biggest problem with this on the MessagePad is screen area - I can't
fit a usable keyboard on the screen, let alone a decent sized word list.
Therefore phrase mode is provided for those situations where the
sufferer has difficulty with the small keyboard. With a bigger screen,
the two modes can be merged, which will make it more effective.
 
The keyboard can be one of many layouts, e.g. QWERTY, Alphabetic, etc. -
the user can choose the most familiar.
 
 
Phrase Mode:
------------
 
Most messages are fairly standard, for example when the sufferer needs
to communicate a simple request to the carer:
  "I want corn flakes for breakfast"
  "Please turn the TV to Channel 4"
etc.
 
Phrase Mode allows the sufferer to communicate these simple everyday
requests, even when they are having an "off" period (and hence reduced
typing ability), by reducing the number of presses required.
 
The user is presented with a list of 16 words in large buttons on the
screen. They chose one, and then the list is refreshed with new words
which follow. In this way, short phrases can be built up with just a few
presses.
 
Initially, the machine will come with a few phrases built in, enough to
get you started. Over time, you can teach the machine your own.
 
The phrases are in fact stored in a tree structure inside the machine. A
diagram might help:
 
 
                                   |--- Tea
                                   |
                       |-- Drink --|--- Coffee
                       |           |
          |--- Food ---|           |--- Water
          |            |
          |            |           |--- Biscuit
          |            |           |
          |            |-- Food ---|--- Chocolate
          |                        |
          |                        |--- (..... etc. ....)
          |
          |
Personal -|--- Toilet (..... etc. ....)
          |
          |--- Move (..... etc. ....)
          |
          |
          |           |--- Hand
          |--- Pain --|
                      |--- Arm
 
 
The user's first choice would be "Personal", followed by "Food" (from a
list of "Food", "Toilet", "Move", "Pain"), followed by "Drink", then
"Water".
 
This allows simple, common phrases to be built up with only one press
per word. The system can be personalised, i.e. if the sufferer only
drinks Coke, and never water.
 
This mode has several problems, which is why it is only used when the
user can't operate the typing mode.
 
    - the phrase must be known by the machine (new ones can be added
      with the keyboard & word predictor from the typing mode).
    - the tree must be structured so the most frequent words appear in
      the top levels.
    - all unnecessary words should be omitted from the phrase. e.g.
      "now want coffee", rather than "Please can I have some coffee
      now". Grammatical niceties are out.
 
 
Question and Answer Mode:
-------------------------
 
There are occasions where the sufferer is obviously distressed, although
the cause is not known. Because they are distressed, they find it
difficult to communicate their need to the carer.
 
Carers will know how frustrating this situation is - it is obvious
something needs to be done, although it is difficult to know what.
 
In these situations, experienced carers go through a process of
elimination, getting the sufferer to signal "yes" or "no" to a series of
questions:
    "Do you have cramp?"
    "Do you need to go to the toilet?"
 
Not solving the sufferer's problem correctly in a short time can cause
additional stress. It is primarily because of these situations that
principal & secondary carers feel uneasy about leaving their sufferer in
the care of occasional carers.
 
Q & A mode leads the carer through a series of questions in order to
ascertain the problem & solution in the shortest possible time. These
questions will have to be programmed by the principal carer before hand.
Basically all the problem diagnosis skills of the carer are placed
within CANDi - an expert system, configured to the sufferer's individual
problems & needs.
 
 
General:
--------
 
The three modes are designed for different levels of disability; this
reflects the fluctuations in symptoms that Parkinsonians can experience.
The important thing is to gain maximum benefit from whatever motor
control the sufferer has.
 
The size of the buttons will be very important; Parkinsonian's physical
co-ordination can be poor, so buttons should be as large as possible. If
the buttons are too large, there won't be many of them on screen. Hence
a rather difficult trade off.
 
Eventually it is hoped the size of the buttons can vary dynamically -
the system can monitor how co-ordinated the person is, and adjust the
button size accordingly.
 
 
Other Functions:
================
 
The "Main Functions" section described the areas I am committed to
completing for my Final Year Project; they form the basis for the
device. This section explores the other ideas I have - long term dreams
rather than plans.
 
A lot of these functions rely on the "central machine", so perhaps it
would be best to describe this first.
 
A user will have a lot of information in their CANDi; perhaps many years
worth of configuration. It is sensible to provide some way for them to
back this up. It is unlikely they will have access to a PC or Mac to do
it themselves.
 
Fortunately, there is a fairly cheap Newton modem available (and people
can share one); users could phone up some central machine, and send all
their data down the phone line. If all users have modems and are
contacting the central machine on a regular basis (every month? every
week?), other things become possible.
 
a) To get the CANDi software, users go down to their local electronics
   shop and buy a Newton and Modem. They then phone up and download the
   software and initial configuration. This is slightly expensive in
   phone bills, but cheaper than organising a distribution network.
 
b) To back their CANDi up, they just phone the central machine.
 
c) As the central machine has all the data from all the machines, it
   could merge them into one big pool. This can be pushed back to
   individual machines - so they all gain from each other's experience,
   learning much faster than they would individually. This is what I
   call "Federated Learning".
 
d) I can release software on a "little & often" basis, without bothering
   the users. This should allow the development of CANDi to proceed
   faster than it would otherwise.
 
e) I can gather metrics on how successful the prediction mechanisms are,
   and adjust them to maximise performance.
 
f) Questionnaires prepared by other researchers can be downloaded and
   answered by people at their convenience. This will be cheaper and
   more effective than any other method.
 
g) Users can decide what reference material they want to download.
 
h) Information can be gathered on how co-ordinated the user is, day-to-
   day, and hour-to-hour. This could allow medical personnel to measure
   the effectiveness of drug treatment.
 
There is no possibility of confidentiality being compromised by sending
data to the central machine; it will be in such a form as to make
reconstruction of original messages impossible.
 
The system could store people's basic preferences - what foods they
don't like, how they take their coffee etc. It could also document their
drug regime and record medical/care history (as entered by the carer).
 
There is no reason why CANDi cannot be used either as a keyboard for a
conventional computer (i.e. IBM PC compatible), or as a serial terminal
onto another system.
 
In addition, the typing mode keyboard can be used instead of the normal
Newton keyboards & handwriting recognition. So all the normal functions
of the Newton are still available to you.
 
A reference library could be included on the device (as a collection of
Newton-format books), to show occasional carers general information
about the disease. Unlike paper, this can be dynamic, so an animation
could show how to move a sufferer, for example.
 
One of the more crazy things is "Automatic Politeness". Often sufferers
want to say "thank you" when they have been helped, or to put "please"
in front of requests. Often its too much hassle - I can remember
spending 10 minutes getting very frustrated with Mum, and all she wanted
to say was "thank you". So automatic politeness would put "please" and
"thank you" in the sentence where appropriate (it sounds a little
automatic, but I think no one would notice after a while).
 
 
How & Why:
==========
 
When I first told them I wanted to predict speech, most people laughed.
This is a very difficult problem!
 
Yet the task becomes easier when you use the symptoms of the disease to
help. Parkinsonians don't want to have wide ranging philosophical
debates (they probably do, but I can't cater for that yet). What they
need to do is communicate needs to their carer. This means the
communication is over a very restricted domain.
 
The grammar can also be simplified - the carer will share a lot of
context with the sufferer, so a lot of the information which is carried
in full grammar can be guessed (i.e. tenses and modality). There is no
need to produce perfect English prose.
 
So:
    - The vocabulary can be restricted
    - The grammar can be simplified & abused.
 
This makes prediction practical. As the technology improves, a wider
range of conversation can be supported.
 
 
Technology:
===========
 
The use of a consumer hardware platform like the Newton brings several
advantages over custom solutions:
    - Development and support should be easy enough that I can to it in
      my spare time.
    - The hardware is going to evolve at a much faster rate, and produce
      more varied products than could ever be achieved with a low volume
      custom device. CANDi should be usable on all Newtons, so pick the
      one that suits you best.
    - The hardware is more powerful than anything I could produce for
      the same price.
    - Newton is available from your local shops.
    - Spares & Servicing can be done by your local dealer.
      Which means I can distribute CANDi far and wide without having to
      worry about a support system.
    - Other Newton software products will be useful: Mail systems, fax,
      books, games etc.
    - Its fun (for me to write, and for you to use).
    - It doesn't look like a "disabled device", more like a yuppie toy.
      There is no embarrassment carrying this in a social function.
 
CANDi at the moment runs on a Newton MessagePad with a 2Mb RAM card.
 
 
The Future:
===========
 
The biggest problem I have at the moment is screen size - especially for
the typing mode. There isn't really enough space to fit everything on in
a useful size.
 
For the moment I will do the best I can with a screen 3" by 4". But
CANDi will only really work on a larger screen - wait for the "SlatePad"
out in the latter half of the year. This should have a much larger,
backlit screen which will help a lot.
 
In the mean time, on a MessagePad, I feel the following features will be
usable:
    - Prediction Mode.
    - Question and Answer mode.
    - Typing mode will not be at its best.
 
As Apple and other manufacturers produce further Newton devices, users
can pick the one most suited to their needs. It may be that a small,
hand held device would be appropriate initially, with an upgrade to a
larger machine later.
 
CANDi is at the moment being privately financed. With the resources I
have it should be possible to place pilot devices with a small selection
of people, provided their Newton is financed from some other source.
However, more resources make progress easier, and there is no doubt that
running this project on a shoestring is an impediment. So offers of
help, or suggestions of who to approach would be very welcome!
 
In addition, if anyone has leaflets or books which they are willing to
have distributed electronically with CANDi, please let me know. I'll
probably start thinking of including a reference library in about 6
months.
 
CANDi will be a very effective way of getting large amounts of
information to people cheaply, and I would hope a good selection would
be forthcoming (as most societies give their information away free, or
at cost).
 
 
Progress to Date:
=================
 
This section describes where I am at this moment in time, and where I
hope to get to soon.
 
    - The phrase mode works, although you can't add any new phrases
      yet.
    - The typing mode works, although the keyboard is too small.
    - The Q & A mode is still very much an idea at the moment, although
      I have a good idea how it will work. It will probably get written
      as "relaxation" during my exams.
 
Before I release beta versions to people who already have Newtons (I
don't want the responsibility of people spending their hard earned cash
on this yet) I need to:
    - build a list of words.
    - build a list of phrases for the prediction mode.
    - solve the problem of how to efficiently get 1.5Mb of initial data
      into each person's machine.
    - write a user manual.
 
I would hope to get this done so people can start playing with this in
Q2/3 1994. I don't know if Q & A mode will get done by then.
 
In March, I disappear for a while, for my final exams. After the exams
(which end in May), I work full time on the project. This will probably
involve all the boring stuff I have to do to pass my degree -
documentation etc. I would hope to get a more polished version done
though, and incorporate any comments from the first version.
 
Once I get out of University, I would hope to get a Version 1.0 release
out in Q4 of 1994, although I may well wait for Apple to release the
SlatePad.
 
So the upshot is:
    - beta versions, which will show potential but not a lot of utility,
      will be out by Q2/3 1994.
    - Version 1.0 will be out in Q4 1994.
 
After the first release, I will launch a programme of incremental
improvements, with a version coming out probably every month. This is
where most of the "Other functions" described above will happen.
 
 
Definitions:
============
 
Carer:            A person engaged in assisting the Sufferer.
                  The distinction between the three types of carer is
                  due to the different amounts of familiarity they have
                  with the suffererUs condition, and hence how much
                  background knowledge they have during communication.
 
Occasional Carer: A person who might care for the sufferer infrequently.
                  For example, nurses on respite care wards, members of
                  the general public, relatives who do not live with the
                  sufferer. These people typically do not have the
                  opportunity to learn the suffererUs condition in
                  detail. In order to care effectively for the sufferer,
                  occasional carers generally need to communicate with
                  them.
 
Principal  Carer: The suffererUs most frequent carer, generally their
                  husband, wife or other close relative. Whilst these
                  people will generally have little or no medical
                  background, they will have the most experience of the
                  sufferer's condition.
 
Secondary Carer:  People who care for the sufferer on a regular basis,
                  generally members of the immediate family. Whilst they
                  may not have all the knowledge of the Principal Carer,
                  they will have a detailed knowledge of the sufferer.
 
Catastrophic Disability :
                  "Disability" has become a very imprecise word, and can
                  be taken to mean almost any impairment of function.
                  Therefore I have used the term "Catastrophic
                  Disability" to emphasise the multiple problems typical
                  of very severe Parkinsonians (and other similar
                  conditions). With such conditions there are a great
                  many problems - sufferers have great difficulty
                  moving, speaking etc.
                  Thankfully such conditions are very rare indeed, and
                  few Parkinsonians ever reach this stage. But I needed
                  to emphasise CANDi is not needed by, or intended for,
                  the typical sufferer.
 
Button:           Area of the screen which has some text in it and does
                  something when pressed.
 
Respite Care:     When the sufferer goes into short term residential
                  care (e.g. into hospital) to give their principal
                  carer a rest.
 
Beta Version:     A release of a software package which has not yet been
                  finished or fully tested. Use at your own risk.
 
 
This document was written by Simon Coles 30/1/94.
Last updated 10/2/94.
--