Donegan, M., Oosthuizen, L., Bates, R., Daunys, G., Hansen, J.P., Joos, M., Majaranta, P. and Signorile, I. (2005) D3.1 User requirements report with observations of difficulties users are experiencing. Communication by Gaze Interaction (COGAIN), IST-2003-511598: Deliverable 3.1. Available in PDF format at http://www.cogain.org/results/reports/COGAIN-D3.1.pdf

Mick Donegan (ACE, contact person, email: [forename(dot)surname(dot)cogain(dot)org])
Lisa Oosthuizen (ACE)
Richard Bates (DMU)
Gintautas Daunys (SU)
John Paulin Hansen (ITU)
Markus Joos (TU DRESDEN)
Päivi Majaranta (UTA)
Isabella Signorile (POLITO)

Table of Contents

1 Executive Summary
  1.1 End-users eye control hardware requirements
  1.2 End-users' eye control software requirements

2 Introduction - What is the purpose of this User Requirements document?
  2.1 Putting the user at the centre
  2.2 Data collection methods and background information
  2.3 Summary

3 Who is currently able to use eye control technology?
  3.1 What the literature says
  3.2 Information from stakeholders
  3.3 Summary

4 Who is currently unable to use eye control technology?
  4.1 What the literature says
    4.1.1 Difficulties with eye control
    4.1.2 Ability to maintain a position in front of the Eyegaze monitor
    4.1.3 Medication side effects that affect eye tracking
  4.2 Information from stakeholders
    Case study - Claire
    Case study - Michael
  4.3 Summary

5 What potential alternatives to eye control are there?
  5.1 What the literature says
    5.1.1 Devices with direct mechanical interaction
    5.1.2 Devices with optical interaction
    5.1.3 Electrical interaction
  5.2 Information from stakeholders
    Case Study - Julius
    Case Study - Paul
  5.3 Summary

6 Choosing between Eye Control and other access methods - the concept of Usability
  6.1 What the literature says
    6.1.1 Usability
    6.1.2 Tools currently in use for measuring the usability of eye control
    6.1.3 Usability measurement and the usability of eye control
    6.1.4 Usability variation in particular environments
    6.1.5 Usability variation in specified users
  6.2 Information from stakeholders
    Case study - Sarah
    Case Study - Ahmar
  6.3 Summary

7 What can people with disabilities currently achieve with access technology overall?
  7.1 What the literature says
    7.1.1 Social Communication and Writing using Assistive Technology
    7.1.2 Software for written communication
    7.1.3 Controlling the Environment using Assistive Technology
    7.1.4 Powered Mobility using Assistive Technology
    7.1.5 Leisure activities using Assistive Technology
  7.2 Information from stakeholders
    Case Study - Stewart
  7.3 Summary

8 What can people with disabilities currently achieve using eye control technology specifically - and how well?
  8.1 What the literature says
    8.1.1 Non-electronic gaze communication
    8.1.2 Computer based gaze communication
    8.1.3 Environmental control
    8.1.4 Entertainment and work
    8.1.5 What improvements need to be made to existing eye control technology and applications?
  8.2 Information from stakeholders
  8.3 Summary

9 Discussion and Recommendations
  9.1 Issues relating to users' eye control hardware requirements
  9.2 Issues relating to users' eye control software requirements
    9.2.1 A wide choice of on-screen software interfaces
    9.2.2 A range of input methods
    9.2.3 A wide choice of output methods
    9.2.4 A choice of languages
    9.2.5 Summary of software requirements
  9.3 Issues relating to safety and reliability

Glossary

References

Appendices

1 Executive Summary

From the literature and data collected it seems that, at present, eye control can only be used effectively to meet a limited range of user requirements. Furthermore, it can only be used effectively by a limited number of people with disabilities who might benefit greatly from it. To address these issues, a number of recommendations are made in this document for consideration by COGAIN partners.

[Back to Contents]

1.1 End-users eye control hardware requirements

It is recommended that a good starting point would be to:

• Measure how effectively the eye control technology available can meet the needs of the full range of users who might benefit from it.

To achieve this aim, it is recommended that WP3 (User Involvement) should:

• Trial as many specialist Eye Control systems as possible[1].

This will provide an opportunity to:

• Feed back to Eye Control System developers how effectively their technology is meeting the needs of the full range of existing and potential users[2], and
• Make observations and suggestions relating to any potential modifications to their systems and/or software that might make it more accessible and/or more effective for more users[3].
• As the above information is acquired, to enable users to make an informed choice of which hardware to consider for their eye control needs, it is recommended that:
• WP3 should add the information gathered from the above investigations to the WP5 catalogue of currently available eye trackers.

The emphasis of the information provided by WP3 should be specifically related to usability issues related to the requirements of end-users with disabilities, e.g. environmental control, portability issues, mounting issues, 'choice of output methods', 'range of access methods', etc[4].

[Back to Contents]

1.2 End-users' eye control software requirements

Features of the wide range of assistive software already being successfully used via a range of access methods in addition to eye control include the following: resizable cells and grids; a range of input methods; a wide choice of output methods; a choice of symbols or text output; a wide choice of text styles and colours; a range of speech facilities; a choice of languages, etc. As a result, it is recommended that the following issues be investigated with the involvement of the users themselves.

• Of the wide range of specialist (non-eye control) software that is already successfully being used by many people with disabilities for access and communication, find out which can be adapted effectively for eye control (e.g. The Grid, SAW).

This will enable COGAIN partners to:

• Make a comparison of how effectively both the existing range of software specifically designed for eye control and the adapted specialist software compare in terms of their efficacy with eye control systems.

As a result, on behalf of the Users, WP3 will be able to:

• Recommend modifications that could be made to the current range of software that can (or could) be used for eye control, so that it meets as many of the needs of as many existing and potential users as possible.

As the above information is acquired, to enable users to make an informed choice of which software to use for eye control, it is recommended that:

• A matrix should be set up on the COGAIN website relating to features of different software that can (or could be) used for eye control.

The comparison would be based on features such as those described above, such as 'choice of output methods', 'range of access methods', 'range of multi-modal access', etc[5].

[Back to Contents]

2 Introduction - What is the purpose of this User Requirements document?

2.1 Putting the user at the centre

"All technology operates within a context, and in designing products it is important to look at that context in addition to the technology itself. From this perspective, all technology is seen as being part of a wider system which must be designed correctly if that technology is to function appropriately."
             USERfitIntroduction, p. 19, ECSC-EC-EAEC, Brussels-Luxembourg 1996.

Over the 5 years duration of the COGAIN Project, partners will be involved in developing technology that will (a) accommodate the specific individual needs and difficulties of users with disabilities and (b) meet as many of their overall requirements of Access Technology as possible. To help to put the next 5 years work into a context, therefore, this document will focus on providing information that will assist COGAIN partners in putting the real needs of end users at the centre of their research and development work.

Figure 2.1. Adapted from 'Diagram of Usability in a wider context', p. 15, ECSC-EC-EAEC, Brussels-Luxembourg 1996.

The aims of the COGAIN Project fit very well with the European Commission's TIDE 'USERfit' ideology (see Figure 2.1). Essentially, this promotes an approach that is:

• User-centred
• System Oriented
• Iterative

A user-centred approach is one, which argues that it is the end user's requirements, rather than technological capabilities, which drives product development. The logical starting point, then, is to take the time to understand the user population in some detail and understand what they need from products before going too far down the route of deciding on specific solutions. Too often, design is driven by technical feasibility, which can lead to a poor match to user's needs. A user-centred approach, on the other hand, is keen to ensure that the proposed solutions:

• Have real value for end users
• Are matched to user capabilities
• Are fit for the purpose for which they were designed

For these reasons, COGAIN is ensuring that potential users of Eye Control Technology should be involved as frequently and extensively as possible. This is why the ACE Centre, as specialists in the field of Access Technology, have been assigned with the responsibility of leading the process of facilitating the representation of user's views. This is a central element of the COGAIN Project.

In order to support COGAIN partners in making decisions about which user requirements to meet and how to meet them, it is important to put eye control into the context of disability and access technology as a whole. This document, therefore, sets out to consider:

• Who is currently able to use Eye Control Technology? (Chapter 3)
• Which new users need to be able to use Eye Control Technology who can't already? (Chapter 4)
• What are the potential alternatives to Eye Control? (Chapter 5)
• Usability issues - what are the potential benefits of Eye Control compared to other forms of access? (Chapter 6)
• What sorts of activities are people with disabilities currently using access technology as whole - not just eye control - for? (Chapter 7)
• If they can use eye control, what sorts of applications are people with disabilities currently using eye control for and how effectively? (Chapter 8)

With this information, it is intended that COGAIN partners will have the background knowledge required to make pragmatic decisions about which types of applications and which accessing issues to take into account in their research and development work under COGAIN. It is acknowledged that this can only be done within the context of a range of practical, reasonable considerations, for example:

• The quality and effectiveness of eye control hardware and software is, of course, dependent on variables such as the amount of development time and the budget available.
• Some users will have more limited requirements of this technology than others, so it is also reasonable to acknowledge that not all users will need to have a top-level eye control system with top-level assistive software.

Whichever decisions COGAIN partners make in meeting user requirements, it is important that users are closely involved as part of a dynamic, iterative process. It is, indeed, the responsibility of the ACE Centre as leaders of WP3 to ensure that the evolving and emerging needs and requirements of users are taken into account on an ongoing basis.

For this reason, this User Requirements Document should be regarded as a working document and one that evolves as the COGAIN Project progresses. Eye Control Technology is a developing field so many of those whose opinions have been sought in relation to this document will not yet have had the opportunity to base their opinions on direct experience. As a result, some of their expectations and requirements might not actually be achievable under COGAIN. As the project, progresses and those involved become more familiar with what is and is not possible through Eye Control Technology, their expectations of it will, of course, be shaped by events. It is important, therefore, that this document captures these evolving requirements. For this reason, it will be periodically reviewed and revised.

[Back to Contents]

2.2 Data collection methods and background information

Because of the relatively small numbers of existing users of eye control systems and their wide geographical spread, it was anticipated that, within the timescale of this document, the numbers from whom information could be gathered and of whom observations could be made would be small. Similarly, the number of non-users with whom it would be possible to trial eye control systems with would also be small. For this reason, the approach to data collection involved a qualitative rather than quantitative approach to with a view to gaining an insight into the range of issues involved for consideration by COGAIN partners, rather than attempting to gather information of any quantitative significance, at this stage. The data collected, therefore, is intended to augment and enrich the information provided by the available literature.

Information from those supporting people with disabilities

• The views of parents and professionals (e.g. teacher, carer, health professional, assistive technology specialist) who support a people who have a disability were gathered through questionnaires[6].

Information from end users

• Information from those people who have a disability and who are (or have been) regular users of eye control systems was gathered through observation and questionnaires[7].
• Information from those people who have a disability and who do not use an eye control system (or have had only a little experience with one) was gathered through questionnaires and informal interviews.
• To gain an insight into the issues involved in calibration and utilisation of eye control systems with users who are not currently using eye control, a number of user trials[8] were carried out with end users not currently using eye control.

Background Literature

COGAIN WP3 partners have written the background literature sections, collaboratively, in Chapters 3-8 inclusive.

[Back to Contents]

2.3 Summary

An essential purpose of this document is to ensure that COGAIN partners are provided with information relating to:

• The wide range of user requirements that exist.
• The wide range of difficulties that need to be overcome in order to meet these requirements.
• The wide range of potential benefits of this technology in comparison with other access technology.
• The wide range of potential beneficiaries who are not yet able to access this technology.

With this information, it is intended that COGAIN partners will be able to share a common understanding of the range of user-centred considerations necessary to promote well-informed, pragmatic decisions about the hardware and software they develop.

[Back to Contents]

3 Who is currently able to use eye control technology?

3.1 What the literature says

Current eye tracking technologies can be divided into the following categories: electro-oculography (EOG), scleral contact lens/search coil, video-oculography (VOG) or photo-oculography (POG), and video-based combined pupil/corneal reflection techniques (Duchowski, 2003). Not all of them are practical for interactive applications. For example, systems that use contact lenses are not practical or convenient for interactive applications, even though they can be very accurate for psychological or physiological studies, for example. Certain EOG systems are impractical for everyday use because they require electrodes to be placed around the eye to measure the skins electric potential differences. However, there are EOG systems aimed for augmentative and alternative communication, e.g. the EagleEyes system (Gips et al., 1993), Figure 3.1.

The most popular and suitable systems for use in interactive systems are video-based. Video-based combined pupil/corneal reflection techniques provide the Point Of Regard (POR) measurement: the system can calculate the direction of gaze (Duchowski, 2003). This requires taking into account the head movements in addition and relative to the eye movements. At least two reference points are required for the gaze point calculation (sometimes called the glint-pupil vector). By measuring the corneal reflection(s) relative to the centre of the pupil, the system can compensate for a certain degree of head movement.

Before the system can calculate the direction of gaze, it must be calibrated. This is done by showing a few (e.g. 9) points on the screen and asking the person to look at the points, one at a time. If the person is not able to direct his or her gaze and focus on the point (e.g. due to eye tremor), the accuracy of the calibration suffers or may become totally impossible. The duration needed to look at each calibration point may vary. For example, the LC Eyegazes calibration procedure simply waits for a good fixation before moving to the next calibration point (LC Eyegaze, 2001).

The corneal reflections are typically from an infrared (IR) or near infrared (NIR-LED) light source. Because IR is invisible to the eye, it does not distract the user. If the light source is located on-axis with the camera and the eye, it causes a so-called bright pupil effect. The light reflects directly back from the eye to the camera (similar to the red eyes effect in photographs taken with flash). If the light source is located off-axis, it the camera sees a dark pupil. There are variations in how well each eye tracking system can track the user's pupil. The size of the pupil has an affect on how well the system is able to track the persons eye. According to Nguyen et al. (2002), there are fairly big differences in the infrared bright pupil response of human eyes. The ethnic background has on effect on how the light is reflected from the retina. On the other hand, in techniques based on the dark pupil affect the colour of the iris matters; the contrast between the iris and the pupil must be distinguishable from the video image.

The camera must have an unobstructed view to the persons eye (especially the pupil) to be able to track it accurately. Eyelids or eyelashes may partially cover the pupil. The frames of the user's glasses may also obstruct the view to the eye, and the frames or the lenses may cause extra reflections. When contact lenses are used, the reflection is obtained from the surface of the contacts, instead of the cornea itself. Small, hard contact lenses may sometimes cause problems, if the lenses move around considerably on the cornea (LC Eyegaze, 2001).

For all the above-mentioned reasons, the calibration may fail even if the person is able-bodied and has normal vision. Indeed, Goldberg and Wichansky (2003) estimate that up to 20% of randomly selected subjects fail to get a good calibration. Eye tracker manufacturers typically report higher success rates. Lower success rates in real life conditions may be caused by ambient light or reflections from the environment. The user may not be sitting still enough for the system to be able to track him/her. Most systems also have an optimal distance and location concerning the positioning of the person in from of the eye-tracking device.

The problems are exacerbated when people who have severe involuntary head or eye movements try to use the systems. The person may also be lying on his or her back, or sit in a divergent position, e.g. her head rotated sideways. The current eye tracking systems have some ways of trying to cope with these problems. Most manufacturers provide accessories for positioning the eye-tracking device (and the computer screen) so that the distance and the angle of the tracker are adjustable (e.g. by using an adjustable arm). If the camera loses the eye, some of the systems automatically start searching for it. For example, the Metrovision VISIOBOARD systems camera automatically moves and scans the surroundings, trying to relocate the 'lost' eye. Whilst this can be very effective to 're-locate' the eye of a user who has moved away from the VISIOBOARD and then returns to it, it is not designed to work at the speed necessary to accommodate the head movement of a user who has ongoing, 'jerky' involuntary head movement.

The Tobii eye-tracking device tracks both eyes and is able to temporarily continue with one eyes data if the other eye is lost (Tobii, 2003). Whilst all current eye trackers have problems in tracking the eye(s) if the users eyes have a tremor or if the user has involuntary head movements, initial trials by The ACE Centre have suggested that the extent to which each system can 'cope' can vary considerably. No comparisons between the different systems in terms of their efficacy in this respect are available. For this reason, it is important for COGAIN to compare different systems with users who experience these difficulties.

If the calibration fails for the user, some systems can be used with a default calibration, and a special filtering of eye movements can be applied if the user has eye movement disorders (Charlier et al., 1997). For example, in the Metrovision VISIOBOARD system allows to use 3 x 2 matrix for people for whom the calibration fails. Naturally the accuracy of the measured point of gaze is very rough in such cases, and the objects on the screen must be large, e.g. only 6-9 large buttons visible on the screen at a time. If only a few buttons are visible at a time, normal on-screen keyboards cannot be used. The characters or commands must then be organized hierarchically. This makes the use of the system slow, because several selections are needed for one command. By using predictive algorithms, the system can speed up the selection process (Frey et al., 1990; Hansen et al., 2003b).

Figure 3.4. Two-level buttons (EagleEyes Staggered Speech application).

Whilst it is acknowledged that the use of larger targets can be slower because it might necessitate two 'hits' instead of one, it must be remembered that larger targets can make the difference between a user being able to use an eye-tracking device effectively and not using it at all.

Figure 3.5. Paul (left picture) was able to use a grid with smaller cells, whereas larger cells were more appropriate for Michael (right).

Making the cells in a grid larger can make the difference between accessing a target with two or three hits and not being able to access it at all (see Figure 3.5).

When compared with scanning, even two or three 'hits' to achieve a required target might be far less tiring, quicker and more efficient for some users than scanning. (Refer to Ahmar's case study in Chapter 6.2).

However, if everything else fails, the eyes might also be used as simple one or two-way switches and the focus can be shifted from one item to another by using a method called scanning (ten Kate et al., 1979). For example, the H.K. EyeCan VisionKey system uses a sequential scanning and requires only coarse vertical eye movements for selection (Kahn et al., 1999).

It must be noted, however, that, in practice, scanning can be considerably slower, and therefore considerably more frustrating than direct eye control for the user so that, given a choice, it is likely that the vast majority of users would choose eye control as opposed to eye blink.

Most eye tracker manufacturers report that the spatial accuracy of the system is about from 0.5 to 1 degree. Even if the calibration succeeds and the measured point of gaze is fairly accurate right after the calibration, the tracking may fail after a while because of the deterioration of a calibration that occurs over time; the calibration drifts. The drift may be caused by the change of pupil size, the eyes may become dry, or a considerable change of the angle, position and distance of the persons eyes in relation to the eye-tracking device. Systems that track both eyes cope with the drift better, because most drift effects are inversely symmetrical between the eyes (Tobii, 2003). There are also algorithms that can dynamically correct the drift (Stampe and Reingold, 1995). The accuracy of the calibration is checked on every successful selection and can then be gradually adjusted over time. The Metrovision VISIOBOARD also allows recalibration on any part of the screen. When the user fixates on any place of the screen for predefined period of time (longer that the normal duration/dwell time needed for a single or double click), a calibration point appears on that spot.

If eye gaze is used to control a mouse, not even 0.5-1 degree accuracy is enough for selecting the small elements in the standard graphical environment. The accuracy problems can be at least partly overcome by using special methods for selecting the tiny objects on the screen. For example, The Quick Glance, VISIOBOARD and ERICA use zooming, screen magnifiers and fish-eye views to select tiny icons and menus on the screen (Lankford, 2000; Rasmusson et al., 1999)

The eye-tracking devices can be table-mounted (remote) or head-mounted (worn on the head). If the device is used for interactive applications, heavy head-mounted trackers that must be tightly attached to the users head are not convenient. There are, however, lightweight head-mounted eye-tracking devices that can be used for a prolonged period of time without too much inconvenience. For example, the VisionKey by H.K. EyeCan is mounted on a pair of standard frames. If the eye-tracking system is used for social communication, the user is likely to want to use it outdoors, on the move, as well as indoors. While systems like the LC Eyegaze can be mounted to a wheelchair, and move around with the user, they are not designed for outdoor use.

With the Quick Glance II system, as with other control systems, the user's eye must be kept within in the camera's field of view. This is about 6 by 6 cm for models 2B and 2S. With the 2SH model, however, the area of 'Allowable Head Movement' is about 10 x 10 cm for model.

This model, therefore, would be the one to try for someone with involuntary head movement because they would stand a better chance of keeping maintaining the camera's view of their eye movement.

As with all eye control systems, however, the only way to evaluate their efficacy is to carry out trials with a range of users to find out how they work, in practice. For this reason, an important aim of COGAIN, on behalf of end-users, is to collaborate with them in order to provide information to assist them in making well-informed choices. This information will relate to the wide range of eye control systems available in terms of usability issues, the software that is already being used for eye control and the software that could potentially be used for eye control. In this way, we can provide support to end users and those who support them in answering such questions as this, from the questionnaire of a parent of a child who she would like to consider eye control for:

"How do we find out about the suitability of different products?"

It is anticipated that Work package 3, in collaboration with COGAIN partners, will help to provide some answers.

[Back to Contents]

3.2 Information from stakeholders

One of the aims of COGAIN Work Package 3 (User Requirements) is to trial software applications developed under COGAIN. To do this, there is a need to find out where the existing users are. Establishing direct contact with those who are already using Eye Control technology in any significant numbers is not a quick and easy process but one that will evolve over time. It will necessitate, for example, raising awareness and gaining the trust and support of a range of manufacturers, suppliers, charities, etc. Publicity for COGAIN, too, through magazine and journal articles, etc. is an essential part of the awareness-raising process and this process is already well underway throughout the COGAIN Project. It is anticipated that the impact of these efforts, in terms of establishing direct contact with a community of existing eye-control users, will be felt increasingly as the project progresses. In the meantime, though the numbers of existing long-term eye control users we have established direct contact with is small, what we have learnt from them is both rich and informative. It provides a revealing insight into what can currently be achieved and what might be achieved in the future.

Contact has already been established with a very severely disabled eye control user called Keith. The information acquired from him provides a valuable insight into the impact of eye control technology on his quality of life:

Case Study - Keith

Keith has ALS. He is completely paralysed and is only able to move his eyes. He can no longer move his head at all. He cannot blink. This means that he experiences great difficulty and discomfort and it is necessary to keep clearing his eyes in order to prevent a film building up on them.

He has been using an LC technologies Eyegaze system for 2 years and he controls it from an upright position in his wheelchair. He uses the system for about 12 hours daily. He runs his laptop through the LC system.

He would also like to be able to use the Eyegaze to take control over his environment "so I could be more independentto change TV channels, turn lights on and answer phone". He says he could do this, but he just has not purchased that system.

Whilst he has not timed the number of words per minute he achieves, but the 'eye response control' (or 'dwell select') is set to 0.20 seconds, which suggests an extremely proficient user. In the experience of The ACE Centre, achieving such a speed with any other pointer method than eye control (e.g. a headmouse) would be very difficult indeed. Eye-writing is quicker for him than when he was able to use his fingers to type: "I am faster with my eyes than I ever was when my fingers used to work".

He regards the process of eye-control of the computer as essential to his quality of life ('I would have no desire to live without this eyegaze system') and uses it for a range of activities. He uses it for social communication, writing, emailing and access to the Internet. Eye-writing is his "only way of communicating". It enables him to "still be a part of other people's lives. Plus, I can still give advice and help others." Through emailing, he keeps in daily contact with people: "It gives me an outlet to feel like I can still make a difference on somebody's life". The Internet is his "only way of keeping up with what's going on in the outside world".

If it were possible, he feels that wheelchair control using his eyes would be very beneficial, as it would provide him with "freedom from always having to ask others for help".

[Back to Contents]

3.3 Summary

At present, whilst certain systems have features that are intended to accommodate certain accessing difficulties, in practice, their effectiveness will vary for different users. Because of the lack of comparative information, it is important that, under COGAIN, the efficacy of different systems with different types of users is compared in order to provide them with the information they need to make a well-informed choice. Similarly, the way in which the software interface is designed (e.g. magnification, larger targets, etc.) offers opportunities to enable more users with accessing difficulties (e.g. those with involuntary head and eye movement) to access this technology effectively. A range of hardware and software will need to be trialled under COGAIN for comparison in relation the efficacy of its use for users with a wide range of needs.

Eye tracking systems aimed for people with disabilities include:

• EagleEyes (http://www.bc.edu/schools/csom/eagleeyes)
• Eye Response Technologies ERICA (http://www.eyeresponse.com)
• EyeTech Digital Systems Quick Glance (http://www.eyetechds.com)
• H.K. EyeCan VisioKey (http://www.eyecan.ca)
• LC Technologies Eyegaze (http://www.eyegaze.com)
• Metrovision VISIOBOARD (http://www.metrovision.fr/)
• TechnoWorks TE-9100 Nursing System (http://www.t-works.co.jp/page011.html)
• Tobii Technologies MyTobii (http://www.tobii.com/).

It is important that as many such potentially beneficial systems (and appropriate software) are trialled by a cross-section of users and potential users as possible and the information gathered made available to all users and potential users.

[Back to Contents]

4 Who is currently unable to use eye control technology?

At present, Eye Control Technology finds it difficult to cope with users who have certain physical or visual difficulties. These include, for example, those with involuntary head movement (e.g. those who have athetoid cerebral palsy) or involuntary eye movement (e.g. due to nystagmus). This chapter sets out to explore the wide range of difficulties and abilities of users with disabilities, who wish to use this technology but who, at present, are unable to do so effectively. It will consider the reasons for this and indicate the technical difficulties that need to be overcome in order to include this currently excluded group.

[Back to Contents]

4.1 What the literature says

Eye tracking for disabled people who have various physical difficulties resulting from, for example, Amyotrophic Lateral Sclerosis (ALS), head Injury, cerebral palsy, Multiple Sclerosis, Muscular Dystrophy, Spinal Muscular Atrophy, Werdnig-Hoffman syndrome, Rett syndrome, Spinal Cord Injuries resulting in Quadriplegia and "locked-in syndrome" may have difficulties using eye control technologies with respect to two possible sources of influence.

The first one is tracking the eyes themselves in terms of reliability and validity, i.e. in order to further process eye tracking data. First of all this data must reflect a) the true gaze position and b) the true temporal characteristics of the eye movement in question. The ability to reliably track users eyes is a prerequisite for the second problem: the eye tracking software has to interpret the data and respond to the user's intention in an appropriate way. From another point of view, the first problem is one of the eye-tracking hardware and the second a problem of the application software using the data from the eye tracking hardware. Influences on the hardware are closely associated with the physical abilities of the users. It is worth noting that that these physical abilities may also have an impact on certain on eye tracking in healthy subjects, but some of those listed in the next section are more prevalent in users with the aforementioned diseases. Since literature on the impact of physical abilities on eye tracking is sparse, most of the information given below came from practitioners with extensive experience.

The following physical conditions may make successful eye tracking difficult[9]:

[Back to Contents]

4.1.1 Difficulties with eye control

To have full control over commercially available eye control (as opposed to eye-blink), systems users must be able to look up, down, left and right. They must be able to direct their gaze on all areas of a computer screen. At a typical distance of a user from the screen this means that a user must be able to move their eyes approximately 25 of visual angle horizontally and 19 of visual angle vertically. In addition, they must be able to focus on a spot for at least 500ms in order for most eye typing systems to work properly. There are certain common eye movement problems that make a satisfactory calibration difficult to achieve:

Nystagmus:

This is a constant involuntary horizontal, vertical, or rotary movement of the eyeballs. It may lead to the inability of users to fix their gaze long enough to make selections. The problem here is that simply reducing the fixation dwell time threshold can increase the risk of involuntarily making selections (see Midas Touch problem below).

Strabismus:

This visual difficulty (also called heterotropia or tropia) results in an inability to attain or maintain binocular vision due to some type of muscle imbalance in one eye, causing that eye to turn in, out, up, or down relative to the other eye. Strabismus can be intermittent (occurring sometimes), constant (occurring all the time), and/or alternating (occurring sometimes with one eye and sometimes with the other eye, whether intermittently or constantly). The problem for eye tracking arising from alternating strabismus is that for monocular systems it is impossible to make sure that the tracked eye is the one not deviating from the line of sight. One solution to this problem is putting a partial eye patch over the nasal side of the eye not being observed by the camera that often solves this tracking problem. Since only the un-patched eye can see the screen, it will continuously focus on the screen. By applying only a nasal-side patch to the other eye, the user will retain peripheral vision on that side.

Visual acuity issues:

Several common vision problems may affect a user's ability to see visual elements clearly on an eye tracking system monitor. These include the following: A) Inadequate Visual acuity: The user must be able to see visual elements on the screen with adequate acuity. If, prior to an injury or the onset of an illness a user wore glasses, he may need corrective lenses to operate an eye tracking system. If a user is over 40 years old and has not had his vision checked recently, he might need reading glasses in order to see the screen clearly. In most cases, eye tracking works well with glasses. The calibration procedure should accommodate for the refractive properties of most lenses. Hard-line bifocals can be a problem if the lens boundary splits the image of the pupil, making it difficult for a system's image processing software to determine the pupil centre accurately. Graded bifocals, however, typically do not interfere with eye tracking. If users wear contact lenses that cover all or most of the cornea eye tracking generally works well. In this case, corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can interfere, if the lens moves around considerably on the cornea and causes the corneal reflection to move across the discontinuity between the contact lens and the cornea. B) Diplopia (double vision): Diplopia may be the result of an injury to the brain, or a side effect of many commonly prescribed medications, and may make it difficult for the user to fix his gaze on a given point (see strabismus). C) Blurred vision: Another occurrence associated with some brain injuries, as well as a side effect of medications (see below). Blurred vision of screen elements decreases the accuracy of eye fixations. D) Cataracts (clouding of the lens of the eye): If a cataract has formed on the portion of the lens that covers the pupil, it may prevent light from passing through the pupil to reflect off the retina. Without a good retinal reflection, eye-tracking systems based on the bright pupil method cannot accurately predict the user's eye fixations. The clouded lens may also make it difficult for a user to see text on the screen clearly. Surgical removal of the cataracts will normally solve the problem and make the use of eye tracking possible. E) Homonymous hemianopsia (blindness or defective vision in the right or left halves of the visual fields of both eyes): This may make calibration almost impossible if the user cannot see calibration points on one side of the screen.

[Back to Contents]

4.1.2 Ability to maintain a position in front of the Eyegaze monitor

It is generally easiest to run an eye tracking system from an upright, seated position, with the head centred in front of the Eyegaze monitor. However, the eye tracking system should be able to be operated from a semi-reclined position if necessary.

[Back to Contents]

4.1.3 Medication side effects that affect eye tracking

Many commonly prescribed medications have potential side effects that can make it difficult to operate an eye tracking system. Anticonvulsants (seizure drugs) can cause nystagmus, blurred vision, diplopia, dizziness, drowsiness, headache and confusion. Some antidepressants can cause blurred vision and mydriasis (abnormally dilated pupil). Some drugs commonly used to decrease muscle spasms (e.g. Baclofen), can cause dizziness, drowsiness, headache, disorientation, blurred vision and mydriasis. Mydriasis can be severe enough to block eye tracking. If the retinal reflection is extremely bright, and the corneal reflection is sitting on top of a big, bright pupil, the corneal reflection may be indistinguishable and therefore unreadable by the computer.

In addition to these physical prerequisites, which can make tracking of users eyes impossible, there is the second problem of how to interpret the gaze data of an eye tracking system. Jacob (1995) pointedly termed this difficulty the Midas Touch Problem: The System has to differentiate attentive saccades with an intended goal of communication from the low-level eye movements that are just random or provoked by external stimulation. In order to answer this question one needs a model of eye movement control in relation to information processing tasks. First of all a very basic function of eye movements is search. If a potentially important object or feature has been discovered it should be perceptually described. Such identification is only hardly possible without eye movements (see also Findlay and Gilchrist, 2003). After identification, the information can be semantically categorized and some of the information may also receive additional self-refential interpretation. Only at this point, the information may be considered to be ready for a possible communication. The crucial parameter to distinguish different levels of processing and thus different states of intention is fixation duration. In many experiments (e.g. Velichkovsky, Pomplun and Rieser, 1996) a threshold value in the order of 450-500ms has proved as a practical solution to the Midas Touch Problem. In terms of eye-tracking-based communication systems, the eye-tracking data should hence be temporarily filtered to avoid unintended selections.

[Back to Contents]

4.2 Information from stakeholders

Under the COGAIN project, it is anticipated that an effective methodology for evaluating the usability of a range of systems and software will be developed in relation to the wide range of user's abilities and requirements. However, to gain a small insight into how well eye control technology could cope with certain kinds of physical and visual difficulties, a small number of brief user trials was carried out. The aim was simply to find out how good a calibration (if any) could be achieved with a small number of users who had visual difficulties, difficulties with head control, or both. In all, there were 10 users involved in these 'introductory' user trials. They included children and adults whose range of difficulties included head injury, stroke, cerebral palsy and athetoid cerebral palsy. With several of them, along with their physical difficulties, they had an associated visual difficulty, e.g. divergent squint, nystagmus.

[Back to Contents]

User Trials

Our (very limited at this stage) user trials suggested that, to a degree, certain eye control technology is already able to cope with certain users who have certain difficulties with head and/or eye control. In turn, even if a calibration has been successfully carried out, eye control might only be effectively be used up to a certain level of accuracy. Two examples follow:

[Back to Contents]

Case study - Claire

Claire has athetoid cerebral palsy, which means it is very difficult for her to control her movements. She is very bright, literate and well motivated. She uses a special joystick to access the computer. Using the joystick and a range of specialised on-screen grids designed within SAW, she is able to use the joystick effectively and accurately to control the computer.

Nonetheless, the method is very time consuming and involves a great deal of physical effort for her because, with her particular condition, there is a great deal of involuntary movement whenever she tries to carry out a manual task. Just reaching out in order to grasp the joystick handle in the first place is, in itself, very difficult, with her hand and arm sometimes 'overshooting' the target. It is not just hand movement that has this effect. Even if Claire just tries to speak, this also triggers off a range of involuntary movements and this, too, can be tiring for her. On the other hand, when there are no physical demands on Claire, she has learnt to sit reasonably still with comparatively little involuntary movement.

When a calibration was tried with Claire, using a Tobii eye control system, the results were encouraging. The system did manage to cope with a certain amount of involuntary head movement, whether forwards, backwards or sideways. When she tried 'eye-writing', using 'dwell-select', despite the cognitive load of concentrating on eye-typing using an unfamiliar grid layout she still managed to remain comparatively still. Nonetheless, the targets had to remain reasonably large to maintain accuracy, using the type of grid shown in Figure 4.3.

Figure 4.4. Claire using two 'hits' to select a 'y' to write the word 'party'.

This grid was simply 'experimental', with no special facilities such as prediction, etc. However, Claire's successful access to the cells she wished to select gave the following, encouraging indications:

• The Tobii system was able to achieve a calibration that was accurate enough for this user who had a certain amount of involuntary head movement to accurately select a letter she required for eye-typing with two hits, e.g. selecting a cell with the letters a-f and then selecting the actual letter she wanted.
• Even without specialist or customised software, there were already strong indications that eye control might have the potential to be (a) quicker (b) less tiring and, because of the sharp reduction in involuntary head movement, more comfortable for Claire.

[Back to Contents]

Case study - Michael

Michael is in his early 40's and has a wife and three boys. He had a severe stroke about 2 years ago and now, after rehabilitation, is back home. He cannot speak but communicates by looking at letters on an E-tran frame[10].

Figure 4.5. Michael using the E-tran frame to communicate with his wife, Wendy.

Before his stroke, Michael was very active and enjoyed a wide range of leisure pursuits. Despite the stroke, he remains a very intelligent man with an excellent sense of humour. He would like to be able to access the computer quickly and efficiently in order to communicate socially and assist with his wife's business. However, at present, his only form of access to technology is via switches. This he finds very slow and frustrating and would very much like to use eye control as a quicker and easier method, if at all possible.

Because of the stroke, Michael has a certain amount of difficulty with head control. In addition, he has nystagmus, which means that he cannot fix his gaze in the same way that most other people can. Both of Michael's eyes have a significant amount of involuntary side-to-side movement (nystagmus) to become more severe when he is tired.

A Tobii system was tried with Michael. With this particular system, if there are any specific areas that require re-calibration, they can be selected and recalibrated individually.

The grid used was 'experimental', with no special facilities such as prediction, etc. The first time it was tried, in the morning, the Tobii system was able to produce a calibration that was accurate enough for Michael to be able to select a letter he required for eye-typing with two hits, e.g. selecting a cell with the letters A-F and then selecting the actual letter he wanted. Unfortunately, when it was tried in the afternoon, when Michael's involuntary eye movement was greater, finding a calibration that would enable him to reliably access the same size of cells proved very difficult. The reason for this can be clearly seen in the screenshots below:

Figure 4.9. 'Before and after' - the calibration on the left was good enough for Michael to eye-write using a large grid
but with a later calibration (right) he was unable to use the same grid effectively.

Two other eye gaze systems were tried with Michael on separate occasions, an eye analysis system under development by QinetiQ, and the Quick Glance II SH. Despite three specialists from QinetiQ working with Michael for several hours, it proved impossible to achieve a satisfactory calibration. The Quick Glance, too, was not particularly successful. Even though a calibration could be achieved, albeit with difficulty, the level of accuracy was very poor indeed - only good enough, for example, to very roughly target each area of a 2 x 2 full-screen grid. Even a 2 x 2 grid, however, could not be used functionally, because the mouse movement was not sufficiently 'dampened' or filtered. For example, it was impossible to achieve any kind of dwell select because the pointer was jumping about so much.

The user trial with Michael gave the following indications:

At present, whichever eye control system of those available was tried, it would be difficult to achieve an effective calibration for Michael when his nystagmus is at its most severe.

Even if a successful calibration can be achieved, account must be taken of the fact that the nature of Michael's visual difficulties fluctuates and a successful calibration at one moment in time might not work effectively for him on another occasion.

On the occasion when Michael did achieve a successful calibration, the significant increase in speed that he was able to achieve in comparison with switches, combined with his greater comfort and satisfaction that the Tobii gave him, meant that he was extremely enthusiastic about eye control as an access method.

His enthusiasm emphasised the importance of the need for developers to try to accommodate people like Michael by taking his kinds of accessing difficulties (involuntary head movement and fluctuating visual difficulties) into account.

[Back to Contents]

4.3 Summary

As described above, the questionnaire response from existing users of eye control was small. However, it was interesting to find a long-term user with athetoid cerebral palsy amongst those who are already an eye-control system successfully. This is encouraging, as many of those with this condition have a certain amount of involuntary head movement. Obviously, the visual and physical abilities and difficulties of any person with any given condition is different form any other person, whether they have the same condition or not. As a result, the amount of involuntary movement that results from having athetoid cerebral palsy, for example, will vary considerably. Therefore, if developers wish to meet the needs of any group of users with complex physical or visual difficulties, therefore, they will need to ensure that their systems are flexible enough to accommodate a wide range of differences.

It is important for COGAIN partners to remember that, despite the benefits that some people with disabilities are already enjoying though the use of eye control technology, many of those who would gain most benefit from its use are still excluded from using it. It is hoped that, by working closely with these potential users, many more of them will be able to use eye control technology by the time the project is completed.

[Back to Contents]

5 What potential alternatives to eye control are there?

This chapter describes a selection of access methods that might be considered as alternatives to eye control to provide a context for the use of Eye Control Technology within the wider fields of Access Technology and Disability. Without such background information, eye control technology could be applied to meet a user's needs inappropriately when alternative access technology might be better.

Eye Control Technology is just one of many ways in which even the most disabled users can access the computer. It is very important, therefore, that those who are involved with it have a clear understanding of where it stands in relation to the very wide range of hardware and software available to the disabled community. Without such an understanding, there is a danger of going to all the trouble and expense of implementing the use of Eye Control Technology with a disabled person when perhaps a cheaper, simpler, more reliable or indeed more effective control method could be provided. It is not within the remit of this document to provide information on every single access device, every single piece of Access Technology and every single piece of access software. These amount to many thousands. However, it is important that COGAIN partners are aware of the categories and purposes of access methods and devices that might provide alternatives to Eye Control Technology. An overview of the range of alternatives to Eye Control Technology is now provided in order to provide a clearer understanding of the issues involved in considering the potential benefits and limitations of Eye Control Technology in relation to other types of Access Technology.

[Back to Contents]

5.1 What the literature says

There are many alternative ways in which assistive technology devices can support people with disabilities. Most of these require a controlled physical movement. Some people might use a part of their body such as a fist, finger or foot. Some might use the blink of an eye or head movement. Obviously this needs to be a deliberate movement and not accidental, such as a tremor. A key issue is that the input device must be activated with a minimum of effort in relation to the result achieved, without causing any kind of discomfort.

Computer input devices for people with disabilities are classified by different features. According to Shein et al. (1992), there are three main groups:

• Switches
• Keyboards
• Pointing devices

In turn, there are 3 ways in which these input devices can be operated:

• Mechanical interaction
• Optical interaction
• Electrical interaction

Mechanical interaction requires the user to make a single, deliberate movement, to activate a switch, e.g. by applying direct pressure or blinking or by breath. With 'optical interaction', interaction between the user and input device occurs by means of light waves in visible or infrared form. With electrical interaction, devices use acquired electrical signals directly from the users skin or tissues.

Also it is pointed out (Shein et al., 1992) that proper seating and positioning that provides stabilization and support for the person with a physical disability. They emphasize that it is the foundation for promoting effective interaction with a computer.

[Back to Contents]

5.1.1 Devices with direct mechanical interaction

Firstly, there are switches that require direct mechanical interaction. A wide range of switches is available. Their shape, size and the pressure required to activate them depends on the requirements and abilities of the user. Some, for example, provide tactile or auditory feedback. These are a few examples from a huge range of choices:

• Pillow Switch, which is suitable for activating with the jaw
• Floor/Foot Switch
• Mercury switch that is activated by tilting 5 degrees from horizontal position
• Grasp Switch

The last one is designed to be activated with a gentle squeeze or pinch. The user must be capable of a well-controlled grasping movement and able to release the grip within a short period of time. One of the special facilities of the switch is that, if it is held for over 2 seconds, then the switch will latch or lock on. If the user does not have a controlled grasping movement, therefore, then the Grasp Switch is not the appropriate switch for them. Below is an example of a jaw-activated switch. Here movements of the jaw are used to activate the switch (Fig. 5.1). The switch also has a headset for auditory feedback.

Figure 5.1. Jaw activated switch.
1, 2 earpieces, 3 head set support, 4 chin strap, 5 adjustable securing means,
6 oval chin cup, 7 chin bar, 8 fastening means, 9 chin bar electrode, 10 conductive wire.

An example of a dual switch is a Sip/Puff switch that is activated by inhalation/ exhalation. The Origin Instruments Corporation Sip/Puff Switch (Fig 5.2) is a head-mounted accessory used to activate a two-position switch by a simple sip or puff. It consists of a head frame with attached mouth tube and a switch box connected to the head frame by a second plastic tube. Sips and puffs are converted to switch closures inside the Switch Box.

In some cases, it is not necessary to develop a new input device to enable a user to interact with the computer. One alternative is to provide the user with a special device that acts as an interface between them and a mainstream input device. For example, for users who find it difficult to control their hands but who can move their heads effectively, a special helmet can be worn. Attached to the helmet is a 'wand' with which they can access an ordinary keyboard. Alternatively, instead of attaching an enabling device to the user, a device can be attached to the mainstream device itself to make it more accessible. For example, a keyguard can be attached to an ordinary keyboard. This can enable those who have a tremor or other difficulties when trying to access a keyboard to use the holes in the keyguard to accurately guide their fingers to the target keys.

Joysticks manipulated by hand, feet, chin, tongue are used to control the cursor on screen. For example, the "Tonguepoint" system (IBM Trackpoint III Trademark) is comprised of a pressure-sensitive joystick fastened to a mouthpiece that can be operated by the user's tongue (Salem and Zhai, 1997). The joystick provides cursor-control, while two switches, a bite switch and a manual switch, allow the user to use the left/right click buttons.

[Back to Contents]

5.1.2 Devices with optical interaction

Optical systems rely on measurements of reflected or emitted light. These systems inevitably have two components: light sources and optical sensors. The light sources might be passive objects that reflect ambient light or active devices that emit internally generated light. Examples of active light sources include light-emitting diodes (LEDs), lasers, or simple light bulbs. Optical sensors can be either discrete devices (photodiodes, phototransistors and other), or image sensors (CCD, CMOS).

Lenses and apertures can be used to project images onto the sensor, indicating the angle to the source. The intensity of light reaching the sensor can be used to estimate the distance to the source. Filters can be added to selectively admit or reject certain wavelengths of light. For example, a sensor system might use infrared light sources in conjunction with filters that only admit infrared light, effectively providing a light channel separate from the ambient visible light. Examples include an eye blink detector or the Headmouse from Origin Instruments

The technology to detect a deliberate blinking action is based on the different amount of reflection from an opened and closed eye. The detection bracket houses both the IR emitting diode and corresponding IR photodetector (phototransistor) (Fig. 5.3) and can be clamped onto the frame of a normal pair of glasses (Shaw et al., 1990). The emitters and detectors (one on each side of the glasses) are focused at the same spot on the sclera, at the lateral corner of the eye. Since the eyelid is both more absorbent and less reflective than the sclera, the IR beam will be less strongly detected when the lid is closed. Thus, a clear threshold can be established between the "open" and "closed" states of the eyelid.

By setting up a 'dwell' time that exceeds the users normal blink time, the system can be set to be activated only when the user makes a deliberate 'wink'. In this way, the eye-blink becomes a single switch and can perform the same function as a mechanical switch pressed with the hand or any other part of the body. Making different dwell times perform different functions can extend the functionality of the eye-blink switch. For example, deliberately closing the eye for 0.5 seconds would produce a different result on the computer to a 'blink' of 1.0 second.

Figure 5.4. Optical blink detector. (left) basic concept, (right) detectors on bracket (Shaw et al., 1990)

For those who have good head control, there are various commercial mouse alternatives. Some systems use infrared emitters that are attached to the users glasses, headband, or cap. Other systems place the transmitter over the monitor and use an infrared reflector that is attached to the users forehead or glasses (see Fig.5.4). The users head movements control the mouse cursor on the screen. Mouse clicks are generated with a physical switch or a software interface. Evans et al. (2000) recently described a head-mounted infrared-emitting control system that is a relative pointing device and acts like a joystick rather than a mouse. Chen et al. (1999) developed a system that contains an infrared transmitter, mounted onto the users eyeglasses, a set of infrared receiving modules that substitute the keys of a computer keyboard, and a tongue-touch panel to activate the infrared beam.

From the commercial sector, the Headmouse uses a infrared to turn head movement into cursor control. Mouse functions can be controlled by dwelling over a key for a set period of time (dwell-select). Alternatively, an external switch or switches can be set up to emulate the mouse button(s), such as a. Sip/Puff Switch.

It must be acknowledged that helmets, electrodes, goggles, and mouthsticks can be uncomfortable to wear or use. Commercial head-mounted devices can often not be adjusted to fit a childs head. However, some users, in particular young children, dislike to be touched on their face and might object to any devices being attached to their heads. Non-mechanical switches, such as the Infrared Jelly Bean switch from AbleNet, Inc. can be operated by moving any part of the body through an infrared beam.

There is a range of software available that enables a camera to be used for tracking used to track body movements such as head movement. The 'CameraMouse' is software that enables for hands-free control of a computer using a video camera to track body movements, (head, for example), and convert those movements into cursor movements on a computer screen. An on-screen toolbar can allow the user to emulate all of the mouse controls.

The CameraMouse works with all standard software and requires no wires, dots, infrared beams, or anything else to be worn on the head. It tracks a body featurefor example, the tip of the nosewith a video camera and uses the detected motion of the feature to directly control the mouse pointer on a computer. The CameraMouse system currently involves two computers that are linked togethera 'vision computer' and a 'user computer'. The vision computer executes the visual tracking algorithm and sends the position of the tracked feature to the user computer. The user computer interprets the received signals and runs any application software the user wishes to use. CameraMouse will work with most USB Cameras that utilize the CCD image sensor. The Logitech QuickCam Pro 4000, 3000, Orbit and Intel Pro Video PC USB cameras have been lab tested and work well. A single-computer version of the system ha also been developed.

[Back to Contents]

5.1.3 Electrical interaction

Another approach is the Brain-Computer Interface (BCI[11]) that uses electroencephalographic (EEG) waves originating in the brain. Users interact with physical devices through nothing more than the voluntary control of their own mental activity. Between the simple EEG and the extremely invasive direct recording of neurons, a researcher might reasonably consider using another established brain-imaging technique (such as magnetoencephalography, functional magnetic resonance imaging, and emission tomography). Nevertheless, all such techniques require sophisticated devices that can be operated only in specially designed medical facilities.

One of the major limitations of BCI systems is the high potential for electromyographic (EMG) contamination. Any muscle movement on the head or neck can produce "noise" contamination from the corresponding EMG signal. From an application standpoint, this can result in difficulties for the user, especially if they have a movement-related disorder such as cerebral palsy.

The EMG/EEG-based Human-Computer Interaction system is an alternative to eye tracking systems and when combined with an on-screen keyboard is fully operational without the user having to initiate a physical movement. The system transforms biosignals into controls for two-dimensional cursor movement. In contrast to eye tracking systems, the HCI system has the potential to be relatively inexpensive and portable and eliminates some of the concerns related to eye tracking systems, such as "dwell time", user training, and loss of calibration.

Current approaches to EEG-based communication can be divided into two groups, those that use time-domain EEG components and those that use frequency-domain components. Frequency-domain methods use spectral analysis and focus on specific frequencies at specific scalp locations. The BCI developed at the Wadsworth Center uses mu (812 Hz) and/or beta (1825 Hz) rhythms recorded over sensorimotor cortex to control a cursor on a video screen. In the simplest case, the amplitude of a single spectral band at a single location on the scalp is used to determine one-dimensional (1-D) cursor movements. The user learns to control this amplitude. Cursor movements are calculated 10 times per second by an empirically derived linear equation. In offline analysis, data from the most recent sessions are used to determine the best location and frequency band for cursor control for the next sessions.

Figure 5.8. Placement of electrodes in a cap for EEG measurement (Millan, 2003).

The 'Cyberlink Interface' is a commercially available system that enables hands-free control of computers and electrical devices. The Cyberlink system detects brain and body signals by the sensors in the headband.

Figure 5.9. The Cyberlink system detects brain and body signals by the sensors in the headband.

These are amplified, digitized and transmitted to the computer in order to control a mouse or an interactive video game, navigate a business application, use a web browser, control almost any Windows application, play musical synthesizers, activate peripheral devices, adjust environmental controls, etc. It has been the experience of the ACE Centre that not all users experience success with system and those who do would find it hard to match the speed of an effective eye control system.

[Back to Contents]

5.2 Information from stakeholders

In the questionnaire, potential eye control users were asked how they felt eye control would compare with their existing access method, in general. Where an opinion was expressed (6 out of 8) they felt that eye control would be 'better' or 'much better.' Their reasons given included:

• 'His eye pointing is very good so I would hope that it would be easier and quicker.'
• 'Less effort, greater choice, more control.'
• 'Her head control is good but becomes unreliable when she is trying to press a switch.'
• 'From the limited testing I have donepotential for increasing my access speed, especially when my body gets tired.'

However, there were certain reservations expressed. These included:

• Potential difficulties in ensuring the correct positioning of the eye control device.
• Outdoor use.

They were also asked specifically about the potential benefits of writing using eye control. Of the 8 potential eye control users who expressed an opinion, 2 thought it would be 'beneficial', 4 thought it would be 'very beneficial' and one thought it would be 'not very beneficial' (he already accesses the computer effectively with mouse and keyboard). Their reasons were related to enabling greater choice, ease of use, independence and speed. Comments included:

• It would allow her to have complete flexibility over what she wants to say.
• It is likely to be easier, quicker and more accurate.

Similarly, positive comments were made about using eye control for all 'standard' PC applications, such as Internet access, email, leisure software, etc. Against these potential benefits, however, there were reservations expressed by the some of the professionals in relation to reliability and/or safety issues when it comes to either environmental control or powered mobility.

As described earlier, facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to control their environment. Of the 3 eye control users, one did not comment. Another, who already uses eye control for their environment, feels it is 'beneficial.' The other, who does not have eye control over their environment, feels it would be 'very beneficial.' Their reasons included the following:

• "Freedom"
• "I'm doing things on my own"

Of the 8 users of other forms of access technology, 4 thought it would be 'very beneficial', 2 'not beneficial', one was able to use standard technology already and the other did not comment. Reasons for considering it to be beneficial included:

• Increased independence, privacy.
• 'He has more control over his eyes than his hands so it is likely to be easier, quicker and more accurate. It would increase the options available to him.'

Their reservations related to potential difficulties with reliability and reduced portability.

• 'It would be a lot easier but reliability would be a big concern. My independence is very important and I simply would not trust it. My environmental control system is extremely portable. I cannot see an eyegaze system ever being as portable.'

Of the 5 professionals, 4 thought that the facility to control the environment using eye control is 'very beneficial' and the other 1 said they thought it was 'quite beneficial.' Their potential benefits included: control of the TV, the ability to keeping up with events, allow the user to be 'wireless' and enable them to call their caregiver, if required.

At present, as far as we are aware, powered mobility using eye control is not an option that is commercially available. However, the views of respondents on its potential use were extremely informative. Of the 3 eye control users, one did not comment, one feels it would be 'beneficial', the other 'very beneficial.' Their reasons included the following:

• "Freedom from always asking others for help"
• "I am not moving around a lot in my chair"

Of the 8 users of other forms of access technology, 2 thought it would be 'very beneficial', one 'not very beneficial', 3 'not beneficial', one was 'unsure' and one did not comment. Reasons for considering it to be beneficial included increased independence. Their reservations related to safety issues, for example, problems with looking at the computer and direction at the same time, problems due to a visual impairment, problems with outdoor use.

Of the 5 professionals, 4 thought that the facility to control a wheelchair using eye control would be 'not beneficial' and the other 1 said they did not know. Their reasons for it not being beneficial included safety issues, as 'the eyes need to be used all the time for various purposes', or errors could be made if the user 'slid down the chair' or was 'under stress'

The questionnaire responses, then, remind us that there are a range of issues to be considered in relation to choosing an access device, including reliability, safety, ease of use and comfort. When users could access other, more established devices for mobility and environmental control, there was a preference for these 'non-eye control' methods, such as switch access, because they were regarded as more 'safe' and reliable alternatives. Two case studies follow which provide examples of the way in which users select different access methods depending on which activity they wish to carry out.

[Back to Contents]

Case Study - Julius

Julius is a young man who was seriously injured as a result of an accident in Nürburg in Germany, in June 2002.

He had a spinal injury at C2/3 and some brain damage to the lower brain/brain stem (full extent unclear). He had a massive 'open book fracture' to the pelvis requiring metal plates and an abdominal cage to stabilise. For the first two months of his recovery, he was either unconscious or semiconscious. After three months, he returned to the UK. He then spent some time in the Spinal Injuries section, Aylesbury before going home in September 2002.

Julius now has a full time job working from home, using the computer and, intellectually, Julius is as able as he always has been. He operates an adapted electric wheelchair, using thumb control. At present, producing speech is not an easy process for him, making voice recognition inappropriate at this stage. He has a limited range of neck movement due to the metal pins used to fuse the break, but he does have very good head control.

Though it is difficult for him to move his arms, he is able to hold a mouse in one hand and operate the mouse buttons. He has visual difficulties in one eye that make it necessary to wear a strong prismatic lens over the right eye, which consequently appears opaque. This made the Tobii system, currently designed for use with both eyes simultaneously, difficult for him to use successfully. For Julius, whilst he has tried eye control and a headmouse as alternatives, his preference is for a 'CameraMouse' system. One important reason is that the system offers him a high level of accuracy in directly accessing all of the Windows applications he needs to use. Combined with a special on-screen keyboard, with built-in prediction, 'Skeleton Keys', Julius is quick and accurate, throughout the day, and able to access all the functions available to every other computer user. He describes himself as able to work at the speed of a "medium paced" user.

Figure 5.11. Even though 'Skeleton Keys' has small targets,
Julius is able to control it quickly and effectively using the CameraMouse.

Key factors that determine his proficiency on the system are the ability to press both buttons on the mouse quickly and selectively and what he describes as "the powerful functionality of Skeleton Keys" with which he can carry out all of the functions he would be able to on an ordinary keyboard. By comparison, at present, an eye control system would have very little to offer him.

• For Julius' computer access needs, an eye control system, no matter how good, would be unlikely to be more quick or accurate. With his CameraMouse, he is able to quickly and accurately select targets as small as the 'Minimise' and 'Close Window' buttons with ease, as well as all of the keys on his small on-screen QWERTY keyboard.
• Even a 'mid-price-range' eye control system would be far more expensive than his CameraMouse system.
• The CameraMouse can be used out of doors as well as indoors more reliably than an eye control system.

[Back to Contents]

Case Study - Paul

Paul is a young man who has cerebral palsy. He uses a powered wheelchair for mobility that he controls, albeit with difficulty, using a centrally mounted joystick. When The ACE Centre first assessed him for access technology, 9 years ago, he was able to use an ordinary keyboard and mouse. However, since then, his physical abilities have changed. The speed at which Paul has been able to move his fingers and arms has slowed down, his range of movement has decreased and the amount of pressure he is able to apply has steadily reduced. For this reason, it has been necessary for The ACE Centre to review Paul's progress and recommend any changes necessary to the access technology Paul uses on an ongoing basis.

In terms of access devices, Paul initially used an ordinary mouse and keyboard, but then, when his range of movement and strength decreased, he needed to use a 'mini-joystick' with an 'on-screen' keyboard and predictor. Now, because he can no longer apply enough pressure to the mini-joystick, he uses a SmartNav headmouse, because he still has a reasonable range of head movement.

He combines the headmouse with a 'dwell-click' utility, 'Dragger 32' that enables him to carry out all of the functions that can be achieved with the buttons of an ordinary mouse.

Initially, he kept the same on-screen keyboard and predictor as before, which was helpful because his predictor's vocabulary had gradually evolved to become increasingly personalised over several years. About a year ago, Paul decided to change to Dasher as his preferred method of text entry. With this, he estimates that he is up to about four times quicker than with his grid-based text entry system.

Apart from having an on-screen method of text entry, however, the headmouse, combined with Dragger, is used to perform exactly the same functions as an ordinary mouse. As a result, Paul has quick and efficient access to the full range of Windows software.

At present, Paul is using his headmouse very effectively. However, from past experience, it is considered possible that head movement might become increasingly difficult and tiring for him. For this reason, it is important for him to consider eye control as his next method of accessing and controlling the computer.

[Back to Contents]

5.3 Summary

There is a range of access devices available to for people with disabilities to use as alternatives to eye control, including switches, pointer-based interfaces (e.g. trackerballs, joysticks, headmice) and electrical interaction (e.g. brain-computer interface and muscle EMG). The factors involved in choosing an appropriate access device (or devices) are complex and depend on a range of issues, including safety, reliability, independence, ease of use, etc. For certain other people with disabilities, however, eye control is their only method of independent control of technology. It is important that, for these people especially, efforts are made to explore the full range of applications used by people with disabilities that can increase their independence and improve their quality of life, including social communication, powered mobility and environmental control.

[Back to Contents]

6 Choosing between Eye Control and other access methods - The concept of Usability

6.1 What the literature says

There is a wide and expanding range of user groups who may benefit from eye or head based pointing direct interaction with an interface. These user groups range from people with no disabilities who may have hands occupied with other tasks and wish to point with head or eye (Jacob, 1995), or similarly people who may have some reluctance, difficulty or discomfort moving their hands or arms (in Europe alone, 6.6% of the working population (aged 16 to 64) suffer from some form of arm or hand disability or related problem(EUROSTAT, 2002)), to people who have little, if any, bodily movement (Chapman, 1991) (such as Amyotrophic Lateral Sclerosis or Motor Neurone Disease which causes, in later stages, a form of locked-in syndrome nearly 350,000 people suffer from these disabilities worldwide (International Alliance of ALS/MND Associations)). Between these extremes lie diverse ranges of motor disabled user groups who may benefit to a greater or lesser extent from using their eyes or head to interact with an interface. These include any disabilities that cause paralysis or impairment of motor function at a high level on the body. Examples include cerebral palsy, brain injury resulting in locked-in syndrome, multiple sclerosis, musculoskeletal diseases, polio, Parkinsonism and injuries to the cervical spinal column causing tetraplegia (currently there are between 30000 and 40000 people in the UK alone with tetraplegia or paraplegia of varying levels of injury[12]).

As the level of motor disability increases, so the number of possible usable computer input devices, or input modalities (such as eye, head or hand pointing), decreases dramatically, with the majority of input devices becoming unusable once hand function is lost. As the level of motor disability approaches neck level only a range of single switch devices, some unusual and limited bandwidth[13] devices such as brain activity and muscle EMG, speech and head and eye movement could be usable with sufficient bandwidth to give interaction for these users (Bates, 2002).

Of these available modalities, it is vitally important to accurately assess which is most usable for the person relying on these limited modalities for their everyday communication needs. It is not sufficient to simply observe that the user is able to generate meaningful input or communication to a system with a given choice of modality it is quite possible that this choice of modality is not the best suited to the user. The user may struggle with the modality, or experience considerable workload and undergo considerable exertion to accomplish tasks. It is also possible that the user can only use this given modality as this is all that is available to them, in this case the user must make do with the modality, but efforts could be made to make the operation of the modality more usable.

To assess what modalities and methods are most usable for a user, a definition of what constitutes usable and usability is required, together with suitable methods of measuring usability in relation to eye based and other control modalities. Knowledge of what available modalities in what configurations would give the most usable eye, or other modality, control for the user is vital for those using these systems and also for those helping others to use these systems.

[Back to Contents]

6.1.1 Usability

A widely accepted definition of usability in the context of computer applications is the Degree to which specified users can achieve specified goals in a particular environment with effectiveness, efficiency and satisfaction (ISO DIS 9241-11). From this the notion of effectiveness refers to the completeness and accuracy with which goals can be achieved. The notion of objective efficiency refers to the effort or resources necessary to achieve these goals (Figure 6.1) and subjective satisfaction refers to the subjective assessment by the user of factors such as workload, comfort, and ease of use whilst achieving these goals (Table 6.1).

Figure 6.1. Possible eye control efficiency metrics (Bevan et al., 1995).

Satisfaction assessment areas and factors
Area	Workload	Comfort	Ease of use
Factors	Physical effort	Headache	Accuracy of pointing
	Mental effort	Eye discomfort	Speed of pointing
	Time pressure	Facial discomfort	Accuracy of selection
	Frustration	Mouth discomfort	Speed of selection
	Performance	Neck discomfort	Ease of system control

Table 6.1 Possible eye control satisfaction metrics (Bates, 2005 (Thesis); ISO, 1998; Smith, 1996; Douglas et al., 1999).

This idea of usability is well suited to examining the use of eye controlled pointing devices by motor-impaired users of computer-based systems. Specified users can refer to the level and type of motor impairment of the users. Specified goals can refer to activities associated with the operational aims of the user, for example using common software (such as word processing, web usage, email) as well as operation of the equipment associated with the pointing device itself (such as calibration). The particular environment refers to the physical environment in which the system will be used as well as the support and social context available for use (the former includes use of wheelchairs or beds, while the latter includes the provision of human helpers). These metrics all help to set and define the scenarios in which the user is operating. The definition then goes on to state the success and satisfaction of the user when using a given modality in the defined scenario.

Here it should be noted that measuring and understanding effectiveness and efficiency are more critical issues from the point of view of the motor-impaired user than the able-bodied user. Motor-impaired users may need to trade off apparently efficient ways of working which are physically demanding, against less efficient methods, which are less demanding but enable longer periods of use. Users may often have a finite amount of energy that has to be measured out over the tasks they wish to achieve. For example, using a communication aid should be sustainable throughout a day even if communication is slow, rather than only be usable for a short period of rapid communication. However there are balances here, where modalities may be chosen for their high effectiveness for short periods (such as playing a game or driving a wheelchair), will fallback modalities available for more sustained usage (such as communication). Thus measuring the satisfaction of a modality is a vital component in understanding the cost and sustainability of modality choices in terms of the available energy quota and the nature of tasks undertaken.

[Back to Contents]

6.1.2 Tools currently in use for measuring the usability of eye control

Reviewing previous work on assessing eye based pointing devices finds that tools for measuring the usability of eye based control fall into two areas; abstract target acquisition tests (for example, MacKenzie, 1992; MacKenzie, 1991; MacKenzie and Buxton, 1992; Accot and Zhai, 1997; Sibert and Jacob, 2000; Douglas et al.,1999; Murata, 1991; Istance and Howarth, 1993; Bates, 1999; Radwin et al., 1990), and simulated real world interaction sequences on user interfaces (for example, Istance et al., 1996a; Jacob, 1993; Hansen et al., 2004; Majaranta et al., 2004). To define these two areas, abstract target acquisition test tools are based on presenting the user with a sequence of targets of varying size and spatial separation on an otherwise blank screen, and asking the user to rapidly point to targets in turn (Figure 6.2). Typically, the data collected from these experiments is sparse, with the time taken to select targets and the number of errors being recorded as usability or performance metrics.

Figure 6.2. Example abstract target acquisition test.[14]

In contrast, simulated real world test tools are typically based on the user performing a small set of tasks or interaction sequences on either a real environment, or a simulated and simplified version of a real environment (Figure 6.3). Often these tests only assess one type of control, such as typing on an on-screen keyboard, that occurs in a real environment. The data from these experiments is usually determined by the nature of the assessment task, for example, words per minute for a typing task, but other metrics such as cursor paths, eye scan paths or user subjective reaction to the test are often recorded, giving a richer data set for usability assessment. The rationale behind these potentially complex real world tests is that, although frequently time consuming and laborious to conduct, the true usability of a device cannot be known unless that device is actually tested on such a real world complex environment. Hence, these tools will be better suited to usability assessment than tools or techniques aimed at simply measuring device performance.

Figure 6.3. Example Real World typing test[15].

Reviewing literature, there appears to be no standard or commonly accepted test for assessing real world usability on an environment for any pointing device. Typically real world evaluation is designed to test or assess a particular element of interaction with specific interest, rather than the full range of interaction that is possible on an environment. This is acceptable if the user only ever wishes to perform these limited tasks, but what if the user wishes to do new or different tasks will the usability of the modality be the same as before?