Gaze Controlled Games

A forthcoming overview/review paper on gaze controlled games.
Isokoski P., Joos, M., Spakov, O., & Martin, B. (in press). Gaze Controlled Games. To appear in Universal Access in the Information Society
Link: [will appear here when the paper comes out of press...]

Eye Tracker Input in First Person Shooter Games

Abstract:
We report ongoing work on using an eye tracker as an input device in first person shooter (FPS) games. In these games player moves in a three-dimensional virtual world that is rendered from the player's point of view. The player interacts with the objects he or she encounters mainly by shooting at them. Typical game storylines reward killing and punish other forms of interaction. The reported work is a part of an effort to evaluate a range of input devices in this context. Our results on the other devices in the same game allow us to compare the efficiency of eye trackers as game controllers against more conventional devices. Our goal regarding eye trackers is to see whether they can help players perform better. Some FPS games are played competitively over the Internet. If using an eye tracker gives an edge in competitive play, players may want to acquire eye tracking equipment. Eye trackers as input devices in FPS games have been investigated before (Jönsson, 2005), but that investigation focused on user impressions rather than on the efficiency and effectiveness of eye trackers in this domain. However, Jönsson's results on eye tracker efficiency in a non-FPS game were encouraging.

Reference:
Isokoski, P., & Martin, B. (2006). Eye Tracker Input in First Person Shooter Games. In Proceedings of COGAIN 2006: Gazing into the Future, 78-81.
Link: http://www.cs.uta.fi/~poika/cogain2006/cogain2006.pdf

Use of Eye Movements for Video Game Control

Abstract:
We present a study that explores the use of a commercially available eye tracker as a control device for video games. We examine its use across multiple gaming genres and present games that utilize the eye tracker in a variety of ways. First, we describe a first person shooter that uses the eyes to control orientation. Second, we study the use of eye movements for more natural interaction with characters in a role playing game. And lastly, we examine the use of eye tracking as a means to control a modified version of the classic action/arcade game Missile Command. Our results indicate that the use of an eye tracker can increase the immersion of a video game and can significantly alter the gameplay experience.

Reference:
Smith, J. D., & Graham, T. C. N. (2006). Use of Eye Movements for Video Game Control. In ACM Advancements in Computer Entertainment Technology (Hollywood, CA, USA, June 14 - 16, 2006). ACE 2006. ACM Press, New York, NY.
Link: http://www.cs.queensu.ca/~smith/papers/ace2006.pdf

If Looks Could Kill - An Evaluation of Eye Tracking in Computer Games

Abstract:
The possibility to track human eye gaze is not new. Different eye tracking devices have been available for several years. The technology has for instance been used in psychological research, usability evaluation and in equipment for disabled people. The devices have often required the user to utilize a chinrest, a bite board or other cumbersome equipment. Hence, the use of eye tracking has been limited to restricted environments.

In recent years, new non-intrusive eye tracking technology has become available. This has made it possible to use eye tracking in new, natural environments. The aim of this study was to evaluate the use of eye tracking in computer games. A literature study was made to gather information about eye tracker systems, existing eye gaze interfaces and computer games. The analysis phase included interviews with people working with human-computer interaction and game development, a focus group session and an evaluation of computer games. The result from the analysis constituted of a summary of interaction sequences, presumable suitable to control with the eyes. Three different prototypes of eye controlled computer games were developed. The first was a shoot'em up game where the player aimed with his eyes to shoot monsters that appeared in random places. The two other prototypes were developed with the Half Life Software Development Kit. In the first Half Life prototype, the player aimed a weapon with his eyes. In the second, the view of sight was controlled with the eyes. The different eye controlled game prototypes were evaluated in a usability study. The subjects played the different prototypes with mouse and eyes respectively. Their experience was evaluated with the thinking aloud method, questionnaires and an interview. The result showed that interaction with the eyes is very fast, easy to learn and perceived to be natural and relaxed. According to the usability study, eye control can provide a more fun and committing gaming experience than ordinary mouse control. Eye controlled computer games is a very new area that needs to be further developed and evaluated. The result of this study suggests that eye based interaction may be very successful in computer games.

Reference:
Jönsson, E. (2005). If Looks Could Kill - An Evaluation of Eye Tracking in Computer Games. Master's Thesis, Department of Numerical Analysis and Computer Science, Royal Insittute of Technology, Stockholm, Sweden.
Link:
http://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2005/rapporter05/jonsson_erika_05125.pdf

EyeChess: the tutoring game with visual attentive interface

Abstract:
Advances in eye tracking have enabled the physically challenged people to type, draw, and control the environment with their eyes. However, entertainment applications for this user group are still few. The EyeChess project described in this paper is a PC based tutorial to assist novices in playing chess endgames. The player always starts first and has to checkmate the Black King in three moves. First, to make a move the player selects a piece and then its destination square. To indicate that some squares could be activated, while other ones were forbidden for selection, color highlighting was applied. A square with a green highlight indicated a valid action, and the red color denoted invalid action. There were three options to make a selection: blinking, eye gesture (i.e., gazing at offscreen targets), and dwell time. If the player does not know how to solve the task, or s/he plays by making mistakes, the tutorial provides a hint. This shows up a blinking green highlight when the gaze points at the right square. Preliminary evaluation of the system revealed that dwell time was the preferred selection technique. The participants reported that the game was fun and easy to play using this method. Meanwhile, both the blinking and eye gesture methods were characterized as quite fatiguing. The tutorial was rated helpful in guiding the decision making process and training the novice users in gaze interaction.

Reference:
Spakov, O. (2005). EyeChess: the tutoring game with visual attentive interface. Alternative Access: Feelings and Games 2005, Department of Computer Sciences, University of Tampere, Finland.
Link: http://www.cs.uta.fi/~oleg/

EyeDraw: A System for Drawing Pictures with Eye Movements

Abstract:
This paper describes the design and development of EyeDraw, a software program that will enable children with severe mobility impairments to use an eye tracker to draw pictures with their eyes so that they can have the same creative developmental experiences as nondisabled children. EyeDraw incorporates computer-control and software application advances that address the special needs of people with motor impairments, with emphasis on the needs of children. The contributions of the project include (a) a new technique for using the eyes to control the computer when accomplishing a spatial task, (b) the crafting of task-relevant functionality to support this new technique in its application to drawing pictures, and (c) a user-tested implementation of the idea within a working computer program. User testing with nondisabled users suggests that we have designed and built an eye-cursor and eye-drawing control system that can be used by almost anyone with normal control of their eyes. The core technique will be generally useful for a range of computer control tasks such as selecting a group of icons on the desktop by drawing a box around them.

Reference:
Hornof, A., Cavender, A., & Hoselton, R. (2004). EyeDraw: A System for Drawing Pictures with Eye Movements. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility.
Link: http://www.cs.uoregon.edu/~hornof/downloads/ASSETS04.pdf

Design of a computer game using an eye-tracking device for eye's activity rehabilitation

Abstract:
An eye mouse interface that can be used to operate a computer using the movement of the eyes is described. We developed this eye-tracking system for eye motion disability rehabilitation. When the user watches the screen of a computer, a charge-coupled device will catch images of the user's eye and transmit it to the computer. A program, based on a new cross-line tracking and stabilizing algorithm, will locate the center point of the pupil in the images. The calibration factors and energy factors are designed for coordinate mapping and blink functions. After the system transfers the coordinates of pupil center in the images to the display coordinate, it will determine the point at which the user gazed on the display, then transfer that location to the game subroutine program. We used this eye-tracking system as a joystick to play a game with an application program in a multimedia environment. The experimental results verify the feasibility and validity of this eye-game system and the rehabilitation effects for the user's visual movement.

Reference:
Lin, C.-S., Huan C.-C., Chan C.-N., Yeh, M.-S., & Chiu C.-C. (2004). Design of a computer game using an eye-tracking device for eye's activity rehabilitation, Optics and lasers in engineering, 42(1), 91-108, Elsevier.
Link:
http://www.foylearts.net/jmagee/Bdes/des514m1/06brf514/Lin%20et%20al%202002-%20eye%20tracking.pdf

Eye movements in an Action Game Tutorial

Abstract:
Action games are controversial and discussed, at the same time they fascinate players all over the world. One way to find out what this attraction is about is to use eye tracking to explore them. This method can show explicit eye gaze direction within the game environment and at the same time point out what the mind determine as important in the different interactions in an action game tutorial. This study wants to lay out the foundations of players´ eye behaviours in the light of training, learning, social behaviour and if there are any visual reinforcements between interactive media compared to a natural situation. Action games are today classified as entertainment products with built in simulation paths at the same time as some organisations bring in commercial games for professional training or evaluating its profit. A study made last year at Rochester University showed that non-video game players could improve their visual attention. In this study, eight subjects were playing and the recording tracked every eye movement and step in choice. The results revealed that facial interest is secondary in task progression, eye behaviour patterns are similair to eye behaviour in car driving and re-fixations occured after search and shooting partly independent of background.

Reference:
Sennersten, C. (2004). Eye movements in an Action Game Tutorial. Student Paper. Department of Cognitive Science. Lund University, Sweden.
Link: http://www.sol.lu.se/humlab/eyetracking/Studentpapers/CharlotteSennersten.pdf

Gaze vs. Mouse: An evaluation of user experience and planning in problem solving games

Abstract:
The aim of this thesis is to investigate whether gaze-based interaction is a suitable means of input for problem solving games. Where a player has to use his/her eyes not only to select objects, but also to visually perceive the puzzle and plan his/her next move in order to solve the puzzle. Two common problem solving puzzles were implemented, the Sudoku and the Tile Slide puzzle (or 15 puzzle). Each puzzle can be played with eye gaze or with the mouse. Although test subjects found gaze interesting, the mouse was still the preferred mode of interaction. We found that gaze selection is more erroneous than mouse selection and that these errors can cause a player to lose concentration from the task at hand. We also found that the user interface and the interaction sequence influences both the planning strategy that the player would use and the amount of time it takes him/her to complete the task.

Reference:
Gowases Tersia (2007) Gaze vs. Mouse: An evaluation of user experience and planning in problem solving games. Master’s thesis May 2, 2007. Department of Computer Science, University of Joensuu, Finland.
Link: ftp://cs.joensuu.fi/pub/Theses/2007_MSc_Gowases_Tersia.pdf

Eye gaze assistance for a Game-like interactive task

Abstract:
Human beings communicate in abbreviated ways dependent on prior interactions and shared knowledge. Furthermore, humans share information about intentions and future actions using eye gaze. Among primates, humans are unique in the whiteness of the sclera and amount of sclera shown, essential for communication via interpretation of eye gaze. This paper extends our previous work in a Game-like interactive task by the use of computerised recognition of eye gaze and fuzzy signature based interpretation of possible intentions. This extends our notion of robot instinctive behaviour to intentional behaviour. We show a good improvement of speed of response in a simple use of eye gaze information. We also show a significant and more sophisticated use of the eye gaze information, which eliminates the need for control actions on the user’s part. We also make a suggestion as to returning visibility of control to the user in these cases.

Reference:
Tom Gedeon, Dingyun Zhu, and Sumudu Mendis (2008) Eye gaze assistance for a Game-like interactive task. International Journal of Computer Games Technology.
Link: http://www.hindawi.com/journals/ijcgt/aip.623725.html

Snap Clutch, a Moded Approach to Solving the Midas Touch Problem

Abstract:
This paper proposes a simple approach to an old problem, that of the 'Midas Touch'. This uses modes to enable different types of mouse behavior to be emulated with gaze and by using gestures to switch between these modes. A light weight gesture is also used to switch gaze control off when it is not needed, thereby removing a major cause of the problem. The ideas have been trialed in Second Life, which is characterized by a feature-rich of set of interaction techniques and a 3D graphical world. The use of gaze with this type of virtual community is of great relevance to severely disabled people as it can enable them to be in the community on a similar basis to able-bodied participants. The assumption here though is that this group will use gaze as a single modality and that dwell will be an important selection technique. The Midas Touch Problem needs to be considered in the context of fast dwell-based interaction. The solution proposed here, Snap Clutch, is incorporated into the mouse emulator software. The user trials reported here show this to be a very promising way in dealing with some of the interaction problems that users of these complex interfaces face when using gaze by dwell.

Reference:
H.O. Istance, R Bates, A. Hyrskykari and S. Vickers (2008) Snap Clutch, a Moded Approach to Solving the Midas Touch Problem Proceedings of the 2008 symposium on Eye tracking research & applications ETRA '08 ACM Press, Savannah, March 2008.

See also the related article on "Eye-tracking interface means gamers' looks can kill" in New Scientist Tech, 5 May 2008.
http://technology.newscientist.com/article/dn13830-eyetracking-interface-means-gamers-looks-can-kill.html

Evaluation of Real-time Eye Gaze Logging by a 3D Game Engine

Abstract:
Human Computer Interaction studies of visual attention in dynamic 3D computer gameplay can be greatly facilitated by automated gaze object logging implemented by integration of eye gaze tracking systems with game engines. This verification study reports the spatial and temporal accuracy of such an integrated system.

Reference:
Charlotte Sennersten and Craig Lindley (2008) Evaluation of Real-time Eye Gaze Logging by a 3D Game Engine. In Proc. 12th IMEKO TC1 & TC7 Joint Symposium on Man Science & Measurement, Annecy, 2008.
Link: http://www.bth.se/fou/forskinfo.nsf/alfs/14fc0ac35cfea843c1257464003e9022

Computational mechanisms for gaze direction in interactive visual environments

Abstract:
Next-generation immersive virtual environments and video games will require virtual agents with human-like visual attention and gaze behaviors. A critical step is to devise efficient visual processing heuristics to select locations that would attract human gaze in complex dynamic environments. One promising approach to designing such heuristics draws on ideas from computational neuroscience. We compared several such heuristics with eye movement recordings from five observers playing video games, and found that heuristics which detect outliers from the global distribution of visual features were better predictors of human gaze than were purely local heuristics. Heuristics sensitive to dynamic events performed best overall. Further, heuristic prediction power differed more between games than between different human observers. Our findings suggest simple neurally inspired algorithmic methods to predict where humans look while playing video games.

Reference:
Peters, R. J., & Itti, L. (2006). Computational mechanisms for gaze direction in interactive visual environments. Proceedings of the ACM Eye Tracking Research and Applications (ETRA) Symposium, 2006. 20-27.
Link: http://ilab.usc.edu/publications/doc/Peters_Itti06etra.pdf

Towards eye based virtual environment interaction for users with high-level motor disabilities

Abstract:
An experiment is reported which extends earlier work on the enhancement of eye pointing in 2D environments through the addition of a zoom facility, to its use in virtual 3D environments using a similar enhancement. A comparison between hand pointing and eye pointing without any enhancement shows a performance advantage for hand based pointing. However, the addition of a 'fly' or 'zoom' enhancement increases both eye and hand based performance, and reduces greatly the difference between these devices. Initial attempts at 'intelligent' fly mechanisms and further enhancements are evaluated.

Reference:
Bates, R., & Istance, H. O. (2005). Towards eye based virtual environment interaction for users with high-level motor disabilities. Special Issue of International Journal of Disability & Human Development: The International Conference Series on Disability, Virtual Reality and Associated Technologies, Vol. 4(3).
Link: http://www.icdvrat.rdg.ac.uk/2004/papers/S09_N2_Bates_Istance_ICDVRAT2004.pdf

Gaze- vs. Hand-Based Pointing in Virtual Environment

Abstract:
This paper contributes to the nascent body of literature on pointing performance in Virtual Environments (VEs), comparing gaze- and hand-based pointing. Contrary to previous findings, preliminary results indicate that gaze-based pointing is slower than hand-based pointing for distant objects.

Reference:
Cournia, N., Smith, J.D., & Duchowski, A.T. (2003). Gaze- vs. Hand-Based Pointing in Virtual Environment, in Proc. SIGCHI 2003 (Short Talks & Interactive Posters), April 5-10, 2003, Ft. Lauderdale, FL.
Link: http://andrewd.ces.clemson.edu/research/vislab/docs/chi03-short.pdf

Evaluating gaze-contingent level of detail rendering of virtual environments using visual search

Abstract:
Level of detail rendering reduces the geometric complexity of objects in virtual reality in order to reduce the computational load on the rendering system. Although the resultant increase in rendering speed is desirable, the behavioral consequences of these techniques for humans performing realistic tasks in complex virtual environments are not well understood. The current study examines the behavior of human observers in virtual environments rendered using a gaze-contingent level of detail criterion. This method takes advantage of the fact that the visual sensitivity of the human visual system is greater at the point of gaze than in the periphery by rendering objects in the periphery with less detail than objects at the point of gaze. In the experiment, participants performed a "virtual search" task, i.e. a visual search task where participants are required to pan the viewport to find a target object among distractors in a virtual environment. Gaze-contingent rendering was employed where the level of detail dropped continuously from the point of gaze. The time to detect and localize the target was measured as a function of the rate of decline in visual detail. Frame rates were allowed to increase with decreasing detail, thus keeping computational load approximately constant. Reaction times to detect the target increased with decreasing detail while reaction times to localize the target decreased with decreasing detail. These results suggest that reduced detail impedes target identification while the increased frame rates due to the reduction in detail facilitates interaction with virtual environments. Overall, these results indicate that the behavioural performance costs of gaze-contingent level of detail techniques can be offset by the behavioural performance gains due to increased rendering speed.

Reference:
Parkhurst, D., Law, I., & Niebur, E. (2001). Evaluating gaze-contingent level of detail rendering of virtual environments using visual search. In Lab Technical Report 2001-02, 1-6.
Link: http://cnslab.mb.jhu.edu/pubs/Parkhurst_etal01c.pdf

Interacting with Eye Movements in Virtual Environments

Abstract:
Eye movement-based interaction offers the potential of easy, natural, and fast ways of interacting in virtual environments. However, there is little empirical evidence about the advantages or disadvantages of this approach. We developed a new interaction technique for eye movement interaction in a virtual environment and compared it to more conventional 3-D pointing. We conducted an experiment to compare performance of the two interaction types and to assess their impacts on spatial memory of subjects and to explore subjects' satisfaction with the two types of interactions. We found that the eye movement based interaction was faster than pointing, especially for distant objects. However, subjects' ability to recall spatial information was weaker in the eye condition than the pointing one. Subjects reported equal satisfaction with both types of interactions, despite the technology limitations of current eye tracking equipment.

Reference:
Tanriverdi, V., & Jacob, R. J. K. (2000). Interacting with Eye Movements in Virtual Environments. In CHI '00 Proceedings, ACM, 265-272.
Link: http://www.cs.tufts.edu/~jacob/papers/chi00.tanriverdi.pdf

Gaze Interaction with Virtual On-Line Communities: Levelling the Playing Field for Disabled Users

Abstract:

Reference:
R Bates, H.O. Istance and S. Vickers (2008) Gaze Interaction with Virtual On-Line Communities: Levelling the Playing Field for Disabled Users. In Proceedings of the 4th Cambridge Workshop on Universal Access and Assistive Technology (CWUAAT), University of Cambridge, 13th-16th April 2008.

COGAIN - Leisure Applications

Gaze-controlled games and leisure applications available via the COGAIN web portal
Link: http://www.cogain.org/downloads/leisure-applications

SpecialEffect GameBase

The SpecialEffect GameBase provides links to accessible computer games. A group of youngsters, all with disabilities, reviewed these games and helped in the creation of the website. It is hoped that the information on this site will help other young players to find games that are suitable for their interests and abilities. Each game review provides information about how a game is controlled, how fast it is, how much it is likely to cost etc., which saves our players from having to spend time/money on games that would not be suitable for them in the first place. Our 'Comments' section provides a format where players can keep adding tips and tricks for each other.
Link: http://www.gameonbeta.org.uk/

Game Accessibility

All about game accessibility. For ALL disabled and interested gamers. Contains contributions to gaze controlled computer games.
See especially:
Resources (papers, videos etc.): http://www.game-accessibility.com/index.php?pagefile=papers
Forum: http://www.game-accessibility.com/forum/index.php

OneSwitch.Org

A resource of fun ideas and 'assistive technology' aimed at moderate to severely learning/physically disabled people.
Link: http://www.oneswitch.org.uk/

Head, Mouth and Eye Controls
At oneswitch.org you can find a list detailing a number of different styles of head, mouth and eye operated controllers. Most of these are for PCs and Apple computers, but there are alternatives for games consoles too. Some of these devices are very expensive, so it is always worth trying to track down some way of trying them out before you buy.
See http://www.oneswitch.org.uk/1/AGS/AGS-head.htm.

Software Downloads
Oneswitch.org provides more than 70 one switch games of different types (adventure, arcade classics, platformers, puzzle & skill games, race games, shoot-em-ups…) for free download. In all likelihood a large number of these are suitible for gaze control. Alongside this you will find articles, instructions and more at
http://www.oneswitch.org.uk/2library.htm
http://www.oneswitch.org.uk/4/games/0index.htm

levelgames.net

A website focusing on switch games designed to be widely accessible for players who have Muscular Dystrophy, Cerebral Palsy, Spinal Injury, Head Injury or other physical disabilities.
Link: http://www.levelgames.net/

Games designed for the MyTobii Eye Tracking System (by Oleg Špakov)

This page contains a list of applications developed to run specifically in the MyTobii environment. Each application registeres itself on installation to be recognized by MyTobii as MyTobii Partner Application.
Link: http://www.cs.uta.fi/~oleg/mytobii.html

World of Warcraft Percept Interface (by Oleg Komogortsev)

Oleg Komogortsev created an interface that allows to play computer games using gaze control without the use of mouse or keyboard. The interface was tested with the virtual reality game World of Warcraft.
Link: http://www.cs.kent.edu/~okomogor/wowpercept/wowpercept.htm

Adventure Game Studio

Adventure Game Studio (AGS for short) allows you to create your own point-and-click adventure games, similar to the early 90's Sierra and Lucasarts adventures. It consists of an easy-to-use development environment, and run-time engine. AGS is free. You need no programming experience to make a game using AGS - setting most game options is just a matter of point-and-click (though scripting is of course available if you prefer).
Link: http://www.adventuregamestudio.co.uk/

Entertainment Software designed for an EOG based Eye Tracking System: EagleEyes

This website contains various applications software designed to be run with EagleEyes and Camera Mouse and other similar systems. (Includes Games, Spell and Speak, System)
Link: http://www.bc.edu/schools/csom/eagleeyes/downloads.html

List of open source games

Open source games are computer games assembled using open-source software and open content. These games are open to modifications, such as implementing gaze control.
Link: http://en.wikipedia.org/wiki/List_of_open_source_games

Game Accessibility Suite

Code library and utilities to enhance accessibility to existing and future games.
Link: http://sourceforge.net/projects/gameaccess/

Retro Remakes Forum

Forum on game accessibility containing threads on eye- and head control.
Link: http://www.retroremakes.com/forum2/forumdisplay.php?f=84

Eye Trackers

Catalogue of currently available eye trackers for interactive applications.
Link: http://www.cogain.org/eyetrackers/

Gaze-aware Space Vampires (by Chris Schmelzle)

Chris Schmelzle trialled how adding information from eye movements to improve the game's artificial intelligence can enhance the gaming experience -- when the game enemies know where the player is looking at.
Link: http://www.cschmelzle.net/eye.html

YouTube video clips on gaze & games

• Eye Based Video Game Control: Quake 2
• Eye Based Video Game Control: Neverwinter Nights
• Eye Based Video Game Control: Missile Command
• Solitaire with an eye-tracker
• Playing Unreal Tournament with an eye-tracker
• House of the Dead with Eye Tracking
• Eye Gaze Driven Second Life - Camera, Locomotion
• Eye Gaze Interaction with World of Warcraft
• Gaze-Controlled First-Person-Shooter, Eye Tracking with IntelliGaze(tm) technology
• Dreamhack Eye-Tracking Experiment, by the BTH game research group.

SpecialEffect

SpecialEffect is a charitable organisation dedicated to helping ALL young people with disabilities to enjoy computer games. For these children, the majority of computer games are simply too quick or too difficult to play, and we can help them and their parents to find out which games they CAN play, and how to adapt those games that they can't.
Link: http://www.specialeffect.org.uk/

IGDA (International Game Developers Association) Game Accessibility Special Interest group (GA-SIG)

The GA-SIG was formed to help the game community strive towards creating mainstream games that are universally accessible to all, regardless of disability.
Link: http://www.igda.org/wiki/index.php/Game_Accessibility_SIG

Pin Interactive

Game Accessibility Development company
Link: http://www.pininteractive.com/