Malaikar, Ms. Madhumita Mankar, Mr.
Vikrant Mehendale, Mr. Parth Mitkari
of Electronics and Telecommunication Engineering
Vidyapeeth College of Engineering, Belpada, Navi Mumbai
Eye-Writer is an Eye Tracking device and a custom software. It is an
application of eye tracking. Eye Tracking is a sensor technology to know where
your eyes are focused. Individuals
suffering from paralysis cannot move their bodies but they can still move their
eyes. So Eye-Writer is an application which will help them to draw or write
without using the rest of the body. The original Eye-Writer was meant for a
motionless head. The Eye-Writer which we have designed has a LED and camera
mounted away from the head which allows slight motion.
The Eye-Writer project is an ongoing, open
source, collaborative research effort to empower people who are suffering from
neuromuscular disorder and injury with creative technology. The Eye-Writer
itself is a pair of low-cost
eye-tracking glasses and custom software that allows artists and graffiti
writers with paralysis resulting from Amyotrophic Lateral Sclerosis, as well as
other neuromuscular disorders and injuries, to draw using only their eyes.
The original design featured a pair of
glasses, but we have a new design, which we’ve called “eye-writer
2.0” which improves the accuracy of the device, and allows people whose
heads are moving slightly to also use an eye tracker. The original eye-writer,
designed for a paralyzed Graffiti artist TEMPT1, was designed to be worn on a
completely motionless head. The 2.0 design, which uses a camera and LED system
mounted away from the head, can be used by people whose heads are moving
slightly, such as MS patients, and people who wear glasses, etc.
Eye tracking is the process of measuring either the point of gaze
(where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Most modern eye
trackers utilize near-infrared technology along with a high-resolution camera
(or other optical sensor) to track gaze direction. The underlying concept,
commonly referred to as PCCR involves the
camera tracking the pupil center, and where light reflects from the
words-: PCCR-Pupil center corneal reflection
Edmund Huey built
an early eye tracker, using a sort of contact lens with a hole for the pupil. The lens was connected to an
aluminium pointer that moved in response to the movement of the eye. Huey
studied and quantified regressions (only a small proportion of saccades are
regressions), and show that some words in a sentence are not fixated. 1
The first non-intrusive eye
trackers were built by Guy Thomas Buswell in Chicago, using beams of light that
were reflected on the eye and then recording them on film. Buswell made
systematic studies into reading and
picture viewing. 3
In the 1950s, Alfred L. Yarbus did important eye tracking research
and his 1967 book is very highly quoted. He showed the task given to a subject
has a very large influence on the subject’s eye movement. He also wrote about
the relation between fixations and interest. 5
In the 1970s, eye tracking
research expanded rapidly, particularly reading research. A good overview of
the research in this period is given by Rayner. 1
In 1980, Just and Carpenter
formulated the influential Strong
eye-mind Hypothesis, the hypothesis that “there is no appreciable lag
between what is fixated and what is processed”. If this hypothesis is
correct, then when a subject looks at a word or object, he or she also thinks
about (process cognitively), and for exactly as long as the recorded fixation.
The hypothesis is often taken for granted by beginning eye tracker researchers.
During the 1980s, the
eye-mind hypothesis was often questioned in light of covert attention, the
attention to something that one is not looking at, which people often do. If
covert attention is common during eye tracking recordings, the resulting scan
path and fixation patterns would often show not where our attention has been,
but only where the eye has been looking, and so eye tracking would not indicate
According to Hoffman, current
consensus is that visual attention is always slightly (100 to 250 ms) ahead of
the eye. But as soon as attention moves to a new position, the eyes will want
to follow. 3
We still cannot infer specific cognitive
processes directly from a fixation on a particular object in a scene. For instance, a fixation on a face in
a picture may indicate recognition, liking, dislike, puzzlement etc. Therefore
eye tracking is often coupled with other methodologies, such as introspective
verbal protocols. 5
III. EYE TRACKING
The term eye tracking here as it is use here means estimation of direction of users gaze. In most cases
estimation of gaze direction means identification of the object upon which the
accuracy of eye movement measurement heavily relies on a clear demarcation of
the pupil and detection of corneal reflection. The visible spectrum is likely
to generate uncontrolled specular reflection, while illuminating the eye with
infrared light – which is not perceivable by the human eye – renders the
demarcation of the pupil and the iris an easy task – while the light directly
enters the pupil, it just reflects from the iris. This means that a clear
reflection is generated (with little noise) and can, therefore be followed with
Near-infrared light is
directed toward the center of the eyes (the pupils) causing visible reflections
in the cornea (the outer-most optical element of the eye), which are tracked by
trackers measure rotations of the eye in one of several ways, but principally
they fall into three categories: a special contact lens with an embedded mirror
or magnetic field sensor. video based eye tracker: optical method for measuring
eye motion(IR rays). Video camera senses these rays. (Non-invasive and less
cost). electric potentials measured with electrodes placed around the eyes.
This steady electric potential field, which can also be detected in total
darkness and if the eyes are closed.
In this third method, electric signal that can be derived using two
pairs of contact electrodes placed on the skin around one eye is
called Electro oculogram (EOG). Analysing the changes in the eye movement
by reading the EOG. Horizontal and vertical movement components. Due to
potential drifts and variable relations between the EOG signal amplitudes and
the saccade sizes make it challenging to use EOG for measuring slow eye
movement and detecting gaze direction.
techniques: The most widely used current designs are video-based eye trackers.
Most modern eye-trackers use contrast to locate the center of the pupil and
use infrared and near-infrared non-collimated light to
create a corneal reflection. Two general types of eye tracking techniques are
used based on the location of the illumination source with respect to the
optics: Bright Pupil Dark Pupil.
Bright pupil is
effective in the indoors where there is a little IR interference during
monitoring. Eye movement is typically divided into fixations and saccades ,
when the eye gaze pauses in a certain position, and when it moves to another
position, respectively. The resulting series of fixations and saccades is
called a scanpath .
Most information from
the eye is made available during a fixation, but not during a saccade.
Scanpaths are useful for analyzing cognitive intent, interest, and salience.
Other biological factors (some as simple as gender) may affect the scanpath.
size / dilation
in pupil size is referred to as pupil dilation, and a decrease in size is
called pupil constriction. Pupil size primarily responds to changes in light
(ambient light) or stimulus material (e.g. video stimulus).
to the screen
Along with pupil size, eye trackers also measure the
distance to the screen and the relative position of the respondent. Leaning
forwards or backwards in front of a remote device is tracked directly and can
Most eye trackers measure the positions of the left
and right eyes independently. This allows the extraction of vergence,
i.e., whether left and right eyes move together or apart from each other. This
phenomenon is just a natural consequence of focusing near and far. Divergence
often happens when our mind drifts away, when losing focus or concentration.
Eye tracking can also provide essential information
on cognitive workload by
monitoring blinks. Cognitively demanding tasks can be associated with delays in
blinks, the so-called attentional blink. However, many
other insights can be derived from blinks. A very low frequency of blinks, for
example, is usually associated with higher levels of concentration. A rather
high frequency is indicative of drowsiness and lower levels of focus and concentration.
V. LEARNING AN END-TO-END MODEL
Working of the model is divided into three
basic steps. First, making LED illuminators for the sides of the screen and the
center. From a technical perspective, the
system works by strobing 3 IR illuminators every frame. On even frames,
it uses the center illuminator (located around the camera lens) and on odd
frames it uses the 2 side illuminators. On even frames, the pupil appears
bright, since the IR light is actually bouncing off the back of your eye, like
red eye effect. On odd frames, our pupil appears dark. The difference between
the two allows us to isolate and track the pupil in realtime. Additionally, the
glints (reflections of the IR illuminators) of the dark frame are tracked, and
these, plus the info on the pupil, is calibrated to screen position using a
least squares fitting process for an equation that provides a mapping of
glint/pupil position to screen position.
In particular, an infrared light source
illuminating the eye of a wearer at a relatively high intensity may generate
specular reflections off the wearer’s cornea, also called ‘glints’. The glints
can be imaged with an infrared camera. When the infrared light sources are
illuminated at a relatively lower intensity, determination of the pupil
location is possible. Glints, in combination with the pupil location, may be
used to accurately determine the gaze direction and eye rotation. The
determined gaze direction could be used in various eye-tracking applications.
By controlling the light sources to change intensity levels and by combining
multiple images of the eye to incorporate multiple glint locations with the
pupil location, eye tracking can be performed with better accuracy and with
fewer light sources.
Second, hacking the PS3 eye camera to
get the vertical sync (when the frame of video is being taken) and to make it
sensitive to IR.
Third, programming and building the
arduino / circuit to control the blinking. Finally, setting up the base for the
system and go through the basics of the software.
i) Using EyeWriter Software –
It is one part
of the EyeWriter project. When the person stares at the box called as the
‘Catch Me Box’, the box will turn green. When the box is fully green then the
box is caught and it will appear somewhere else. We can keep catching the boxes
to test the eye-tracking calibration.
Using EyeWriter Software- Typing
virtual keyboard is visible on the screen. We can stare at whichever key that
we wish to press. After staring, the key colour will turn green and then flash
blue. When the key flashes blue, it has been pressed. Whatever is typed can be
seen at the top of the screen.
VII. FUTURE SCOPE
The project will inspire the formation of
social networks, broadening the scope of research into its potency not only as
a tool for expression by drawing, but also in the fields of medicine and
can build and use the Eye-Writer by utilizing a combination of computers and
other familiar instruments, and installing software that is available for free.
It is probably the most versatile device for drawing with the eyes.
Thus Eye-Writer is a
boon to the disabled people suffering from neuromuscular diseases. It’s cost is
low which is affordable even to the common man. It improves the accuracy of the device and allows
people whose heads are moving even slightly to use an Eye-Writer, unlike the
previous version where eye tracking glasses were used. It
is not a bulky setup as the person using Eye-Writer will not have to wear the
T. Baltrusaitis, P. Robinson, and L.-P. Morency. Constrained local neural
fields for robust facial landmark detection in the wild. In Com-puter Vision
Workshops(ICCVW),2013IEEE InternationalConfer- ence on, pages 354–361. IEEE,
A.Vetrivel et al, International Journal of Computer Science and
Mobile Computing, Vol.5 Issue.3, pg. 423-430 , March- 2016
3 D. Beymer,
D. Russell and P.
Orton An eye tracking study of
how font size and type influence online reading, Proceedings of the 22nd
British HCI Group Annual Conference on People and Computers: Culture,
Creativity, Interaction – Volume 2, Pages 15-18, 2008.
Z Hao, and Q Lei, “Vision-based Interface: Using Face and Eye Blinking Tracking
with Camara”, Second
International Symposium on
Intelligent Information Technology Applications, pp 306-310., 2008.
N H Cuong, and H T Hoang, “Eye Gaze Detection with a Single WebCAM Based on
Geometry Features Extraction”, 2010
11th International Conference
on Control, Automation, Robotics and vision, Singapore, pp 2507-2512., 7-10th
Yang, L Wang, and B Tang,
on Gray Projection”, 2009 Third International Symposium on Intelligent Information
Z Hao, and Q Lei, “Vision-based Interface: Using Face and Eye Blinking
Tracking with Camara”, Second
International Symposium on Intelligent Information Technology Applications, pp 306-310, 2008.
8 Hari Singh and Dr. Jaswinder Singh, “Human Eye Tracking and Related
Issues: A Review”, International Journal of
Scientific and Research Publications, Volume 2, Issue 9, ISSN 2250-3153
, September 2012
9 A. Vetrivel
and T. Pandiyavathi, “Tracking Eye Movement by characterizing a sequence
through hierarchical clustering”, A.Vetrivel et al, International Journal of Computer Science and Mobile Computing, Vol.5 Issue.3, pg. 423-430, March- 2016