Abstract—Emotions and facialexpressions plays an important role in communication in social interaction withother human beings which delivers rich information about their mood. The “BLUEEYES TECHNOLOGY” aims to create computational mechanisms that have perceptual andsensory abilities like those of human beings which enables the computer togather facts about humans and interact with them. This paper implements thedetection of emotions (happy, sad, fear, surprised, disgust and anger) bytaking in account the human eye expressions and by using an emotion mouse. The emotionmouse obtains physiological data and emotional state of a person throughthe single touch of mouse having different sensors. Emotions are alsodetermined by human eye expression in which the eye region from a videosequence is analyzed.
From the different frames of the video stream, the human eyes can beextracted using the edge operator and then can be classified using a SupportVector machine (SVM) classifier. After the classification we use standardlearning tool, Hidden Markov Model (HMM) to recognize the emotions from thehuman eye expressions. After the detection of emotion, suitable audiotrack will be played. Keywords- Blue eyes, emotion mouse, emotionrecognition, eye expressions, SupportVector Machine (SVM), HiddenMarkov Model (HMM).
I. INTRODUCTIONThe”BLUE EYES” technology aims at creating computational machines by adding incomputers some amazing perceptual abilities that helps them to verify human’sidentity, feel their presence, and interact with them. Human recognition dependsprimarily on the stability to observe, understand, and integrate audio/visualsand sensoring information. Blue eyes technology makes a computer to sense andunderstand user’s feelings and their behavior and enables the computer torespond according to the detected emotional level.
The chief aim of blue eyestechnology is to give human abilities or power to a computer, so that themachine can naturally interact with human beings as humans interacts amongthemselves.The projectedmethodologies in this paper detect human emotions are emotion mouse and emotionrecognition by human eye expressions. Emotion mouse is an input device which isdesigned in such a way that it can track the emotions of a user by a simpletouch on it. The emotion mouse is used to evaluate and identify the user’semotions (such as happy, sad, anger, fear, disgust, surprised, etc). when the useris interacting with computer. Human’s emotion recognition isan important component for efficient man-machine interaction. It plays acritical role in communication by allowing people to express oneself beyond theverbal domain. Analysis of emotions from human eye expression involves thedetection and categorization of various human emotions or different state ofmind.
For example, in security and surveillance, they can predict the offenderor criminal’s behavior by analyzing the images of their face from the frames ofthe video sequence. The analysis of human emotions can be applied in a varietyof application domains, such as video surveillance and human – computerinteraction systems. In some cases, the results of such analysis can be appliedto identify and categorize the various human emotions automatically from thevideos. II. RELATED WORKManyapproaches for blue eye technology and human emotion recognition have beenproposed in the last two decades.MiznaRehman Mizna et.
al. 1 this paperpresents a technique which identifies human emotions (happy, surprise, sad orexcited ) using image processing by taking out only the eye portion from thecaptured image which is further compared with images that are already stored indatabase. This paper intends two results of emotional sensory world.First, observation reveals the fact that different eye colors and theirintensity results in change in emotions. It changes without giving anyinformation on shape and actual detected emotion.
It is used to successfullyrecognize four different emotions of eyes. S.R. Vinothaet. al. 2, this paper uses the feature extraction technique to extract theeyes, support vector machine (SVM) classifier and a HMM to build a humanemotion recognition system.
The proposed system presents a human emotion recognition system thatanalyzes the human eye region from video sequences. From the frames of thevideo stream the human eyes can be extracted using the well-known canny edgeoperator and classified using a non – linear Support Vector machine (SVM)classifier. Finally, standard learning tool is used, Hidden Markov Model (HMM)for recognizing the emotions from the human eye expressions.Mohammad Soleymani et. al. 3 this paper presents themethodology in which instantaneous detection of the user’s emotions from facialexpression and electroencephalogram (EEG) signals is used. A set of videoshaving different emotional level were shown to a group of people and their physiologicalresponses and facial expressions were recorded. Five annotators annotates the valence (from negative to positive) in theuser’s face videos.
A continuous annotation of arousal dimensions and valenceis also taken for stimuli videos. Continuous Conditional Random Fields (CCRF)and Long-short-term-memory recurrent neural networks (LSTM-RNN) were used indetecting emotions continuously and automatically. The analyzed effect of theinterference of facial muscle activities on EEG signals shows that most of theemotionally valued content in EEG features are as a result of this interference.However, the arithmetical analysis showed that EEG signalscarries complementary information in presence of facial expressions.T. Moriyama et. al. 4 thispaper presents a system that has capabilities of giving detailedanalysis of eye region images in terms of the position of the iris, angle ofeyelid opening, and the texture, shape and complexity of the eyelids.
Thesystem uses an eye region model that parameterizes the motion and finestructure of an eye. The structural factors represent structural individualityof the eye, including the colour and size of the iris, the complexity, boldnessand width of the eyelids, the width of the illumination reflection on the bulgeand the width of the bulge below the eye. The motion factors represent movementof the eye, including the 2D position of the iris and the up-down motion andposition of the upper and lower eyelids. Renu Nagpal et.
al. 5 this paper presents the world’s first publiclyavailable dataset of labeled data that has been recorded over the Internet ofpeople naturally viewing online media. The AM-FED contains, 1) More than 200 webcamvideos recorded in real-world conditions, 2) More than 1.5 lakhs frames labeledfor the presence of 10 symmetrical FACS action units, 4 asymmetric (unilateral)FACS action units, 2 head movements, smile, general expressiveness, featuretracker fails and gender, 3) locations of 22 automatically detect landmarkpoints, 4) baseline performance of detection algorithms on this dataset and baselineclassifier outputs for smile.
5) Self-report responses of familiarity with,liking of and desire to watch again for the stimuli videos. This represents arich and extensively coded resource for researchers working in the domains offacial expression recognition, affective computing, psychology and marketing.The videos in this dataset were recorded in real-world conditions. Inparticular, they exhibit non-uniform frame rate and non-uniform lighting. Thecamera position relative the viewer varies from video to video and in somecases the screen of the laptop is the only source of illumination.
The videoscontain viewers from a range of ages and customs some with glasses and facialhair. The dataset contains a large number of frames with agreed presence offacial action units and other labels. III.
METHODOLOGYUSEDA. EmotionRecognition From Human EyesFacial expressions play an vitalrole in communications in social interactions with other human beings which conveysinformation about their emotions. The most crucial feature of human interactionthat grants naturalism to the process is our ability to conclude the emotionalstates of others. Our goal is to classify the different human emotions fromtheir eye expressions. The proposed system presents a human emotion recognitionsystem that analyzes the human eye region from thevideosequences.
From all the frames of the video stream the humaneyes can be extracted using the well-known canny edge operator and classifiedusing a non – linear Support Vector machine (SVM) classifier. Finally, astandard learning tool is used, Hidden Markov Model (HMM) for recognizing theemotions from the human eye expressions. Surprised Sad Happy Anger Fear Disgust Fig. 1: Sample eyeexpressionsHuman emotionrecognition is an important component for efficient human – computerinteraction. It plays a critical role in communication, allowing people toexpress themselves beyond the verbal domain. Analysis of emotions from humaneye expression involves the detection and categorization of various humanemotions and state of mind.
The analysis of human emotions can be applied in avariety of application domains, such as video surveillance and human – computerinteraction systems. In some cases, the results of such analysis can be appliedto identify and categorize the various human emotions automatically from the videos.The six primary or main types of emotions are shown in Fig.
1: surprised, sad,happy, anger, fear, disgust. Our method is to use the feature extractiontechnique to extract the eyes, support vector machine (SVM) classifier and aHMM to build a human emotion recognition system. Themethodology of emotionrecognition from human eye expression is shown in Fig. 2. In thismethodology image of the user sitting in front of the camera is captured. Thenimage representing a set of frames is preprocessed and a noise free image isobtained.
The noise free image is edge detected using Canny Edge Operator.Using the feature extraction process, the eye regions are extracted from theresultant edge detected image. The extracted eye regions are classified usingSVM classifier. Finally, the corresponding emotions are recognized. B. EmotionMouseOneproposed, non-invasive method for gaining user information through touch is viaa computer input device, the mouse. This then allows the user to relate thecardiac rhythm, the body temperature and other physiological attributes with the mood.
Fig. 3: Block Diagramof Emotion Mouse The block diagram of emotion mouse is shownin Fig. 3, this device can measure heart rate and temperature and matches themwith six emotional states: happiness, surprise, anger, fear, sadness anddisgust. The mouse includes a set ofsensors, including infrared detectors and temperature-sensitive chips. Thesecomponents can also be crafted into other commonly used items such as theoffice chair, the steering wheel, the keyboard and the phone handle.Integrating the system into the steering wheel, for instance, could allow analert to be sounded when a driver becomes drowsy. Heart rateis taken by IR on the thumb and temperature is taken using a thermistor chip.
These values are input into a series of discriminate function analyses andcorrelated to an emotional state. Specifically, for the mouse, discriminatefunction analysis is used in accordance with basic principles to determine abaseline relationship, that is, the relationship between each set ofcalibration physiological signals and the associated emotion. IV. SYSTEMMODEL In this system, two methodologies namelyemotion mouse and emotion recognition from eye expression are used. Emotionmouse will consider the physiological as well as biological parameters such ascardiac rhythm and body temperature, whereas on the other side emotionrecognition from human eye expression considers facial expression for thedetection of human emotion and mood. Fig. 4: Block diagramof the systemFig.
4 shows theblock diagram of the system. In this system the data from the heartbeat sensorand temperature sensor of the emotion mouse is given to the microcontroller.The output of the microcontroller is then fed to the computer. The value ofheartbeat sensor and temperature sensor is compared with the standard range of eachemotion and the suitable emotion is selected on the other hand a webcam isconnected with the computer which will take the image of the person from avideo sequence and will further recognize the emotion by detecting the eyepart. The captured eye section will be compared to the images stored indatabase to detect mood of the person. After detecting the mood, the musicoraudio command is played according to the detected mood.
V. RESULTIn proposed system, there are two results ofthe mentioned methodologies. Firstly, different eye expressions of the differentpeople are taken in consideration by edge detection of eyes. Further each eyeexpression is categorized into a given set of emotions (happy, sad, fear,surprised, disgust, anger} to take in account a single standard expression foreach emotion. Thus emotion of a person can be detected by comparing the eyeexpression of the person with the standard eye expressions of each emotion.Secondly, the values of heartbeat sensor and temperature sensor are comparedwith the standard value range of each emotion and accordingly the value rangeof a emotion that matches with the data values of the user is considered as theemotional state of the user. According to the detected emotion the music oraudio command is played.
VI. CONCLUSIONRecent research documents tellthat the understanding and recognition of emotional expressions plays a veryimportant role in the maintenance and development of social relationships. Thispaper gives an approach of creating computational machines that have perceptualand sensory ability like those of human beings which enables the computer togather information about you through special techniques like facial expressionsrecognition and considering biological factors such as cardiac rhythm and bodytemperature. This makes it possible for computer and machines to detect theemotion of the human and respond to it.