Development of EEG based Emotion Classifier

Abstract

Analysis and study of abstract human relations have always posed a daunting challenge for technocrats engaged in the field of psychometric analysis. The study on emotion recognition is all the more demanding as it involves integration of abstract phenomenon of emotion causation and emotion appraisal through physiological and brain signals. Emotion is most commonly defined as short and intense reaction of humans occurring on account of a stimulus. Occurrence of emotion may bring a noticeable change in physiological parameters such as respiration rate, heart rate, Galvanic Skin Resistance (GSR), body temperature and ElectroEncephaloGram (EEG) etc. Changes in physical parameters such as color of the skin, eye gaze, eye blink rate and shape of the face are also perceived. The study of complex human emotions for developing an affective Brain Computer Interface (BCI) has for long been an area attracting biomedical scientists. Moreover emotion recognition plays an important role for all the personnel involved in mission critical tasks like for pilots, nuclear plant operators and air traffic controllers etc. The challenge to develop an affective BCI demanded understanding of emotions psychologically, physiologically as well as analysis from engineer’s point of view. To make the analysis and classification of emotions possible, emotions have been represented in a 2-dimensional or 3-dimensional space represented by arousal and valence domains or arousal, valence and dominance domains respectively. Interestingly, the classification of emotions along any of the domains is possible by utilizing the orthogonal nature of emotions. One of the effective ways to classify emotions is by use of Event Related Potential (ERP) of EEG signals. This requires projection of emotion evoking stimulus on one computer system while simultaneously putting a mark on another computer system acquiring EEG. It is generally achieved by using costly modules to synchronize stimulus presentation system with the data acquisition system. Apart from emotion recognition system, this study describes an innovative low cost technique to achieve simultaneous triggering on the second computer system using parallel operation of mechanical keyboards. The latency aspect of both USB and PS/2 keyboards with their two keys galvanically connected have been experimentally analyzed and compared. The synchronization error between the two USB keyboards has been found to be lower than or equal to 1 millisecond for nearly 70% of keystrokes. Even in the worst case the synchronization error does not exceed 8 millisecond. Our window of ERP is ±20 millisecond and hence the error of this magnitude is acceptable. The use of this synchronization setup has saved expenditure to the tune of $3000. EEG signals have been acquired from 24 right handed male subjects to classify emotions into four classes, namely low valence high arousal (LVHA), high valence high arousal (HVHA), high valence low arousal (HVLA) and low valence low arousal (LVLA). For emotion evocation, the visuals from International Affective Picture System (IAPS) have been used. For each class of emotion, 40 IAPS images classified on the basis of arousal and valence on a scale of 1 to 9 have been used with an epoch time of 2.5 second. The evoked EEG signals have been acquired in a unipolar mode on 10 Ag/AgCl electrodes namely Fp1, Fp2, F3, F4, F8, Fz, Cz, Pz, P3 and P4 at a sampling frequency of 500 samples per second by using Biopac MP150 system and EEG100C EEG cap. Emotion classification using EEG signals can primarily be done either in offline mode by taking average of EEG signals acquired from several trials or in online mode by taking a single trial. In this study, emotion classification has been obtained by taking both the cases into consideration viz; by using single trial and average of EEG signals. The single trial ERP features have been used for both subject dependent and subject independent emotion classification whereas average ERP features have been used for subject independent emotion classification. Apart from ERP features, the difference of ERPs both average and single trial have been used to develop subject independent four class emotion classifiers. In other words, five categories of four class emotion classifier have been developed and reported in this study, namely, subject dependent emotion classifier using single trial ERP features (accuracy 68.2%), subject independent emotion classifier using single trial ERP features (accuracy 39%), subject independent emotion classifier using difference of single trial ERP features (accuracy 55%), subject independent emotion classifier using average ERP features (accuracy 83%) and subject independent emotion classifier using difference of average ERP features (accuracy 77%). In all cases we have considered self assessment as gold standard for training, testing and validation of four class emotion classifier. We have found that the subject independent emotion classifier using average ERPs has the best accuracy of 83%. The results are better as compared to the existing study on average ERPs. The existing study on average ERPs reported four class emotion classification accuracy in the range of 68 - 82% with mid range accuracy of 75% whereas in the proposed classifier, the four class emotion classification accuracy lies between 82% - 88% with mid range of 85%. The proposed classifier is better in performance on account of electrode selection, order of Support Vector Machine (SVM) polynomial classifier and feature reduction. It is prudent to mention here that the subject dependent emotion classifier requires training each time the subject is to be tested for emotion because of the day to day variations in external and internal conditions. This limits its practical utility. The subject independent classifier is trained by collecting several trials on different days from large number of subjects possessing unique behavior and personality. Thus a subject independent classifier once trained would have more practical utility in classifying emotions without the need of training again before testing on subjects. The advantage of single trial emotion recognition is instantaneous result, while in offline mode; it takes several minutes to reach any conclusion. Since emotion is a short lived phenomenon, assuming that this remains the same for several minutes of data acquisition, may be erroneous. This at best would give the average emotional state. However, the four class classification accuracy using average and difference of average ERP attributes is much higher as compared to the accuracies obtained using single trial modes. This is due to the fact that averaging eliminates the noise interference in signals. To improve the accuracy results using single trial EEG, difference of single trial ERP attributes for classification of emotions have been proposed in this study. Taking a difference of local maxima and minima (maximum ERP and minimum ERP) acquired within the fixed latency period eliminates the common noise affecting the acquired signal. This is evident from the emotion classification results obtained on difference of single trial ERP attributes. Apart from the development of emotion classifier, an intervention technique has been applied on 20 right handed male subjects. These 20 subjects had been found to be in LVHA state of emotion using self assessment and EEG classifier. Intervention was given to these 20 subjects with an aim to bring them to HVLA state. Of the 20 subjects, 16 could comply with the intervention and reported through self assessment the transition to HVLA state. Application of a four class subject independent emotion classifier validated 13 of 16 subjects in HVLA state.