Emotion Recognition from EEG and Facial Expressions: A Multimodal Approach

Valentina Chaparro, Alejandro Gomez, Alejandro Salgado, O. Lucia Quintero, Natalia Lopez, Luisa F. Villa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

The understanding of a psychological phenomena such as emotion is of paramount importance for psychologists, since it allows to recognize a pathology and to prescribe a due treatment for a patient. While approaching this problem, mathematicians and computational science engineers have proposed different unimodal techniques for emotion recognition from voice, electroencephalography, facial expression, and physiological data. It is also well known that identifying emotions is a multimodal process. The main goal in this work is to train a computer to do so. In this paper we will present our first approach to a multimodal emotion recognition via data fusion of Electroencephalography and facial expressions. The selected strategy was a feature-level fusion of both Electroencephalography and facial microexpressions, and the classification schemes used were a neural network model and a random forest classifier. Experimental set up was out with the balanced multimodal database MAHNOB-HCI. Results are promising compared to results from other authors with a 97% of accuracy. The feature-level fusion approach used in this work improves our unimodal techniques up to 12% per emotion. Therefore, we may conclude that our simple but effective approach improves the overall results of accuracy.

Original languageEnglish
Title of host publication40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages530-533
Number of pages4
Volume2018-July
ISBN (Electronic)9781538636466
DOIs
StatePublished - 26 Oct 2018
Event40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018 - Honolulu, United States
Duration: 18 Jul 201821 Jul 2018

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume2018-July
ISSN (Print)1557-170X

Conference

Conference40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018
CountryUnited States
CityHonolulu
Period18/07/1821/07/18

Fingerprint Dive into the research topics of 'Emotion Recognition from EEG and Facial Expressions: A Multimodal Approach'. Together they form a unique fingerprint.

  • Cite this

    Chaparro, V., Gomez, A., Salgado, A., Quintero, O. L., Lopez, N., & Villa, L. F. (2018). Emotion Recognition from EEG and Facial Expressions: A Multimodal Approach. In 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018 (Vol. 2018-July, pp. 530-533). [8512407] (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; Vol. 2018-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EMBC.2018.8512407