Emotion recognition using EEG and physiological data for robot-assisted rehabilitation systems

Elif Gümüslü, Duygun Erol Barkana, Hatice Köse

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

20 Citations (Scopus)

Abstract

Robot-assisted rehabilitation systems are developed to monitor the performance of the patients and adapt the rehabilitation task intensity and difficulty level accordingly to meet the needs of the patients. The robot-assisted rehabilitation systems can be more prosperous if they are able to recognize the emotions of patients, and modify the difficulty level of task considering these emotions to increase patient's engagement. We aim to develop an emotion recognition model using electroencephalography (EEG) and physiological signals (blood volume pulse (BVP), skin temperature (ST) and skin conductance (SC)) for a robot-assisted rehabilitation system. The emotions are grouped into three categories, which are positive (pleasant), negative (unpleasant) or neutral. A machine-learning algorithm called Gradient Boosting Machines (GBM) and a deep learning algorithm called Convolutional Neural Networks (CNN) are used to classify pleasant, unpleasant and neutral emotions from the recorded EEG and physiological signals. We ask the subjects to look at pleasant, unpleasant and neutral images from IAPS database and collect EEG and physiological signals during the experiments. The classification accuracies are compared for both GBM and CNN methods when only one sensory data (EEG, BVP, SC and ST) or the combination of the sensory data from both EEG and physiological signals are used.

Original languageEnglish
Title of host publicationICMI 2020 Companion - Companion Publication of the 2020 International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery, Inc
Pages379-387
Number of pages9
ISBN (Electronic)9781450380027
DOIs
Publication statusPublished - 25 Oct 2020
Event2020 International Conference on Multimodal Interaction, ICMI 2020 - Virtual, Online, Netherlands
Duration: 25 Oct 202029 Oct 2020

Publication series

NameICMI 2020 Companion - Companion Publication of the 2020 International Conference on Multimodal Interaction

Conference

Conference2020 International Conference on Multimodal Interaction, ICMI 2020
Country/TerritoryNetherlands
CityVirtual, Online
Period25/10/2029/10/20

Bibliographical note

Publisher Copyright:
© 2020 ACM.

Funding

This study is supported by the Turkish Academy of Sciences in scheme of the Outstanding Young Scientist Award (TÜBA-GEBİP).

FundersFunder number
TÜBA-GEBİP
Türkiye Bilimler Akademisi

    Keywords

    • Assistive robotic systems
    • Convolutional neural networks
    • Emotion recognition
    • Gradient boosting machines
    • Physiological signal

    Fingerprint

    Dive into the research topics of 'Emotion recognition using EEG and physiological data for robot-assisted rehabilitation systems'. Together they form a unique fingerprint.

    Cite this