Human Action Recognition Using Deep Learning Methods on Limited Sensory Data

Nilay Tufek; Murat Yalcin; Mucahit Altintas; Fatma Kalaoglu; Yi Li; Senem Kursun Bahadir

doi:10.1109/JSEN.2019.2956901

Human Action Recognition Using Deep Learning Methods on Limited Sensory Data

Nilay Tufek, Murat Yalcin, Mucahit Altintas, Fatma Kalaoglu, Yi Li, Senem Kursun Bahadir^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

78 Citations (Scopus)

Abstract

In recent years, due to the widespread usage of various sensors action recognition is becoming more popular in many fields such as person surveillance, human-robot interaction etc. In this study, we aimed to develop an action recognition system by using only limited accelerometer and gyroscope data. Several deep learning methods like Convolutional Neural Network(CNN), Long-Short Term Memory (LSTM) with classical machine learning algorithms and their combinations were implemented and a performance analysis was carried out. Data balancing and data augmentation methods were applied and accuracy rates were increased noticeably. We achieved new state-of-the-art result on the UCI HAR dataset by 97.4% accuracy rate with using 3 layer LSTM model. Also, we implemented same model on collected dataset (ETEXWELD) and 99.0% accuracy rate was obtained which means a solid contribution. Moreover, the performance analysis is not only based on accuracy results, but also includes precision, recall and f1-score metrics. Additionally, a real-time application was developed by using 3 layer LSTM network for evaluating how the best model classifies activities robustly.

Original language	English
Article number	8918509
Pages (from-to)	3101-3112
Number of pages	12
Journal	IEEE Sensors Journal
Volume	20
Issue number	6
DOIs	https://doi.org/10.1109/JSEN.2019.2956901
Publication status	Published - 15 Mar 2020

Bibliographical note

Publisher Copyright:
© 2001-2012 IEEE.

Funding

Manuscript received October 12, 2019; accepted November 13, 2019. Date of publication December 2, 2019; date of current version February 14, 2020. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 644268. Welding of E-Textiles for Interactive Clothing (ETEXWELD) Project. The associate editor coordinating the review of this article and approving it for publication was Prof. Chang-Hee Won. (Corresponding author: Senem Kursun Bahadir.) N.Tufek,M.Yalcinand M.AltintasarewiththeDepartmentofComputer Engineering, Istanbul Technical University (ITU), 34467 Istanbul, Turkey (e-mail: [email protected]; [email protected]; [email protected]).

Funders	Funder number
Marie Skłodowska-Curie
Horizon 2020 Framework Programme	644268

Keywords

Activity recognition
CNN
LSTM
data augmentation
data balancing
deep learning

Access to Document

10.1109/JSEN.2019.2956901

Cite this

@article{7752eefc2aed4d9794c9eb874f4f471c,

title = "Human Action Recognition Using Deep Learning Methods on Limited Sensory Data",

abstract = "In recent years, due to the widespread usage of various sensors action recognition is becoming more popular in many fields such as person surveillance, human-robot interaction etc. In this study, we aimed to develop an action recognition system by using only limited accelerometer and gyroscope data. Several deep learning methods like Convolutional Neural Network(CNN), Long-Short Term Memory (LSTM) with classical machine learning algorithms and their combinations were implemented and a performance analysis was carried out. Data balancing and data augmentation methods were applied and accuracy rates were increased noticeably. We achieved new state-of-the-art result on the UCI HAR dataset by 97.4% accuracy rate with using 3 layer LSTM model. Also, we implemented same model on collected dataset (ETEXWELD) and 99.0% accuracy rate was obtained which means a solid contribution. Moreover, the performance analysis is not only based on accuracy results, but also includes precision, recall and f1-score metrics. Additionally, a real-time application was developed by using 3 layer LSTM network for evaluating how the best model classifies activities robustly.",

keywords = "Activity recognition, CNN, LSTM, data augmentation, data balancing, deep learning",

author = "Nilay Tufek and Murat Yalcin and Mucahit Altintas and Fatma Kalaoglu and Yi Li and Bahadir, {Senem Kursun}",

note = "Publisher Copyright: {\textcopyright} 2001-2012 IEEE.",

year = "2020",

month = mar,

day = "15",

doi = "10.1109/JSEN.2019.2956901",

language = "English",

volume = "20",

pages = "3101--3112",

journal = "IEEE Sensors Journal",

issn = "1530-437X",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "6",

}

TY - JOUR

T1 - Human Action Recognition Using Deep Learning Methods on Limited Sensory Data

AU - Tufek, Nilay

AU - Yalcin, Murat

AU - Altintas, Mucahit

AU - Kalaoglu, Fatma

AU - Li, Yi

AU - Bahadir, Senem Kursun

PY - 2020/3/15

Y1 - 2020/3/15

N2 - In recent years, due to the widespread usage of various sensors action recognition is becoming more popular in many fields such as person surveillance, human-robot interaction etc. In this study, we aimed to develop an action recognition system by using only limited accelerometer and gyroscope data. Several deep learning methods like Convolutional Neural Network(CNN), Long-Short Term Memory (LSTM) with classical machine learning algorithms and their combinations were implemented and a performance analysis was carried out. Data balancing and data augmentation methods were applied and accuracy rates were increased noticeably. We achieved new state-of-the-art result on the UCI HAR dataset by 97.4% accuracy rate with using 3 layer LSTM model. Also, we implemented same model on collected dataset (ETEXWELD) and 99.0% accuracy rate was obtained which means a solid contribution. Moreover, the performance analysis is not only based on accuracy results, but also includes precision, recall and f1-score metrics. Additionally, a real-time application was developed by using 3 layer LSTM network for evaluating how the best model classifies activities robustly.

AB - In recent years, due to the widespread usage of various sensors action recognition is becoming more popular in many fields such as person surveillance, human-robot interaction etc. In this study, we aimed to develop an action recognition system by using only limited accelerometer and gyroscope data. Several deep learning methods like Convolutional Neural Network(CNN), Long-Short Term Memory (LSTM) with classical machine learning algorithms and their combinations were implemented and a performance analysis was carried out. Data balancing and data augmentation methods were applied and accuracy rates were increased noticeably. We achieved new state-of-the-art result on the UCI HAR dataset by 97.4% accuracy rate with using 3 layer LSTM model. Also, we implemented same model on collected dataset (ETEXWELD) and 99.0% accuracy rate was obtained which means a solid contribution. Moreover, the performance analysis is not only based on accuracy results, but also includes precision, recall and f1-score metrics. Additionally, a real-time application was developed by using 3 layer LSTM network for evaluating how the best model classifies activities robustly.

KW - Activity recognition

KW - CNN

KW - LSTM

KW - data augmentation

KW - data balancing

KW - deep learning

UR - http://www.scopus.com/inward/record.url?scp=85079739124&partnerID=8YFLogxK

U2 - 10.1109/JSEN.2019.2956901

DO - 10.1109/JSEN.2019.2956901

M3 - Article

AN - SCOPUS:85079739124

SN - 1530-437X

VL - 20

SP - 3101

EP - 3112

JO - IEEE Sensors Journal

JF - IEEE Sensors Journal

IS - 6

M1 - 8918509

ER -

Human Action Recognition Using Deep Learning Methods on Limited Sensory Data

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this