Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

A. Alp Kindiroglu; H. Yalcin; O. Aran; M. Hrúz; P. Campr; L. Akarun; A. Karpov

doi:10.1134/S1054661812040086

Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

A. Alp Kindiroglu, H. Yalcin, O. Aran, M. Hrúz, P. Campr, L. Akarun, A. Karpov

Research output: Contribution to journal › Article › peer-review

7 Citations (Scopus)

Abstract

This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

Original language	English
Pages (from-to)	527-536
Number of pages	10
Journal	Pattern Recognition and Image Analysis
Volume	22
Issue number	4
DOIs	https://doi.org/10.1134/S1054661812040086
Publication status	Published - Dec 2012
Externally published	Yes

Keywords

assistive technologies
fingerspelling recognition
sign language

Access to Document

10.1134/S1054661812040086

Cite this

@article{ae45a99bb3484e3dbe8a8dda436dcd4f,

title = "Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled",

abstract = "This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.",

keywords = "assistive technologies, fingerspelling recognition, sign language",

author = "Kindiroglu, {A. Alp} and H. Yalcin and O. Aran and M. Hr{\'u}z and P. Campr and L. Akarun and A. Karpov",

year = "2012",

month = dec,

doi = "10.1134/S1054661812040086",

language = "English",

volume = "22",

pages = "527--536",

journal = "Pattern Recognition and Image Analysis",

issn = "1054-6618",

publisher = "Pleiades Publishing",

number = "4",

}

TY - JOUR

T1 - Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

AU - Kindiroglu, A. Alp

AU - Yalcin, H.

AU - Aran, O.

AU - Hrúz, M.

AU - Campr, P.

AU - Akarun, L.

AU - Karpov, A.

PY - 2012/12

Y1 - 2012/12

N2 - This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

AB - This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

KW - assistive technologies

KW - fingerspelling recognition

KW - sign language

UR - http://www.scopus.com/inward/record.url?scp=84871324891&partnerID=8YFLogxK

U2 - 10.1134/S1054661812040086

DO - 10.1134/S1054661812040086

M3 - Article

AN - SCOPUS:84871324891

SN - 1054-6618

VL - 22

SP - 527

EP - 536

JO - Pattern Recognition and Image Analysis

JF - Pattern Recognition and Image Analysis

IS - 4

ER -

Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this