Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

A. Alp Kindiroglu; H. Yalcin; O. Aran; M. Hrúz; P. Campr; L. Akarun; A. Karpov

doi:10.1134/S1054661812040086

Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

A. Alp Kindiroglu
, H. Yalcin
, O. Aran
, M. Hrúz
, P. Campr
, L. Akarun
, A. Karpov

Bogazici University
IDIAP Research Institute
University of West Bohemia
Russian Academy of Sciences

Araştırma çıktısı: Dergi yayını › Makale › Hakem

7 Atıf (Scopus)

Özet

This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

Orijinal dil	İngilizce
Sayfa (başlangıç-bitiş)	527-536
Sayfa sayısı	10
Dergi	Pattern Recognition and Image Analysis
Hacim	22
Basın numarası	4
DOI'lar	https://doi.org/10.1134/S1054661812040086
Yayın durumu	Yayınlandı - Ara 2012
Harici olarak yayınlandı	Evet

Finansman

6. ACKNOWLEDGMENTS This research was supported by the Ministry of Edu cation of the Czech Republic, project no. ME08106; by the EU and the Ministry of Education of the Czech Republic, project no. CZ.1.07/2.3.00/09.0050; by the Grant Agency of Academy of Sciences of the Czech Republic, project no. 1ET101470416; by the Ministry of Education and Science of Russia, contract no. 11.519.11.4025; by the grant of the President of Russia, project no. MK 1880.2012.8; by the Bogazici University Research Fund, grant agreement no. 6061.

Finansörler	Finansör numarası
Bogazici University Research Fund	6061
Ministry of Edu cation of the Czech Republic	ME08106
European Commission
Ministerstvo Školství, Mládeže a Tělovýchovy	CZ.1.07/2.3.00/09.0050
Ministry of Education and Science of the Russian Federation	11.519.11.4025
Akademie Věd České Republiky	1ET101470416

Belgeye Erişim

10.1134/S1054661812040086

Diğer Dosyalar ve Linkler

Link to publication in Scopus

Alıntı Yap

@article{ae45a99bb3484e3dbe8a8dda436dcd4f,

title = "Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled",

abstract = "This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.",

keywords = "assistive technologies, fingerspelling recognition, sign language",

author = "Kindiroglu, \{A. Alp\} and H. Yalcin and O. Aran and M. Hr{\'u}z and P. Campr and L. Akarun and A. Karpov",

year = "2012",

month = dec,

doi = "10.1134/S1054661812040086",

language = "English",

volume = "22",

pages = "527--536",

journal = "Pattern Recognition and Image Analysis",

issn = "1054-6618",

publisher = "Pleiades Publishing",

number = "4",

}

TY - JOUR

T1 - Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

AU - Kindiroglu, A. Alp

AU - Yalcin, H.

AU - Aran, O.

AU - Hrúz, M.

AU - Campr, P.

AU - Akarun, L.

AU - Karpov, A.

PY - 2012/12

Y1 - 2012/12

N2 - This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

AB - This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.

KW - assistive technologies

KW - fingerspelling recognition

KW - sign language

UR - https://www.scopus.com/pages/publications/84871324891

U2 - 10.1134/S1054661812040086

DO - 10.1134/S1054661812040086

M3 - Article

AN - SCOPUS:84871324891

SN - 1054-6618

VL - 22

SP - 527

EP - 536

JO - Pattern Recognition and Image Analysis

JF - Pattern Recognition and Image Analysis

IS - 4

ER -

Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

Özet

Finansman

Belgeye Erişim

Diğer Dosyalar ve Linkler

Parmak izi

Alıntı Yap