Abstract
This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.
| Original language | English |
|---|---|
| Pages (from-to) | 527-536 |
| Number of pages | 10 |
| Journal | Pattern Recognition and Image Analysis |
| Volume | 22 |
| Issue number | 4 |
| DOIs | |
| Publication status | Published - Dec 2012 |
| Externally published | Yes |
Funding
6. ACKNOWLEDGMENTS This research was supported by the Ministry of Edu cation of the Czech Republic, project no. ME08106; by the EU and the Ministry of Education of the Czech Republic, project no. CZ.1.07/2.3.00/09.0050; by the Grant Agency of Academy of Sciences of the Czech Republic, project no. 1ET101470416; by the Ministry of Education and Science of Russia, contract no. 11.519.11.4025; by the grant of the President of Russia, project no. MK 1880.2012.8; by the Bogazici University Research Fund, grant agreement no. 6061.
| Funders | Funder number |
|---|---|
| Bogazici University Research Fund | 6061 |
| Ministry of Edu cation of the Czech Republic | ME08106 |
| European Commission | |
| Ministerstvo Školství, Mládeže a Tělovýchovy | CZ.1.07/2.3.00/09.0050 |
| Ministry of Education and Science of the Russian Federation | 11.519.11.4025 |
| Akademie Věd České Republiky | 1ET101470416 |
Keywords
- assistive technologies
- fingerspelling recognition
- sign language