From Gesture to Communication: Recognition of Sign Languages with the Help of AI-Powered Wearable Bracelet, Turkish Sign Language Case

Deaf people around the world face great difficulties in communicating in daily life. Sign language is one of the main tools these individuals use to express themselves, but the fact that sign language is not widely known and understood further deepens communication barriers. In this context, the ability to recognize sign language into text or audio emerges as an important solution that will enable deaf individuals to participate more actively in social life. This study aims to recognize Turkish Sign Language (TSL) in real-time using wearable sensor technologies with the help of machine learning. Within the scope of the project, a smart bracelet containing an accelerometer, and a gyroscope and being appropriate for future installation of different sensors like an electromyography sensor (EMG) was designed and sensor data of ten basic sign expressions made by nine different people were collected and augmented with appropriate noise-adding/removing methods. These data were processed with Extreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), and Long Short-Term Memory (LSTM) algorithms. Among the models, LSTM has the highest average accuracy rate of %92.5. This study offers a potential solution for deaf individuals to communicate more comfortably in society and presents a data set that will form the basis of future studies.