Abstract
Sign language is a communication medium for the deaf and vocally impaired. However, this language is not practised in public due to the deaf community being a minority and it takes time to learn and skilled manpower to assist the deaf in public interaction. Thus, this study aims to produce a Malaysian Sign Language Recognition (MSLR) application to recognise MSL alphabets to help normal people communicate with the deaf. The proposed work involves a few stages that consist of background subtraction to detect the moving hand, skin segmentation based on skin tones using YCbCr (Luminance, Chrominance) colour space for robustness in illumination and a 2D Convolutional Neural Network (CNN) model for feature extraction and classification of 24 alphabets. As MSL alphabets are similar to American Sign Language (ASL) alphabets, the ASL FingerSpelling Dataset from the University of Surrey’s Center for Vision, Speech and Signal Processing is used for model training and testing. Evaluation criteria include micro averages in Precision, Recall and F1-Score. The test accuracy is 79.54% with misclassifications on letters such as ‘E’ and ‘Q’ due to the signing orientation and similarity in finger articulation.
Keywords: 2D CNN, Computer Vision, Fingerspelling, Machine Learning, Malaysian Sign Language Recognition
Authors
Alison Liew Shu Lien [1] ; Lim Khai Yin [2]
[1][2] Department of Computer Science and Embedded Systems, Tunku Abdul Rahman University of Management and Technology, Kuala Lumpur, Malaysia
[1] alisonlsl-wm17@student.tarc.edu.my ; [2] limky@tarc.edu.my
Cite Me
Plain Text:
L.S.L.Alison, K.Y.Lim, "Gesture Recognition-Malaysian Sign Language Recognition with Convolutional Neural Network," International Conference on Digital Transformation and Applications (ICDXA) 2020, 2020, pp. 100-105, doi: https://doi.org/10.56453/icdxa.2020.1010.
BibTex:
@INPROCEEDINGS{ICDXA2020T203,
author={Alison, Liew Shu Lien and Lim, Khai Yin},
booktitle={International Conference on Digital Transformation and Applications (ICDXA) 2020},
title={Gesture Recognition-Malaysian Sign Language Recognition with Convolutional Neural Network},
year={2020},
volume={},
number={},
pages={100-105},
doi={https://doi.org/10.56453/icdxa.2020.1010}}
