MediSign: An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community

Ihsan, MA, Eram, AF, Nahar, L ORCID: 0000-0002-1157-2405 and Kadir, MA (2024) MediSign: An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community. IEEE Access, 12. pp. 33803-33815.

Preview

Text MediSign An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community.pdf - Published Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (2MB) | Preview

Abstract

Along with day-to-day communication, receiving medical care is quite challenging for the hearing impaired and mute population, especially in developing countries where medical facilities are not as modernized as in the West. A word-level sign language interpretation system that is aimed toward detecting medically relevant signs can allow smooth communication between doctors and hearing impaired patients, ensuring seamless medical care. To that end, a dataset from twenty distinct signers of diverse backgrounds performing 30 frequently used words in patient-doctor interaction was created. The proposed system has been built employing MobileNetV2 in conjunction with an attention-based Bidirectional LSTM network to achieve robust classification, where the validation accuracy and f1-scores were 95.83% and 93%, respectively. Notably, the accuracy of the proposed model surpasses the recent word-level sign language classification method in a medical context by 5%. Furthermore, the comparison of evaluation metrics with contemporary word-level sign language recognition models in American, Arabic, and German Sign Language further affirmed the capability of the proposed architecture.

Item Type:	Article
Uncontrolled Keywords:	Sign language; Assistive technologies; Hidden Markov models; Medical services; Error analysis; Patient monitoring; Deafness; Convolutional neural networks; Long short term memory; Bidirectional control; Attention; BiLSTM; MobileNetV2; patient-doctor interaction; sign language; 08 Information and Computing Sciences; 09 Engineering; 10 Technology
Subjects:	Q Science > QH Natural history > QH301 Biology R Medicine > R Medicine (General) R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine
Divisions:	Pharmacy and Biomolecular Sciences
Publisher:	Institute of Electrical and Electronics Engineers (IEEE)
Date of acceptance:	21 February 2024
Date of first compliant Open Access:	30 October 2024
Date Deposited:	30 Oct 2024 14:13
Last Modified:	03 Jul 2025 18:30
DOI or ID number:	10.1109/ACCESS.2024.3370684
URI:	https://researchonline.ljmu.ac.uk/id/eprint/24627

View Item