Facial reconstruction

Search LJMU Research Online

Browse Repository | Browse E-Theses

MediSign: An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community

Ihsan, MA, Eram, AF, Nahar, L and Kadir, MA (2024) MediSign: An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community. IEEE Access, 12. pp. 33803-33815.

[img]
Preview
Text
MediSign An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB) | Preview

Abstract

Along with day-to-day communication, receiving medical care is quite challenging for the hearing impaired and mute population, especially in developing countries where medical facilities are not as modernized as in the West. A word-level sign language interpretation system that is aimed toward detecting medically relevant signs can allow smooth communication between doctors and hearing impaired patients, ensuring seamless medical care. To that end, a dataset from twenty distinct signers of diverse backgrounds performing 30 frequently used words in patient-doctor interaction was created. The proposed system has been built employing MobileNetV2 in conjunction with an attention-based Bidirectional LSTM network to achieve robust classification, where the validation accuracy and f1-scores were 95.83% and 93%, respectively. Notably, the accuracy of the proposed model surpasses the recent word-level sign language classification method in a medical context by 5%. Furthermore, the comparison of evaluation metrics with contemporary word-level sign language recognition models in American, Arabic, and German Sign Language further affirmed the capability of the proposed architecture.

Item Type: Article
Uncontrolled Keywords: Sign language; Assistive technologies; Hidden Markov models; Medical services; Error analysis; Patient monitoring; Deafness; Convolutional neural networks; Long short term memory; Bidirectional control; Attention; BiLSTM; MobileNetV2; patient-doctor interaction; sign language; 08 Information and Computing Sciences; 09 Engineering; 10 Technology
Subjects: Q Science > QH Natural history > QH301 Biology
R Medicine > R Medicine (General)
R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine
Divisions: Pharmacy and Biomolecular Sciences
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
SWORD Depositor: A Symplectic
Date Deposited: 30 Oct 2024 14:13
Last Modified: 30 Oct 2024 14:15
DOI or ID number: 10.1109/ACCESS.2024.3370684
URI: https://researchonline.ljmu.ac.uk/id/eprint/24627
View Item View Item