Tools
Tools
Kumar, A, Natalia, F, Sudirman, S and Al-Jumeily, D (2025) Technical Document Query System using Transformer Model-based Machine Reading Comprehension. In: 2024 17th International Conference on Development in eSystem Engineering (DeSE) . pp. 497-502. (2024 17th International Conference on Development in eSystem Engineering (DeSE), 6th - 8th Nov 2024, Khorfakkan, United Arab Emirates).
|
Text
Amit Kumar Final.pdf - Accepted Version Download (988kB) | Preview |
Abstract
Constructing a Question Answering system is a challenging task despite a significant amount of study that has been conducted in recent times on this topic. It is even more difficult to provide satisfactory responses to the inquiries raised by users in an organizational setting as opposed to in an informal setting. We present in this paper, the results of our study into the use of a transformer-based model in the development of a technical document query system with machine reading comprehension. Our method fine-tunes a pre-trained transformer model with hyperparameter optimization using a pre-processed training dataset and tested on a different dataset. We experimented using eight pre-trained models from seven different variations of the BERT transformer architecture including BERT, RoBERTa, XLM-RoBERTa, ELECTRA, ALBERT, MobileBERT, and MPNet using the SQuAD1.1 dataset for fine-tuning and the Oracle Knowledge Documentation for testing. We found that the ALBERT pre-trained model is the best model achieving 0.891, 0.950, and 0.882 performance when measured using the Exact Match, F1 score, and Confidence Score metrics - despite its relatively small model size.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Additional Information: | © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Computer Science and Mathematics |
| Publisher: | IEEE |
| SWORD Depositor: | A Symplectic |
| Date Deposited: | 13 Mar 2025 13:41 |
| Last Modified: | 13 Mar 2025 13:41 |
| DOI or ID number: | 10.1109/dese63988.2024.10911893 |
| URI: | https://researchonline.ljmu.ac.uk/id/eprint/25871 |
 |
View Item |