Auto-assessment of assessment: A human-in-the-loop AI framework addressing policy gaps in academic assessment

Khan, W ORCID: 0000-0002-7511-3873, Topham, L ORCID: 0000-0002-6689-7944, Jones, N, Atherton, P ORCID: 0000-0003-3258-0436, Al-Shabandar, R, Kolivand, H ORCID: 0000-0001-5460-5679, Khan, I ORCID: 0000-0002-4206-7663, Alatrany, A and Hussain, A (2026) Auto-assessment of assessment: A human-in-the-loop AI framework addressing policy gaps in academic assessment. PLoS One, 21 (4). ISSN 1932-6203

Preview

Text journal.pone.0346815.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Education is being transformed by rapid advances in Artificial Intelligence (AI), including emerging Generative Artificial Intelligence (GAI). Such technology can significantly support academics and students by automating monotonous tasks and making personalised suggestions. However, despite the potential of the technology, there are significant concerns regarding AI misuse, particularly by students in assessments. There are two schools of thought: one advocates for a complete ban on it, while the other views it as a valuable educational tool, provided it is governed by a robust usage policy. This contradiction clearly indicates a major policy gap in academic practices, and new policies are required to uphold academic standards while enabling staff and students to benefit from technological advancements. We surveyed 117 academics from three countries (the UK, the UAE, and Iraq) and identified that most academics retain positive opinions regarding AI in education. For example, the majority of experienced academics do not favour complete bans, and they see potential benefits of AI for students, teaching staff, and academic institutions. Importantly, academics specifically identified the particular benefits of AI for autonomous assessment (71.79% of respondents agreed). Therefore, for the first time, we introduce a novel AI framework for evaluating students' work (e.g., reports, coursework, etc.) within a human-in-the-loop assessment process, in which automated grade suggestions are generated and subsequently reviewed by qualified instructors based on students' knowledge and in-depth understanding of the submitted content. The survey results and evaluation outcomes further highlight a significant lack of awareness of modern AI-based tools among experienced academics, a gap that must be addressed to uphold educational standards.

Item Type:	Article
Uncontrolled Keywords:	Humans; Artificial Intelligence; Educational Measurement; Surveys and Questionnaires; Students; United Kingdom; General Science & Technology
Subjects:	L Education > LB Theory and practice of education L Education > LB Theory and practice of education > LB2300 Higher Education Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Divisions:	Computer Science and Mathematics Education Pharmacy and Biomolecular Sciences
Publisher:	Public Library of Science (PLoS)
Date of acceptance:	23 March 2026
Date of first compliant Open Access:	20 April 2026
Date Deposited:	20 Apr 2026 13:24
Last Modified:	20 Apr 2026 13:24
DOI or ID number:	10.1371/journal.pone.0346815
Editors:	Fernandes, TP
URI:	https://researchonline.ljmu.ac.uk/id/eprint/28407

View Item