Export Citation
Export Citation
Iram, S, Al-Jumeily, D, Fergus, P and Hussain, A (2014) Exploring the Hidden Challenges Associated with the Evaluation of Multi-class Datasets using Multiple Classifiers. In: 2014 EIGHTH INTERNATIONAL CONFERENCE ON COMPLEX, INTELLIGENT AND SOFTWARE INTENSIVE SYSTEMS (CISIS), . (8th International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), 02 July 2014 - 04 July 2014, Birmingham City Univ, Birmingham, UNITED KINGDOM).
Preview |
Text
CISIS-Final.pdf - Accepted Version Download (186kB) | Preview |
Abstract
The optimization and evaluation of a pattern recognition system requires different problems like multi-class and imbalanced datasets be addressed. This paper presents the classification of multi-class datasets which present more challenges when compare to binary class datasets in machine learning. Furthermore, it argues that the performance evaluation of a classification model for multi-class imbalanced datasets in terms of simple “accuracy rate” can possibly provide misleading results. Other parameters such as failure avoidance, true identification of positive and negative instances of a class and class discrimination are also very important. We, in this paper, hypothesize that “misclassification of true positive patterns should not necessarily be categorized as false negative while evaluating a classifier for multi-class datasets”, a common practice that has been observed in the existing literature. In order to address these hidden challenges for the generalization of a particular classifier, several evaluation metrics are compared for a multi-class dataset with four classes; three of them belong to different neurodegenerative diseases and one to control subjects. Three classifiers, linear discriminant, quadratic discriminant and Parzen are selected to demonstrate the results with examples.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Uncontrolled Keywords: | Science & Technology; Technology; Computer Science, Artificial Intelligence; Computer Science, Software Engineering; Computer Science, Theory & Methods; Computer Science; Classifier evaluation; multi-class dataset; pattern recognition; neurodegenerative diseases; multiple classifiers; CLASSIFICATION; ROC |
| Subjects: | Q Science > QA Mathematics > QA76 Computer software |
| Divisions: | Computer Science and Mathematics |
| Publisher: | IEEE |
| Related URLs: | |
| Date of acceptance: | 4 July 2014 |
| Date of first compliant Open Access: | 27 November 2015 |
| Date Deposited: | 27 Nov 2015 09:26 |
| Last Modified: | 13 Apr 2022 15:14 |
| DOI or ID number: | 10.1109/CISIS.2014.48 |
| URI: | https://researchonline.ljmu.ac.uk/id/eprint/2397 |
 |
View Item |