Iram, S and Al-Jumeily, D and Fergus, P and Hussain, A (2014) Exploring the Hidden Challenges Associated with the Evaluation of Multi-class Datasets using Multiple Classifiers. In: 2014 EIGHTH INTERNATIONAL CONFERENCE ON COMPLEX, INTELLIGENT AND SOFTWARE INTENSIVE SYSTEMS (CISIS), . (8th International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), 02 July 2014 - 04 July 2014, Birmingham City Univ, Birmingham, UNITED KINGDOM).
CISIS-Final.pdf - Accepted Version
The optimization and evaluation of a pattern recognition system requires different problems like multi-class and imbalanced datasets be addressed. This paper presents the classification of multi-class datasets which present more challenges when compare to binary class datasets in machine learning. Furthermore, it argues that the performance evaluation of a classification model for multi-class imbalanced datasets in terms of simple “accuracy rate” can possibly provide misleading results. Other parameters such as failure avoidance, true identification of positive and negative instances of a class and class discrimination are also very important. We, in this paper, hypothesize that “misclassification of true positive patterns should not necessarily be categorized as false negative while evaluating a classifier for multi-class datasets”, a common practice that has been observed in the existing literature. In order to address these hidden challenges for the generalization of a particular classifier, several evaluation metrics are compared for a multi-class dataset with four classes; three of them belong to different neurodegenerative diseases and one to control subjects. Three classifiers, linear discriminant, quadratic discriminant and Parzen are selected to demonstrate the results with examples.
|Item Type:||Conference or Workshop Item (Paper)|
|Uncontrolled Keywords:||Science & Technology; Technology; Computer Science, Artificial Intelligence; Computer Science, Software Engineering; Computer Science, Theory & Methods; Computer Science; Classifier evaluation; multi-class dataset; pattern recognition; neurodegenerative diseases; multiple classifiers; CLASSIFICATION; ROC|
|Subjects:||Q Science > QA Mathematics > QA76 Computer software|
|Date Deposited:||27 Nov 2015 09:26|
|Last Modified:||27 Nov 2015 09:26|
|DOI or Identification number:||10.1109/CISIS.2014.48|
Actions (login required)