TY - GEN
T1 - AUC estimation and concept drift detection for imbalanced data streams with multiple classes
AU - Wang, Shuo
AU - Minku, Leandro
PY - 2020/9/28
Y1 - 2020/9/28
N2 - Online class imbalance learning deals with data streams having very skewed class distributions. When learning from data streams, concept drift is one of the major challenges that deteriorate the classification performance. Although several approaches have been recently proposed to overcome concept drift in imbalanced data, they are all limited to two-class cases. Multi-class imbalance imposes additional challenges in concept drift detection and performance evaluation, such as a more severe imbalanced distribution and the limited choice of performance measures. This paper extends AUC for evaluating classifiers on multi-class imbalanced data in online learning scenarios. The proposed metrics, PMAUC, WAUC and EWAUC, are studied through comprehensive experiments, focusing on their characteristics on time-changing data streams and whether and how they can be used to detect concept drift. The AUC-based metrics show effectiveness in detecting concept drift in a variety of artificial data streams and a real-world data application with multiple classes. In particular, EWAUC is shown to be both effective and efficient.
AB - Online class imbalance learning deals with data streams having very skewed class distributions. When learning from data streams, concept drift is one of the major challenges that deteriorate the classification performance. Although several approaches have been recently proposed to overcome concept drift in imbalanced data, they are all limited to two-class cases. Multi-class imbalance imposes additional challenges in concept drift detection and performance evaluation, such as a more severe imbalanced distribution and the limited choice of performance measures. This paper extends AUC for evaluating classifiers on multi-class imbalanced data in online learning scenarios. The proposed metrics, PMAUC, WAUC and EWAUC, are studied through comprehensive experiments, focusing on their characteristics on time-changing data streams and whether and how they can be used to detect concept drift. The AUC-based metrics show effectiveness in detecting concept drift in a variety of artificial data streams and a real-world data application with multiple classes. In particular, EWAUC is shown to be both effective and efficient.
KW - Class imbalance learning
KW - Concept drift detection
KW - Online learning
UR - http://www.scopus.com/inward/record.url?scp=85093857588&partnerID=8YFLogxK
U2 - 10.1109/IJCNN48605.2020.9207377
DO - 10.1109/IJCNN48605.2020.9207377
M3 - Conference contribution
SN - 978-1-7281-6927-9 (PoD)
T3 - Proceedings of International Joint Conference on Neural Networks
BT - Proceedings of the International Joint Conference on Neural Networks (IJCNN), World Congress on Computational Intelligence, 2020
PB - IEEE Computer Society Press
T2 - IEEE International Joint Conference on Neural Networks (IJCNN), 2020
Y2 - 19 July 2020 through 24 July 2020
ER -