Hierarchical reduced-space drift detection framework for multivariate supervised data streams

Shuyi Zhang, Peter Tino, Xin Yao

Research output: Contribution to journalArticlepeer-review

302 Downloads (Pure)

Abstract

In a streaming environment, the characteristics of the data themselves and their relationship with the labels are likely to experience changes as time goes on. Most drift detection methods for supervised data streams are performance-based, that is, they detect changes only after the classication accuracy deteriorates. This may not be sufcient in many application areas where the reason behind a drift is also important. Another category of drift detectors are data distribution-based detectors. Although they can detect some drifts within the input space, changes affecting only the labelling mechanism cannot be identied. Furthermore, little work is available on drift detection for high-dimensional supervised data streams. In this paper we propose an advanced Hierarchical Reduced-space Drift Detection Framework for Supervised Data Streams (HRDS) which captures drifts regardless of their effects on classication performance. This framework suggests monitoring both marginal and class-conditional distributions within a lower-dimensional space specically relevant to the assigned classication task. Experimental comparisons have demonstrated that the proposed HRDS not only achieves high-quality performance on high-dimensional data streams, but also outperforms its competitors in terms of detection recall, precision and F-measure across a wide range of different concept drift types including subtle drifts.
Original languageEnglish
Pages (from-to)2628-2640
Number of pages13
JournalIEEE Transactions on Knowledge and Data Engineering
Volume35
Issue number3
Early online date16 Sept 2022
DOIs
Publication statusPublished - 1 Mar 2023

Keywords

  • Concept drift
  • Delays
  • Detectors
  • Feature extraction
  • Licenses
  • Monitoring
  • Task analysis
  • Training
  • data stream mining
  • drift detection
  • online learning

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Hierarchical reduced-space drift detection framework for multivariate supervised data streams'. Together they form a unique fingerprint.

Cite this