A nearest-neighbor-based ensemble classifier and its large-sample optimality

Majid Mojirsheibani, William Pouliot

Research output: Contribution to journalArticlepeer-review

13 Downloads (Pure)


A nonparametric approach is proposed to combine several individual classifiers in order to construct an asymptotically more accurate classification rule in the sense that its misclassification error rate is, asymptotically, at least as low as that of the best individual classifier. The proposed method uses a nearest neighbour type approach to estimate the conditional expectation of the class associated with a new observation (conditional on the vector of individual predictions). Both mechanics and the theoretical validity of the proposed approach are discussed. As an interesting by product of our results, it is shown that the proposed method can also be applied to any single classifier in which case the resulting new classifier will be at least as good as the original one. Several numerical examples, involving both real and simulated data, are also given. These numerical studies further confirm the superiority of the proposed classifier.
Original languageEnglish
Pages (from-to)1-17
Number of pages17
JournalJournal of Statistical Computation and Simulation
Issue number10
Early online date10 Feb 2021
Publication statusE-pub ahead of print - 10 Feb 2021

Bibliographical note

Funding Information:
This work was supported by the NSF under Grant DMS-1916161 of Majid Mojirsheibani.

Publisher Copyright:
© 2021 Informa UK Limited, trading as Taylor & Francis Group.


  • Nonparametric
  • asymptotics
  • classification

ASJC Scopus subject areas

  • Statistics and Probability
  • Modelling and Simulation
  • Statistics, Probability and Uncertainty
  • Applied Mathematics


Dive into the research topics of 'A nearest-neighbor-based ensemble classifier and its large-sample optimality'. Together they form a unique fingerprint.

Cite this