A nearest-neighbor-based ensemble classifier and its large-sample optimality

Research output: Contribution to journalArticlepeer-review


Colleges, School and Institutes

External organisations

  • Department of Mathematics, California State University, Northridge


A nonparametric approach is proposed to combine several individual classifiers in order to construct an asymptotically more accurate classification rule in the sense that its misclassification error rate is, asymptotically, at least as low as that of the best individual classifier. The proposed method uses a nearest neighbour type approach to estimate the conditional expectation of the class associated with a new observation (conditional on the vector of individual predictions). Both mechanics and the theoretical validity of the proposed approach are discussed. As an interesting by product of our results, it is shown that the proposed method can also be applied to any single classifier in which case the resulting new classifier will be at least as good as the original one. Several numerical examples, involving both real and simulated data, are also given. These numerical studies further confirm the superiority of the proposed classifier.

Bibliographic note

Funding Information: This work was supported by the NSF under Grant DMS-1916161 of Majid Mojirsheibani. Publisher Copyright: © 2021 Informa UK Limited, trading as Taylor & Francis Group.


Original languageEnglish
Pages (from-to)1-17
Number of pages17
JournalJournal of Statistical Computation and Simulation
Early online date10 Feb 2021
Publication statusE-pub ahead of print - 10 Feb 2021


  • Nonparametric, asymptotics, classification