Heterogeneous sets in dimensionality reduction and ensemble learning

Henry Reeve, Ata Kaban*, Jakramate Bootkrajang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

33 Downloads (Pure)

Abstract

We present a general framework for dealing with set heterogeneity in data and learning problems, which is able to exploit low complexity components.
The main ingredients are (i) a definition of complexity for elements of a convex union that takes into account the complexities of their individual composition -- this is used to cover the heterogeneous convex union, and (ii) upper bounds on the complexities of restricted subsets. We demonstrate this approach in two different application areas, highlighting their conceptual connection. (1) In random projection based dimensionality reduction, we obtain improved bounds on the uniform preservation of Euclidean norms and distances when low complexity components are present in the union. (2) In statistical learning, our generalisation bounds justify heterogeneous ensemble learning methods that were incompletely understood before. We exemplify empirical results with boosting type random subspace and random projection ensembles that implement our bounds.
Original languageEnglish
Number of pages22
JournalMachine Learning
Early online date28 Oct 2022
DOIs
Publication statusE-pub ahead of print - 28 Oct 2022

Keywords

  • Heterogeneous ensembles
  • Random projection
  • Suprema of empirical processes

Fingerprint

Dive into the research topics of 'Heterogeneous sets in dimensionality reduction and ensemble learning'. Together they form a unique fingerprint.

Cite this