Feature relevance determination for ordinal regression in the context of feature redundancies and privileged information

Lukas Pfannschmidt, Jonathan Jacob, Fabian Hinder, Michael Biehl, Peter Tino, Barbara Hammer

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
91 Downloads (Pure)


Advances in machine learning technologies have led to increasingly powerful models in particular in the context of big data. Yet, many application scenarios demand for robustly interpretable models rather than optimum model accuracy; as an example, this is the case if potential biomarkers or causal factors should be discovered based on a set of given measurements. In this contribution, we focus on feature selection paradigms, which enable us to uncover relevant factors of a given regularity based on a sparse model. We focus on the important specific setting of linear ordinal regression, i.e. data have to be ranked into one of a finite number of ordered categories by a linear projection. Unlike previous work, we consider the case that features are potentially redundant, such that no unique minimum set of relevant features exists. We aim for an identification of all strongly and all weakly relevant features as well as their type of relevance (strong or weak); we achieve this goal by determining feature relevance bounds, which correspond to the minimum and maximum feature relevance, respectively, if searched over all equivalent models. In addition, we discuss how this setting enables us to substitute some of the features, e.g. due to their semantics, and how to extend the framework of feature relevance intervals to the setting of privileged information, i.e. potentially relevant information is available for training purposes only, but cannot be used for the prediction itself.
Original languageEnglish
Pages (from-to)266-279
Number of pages14
Early online date9 Apr 2020
Publication statusE-pub ahead of print - 9 Apr 2020


  • Feature selection
  • Global feature relevance
  • Interpretability
  • Ordinal regression
  • Privileged information


Dive into the research topics of 'Feature relevance determination for ordinal regression in the context of feature redundancies and privileged information'. Together they form a unique fingerprint.

Cite this