Online estimation of discrete, continuous, and conditional joint densities using classifier chains

Michael Geilke, Andreas Karwath, Eibe Frank, Stefan Kramer

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


We address the problem of estimating discrete, continuous, and conditional joint densities online, i.e., the algorithm is only provided the current example and its current estimate for its update. The family of proposed online density estimators, estimation of densities online (EDO), uses classifier chains to model dependencies among features, where each classifier in the chain estimates the probability of one particular feature. Because a single chain may not provide a reliable estimate, we also consider ensembles of classifier chains and ensembles of weighted classifier chains. For all density estimators, we provide consistency proofs and propose algorithms to perform certain inference tasks. The empirical evaluation of the estimators is conducted in several experiments and on datasets of up to several millions of instances. In the discrete case, we compare our estimators to density estimates computed by Bayesian structure learners. In the continuous case, we compare them to a state-of-the-art online density estimator. Our experiments demonstrate that, even though designed to work online, EDO delivers estimators of competitive accuracy compared to other density estimators (batch Bayesian structure learners on discrete datasets and the state-of-the-art online density estimator on continuous datasets). Besides achieving similar performance in these cases, EDO is also able to estimate densities with mixed types of variables, i.e., discrete and continuous random variables.
Original languageEnglish
Pages (from-to)561-603
JournalData Mining and Knowledge Discovery
Issue number3
Early online date25 Nov 2017
Publication statusE-pub ahead of print - 25 Nov 2017


  • density estimation
  • machine learning
  • stream mining


Dive into the research topics of 'Online estimation of discrete, continuous, and conditional joint densities using classifier chains'. Together they form a unique fingerprint.

Cite this