Performance assessment of a system for reasoning under uncertainty

Branko Ristic, Christopher Gilliam, Marion Byrne

Research output: Contribution to journalLetterpeer-review

Abstract

From the early developments of machines for reasoning and decision making in higher-level information fusion, there was a need for a systematic and reliable evaluation of their performance. Performance evaluation is important for comparison and assessment of alternative solutions to real-world problems. In this paper we focus on one aspect of performance assessment for reasoning under uncertainty: the accuracy of the resulting belief (prediction or estimate). We propose a framework for assessment based on the assumption that the system under investigation is uncertain only due to stochastic variability (randomness), which is partially known. In this context we formulate a distance measure between the “ground truth” and the output of an automated system for reasoning in the framework of one of the non-additive uncertainty formalisms (such as imprecise probability theory, belief function theory or possibility theory). The proposed assessment framework is demonstrated with a simple numerical example.

Original languageEnglish
Pages (from-to)11-16
Number of pages6
JournalInformation Fusion
Volume71
Early online date15 Jan 2021
DOIs
Publication statusPublished - Jul 2021

Bibliographical note

Funding Information:
This research was supported by DST Group, Australia under the Research Agreement “Classification decisions using uncertain knowledge-base” (2019–2020).

Publisher Copyright:
© 2021 Elsevier B.V.

Keywords

  • Automated reasoning under uncertainty
  • Belief function theory
  • Imprecise probability theory
  • Performance evaluation
  • Possibility theory

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Performance assessment of a system for reasoning under uncertainty'. Together they form a unique fingerprint.

Cite this