Branching time active inference: the theory and its generality

Théophile Champion; Lancelot Da Costa; Howard Bowman; Marek Grześ

doi:10.1016/j.neunet.2022.03.036

Branching time active inference: the theory and its generality

Théophile Champion, Lancelot Da Costa, Howard Bowman, Marek Grześ

Psychology

Research output: Contribution to journal › Article › peer-review

130 Downloads (Pure)

Abstract

Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.

Original language	English
Pages (from-to)	295-316
Number of pages	22
Journal	Neural Networks
Volume	151
Early online date	6 Apr 2022
DOIs	https://doi.org/10.1016/j.neunet.2022.03.036
Publication status	Published - Jul 2022

Bibliographical note

Funding Information:
We would like to thank the reviewers for their valuable feedback, which greatly improved the quality of the present paper. LD is supported by the Fonds National de la Recherche, Luxembourg (Project code: 13568875). This publication is based on work partially supported by the EPSRC, United Kingdom Centre for Doctoral Training in Mathematics of Random Systems: Analysis, Modelling and Simulation (EP/S023925/1).

Publisher Copyright:
© 2022 The Author(s)

Keywords

Active inference
Free energy principle
Planning
Tree search
Variational message passing

ASJC Scopus subject areas

Cognitive Neuroscience
Artificial Intelligence

Access to Document

10.1016/j.neunet.2022.03.036Licence: Creative Commons: Attribution (CC BY)

ChampionT2022BranchingFinal published version, 744 KBLicence: Creative Commons: Attribution (CC BY)

Cite this

@article{462784dcb35244328c056cba78ac6e57,

title = "Branching time active inference: the theory and its generality",

abstract = "Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.",

keywords = "Active inference, Free energy principle, Planning, Tree search, Variational message passing",

author = "Th{\'e}ophile Champion and {Da Costa}, Lancelot and Howard Bowman and Marek Grze{\'s}",

note = "Funding Information: We would like to thank the reviewers for their valuable feedback, which greatly improved the quality of the present paper. LD is supported by the Fonds National de la Recherche, Luxembourg (Project code: 13568875). This publication is based on work partially supported by the EPSRC, United Kingdom Centre for Doctoral Training in Mathematics of Random Systems: Analysis, Modelling and Simulation (EP/S023925/1). Publisher Copyright: {\textcopyright} 2022 The Author(s)",

year = "2022",

month = jul,

doi = "10.1016/j.neunet.2022.03.036",

language = "English",

volume = "151",

pages = "295--316",

journal = "Neural Networks",

issn = "0893-6080",

publisher = "Elsevier",

}

TY - JOUR

T1 - Branching time active inference

T2 - the theory and its generality

AU - Champion, Théophile

AU - Da Costa, Lancelot

AU - Bowman, Howard

AU - Grześ, Marek

N1 - Funding Information: We would like to thank the reviewers for their valuable feedback, which greatly improved the quality of the present paper. LD is supported by the Fonds National de la Recherche, Luxembourg (Project code: 13568875). This publication is based on work partially supported by the EPSRC, United Kingdom Centre for Doctoral Training in Mathematics of Random Systems: Analysis, Modelling and Simulation (EP/S023925/1). Publisher Copyright: © 2022 The Author(s)

PY - 2022/7

Y1 - 2022/7

N2 - Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.

AB - Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.

KW - Active inference

KW - Free energy principle

KW - Planning

KW - Tree search

KW - Variational message passing

UR - http://www.scopus.com/inward/record.url?scp=85129542034&partnerID=8YFLogxK

U2 - 10.1016/j.neunet.2022.03.036

DO - 10.1016/j.neunet.2022.03.036

M3 - Article

C2 - 35468491

SN - 0893-6080

VL - 151

SP - 295

EP - 316

JO - Neural Networks

JF - Neural Networks

ER -

Branching time active inference: the theory and its generality

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this