United We Stand: Population Based Methods for Solving Unknown POMDPs

NH Welsh; Jeremy Wyatt; S Girgin; M Loth

doi:10.1007/978-3-540-89722-4_19

United We Stand: Population Based Methods for Solving Unknown POMDPs

NH Welsh, Jeremy Wyatt, S Girgin, M Loth

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

Solving large unknown POMDPs is an open research problem. Policy search is one solution method that is attractive as it scales in the size of the policy, which is typically much simpler than the environment. We present a global search algorithm capable of finding good policies for POMDPs that are substantially larger than previously reported results. Our algorithm is general; we show it can be used with, and improves the performance of, existing local search techniques such as gradient ascent. Sharing information between the members of the population is the key to our algorithm and we show it results in better performance than equivalent parallel searches that do not share information. Unlike previous work our algorithm does not require the size of the policy to be known in advance.

Original language	English
Title of host publication	Recent Advances in Reinforcement Learning
Volume	LNAI 5323
ISBN (Electronic)	9783540897224
DOIs	https://doi.org/10.1007/978-3-540-89722-4_19
Publication status	Published - 27 Nov 2008

Access to Document

10.1007/978-3-540-89722-4_19

Cite this

@inbook{d063298d99fe4f648f8aa598136f9a03,

title = "United We Stand: Population Based Methods for Solving Unknown POMDPs",

abstract = "Solving large unknown POMDPs is an open research problem. Policy search is one solution method that is attractive as it scales in the size of the policy, which is typically much simpler than the environment. We present a global search algorithm capable of finding good policies for POMDPs that are substantially larger than previously reported results. Our algorithm is general; we show it can be used with, and improves the performance of, existing local search techniques such as gradient ascent. Sharing information between the members of the population is the key to our algorithm and we show it results in better performance than equivalent parallel searches that do not share information. Unlike previous work our algorithm does not require the size of the policy to be known in advance.",

author = "NH Welsh and Jeremy Wyatt and S Girgin and M Loth",

year = "2008",

month = nov,

day = "27",

doi = "10.1007/978-3-540-89722-4_19",

language = "English",

isbn = "9783540897217",

volume = "LNAI 5323",

booktitle = "Recent Advances in Reinforcement Learning",

}

TY - CHAP

T1 - United We Stand: Population Based Methods for Solving Unknown POMDPs

AU - Welsh, NH

AU - Wyatt, Jeremy

AU - Girgin, S

AU - Loth, M

PY - 2008/11/27

Y1 - 2008/11/27

N2 - Solving large unknown POMDPs is an open research problem. Policy search is one solution method that is attractive as it scales in the size of the policy, which is typically much simpler than the environment. We present a global search algorithm capable of finding good policies for POMDPs that are substantially larger than previously reported results. Our algorithm is general; we show it can be used with, and improves the performance of, existing local search techniques such as gradient ascent. Sharing information between the members of the population is the key to our algorithm and we show it results in better performance than equivalent parallel searches that do not share information. Unlike previous work our algorithm does not require the size of the policy to be known in advance.

AB - Solving large unknown POMDPs is an open research problem. Policy search is one solution method that is attractive as it scales in the size of the policy, which is typically much simpler than the environment. We present a global search algorithm capable of finding good policies for POMDPs that are substantially larger than previously reported results. Our algorithm is general; we show it can be used with, and improves the performance of, existing local search techniques such as gradient ascent. Sharing information between the members of the population is the key to our algorithm and we show it results in better performance than equivalent parallel searches that do not share information. Unlike previous work our algorithm does not require the size of the policy to be known in advance.

U2 - 10.1007/978-3-540-89722-4_19

DO - 10.1007/978-3-540-89722-4_19

M3 - Chapter

SN - 9783540897217

VL - LNAI 5323

BT - Recent Advances in Reinforcement Learning

ER -