Discovering Domain Axioms Using Relational Reinforcement Learning and Declarative Programming

This paper presents an architecture that integrates declarative programming and relational reinforcement learning to support incremental and interactive discovery of previously unknown axioms governing domain dynamics. Specifically, Answer Set Prolog (ASP), a declarative programming paradigm, is used to represent and reason with incomplete commonsense domain knowledge. For any given goal, any unexplained failure of plans created by inference in the ASP program is taken to indicate the existence of unknown domain axioms. The task of discovering these axioms is formulated as a Reinforcement Learning problem, and decisiontree regression with a relational representation is used to incrementally generalize from specific axioms identified over time. These new axioms are added to the ASP program for subsequent inference. We demonstrate and evaluate the capabilities of our architecture in two simulated domains: Blocks World and Simple Mario.