Decision making in uncertain real-world domains using DT-golog

In this paper we propose an extension of the action language GOLOG that integrates linguistic terms in non-deterministic argument choices and the reward function for decision-theoretic planning. It is often cumbersome to specify the set of values to pick from in the non-deterministic-choice-of-argument statement. Also, specifying a reward function is not always easy, even for domain experts. Instead of providing a finite domain for values in the non-deterministic-choice-of-argument statement in GOLOG, we now allow for stating the argument domain by simply providing a formula over linguistic terms and fuzzy fluents. In GOLOG's forward-search DT planning algorithm, these formulas are evaluated in order to find the agent's optimal policy. We illustrate this in the Diner Domain where the agent needs to calculate the optimal serving order .

Lecture Notes in Computer Science

Readylog is a logic-based agent programming language and combines many important features from other Golog dialects. One of the features of Readylog is to make use of decision-theoretic planning for specifying the behavior of an agent or robot. In this paper we show a method to reduce the planning time for decision-theoretic planning in the Readylog framework. Instead of planning policies on the fly over and over again, we calculate an abstract policy once and store it in a plan library. This policy can later be re-instantiated. With this plan library the on-line planning time can be significantly reduced. We compare computing policies on the fly with those stored in our plan library with examples from the robotic soccer domain. In the 2D soccer simulation league we show the significant speed-up when using our plan library approach. Moreover, the use of the plan library together with a suitable state space abstraction for the soccer domain makes it possible to apply macro-actions in an otherwise continuous domain.

International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems

In this paper we present an extension of the action language Golog that allows for using fuzzy notions in non-deterministic argument choices and the reward function in decision-theoretic planning. Often, in decision-theoretic planning, it is cumbersome to specify the set of values to pick from in the non-deterministic-choice-of-argument statement. Also, even for domain experts, it is not always easy to specify a reward function. Instead of providing a finite domain for values in the non-deterministic-choice-of-argument statement in Golog, we now allow for stating the argument domain by simply providing a formula over linguistic terms and fuzzy uents. In Golog’s forward-search DT planning algorithm, these formulas are evaluated in order to find the agent’s optimal policy. We illustrate this in the Diner Domain where the agent needs to calculate the optimal serving order.

2006

homepage.cs.latrobe.edu.au

J. Mach. Learn. Res., 2017

POMDPs.jl is an open-source framework for solving Markov decision processes (MDPs) and partially observable MDPs (POMDPs). POMDPs.jl allows users to specify sequential decision making problems with minimal effort without sacrificing the expressive nature of POMDPs, making this framework viable for both educational and research purposes. It is written in the Julia language to allow flexible prototyping and large-scale computation that leverages the high-performance nature of the language. The associated JuliaPOMDP community also provides a number of state-of-the-art MDP and POMDP solvers and a rich library of support tools to help with implementing new solvers and evaluating the solution results. The most recent version of POMDPs.jl, the related packages, and documentation can be found at https://github.com/JuliaPOMDP/POMDPs.jl.

2008

We present a novel approach to adaptive multi-agent programming, which is based on an integration of the agent programming language GTGolog with adaptive dynamic programming techniques. GTGolog combines explicit agent programming in Golog with multi-agent planning in stochastic games. A drawback of this framework, however, is that the transition probabilities and immediate rewards of the domain must be known in advance and then cannot change anymore. But such data is often not available in advance and may also change over time. The adaptive generalization of GTGolog in this paper is directed towards letting the agents themselves explore and adapt these data, which is more useful for realistic applications. We present an algorithm for learning policies and show that it converges and produces optimal policies. This multi-agent learning algorithm includes as a special case a single-agent learning algorithm for DTGolog. We use highlevel programs for generating both abstract states and o...

Proceeding of the …, 2008

Résumé : Time is a crucial variable in planning and often requires special at-tention since it introduces a specific structure along with additional complexity, especially in the case of decision under uncertainty. In this paper, after reviewing and comparing MDP frameworks ...

2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, 2009

Markov decision processes (MDPs) have become a standard method for planning under uncertainty, however they usually assume a sequential process, so a single action is executed at each time step. In some applications, as in robotics, it is required to execute several actions concurrently. For this we propose a framework based on a functional decomposition of the problem into several sub-problems, each represented as a subM DP . Each subM DP is solved independently and their policies are combined to obtain a global solution, such that the actions of each subM DP can be executed concurrently. As we combine the local policies, conflicts between them can arise. We define two kinds of conflicts, resource and behavior conflicts, and propose solutions for both. Resource conflicts are solved off-line via a two-phase process which guarantees a near-optimal global policy. Behavior conflicts are solved on-line based on a set of restrictions specified by the user. If there are no restrictions, all the actions are executed concurrently; otherwise, an arbiter selects the action(s) with higher expected utility. We present experimental results in two cases: (i) a simulated robot navigation problem, with resource conflicts, and (ii) a simulated robot in a message delivery task, with behavior conflicts.

1997

We are working on a logic to combine the advantages of first-order logic, but using Bayesian decision theory (or more generally game theory) as a basis for handing uncertainty. This forms a logic for multiple agents under uncertainty. These agents act asynchronously, can have their own goals, have noisy sensors, and imperfect effectors. Recently we have developed the independent choice logic that incorporates all of these features. In this paper we discuss two different representations of time within this framework: the situation calculus and what is essentially the event calculus. We show how they both can be used, and compare the different ontological commitments made by each. Uncertainty is handled in terms of a logic which allows for independent choices and a logic program that gives the consequences of the choices. There are probabilities over the choices by nature. As part of the ¡ This work was supported by Institute for Robotics and Intelligent Systems, Project IC-7 and Natural Sciences and Engineering Research Council of Canada Operating Grant OGPOO44121. consequences are a specification of the utility of (final) states. In the situation calculus, agents adopt programs and programs lead to situations in possible worlds (which correspond to the possible outcomes of complete histories); given a probability distribution over possible worlds, we can get the expected utility of a program. In the event calculus view, actions are propositions and agents adopt policies which are logic programs to imply what the agent will do based on what it observes. Again the expected value of a policy can be computed. The aim is to choose the plan or policy that maximizes the expected utility. This paper overviews both approaches, and explains why I think the event calculus is the most promising approach. 2 The Independent Choice Logic An independent choice space theory is make of two principal components: a choice space that specifies what choice can be made, and a set of facts that specifies what follows from the choices.