Academia.eduAcademia.edu

Outline

Multiagent Metareasoning through Organizational Design

Proceedings of the AAAI Conference on Artificial Intelligence

https://doi.org/10.1609/AAAI.V28I1.8892

Abstract

We formulate an approach to multiagent metareasoning that uses organizational design to focus each agent's reasoning on the aspects of its local problem that let it make the most worthwhile contributions to joint behavior. By employing the decentralized Markov decision process framework, we characterize an organizational design problem that explicitly considers the quantitative impact that a design has on both the quality of the agents' behaviors and their reasoning costs. We describe an automated organizational design process that can approximately solve our organizational design problem via incremental search, and present techniques that efficiently estimate the incremental impact of a candidate organizational influence. Our empirical evaluation confirms that our process generates organizational designs that impart a desired metareasoning regime upon the agents.

References (23)

  1. Agogino, A. K., and Tumer, K. 2005. Multi-agent reward analysis for learning in noisy domains. In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, 81-88.
  2. Alexander, G.; Raja, A.; Durfee, E. H.; and Musliner, D. J. 2007. Design paradigms for meta-control in multi-agent sys- tems. In Proceedings of AAMAS 2007 Workshop on Metarea- soning in Agent-based Systems, 92-103.
  3. Becker, R.; Zilberstein, S.; Lesser, V.; and Goldman, C. V. 2004. Solving transition independent decentralized Markov decision processes. Journal of Artificial Intelligence Research 22(1):423-455.
  4. Bratman, J.; Singh, S.; Sorg, J.; and Lewis, R. 2012. Strong mitigation: Nesting search for good policies within search for good reward. In Proceedings of the Eleventh International Conference on Autonomous Agents and Multiagent Systems, 407-414.
  5. Cox, M. T., and Raja, A. 2011. Metareasoning: Thinking About Thinking. MIT Press.
  6. Dignum, V., and Padget, J. 2012. Multiagent organizations. In Weiss, G., ed., Multiagent Systems. MIT Press.
  7. Dignum, V.; Vázquez-Salceda, J.; and Dignum, F. 2005. Omni: Introducing social structure, norms and ontologies into agent organizations. In Programming Multi-Agent Systems. Springer. 181-198.
  8. Durfee, E. H., and Zilberstein, S. 2012. Multiagent planning, control, and execution. In Weiss, G., ed., Multiagent Systems. MIT Press.
  9. Hansen, E. A., and Zilberstein, S. 2001a. LAO*: A heuristic search algorithm that finds solutions with loops. Artificial Intelligence 129(1):35-62.
  10. Hansen, E. A., and Zilberstein, S. 2001b. Monitoring and control of anytime algorithms: A dynamic programming ap- proach. Artificial Intelligence 126(1):139-157.
  11. Horling, B., and Lesser, V. 2008. Using quantitative models to search for appropriate organizational designs. Autonomous Agents and Multiagent Systems 16(2):95-149.
  12. IBM. 2012. IBM ILOG CPLEX. See http://www-01.ibm. com/software/integration/optimization/cplex-optimizer/.
  13. Kallenberg, L. C. M. 1983. Linear Programming and Finite Markovian Control. Mathematical Centre Tracts.
  14. Littman, M.; Dean, T.; and Kaelbling, L. 1995. On the com- plexity of solving Markov decision problems. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intel- ligence, 394-402.
  15. Oliehoek, F. A.; Spaan, M. T. J.; Amato, C.; and Whiteson, S. 2013. Incremental clustering and expansion for faster optimal planning in decentralized POMDPs. Journal of Artificial Intelligence Research 46:449-509.
  16. Oliehoek, F. A.; Whiteson, S.; and Spaan, M. T. J. 2013. Approximate solutions for factored Dec-POMDPs with many agents. In Proceedings of the Twelfth International Confer- ence on Autonomous Agents and Multiagent Systems, 563- 570. Pacheco, O., and Carmo, J. 2003. A role based model for the normative specification of organized collective agency and agents interaction. Autonomous Agents and Multi-Agent Systems 6(2):145-184.
  17. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc.
  18. Raja, A., and Lesser, V. 2007. A framework for meta-level control in multi-agent systems. Autonomous Agents and Multi-Agent Systems 15(2):147-196.
  19. Shoham, Y., and Tennenholtz, M. 1995. On social laws for ar- tificial agent societies: Off-line design. Artificial Intelligence 73(1-2):231-252.
  20. Sleight, J., and Durfee, E. H. 2013. Organizational design principles and techniques for decision-theoretic agents. In Proceedings of the Twelfth International Conference on Au- tonomous Agents and Multiagent Systems, 463-470.
  21. Velagapudi, P.; Varakantham, P.; Sycara, K.; and Scerri, P. 2011. Distributed model shaping for scaling to decentralized POMDPs with hundreds of agents. In Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems, 955-962.
  22. Witwicki, S. J., and Durfee, E. H. 2010. Influence-based policy abstraction for weakly-coupled Dec-POMDPs. In Proceedings of the Twentieth International Conference on Automated Planning and Scheduling, 185-192.
  23. Zhang, C., and Lesser, V. 2013. Coordinating multi-agent reinforcement learning with limited communication. In Pro- ceedings of the Twelfth International Conference on Au- tonomous Agents and Multiagent Systems, 1101-1108.