Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Correlated Equilibria in Stopping Games
Selection of a Correlated Rational Equilibria
Conclusions and Final Remarks
References
All Topics
Mathematics
Probability

Selection of a correlated equilibrium in Markov stopping games

Profile image of David RamseyDavid Ramsey

2008, European Journal of Operational Research

https://doi.org/10.1016/J.EJOR.2006.10.050
visibility

…

description

22 pages

link

1 file

Abstract

This paper deals with an extension of the concept of correlated strategies to Markov stopping games. The Nash equilibrium approach to solving nonzero-sum stopping games may give multiple solutions. An arbitrator can suggest to each player the decision to be applied at each stage based on a joint distribution over the players' decisions according to some optimality criterion. This is a form of equilibrium selection. Examples of correlated equilibria in nonzero-sum games related to the best choice problem are given. Several concepts of criteria for selecting a correlated equilibrium are used.

Related papers

The Generalized Stackelberg Equilibrium of the Two-Person Stopping Game
Krzysztof Szajowski

Journal of the Operations Research Society of China

In modeling the bilateral selection of states of the process, Dynkin (Dokl Akad Nauk USSR 185:241–288, 1969) proposed a two-person game in which players use stopping moments as strategies. The purpose of this work is to present a model of the game in which the players have different information about the process itself, as well as various laws to stop the process and accept its state. The game model uses the stochastic process apparatus, in particular, the ability to create different filters for the same process. The sets of stopping moments based on different filters are not identical, which allows us to model different sets of strategies for players. We show that the follower, by observing the behavior of a rational leader, can recover information that is lost due to the lack of complete observation of the state of the process. In the competition of two opponents for the maximum of the i.i.d. sequence, one of whom has access to full information and the other only knows their relat...

View PDFchevron_right
Maximum causal entropy correlated equilibria for Markov games
Brian Ziebart

… Agents and Multi-Agent Systems (AAMAS …, 2011

Motivated by a machine learning perspective-that gametheoretic equilibria constraints should serve as guidelines for predicting agents' strategies, we introduce maximum causal entropy correlated equilibria (MCECE), a novel solution concept for general-sum Markov games. In line with this perspective, a MCECE strategy profile is a uniquely-defined joint probability distribution over actions for each game state that minimizes the worst-case prediction of agents' actions under log-loss. Equivalently, it maximizes the worstcase growth rate for gambling on the sequences of agents' joint actions under uniform odds. We present a convex optimization technique for obtaining MCECE strategy profiles that resembles value iteration in finite-horizon games. We assess the predictive benefits of our approach by predicting the strategies generated by previously proposed correlated equilibria solution concepts, and compare against those previous approaches on that same prediction task.

View PDFchevron_right
Optimal stopping problems by two or more decision makers: a survey
Saoussen Krichen

Computational Management Science, 2006

A review of the optimal stopping problem with more than a single decision maker (DM) is presented in this paper. We classify the existing literature according to the arrival of the offers, the utility of the DMs, the length of the sequence of offers, the nature of the game and the number of offers to be selected. We enumerate various definitions for this problem and describe some dynamic approaches.

View PDFchevron_right
On infinite horizon multi-person stopping games with priorities
Elzbieta Ferenstein

Banach Center Publications, 2006

We study nonzero-sum multi-person multiple stopping games with players' priorities. The existence of Nash equilibrium is proved. Examples of multi stopping of Markov chains are considered. The game may also be presented as a special case of a stochastic game which leads to many variations of it, in which stopping is a part of players' strategies.

View PDFchevron_right
A model of a 2-player stopping game with priority and asynchronous observation
David Ramsey

Mathematical Methods of Operations Research, 2007

Various models of 2-player stopping games have been considered which assume that players simultaneously observe a sequence of objects. Nash equilibria for such games can be found by first solving the optimal stopping problems arising when one player remains and then defining by recursion the normal form of the game played at each stage when both players are still searching (a 2 × 2 matrix game). The model considered here assumes that Player 1 always observes an object before Player 2. If Player 1 accepts the object, then Player 2 does not see that object. If Player 1 rejects an object, then Player 2 observes it and may choose to accept or reject it. It is shown that such a game can be solved using recursion by solving appropriately defined subgames, which are played at each moment when both players are still searching. In these subgames Player 1 chooses a threshold, such that an object is accepted iff its value is above this threshold. The strategy of Player 2 in this subgame is a stopping rule to be used when Player 1 accepts this object, together with a threshold to be used when Player 1 rejects the object. Whenever the payoff of Player 1 does not depend on the value of the object taken by Player 2, such a game can be treated as two optimisation problems. Two examples are given to illustrate these approaches.

View PDFchevron_right
Nonzero-Sum Stochastic Games
Krzysztof J. Szajowski

Stochastic and Differential Games, 1999

This paper treats stochastic games. A nonzero-sum average payoff stochastic games with arbitrary state spaces and the stopping games are considered. Such models of games very well fit in some studies in economic theory and operations research. A correlation of strategies of the players, involving "public signals", is allowed in the nonzero-sum average payoff stochastic games. The main result is an extension of the correlated equilibrium theorem proved recently by Nowak and Raghavan for dynamic games with discounting to the average payoff stochastic games. The stopping games are special model of stochastic games. The version of Dynkin's game related to observation of Markov process with random priority assignment mechanism of states is presented in the paper. The zero-sum and nonzero-sum games are considered. The paper also provides a brief overview of the theory of nonzero-sum stochastic games and stopping games which are very far from being complete.

View PDFchevron_right
Voting Procedure on Stopping Games of Markov Chain
Masami Yasuda

The paper deals with a p person, non-cooperative game related to the observation of a Markov chain. The players observe the process up to a random moment defined by a monotonic logical function based on an individual players’ decision. The concept of Nash equilibrium is used. The solution of the game for finite and infinite horizon problems is derived. A simple example is presented.

View PDFchevron_right
On the value of optimal stopping games
stephane villeneuve

The Annals of Applied Probability, 2006

We show, under weaker assumptions than in the previous literature, that a perpetual optimal stopping game always has a value. We also show that there exists an optimal stopping time for the seller, but not necessarily for the buyer. Moreover, conditions are provided under which the existence of an optimal stopping time for the buyer is guaranteed. The results are illustrated explicitly in two examples.

View PDFchevron_right
Risk-sensitive Markov stopping games with an absorbing state
Rolando Cavazos-Cadena

Kybernetika, 2022

This work is concerned with discrete-time Markov stopping games with two players. At each decision time player II can stop the game paying a terminal reward to player I, or can let the system to continue its evolution. In this latter case player I applies an action affecting the transitions and entitling him to receive a running reward from player II. It is supposed that player I has a no-null and constant risk-sensitivity coefficient, and that player II tries to minimize the utility of player I. The performance of a pair of decision strategies is measured by the risk-sensitive (expected) total reward of player I and, besides mild continuity-compactness conditions, the main structural assumption on the model is the existence of an absorbing state which is accessible from any starting point. In this context, it is shown that the value function of the game is characterized by an equilibrium equation, and the existence of a Nash equilibrium is established.

View PDFchevron_right
Incentive compatibility in sender-receiver stopping games
Dries Vermeulen

2020

We introduce a model of sender-receiver stopping games, where the state of the world follows an iid--process throughout the game. At each period, the sender observes the current state, and sends a message to the receiver, suggesting either to stop or to continue. The receiver, only seeing the message but not the state, decides either to stop the game, or to continue which takes the game to the next period. The payoff to each player is a function of the state when the receiver quits, with higher states leading to better payoffs. The horizon of the game can be finite or infinite. We prove existence and uniqueness of responsive (i.e. non-babbling) Perfect Bayesian Equilibrium (PBE) under mild conditions on the game primitives in the case where the players are sufficiently patient. The responsive PBE has a remarkably simple structure, which builds on the identification of an easy-to-implement and compute class of threshold strategies for the sender. With the help of these threshold stra...

View PDFchevron_right

References (56)

  1. R.J. Aumann, Subjectivity and correlation in randomized strategies, Journal of Mathematical Economics 1 (1974) 67-96.
  2. F. Ben Abdelaziz, S. Krichen, An interactive method for the optimal selection problem with two decision makers, European Journal of Operational Research 162 (3) (2005) 602-609.
  3. D.T. Bishop, C. Cannings, A generalized war of attrition, Journal of Theoretical Biology 70 (1978) 85-124.
  4. A. Calvo ´-Armengol, The set of correlated equilibria of 2 • 2 games, Tech. report, Department of Economics, Edifici B, 08193 Bellaterra (Barcelona), Spain, 2004, http://selene.uab.es/acalvo/correlated.pdf.
  5. V. Domansky, Dynkin's games with randomized optimal stopping rules, in: K. Szajowski, A.S. Nowak (Eds.), Advances in Dynamic Games: Applications to Economics, Finance, Optimization, and Stochastic Control (Boston), Annals of the International Society of Dynamic Games, vol. 7, Birkhaser, 2005, pp. 247-262.
  6. E.G. Enns, E.Z. Ferenstein, On a multi-person time-sequential game with priorities, Sequential Analysis 6 (1987) 239- 256.
  7. Fe S. Evangelista, T.E.S. Raghavan, A note on correlated equilibrium, International Journal of Game Theory 25 (1) (1996) 35-41.
  8. E.Z. Ferenstein, Two-person non-zero-sum games with priorities, in: F. Thomas Bruss, Thomas S. Ferguson, Stephen M. Samuels (Eds.), Strategies for Sequential Search and Selection in Real Time, Proceedings of the AMS-IMS-SIAM Join Summer Research Conferences held June 21-27, 1990, University of Massachusetts at Amherst, Contemporary Mathematics, vol. 125. Oxford University Press, 1992, pp. 119-133.
  9. E.Z. Ferenstein, On randomized stopping game, in: K. Szajowski, A.S. Nowak (Eds.), Advances in Dynamic Games: Applications to Economics, Finance, Optimization, and Stochastic Control (Boston), Annals of the International Society of Dynamic Games, vol. 7, Birkha ¨ser, 2005, pp. 223-233.
  10. T.S. Ferguson, Selection by committee, in: K. Szajowski, A.S. Nowak (Eds.), Advances in Dynamic Games: Applications to Economics, Finance, Optimization, and Stochastic Control (Boston), Annals of the International Society of Dynamic Games, vol. 7, Birkha ¨ser, 2005, pp. 203-209.
  11. F. Forges, An approach to communication equilibria, Econometrica 54 (1986) 1375-1385.
  12. F. Forges, Communication equilibria in repeated games with incomplete information, Mathematics of Operations Research 13 (2) (1988) 191-231.
  13. M. Fushimi, The secretary problem in a competitive situation, Journal of the Operations Research Society of Japan 24 (1981) 350- 358.
  14. J.P. Gilbert, F. Mosteller, Recognizing the maximum of a sequence, Journal of American Statistical Association 61 (313) (1966) 35- 73.
  15. I. Gilboa, E. Zemel, Nash and correlated equilibria: Some complexity considerations, Games and Economic Behavior 1 (1) (1989) 80- 93.
  16. A. Greenwald, K. Hall, Correlated Q-learning, in: Tom Fawcett, Nina Mishra (Eds.), Proceedings of Twentieth International Conference on Machine Learning (ICML-2003), August 21-24, The AAAI Press, Washington, DC, 2003, pp. 242-249.
  17. R. Hendrickx, R. Peeters, J. Potters, A relation between Nash equilibria and correlated equilibria, International Game Theory Review 4 (4) (2002) 405-413.
  18. P. Jean-Jacques Herings, Ronald J.A.P. Peeters, Equilibrium selection in stochastic games, International Game Theory Review 5 (4) (2003) 307-326.
  19. P. Jean-Jacques Herings, Ronald J.A.P. Peeters, Stationary equilibria in stochastic games: Structure, selection, and computation, Journal of Economic Theory 118 (1) (2004) 32-60.
  20. M. Kurano, M. Yasuda, J. Nakagami, Multi-variate stopping problem with a majority rule, Journal of the Operations Research Society of Japan 23 (1980) 205-223.
  21. V.V. Mazalov, A game related to optimal stopping of two sequences of independent random variables having different distributions, Mathematica Japonica 43 (1996) 121-128.
  22. J.-F. Mertens, Correlated-and communication equilibria, in: J.-F. Mertens et al. (Eds.), Game-theoretic methods in general equilibrium analysis. Proceedings of the NATO Advanced Study Institute, Long Island, NY (USA), July 1-12, 1991, NATO ASI Series. Series D. Behavioural and Social Sciences, vol. 77, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1994, pp. 243- 248.
  23. H. Moulin, Game Theory for the Social Sciences, New York University Press, New York, 1986.
  24. R.B. Myerson, Acceptable and predominant correlated equilibria, International Journal of Game Theory 15 (1986) 133-154.
  25. R.B. Myerson, Multistage games with communication, Econometrica 54 (1986) 323-358.
  26. R.B. Myerson, Communication, correlated equilibria and incentive compatibility, in: R.J. Aumann et al. (Eds.), Handbook of Game Theory with Economic Applications, Handbooks in Economics, vol. 2, Elsevier, Amsterdam, 1994, pp. 827-847.
  27. R. Nau, S.G. Canovas, P. Hansen, On the geometry of Nash equilibria and correlated equilibria, International Journal of Game Theory (2004) 443-453.
  28. P. Neumann, D. Ramsey, K. Szajowski, Randomized stopping times in Dynkin games, ZAMM 82 (11-12) (2002) 811-819.
  29. A.S. Nowak, Existence of correlated weak equilibria in discounted stochastic games with general state space, Stochastic Games and Related Topics. In Honor of Prof. L.S. Shapley, Proc. Workshop, Chicago/IL (USA) 1987Theory Decis. Libr., Ser. C 7, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1991, pp. 135-143.
  30. A.S. Nowak, Correlated relaxed equilibria in nonzero-sum linear differential games, Journal of Mathematical Analysis and Applications 163 (1992) 104-112.
  31. A.S. Nowak, T.E.S. Raghavan, Existence of stationary correlated equilibria with symmetric information for discounted stochastic games, Mathematics of Operations Research 17 (3) (1992) 519-526.
  32. A.S. Nowak, K. Szajowski, Nonzero-sum stochastic games, in: T. Parthasarathy, M. Bardi, T.E.S. Raghavan (Eds.), Stochastic and Differential Games. Theory and Numerical Methods (Boston), Annals of the International Society of Dynamic Games, Birkha ¨ser, 1998, pp. 297-342.
  33. Y. Ohtsubo, A nonzero-sum extension of Dynkin's stopping problem, Mathematics of Operations Research 12 (1987) 277-296.
  34. H. Peters, K. Vrieze, Nash refinements of equilibria, Journal of Optimization Theory and Applications 83 (2) (1994) 355-373.
  35. T. Radzik, On a new solution concept for bargaining problems, Applied Mathematics 25 (3) (1998) 285-294.
  36. D. Rosenberg, E. Solan, N. Vieille, Stopping games with randomized strategies, Probability Theory and Related Fields 119 (2001) 433-451.
  37. M. Sakaguchi, Non-zero-sum games related to the secretary problem, Journal of the Operations Research Society of Japan 23 (1980) 287-293.
  38. M. Sakaguchi, A simplified two-person multi-stage poker with optimal stopping, Mathematica Japonica 28 (1983) 287-303.
  39. M. Sakaguchi, Bilateral sequential games related to the no-information secretary problem, Mathematica Japonica 29 (1984) 961-974.
  40. M. Sakaguchi, Some two-person bilateral games in the generalized secretary problem, Mathematica Japonica 33 (1988) 637-654.
  41. M. Sakaguchi, Optimal stopping games-a review, Mathematica Japonica 42 (1995) 343-351.
  42. M. Sakaguchi, Optimal stopping games where players have weighted privilege, Game Theory and Application 4 (2001) 116-131.
  43. A.N. Shiryaev, Optimal Stopping Rules, Springer-Verlag, New York, Heidelberg, Berlin, 1978.
  44. E. Shmaya, E. Solan, Two-player nonzero-sum stopping games in discrete time, Annals of Probability 32 (3B) (2004) 2733-2764.
  45. E. Solan, Characterization of correlated equilibria in stochastic games, International Journal of Game Theory 30 (2) (2001) 259-277.
  46. E. Solan, N. Vieille, Quitting games, Mathematics of Operations Research 26 (2) (2001) 265-285.
  47. E. Solan, N. Vieille, Correlated equilibrium in stochastic games, Games and Economic Behavior 38 (2002) 362-399.
  48. E. Solan, R.V. Vohra, Correlated equilibrium in quitting games, Mathematics of Operations Research 26 (3) (2001) 601-610.
  49. S. Sorin, Communication, correlation and cooperation, in: S. Hart et al. (Eds.), Cooperation: Game-theoretic approaches, Proceedings of the NATO ASI, SUNY, Stony Brook, NY, USA, July 18-29, 1994, NATO ASI Ser., Ser. F, Comput. Syst. Sci., vol. 155, Springer, Berlin, 1997, pp. 199-218.
  50. W.E. Stein, D.A. Seale, A. Rapoport, Analysis of heuristic solutions to the best choice problem, European Journal of Operational Research 151 (1) (2003) 140-152.
  51. K. Szajowski, Markov stopping games with random priority, Zeitschrift fu ¨r Operations Research 39 (1) (1994) 69-84.
  52. K. Szajowski, M. Yasuda, Voting procedure on stopping games of Markov chain, in: Anthony H. Christer Shunji Osaki, C. Thomas Lyn (Eds.), UK-Japanese Research Workshop on Stochastic Modelling in Innovative Manufecuring, July 21-22, 1995, Moller Centre, Churchill College, University of Cambridge, UK, Lecture Notes in Economics and Mathematical Systems, vol. 445, Springer, 1996, pp. 68-80.
  53. W. Thomson, Cooperative models of bargaining, in: R.J. Aumann et al. (Eds.), Handbook of Game Theory with Economic Applications, Handbooks in Economics, vol. 2, Elsevier, Amsterdam, 1994, pp. 1237-1284.
  54. B. Tolwin ´ski, A. Haurie, G. Leitmann, Cooperative equilibria in differential games, Journal of Mathematical Analysis and Applications 119 (1986) 182-202.
  55. M. Yasuda, On a randomized strategy in Neveu's stopping problem, Stochastic Processes and their Applications 21 (1985) 159-166.
  56. M. Yasuda, J. Nakagami, M. Kurano, Multi-variate stopping problem with a monoton rule, Journal of the Operations Research Society of Japan 25 (1982) 334-350.

Related papers

Correlated equilibria in Markov stopping games. The numerical methods and examples
David Ramsey

2004

View PDFchevron_right
Correlated equilibria in Markov stopping games
David Ramsey
View PDFchevron_right
Sequential correlated equilibrium in stopping games
Yuval Heller

Operations Research, 2012

In many situations, such as trade in stock exchanges, agents have many opportunities to act within a short interval of time. The agents in such situations can often coordinate their actions in advance, but coordination during the game consumes too much time. An equilibrium in such situations has to be sequential in order to handle mistakes made by players. In this paper, we present a new solution concept for infinite-horizon dynamic games, which is appropriate for such situations: a sequential normal-form correlated approximate equilibrium. Under additional assumptions, we show that every such game admits this kind of equilibrium.

View PDFchevron_right
Sequential Correlated Equilibria in Stopping Games
Yuval Heller

Social Science Research Network, 2009

In many situations, such as trade in stock exchanges, agents have many opportunities to act within a short interval of time. The agents in such situations can often coordinate their actions in advance, but coordination during the game consumes too much time. An equilibrium in such situations has to be sequential in order to handle mistakes made by players. In this paper, we present a new solution concept for infinite-horizon dynamic games, which is appropriate for such situations: a sequential normal-form correlated approximate equilibrium. Under additional assumptions, we show that every such game admits this kind of equilibrium.

View PDFchevron_right
Randomized stopping games and Markov market games
Elzbieta Ferenstein

Mathematical Methods of Operations Research, 2007

We study nonzero-sum stopping games with randomized stopping strategies. The existence of Nash equilibrium and ε-equilibrium strategies are discussed under various assumptions on players random payoffs and utility functions dependent on the observed discrete time Markov process. Then we will present a model of a market game in which randomized stopping times are involved. The model is a mixture of a stochastic game and stopping game. Keywords Stopping games • Stochastic games • Nash equilibrium • Markov chain Research supported by grant PBZ-KBN-016/P03/99.

View PDFchevron_right

Related topics

  • Mathematics
  • Computer Science
  • Secretary problem
  • Nash Equilibrium
  • Markov Process
  • Equilibrium selection
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved