Qualitative Logics and Equivalences for Probabilistic Systems
2009, Logical Methods in Computer Science
https://doi.org/10.2168/LMCSAbstract
We investigate logics and equivalence relations that capture the qualitative behavior of Markov Decision Processes (MDPs). We present Qualitative Randomized Ctl (Qrctl): formulas of this logic can express the fact that certain temporal properties hold over all paths, or with probability 0 or 1, but they do not distinguish among intermediate probability values. We present a symbolic, polynomial time model-checking algorithm for Qrctl on MDPs. The logic Qrctl induces an equivalence relation over states of an MDP that we call qualitative equivalence: informally, two states are qualitatively equivalent if the sets of formulas that hold with probability 0 or 1 at the two states are the same. We show that for finite alternating MDPs, where nondeterministic and probabilistic choices occur in different states, qualitative equivalence coincides with alternating bisimulation, and can thus be computed via efficient partition-refinement algorithms. On the other hand, in nonalternating MDPs the equivalence relations cannot be computed via partition-refinement algorithms, but rather, they require non-local computation. Finally, we consider Qrctl * , that extends Qrctl with nested temporal operators in the same manner in which Ctl * extends Ctl. We show that Qrctl and Qrctl * induce the same qualitative equivalence on alternating MDPs, while on non-alternating MDPs, the equivalence arising from Qrctl * can be strictly finer. We also provide a full characterization of the relation between qualitative equivalence, bisimulation, and alternating bisimulation, according to whether the MDPs are finite, and to whether their transition relations are finitely-branching.
References (30)
- C. Álvarez, J. L. Balcázar, J. Gabarró, and M. Sántha. Parallel complexity in the design and analysis of concurrent systems. In PARLE '91: Proc. on Parallel architectures and languages Europe. Springer-Verlag, 1991.
- R. Alur, T.A. Henzinger, and O. Kupferman. Alternating time temporal logic. J. ACM, 49:672- 713, 2002.
- R. Alur, T.A. Henzinger, O. Kupferman, and M.Y. Vardi. Alternating refinement relations. In CONCUR 98: Concurrency Theory. 9th Int. Conf., volume 1466 of Lect. Notes in Comp. Sci., pages 163-178. Springer-Verlag, 1998.
- ASB + 95] A. Aziz, V. Singhal, F. Balarin, R.K. Brayton, and A.L. Sangiovanni-Vincentelli. It usually works: The temporal logic of stochastic systems. In Computer Aided Verification, volume 939 of Lect. Notes in Comp. Sci. Springer-Verlag, 1995.
- G. Bhat and R. Cleaveland. Efficient model checking via the equational µ-calculus. In Proc. 11th IEEE Symp. Logic in Comp. Sci., pages 304-312, 1996.
- A. Bianco and L. de Alfaro. Model checking of probabilistic and nondeterministic systems. In Found. of Software Tech. and Theor. Comp. Sci., volume 1026 of Lect. Notes in Comp. Sci., pages 499-513. Springer-Verlag, 1995.
- D.P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 1995. Volumes I and II.
- E. Bandini and R. Segala. Axiomatizations for probabilistic bisimulation. In Proc. 28th Int. Colloq. Aut. Lang. Prog., volume 2076 of Lect. Notes in Comp. Sci., pages 370-381. Springer- Verlag, 2001.
- K. Chatterjee, L. de Alfaro, and T.A. Henzinger. Trading memory for randomness. In QEST 04. IEEE Computer Society Press, 2004.
- E.M. Clarke and E.A. Emerson. Design and synthesis of synchronization skeletons using branch- ing time temporal logic. In Proc. Workshop on Logic of Programs, volume 131 of Lect. Notes in Comp. Sci., pages 52-71. Springer-Verlag, 1981.
- C. Courcoubetis and M. Yannakakis. The complexity of probabilistic verification. J. ACM, 42(4):857-907, 1995.
- L. de Alfaro. Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, 1997. Technical Report STAN-CS-TR-98-1601.
- L. de Alfaro. Temporal logics for the specification of performance and reliability. In Proc. of Symp. on Theor. Asp. of Comp. Sci., volume 1200 of Lect. Notes in Comp. Sci., pages 165-176. Springer-Verlag, 1997.
- L. de Alfaro, M. Faella, R. Majumdar, and V. Raman. Code-aware resource management. In EMSOFT 05: ACM Conference on Embedded Software, Lect. Notes in Comp. Sci. Springer- Verlag, 2005.
- L. de Alfaro and T.A. Henzinger. Concurrent omega-regular games. In Proc. 15th IEEE Symp. Logic in Comp. Sci., pages 141-154, 2000.
- L. de Alfaro, T.A. Henzinger, and O. Kupferman. Concurrent reachability games. In Proc. 39th IEEE Symp. Found. of Comp. Sci., pages 564-575. IEEE Computer Society Press, 1998.
- L. de Alfaro, T.A. Henzinger, and R. Majumdar. From verification to control: Dynamic pro- grams for omega-regular objectives. In Proc. 16th IEEE Symp. Logic in Comp. Sci., pages 279-290. IEEE Press, 2001. [dAKN + 00] L. de Alfaro, M. Kwiatkowska, G. Norman, D. Parker, and R. Segala. Symbolic model check- ing of concurrent probabilistic processes using MTBDDs and the Kronecker representation. In TACAS: Tools and Algorithms for the Construction and Analysis of Systems, volume 1785 of Lect. Notes in Comp. Sci., pages 395-410. Springer-Verlag, 2000.
- C. Derman. Finite State Markovian Decision Processes. Academic Press, 1970.
- J. Desharnais, V. Gupta, R. Jagadeesan, and P. Panangaden. Metrics for labelled markov systems. In CONCUR'99: Concurrency Theory. 10th Int. Conf., volume 1664 of Lect. Notes in Comp. Sci., pages 258-273. Springer, 1999.
- H. Hansson and B. Jonsson. A logic for reasoning about time and probability. Formal Aspects of Computing, 6(5):512-535, 1994.
- M. Kwiatkowska, G. Norman, and D. Parker. Verifying randomized distributed algorithms with prism. In Workshop on Advances in Verification (WAVE'00), 2000.
- D. Kozen. Results on the propositional µ-calculus. Theoretical Computer Science, 27(3):333- 354, 1983.
- R. Milner. Operational and algebraic semantics of concurrent processes. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science, volume B, pages 1202-1242. Elsevier Science Publishers (North-Holland), Amsterdam, 1990.
- Z. Manna and A. Pnueli. The Temporal Logic of Reactive and Concurrent Systems: Specification. Springer-Verlag, New York, 1991.
- A. Pogosyants, R. Segala, and N. Lynch. Verification of the randomized consensus algorithm of Aspnes and Herlihy: a case study. Distributed Computing, 13(3):155-186, July 2000.
- R. Segala. Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis, MIT, 1995. Technical Report MIT/LCS/TR-676.
- R. Segala and N.A. Lynch. Probabilistic simulations for probabilistic processes. In CONCUR'94: Concurrency Theory. 5th Int. Conf., volume 836 of Lect. Notes in Comp. Sci., pages 481-496. Springer-Verlag, 1994.
- R. Segala and A. Turrini. Comparative analysis of bisimulation relations on alternating and non-alternating probabilistic models. In QEST 05. IEEE, 2005.
- M.I.A. Stoelinga. Fun with FireWire: Experiments with verifying the IEEE1394 root contention protocol. In Formal Aspects of Computing, 2002.
- M.Y. Vardi and P. Wolper. Automata theoretic techniques for modal logics of programs. J. Comp. Sys. Sci., 32:183-221, 1986.