Fuzzy Perceptive Values for MDPs with Discounting

Kybernetika

The paper concerns Markov decision processes (MDPs) with both the state and the decision spaces being finite and with the total reward as the objective function. For such a kind of MDPs, the authors assume that the reward function is of a fuzzy type. Specifically, this fuzzy reward function is of a suitable trapezoidal shape which is a function of a standard non-fuzzy reward. The fuzzy control problem consists of determining a control policy that maximizes the fuzzy expected total reward, where the maximization is made with respect to the partial order on the α-cuts of fuzzy numbers. The optimal policy and the optimal value function for the fuzzy optimal control problem are characterized by means of the dynamic programming equation of the standard optimal control problem and, as main conclusions, it is obtained that the optimal policy of the standard problem and the fuzzy one coincide and the fuzzy optimal value function is of a convenient trapezoidal form. As illustrations, fuzzy extensions of an optimal stopping problem and of a red-black gambling model are presented.

In this paper, we consider the model that the information on the rewards in vector-valued Markov decision processes includes imprecision or ambiguity. The fuzzy reward model is analyzed as follows: The fuzzy reward is represented by the fuzzy set on the multi-dimensional Euclidian space Rp and the infinite horizon fuzzy expected discounted reward(FEDR) from any stationary policy is characterized as a unique fixed point of the corresponding contractive op- erator. Also, we fined a Pareto optimal policy which maximizes the infinite horizon FEDR over all stationary policies under the pseudo order induced by a convex coneRp. As a numerical example, the machine maintenance problem is considered.

In this paper, we consider the model that the information on the rewards in vector-valued Markov decision processes includes imprecision or ambiguity. The fuzzy reward model is analyzed as follows: The fuzzy reward is represented by the fuzzy set on the multi-dimensional Euclidian space R and the infinite horizon fuzzy expected discounted reward(FEDR) from any stationary policy is characterized as a unique fixed point of the corresponding contractive operator. Also, we fined a Pareto optimal policy which maximizes the infinite horizon FEDR over all stationary policies under the pseudo order induced by a convex cone R . As a numerical example, the machine maintenance problem is considered.

Bulletin of informatics and cybernetics, 1996

A utility treatment is studied in the framework of discounted Markov decision processes. We will define a new index called a utility deviation related to the risk premium, which is characterized by an iterative formula. Examples are given in the quadratic and the exponential utility cases.

Fuzzy Sets and Systems, 2003

A continuous-time decision process with fuzzy transitions, which is termed fuzzy decision process, is discussed from a viewpoint of a dynamic fuzzy system by the authors. The discounted total reward is given by a fuzzy number and is discussed by the fuzzy max order. This paper gives the optimality equation for the fuzzy decision model by a fuzzy di erential equation. The optimization problem of the discounted total rewards is also discussed by a fuzzy expectation generated from a fuzzy goal, and an optimal policy and the permissible range of the decision maker's optimal expected payo s are given for this problem. Also, an illustrated example is given to explain the idea of the paper.

Fuzzy Sets and Systems, 1994

This paper constructs a Markov fuzzy process, which represents transitions of grades of fuzzy sets, with a transition possibility measure and a general state space. We analyse Snell's optimal stopping problem for the process and we apply the results to solve fuzzy dynamic programming with optimal stopping times and with general state spaces and action spaces under fuzzy transitions.

Fuzzy Sets and Systems, 2006

This article was originally published in a journal published by Elsevier, and the attached copy is provided by Elsevier for the author's benefit and for the benefit of the author's institution, for non-commercial research and educational use including without limitation use in instruction at your institution, sending it to specific colleagues that you know, and providing a copy to your institution's administrator.

Wiley Encyclopedia of Operations Research and Management Science, 2011

This article describes the results on the existence of optimal and nearly optimal policies for Markov Decision Processes (MDPs) with total expected discounted rewards. The problem of optimization of total expected discounted rewards for MDPs is also known under the name of discounted dynamic programming.

Top, 2006

This paper is a survey of recent results on continuous-time Markov decision processes (MDPs) withunbounded transition rates, and reward rates that may beunbounded from above and from below. These results pertain to discounted and average reward optimality criteria, which are the most commonly used criteria, and also to more selective concepts, such as bias optimality and sensitive discount criteria. For concreteness, we consider only MDPs with a countable state space, but we indicate how the results can be extended to more general MDPs or to Markov games.

In this paper, we shall develop a fuzzy treatment for uncertain Markov decision processes which allow for a fluctuating transition matrix at each step in time. The decision model with uncertain transition matrices is described by the use of fuzzy sets in which we find a Pareto optimal policy maximizing the infinite horizon fuzzy expected discounted reward over all stationary policies under some partial order. The Pareto optimal policies are characterized by maximal solutions of an optimal equation including efficient set-functions. As a numerical example, the machine maintenance problem is considered.