Non-constant discounting in continuous time

IEEE Transactions on Automatic Control, 1999

2014

We study the existence of optimal strategies and value function of non stationary Markov Decision Processes under variable discounted criteria, when the action space is assumed to be Borel and the action space to be compact. With this new way of defining the value of a policy, we show existence of Markov deterministic optimal policies in the finite-horizon case, and a recursive method to obtain such ones. For the infinite horizon problem we characterize the value function and show existence of stationary deterministic policies. The approach presented is based on the use of adequate dynamic programming operators.

Annals of Operations Research, 2011

Journal of Economic Dynamics & Control, 2009

This paper derives the HJB (Hamilton-Jacobi-Bellman) equation for sophisticated agents in a finite horizon dynamic optimization problem with non-constant discounting in a continuous setting, by using a dynamic programming approach. A simple example is used in order to illustrate the applicability of this HJB equation, by suggesting a method for constructing the subgame perfect equilibrium solution to the problem. Conditions for the observational equivalence with an associated problem with constant discounting are analyzed. Special attention is paid to the case of free terminal time. Strotz's model (an eating cake problem of a nonrenewable resource with non-constant discounting) is revisited. JEL Classification: C61; D83; C73 Keywords: Non-constant discounting, naive and sophisticated agents, free terminal time, observational equivalence

SSRN Electronic Journal, 2000

The possibility of non-constant discounting is important in environmental and resource management problems where current decisions affect welfare in the far-distant future, as with climate change. The difficulty of analyzing models with non-constant discounting limits their application. We describe and provide software to implement an algorithm to numerically obtain a Markov Perfect Equilibrium for an optimal control problem with non-constant discounting. The software is available online. We illustrate the approach by studying welfare and observational equivalence for a particular renewable resource management problem.

Operations Research Letters, 2014

In this note we study a deterministic dynamic programming model with generalised discounting. We use a modified weighted norm approach and an approximation technique to a study of the Bellman equation for unbounded return functions. Furthermore, we apply this theory to economic growth models.

Kybernetika, 2011

In this paper a problem of consumption and investment is presented as a model of a discounted Markov decision process with discrete-time. In this problem, it is assumed that the wealth is affected by a production function. This assumption gives the investor a chance to increase his wealth before the investment. For the solution of the problem there is established a suitable version of the Euler Equation (EE) which characterizes its optimal policy completely, that is, there are provided conditions which guarantee that a policy is optimal for the problem if and only if it satisfies the EE. The problem is exemplified in two particular cases: for a logarithmic utility and for a Cobb-Douglas utility. In both cases explicit formulas for the optimal policy and the optimal value function are supplied.

IMF Working Papers, 2006

We derive a stationary equilibrium in a two-player multi-stage game with endogenous discounting. At each stage, the probability to reach the next stage is determined by the players' current actions. We assume that the players are myopic in the sense that they take the future strategies of their opponents as given. We find that the stationary myopic equilibrium of the infinite-horizon multi-stage game corresponds to the infinite repetition of a Nash equilibrium of an induced (one-shot) limit game. Interestingly, this stationary myopic equilibrium is singled out when studying limiting equilibria of the associated multi-stage game with a finite horizon.

Mathematical Methods of Operations Research, 2004

This paper presents three conditions. Each of them guarantees the uniqueness of optimal policies of discounted Markov decision processes. The conditions presented here impose hypotheses specifically on the state space X, the action space A, the admissible action sets A(x),x∈X, the transition probability Q, and on the cost function c. Two of these conditions require mainly convexity assumptions, but the third one does not need this kind of assumptions. However, it needs certain stochastic order relations in Q, and the cost function c to reach its minimum with respect to the actions, just in one action. We illustrate the conditions with several examples including, in particular, discrete models, the linear regulator problem, and also a model of an inventory control system.

Two-Stage Exponential (TSE) discounting, the model developed here, generalises exponential discounting in a parsimonious way. It can be seen as an extension of Quasi-Hyperbolic discounting to continuous time. A TSE discounter has a constant rate of time preference before and after some threshold time; the switch point. If the switch point is expressed in calendar time, TSE discounting captures time consistent behaviour. If it is expressed in waiting time, TSE discounting captures time invariant behaviour. We provide preference foundations for all cases, showing how the switch point is derived endogenously from behaviour. We apply each case to Rubinstein’s infinite-horizon, alternating-offers bargaining model.