Parameter estimation for partially observed queues

This diploma thesis considers the problem of parameter estimation for time-continuous Markov processes on discrete state spaces. A Markov process is a mathematical model for the random evolution of a system of events over the time axis. The stochastic process has the Markov property, i.e. the likelihood of a given future state only depends on the present state and not on the past evolution of the process. Therefore, a Markov process is also called a memoryless process. These processes are very important for descriptions of natural phenomena in physics, the sciences and economics. If we consider the board game Monopoly, we can simulate the movement of a player's pawn using a Markov process. The discrete state space is the board consisting of forty spaces. The pawn's moves are determined entirely by dice rolls and by its starting points. It does not depend on how the pawn has reached the current state or even the previous moves. Therefore the movement process has the Markov property. Of course we exclude some of the 'community chest' states, which have a direct in uence on the pawn's movement, e.g. the card 'Go directly to jail, ...'. Furthermore, Markov processes are used in economics to model a great variety of di erent phenomena, as for example asset prices or market crashes. Another typical example is a random walk on the number line which starts at zero and transitions occur one step upward or downward with the same probability at each step. The position of the process in the following transitions only depends on the present position and not on the way the process has walked before it reached its present state. For example zero-sum games can be modeled with this process. Thereby it moves up at each win and down at each loose. Therefore, the present state represents the di erence of both wins and looses. Another extensive eld of applications is the queueing theory. Considering the length of a queue on a discrete state space, a Markov process allows us to model the stochastic behavior of a queue. The Markov property is immediately ensured since the next state of the queue length only depends on the present one. In contrast to the Monopoly example, this case is modeled by a continuous-time Markov process since events do not occur as before at each dice roll, but rather new customers could arrive at every single moment. This leads us to many other di erent examples: such as Birth-and-Death processes where the number of citizens is counted or customer lines in a supermarket. Furthermore, we could consider a queue of customers in a telephone system waiting for connection. Hereby, the telephone company is interested in lowering its running costs as well as in satisfying its customers. If the capacity is not big enough, it can happen in peak times that someone has to wait very long for a free line before getting connected. The queueing theory is able to give us an answer about over ows of the system as well as about the necessary number of connection lines to prevent such over ows. Another 7 8 Chapter 1. Introduction applied example is when the owner of a supermarket gets an instrument to identify the needed number of open cash desks to minimize waiting times for customers. In summary, queueing theory is mainly used to optimize the problems connected any sort of waiting lines. Now the question arises how these kinds of queueing systems can be modeled. Normally, we do not have exact information about the system and its parameters. Because of this, the classical queueing system has Poisson arrival processes with exponentially distributed inter arrival times as well as exponentially distributed service times, for instance with parameter  for the arrival process and for the service times. The size of the waiting room is variable but usually assumed to be countable and in nite. This kind of queueing system can be written in Kendall's notation as M=M=1=1 system where the rst and second M=Markovian stands for the mentioned arrival and service process, the 1 represents the number of service stations and the in nity symbol stands for the size of the waiting room. Since the parameters  and are unknown they must be estimated based on observation of the process, clearly, best results are obtained by continuous observation of the process. This would imply staying next to a supermarket's cash desk the whole time and recording all waiting line changes, including event times points. We would thereby not lose track of any events and would gain accurate information from the very beginning. This leads us to the question about how to use the information then: the maximum likelihood estimation of the parameters, which is simple and well-known, is normally used, see e.g. [2]Billingsley from 1961. Many others have investigated this kind of estimation since then and the main idea of the likelihood estimation has remained the same. Given an observation over a period of time we take those possible values for the parameters which would maximize the probability to get exactly that observation. Consequently, we need the transition probability functions. Since we assume the queue to t to a wellknown parametric model, for example the M=M=1=1 model, the distribution of both, the arrival and service processes, are known whereas only its corresponding parameters are to be estimated. Therefore, the theory of Markov processes and especially the queueing system theory give us the transition probabilities between the di erent states with respect to the parameters. With some stationarity conditions on the process and with knowledge of the right parameters, we could say for example how great the probability is for a special length of a queue at a random point of time. Maximum likelihood estimation of the parameters of the mentioned Birth-and-Death processes was rst studied by [10] Keiding in 1975. He showed how to estimate the birth and death intensities, if a continuous observation over a xed time interval of the process is available. Realistically speaking, however, a non-stop observation of a waiting line is rarely possible. In their work [7]Dehay & Yao considered the likelihood estimation for discretely observed Markov processes. This means that an observer comes back after a speci c period of time to count the length of the queue. They de ne this period of time to be xed and the process is therefore called equidistantly observed. The occurring observations are then called to be an observation chain. However, if a Markov process is only observed at discrete equidistant time points, the situation is 9 more complex. Whereas in the continuous observation case noted previously, we had accurate information without having missed any events, here, the case is di erent. It could happen that more customers arrive between two observation points and meanwhile the same amount of customers is being served at the front of the queue. Since we do not di erentiate between them in the queue, but only count the amount of the customers, we do not record any change of queueing length in that case. Nevertheless it is possible for Dehay and Yao to de ne a likelihood estimation function and to show some asymptotic properties for the maximum likelihood estimator in that case. The requirements for their theory are very clear. The process has to be a Markov process on a countable state space. Note: in this situation, it is possible to use an in nite state space. In addition the process is not allowed to have an in nite expected sojourn time at any state of queue. The intensity matrix Q must be conservative; this means that a process should have the same intensity to jump in and out of any state. Furthermore,

This is an annotated bibliography on estimation and inference results for queues and related stochastic models. The purpose of this document is to collect and categorise works in the field, allowing for researchers and practitioners to explore the various types of results that exist. This bibliography attempts to include all known works that satisfy both of these requirements: • Works that deal with queueing models. • Works that contain contributions related to methodology of parameter estimation, state estimation, hypothesis testing, confidence interval and/or actual data-sets of application areas. It also includes reference to selected additional related material in Section 8. There are additional works not mentioned in this bibliography that are mildly related. This includes methods for parameter estimation of point processes, methods for parameter estimation of stochastic matrix analytic models as well as inference, estimation and tomography of communication networks not directly modelled as queueing networks. Our attempt is to make this bibliography exhaustive, yet there are possibly some papers that we have missed. As it is updated continuously, additions and comments are welcomed. The sections below categorise the works based on several categories. A single paper may appear in several categories simultaneously. The final section lists all works in chronological order along with short descriptions of the contributions.

Queueing Systems, 2021

We present a broad literature survey of parameter and state estimation for queueing systems. Our approach is based on various inference activities, queueing models, observations schemes, and statistical methods. We categorize these into branches of research that we call estimation paradigms. These include: the classical sampling approach, inverse problems, inference for noninteracting systems, inference with discrete sampling, inference with queueing fundamentals, queue inference engine problems, Bayesian approaches, online prediction, implicit models, and control, design, and uncertainty quantification. For each of these estimation paradigms, we outline the principles and ideas, while surveying key references. We also present various simple numerical experiments. In addition to some key references mentioned here, a periodically-updated comprehensive list of references dealing with parameter and state estimation of queues will be kept in an accompanying annotated bibliography.

Performance Evaluation, 2006

Performance modeling has become increasingly important in the design, engineering and optimization of information technology (IT) infrastructures and applications. However, modeling work itself is time consuming and requires a good knowledge not only of the system, but also of modeling techniques. One of the biggest challenges in modeling complex IT systems consists in the calibration of model parameters, such as the service requirements of various job classes. We present an approach for solving this problem in the queueing network framework using inference techniques. This is done through a mathematical programming formulation, for which we propose an efficient and robust solution method. The necessary input data are end-to-end measurements which are usually easy to obtain. The robustness of our method means that the inferred model performs well in the presence of noisy data and further, is able to detect and remove outlying data sets. We present numerical experiments using data from real IT practice to demonstrate the promise of our framework and algorithm.

Transportation Planning and Technology, 2014

Queues are often associated with uncertainty or unreliability, which can arise from chance or climatic events, phase changes in system behaviour, or inherent randomness. Knowing the probability distribution of the number of customers in a queue is important for estimating the risk of stress or disruption to routine services and upstream blocking, potentially leading to exceeding critical limits, gridlock or incidents. The present paper focuses on time-varying queues produced by transient oversaturation during demand peaks where there is randomness in arrivals and service. The objective is to present practical methods for estimating a probability distribution from knowledge of the mean, variance and utilisation (degree of saturation) of a queue available from computationally efficient, if approximate, time-dependent calculation. This is made possible by a novel expression for time-dependent queue variance. The queue processes considered are those commonly used to represent isolated priority (M/M/1) and signal-like (M/D/1) systems, plus some statistical variations within the common Pollaczek-Khinchin framework. Results are verified by comparison with Markov simulation based on recurrence relations.