A flexible two-piece normal dynamic linear model

Computational Statistics & Data Analysis, 2009

A Bayesian Markov chain Monte Carlo methodology is developed for the estimation of multivariate linear Gaussian state space models. In particular, an e¢ cient simulation smoothing algorithm is proposed that makes use of the univariate representation of the state space model. Substantial gains over existing algorithms in computational e¢ ciency are achieved using the new simulation smoother for the analysis of high dimensional multivariate time series.

IEEE Transactions on Signal Processing, 2015

Bayesian filtering aims at tracking sequentially a hidden process from an observed one. In particular, sequential Monte Carlo (SMC) techniques propagate in time weighted trajectories which represent the posterior probability density function (pdf) of the hidden process given the available observations. On the other hand, Conditional Monte Carlo (CMC) is a variance reduction technique which replaces the estimator of a moment of interest by its conditional expectation given another variable. In this paper we show that up to some adaptations, one can make use of the time recursive nature of SMC algorithms in order to propose natural temporal CMC estimators of some point estimates of the hidden process, which outperform the associated crude Monte Carlo (MC) estimator whatever the number of samples. We next show that our Bayesian CMC estimators can be computed exactly, or approximated efficiently, in some hidden Markov chain (HMC) models; in some jump Markov state-space systems (JMSS); as well as in multitarget filtering. Finally our algorithms are validated via simulations.

The New Palgrave Dictionary of Economics, 2008

This article describes the use of Bayesian methods in the statistical analysis of time series. The use of Markov chain Monte Carlo methods has made even the more complex time series models amenable to Bayesian analysis. Models discussed in some detail are ARIMA models and their fractionally integrated counterparts, state-space models, Markov switching and mixture models, and models allowing for timevarying volatility. A final section reviews some recent approaches to nonparametric Bayesian modelling of time series.

Multidimensional Systems and Signal Processing, 2010

We present a novel and general methodology for modeling time-varying vector autoregressive processes which are widely used in many areas such as modeling of chemical processes, mobile communication channels and biomedical signals. In the literature, most work utilize multivariate Gaussian models for the mentioned applications, mainly due to the lack of efficient analytical tools for modeling with non-Gaussian distributions. In this paper, we propose a particle filtering approach which can model non-Gaussian autoregressive processes having cross-correlations among them. Moreover, time-varying parameters of the process can be modeled as the most general case by using this sequential Bayesian estimation method. Simulation results justify the performance of the proposed technique, which potentially can model also Gaussian processes as a sub-case.

Journal of Statistical Software, 2014

Journal of Statistical Software 3 algorithms. In particular it contains classes for the Gibbs sampler, Metropolis Hastings (MH), independent MH, random walk MH, orientational Monte Carlo (OBMC) as well as the slice sampler.

We introduce state-space models where the functionals of the observational and the evolutionary equations are unknown, and treated as random functions evolving with time. Thus, our model is nonparametric and generalizes parametric state-space models, such as the extended Kalman filter. This random function approach also frees us from the restrictive assumption that the functional forms, although time-dependent, are of fixed forms. We specify Gaussian processes as priors of the random functions and exploit the "look-up table approach" of Bhattacharya (2007) to efficiently handle the dynamic structure of the model. We consider both univariate and multivariate situations, using the Markov chain Monte Carlo (MCMC) approach for studying the posterior distributions of interest. In the case of challenging multivariate situations we demonstrate that the newly developed Transformation-based MCMC (TMCMC) provides interesting and efficient alternatives to the usual proposal distributions. We illustrate our methods with simulated data sets, obtaining very encouraging results in both univariate and multivariate situations. Moreover, using our Gaussian process approach we analysed a real data set, which has also been analysed by and using the linearity assumption. Our analyses show that towards the end of the time series, the linearity assumption of the previous authors breaks down.

Applied Stochastic Models in Business and Industry, 2006

This thesis presents a Bayesian analysis of a non-linear time series model. In particular, we deal with a mixture of normal distributions whose means are linear functions of the past values of the observed variable. Since the component densities of the mixture can be viewed as the conditional distributions of different Gaussian autoregressive models, the model is referred as mixture of autoregressive components. Bayesian perspective implies some advantages, especially in terms of model determination. First of all, it has been recognized that usual criterions like AIC and BIC are not satisfactory for the mixture of autoregressive components. In addition, these standard approaches do not take into account model uncertainty because they select a single model and then make inference based on this model. On the contrary, our Bayesian approach maintains consideration of several models, with the input of each into the analysis weighted by the model posterior probability. Both parameter estimation and model selection do not lend themselves to analytic solutions and we use Markov Chain Monte Carlo (or MCMC) approximation methods, which have had a real explosion over the last years, especially in Bayesian statistics. Our work takes into account the stationarity conditions of the autoregressive coefficients of the mixture components through a reparametrization in terms of partial autocorrelations. Finally, this thesis addresses the important task of modelling and forecasting return volatility. Several stylized facts about volatility have been recognized and they are captured by the mixture of autoregressive components. i N (a,b) (θ|µ, σ 2 ) = N(θ|µ, σ 2 ) F N (b|µ, σ 2 ) -F N (a|µ, σ 2 ) where F N is the normal cumulative distribution function.

Applied Mathematics and Computation, 2008

We consider the problem of optimal state estimation for a wide class of nonlinear time series models. A modified sigma point filter is proposed, which uses a new procedure for generating sigma points. Unlike the existing sigma point generation methodologies in engineering where negative probability weights may occur, we develop an algorithm capable of generating sample points that always form a valid probability distribution while still allowing the user to sample using a random number generator. The effectiveness of the new filtering procedure is assessed through simulation examples.

Journal of the American Statistical Association, 1985

Bayesian Forecasting MIKE WEST, P. JEFF HARRISON, and HELIO S. MIGON* Dynamic Bayesian models are developed for application in nonlinear, non-normal time series and regression problems, providing dynamic extensions of standard generalized linear models. A key feature of the analysis is the use of conjugate prior and posterior distributions for the exponential family parameters. This leads to the calculation of closed, standard-form predictive distributions for forecasting and model criticism. The structure of the models depends on the time evolution of underlying state variables, and the feedback of observational information to these variables is achieved using linear Bayesian prediction methods. Data analytic aspects of the models concerning scale parameters and outliers are discussed, and some applications are provided.

State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference and learning (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gaussian process prior over the state transition dynamics, resulting in a flexible model able to capture complex dynamical phenomena. To enable efficient inference, we marginalize over the transition dynamics function and infer directly the joint smoothing distribution using specially tailored Particle Markov Chain Monte Carlo samplers. Once a sample from the smoothing distribution is computed, the state transition predictive distribution can be formulated analytically. Our approach preserves the full nonparametric expressivity of the model and can make use of sparse Gaussian processes to greatly reduce computational complexity.