A Nonparametric Bayesian Approach to Inverse Problems

2002

Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model parameters up to more realistic advanced problems of myopic or blind inversion where we may be uncertain about the forward model and we may have noisy data.

Inverse Problems in Science and Engineering, 2008

This work deals with the use of radial basis functions for the interpolation of the likelihood function, in a parameter estimation problem solved with the Bayesian technique. The proposed interpolation of the likelihood function is applied to a test-case involving the estimation of parameters in the dispersion of a tracer in saturated soils. The use of the interpolated likelihood function reduces significantly the computational cost associated with the implementation of Markov Chain Monte Carlo methods to the solution of the present inverse problem.

Geophysical Journal International, 1994

We present a practical algorithm for determining the Bayesian solution of non-linear inverse problems with a limited number of parameters. This approach allows the use of very general conditional probability density functions (pdfs of the data given the model parameters) and a priori pdfs (prior beliefs upon the parameters). The results consist in the a posteriori marginal pdfs of the model parameters. The marginal pdfs describe the additional information (if any) brought by the data to the prior knowledge about the model parameters. We have paid a special attention to the numerical calculation of the a posteriori pdfs and we propose a solution which enables the simultaneous determination of the numerical estimates of the a posteriori pdfs and their uncertainties. With the help of a practical example (the 1-D magnetotelluric inverse problem) with synthetic and real data, we show that in most cases, the a posteriori marginal pdfs are complicated functions and cannot be predicted from the data pdf or the a priori pdfs. Consequently, the standard analyses applied to the same data sets such as the maximum likelihood techniques or the asymptotic estimation technique would lead to severely biased estimates of the parameters. This remark is likely to be true for any non-linear inverse problem.

Maximum Entropy and Bayesian Methods, 1996

The main object of this paper is to present some general concepts of Bayesian inference and more specifically the estimation of the hyperparameters in inverse problems. We consider a general linear situation where we are given some data y related to the unknown parameters x by y = Ax + n and where we can assign the probability laws p(x|θ), p(y|x, β), p(β) and p(θ). The main discussion is then how to infer x, θ and β either individually or any combinations of them. Different situations are considered and discussed. As an important example, we consider the case where θ and β are the precision parameters of the Gaussian laws to whom we assign Gamma priors and we propose some new and practical algorithms to estimate them simultaneously. Comparisons and links with other classical methods such as maximum likelihood are presented.

SIAM Journal on Scientific Computing, 2014

The Bayesian approach to inverse problems typically relies on posterior sampling approaches, such as Markov chain Monte Carlo, for which the generation of each sample requires one or more evaluations of the parameter-to-observable map or forward model. When these evaluations are computationally intensive, approximations of the forward model are essential to accelerating sample-based inference. Yet the construction of globally accurate approximations for nonlinear forward models can be computationally prohibitive and in fact unnecessary, as the posterior distribution typically concentrates on a small fraction of the support of the prior distribution. We present a new approach that uses stochastic optimization to construct polynomial approximations over a sequence of measures adaptively determined from the data, eventually concentrating on the posterior distribution. The approach yields substantial gains in efficiency and accuracy over prior-based surrogates, as demonstrated via application to inverse problems in partial differential equations.

2001

The Bayesian approach has proved to be a coherent approach to handle ill posed Inverse problems. However, the Bayesian calculations need either an optimization or an integral calculation. The maximum a posteriori (MAP) estimation requires the minimization of a compound criterion which, in general, has two parts: a data fitting part and a prior part. In many situations the criterion to be minimized becomes multimodal. The cost of the Simulated Annealing (SA) based techniques is in general huge for inverse problems.

Springer eBooks, 2017

These lecture notes highlight the mathematical and computational structure relating to the formulation of, and development of algorithms for, the Bayesian approach to inverse problems in differential equations. This approach is fundamental in the quantification of uncertainty within applications involving the blending of mathematical models with data. The finite dimensional situation is described first, along with some motivational examples. Then the development of probability measures on separable Banach space is undertaken, using a random series over an infinite set of functions to construct draws; these probability measures are used as priors in the Bayesian approach to inverse problems. Regularity of draws from the priors is studied in the natural Sobolev or Besov spaces implied by the choice of functions in the random series construction, and the Kolmogorov continuity theorem is used to extend regularity considerations to the space of Hölder continuous functions. Bayes' theorem is derived in this prior setting, and here interpreted as finding conditions under which the posterior is absolutely continuous with respect to the prior, and determining a formula for the Radon-Nikodym derivative in terms of the likelihood of the data. Having established the form of the posterior, we then describe various properties common to it in the infinite dimensional setting. These properties include well-posedness, approximation theory, and the existence of maximum a posteriori estimators. We then describe measure-preserving dynamics, again on the infinite dimensional space, including Markov chain-Monte Carlo and sequential Monte Carlo methods, and measure-preserving reversible stochastic differential equations. By formulating the theory and algorithms on the underlying infinite dimensional space, we obtain a framework suitable for rigorous analysis of the accuracy of reconstructions, of computational complexity, as well as naturally constructing algorithms which perform well under mesh refinement, since they are inherently well-defined in infinite dimensions.

Journal of physics, 2012

Variational Bayesian Approximation (VBA) methods are recent tools for effective Bayesian computations. In this paper, these tools are used for inverse problems where the prior models include hidden variables and where where the estimation of the hyper parameters has also to be addressed. In particular two specific prior models (Student-t and mixture of Gaussian models) are considered and details of the algorithms are given.

In this paper, first the basics of Bayesian inference with a parametric model of the data is presented. Then, the needed extensions are given when dealing with inverse problems and in particular the linear models such as Deconvolution or image reconstruction in Computed Tomography (CT). The main point to discuss then is the prior modeling of signals and images. A classification of these priors is presented, first in separable and Markovien models and then in simple or hierarchical with hidden variables. For practical applications, we need also to consider the estimation of the hyper parameters. Finally, we see that we have to infer simultaneously on the unknowns, the hidden variables and the hyper parameters. Very often, the expression of this joint posterior law is too complex to be handled directly. Indeed, rarely we can obtain analytical solutions to any point estimators such the Maximum A posteriori (MAP) or Posterior Mean (PM). Three main tools are then can be used: Laplace approximation (LAP), Markov Chain Monte Carlo (MCMC) and Bayesian Variational Approximations (BVA). To illustrate all these aspects, we will consider a deconvolution problem where we know that the input signal is sparse and propose to use a Student-t prior for that. Then, to handle the Bayesian computations with this model, we use the property of Student-t which is modelling it via an infinite mixture of Gaussians, introducing thus hidden variables which are the variances. Then, the expression of the joint posterior of the input signal samples, the hidden variables (which are here the inverse variances of those samples) and the hyper-parameters of the problem (for example the variance of the noise) is given. From this point, we will present the joint maximization by alternate optimization and the three possible approximation methods. Finally, the proposed methodology is applied in different applications such as mass spectrometry, spectrum estimation of quasi periodic biological signals and X ray computed tomography.

Communications in Computational Physics, 2009

We present an efficient numerical strategy for the Bayesian solution of inverse problems. Stochastic collocation methods, based on generalized polynomial chaos (gPC), are used to construct a polynomial approximation of the forward solution over the support of the prior distribution. This approximation then defines a surrogate posterior probability density that can be evaluated repeatedly at minimal computational cost. The ability to simulate a large number of samples from the posterior distribution results in very accurate estimates of the inverse solution and its associated uncertainty. Combined with high accuracy of the gPC-based forward solver, the new algorithm can provide great efficiency in practical applications. A rigorous error analysis of the algorithm is conducted, where we establish convergence of the approximate posterior to the true posterior and obtain an estimate of the convergence rate. It is proved that fast (exponential) convergence of the gPC forward solution yields similarly fast (exponential) convergence of the posterior. The numerical strategy and the predicted convergence rates are then demonstrated on nonlinear inverse problems of varying smoothness and dimension.