Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
General Sensitivity Analysis
Sensitivity Analysis via Finite Differences
Sensitivity Analysis and Data Assimilation
Method for Estimating the Hessian Matrix
References
All Topics
Computer Science
Theoretical Computer Science

Second-Order Information in Data Assimilation

Profile image of Dacian DaescuDacian DaescuProfile image of Ionel M NavonIonel M Navon

2002, Monthly Weather Review

https://doi.org/10.1175/1520-0493(2002)130<0629:SOIIDA>2.0.CO;2
visibility

…

description

61 pages

link

1 file

Abstract

In variational data assimilation (VDA) for meteorological and/or oceanic models, the assimilated fields are deduced by combining the model and the gradient of a cost functional measuring discrepancy between model solution and observation, via a first order optimality system. However existence and uniqueness of the VDA problem along with convergence of the algorithms for its implementation depend on the convexity of the cost function. Properties of local convexity can be deduced by studying the Hessian of the cost function in the vicinity of the optimum. This shows the necessity of second order information to ensure a unique solution to the VDA problem.

Related papers

A Second Order Adjoint Method to Targeted Observations
Humberto C Godinez

Lecture Notes in Computer Science, 2009

The role of the second order adjoint in targeting strategies is studied and analyzed. Most targeting strategies use the first order adjoint to identify regions where additional information is of potential benefit to a data assimilation system. The first order adjoint posses a restriction on the targeting time for which the linear approximation accurately tracks the evolution of perturbation. Using second order adjoint information it is possible to maintain some accuracy for longer time intervals, which can lead to an increase on the target time. We propose the use of the dominant eigenvectors of the Hessian matrix as an indicator of the directions of maximal error growth for a given targeting functional. These vectors are a natural choice to be included in the targeting strategies given their mathematical properties.

View PDFchevron_right
A reduced adjoint approach to variational data assimilation
Ibrahim Hoteit

2012

The adjoint method has been used very often for variational data assimilation. The computational cost to run the adjoint model often exceeds several original model runs and the method needs significant programming efforts to implement the adjoint model code. The work proposed here is variational data assimilation based on proper orthogonal decomposition (POD) which avoids the implementation of the adjoint of the tangent linear approximation of the original nonlinear model.

View PDFchevron_right
Efficiency of a POD-based reduced second-order adjoint model in 4D-Var data assimilation
Ionel M Navon

2006

Order reduction strategies aim to alleviate the computational burden of the four-dimensional variational data assimilation by performing the optimization in a low-order control space. The proper orthogonal decomposition (POD) approach to model reduction is used to identify a reduced-order control space for a two-dimensional global shallow water model. A reduced second-order adjoint (SOA) model is developed and used to facilitate the implementation of a Hessian-free truncated-Newton (HFTN) minimization algorithm in the POD-based space. The efficiency of the SOA/HFTN implementation is analysed by comparison with the quasi-Newton BFGS and a nonlinear conjugate gradient algorithm. Several data assimilation experiments that differ only in the optimization algorithm employed are performed in the reduced control space. Numerical results indicate that first-order derivative methods are effective during the initial stages of the assimilation; in the later stages, the use of second-order derivative information is of benefit and HFTN provided significant CPU time savings when compared to the BFGS and CG algorithms. A comparison with data assimilation experiments in the full model space shows that with an appropriate selection of the basis functions the optimization in the POD space is able to provide accurate results at a reduced computational cost. The HFTN algorithm benefited most from the order reduction since computational savings were achieved both in the outer and inner iterations of the method. Further experiments are required to validate the approach for comprehensive global circulation models.

View PDFchevron_right
The extended adjoint method
Stanislas Larnier

ESAIM: Mathematical Modelling and Numerical Analysis, 2013

Searching for the optimal partitioning of a domain leads to the use of the adjoint method in topological asymptotic expansions to know the influence of a domain perturbation on a cost function. Our approach works by restricting to local subproblems containing the perturbation and outperforms the adjoint method by providing approximations of higher order. It is a universal tool, easily adapted to different kinds of real problems and does not need the fundamental solution of the problem; furthermore our approach allows to consider finite perturbations and not infinitesimal ones. This paper provides theoretical justifications in the linear case and presents some applications with topological perturbations, continuous perturbations and mesh perturbations. This proposed approach can also be used to update the solution of singularly perturbed problems.

View PDFchevron_right
Comments on "Tangent linear and adjoint of 'on-off' processes and their feasibility for use in 4-dimensional variational data assimilation
Qin Xu

Tellus A, 1998

View PDFchevron_right
Adjoint Methods in CFD-Based Optimization-Gradient Computation & Beyond
Vaggelis Papoutsis

2012

An overview of recent achievements in the development of adjoint-based methods and tools in the fields of aero/hydrodynamic shape optimization, optimal active flow control and topology optimization, in compressible and incompressible flows, is presented. In the first part, the continuous adjoint approach to widely-used turbulence models, such as the low-Reynolds Spalart-Allmaras and the high-Reynolds k-ε ones is discussed. The relevant developments allow the computation of the exact gradient of the objective function using continuous adjoint, even for RANS assisted by wall functions, and overcome the frequently made assumption of negligible turbulence variations. The second part of this paper deals with higher-order sensitivity analysis based on the combined use of the adjoint approach and direct differentiation. Through the methods exposed herein, the aforementioned optimization problems can be solved faster via the Newton’s method; also, robust design problems can be tackled using...

View PDFchevron_right
Adjoint sensitivity in PDE constrained least squares problems as a multiphysics problem
Wouter Mulckhuyse

Compel-the International Journal for Computation and Mathematics in Electrical and Electronic Engineering, 2012

Purpose-The purpose of this paper is to provide a framework for the implementation of an adjoint sensitivity formulation for least-squares partial differential equations constrained optimization problems exploiting a multiphysics finite elements package. The estimation of the diffusion coefficient in a Poisson-type diffusion equation is used as an example. Design/methodology/approach-The authors derive the adjoint formulation in a continuous setting allowing to attribute to the direct and adjoint states the role of different fields to be solved for. They are one-way coupled through the mismatch between measured and direct states acting as a source term in the adjoint equation. Having solved for the direct and adjoint state, the sensitivity of the cost function with respect to the design variables can then be obtained by a suitable post-processing procedure. This sensitivity can then be used to efficiently solve the least-squares problem. Findings-The authors derived the adjoint formulation in a continuous setting allowing the direct and adjoint states to be attributed the role of different fields to be solved. They are one-way coupled through the mismatch between measured and direct states acting as a source term in the adjoint equation. It is found that, having solved for the direct and adjoint state, the sensitivity of the cost function with respect to the design variables can then be obtained by a suitable post-processing procedure. Research limitations/implications-This paper implies that modern multiphysics finite elements packages provide a flexible and extendable software environment for the experimentation with different adjoint formulations. Such tools are therefore expected to become increasingly important in solving notoriously difficult partial differential equation (PDE)-constrained least-squares problems. The framework also provides the possibility of experimentation with different regularization techniques (total variation and multiscale techniques for instance) to handle the ill-posedness of the problem. Originality/value-In this paper the adjoint sensitivity computation is casted as a multiphysics problem allowing for a flexible and extendable implementation.

View PDFchevron_right
Approximation of weak adjoints by reverse automatic differentiation of BDF methods
Leonard Wirsching

2011

With this contribution, we shed light on the relation between the discrete adjoints of multistep backward differentiation formula (BDF) methods and the solution of the adjoint differential equation. To this end, we develop a functional-analytic framework based on a constrained variational problem and introduce the notion of weak adjoint solutions. We devise a finite element Petrov-Galerkin interpretation of the BDF

View PDFchevron_right
Solving regularized nonlinear least-squares problem indual space with application to variational data assimilation
Selime Gürol

2013

Cette thèseétudie la méthode du gradient conjugué et la méthode de Lanczos pour la résolution de problèmes aux moindres carrés non-linéaires sousdéterminés et régularisés par un terme de pénalisation quadratique. Ces problèmes résultent souvent d'une approche du maximum de vraisemblance, et impliquent un ensemble de m observations physiques et n inconnues estimées par régression non linéaire. Nous supposons ici que n est grand par rapportà m. Un tel cas se présente lorsque des champs tridimensionnels sont estimésà partir d'observations physiques, par exemple dans l'assimilation de données appliquée aux modèles du système terrestre. Un algorithme largement utilisé dans ce contexte est la méthode de Gauss-Newton (GN), connue dans la communauté d'assimilation de données sous le nom d'assimilation variationnelle des données quadridimensionnelles. Le procédé GN repose sur la résolution approchée d'une séquence de moindres carrés linéaires optimale dans laquelle la fonction coût non-linéaire des moindres carrés est approximée par une fonction quadratique dans le voisinage de l'itération non linéaire en cours. Cependant, il est bien connu que cette simple variante de l'algorithme de Gauss-Newton ne garantit pas une diminution monotone de la fonction coût et sa convergence n'est donc pas garantie. Cette difficulté est généralement surmontée en utilisant une recherche linéaire (Dennis and Schnabel, 1983) ou une méthode de région de confiance (Conn, Gould and Toint, 2000), qui assure la convergence globale des points critiques du premier ordre sous des hypothèses faibles. Nous considérons la seconde de ces approches dans cette thèse. En outre, compte tenu de la grandeéchelle de ce problème, nous proposons ici d'utiliser un algorithme de région de confiance particulier s'appuyant sur la méthode du gradient conjugué tronqué de Steihaug-Toint pour la résolution approchée du sous-problème (Conn, Gould and Toint, 2000, p. 133-139) La résolution de ce sous-problème dans un espaceà n dimensions (par CG ou Lanczos) est considérée comme l'approche primale. Comme alternative, une réduction significative du coût de calcul est possible en réécrivant l'approximation quadratique dans l'espaceà m dimensions associé aux observations. Ceci est important pour les applicationsà grandeéchelle telles que celles quotidiennement traitées dans les systèmes de prévisions météorologiques. Cette approche, qui effectue la minimisation de l'espaceà m dimensionsà l'aide CG ou de ces variantes, est considérée comme l'approche duale.

View PDFchevron_right
Observation targeting with a second-order adjoint method for increased predictability
Humberto C Godinez

Computational Geosciences, 2011

The efficiency of current adjoint-based observations targeting strategies in variational data assimilation is closely determined by the underlying assumption of a linear propagation of initial condition errors into the model forecasts. A novel targeting strategy is proposed in the context of four-dimensional variational data assimilation (4D-Var) to account for nonlinear error growth as the forecast lead time increases. A quadratic error growth model is shown to maintain the accuracy in tracking the nonlinear evolution of initial condition perturbations, as compared to the first-order approximation. A second-order adjoint model is used to provide the derivative information that is necessary in the higher-order Taylor series approximation. The observation targeting approach relies on the dominant eigenvectors of the Hessian matrix associated with a specific forecast error aspect as an indicator of the directions of largest quadratic error growth. A compar-ative qualitative analysis between observation targeting based on first-and second-order adjoint information is presented in idealized 4D-Var experiments with a twodimensional global shallow-water model. The results indicate that accounting for the quadratic error growth in the targeting strategy is of particular benefit as the forecast lead time increases.

View PDFchevron_right

References (110)

  1. Abate,J., C. Bischof, A. Carle, and L. Roh,1997:Algorithms and Design for a Second-Order Automatic Differentiation Module.in Proc. Int. Symposium on Sym- bolic and Algebraic Computing (ISSAC) '97,Association of Computing Machinery, New York,149-155.
  2. Alekseev, K. A. and Navon I. M., 2001a: The Analysis of an Ill-Posed Problem Using Multiscale Resolution and Second Order Adjoint Techniques. Computer Methods in Applied Mechanics and Engineering, 190, 15-17 ,1937-1953 .
  3. Alekseev, K.A. and Navon I.M., 2001b:On Estimation of Temperature Un- certainty Using the Order Adjoint Problem. Accepted for publication in International Journal of Computational Fluid Dynamics, Available from http://www.csit.fsu.edu/ navon/pubs/.
  4. Arian,E. and S. Ta'asan,1999:Analysis of the Hessian for aerodynamic optimization: inviscid flow.Computers & Fluids,28: (7), 853-877.
  5. Averbukh V.Z., Figueroa S. and Schlick T.,1994: Remark on Algorithm-566. ACM Trans. Math Software 20: (3) 282-285 .
  6. Barkmeijer, J., Buizza R. and Palmer T.N.,1999: 3D-Var Hessian singular vectors and their potential use in the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc.125, Part B 2333-2351. .
  7. Barkmeijer, J., Van Gijzen, M. and F. Bouttier ,1998: Singular vectors and estimates of the analysis-error covariance metric. Quart. J. Roy. Meteor. Soc. 124, Part A, 1695-1713 .
  8. Bischof, C. H.,1995: Automatic Differentiation, Tangent Linear Models, and (Pseudo)Adjoints.in Proceedings of the Workshop on High-Performance Computing in the Geosciences, Francois-Xavier Le Dimet, ed.,NATO Advanced Science Insti- tutes Series : C, Mathematical and Physical Sciences , 462, Kluwer Academic Pub- lishers B.V.
  9. Burger,J., J. L. Brizaut and M. Pogu, 1992: Comparison of two methods for the calculation of the gradient and of the Hessian of the cost functions associated with differential systems. Mathematics and Computers in Simulation, 34, 551-562.
  10. Bus, J. C. P., 1977: Convergence of Newton-like methods for solving systems of nonlinear equations. Numerische Mathematik, 27, 271-281.
  11. Coleman, T. F. and Cai J. Y.,1986: The Cyclic Coloring Problem and Estimation of Sparse Hessian Matrices .SIAM J. Algebra Discr. 7 (2), 221-235 .
  12. Coleman, T.F., B.S. Garbow and J.J. More ,1985: Software for estimating Sparse Hessian Matrices. ACM Trans Math Software 11: (4), 363-377 .
  13. Coleman, T.F., B.S. Garbow and J.J. More ,1985a: Fortran Subroutines for estimat- ing Sparse Hessian Matrices. ACM Trans Math Software 11: (4), 378-378.
  14. Coleman TF, More J.J.,1984: Estimation of sparse Hessian Matrices and Graph- coloring Problems. Math Program,28: (3), 243-270 .
  15. Courtier, P. and O. Talagrand, 1987: Variational assimilation of meteorological obser- vations with the adjoint equations Part 2. Numerical results. Quart. J. Roy. Meteor. Soc., 113, 1329-1347.
  16. Courtier, P., J.-N. Thepaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4D-Var, using an incremental approach. Quart J. Roy. Meteor. Soc., 120, 1367-1388.
  17. Courtier, P.,1993: Introduction to Numerical Weather prediction Data Assimilation Methods. Proc. ECMWF Seminar on Developments in the use of Satellite Data In Numerical Weather Prediction. 6-10 September 1993.
  18. Davidon, W. C., 1959: Variable metric method for minimization. A. E. C. Research and Development Report, ANL-5990 (Rev.).
  19. Davidon, W. C., 1991: Variable metric method for minimization. SIAM Journal on Optimization,1,1-17.
  20. Davidson E.R.,1975: The iterative calculation of a few of the lowest eigenvalues and corresponding eigenvectors of a large real symmetric matrices. J. Comput. Phys., 17, 87-94.
  21. Dembo, R. S., S. C. Eisenstat and T. Steihaug, 1982: Inexact Newton methods. SIAM Journal of Numerical analysis, 19, 400-408.
  22. Dembo, R. S. and T. Steihaug, 1983: Truncated-Newton algorithms for large-scale unconstrained optimization. Math. Prog., 26, 190-212.
  23. Dennis, J. and R. Schnabel, 1983: Numerical methods for unconstrained Optimization and nonlinear equations. EngleWood Cliffs, NJ: Prentice-Hall, 378pp.
  24. Dennis, J., 1994: Personal communication. Dept. of Math., Rice University, Houston, TX.
  25. Dixon, L. C. W., 1991: Use of automatic differentiation for calculating Hessians and Newton steps, in Automatic differentiation of algorithms: theory, implementation, and application, Andreas Griewank and George F. Corliss (ed.), SIAM, Philadelphia, 115-125.
  26. Ehrendorfer, M. and F. Bouttier,1998: An explicit low-resolution extended Kalman filter: Implementation and preliminary experimentation. Sept 1998. ECMWF Tech- nical Memorandum No 259.
  27. Errico, Ronald M., Tomislava Vukićević, and Kevin Raeder, 1993: Examination of the accuracy of a tangent linear model. Tellus, 45A, 462-477.
  28. Fisher, M. and Courtier, P., 1995:Estimating the covariance matrices of analysis and forecast errors in variational data assimilation.ECMWF Tech Memorandum, No. 220, 28pp.
  29. Fisher, M.,1998: Development of a simplified Kalman filter.ECMWF Tech Memo- randum, No. 260,16pp.
  30. Forsythe ,G. E. and E. G. Strauss,1955: On best conditioned matrices.Proc. Amer. Math. Soc., 6, 340-345.
  31. Gay , D.M.,1996: More AD of Nonlinear AMPL Models: Computing Hessian Infor- mation and Exploiting Partial Separability in Computational Differentiation: Tech- niques, Applications, and Tools Martin Berz, Christian Bischof, George Corliss, and Andreas Griewank, Editors, Proceedings in Applied Mathematics 89 .SIAM 1996 / xvi + 421 pages.
  32. Giering, R. and T. Kaminski ,1998a: Recipes for adjoint code construction ACM Trans Math Software,24: (4) ,437-474.
  33. Giering, R. and T. Kaminski ,1998b:Using TAMC to generate efficient adjoint code: Comparison of automatically generated code for evaluation of first and second or- der derivatives to hand written code from the Minpack-2 collection.Extended ab- stracts of the Session Automatic Differentiation for adjoint code generation at the IMAC Conference on Applications of Computer Algebra, Prag, 1998. Available at:http://puddle.mit.edu/ ralf/tamc/minpack/gierkam/node1.html.
  34. Gilbert, J. C., 1992: Automatic differentiation and iterative processes. Optimization Methods and Software, 1, 13-21.
  35. Gill, P. E. and W. Murray, 1972: Quasi-Newton methods for unconstrained opti- mization. J. Inst. Maths Applics, 9, 91-108.
  36. Gill, P. E. and W. Murray, 1974: Numerical methods for unconstrained optimization. Academic Press, London and New-York, 283pp.
  37. Gill, P. E. and W. Murray, 1979: Newton-type methods for unconstrained and lin- early constrained optimization. Math. Prog., 28, 311-350.
  38. Gill P. E., Walter Murray and Margaret H. Wright, 1981 : Practical Optimization. Academic Press, London, 401pp.
  39. Gill, P. E., W. Murray, M. C. Saunders and M. H. Wright, 1983: Computing forward- difference intervals for numerical optimization. SIAM J. Sci. Stat. Comput., 4, 310- 321.
  40. Grammeltvedt, A., 1969: A survey of finite-difference schemes for the primitive equa- tions for a barotropic fluid. Mon. Wea. Rev., 97, 387-404.
  41. Griewank, A. and George F. Corliss., 1991: Automatic differentiation of algorithms: theory, implementation, and application, SIAM, Philadelphia, 353pp.
  42. Griewank, A.,1993: Some bounds on the complexity of gradients, Jacobians, and Hes- sians,Complexity in Nonlinear Optimization (P.M. Pardalos, ed.), World Scientific Publishers, , pp. 128-161.
  43. Griewank, A.,2000: Evaluating Derivatives: Principles and Techniques of Algo- rithmic Differentiation. Frontiers in Applied Mathematics 19, SIAM , Philadelphia, 369pp.
  44. Griewank, A.,2001: Complexity of gradients, Jacobians and Hessians,SA. In Encyclopaedia of Optimization, edited by Floudas and Pardalos. Published by Kluwer Academic Publishers. Kluwer Academic Publishers. Dorderecht, The Nether- lands,3200pp.
  45. Hamming, R. W., 1973: Numerical methods for scientists and engineers, 2nd edition, McGraw-Hill and New York.
  46. Hovland ,P.,1995: Using ADIFOR 1.0 to Compute Hessians, CRPC-TR95540- S,Center for Research on Parallel Computation, Rice University, Houston, TX, 12 pp.
  47. Ide, K., P. Courtier, M. Ghil, and A. Lorenc, 1997: Unified notation for data assim- ilation: Operational sequential and variational. In Data Assimilation, Meteorology and Oceanography: Theory and Practice, J. Meteorol. Soc Japan, 75, 1B, 71-79.
  48. Hou, G J.-W. and J. Sheen, 1993: Numerical methods for second order shape sensi- tivity analysis with application to heat conduction problems. Int. J. Num. Meth. in Engr., 36, 417-435.
  49. Jackson R.H.F. and G.P. McCormic: Second order sensitivity analysis in factorable programming: Theory and Applications.Mathematical Programming, 41, No. 1, 1-28.
  50. Kalnay E.,S. K. Park, Z-X. Pu and J Gao, 2000:Application of the quasi-inverse method to data assimilation. Mon. Wea. Rev., 128, 864-875.
  51. Lacarra, J. F. and O. Talagrand, 1988: Short-range evolution of small perturbations in a barotropic model. Tellus, 40A, 81-95.
  52. Le Dimet, F. X. and O. Talagrand, 1986: Variational algorithms for analysis and assimilation of meteorological observations: Theoretical aspects. Tellus, 38A,97-110.
  53. Le Dimet F. X., H. E. Ngodock , B. Luong and J. Verron, 1997: Sensitivity analysis in variational data assimilation. J Meteorol Soc Japan, 75: (1B), 245-255.
  54. Le Dimet , F. X.and I Charpentier,1998: Methodes de second order en assimilation de donnees.In Equations aux dérivées partielles et applications (articles dédiées à Jacques-Louis Lions) , Gauthier-Villars, Paris 623-640.808pp.
  55. Lehoucq R.B., D.C. Sorensen and C. Yang, 1998: Arpack Users' Guide: Solution of Large-scale eigenvalue Problems with Implicitly Restarted Arnoldi Methods. Soft- ware, Environments, and Tools 6 ,SIAM, Philadelphia, 160pp.
  56. Lewis, R.M.,1997:A Nonlinear Programming perspective on sensitivity calculations for systems governed by state equations.ICASE Report,No. 97-12,35pp.
  57. Liu, D. C and Jorge Nocedal, 1989: On the limited memory BFGS method for large scale minimization. Math. Prog., 45, 503-528.
  58. Moré, J.J., B. S. Garbow and K.E. Hillstrom, 1981:Testing Unconstrained Optimiza- tion Software. ACM Transactions on Mathematical Software, 7, 17-41.
  59. Nash, S. G., 1982: Truncated-Newton methods. Ph.D. Thesis, Computer Science Department, Stanford University, CA.
  60. Nash, S. G., 1984: Newton-type minimization via the Lanczos method. SIAM J. Numer. Anal., 21 (4), 770-788.
  61. Nash, S. G., 1984: Truncated-Newton methods for large-scale function minimization. Applications of Nonlinear Programming to Optimization and Control, H.E. Rauch (ed.), Pergamon Press, Oxford, 91-100.
  62. Nash, S. G., 1984: User's guide for TN/TNBC: Fortran routines for nonlinear opti- mization. Tech. Rep. No., 397, 17pp.
  63. Nash, S. G., 1984: Solving nonlinear programming problems using truncated-Newton techniques. Numerical optimization, P. T. Boggs, R. H. Byrd and R. B. Schnabel (ed.), SIAM, Philadelphia, 119-136.
  64. Nash, S. G., 1985: Preconditioning of truncated-Newton methods. SIAM J. Sci. Stat. Comput., 6 (3), 599-616.
  65. Nash, S. G. and A. Sofer, 1989: Block truncated-Newton methods for parallel opti- mization. Math. Prog., 45, 529-546.
  66. Nash, S. G. and A. Sofer, 1989: A parallel line search for Newton type methods in computer science and statistics. Proceeding 21-st Symposium on the Interface, K. Berk and L. Malone (ed.), ASA, 134-137.
  67. Nash, S.G. and J. Nocedal,1991: A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization,SIAM J. Optimization,1, pp. 358-372.
  68. Nash, S. G. and A. Sofer, 1990: Assessing a search direction within a truncated- Newton method. Operations Research Letters, 9 (4), 219-221.
  69. Nash S.G.,2000: A Survey of Truncated-Newton Methods,Journal of Computational and Applied Mathematics, 124, 45-59.
  70. Nash, S. G. and A. Sofer, 1996: Linear and Nonlinear Programming, McGraw-Hill (New York),ISBN 0-07-046065-5.
  71. Navon, I. M. and D. M. Legler, 1987: Conjugate gradient methods for large scale minimization in meteorology. Mon. Wea. Rev., 115, 1479-1502. conjugate gradient method
  72. Navon, I. M. , X. L. Zou, J. Derber and J. Sela, 1992: Variational data assimilation with an adiabatic version of the NMC Spectral Model. Mon. Wea. Rev.,122, 1433- 1446.
  73. Ngodock, H. E.,1996:Data Assimilation and sensitivity Analysis.Ph. D. Thesis, LMC- IMAG Laboratory, University Joseph Fourier, Grenoble, France, 213pp.
  74. Nocedal, J., 1980: Updating quasi-Newton matrices with limited storage. Mathemat- ics of Computation, 35, 773-782.
  75. Nocedal,J. and S.J. Wright,1999: Numerical Optimization.(Springer Series in Oper- ations Research. Ed.: P. Glynn) Springer Verlag Series in Operations Research,656 pp., 85 illus.
  76. O'Leary, D. P., 1983: A discrete Newton algorithm for minimizing a function of many variables. Math. Prog., 23, 20-23.
  77. Park, S.K., and E. Kalnay, 1999: Application of the quasi-inverse method to storm- scale data assimilation. Abstracts, 5th SIAM Conference on Mathematical and Com- putational Issues in the Geosciences, March 24-27, 1999, San Antonio, Texas, SIAM, 104.
  78. Powell M.J.D., Toint P.L.,1979: Estimation of sparse Hessian matrices. SIAM J. Numer. Anal.,16: (6), 1060-1074.
  79. Pu,Z.X. and E. Kalnay,1999: Targeting observations with the quasi-inverse linear and adjoint NCEP global models: Performance during FASTEX.Quart. J. Roy. Meteor. Soc., 125 No. 561,pp 3329-3337
  80. Pu Z.X., E. Kalnay , J. Sela , et al.,1997: Sensitivity of forecast errors to initial conditions with a quasi-inverse linear method Mon. Wea. Rev ,125: (10), 2479- 2503.
  81. Rabier, F. and P. Courtier , 1992:Four dimensional assimilation in the presence of baroclinic instability.Quart. J. Roy. Meterol. Soc., 118, 649-672.
  82. Rabier, F., J-F. Mahfouf, M. Fisher, H. Jarvinen, A. Simmons, E. Andersson, F. Bouttier, P. Courtier, M. Hamrud, J. Haseler, A. Hollingsworth, L. Isaksen, E. Klinker, S. Saarinen, C.Temperton, J-N. Thepaut, P. Unden, and D. Vasiljevic,1997: Recent experimentation on 4D-Var and first results from a Simplified Kalman Filter. October 1997.ECMWF Technical Memorandum, No. 240.
  83. Reuther, J. J.,1996: Aerodynamic Shape Optimization Using Control Theory.Ph.D. Dissertation, University of California, Davis, 1996.Available from University of Cal- ifornia, Davis.
  84. Santosa, Fadil and William W. Symes, 1988: Computation of the Hessian for least- squares solutions of inverse problems of reflection seismology. Inverse Problems, 4, 211-233.
  85. Santosa, Fadil and William W. Symes, 1989: An analysis of least squares velocity inversion. Society of Exploration Geophysicists, Geophysical monograph 4, Tulsa.
  86. Schlick, T. and A. Fogelson, 1992: TNPACK-A truncated Newton minimization Package for large-scale problems: I. Algorithm and usage. ACM Trans on Math. Soft., 18 (1), 46-70.
  87. Schlick, T. and A. Fogelson, 1992: TNPACK-A Truncated Newton minimization package for large-scale problems: II. Implementation examples. ACM Trans Math. Soft., 18 (1), 71-111.
  88. Shanno, D. F. and K. H. Phua, 1980: Remark on algorithm 500 -a variable method subroutine for unconstrained nonlinear minimization. ACM Trans. on Math. Soft., 6, 618-622.
  89. Sleijpen G.L.G., H.A. Van der Vorst ,1996: A Jacobi-Davidson iteration method for linear eigenvalue problems. SIAM J Matrix Anal A 17: (2), 401-425 .
  90. Stewart III, G. W., 1967: A modification of Davidon's minimization method to accept difference approximations of derivatives. Journal of the Association for Computing Machinery, 14 (1), 72-83.
  91. Symes, William W., 1990:Velocity inversion: A case study in infinite-dimensional optimization. Math. Programming, 48, 71-102.
  92. Symes, William W., 1991: A differential semblance algorithm for the inverse problem of reflection seismology. Computers Math. Applic., 22(4/5), 147-178. Math.,
  93. Symes, William W., 1993:A differential semblance algorithm for the inversion of Multioffset Seismic Reflection Data. Jour. Geophys. Res., 98 B2, 2061-2073.
  94. Talagrand, O. and P. Courtier, 1987: Variational assimilation of meteorological obser- vations with the adjoint vorticity equation-Part 1. Theory. Quart. J. Roy. Meteorol. Soc., 113, 1311-1328.
  95. Thacker, W. C.,1989: The Role of Hessian Matrix in Fitting models to Measurements. J. Geophys. Res., 94, 6177-6196.
  96. Thepaut , J. N. and Courtier., Philip,1991: Four-dimensional variational assimilation using the adjoint of a multilevel primitive equation model. Quart. J. Roy. Meteorol. Soc., 117, 1225-1254. assimilation
  97. Thepaut, J.-N. and Moll, P.,1990: Variational inversion of simulated TOVS radiances using the adjoint technique.Quart. J. Roy. Meteorol. Soc., 116, 1425-1448.
  98. Thomas, R. Cuthbert, Jr., 1987: Optimization using personal computers. John Wiley & Sons, New York, pp. 474.
  99. Veerse F.,1999: Variable-storage quasi-Newton operators as inverse forecast/analysis error covariance matrices in data assimilation. INRIA Technical Report,No. 3685 , Theme 4,28pp.
  100. Wang, Z., I. M. Navon, F. X. Le Dimet and X. Zou, 1992: The Second Order Adjoint Analysis: Theory and Application. Meteorology and Atmospheric Physics, 50, 3-20.
  101. Wang, Z., I. M. Navon and X. Zou, 1993: The adjoint truncated Newton algorithm for large-scale unconstrained optimization. Tech. Rep. FSU-SCRI-92-170, Florida State University, Tallahassee, Florida, 44 pp.
  102. Wang, Z., 1993: Variational data assimilation with 2-D shallow water equations and 3-D FSU global spectral models. Tech. Rep. FSU-SCRI-93T-149, Florida State University, Tallahassee, Florida, 235 pp.
  103. Wang, Z., I. M. Navon, X. Zou and F.X. Le Dimet,1995: A Truncated -Newton Optimization Algorithm in Meteorology Applications with Analytic Hessian/vector Products, Computational Optimization and Applications,4 , 241-262.
  104. Wang, Z., Kelvin K. Droegemeier, L. White and I. M. Navon ,1997: Application of a New Adjoint Newton Algorithm to the 3-D ARPS Storm Scale Model Using Simulated Data, Monthly Weather Review,125, No 10, 2460-2478.
  105. Wang, Z. , Kelvin K. Droegemeier and L. White, 1998 :The Adjoint New- ton algorithm for large-scale unconstrained optimization in meteorology applica- tions.Computational Optimization and Applications,10 , 283-320.
  106. Zou, X., I. M. Navon, F. X. Le Dimet, A. Nouailler and T. Schlick, 1990: A comparison of efficient large-scale minimization algorithms for optimal control ap- plications in meteorology. Tech. Rep. FSU-SCRI-90-167, Florida State University, Tallahassee, Florida, 44pp.
  107. Zou, X., I. M. Navon, M. Berger, Paul K. H. Phua, T. Schlick and F. X. Le Dimet, 1993: Numerical experience with limited-Memory Quasi-Newton methods and Trun- cated Newton methods. SIAM Jour. on Numerical Optimization, 3, 582-608.
  108. Zou, X., I. M. Navon and F. X. Le Dimet, 1992: Incomplete observations and control of gravity waves in variational data assimilation. Tellus, 44A, 273-296.
  109. Zupanski, M. 1996 : A preconditioning algorithm for four-dimensional variational data assimilation. Mon. Wea Rev., 124: (11) ,2562-2573.
  110. Yang, W., I. M. Navon and P. Courtier,1996: A New Hessian Preconditioning Method Applied to Variational Data Assimilation Experiments using Adiabatic Ver- sion of NASA/GEOS-1 GCM . Monthly Weather Review, 124, No 5 1000-1017.

Related papers

The Second Order Adjoint Analysis: Theory and Application
Ionel M Navon

1992

The adjoint method application in variational data assimilation provides a way of obtaining the exact gradient of the cost function J with respect to the control variables. Additional information may be obtained by using second order information. This paper presents a second order adjoint model (SOA) for a shallow-water equation model on a limited-area domain. One integration of such a model yields a value of the Hessian (the matrix of second partial derivatives, VzJ) multiplied by a vector or a column of the Hessian of the cost function with respect to the initial conditions. The SOA model was then used to conduct a sensitivity analysis of the cost function with respect to distributed observations and to study the evolution of the condition number (the ratio of the largest to smallest eigenvalues) of the Hessian during the course of the minimization. The condition number is strongly related to the convergence rate of the minimization. It is proved that the Hessian is positive definite during the process of the minimization, which in turn proves the uniqueness of the optimal solution for the test problem.

View PDFchevron_right
The second order adjoint analysis: Theory and applications
Ionel M Navon

Meteorology and Atmospheric Physics, 1992

The adjoint method application in variational data assimilation provides a way of obtaining the exact gradient of the cost function J with respect to the control variables. Additional information may be obtained by using second order information. This paper presents a second order adjoint model (SOA) for a shallow-water equation model on a limited-area domain. One integration of such a model yields a value of the Hessian (the matrix of second partial derivatives, VzJ) multiplied by a vector or a column of the Hessian of the cost function with respect to the initial conditions. The SOA model was then used to conduct a sensitivity analysis of the cost function with respect to distributed observations and to study the evolution of the condition number (the ratio of the largest to smallest eigenvalues) of the Hessian during the course of the minimization. The condition number is strongly related to the convergence rate of the minimization. It is proved that the Hessian is positive definite during the process of the minimization, which in turn proves the uniqueness of the optimal solution for the test problem.

View PDFchevron_right
Second-order adjoints for solving PDE-constrained optimization problems
Alexandru Alex

Optimization Methods & Software, 2012

Inverse problems are of the utmost importance in many fields of science and engineering. In the variational approach, inverse problems are formulated as partial differential equation-constrained optimization problems, where the optimal estimate of the uncertain parameters is the minimizer of a certain cost functional subject to the constraints posed by the model equations. The numerical solution of such optimization problems requires the computation of derivatives of the model output with respect to model parameters. The first-order derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through first-order adjoint (FOA) sensitivity analysis. Second-order adjoint (SOA) models give second derivative information in the form of matrix–vector products between the Hessian of the cost functional and user-defined vectors. Traditionally, the construction of second-order derivatives for large-scale models has been considered too costly. Consequently, data assimilation applications employ optimization algorithms that use only first-order derivative information, such as nonlinear conjugate gradients and quasi-Newton methods.In this paper, we discuss the mathematical foundations of SOA sensitivity analysis and show that it provides an efficient approach to obtain Hessian-vector products. We study the benefits of using second-order information in the numerical optimization process for data assimilation applications. The numerical studies are performed in a twin experiment setting with a two-dimensional shallow water model. Different scenarios are considered with different discretization approaches, observation sets, and noise levels. Optimization algorithms that employ second-order derivatives are tested against widely used methods that require only first-order derivatives. Conclusions are drawn regarding the potential benefits and the limitations of using high-order information in large-scale data assimilation problems.

View PDFchevron_right
order adjoints for solving PDE-constrained optimization problems. Optim. Methods Softw. 2011, submitted for publication
Alexandru Cioaca

2020

Inverse problems are of utmost importance in many fields of science and engineering. In the variational approach inverse problems are formulated as PDE-constrained optimization problems, where the optimal estimate of the uncertain parameters is the minimizer of a certain cost functional subject to the constraints posed by the model equations. The numerical solution of such optimization problems requires the computation of derivatives of the model output with respect to model parameters. The first order derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through first order adjoint sensitivity analysis. Second order adjoint models give second derivative information in the form of matrix-vector products between the Hessian of the cost functional and user defined vectors. Traditionally, the construction of second order derivatives for large scale models has been considered too costly. Consequent...

View PDFchevron_right
Intercomparison of the primal and dual formulations of variational data assimilation
Simon Pellerin

Quarterly Journal of the Royal Meteorological Society, 2008

Two approaches can be used to solve the variational data assimilation problem. The primal form corresponds to the 3D/4D-Var used now in many operational NWP centres. An alternative approach, called dual or 3D/4D-PSAS, consists in solving the problem in the dual of observation space. Both forms use the same basic operators so that once one method is developed, it should be possible to obtain the other easily provided these operators have a modular form. It has been shown that, with proper conditioning of the minimization problem, the two algorithms should have similar convergence rates and computational performances. In the presence of nonlinearities, the incremental form of 3D/4D-Var extends the equivalence to the so-called 3D/4D-PSAS. The first objective of this paper is to present results obtained with the variational data assimilation of the Meteorological Service of Canada to show the equivalence between the 3D-Var and the PSAS systems. This exercise has forced us to have a close look at the modularity of the operational 3D/4D-Var which then makes it possible to obtain the 3D-PSAS scheme. This paper then focuses on these two quadratic problems that show similar convergence rates. However, the minimization of 3D-PSAS is examined more thoroughly as some parameters are shown to be determining elements in the minimization process. Lastly, preconditioning properties are studied and the Hessians of the two problems are shown to be directly related to one another through their singular vectors, which makes it possible to cycle the Hessian of the PSAS form.

View PDFchevron_right

Related topics

  • Computational Complexity
  • Data Assimilation
  • Atmospheric sciences
  • First-Order Logic
  • Automatic differentiation
  • Variational Data Assimilation

    • Cited by

    Second-order adjoints for solving PDE-constrained optimization problems
    Alexandru Alex

    Optimization Methods & Software, 2012

    Inverse problems are of the utmost importance in many fields of science and engineering. In the variational approach, inverse problems are formulated as partial differential equation-constrained optimization problems, where the optimal estimate of the uncertain parameters is the minimizer of a certain cost functional subject to the constraints posed by the model equations. The numerical solution of such optimization problems requires the computation of derivatives of the model output with respect to model parameters. The first-order derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through first-order adjoint (FOA) sensitivity analysis. Second-order adjoint (SOA) models give second derivative information in the form of matrix–vector products between the Hessian of the cost functional and user-defined vectors. Traditionally, the construction of second-order derivatives for large-scale models has been considered too costly. Consequently, data assimilation applications employ optimization algorithms that use only first-order derivative information, such as nonlinear conjugate gradients and quasi-Newton methods.In this paper, we discuss the mathematical foundations of SOA sensitivity analysis and show that it provides an efficient approach to obtain Hessian-vector products. We study the benefits of using second-order information in the numerical optimization process for data assimilation applications. The numerical studies are performed in a twin experiment setting with a two-dimensional shallow water model. Different scenarios are considered with different discretization approaches, observation sets, and noise levels. Optimization algorithms that employ second-order derivatives are tested against widely used methods that require only first-order derivatives. Conclusions are drawn regarding the potential benefits and the limitations of using high-order information in large-scale data assimilation problems.

    View PDFchevron_right
    Automatic Differentiation of the General-Purpose Computational Fluid Dynamics Package FLUENT
    Arno Rasch, Bruno Lang

    Journal of Fluids Engineering, 2007

    Derivatives are a crucial ingredient to a broad variety of computational techniques in science and engineering. While numerical approaches for evaluating derivatives suffer from truncation error, automatic differentiation is accurate up to machine precision. The term automatic differentiation comprises a set of techniques for mechanically transforming a given computer program to another one capable of evaluating derivatives. A common misconception about automatic differentiation is that this technique only works on local pieces of fairly simple code. Here, it is shown that automatic differentiation is not only applicable to small academic codes, but scales to advanced industrial software packages. In particular, the general-purpose computational fluid dynamics software package FLUENT is transformed by automatic differentiation.

    View PDFchevron_right
    Solving ill-posed Image Processing problems using Data Assimilation
    Isabelle HERLIN

    Numerical Algorithms, 2011

    L'Assimilation de Données est un cadre méthodologique utilisé en sciences de l'environnement pour la prédiction des systèmes complexes, que sont les modèles météorologiques, océanographiques, ou encore de qualité de l'air. L'Assimilation de Données opère en résolvant un systèmeà trois composantes: une premièreéquation décrit l'évolution temporelle du vecteur d'état; une secondeéquation décrit la relation entre le vecteur d'état et l'observation; enfin, une dernièreéquation décrit la condition initiale. Dans ce rapport, nous utilisons ce cadre mathématique pour l'étude des problèmes mal posés du traitement d'image, habituellement résolus par des techniques de régularisation. Pour cela, le problème est formulé au moyen du systèmeà trois composantes de l'Assimilation de Données. Comme illustration, cette approche est appliquée au calcul du flot optique.

    View PDFchevron_right
    Observation targeting with a second-order adjoint method for increased predictability
    Humberto C Godinez

    Computational Geosciences, 2011

    The efficiency of current adjoint-based observations targeting strategies in variational data assimilation is closely determined by the underlying assumption of a linear propagation of initial condition errors into the model forecasts. A novel targeting strategy is proposed in the context of four-dimensional variational data assimilation (4D-Var) to account for nonlinear error growth as the forecast lead time increases. A quadratic error growth model is shown to maintain the accuracy in tracking the nonlinear evolution of initial condition perturbations, as compared to the first-order approximation. A second-order adjoint model is used to provide the derivative information that is necessary in the higher-order Taylor series approximation. The observation targeting approach relies on the dominant eigenvectors of the Hessian matrix associated with a specific forecast error aspect as an indicator of the directions of largest quadratic error growth. A compar-ative qualitative analysis between observation targeting based on first-and second-order adjoint information is presented in idealized 4D-Var experiments with a twodimensional global shallow-water model. The results indicate that accounting for the quadratic error growth in the targeting strategy is of particular benefit as the forecast lead time increases.

    View PDFchevron_right
    A Second Order Adjoint Method to Targeted Observations
    Humberto C Godinez

    Lecture Notes in Computer Science, 2009

    The role of the second order adjoint in targeting strategies is studied and analyzed. Most targeting strategies use the first order adjoint to identify regions where additional information is of potential benefit to a data assimilation system. The first order adjoint posses a restriction on the targeting time for which the linear approximation accurately tracks the evolution of perturbation. Using second order adjoint information it is possible to maintain some accuracy for longer time intervals, which can lead to an increase on the target time. We propose the use of the dominant eigenvectors of the Hessian matrix as an indicator of the directions of maximal error growth for a given targeting functional. These vectors are a natural choice to be included in the targeting strategies given their mathematical properties.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved