Juarez Valencia ROJAS 2022a - Revision history

Gstinoco at 22:22, 24 December 2022

2022-12-24T22:22:44Z

Beto1506: Beto1506 moved page Review 590035020140 to Juarez Valencia ROJAS 2022a

2022-11-26T15:37:52Z

Beto1506 moved page Review 590035020140 to Juarez Valencia ROJAS 2022a

Hect at 20:52, 23 November 2022

2022-11-23T20:52:37Z

Hect at 20:32, 23 November 2022

2022-11-23T20:32:55Z

Hect at 20:12, 23 November 2022

2022-11-23T20:12:22Z

Hect at 20:10, 23 November 2022

2022-11-23T20:10:00Z

Hect at 20:09, 23 November 2022

2022-11-23T20:09:02Z

Hect at 20:05, 23 November 2022

2022-11-23T20:05:53Z

Hect at 19:58, 23 November 2022

2022-11-23T19:58:43Z

Hect at 19:53, 23 November 2022

2022-11-23T19:53:38Z

@@ Line 870: / Line 870: @@
 <div id="cite-1"></div>
-'''[[#citeF-1|[1]]]'''  I. K. Argyros, M. A. Hernández-Verón, M. J. Rubio, M. J., On the Convergence of Secant-Like Methods, Current Trends in Mathematical Analysis and Its Interdisciplinary Applications (2019) 141&#8211;183.
+'''[[#citeF-1|[1]]]'''  I. K. Argyros, M. A. Hernández-Verón, M. J. Rubio, M. J., ''On the Convergence of Secant-Like Methods'', in Current Trends in Mathematical Analysis and Its Interdisciplinary Applications (2019) 141&#8211;183.
 <div id="cite-2"></div>
@@ Line 885: / Line 885: @@
 <div id="cite-6"></div>
-'''[[#citeF-6|[6]]]'''  Calderhead, B., Girolami, M. ''Estimating Bayes factors via thermodynamic integra- tion and population'' MCMC. Comput. Stat. Data Anal. 53 (12), (2009), 4028&#8211;4045.
+'''[[#citeF-6|[6]]]'''  Calderhead, B., Girolami, M. ''Estimating Bayes factors via thermodynamic integration and population MCMC'' Comput. Stat. Data Anal. 53 (12), (2009), 4028&#8211;4045.
 <div id="cite-7"></div>
@@ Line 906: / Line 906: @@
 <div id="cite-13"></div>
-'''[[#citeF-13|[13]]]'''  Ramsay, J., Hooker, H., Campbell, D., Cao, J.  Parameter estimation for differential equations: a generalized smoothing approach. J. R. Stat. Soc. Ser. B 69 (5), (2007) 741&#8211;796.
+'''[[#citeF-13|[13]]]'''  Ramsay, J., Hooker, H., Campbell, D., Cao, J.  ''Parameter estimation for differential equations: a generalized smoothing approach''. J. R. Stat. Soc. Ser. B 69 (5), (2007) 741&#8211;796.
 <div id="cite-14"></div>

@@ Line 765: / Line 765: @@
 |- style="border-top: 2px solid;"
 | style="border-right: 2px solid;" | <math display="inline">noise\_level</math>
-| style="border-left: 2px solid;border-right: 2px solid;" | <math>0.05 (5%)</math>
+| style="border-left: 2px solid;border-right: 2px solid;" | <math>0.05\ (5%)</math>
-| style="border-left: 2px solid;" | <math>0.15 (15%)</math>
+| style="border-left: 2px solid;" | <math>0.15\ (15%)</math>
 |- style="border-top: 2px solid;"
 | style="border-right: 2px solid;" | <math display="inline">\boldsymbol{\theta }^0</math>

@@ Line 652: / Line 652: @@
 |-
 | style="border-right: 2px solid;" | <math>\epsilon </math>, no. iters.
-| style="border-left: 2px solid;border-right: 2px solid;" | <math>\epsilon = 10^{-8}</math>,168
+| style="border-left: 2px solid;border-right: 2px solid;" | <math>\epsilon = 10^{-8}</math>, 168
 | style="border-left: 2px solid;" | <math>\epsilon = 10^{-8}</math>, 14
 |- style="border-top: 2px solid;"
@@ Line 718: / Line 718: @@
 |-
 | style="border-right: 2px solid;" | <math>\epsilon </math>,  no. iters.
-| style="border-left: 2px solid;border-right: 2px solid;" | <math>10^{-5}</math>,229
+| style="border-left: 2px solid;border-right: 2px solid;" | <math>10^{-5}</math>, 229
 | style="border-left: 2px solid;" | <math>10^{-6}</math>, 41
 |- style="border-top: 2px solid;"
@@ Line 781: / Line 781: @@
 |-
 | style="border-right: 2px solid;" | <math>\epsilon </math>,  no. iters.
-| style="border-left: 2px solid;border-right: 2px solid;" | <math>10^{-8}</math>,30
+| style="border-left: 2px solid;border-right: 2px solid;" | <math>10^{-8}</math>, 30
 | style="border-left: 2px solid;" | <math>10^{-8}</math>, 43
 |- style="border-top: 2px solid;"

@@ Line 7: / Line 7: @@
 ==1 Introduction==
-Systems of ordinary (and partial) differential equations are an important tool to model the physical state of a real phenomenon that arise in many areas of applied sciences and engineering. Predicting the future behaviour or allowing control of these processes requires not only accurately describing the system but also finding or improving its parameters. Estimation of unknown or inaccurate parameters in turn requires fitting partially observed noisy data or experimental measurements to the model. More generally, in the statistics and machine learning literature various methods have been employed to fit differential equations to data, from maximum likelihood approaches, <span id='citeF-13'></span>[[#cite-13|[13]]] to Bayesian sampling algorithms, <span id='citeF-6'></span>[[#cite-6|[6]]] or traditional deterministic approaches. Thus, parameter estimation needs efficient ODE (forward) solver, optimization routines, statistical and possible stochastic procedures. There are several stochastic, deterministic and hybrid optimization routines <span id='citeF-2'></span>[[#cite-2|[2]]]. Contrary to stochastic algorithms, deterministic ones are computationally efficient but they tend to converge to local minima. However, they are the departure point in many applications and in the design of better and efficient procedures. For these methods the gradient is combined with information from line searches or methods involving a Newton, quasi-Newton (low-rank) or Fisher information based curvature estimators to update model parameters, <span id='citeF-9'></span>[[#cite-9|[9]]]. The main computational bottleneck in these algorithms is the computation of the gradient (or the curvature) of the parametric cost function. Then, efficient methods to evaluate gradients or for parametric sensitivity analysis of differential&#8211;algebraic models is important, no only for the determination of parameters, but also in other areas of application like model simplification, data assimilation, optimal control, process sensitivity, uncertainty analysis, and experimental design, among others, for a wide range of scientific and engineering problems.
+Systems of ordinary (and partial) differential equations are an important tool to model the physical state of a real phenomenon that arise in many areas of applied sciences and engineering. Predicting the future behaviour or allowing control of these processes requires not only accurately describing the system but also finding or improving its parameters. Estimation of unknown or inaccurate parameters in turn requires fitting partially observed noisy data or experimental measurements to the model. More generally, in the statistics and machine learning literature various methods have been employed to fit differential equations to data, from maximum likelihood approaches, <span id='citeF-13'></span>[[#cite-13|[13]]] to Bayesian sampling algorithms, <span id='citeF-6'></span>[[#cite-6|[6]]] or traditional deterministic approaches. Thus, parameter estimation needs efficient ODE (forward) solvers, optimization routines, statistical and possible stochastic procedures. There are several stochastic, deterministic and hybrid optimization routines <span id='citeF-2'></span>[[#cite-2|[2]]]. Contrary to stochastic algorithms, deterministic ones are computationally efficient but they tend to converge to local minima. However, they are the departure point in many applications and in the design of better and efficient procedures. For these methods the gradient is combined with information from line searches or methods involving a Newton, quasi-Newton (low-rank) or Fisher information based curvature estimators to update model parameters, <span id='citeF-9'></span>[[#cite-9|[9]]]. The main computational bottleneck in these algorithms is the computation of the gradient (or the curvature) of the parametric cost function. Then, efficient methods to evaluate gradients or for parametric sensitivity analysis of differential&#8211;algebraic models is important, no only for the determination of parameters, but also in other areas of application like model simplification, data assimilation, optimal control, process sensitivity, uncertainty analysis, and experimental design, among others, for a wide range of scientific and engineering problems.
 In this work we concentrate in deterministic optimization procedures, mainly those for quadratic non-linear programming and based on gradient methods and quasi-Newton algorithms. Our purpose is to explain in some detail modelling, algorithmic and computational issues to a wide, and possible non-expert, audience. In Section 2 we introduce the quadratic non-linear model that incorporates noisy data into the cost function and it is adapted to the SEIRD epidemiological deterministic model, which is employed to illustrate the algorithms and their related computational issues. Section 3 is devoted to explaining the adjoint equation method for computing gradients of the given cost function and its advantages over other methods. In section 4 we describe the CG and BFGS optimization algorithms and discuss important computational issues. Numerical results are shown in Section 5 and finally, in Section 6, we give some conclusions and perspectives for future work in order to improve the model and overcome some drawbacks and algorithmic problems.

@@ Line 3: / Line 3: @@
 '''Abstract.''' In this work we discuss a variational approach for the determination of the parameters of systems of ordinary differential equations (ODE). We construct a model for fitting observed noisy data into the given dynamical system. Also we explain in detail the advantage of using the adjoint equation method to compute the derivatives or gradients, which are needed for the application of gradient methods and quasi-Newton algorithms to find the minimum of the cost function. In particular, we consider two classic iterative algorithms: the conjugate gradient (CG) algorithm and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. For educational purposes we try to explain several numerical and computational issues with some detail and illustrate them with the Susceptible, Exposed, Infected, Recovered, and Deceased (SEIRD) epidemiological model.
-'''Keywords.''' Inverse problem; noisy data; parameter determination; adjoint method, variational approach.
+'''Keywords.''' Inverse problem; noisy data; parameter determination; adjoint method; variational approach.
 ==1 Introduction==

← Older revision	Revision as of 15:37, 26 November 2022
(No difference)

@@ Line 1: / Line 1: @@
 ==Parameter estimation in ODEs. Modelling and computational issues==
-'''Abstract.''' In this work we discuss a variational approach for the determination of the parameters of systems of ordinary differential equations (ODE). We construct a model for fitting observed noisy data into the given dynamical system. Also we explain in detail the advantage of using the adjoint equation method to compute the derivatives or gradients, which are needed for the application of gradient methods and quasi-Newton algorithms to find the minimum of the cost function. In particular we consider two classic iterative algorithms: the conjugate gradient (CG) algorithm and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. For educational purposes we try to explain several numerical and computational issues with some detail and illustrate them with the Susceptible, Exposed, Infected, Recovered, and Deceased (SEIRD) epidemiological model.
+'''Abstract.''' In this work we discuss a variational approach for the determination of the parameters of systems of ordinary differential equations (ODE). We construct a model for fitting observed noisy data into the given dynamical system. Also we explain in detail the advantage of using the adjoint equation method to compute the derivatives or gradients, which are needed for the application of gradient methods and quasi-Newton algorithms to find the minimum of the cost function. In particular, we consider two classic iterative algorithms: the conjugate gradient (CG) algorithm and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. For educational purposes we try to explain several numerical and computational issues with some detail and illustrate them with the Susceptible, Exposed, Infected, Recovered, and Deceased (SEIRD) epidemiological model.
 '''Keywords.''' Inverse problem; noisy data; parameter determination; adjoint method, variational approach.