Some iterative regularized methods for highly nonlinear least squares problems

This report treats numerical methods for highly nonlinear least squares problems for which procedural and rounding errors are unavoidable, e.g. those arising in the development of various nonlinear system identification techniques based on input‐output representation of the model such as training of artificial neural networks. Let F be a Frechet‐differentiable operator acting between Hilbert spaces H1 and H2 and such that the range of its first derivative is not necessarily closed. For solving the equation F(x) = 0 or minimizing the functional f(x) = ½ ‖F(x)‖2 , x H 1, two‐parameter iterative regularization methods based on the Gauss‐Newton method under certain condition on a test function and the required solution are developed, their computational aspects are discussed and a local convergence theorem is proved. First published online: 14 Oct 2010