Search

This paper considers an explicit continuation method with the trusty time-stepping scheme and the limited-memory BFGS (L-BFGS) updating formula (Eptctr) for the linearly constrained optimization problem. At every iteration, Eptctr only involves three pairs of the inner product of vector and one matrix-vector product, other than the traditional and representative optimization method such as the sequential quadratic programming (SQP) or the latest continuation method such as Ptctr \cite{LLS2020}, which needs to solve a quadratic programming subproblem (SQP) or a linear system of equations (Ptctr). Thus, Eptctr can save much more computational time than SQP or Ptctr. Numerical results also show that the consumed time of EPtctr is about one tenth of that of Ptctr or one fifteenth to 0.4 percent of that of SQP. Furthermore, Eptctr can save the storage space of an $(n+m) \times (n+m)$ large-scale matrix, in comparison to SQP. The required memory of Eptctr is about one fifth of that of SQP. Finally, we also give the global convergence analysis of the new method under the standard assumptions., Comment: arXiv admin note: substantial text overlap with arXiv:2012.14808; text overlap with arXiv:2005.05965

This paper considers an explicit continuation method and the trust-region updating strategy for the unconstrained optimization problem. Moreover, in order to improve its computational efficiency and robustness, the new method uses the switching preconditioning technique. In the well-conditioned phase, the new method uses the L-BFGS method as the preconditioning technique in order to improve its computational efficiency. Otherwise, the new method uses the inverse of the Hessian matrix as the pre-conditioner in order to improve its robustness. Numerical results aslo show that the new method is more robust and faster than the traditional optimization method such as the trust-region method and the line search method. The computational time of the new method is about one percent of that of the trust-region method (the subroutine fminunc.m of the MATLAB2019a environment, it is set by the trust-region method) or one fifth of that the line search method (fminunc.m is set by the quasi-Newton method) for the large-scale problem. Finally, the global convergence analysis of the new method is also given.

The LSTM network was proposed to overcome the difficulty in learning long-term dependence, and has made significant advancements in applications. With its success and drawbacks in mind, this paper raises the question - do RNN and LSTM have long memory? We answer it partially by proving that RNN and LSTM do not have long memory from a statistical perspective. A new definition for long memory networks is further introduced, and it requires the model weights to decay at a polynomial rate. To verify our theory, we convert RNN and LSTM into long memory networks by making a minimal modification, and their superiority is illustrated in modeling long-term dependence of various datasets., Comment: Accepted by ICML 2020. Added references, experiments and acknowledgements

For nonlinear equations, the homotopy methods (continuation methods) are popular in engineering fields since their convergence regions are large and they are quite reliable to find a solution. The disadvantage of the classical homotopy methods is that their computational time is heavy since they need to solve many auxiliary nonlinear systems during the intermediate continuation processes. In order to overcome this shortcoming, we consider the special explicit continuation Newton method with the residual trust-region time-stepping scheme for this problem. According to our numerical experiments, the new method is more robust and faster to find the required solution of the real-world problem than the traditional optimization method (the built-in subroutine fsolve.m of the MATLAB environment) and the homotopy continuation methods(HOMPACK90 and NAClab). Furthermore, we analyze the global convergence and the local superlinear convergence of the new method.

The nonlinear optimization problem with linear constraints has many applications in engineering fields such as the visual-inertial navigation and localization of an unmanned aerial vehicle maintaining the horizontal flight. In order to solve this practical problem efficiently, this paper constructs a continuation method with the trusty time-stepping scheme for the linearly equality-constrained optimization problem at every sampling time. At every iteration, the new method only solves a system of linear equations other than the traditional optimization method such as the sequential quadratic programming (SQP) method, which needs to solve a quadratic programming subproblem. Consequently, the new method can save much more computational time than SQP. Numerical results show that the new method works well for this problem and its consumed time is about one fifth of that of SQP (the built-in subroutine fmincon.m of the MATLAB2018a environment) or that of the traditional dynamical method (the built-in subroutine ode15s.m of the MATLAB2018a environment). Furthermore, we also give the global convergence analysis of the new method.

This paper investigates the localization problem of high-speed high-altitude unmanned aerial vehicle (UAV) with a monocular camera and inertial navigation system. It proposes a navigation method utilizing the complementarity of vision and inertial devices to overcome the singularity which arises from the horizontal flight of UAV. Furthermore, it modifies the mathematical model of localization problem via separating linear parts from nonlinear parts and replaces a nonlinear least-squares problem with a linearly equality-constrained optimization problem. In order to avoid the ill-condition property near the optimal point of sequential unconstrained minimization techniques(penalty methods), it constructs a semi-implicit continuous method with a trust-region technique based on a differential-algebraic dynamical system to solve the linearly equality-constrained optimization problem. It also analyzes the global convergence property of the semi-implicit continuous method in an infinity integrated interval other than the traditional convergence analysis of numerical methods for ordinary differential equations in a finite integrated interval. Finally, the promising numerical results are also presented.