Alternating cyclic vector extrapolation technique for accelerating nonlinear optimization algorithms and fixed-point mapping applications.

Vector extrapolations for fixed-point iterations are shown to converge faster when their step lengths are computed from two or three consecutive maps alternately. Based on this finding, cyclic extrapolation methods are proposed which require few objective function evaluations, no matrix inversion, and little extra memory. They are efficient in high-dimensional contexts and do not require problem-specific adaptation. A convergence analysis is done for symmetric positive definite linear systems and for contraction mappings. The proposed methods rivaled common quasi-Newton alternatives in eight mapping applications that included gradient descent for constrained and unconstrained optimization. • Vector extrapolation methods converge faster when alternating between 2 or 3 maps. • ACX require little computation and converge fast. • ACX accelerate gradient descent. • ACX are used for nonlinear optimization. • ACX accelerate the power method for dominant eigenvalues. [ABSTRACT FROM AUTHOR]