Search

Vector extrapolations for fixed-point iterations are shown to converge faster when their step lengths are computed from two or three consecutive maps alternately. Based on this finding, cyclic extrapolation methods are proposed which require few objective function evaluations, no matrix inversion, and little extra memory. They are efficient in high-dimensional contexts and do not require problem-specific adaptation. A convergence analysis is done for symmetric positive definite linear systems and for contraction mappings. The proposed methods rivaled common quasi-Newton alternatives in eight mapping applications that included gradient descent for constrained and unconstrained optimization. • Vector extrapolation methods converge faster when alternating between 2 or 3 maps. • ACX require little computation and converge fast. • ACX accelerate gradient descent. • ACX are used for nonlinear optimization. • ACX accelerate the power method for dominant eigenvalues. [ABSTRACT FROM AUTHOR]

Process monitoring based on neural networks is getting more and more attention. Compared with classical neural networks, high-order neural networks have natural advantages in dealing with heteroscedastic data. However, high-order neural networks might bring the risk of overfitting, which learning both the key information from original data and noises or anomalies. Orthogonal constraints can greatly reduce correlations between extracted features, thereby reducing the overfitting risk. This paper proposes a novel fault detection method called second-order component analysis (SCA). SCA rules out the heteroscedasticity of process data by optimizing a second-order autoencoder with orthogonal constraints. In order to deal with this constrained optimization problem, a geometric conjugate gradient algorithm is adopted in this paper, which performs geometric optimization on the combination of Stiefel manifold and Euclidean manifold. Extensive experiments on the Tennessee-Eastman benchmark process show that SCA outperforms the compared state-of-the-art methods with missed detection rate (MDR) and false alarm rate (FAR). • Second-order component analysis (SCA) is proposed for fault detection. • SCA uses the structure of autoencoder and adds second-order terms to improve nonlinear mapping capabilities. • SCA adopts orthogonal constraints to reduce the overfitting problem. • SCA adopts a geometric conjugate gradient (GCG) algorithm to deal with constrained optimization problem. [ABSTRACT FROM AUTHOR]