Search

Graphical abstract Highlights • A non-iterative and incremental learning method for neural networks is proposed. • It is a hyperparameter-free learning method. • Weights are obtained by linear equations and Singular Value Decomposition. • The method’s accuracy is comparable to some other state-of-the-art approaches. • Its efficiency is better making the method proper to deal with large-scale problems. Abstract In machine learning literature, and especially in the literature referring to artificial neural networks, most methods are iterative and operate in batch mode. However, many of the standard algorithms are not suitable for efficiently managing the emerging large-scale data sets obtained from new real-world applications. Novel proposals to address these challenges are mainly iterative approaches based on incremental or distributed learning algorithms. However, the state-of-the-art is such that there are few learning methods based on non-iterative approaches, which have certain advantages over iterative models in dealing more efficiently with these new challenges. We have developed a non-iterative, incremental and hyperparameter-free learning method for one-layer feedforward neural networks without hidden layers. This method efficiently obtains the optimal parameters of the network, regardless of whether the data contains a greater number of samples than variables or vice versa. It does this by using a square loss function that measures errors before the output activation functions and scales them by the slope of these functions at each data point. The outcome is a system of linear equations that obtain the network's weights and that is further transformed using Singular Value Decomposition. We analyze the behavior of the algorithm, comparing its performance and scaling properties to other state-of-the-art approaches. Experimental results demonstrate that the proposed method appropriately solves a wide range of classification problems and is able to efficiently deal with large-scale tasks. [ABSTRACT FROM AUTHOR]

We propose an adaptive inverse control scheme, which employs a neural network for the system identification phase and updates its weights in online mode. The theoretical basis of the method is given and its performance is illustrated by means of its application to different control problems showing that our proposal is able to overcome the problems generated by dynamic nature of the process or by physical changes of the system which originate important modifications in the process. A comparative experimental study is presented in order to show the more stable behavior of the proposed method in several working ranks. [ABSTRACT FROM AUTHOR]

Abstract: Recent machine learning challenges require the capability of learning in non-stationary environments. These challenges imply the development of new algorithms that are able to deal with changes in the underlying problem to be learnt. These changes can be gradual or trend changes, abrupt changes and recurring contexts. As the dynamics of the changes can be very different, existing machine learning algorithms exhibit difficulties to cope with them. Several methods using, for instance, ensembles or variable length windowing have been proposed to approach this task. In this work we propose a new method, for single-layer neural networks, that is based on the introduction of a forgetting function in an incremental online learning algorithm. This forgetting function gives a monotonically increasing importance to new data. Due to the combination of incremental learning and increasing importance assignment the network forgets rapidly in the presence of changes while maintaining a stable behavior when the context is stationary. The performance of the method has been tested over several regression and classification problems and its results compared with those of previous works. The proposed algorithm has demonstrated high adaptation to changes while maintaining a low consumption of computational resources. [Copyright &y& Elsevier]