Search

This paper introduces a methodology for neural network global optimization. The aim is the simultaneous optimization of multilayer perceptron (MLP) network weights and architectures, in order to generate topologies with few connections and high classification performance for any data sets. The approach combines the advantages of simulated annealing, tabu search and the backpropagation training algorithm in order to generate an automatic process for producing networks with high classification performance and low complexity. Experimental results obtained with four classification problems and one prediction problem has shown to be better than those obtained by the most commonly used optimization techniques.

This paper presents an approach of using Simulated Annealing and Tabu Search for the simultaneous optimization of neural network architectures and weights. The problem considered is the odor recognition in an artificial nose. Both methods have produced networks with high classification performance and low complexity. Generalization has been improved by using the backpropagation algorithm for fine tuning. The combination of simple and traditional search methods has shown to be very suitable for generating compact and efficient networks.

Abstract: This paper introduces an approach called Clustering and Co-evolution to Construct Neural Network Ensembles (CONE). This approach creates neural network ensembles in an innovative way, by explicitly partitioning the input space through a clustering method. The clustering method allows a reduction in the number of nodes of the neural networks that compose the ensemble, thus reducing the execution time of the learning process. This is an important characteristic especially when evolutionary algorithms are used. The clustering method also ensures that different neural networks specialize in different regions of the input space, working in a divide-and-conquer way, to maintain and improve the accuracy. Besides, the clustering method facilitates the understanding of the system and makes a straightforward distributed implementation possible. The experiments performed with seven classification databases and three different co-evolutionary algorithms show that CONE considerably reduces the execution time without prejudicing (and even improving) the accuracy, even when a distributed implementation is not used. [Copyright &y& Elsevier]

Meta-Learning has been successfully applied to acquire knowledge used to support the selection of learning algorithms. Each training example in Meta-Learning (i.e. each meta-example) is related to a learning problem and stores the experience obtained in the empirical evaluation of a set of candidate algorithms when applied to the problem. The generation of a good set of meta-examples can be a costly process depending for instance on the number of available learning problems and the complexity of the candidate algorithms. In this work, we proposed the Active Meta-Learning, in which Active Learning techniques are used to reduce the set of meta-examples by selecting only the most relevant problems for meta-example generation. In an implemented prototype, we evaluated the use of two different Active Learning techniques applied in two different Meta-Learning tasks. The performed experiments revealed a significant gain in the Meta-Learning performance when the active techniques were used to support the meta-example generation. [ABSTRACT FROM AUTHOR]

We present here an original work that applies meta-learning approaches to select models for time-series forecasting. In our work, we investigated two meta-learning approaches, each one used in a different case study. Initially, we used a single machine learning algorithm to select among two models to forecast stationary time series (case study I). Following, we used the NOEMON approach, a more recent work in the meta-learning area, to rank three models used to forecast time series of the M3-Competition (case study II). The experiments performed in both case studies revealed encouraging results. [Copyright &y& Elsevier]

The selection of a good model for forecasting a time series is a task that involves experience and knowledge. Employing machine learning algorithms is a promising approach to acquiring knowledge in regards to this task. A supervised classification method originating from the symbolic data analysis field is proposed for the model selection problem. This method was applied in the task of selecting between two widespread models, and compared to other learning algorithms. To date, it has obtained the lowest classification errors among all the tested algorithms. [Copyright &y& Elsevier]

A novel way to simulate Turing Machines (TMs) by Artificial Neural Networks (ANNs) is proposed. We claim that the proposed simulation is in agreement with the correct interpretation of Turing's analysis of computation; compatible with the current approaches to analyze cognition as an interactive agent-environment process; and physically realizable since it does not use connection weights with unbounded precision. A full description of an implementation of a universal TM into a recurrent sigmoid ANN focusing on the TM finite state control is given, leaving the tape, an infinite resource, as an external non-intrinsic feature. Also, motivated by the results on the limit of what can actually be computed by ANNs when noise is taken into account, we introduce the notion of Definite Turing Machine and investigate some of its properties. [ABSTRACT FROM AUTHOR]

This work addresses the problem of finding the adjustable parameters of a learning algorithm using Genetic Algorithms. This problem is also known as the model selection problem. In this paper, some model selection techniques (e.g., crossvalidation and bootstrap) are used as objective functions of a Genetic Algorithm. The Genetic Algorithm is modified in order to allow the efficient use of these objective functions by means of occam's razor, growing, and other heuristics. Some modifications explore intrinsic features of Genetic Algorithms, such as their ability to handle multiple and noise objective functions. The proposed techniques are very general and may be applied to a large range of learning algorithms. [ABSTRACT FROM AUTHOR]

A large number of classification algorithms have been proposed in the machine learning literature. These algorithms have different pros and cons, and no algorithm is the best for all datasets. Hence, a challenging problem consists of choosing the best classification algorithm with its best hyper-parameter settings for a given input dataset. In the last few years, Automated Machine Learning (Auto-ML) has emerged as a promising approach for tackling this problem, by doing a heuristic search in a large space of candidate classification algorithms and their hyper-parameter settings. In this work we propose an improved version of our previous Evolutionary Algorithm (EA) – more precisely, an Estimation of Distribution Algorithm – for the Auto-ML task of automatically selecting the best classifier ensemble and its best hyper-parameter settings for an input dataset. The new version of this EA was compared against its previous version, as well as against a random forest algorithm (a strong ensemble algorithm) and a version of the well-known Auto-ML method Auto-WEKA adapted to search in the same space of classifier ensembles as the proposed EA. In general, in experiments with 21 datasets, the new EA version obtained the best results among all methods in terms of four popular predictive accuracy measures: error rate, precision, recall and F-measure. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the guest editors discuss various reports within the issue on topics including selection of algorithms to solve traveling salesman problem using meta-learning, selecting features to improve time series forecasting using hybrid intelligent techniques, and comparison of single classification methods and classifier ensembles performances.

Abstract: A supervised learning algorithm for quantum neural networks (QNN) based on a novel quantum neuron node implemented as a very simple quantum circuit is proposed and investigated. In contrast to the QNN published in the literature, the proposed model can perform both quantum learning and simulate the classical models. This is partly due to the neural model used elsewhere which has weights and non-linear activations functions. Here a quantum weightless neural network model is proposed as a quantisation of the classical weightless neural networks (WNN). The theoretical and practical results on WNN can be inherited by these quantum weightless neural networks (qWNN). In the quantum learning algorithm proposed here patterns of the training set are presented concurrently in superposition. This superposition-based learning algorithm (SLA) has computational cost polynomial on the number of patterns in the training set. [Copyright &y& Elsevier]

Millions of people throughout the world describe themselves as being deaf. Some of them suffer from severe hearing loss and consequently use an alternative manner with which to communicate with society by means of either written or visual language. There are several sign languages capable of dealing with such a need. Nonetheless, a communication gap still exists even when using such languages, since only a small fraction of the population is able to use them. Over the last few years, due to the increasing need for universal accessibility when using computational resources, gesture recognition has been widely researched. Thus, in an attempt to reduce this communication gap, our approach proposes a computational solution in order to translate static gesture symbols into text symbols, through computer vision, without the use of hand sensors or gloves. In order to guarantee the highest quality, with emphasis on the reliability of the system and real-time translation, we have developed an approach based on the Extreme Learning Machine (ELM) pattern recognition algorithms fully implemented in hardware, and have assessed it to measure these two metrics. Hardware components were designed in order to perform the best image processing and pattern recognition tasks used within the project. As a case study, and so as to validate the technique, a recognition system for the Brazilian Sign Language (LIBRAS) was implemented. Besides ensuring that this approach could be used for any static hand gesture symbol recognition, our main goal was to guarantee fast, reliable gesture recognition for communication between humans. Experimental results have demonstrated that the system is able to recognize LIBRAS symbols with an accuracy of 97%, a response time of 6.5ms per letter recognition, and using only 43% (about 64,851 logic elements) of the FPGA area. [ABSTRACT FROM AUTHOR]