Search

Wind and solar energy forecasting have become crucial for the inclusion of renewable energy in electrical power systems. Although most works have focused on point prediction, it is currently becoming important to also estimate the forecast uncertainty. With regard to forecasting methods, deep neural networks have shown good performance in many fields. However, the use of these networks for comparative studies of probabilistic forecasts of renewable energies, especially for regional forecasts, has not yet received much attention. The aim of this article is to study the performance of deep networks for estimating multiple conditional quantiles on regional renewable electricity production and compare them with widely used quantile regression methods such as the linear, support vector quantile regression, gradient boosting quantile regression, natural gradient boosting and quantile regression forest methods. A grid of numerical weather prediction variables covers the region of interest. These variables act as the predictors of the regional model. In addition to quantiles, prediction intervals are also constructed, and the models are evaluated using different metrics. These prediction intervals are further improved through an adapted conformalized quantile regression methodology. Overall, the results show that deep networks are the best performing method for both solar and wind energy regions, producing narrow prediction intervals with good coverage.

Deep neural networks (DNN) are becoming increasingly relevant for probabilistic forecasting because of their ability to estimate prediction intervals (PIs). Two different ways for estimating PIs with neural networks stand out: quantile estimation for posterior PI construction and direct PI estimation. The former first estimates quantiles, which are then used to construct PIs, while the latter directly obtains the lower and upper PI bounds by optimizing some loss functions, with the advantage that PI width is directly considered in the optimization process and thus may result in narrower intervals. In this work, two different DNN-based models are studied for direct PI estimation, and compared with DNN for quantile estimation in the context of solar and wind regional energy forecasting. The first approach is based on the recent quality-driven loss and is formulated to estimate multiple PIs with a single model. The second is a novel approach that employs hypernetworks (HN), where direct PI estimation is formulated as a multi-objective problem, returning a Pareto front of solutions that contains all possible coverage-width optimal trade-offs. This formulation allows HN to obtain optimal PIs for all possible coverages without increasing the number of network outputs or adjusting additional hyperparameters, as opposed to the first direct model. Results show that prediction intervals from direct estimation are narrower (up to 20%) than those of quantile estimation, for target coverages 70%–80% for all regions, and also 85%, 90%, and 95% depending on the region, while HN always achieves the required coverage for the higher target coverages. This publication is part of the I+D+i project PID2019-107455RBC22, funded by MCIN /AEI/10.13039/501100011033. This work was also supported by the Comunidad de Madrid Excellence Program. Funding for APC: Universidad Carlos III de Madrid (Read & Publish Agreement CRUE-CSIC 2022)

In the last years, there is an increasing interest for enhanced method for assessing and monitoring the level of the global horizontal irradiance (GHI) in photovoltaic (PV) systems, fostered by the massive deployment of this energy. Thermopile or photodiode pyranometers provide point measurements, which may not be adequate in cases when areal information is important (as for PV network or large PV plants monitoring). The use of All Sky Imagers paired convolutional neural networks, a powerful technique for estimation, has been proposed as a plausible alternative. In this work, a convolutional neural network architecture is presented to estimate solar irradiance from sets of ground-level Total Sky Images. This neural network is capable of combining images from three cameras. Results show that this approach is more accurate than using only images from a single camera. It has also been shown to improve the performance of two other approaches: a cloud fraction model and a feature extraction model. This work has been made possible by the Ministerio de Economia y Empresa of Spain, under the project PROSOL (ENE2014-56126-C2). Authors from the University of Jaen are supported by the Junta de Andalucía (Research group TEP-220) and by FEDER funds. This work has been made possible by projects funded by Agencia Estatal de Investigación (PID2019-107455RB-C21 and PID2019-107455RB-C22 / AEI / 10.13039/501100011033).

In this article we explore the blending of the four models (Satellite, WRF-Solar, Smart Persistence and CIADCast) studied in Part 1 by means of Support Vector Machines with the aim of improving GHI and DNI forecasts. Two blending approaches that use the four models as predictors have been studied: the horizon approach constructs a different blending model for each forecast horizon, while the general approach trains a single model valid for all horizons. The influence on the blending models of adding information about weather types is also studied. The approaches have been evaluated in the same four Iberian Peninsula stations of Part 1. Blending approaches have been extended to a regional context with the goal of obtaining improved regional forecasts. In general, results show that blending greatly outperforms the individual predictors, with no large differences between the blending approaches themselves. Horizon approaches were more suitable to minimize rRMSE and general approaches work better for rMAE. The relative improvement in rRMSE obtained by model blending was up to 17% for GHI (16% for DNI), and up to 15% for rMAE. Similar improvements were observed for the regional forecast. An analysis of performance depending on the horizon shows that while the advantage of blending for GHI remains more or less constant along horizons, it tends to increase with horizon for DNI, with the largest improvements occurring at 6 h. The knowledge of weather conditions helped to slightly improve further the forecasts (up to 3%), but only at some locations and for rRMSE. The authors are supported by the Spanish Ministry of Economy and Competitiveness, projects ENE2014-56126-C2-1-R and ENE2014-56126-C2-2-R (http://prosol.uc3m.es). The University of Jan team are also supported by FEDER funds and by the Junta de Andalucia (Research group TEP-220).The authors are in debt with the National Centers for Environmental Prediction (NCEP), EUMETSAT, Faculdade de Ciencias da Universidade de Lisboa, Grupo de Energa Solar of the Universidad Politcnica de Madrid and Abengoa Solar for providing the data used in this work.

The aim of data transformation is to transform the original feature space of data into another space with better properties. This is typically combined with dimensionality reduction, so that the dimensionality of the transformed space is smaller. A widely used method for data transformation and dimensionality reduction is Principal Component Analysis (PCA). PCA finds a subspace that explains most of the data variance. While the new PCA feature space has interesting properties, such as removing linear correlation, PCA is an unsupervised method. Therefore, there is no guarantee that the PCA feature space will be the most appropriate for supervised tasks, such as classification or regression. On the other hand, 3-layer Multi Layer Perceptrons (MLP), which are supervised methods, can also be understood as a data transformation carried out by the hidden layer, followed by a classification/regression operation performed by the output layer. Given that the hidden layer is obtained after a supervised training process, it can be considered that it is performing a supervised data transformation. And if the number of hidden neurons is smaller than the input, also dimensionality reduction. Despite this kind of transformation being widely available (any neural network package that allows access to the hidden layer weights can be used), no extensive experimentation on the quality of 3-layer MLP data transformation has been carried out. The aim of this article is to carry out this research for classification problems. Results show that, overall, this transformation offers better results than the PCA unsupervised transformation method. This work has been supported by Agencia Estatal de Investigación (PID2019-107455RB-C22 /AEI/ 10.13039/501100011033), and Spanish Ministry of Science and Education under TIN2017-85727-C4-3-P (DeepBio) grant

Prediction Intervals are pairs of lower and upper bounds on point forecasts and are useful to take into account the uncertainty on predictions. This article studies the influence of using measured solar power, available at prediction time, on the quality of prediction intervals. While previous studies have suggested that using measured variables can improve point forecasts, not much research has been done on the usefulness of that additional information, so that prediction intervals with less uncertainty can be obtained. With this aim, a multi-objective particle swarm optimization method was used to train neural networks whose outputs are the interval bounds. The inputs to the network used measured solar power in addition to hourly meteorological forecasts. This study was carried out on data from three different locations and for five forecast horizons, from 1 to 5 h. The results were compared with two benchmark methods (quantile regression and quantile regression forests). The Wilcoxon test was used to assess statistical significance. The results show that using measured power reduces the uncertainty associated to the prediction intervals, but mainly for the short forecasting horizons This work was funded by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R (AOPRIN-SOL project).

Recent research has shown that the integration or blending of different forecasting models is able to improve the predictions of solar radiation. However, most works perform model blending to improve point forecasts, but the integration of forecasting models to improve probabilistic forecasting has not received much attention. In this work the estimation of prediction intervals for the integration of four Global Horizontal Irradiance (GHI) forecasting models (Smart Persistence, WRF-solar, CIADcast, and Satellite) is addressed. Several short-term forecasting horizons, up to one hour ahead, have been analyzed. Within this context, one of the aims of the article is to study whether knowledge about the synoptic weather conditions, which are related to the stability of weather, might help to reduce the uncertainty represented by prediction intervals. In order to deal with this issue, information about which weather type is present at the time of prediction, has been used by the blending model. Four weather types have been considered. A multi-objective variant of the Lower Upper Bound Estimation approach has been used in this work for prediction interval estimation and compared with two baseline methods: Quantile Regression (QR) and Gradient Boosting (GBR). An exhaustive experimental validation has been carried out, using data registered at Seville in the Southern Iberian Peninsula. Results show that, in general, using weather type information reduces uncertainty of prediction intervals, according to all performance metrics used. More specifically, and with respect to one of the metrics (the ratio between interval coverage and width), for high-coverage (0.90, 0.95) prediction intervals, using weather type enhances the ratio of the multi-objective approach by 2%¿. Also, comparing the multi-objective approach versus the two baselines for high-coverage intervals, the improvement is 11%¿% over QR and 10%¿% over GBR. Improvements for low-coverage intervals (0.85) are smaller. The authors are supported by projects funded by Agencia Estatal de Investigación, Spain (PID2019-107455RB-C21 and PID2019-107455RB-C22/AEI/10.13039/501100011033). Also supported by Spanish Ministry of Economy and Competitiveness, project ENE2014-56126-C2-1-R and ENE2014-56126-C2-2-R (http://prosol.uc3m.es). The University of Jaén team is also supported by FEDER, Spain funds and by the Junta de Andalucía, Spain (Research group TEP-220)

Based on a large and recently developed database of 1-min irradiance and ancillary data observations at 54 world stations, this study uses the gradient boosting Machine Learning (ML) technique to improve the process of components separation, through which the direct and diffuse solar radiation components are estimated from 1-min global horizontal irradiance data. Here, the XGBoost implementation of gradient boosting is used both with ensembles of linear and ensembles of non-linear weak prediction models. The predictions of 140 separation models of the literature are combined using XGBoost to overall improve the random errors of the predictions of the individual separation models at any of the validation sites. The minimum prediction error is essentially achieved by a combination of 26 out of the original 140 models, with no meaningful reduction in error by combining more models. Most of these 26 models use at least three inputs in addition to clearness index. In parallel, XGBoost is also used to separate the components directly from the inputs to the separation models. From the 24 possible inputs used in the original 140 separation models, only 14 are found relevant. These 14 inputs could be used with appropriate formalism to subsequently develop a better separation model. It is found that when the training and validation datasets are not collocated, the RMSD of the predictions increases, on average, 2% with respect to the case of collocated datasets. Overall, the present results indicate that a data-driven ML approach combining a limited number of existing models can be used to considerably decrease the currently large random errors associated with such models when used separately at high temporal frequency The first two authors have been funded by the Spanish Ministry of Science under project ENE2014-56126-C2-2-R (AOPRIN-SOL project). Jose Antonio Ruiz Arias was supported by the Spanish Ministry of Economy and Competitiveness under the project ENE2014-56126-C2-1-R.

This paper has been presented at: 14th IFIP International Conference on Artificial Intelligence Applications and Innovations In this work two different machine learning approaches have been studied to predict wind power for different time horizons: individual and global models. The individual approach constructs a model for each horizon while the global approach obtains a single model that can be used for all horizons. Both approaches have advantages and disadvantages. Each individual model is trained with data pertaining to a single horizon, thus it can be specific for that horizon, but can use fewer data for training than the global model, which is constructed with data belonging to all horizons. Support Vector Machines have been used for constructing the individual and global models. This study has been tested on energy production data obtained from the Sotavento wind farm and meteorological data from the European Centre for Medium-Range Weather Forecasts, for a 5 × 5 grid around Sotavento. Also, given the large amount of variables involved, a feature selection algorithm (Sequential Forward Selection) has been used in order to improve the performance of the models. Experimental results show that the global model is more accurate than the individual ones, specially when feature selection is used. The authors acknowledge financial support granted by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R.

Image clustering is a critical and essential component of image analysis to several fields and could be considered as an optimization problem. Cuckoo Search (CS) algorithm is an optimization algorithm that simulates the aggressive reproduction strategy of some cuckoo species.In this paper, a combination of CS and classical algorithms (KM, FCM, and KHM) is proposed for unsupervised satellite image classification. Comparisons with classical algorithms and also with CS are performed using three cluster validity indices namely DB, XB, and WB on synthetic and real data sets. Experimental results confirm the effectiveness of the proposed approach.

In this article, a filter feature weighting technique for attribute selection in classification problems is proposed (LIA). It has two main characteristics. First, unlike feature weighting methods, it is able to consider attribute interactions in the weighting process, rather than only evaluating single features. Attribute subsets are evaluated by projecting instances into a grid defined by attributes in the subset. Then, the joint relevance of the subset is computed by measuring the information present in the cells of the grid. The final weight for each attribute is computed by taking into account its performance in each of the grids it participates. Second, many real problems contain low signal-to-noise ratios, due to instance of high noise levels, class overlap, class imbalance, or small training samples. LIA computes reliable local information for each of the cells by estimating the number of target class instances not due to chance, given a confidence value. In order to study its properties, LIA has been evaluated with a collection of 18 real datasets and compared to two feature weighting methods (Chi-Squared and ReliefF) and a subset feature selection algorithm (CFS). Results show that the method is significantly better in many cases, and never significantly worse. LIA has also been tested with different grid dimensions (1, 2, and 3). The method works best when evaluating attribute subsets larger than 1, hence showing the usefulness of considering attribute interactions. The authors acknowledge financial support granted by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R.

This paper has been presented at: 19th Intelligent Data Engineering and Automated Learning (IDEAL 2018) While it is common to make point forecasts for solar energy generation, estimating the forecast uncertainty has received less attention. In this article, prediction intervals are computed within a multi-objective approach in order to obtain an optimal coverage/width tradeoff. In particular, it is studied whether using measured power as an another input, additionally to the meteorological forecast variables, is able to improve the properties of prediction intervals for short time horizons (up to three hours). Results show that they tend to be narrower (i.e. less uncertain), and the ratio between coverage and width is larger. The method has shown to obtain intervals with better properties than baseline Quantile Regression. This work has been funded by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R (AOPRIN-SOL project).

Optimization of channel selection and spatial filter for Brain Computer Interfaces.A multi-objective evolutionary algorithm simultaneously optimizes filter and channels.Multi-objective approach returns a set of solutions (channels vs. error tradeoffs).A simple threshold is encoded to select channels instead of a longer binary mask.Joint optimization of filter and channels performs better than only the channels. Obtaining high accuracy classification from Brain Computer Interfaces require to attach many electrodes on the scalp of subjects. On the other hand, their placement on the scalp involves generally a laborious and time consuming process. Therefore, it is important for the practitioner to estimate how many electrodes, and which ones, are needed to obtain the required accuracy. With this purpose, a multi-objective formulation is proposed in order to obtain a set of solutions (Pareto front) that represent all the optimal tradeoffs between the number of channels and the classification accuracy, from where the practitioner can choose. Additionally, previous research has shown that classification accuracy highly depends on the proper tuning of the filter used to preprocess the electroencephalogram. Therefore, in this work, the Non-dominated Sorting Genetic Algorithm II is used for optimizing both the number of electrodes and the classification error, through the optimization of a spatial filter encoded in the solution. The fact that the filter is part of the solution allows to determine which electrodes are to be selected by using a simple threshold, instead of a long binary mask as in other approaches. Empirical results show that indeed, the simultaneous optimization of the spatial filter and selected electrodes is crucial to obtain a low classification error, compared to other approaches that reduce the number of electrodes but do not modify the filter.

This article addresses two issues in solar energy forecasting from the numerical weather prediction NWP models using machine learning. First, we are interested in determining the relevant information for the forecasting task. With this purpose, a study has been carried out to evaluate the influence on accuracy of the number of NWP grid nodes used as input for the forecasting model, as well as their relative importance. Several machine learning support vector machines and gradient boosting and feature selection algorithms linear, ReliefF, and local information analysis have been used in this study. The second aim is to be able to predict solar energy for locations where no previous production data are available. To address this goal, an approach consisting on modeling regions in the grid is proposed. Models aggregate models use as input attributes the meteorological variables relevant for the region and two new inputs to identify the location of each station: the latitude and the longitude. Those models can be used to predict energy production for existing stations and for new locations, represented by latitude and longitude. Copyright © 2015 John Wiley & Sons, Ltd.

In the context of forecasting for renewable energy, it is common to produce point forecasts but it is also important to have information about the uncertainty of the forecast. To this extent, instead of providing a single measure for the prediction, lower and upper bound for the expected value for the solar radiation are used (prediction interval). This estimation of optimal prediction intervals requires the simultaneous optimization of two objective measures: on one hand, it is important that the coverage probability of the interval is as close as possible to a given target value. On the other, in order to bound uncertainty, intervals must be narrow; this means that there is a trade-off between both objectives, as narrow intervals reduce the coverage probability for those solutions, as the actual value of solar radiation is more likely to fall outside the predicted margins. In this work we propose a multi-objective evolutionary approach that is able to optimize both goals simultaneously. The proposal uses neural networks to create complex non-linear models whose outputs are the upper and lower limits of the prediction intervals. Results have been compared with a single-objective optimization of similar neural network architectures and a baseline algorithm (quantile estimation with gradient boosting). Results show that the neural network is able to provide accurate results. Also, the multi-objective approach is able to obtain the best results and has also the advantage that a single run is able to obtain prediction intervals for any target coverage probability. This work has been funded by the Spanish Ministry of Science under contract ENE2014-56126-C2-2-R (AOPRIN-SOL project).

This work deals with wind energy prediction using meteorological variables estimated by a Numerical Weather Prediction model in a grid around the wind farm of interest. Two machine learning techniques have been tested, Support Vector Machine and Gradient Boosting Regression, in order to study their performance and compare the results. The use of meteorological variables estimated in a grid generally implies a large number of inputs to the models and the performance of models might decrease. Hence, in this context, the use of feature selection algorithms might be interesting in order to improve the generalization capability of models and/or reduce the number of attributes. We have compared three feature selection techniques based on different paradigms: Principal Components Analysis, ReliefF, and Sequential Forward Selection. Energy production data has been obtained from the Sotavento experimental wind farm. Meteorological variables have been obtained from European Centre for Medium-Range Weather Forecasts, for a 5\(\,\times \,\)5 grid around Sotavento.

An appropriate preprocessing of EEG signals is crucial to get high classification accuracy for Brain–Computer Interfaces (BCI). The raw EEG data are continuous signals in the time-domain that can be transformed by means of filters. Among them, spatial filters and selecting the most appropriate frequency-bands in the frequency domain are known to improve classification accuracy. However, because of the high variability among users, the filters must be properly adjusted to every user’s data before competitive results can be obtained. In this paper we propose to use the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for automatically tuning the filters. Spatial and frequency-selection filters are evolved to minimize both classification error and the number of frequency bands used. This evolutionary approach to filter optimization has been tested on data for different users from the BCI-III competition. The evolved filters provide higher accuracy than approaches used in the competition. Results are also consistent across different runs of CMA-ES. This work has been funded by the Spanish Ministry of Science under Contract TIN2008-06491-C04-03 (MSTAR project) and TIN2011-28336 (MOVES project). Publicado

Traditional mean–variance financial portfolio optimization is based on two sets of parameters, estimates for the asset returns and the variance–covariance matrix. The allocations resulting from both traditional methods and heuristics are very dependent on these values. Given the unreliability of these forecasts, the expected risk and return for the portfolios in the efficient frontier often differ from the expected ones. In this work we present a resampling method based on time-stamping to control the problem. The approach, which is compatible with different evolutionary multiobjective algorithms, is tested with four different alternatives. We also introduce new metrics to assess the reliability of forecast efficient frontiers. The authors acknowledge financial support granted by the Spanish Ministry of Science under contract TIN2008-06491-C04- 03 (MSTAR), TIN2011-28336 (MOVES) and Comunidad de Madrid (CCG10-UC3M/TIC-5029). Publicado

In this paper, we propose a lazy learning strategy for building classification learning models. Instead of learning the models with the whole training data set before observing the new instance, a selection of patterns is made depending on the new query received and a classification model is learnt with those selected patterns. The selection of patterns is not homogeneous, in the sense that the number of selected patterns depends on the position of the query instance in the input space. That selection is made using a weighting function to give more importance to the training patterns that are more similar to the query instance. Our intention is to provide a lazy learning mechanism suited to any machine learning classification algorithm. For this reason, we study two different methods to avoid fixing any parameter. Experimental results show that classification rates of traditional machine learning algorithms based on trees, rules, or functions can be improved when they are learnt with the lazy learning approach proposed. This work has been funded by the Spanish Ministry of Science under contract TIN2008-06491-C04-03 (MSTAR project). Publicado

Lazy learning methods have been used to deal with problems in which the learning examples are not evenly distributed in the input space. They are based on the selection of a subset of training patterns when a new query is received. Usually, that selection is based on the k closest neighbors and it is a static selection, because the number of patterns selected does not depend on the input space region in which the new query is placed. In this paper, a lazy strategy is applied to train radial basis neural networks. That strategy incorporates a dynamic selection of patterns, and that selection is based on two different kernel functions, the Gaussian and the inverse function. This lazy learning method is compared with the classical lazy machine learning methods and with eagerly trained radial basis neural networks. Publicado

In this paper, a combustion temperature retrieval approximation for high-resolution infrared ground-based measurements has been developed based on a multilayer perceptron (MLP) technique. The introduction of a selection subset of features is mandatory due to the problems related to the high dimensionality data and the worse performance of MLPs with this high input dimensionality. Principal component analysis is used to reduce the input data dimensionality, selecting the physically important features in order to improve MLP performance. The use of a priori physical information over other methods in the chosen feature’s phase has been tested and has appeared jointly with the MLP technique as a good alternative for this problem. Publicado

Proceedings of: 8th International Symposium on Intelligent Distributed Computing (IDC'2014). Madrid, September 3-5, 2014 Forecasting solar energy is becoming an important issue in the context of renewable energy sources and Machine Learning Algorithms play an important rule in this field. The prediction of solar energy can be addressed as a time series prediction problem using historical data. Also, solar energy forecasting can be derived from numerical weather prediction models (NWP). Our interest is focused on the latter approach.We focus on the problem of predicting solar energy from NWP computed from GEFS, the Global Ensemble Forecast System, which predicts meteorological variables for points in a grid. In this context, it can be useful to know how prediction accuracy improves depending on the number of grid nodes used as input for the machine learning techniques. However, using the variables from a large number of grid nodes can result in many attributes which might degrade the generalization performance of the learning algorithms. In this paper both issues are studied using data supplied by Kaggle for the State of Oklahoma comparing Support Vector Machines and Gradient Boosted Regression. Also, three different feature selection methods have been tested: Linear Correlation, the ReliefF algorithm and, a new method based on local information analysis. Publicado

In this work, a methodology based on the combined use of a multilayer perceptron model fed using selected spectral information is presented to invert the radiative transfer equation (RTE) and to recover the spatial temperature profile inside an axisymmetric flame. The spectral information is provided by the measurement of the infrared CO2 emission band in the 3-5 mu m spectral region. A guided spectral feature selection was carried out using a joint criterion of principal component analysis and a priori physical knowledge of the radiative problem. After applying this guided feature selection, a subset of 17 wavenumbers was selected. The proposed methodology was applied over synthetic scenarios. Also, an experimental validation was carried out by measuring the spectral emission of the exhaust hot gas plume in a microjet engine with a Fourier transform-based spectroradiometer. Temperatures retrieved using the proposed methodology were compared with classical thermocouple measurements, showing a good agreement between them. Results obtained using the proposed methodology are very promising and can encourage the use of sensor systems based on the spectral measurement of the CO2 emission band in the 3-5 pm spectral window to monitor combustion processes in a nonintrusive way. The authors acknowledge the Spanish Ministry of Education for financial support under the project TRA2005-08892-C02-01.

Real world optimization of financial portfolios pose a challenging multiobjective problem that can be tackled using Evolutionary Algorithms. The fact that the optimization process is subject to the presence of uncertainty concerning asset returns is likely to lead to unreliable solutions. This work suggests extending the classic mean-variance optimization problem with a third explicit robustness objective. This results on sets of portfolios that can be subsequently grouped together according to their reliability. This additional information allows for a better informed decision making regarding asset allocation. © 2014 - IOS Press and the authors. All rights reserved. Financial support granted by the Spanish Ministry of Science under contract TIN2011-28336 (MOVES).

Multi-step prediction is a difficult task that has attracted increasing interest in recent years. It tries to achieve predictions several steps ahead into the future starting from current information. The interest in this work is the development of nonlinear neural models for the purpose of building multi-step time series prediction schemes. In that context, the most popular neural models are based on the traditional feedforward neural networks. However, this kind of model may present some disadvantages when a long-term prediction problem is formulated because they are trained to predict only the next sampling time. In this paper, a neural model based on a partially recurrent neural network is proposed as a better alternative. For the recurrent model, a learning phase with the purpose of long-term prediction is imposed, which allows to obtain better predictions of time series in the future. In order to validate the performance of the recurrent neural model to predict the dynamic behaviour of the series in the future, three different data time series have been used as study cases. An artificial data time series, the logistic map, and two real time series, sunspots and laser data. Models based on feedforward neural networks have also been used and compared against the proposed model. The results suggest than the recurrent model can help in improving the prediction accuracy.

Time series analysis using nonlinear dynamics systems theory and multilayer neural networks models have been applied to the time sequence of water level data recorded every hour at ‘Punta della Salute’ from Venice Lagoon during the years 1980–1994. The first method is based on the reconstruction of the state space attractor using time delay embedding vectors and on the characterisation of invariant properties which define its dynamics. The results suggest the existence of a low dimensional chaotic attractor with a Lyapunov dimension, DL, of around 6.6 and a predictability between 8 and 13 hours ahead. Furthermore, once the attractor has been reconstructed it is possible to make predictions by mapping local-neighbourhood to local-neighbourhood in the reconstructed phase space. To compare the prediction results with another nonlinear method, two nonlinear autoregressive models (NAR) based on multilayer feedforward neural networks have been developed. From the study, it can be observed that nonlinear forecasting produces adequate results for the ‘normal’ dynamic behaviour of the water level of Venice Lagoon, outperforming linear algorithms, however, both methods fail to forecast the ‘high water’ phenomenon more than 2–3 hours ahead.

Although nonlinear inverse and predictive control techniques based on artificial neural networks have been extensively applied to nonlinear systems, their use in real time applications is generally limited. In this paper neural inverse and predictive control systems have been applied to the real-time control of the heat transfer fluid temperature in a pilot chemical reactor. The training of the inverse control system is carried out using both generalised and specialised learning. This allows the preparation of weights of the controller acting in real-time and appropriate performances of inverse neural controller can be achieved. The predictive control system makes use of a neural network to calculate the control action. Thus, the problems related to the high computational effort involved in nonlinear model-predictive control systems are reduced. The performance of the neural controllers is compared against the self-tuning PID controller currently installed in the plant. The results show that neural-based controllers improve the performance of the real plant.

In the field of brain–computer interfaces, one of the main issues is to classify the electroencephalogram (EEG) accurately. EEG signals have a good temporal resolution, but a low spatial one. In this article, metaheuristics are used to compute spatial filters to improve the spatial resolution. Additionally, from a physiological point of view, not all frequency bands are equally relevant. Both spatial filters and relevant frequency bands are user-dependent. In this article a multi-objective formulation for spatial filter optimization and frequency-band selection is proposed. Several multi-objective metaheuristics have been tested for this purpose. The experimental results show, in general, that multi-objective algorithms are able to select a subset of the available frequency bands, while maintaining or improving the accuracy obtained with the whole set. Also, among the different metaheuristics tested, GDE3, which is based on differential evolution, is the most useful algorithm in this context This work has been funded by the Spanish Ministry of Science under contract TIN2008-06491-C04-03 (MSTAR project). Publicado

The main motivation of this paper is to propose a method to extract the output structure and find the input data manifold that best represents that output structure in a multivariate regression problem. A graph similarity viewpoint is used to develop an algorithm based on LDA, and to find out different output models which are learned as an input subspace. The main novelty of the algorithm is related with finding different structured groups and apply different models to fit better those structures. Finally, the proposed method is applied to a real remote sensing retrieval problem where we want to recover the physical parameters from a spectrum of energy.

Proceeding of: Intelligent Data Engineering and Automated Learning – IDEAL 2011: 12th International Conference, Norwich, UK, September 7-9, 2011 The subject of financial portfolio optimization under real-world constraints is a difficult problem that can be tackled using multiobjective evolutionary algorithms. One of the most problematic issues is the dependence of the results on the estimates for a set of parameters, that is, the robustness of solutions. These estimates are often inaccurate and this may result on solutions that, in theory, offered an appropriate risk/return balance and, in practice, resulted being very poor. In this paper we suggest that using a resampling mechanism may filter out the most unstable. We test this idea on real data using SPEA2 as optimization algorithm and the results show that the use of resampling increases significantly the reliability of the resulting portfolios. The authors acknowledge financial support granted by the Spanish Ministry of Science under contract TIN2008-06491-C04-03 (MSTAR) and Comunidad de Madrid (CCG10- UC3M/TIC-5029). Publicado

Proceeding of: 2010 IEEE World Congress in Computational Intelligence (WCCI 2010), Barcelona, Spain, July 18-23, 2010 Abstract—Machine Learning techniques are routinely applied to Brain Computer Interfaces in order to learn a classifier for a particular user. However, research has shown that classiffication techniques perform better if the EEG signal is previously preprocessed to provide high quality attributes to the classifier. Spatial and frequency-selection filters can be applied for this purpose. In this paper, we propose to automatically optimize these filters by means of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The technique has been tested on data from the BCI-III competition, because both raw and manually filtered datasets were supplied, allowing to compare them. Results show that the CMA-ES is able to obtain higher accuracies than the datasets preprocessed by manually tuned filters. This work has been funded by the Spanish Ministry of Science under contract TIN2008-06491-C04-03 (MSTAR project) Publicado

This paper deals with the classification of signals for brain-computer interfaces (BCI). We take advantage of the fact that thoughts last for a period, and therefore EEG samples run in sequences belonging to the same class (thought). Thus, the classification problem can be reformulated into two subproblems: detecting class transitions and determining the class for sequences of samples between transitions. The method detects transitions when the L1 norm between the power spectra at two different times is larger than a threshold. To tackle the second problem, samples are classified by taking into account a window of previous predictions. Two types of windows have been tested: a constant-size moving window and a variable-size growing window. In both cases, results are competitive with those obtained in the BCI III competition.

Proceeding of: Artificial Neural Networks - ICANN 2010. 20th International Conference, Tessaloniki, Greece, September 15-18, 2010. This is an extended version (the paper in the conference proceedings had to be reduced to 10 pages) The aim of this paper is to study the use of Evolutionary Multiobjective Techniques to improve the performance of Neural Net- works (NN). In particular, we will focus on classi¯cation problems where classes are imbalanced. We propose an evolutionary multiobjective ap- proach where the accuracy rate of all the classes is optimized at the same time. Thus, all classes will be treated equally independently of their pres- ence in the training data set. The chromosome of the evolutionary algo- rithm encodes only the weights of the training patterns missclassi¯ed by the NN, instead of all the parameters of the NN as in other approaches. Results show that the multiobjective approach is able to consider all classes at the same time, disregarding to some extent their abundance in the training set or other biases that restrain some of the classes of being learned properly. MSTAR::UC3M, Ref:TIN2008-06491-C04-03 Publicado

Nearest prototype methods can be quite successful on many pattern classification problems. In these methods, a collection of prototypes has to be found that accurately represents the input patterns. The classifier then assigns classes based on the nearest prototype in this collection. In this paper, we first use the standard particle swarm optimizer (PSO) algorithm to find those prototypes. Second, we present a new algorithm, called adaptive Michigan PSO (AMPSO) in order to reduce the dimension of the search space and provide more flexibility than the former in this application. AMPSO is based on a different approach to particle swarms as each particle in the swarm represents a single prototype in the solution. The swarm does not converge to a single solution; instead, each particle is a local classifier, and the whole swarm is taken as the solution to the problem. It uses modified PSO equations with both particle competition and cooperation and a dynamic neighborhood. As an additional feature, in AMPSO, the number of prototypes represented in the swarm is able to adapt to the problem, increasing as needed the number of prototypes and classes of the prototypes that make the solution to the problem. We compared the results of the standard PSO and AMPSO in several benchmark problems from the University of California, Irvine, data sets and find that AMPSO always found a better solution than the standard PSO. We also found that it was able to improve the results of the Nearest Neighbor classifiers, and it is also competitive with some of the algorithms most commonly used for classification. This work was supported by the Spanish founded research Project MSTAR::UC3M, Ref: TIN2008-06491-C04-03 and CAM Project CCG06-UC3M/ESP-0774. Publicado

In regression problems where the number of predictors exceeds the number of observations and the correlation between the predictors is high, a dimensionality reduction or a variable selection approach is demanded. In this paper we deal with a real application where we want to retrieve the physical characteristics of a combustion process from the measurements obtained with a spectroscopic sensor. This application shows up a multicollinearity problem but furthermore it is considered an ill-posed problem.Guided by this application scenario, we propose a clustering approach to find out homogeneous subsets of data which are embedded in arbitrary oriented linear manifold. This model is developed under certain assumptions guided by a priori problem knowledge. The resulting division preserves both, the priori assumptions and the homogeneity in the models. Thereby we break the whole problem in n subproblems improving its individual prediction accuracy versus a global solution. We show the obtained improvements in a real application scenario related with estimating the temperature from spectroscopic data in a remote sensing framework.

This paper presents a new approach to Particle Swarm Optimization, called Michigan Approach PSO (MPSO), and its applica- tion to continuous classi cation problems as a Nearest Prototype (NP) classi er. In Nearest Prototype classi ers, a collection of prototypes has to be found that accurately represents the input patterns. The classi er then assigns classes based on the nearest prototype in this collection. The MPSO algorithm is used to process training data to nd those prototypes. In the MPSO algorithm each particle in a swarm represents a single pro- totype in the solution and it uses modi ed movement rules with particle competition and cooperation that ensure particle diversity. The proposed method is tested both with arti cial problems and with real benchmark problems and compared with several algorithms of the same family. Re- sults show that the particles are able to recognize clusters, nd decision boundaries and reach stable situations that also retain adaptation po- tential. The MPSO algorithm is able to improve the accuracy of 1-NN classi ers, obtains results comparable to the best among other classi ers, and improves the accuracy reported in literature for one of the problems. Publicado

Most machine learning algorithms are eager methods in the sense that a model is generated with the complete training data set and, afterwards, this model is used to generalize the new test instances. In this work we study the performance of different machine learning algorithms when they are learned using a lazy approach. The idea is to build a classification model once the test instance is received and this model will only learn a selection of training patterns, the most relevant for the test instance. The method presented here incorporates a dynamic selection of training patterns using a weighting function. The lazy approach is applied to machine learning algorithms based on different paradigms and is validated in different classification domains.

Proceeding of: 5th International Conference, EMO 2009, Nantes, France, April 7-10, 2009 This work presents the application of a parallel coopera- tive optimization approach to the broadcast operation in mobile ad-hoc networks (manets). The optimization of the broadcast operation im- plies satisfying several objectives simultaneously, so a multi-objective approach has been designed. The optimization lies on searching the best configurations of the dfcn broadcast protocol for a given manet sce- nario. The cooperation of a team of multi-objective evolutionary al- gorithms has been performed with a novel optimization model. Such model is a hybrid parallel algorithm that combines a parallel island- based scheme with a hyperheuristic approach. Results achieved by the algorithms in different stages of the search process are analyzed in order to grant more computational resources to the most suitable algorithms. The obtained results for a manets scenario, representing a mall, demon- strate the validity of the new proposed approach. This work has been supported by the ec (feder) and the Spanish Ministry of Education and Science inside the ‘Plan Nacional de i+d+i’ (tin2005-08818-c04) and (tin2008-06491-c04-02). The work of Gara Miranda has been developed under grant fpu-ap2004-2290. Publicado

Proceeding of: 10th InternationalWork-Conference on Artificial Neural Networks, IWANN 2009 Salamanca, Spain, June 10-12, 2009 The aim os this paper is to study the hybridization of two multi-objective algorithms in the context of a real problem, the MANETs problem. The algorithms studied are Particle Swarm Optimization (MOPSO) and a new multiobjective algorithm based in the combination of NSGA-II with Evolution Strategies (ESN). This work analyzes the improvement produced by hybridization over the Pareto’s fronts compared with the non-hybridized algorithms. The purpose of this work is to validate how hybridization of two evolutionary algorithms of different families may help to solve certain problems together in the context of MANETs problem. The hybridization used for this work consists on a sequential execution of the two algorithms and using the final population of the first algorithm as initial population of the second one. This article has been financed by the Spanish founded research MEC projects OPLINK::UC3M, Ref:TIN2005-08818- C04-02 and MSTAR::UC3M, Ref:TIN2008-06491-C04-03. Publicado

Proceedings of: Second International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2007, La Manga del Mar Menor, Spain, June 18-21, 2007. Nearest Prototype methods can be quite successful on many pattern classification problems. In these methods, a collection of prototypes has to be found that accurately represents the input patterns. The classifier then assigns classes based on the nearest prototype in this collection. In this paper we develop a new algorithm (called AMPSO), based on the Particle Swarm Optimization (PSO) algorithm, that can be used to find those prototypes. Each particle in a swarm represents a single prototype in the solution; the swarm evolves using modified PSO equations with both particle competition and cooperation. Experimentation includes an artificial problem and six common application problems from the UCI data sets. The results show that the AMPSO algorithm is able to find solutions with a reduced number of prototypes that classify data with comparable or better accuracy than the 1-NN classifier. The algorithm can also be compared or improves the results of many classical algorithms in each of those problems; and the results show that AMPSO also performs significantly better than any tested algorithm in one of the problems. This article has been financed by the Spanish founded research MEC project OPLINK::UC3M, Ref: TIN2005-08818-C04-02 and CAM project UC3M-TEC-05-029.

IEEE Swarm Intelligence Symposium. Honolulu, HI, 1-5 april 2007 This paper presents an application of particle swarm optimization (PSO) to continuous classification problems, using a Michigan approach. In this work, PSO is used to process training data to find a reduced set of prototypes to be used to classify the patterns, maintaining or increasing the accuracy of the nearest neighbor classifiers. The Michigan approach PSO represents each prototype by a particle and uses modified movement rules with particle competition and cooperation that ensure particle diversity. The result is that the particles are able to recognize clusters, find decision boundaries and achieve stable situations that also retain adaptation potential. The proposed method is tested both with artificial problems and with three real benchmark problems with quite promising results.

Proceeding of: 7th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2006 (Burgos, Spain, September 20-23, 2006) The use of high spectral resolution measurements to obtain a retrieval of certain physical properties related with the radiative transfer of energy leads a priori to a better accuracy. But this improvement in accuracy is not easy to achieve due to the great amount of data which makes difficult any treatment over it and it's redundancies. To solve this problem, a pick selection based on principal component analysis has been adopted in order to make the mandatory feature selection over the different channels. In this paper, the capability to retrieve the temperature profile in a combustion environment using neural networks jointly with this spectral high resolution feature selection method is studied. Publicado

Proceeding of: International Conference on Computational Intelligence for Modelling, Control and Automation, 2005 and International Conference on Intelligent Agents, Web Technologies and Internet Commerce, Vienna, Austria 28-30 Nov. 2005 Neural networks appear to be a promising tool to solve the so-called inverse problems focused to obtain a retrieval of certain physical properties related to the radiative transference of energy. In this paper the capability of neural networks to retrieve the temperature profile in a combustion environment is proposed. Temperature profile retrieval will be obtained from the measurement of the spectral distribution of energy radiated by the hot gases (combustion products) at wavelengths corresponding to the infrared region. High spectral resolution is usually needed to gain a certain accuracy in the retrieval process. However, this great amount of information makes mandatory a reduction of the dimensionality of the problem. In this sense a careful selection of wavelengths in the spectrum must be performed. With this purpose principal component analysis technique is used to automatically determine those wavelengths in the spectrum that carry relevant information on temperature distribution. A multilayer perceptron will be trained with the different energies associated to the selected wavelengths. The results presented show that multilayer perceptron combined with principal component analysis is a suitable alternative in this field. Publicado

Proceeding of: 15th International Conference on Artificial Neural Networks ICANN 2005, Poland, 11-15 September, 2005 In BCI (Brain Computer Interface) research, the classification of EEG signals is a domain where raw data has to undergo some preprocessing, so that the right attributes for classification are obtained. Several transformational techniques have been used for this purpose: Principal Component Analysis, the Adaptive Autoregressive Model, FFT or Wavelet Transforms, etc. However, it would be useful to automatically build significant attributes appropriate for each particular problem. In this paper, we use Genetic Programming to evolve projections that translate EEG data into a new vectorial space (coordinates of this space being the new attributes), where projected data can be more easily classified. Although our method is applied here in a straightforward way to check for feasibility, it has achieved reasonable classification results that are comparable to those obtained by other state of the art algorithms. In the future, we expect that by choosing carefully primitive functions, Genetic Programming will be able to give original results that cannot be matched by other machine learning classification algorithms. Publicado

Radial Basis Neural Networks have been successfully used in a large number of applications having in its rapid convergence time one of its most important advantages. However, the level of generalization is usually poor and very dependent on the quality of the training data because some of the training patterns can be redundant or irrelevant. In this paper, we present a learning method that automatically selects the training patterns more appropriate to the new sample to be approximated. This training method follows a lazy learning strategy, in the sense that it builds approximations centered around the novel sample. The proposed method has been applied to three different domains an artificial regression problem and two time series prediction problems. Results have been compared to standard training method using the complete training data set and the new method shows better generalization abilities. Publicado

Proceeding of: IEEE International Joint Conference on Neural Networks, IJCNN 2004, Budapest, 25-29 July 2004 Many methods to codify artificial neural networks have been developed to avoid the disadvantages of direct encoding schema, improving the search into the solution's space. A method to analyse how the search space is covered and how are the movements along search process applying genetic operators is needed in order to evaluate the different encoding strategies for multilayer perceptrons (MLP). In this paper, the generative capacity, this is how the search space is covered for a indirect scheme based on cellular systems, is studied. The capacity of the methods to cover the search space (topologies of MLP space) is compared with the direct encoding scheme. Publicado

Proceeding of: 2002 Congress on Evolutionary Computation, 2002. CEC'02, may 12-17, 2002. Honolulu, Hawaii, USA. Designing the optimal architecture can be formulated as a search problem in the architectures space, where each point represents an architecture. The search space of all possible architectures is very large, and the task of finding the simplest architecture may be an arduous and mostly a random task. Methods based in indirect encoding have been used to reduce the chromosome length. In this work a new indirect encoding method is proposed and an analysis of the generative capacity of the method is presented.