Search

In this paper we extend the hierarchical self-organizing maps model (HSOM) to address the problem of learning topological drift under non-stationary and noisy environments. The new model, called robust and flexible hierarchical self-organizing maps (RoFlex-HSOM), combines the capabilities of robustness against noise and the flexibility to adapt to the changing environment. The RoFlex-HSOM model consists of a hierarchical tree structure of growing self-organizing maps (SOMs) that adapts its architecture based on the data. The model preserves the topology mapping from the high-dimensional time-dependent input space onto a neuron position in a low-dimensional hierarchical output space grid. Furthermore, the RoFlex-HSOM algorithm has the plasticity to track and adapt to the topological drift, it gradually forgets (but no catastrophically) previous learned patterns and it is resistant to the presence of noise. We empirically show the capabilities of our model with experimental results using synthetic sequential data sets and the “El Niño” real world data. [Copyright &y& Elsevier]

Abstract: We present a semi-supervised learning method for the growing self-organising maps (GSOM) that allows fast visualisation of data class structure on the 2D feature map. Instead of discarding data with missing values, the network can be trained from data with up to 60% of their class labels and 25% of attribute values missing, while able to make class prediction with over 90% accuracy for the benchmark datasets used. The proposed algorithm is compared to three variants of semi-supervised K-means learning on four real-world benchmark datasets and showed comparable performance and better generalisation. [Copyright &y& Elsevier]

• Prognostics under dynamic operating regimes pose significant challenges. • Baselining consists in detecting and normalizing the data of different regimes. • A self-organizing map is proposed to detect the regimes. • An unsupervised multi-layer perceptron normalizes the data within each regime. • Our findings suggest the proposed combined model can baseline the data effectively. When the influence of changing operational and environmental conditions, such as temperature and external loading, is not factored out from sensor data it can be difficult to observe a clear deterioration path. This can significantly affect the task of engineering prognostics and other health management operations. To address this problem of dynamic operating regimes, it is necessary to baseline the data, typically by first finding the operating regimes and then normalizing the data within each regime. This paper describes a baselining solution based on neural networks. A self-organizing map is used to identify the regimes, and a multi-layer perceptron is used to normalize the sensor data according to the detected regimes. Tests are performed on public datasets from a turbofan simulator. The approach can produce similar results to classical methods without the need to specify in advance the number of regimes and the explicit computation of the statistical properties of a hold-out dataset. Importantly, the techniques can be integrated into a deep learning system to perform prognostics in a single pass. [ABSTRACT FROM AUTHOR]

This paper aims to extend collective interpretation to networks with complicated components. The collective interpretation is used to generate an internally interpretable model independently of specific inputs and learning conditions. The internally interpretable model is obtained by network compression where multi-layers are sequentially compressed, taking into account all possible routes from inputs to outputs. The network compression is easily applied to fully connected networks, but it cannot be applied to some networks with complicated components. Thus, to make the compression possible, we black-box partially and minimally these components to be replaced by the ordinary components. For demonstrating the effectiveness of this technique, we use here a new model based on the self-organizing map (SOM). Then, we introduce the convolutional neural networks (CNN) for dealing with SOM knowledge, usually represented in two-dimensional lattices. Because our network compression cannot deal with those convolutional components, we temporarily black-box the CNN components. Fixing the other connection weights, we re-train the partially black-boxed network to obtain the simplest prototype model for interpretation. The method was applied to two well-known data sets, and we demonstrated that the present method could compress the networks to get the simplest and interpretable ones. In addition, very stable compressed weights for interpretation could be obtained for easy interpretation. The results suggest that the main mechanism of multi-layered neural networks is based on linear relations between individual inputs and targets, to which peripheral non-linear ones are added. [ABSTRACT FROM AUTHOR]

Data visualization plays an important role in data analysis by displaying data to observers in an interpretable way. Visualizing multidimensional data requires projecting the data into a low-dimensional space that is visible to humans. In this paper, we propose a neural network model that can generate such projections while preserving the topology relationships within data points, which is named Visible Self Organizing Incremental Neural Network (V-SOINN). V-SOINN is able to construct a topology preserving visible network automatically and classify visible nodes to different classes in the low-dimensional space. The thought of topology preserving visualization stems from Self-Organizing Map (SOM). Compared to SOM, the main advantage of V-SOINN is that it does not need prior decision of network structure, including the number of nodes and grid in the output layer. V-SOINN can show the density distribution of datasets by using the activation counts of datasets. V-SOINN is able to depict the number of classes in the low-dimensional space as well. We perform experiments on artificial and real-world datasets, and V-SOINN outperforms PCA, MDS, t-SNE, Neural Gas and SOM on the datasets. Experiments show that V-SOINN can preserve the topology and V-SOINN can produce the correct classification result when the number of samples is small. [ABSTRACT FROM AUTHOR]

The emergence of Generative Adversarial Nets (GANs) has pushed the research of generative models to a new climax. Backing this progress are the massive improvements in terms of architecture, loss function design, and regularization. Nevertheless, in the face of complex and diverse data distribution, various GAN variants are still plagued by incomplete mode coverage or even mode collapse. In this paper, we propose a new approach to train the GANs with one generator and a mixture of discriminators to overcome the mode collapse problem. In our model, each discriminator not only distinguishes real and fake samples but also differentiates modes in datasets. In essence, it combines a classical clustering idea called Self-Organizing Map and multiple discriminators into a unified optimization objective. Specifically, we define a topological structure over the multiple discriminators in order to diversify the generated samples and to capture multi-modes. We term this method Self-Organizing Map Generative Adversarial Nets (SOMGAN). By utilizing the parameter sharing trick, the proposed model requires trivial extra computation compared with GANs with a single discriminator. In our experiment, the method covers diverse data modes and gives outstanding performance in qualitative and quantitative evaluations. Since the topological constraint of discriminators is irrelevant to the generator, the SOM-based framework can be embedded into arbitrary GAN frameworks to maximize the generative capacity of the target model. [ABSTRACT FROM AUTHOR]

A self-organizing map (SOM) is an unsupervised artificial neural network that is widely used in, e.g., data mining and visualization. Supervised and semi-supervised learning methods have been proposed for the SOM. However, their teacher labels do not describe the relationship between the data and the location of nodes. This study proposes a landmark map (LAMA), which is an extension of SOMs that utilizes several landmarks, e.g., pairs of nodes and data points. LAMA is designed to obtain a user-intended nonlinear projection to achieve, e.g., the landmark-oriented data visualization. To reveal the learning properties of LAMA, the Zoo dataset from the UCI Machine Learning Repository, the McDonald's dataset from Kaggle, and an artificial formant dataset were analyzed. The analysis results of the Zoo dataset indicated that LAMA could provide a new data view such as the landmark-centered data visualization. McDonald's dataset analysis demonstrated menu recommendation examples based on a few designated items. Furthermore, the artificial formant data analysis revealed that LAMA successfully provided the intended nonlinear projection associating articular movement with vertical and horizontal movement of a computer cursor. Potential applications of LAMA include data mining, recommendation systems, and human–computer interaction. [ABSTRACT FROM AUTHOR]

This paper presents a novel dimensionality reduction technique based on ELM and SOM: ELM-SOM+. This technique preserves the intrinsic quality of Self-Organizing Map (SOM): it is nonlinear and suitable for big data. It also brings continuity to the projection using two Extreme Learning Machine (ELM) models, the first one to perform the dimensionality reduction and the second one to perform the reconstruction. ELM-SOM+ is tested successfully on nine diverse datasets. Regarding reconstruction error, the new methodology shows considerable improvement over SOM and brings continuity. [ABSTRACT FROM AUTHOR]

This paper presents a novel unsupervised learning procedure called the Growing Self-Organizing Array (GSOA) that is inspired by principles of the self-organizing maps for the Traveling Salesman Problem (TSP). The proposed procedure is a consolidation of principles deployed in solution of several variants of the generalized TSP with Neighborhoods (TSPN) for which the main ideas of the proposed unsupervised learning already demonstrates a wide range of applicability. The herein presented learning procedure is a conceptually simple algorithm which outperforms previous self-organizing map based approaches for the TSP in terms of the solution quality and required computational time. The main benefit of the proposed learning procedure is in solving routing problems that combine a combinatorial solution of some variant of the TSP with the continuous optimization, i.e., problems where it is needed to determine a sequence of visits to the given sets with determination of the particular waypoint location from each (possibly infinite) set. Low computational requirements of the proposed method are demonstrated in a solution of the Close-Enough Traveling Salesman Problem (CETSP), which is a special case of the TSPN with the disk-shaped neighborhoods. The results indicate the proposed procedure provides competitive solutions to the existing heuristics while it is about one order of magnitude faster and at least about two orders of magnitude faster than a heuristic solution of the discretized variant of the CETSP considered as the Generalized TSP. [ABSTRACT FROM AUTHOR]

• The paper presents Self-organizing Green Virtual Network Embedding (SoGVNE) method as a embedding algorithm based on supervised self-organizing map (SOM) neural network. • Proposed SoGVNE uses virtual nodes historical data to learn the mapping process in the future. • The paper uses SOM connected lattice network to be aware of data center network topology. • The paper updates the SOM weight matrix in VNE process to be self-organized. • The paper uses renewable energy resources whenever they are available. Virtualization in the data center network is used to overcome the resistance of Internet ossification. Node mapping in virtual network embedding (VNE) process is more important than link mapping because each node mapping has a lot of choices with high probability. Historical data always have valuable information which displays how to be mapped virtual networks with special features. Previous works use physical network historical information, and they do not pay attention to the characteristics of virtual networks during the embedding process. To address this problem, this paper proposes SoGVNE (Self-organized Green Virtual Network Embedding) which uses previous successful virtual node mapping information to train a self-organizing map (SOM) neural network. Then, it will respond to future requests quickly and with fewer resources. SoGVNE uses virtual networks historical data and is able to embed more than one virtual network concurrently. Using supervised SOM makes the SoGVNE aware of resources with renewable energy and uses them to manage energy consumption. This paper evaluates SoGVNE with reinforcement-learning, Monte-Carlo and Presto methods. The results indicate the high efficiency of the proposed method in learning the node mapping process and achieving high profit and acceptance ratio with low cost. [ABSTRACT FROM AUTHOR]

Abstract In recent years, least squares support vector machines (LSSVMs) with various kernel functions have been widely used in the field of machine learning. However, the selection of kernel functions is often ignored in practice. In this paper, an improved LSSVM method based on self-organizing multiple kernel learning is proposed for black-box problems. To strengthen the generalization ability of the LSSVM, some appropriate kernel functions are selected and the corresponding model parameters are optimized using a differential evolution algorithm based on an improved mutation strategy. Due to the large computation cost, a sparse selection strategy is developed to extract useful data and remove redundant data without loss of accuracy. To demonstrate the effectiveness of the proposed method, some benchmark problems from the UCI machine learning repository are tested. The results show that the proposed method performs better than other state-of-the-art methods. In addition, to verify the practicability of the proposed method, it is applied to a real-world converter steelmaking process. The results illustrate that the proposed model can precisely predict the molten steel quality and satisfy the actual production demand. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes an algorithm to deal with continuous state/action space in the reinforcement learning (RL) problem. Extensive studies have been done to solve the continuous state RL problems, but more research should be carried out for RL problems with continuous action spaces. Due to non-stationary, very large size, and continuous nature of RL problems, the proposed algorithm uses two growing self-organizing maps (GSOM) to elegantly approximate the state/action space through addition and deletion of neurons. It has been demonstrated that GSOM has a better performance in topology preservation, quantization error reduction, and non-stationary distribution approximation than the standard SOM. The novel algorithm proposed in this paper attempts to simultaneously find the best representation for the state space, accurate estimation of Q-values, and appropriate representation for highly rewarded regions in the action space. Experimental results on delayed reward, non-stationary, and large-scale problems demonstrate very satisfactory performance of the proposed algorithm. [Copyright &y& Elsevier]

The self-organizing map (SOM) represents high-dimensional input samples by a 2-dimensional output topological structure, whereby similar input samples are mapped onto the same output unit or neighboring units on a map for visualization. Although many extended SOM models have been proposed, the need to determine a grid structure of SOM before learning, and the lack of adaptability to rapid changes in input data have not yet been fully overcome. This research proposes an adaptive growing grid (AGG) model, which is a novel neural self-organizing map (SOM), for projecting high-dimensional input samples onto an output grid. Due to the need for a grid structure for visualization, the AGG uses both growing and pruning functions and an adaptive learning process in order to adapt its output grid structure and learning function to constantly and rapidly changing input data in a non-stationary environment. The proposed AGG is tested on four basic data sets and one cross-domain data set. In addition, the t -test is used to test whether the proposed AGG outperforms the benchmark model, the growing grid (GG). Based on three evaluation measures, i.e. average quantization error (AQE), topographic error (TE) and dead unit ratio (DUR), the AGG significantly outperforms the GG in a non-stationary environment. [ABSTRACT FROM AUTHOR]

Self-organizing maps (SOM) are a useful tool for exploring data. In its original version, the SOM algorithm was designed for numerical vectors. Since then, several extensions have been proposed to handle complex datasets described by (dis)similarities. Most of these extensions represent prototypes by a list of (dis)similarities with the entire dataset and suffer from several drawbacks: their complexity is increased – it becomes quadratic instead of linear –, the stability is reduced and the interpretability of the prototypes is lost. In the present article, we propose and compare two extensions of the stochastic SOM for (dis)similarity data: the first one takes advantage of the online setting in order to maintain a sparse representation of the prototypes at each step of the algorithm, while the second one uses a dimension reduction in a feature space defined by the (dis)similarity. Our contributions to the analysis of (dis)similarity data with topographic maps are thus twofolds: first, we present a new version of the SOM algorithm which ensures a sparse representation of the prototypes through online updates. Second, this approach is compared on several benchmarks to a standard dimension reduction technique (K-PCA), which is itself adapted to large datasets with the Nyström approximation. Results demonstrate that both approaches lead to reduce the prototypes dimensionality while providing accurate results in a reasonable computational time. Selecting one of these two strategies depends on the dataset size, the need to easily interpret the results and the computational facilities available. The conclusion tries to provide some recommendations to help the user making this choice. [ABSTRACT FROM AUTHOR]

The growth of the Internet and, consequently, the number of interconnected computers, has exposed significant amounts of information to intruders and attackers. Firewalls aim to detect violations according to a predefined rule-set and usually block potentially dangerous incoming traffic. However, with the evolution of attack techniques, it is more difficult to distinguish anomalies from normal traffic. Different detection approaches have been proposed, including the use of machine learning techniques based on neural models such as Self-Organizing Maps (SOMs). In this paper, we present a classification approach that hybridizes statistical techniques and SOM for network anomaly detection. Thus, while Principal Component Analysis (PCA) and Fisher Discriminant Ratio (FDR) have been considered for feature selection and noise removal, Probabilistic Self-Organizing Maps (PSOM) aim to model the feature space and enable distinguishing between normal and anomalous connections. The detection capabilities of the proposed system can be modified without retraining the map, but only by modifying the units activation probabilities. This deals with fast implementations of Intrusion Detection Systems (IDS) necessary to cope with current link bandwidths. [ABSTRACT FROM AUTHOR]

Most of the work done on self-organizing maps relies on the minimization of the mean squared error. This nonrobust approach leads to poor performance in the presence of outliers. Here we consider robust M-estimators as an alternative for least squares in the context of self-organization. New learning rules are derived, so that the original Kohonen׳s SOFM learning rule is a particular case. Experimental results are presented which demonstrate the robustness of our method against outliers, when compared to other robust self-organizing maps. [ABSTRACT FROM AUTHOR]

Interval-valued data is most utilized to represent either the uncertainty related to a single measurement, or the variability of the information inherent to a group rather than an individual. In this paper, we focus on Kohonen self-organizing maps (SOMs) for interval-valued data, and design a new Batch SOM algorithm that optimizes an explicit objective function. This algorithm can handle, respectively, suitable City-Block, Euclidean and Hausdorff distances with the purpose to compare interval-valued data during the training of the SOM. Moreover, most often conventional batch SOM algorithms consider that all variables are equally important in the training of the SOM. However, in real situations, some variables may be more or less important or even irrelevant for this task. Thanks to a parameterized definition of the above-mentioned distances, we propose also an adaptive version of the new algorithm that tackles this problem with an additional step where a relevance weight is automatically learned for each interval-valued variable. Several examples with synthetic and real interval-valued data sets illustrate the usefulness of the two novel batch SOM algorithms. [ABSTRACT FROM AUTHOR]

Interval Type-2 fuzzy-neural-network (IT2FNN) has been widely used to model nonlinear systems. In current IT2FNN-based schemes, however, one of the main drawbacks is that the structure of IT2FNN is hard to be determined. In this paper, a self-organizing interval Type-2 fuzzy-neural-network (SOIT2FNN) is introduced via considering the structure adjustment and the parameters learning process simultaneously. Two main contributions of SOIT2FNN are summarized: Firstly, an intensity of information transmission algorithm, which can evaluate the independent component contributions of fuzzy rules, is introduced to optimize the structure of SOIT2FNN. Secondly, an adaptive second-order algorithm, which can obtain fast convergence, is developed to adjust the parameters of SOIT2FNN. To demonstrate the merits of SOIT2FNN, several benchmark nonlinear systems and a real world application are examined with comparisons against other existing methods. Moreover, a statistical analysis of the performance results indicates that the proposed SOIT2FNN performs better and is more suitable for modeling nonlinear systems than some existing methods. [ABSTRACT FROM AUTHOR]

Artificial raindrop algorithm (ARA) has been shown to be an effective methodology for solving optimization problems over continuous space. To further enhance the search performance of ARA, self-organizing map (SOM) and binomial crossover operator (BCO) are integrated into ARA. The purpose of the former is to enhance the diversity of population by dynamically dividing the current population into several sub-populations. The latter is to make the function landscape be pseudo-separable by covariance matrix learning, which efficiently directs the evolution of the population toward a global optimum. By incorporating the SOM and BCO into ARA, an enhanced ARA variant, which is called ARA e -SOM+BCO for short, is presented in this paper. For the comparison purposes, the forehand fourteen CEC2005 contest benchmark functions are selected as the test suite to evaluate the performance of the proposed ARA e -SOM+BCO. The experiment results show that the performance of ARA e -SOM+BCO significantly outperforms ARA and its extension variant, and is also competitive with other state-of-the-art evolutionary algorithms in most test functions. [ABSTRACT FROM AUTHOR]

Abstract: In this work, the problem of real-time monitoring of products’ properties from spectrophotoscopic measurements is presented. Light absorbance spectra are used as inputs to software sensors that estimate outputs otherwise difficult to measure on-line. We approached the problems associated to calibrating the estimation models from very high-dimensional inputs and a reduced number of observations by selecting only a subset of relevant inputs emerging from the topological structure of the data. The topologically preserving representation is performed using the self-organizing map (SOM) where the input significance to the output is computed with the measure of topological relevance (MTR on SOM). As a result, we found that spectral inputs with a topology that is close to the output''s are also associated to the wavelengths that chemically explain the influence of the spectra to the property of interest. Being based on a selection of original spectral variables, the resulting models retain the chemical interpretability of the underlying system. Moreover, the selection approach is independent on the regression model to be embedded in the soft sensors. To support the presentation, the utility of the MTR on SOM is discussed on full-scale problems from pharmaceutical and refining industry. Based on our results, the approach leads to accurate and parsimonious models that can be efficiently implemented in industrial settings. [Copyright &y& Elsevier]

Data streams are found in many large-scale systems such as security, finance, and internet. In many of the data streams, the class distribution is imbalanced, and hence most of the traditional classification modeler fails to produce high accuracy for the samples from minority class. In addition, data streams are changing and the model should be updated to maintain the classification performance. However, obtaining the true class labels of the data samples is not an easy task since labeling process is extremely time-consuming and very often class labels are not available immediately after classification. The goal of this research is to reduce the labeling for an imbalanced data stream, and to produce high classification performance when compared to fully labeling setting. In an imbalanced data stream, the challenging part is to find and label minority class samples. In this paper, we propose RLS-SOM (Reduced labeled Samples-Self Organizing Map) framework for classification of the imbalanced data stream in a non-stationary environment. RLS-SOM locates the minority class samples in the feature space using SOM. It maintains an ensemble of the classifiers and builds a new model when the changes occur, using only partial labeled samples. In RLS-SOM, the classification results are obtained from the ensemble, as well as each individual model in the ensemble. An individual model classification results are selected over ensemble results, if its performance is higher than the ensemble's performance. This comparison is performed to improve the performance as there may be one model in the ensemble that produces higher performance than the ensemble. Our experimental results demonstrate that RLS-SOM obtains higher performance when it is compared with several partially labeling techniques over benchmark data sets. In addition, the experimental results with other state of the art fully labeling methods such as UCB, SERA, SEA, and Learn++.CDS shows RLS-SOM maintains equivalent classification performance by using 10–30% labeling, on average. [ABSTRACT FROM AUTHOR]