Search

In this survey, kinship verification is defined as the automatic process of verifying whether two or more persons are blood relatives (kin) by analyzing images of their faces. Kinship verification is an important research field in computer vision with many applications such as finding missing persons, family album organization, and online image search. Although substantial progress has been made in kinship verification in the past decade, there are still challenges such as intrinsic (face i.e., differences in facial appearance) and extrinsic (acquisition i.e., varying imaging conditions) problems. And there is still a demand for more diverse datasets.Therefore, this paper provides a survey on kinship verification methods and datasets. The survey starts with the definition of kinship verification and its corresponding intrinsic and extrinsic challenges. Then, an overview of kinship verification methods and datasets is given. Finally, a new multi-modal dataset (Nemo-Kinship Dataset) is proposed as a benchmark dataset addressing large inter-subject age variations consisting of 4216 videos of 248 persons from 85 families. The newly collected dataset is used to systematically test and analyze state-of-the-art methods.

Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.

This paper proposes a methodology for identifying data samples that are likely to be mislabeled in a c-class classification problem (dataset). The methodology relies on an assumption that the generalization error of a model learned from the data decreases if a label of some mislabeled sample is changed to its correct class. A general classification model used in the paper is OP-ELM; it also provides a fast way to estimate the generalization error by PRESS Leave-One-Out. It is tested on two toy datasets, as well as on real life datasets for one of which expert knowledge about the identified potential mislabels has been sought.

The advent of the Social Web has enabled anyone with an Internet connection to easily create and share their ideas, opinions and content with millions of other people around the world. In pace with a global deluge of videos from billions of computers, smartphones, tablets, university projectors and security cameras, the amount of multimodal content on the Web has been growing exponentially, and with that comes the need for decoding such information into useful knowledge. In this paper, a multimodal affective data analysis framework is proposed to extract user opinion and emotions from video content. In particular, multiple kernel learning is used to combine visual, audio and textual modalities. The proposed framework outperforms the state-of-the-art model in multimodal sentiment analysis research with a margin of 10–13% and 3–5% accuracy on polarity detection and emotion recognition, respectively. The paper also proposes an extensive study on decision-level fusion.

This paper focuses on the application of Extreme Learning Machines (ELM) to the classification of remote sensing hyperspectral data. The specific aim of the work is to obtain accurate thematic maps of soybean crops, which have proven to be difficult to identify by automated procedures. The classification process carried out is as follows: First, spectral data is transformed into a hyper-spherical representation. Second, a robust image gradient is computed over the hyper-spherical representation allowing an image segmentation that identifies major crop plots. Third, feature selection is achieved by a greedy wrapper approach. Finally, a classifier is trained and tested on the selected image pixel features. The classifiers used for feature selection and final classification are Single Layer Feedforward Networks (SLFN) trained with either the ELM or the incremental OP-ELM. Original image pixel features are computed following a Functional Data Analysis (FDA) characterization of the spectral data. Conventional ELM training of the SLFN improves over the classification performance of state of the art algorithms reported in the literature dealing with the data treated in this paper. Moreover, SLFN-ELM uses less features than the referred algorithms. OP-ELM is able to find competitive results using the FDA features from a single spectral band.

Many real-life problems involve dealing with multiple objectives. For example, in network routing the criteria may consist of energy consumption, latency, and channel capacity, which are in essence conflicting objectives. As in many problems there may be multiple (conflicting) objectives, there usually does not exist a single optimal solution. In those cases, it is desirable to obtain a set of trade-off solutions between the objectives. This problem has in the last decade also gained the attention of many researchers in the field of reinforcement learning (RL). RL addresses sequential decision problems in initially (possibly) unknown stochastic environments. The goal is the maximization of the agent's reward in an environment that is not always completely observable. The purpose of this special issue is to obtain a broader picture on the algorithmic techniques at the confluence between multi-objective optimization and reinforcement learning. The growing interest in multi-objective reinforcement learning (MORL) was reflected in the quantity and quality of submissions received for this special issue. After a rigorous review process, seven papers were accepted for publication, and they reflect the diversity of research being carried out within this emerging field of research. The accepted papers consider many different aspects of algorithmic design and the evaluation and this editorial puts them in a unified framework.

This paper is intended to solve the fully distributed secure bipartite consensus problem of nonlinear multi-agent systems (MASs) with quantized information under Denial-of-Service (DoS) attacks. The attacks, which constrained on attack frequency and duration are studied. Firstly, we propose a novel secure output feedback control protocol integrated of the logarithmic quantizer and relative output measurements of neighboring agents, which can realize secure control under DoS attacks by choosing the design parameters correctly. Secondly, an adaptive control protocol that includes dynamic coupling strengths into the control law and the state observer function is developed. Contrast to the single adaptive control strategy, two adaptive couplings constructed in sensor-to-observer, and controller-to-actuator channels, respectively, which can alleviate the burden of the limited bandwidth and energy consumption more effectively. Furthermore, this control strategy with dynamic coupling gains is fully distributed, under which agents are not required to know a priori knowledge of any global information and the quantizer only needs to quantize the output state error information of agents. Then, theoretical guarantees on the effectiveness of the proposed controllers in steering the system to a secure bipartite (bounded) consensus under quantized output measurements and intermittent DoS attacks are derived. Finally, the numerical simulation inspired by a real-world physical network system is developed to verify the usefulness of the presented controllers.

The paper presents an approach for performing regression on large data sets in reasonable time, using an ensemble of extreme learning machines (ELMs). The main purpose and contribution of this paper are to explore how the evaluation of this ensemble of ELMs can be accelerated in three distinct ways: (1) training and model structure selection of the individual ELMs are accelerated by performing these steps on the graphics processing unit (GPU), instead of the processor (CPU); (2) the training of ELM is performed in such a way that computed results can be reused in the model structure selection, making training plus model structure selection more efficient; (3) the modularity of the ensemble model is exploited and the process of model training and model structure selection is parallelized across multiple GPU and CPU cores, such that multiple models can be built at the same time. The experiments show that competitive performance is obtained on the regression tasks, and that the GPU-accelerated and parallelized ELM ensemble achieves attractive speedups over using a single CPU. Furthermore, the proposed approach is not limited to a specific type of ELM and can be employed for a large variety of ELMs.

This paper considers the leader-following bipartite consensus for a class of nonlinear multi-agent systems (MASs) subject to exogenous disturbances under directed fixed and switching topologies, respectively. Firstly, two new output feedback control protocols involving signs of link weights are introduced based on relative output measurements of neighboring agents. In order to estimate the disturbances produced by an exogenous system, a disturbance observer-based approach is developed. Then, sufficient conditions for leader-following bipartite consensus with directed fixed topologies are derived. Furthermore, by assuming that each switching topology contains a directed spanning tree, it is proved that the leader-following bipartite consensus can be realized with the designed output feedback control protocol if the dwell time is larger than a non-negative threshold. Finally, numerical simulations inspired by a real-world DC motors are provided to illustrate the effectiveness of the proposed controllers.

Long-term time series prediction is a difficult task. This is due to accumulation of errors and inherent uncertainties of a long-term prediction, which leads to deteriorated estimates of the future instances. In order to make accurate predictions, this paper presents a methodology that uses input processing before building the model. Input processing is a necessary step due to the curse of dimensionality, where the aim is to reduce the number of input variables or features. In the paper, we consider the combination of the delta test and the genetic algorithm to obtain two aspects of reduction: scaling and projection. After input processing, two fast models are used to make the predictions: optimally pruned extreme learning machine and optimally pruned k-nearest neighbors. Both models have fast training times, which makes them suitable choice for direct strategy for long-term prediction. The methodology is tested on three different data sets: two time series competition data sets and one financial data set.

Computational dentistry uses computerized methods and mathematical models for dental image analysis. One of the fundamental problems in computational dentistry is accurate tooth instance segmentation in high-resolution mesh data of intra-oral scans (IOS). This paper presents a new computational model based on deep neural networks, called Mask-MCNet, for end-to-end learning of tooth instance segmentation in 3D point cloud data of IOS. The proposed Mask-MCNet localizes each tooth instance by predicting its 3D bounding box and simultaneously segments the points that belong to each individual tooth instance. The proposed model processes the input raw 3D point cloud in its original spatial resolution without employing a voxelization or down-sampling technique. Such a characteristic preserves the finely detailed context in data like fine curvatures in the border between adjacent teeth and leads to a highly accurate segmentation as required for clinical practice (e.g. orthodontic planning). The experiments show that the Mask-MCNet outperforms state-of-the-art models by achieving 98% Intersection over Union (IoU) score on tooth instance segmentation which is very close to human expert performance.

In this paper we propose two methods for instance transfer based on conformal prediction. As a distinctive character, both of the methods are model independent and combine feature selection and source-instance selection to avoid negative transfer. The methods have been tested experimentally for different types of classification model on several benchmark data sets. The experimental results demonstrate that the new methods are capable of outperforming significantly standard instance transfer methods. (C) 2019 Elsevier B.V. All rights reserved.

Highlights • A hierarchical temporal attention based model is proposed to support short-term and long-term human mobility sequence prediction. • The proposed hierarchical temporal attention incorporates individual mobility patterns into the model architecture. • The model is compared with four baseline methods on individual trajectory datasets with varying degree of traveling uncertainty. • Experiments demonstrate the outperformance of the proposed method using three evaluation metrics. • The proposed model uncovers individual frequential and periodical mobility patterns in an interpretable manner., Prediction of individual mobility is crucial in human mobility related applications. Whereas, existing research on individual mobility prediction mainly focuses on next location prediction and short-term dependencies between traveling locations. Long-term location sequence prediction is of great importance for long-time traffic planning and location advertising, and long-term dependencies exist as individual mobility regularity typically occurs daily and weekly. This paper proposes a novel hierarchical temporal attention-based LSTM encoder-decoder model for individual location sequence prediction. The proposed hierarchical attention mechanism captures both long-term and short-term dependencies underlying in individual longitudinal trajectories, and uncovers frequential and periodical mobility patterns in an interpretable manner by incorporating the calendar cycle of individual travel regularities into location prediction. More specifically, the hierarchical attention consists of local temporal attention to identify highly related locations in each day, and global temporal attention to discern important travel regularities over a week. Experiments on individual trajectory datasets with varying degree of traveling uncertainty demonstrate that our method outperforms four baseline methods on three evaluation metrics. In addition, we explore the interpretability of the proposed model in understanding individual daily, and weekly mobility patterns by visualizing the temporal attention weights and frequent traveling patterns associated with locations.

The non-stationary nature of electroencephalography (EEG) signals makes an EEG-based brain-computer interface (BCI) a dynamic system, thus improving its performance is a challenging task. In addition, it is well-known that due to non-stationarity based covariate shifts, the input data distributions of EEG-based BCI systems change during inter- and intra-session transitions, which poses great difficulty for developments of online adaptive data-driven systems. Ensemble learning approaches have been used previously to tackle this challenge. However, passive scheme based implementation leads to poor efficiency while increasing high computational cost. This paper presents a novel integration of covariate shift estimation and unsupervised adaptive ensemble learning (CSE-UAEL) to tackle non-stationarity in motor-imagery (MI) related EEG classification. The proposed method first employs an exponentially weighted moving average model to detect the covariate shifts in the common spatial pattern features extracted from MI related brain responses. Then, a classifier ensemble was created and updated over time to account for changes in streaming input data distribution wherein new classifiers are added to the ensemble in accordance with estimated shifts. Furthermore, using two publicly available BCI-related EEG datasets, the proposed method was extensively compared with the state-of-the-art single-classifier based passive scheme, single-classifier based active scheme and ensemble based passive schemes. The experimental results show that the proposed active scheme based ensemble learning algorithm significantly enhances the BCI performance in MI classifications., 28 Pages, 3 figures, Neurocomputing

This paper introduces a novel hybrid deep neural kernel framework. The proposed deep learning model makes a combination of a neural networks based architecture and a kernel based model. In particular, here an explicit feature map, based on random Fourier features, is used to make the transition between the two architectures more straightforward as well as making the model scalable to large datasets by solving the optimization problem in the primal. Furthermore, the introduced framework is considered as the first building block for the development of even deeper models and more advanced architectures. Experimental results show an improvement over shallow models and the standard non-hybrid neural networks architecture on several medium to large scale real-life datasets. (C) 2018 Elsevier B.V. All rights reserved.

In this paper, we consider a linear supervised dimension reduction method for classification settings: stochastic discriminant analysis (SDA). This method matches similarities between points in the projection space with those in a response space. The similarities are represented by transforming distances between points to joint probabilities using a transformation which resembles Student’s t-distribution. The matching is done by minimizing the Kullback–Leibler divergence between the two probability distributions. We compare the performance of our SDA method against several state-of-the-art methods for supervised linear dimension reduction. In our experiments, we found that the performance of the SDA method is often better and typically at least equal to the compared methods. We have made experiments with various types of data sets having low, medium, or high dimensions and quite different numbers of samples, and with both sparse and dense data sets. If there are several classes in the studied data set, the low-dimensional projections computed using our SDA method provide often higher classification accuracies than the compared methods.

Concept-based image search is an emerging search paradigm that utilizes a set of concepts as intermediate semantic descriptors of images to bridge the semantic gap. Typically, a user query is rather complex and cannot be well described using a single concept. However, it is less effective to tackle such complex queries by simply aggregating the individual search results for the constituent concepts. In this paper, we propose to introduce the learning to rank techniques to concept-based image search for complex queries. With freely available social tagged images, we first build concept detectors by jointly leveraging the heterogeneous visual features. Then, to formulate the image relevance, we explicitly model the individual weight of each constituent concept in a complex query. The dependence among constituent concepts, as well as the relatedness between query and non-query concepts, are also considered through modeling the pairwise concept correlations in a factorization way. Finally, we train our model to directly optimize the image ranking performance for complex queries under a pairwise learning to rank framework. Extensive experiments on two benchmark datasets well verified the promise of our approach. peerReviewed

Clustering ensemble has emerged as an important tool for data analysis, by which a more robust and accurate consensus clustering can be generated. On forming the ensembles, empirical studies have suggested that better ensembles can be obtained by simultaneously considering the quality of the ensembles and the diversity among ensemble members. However, little research efforts have been paid to incorporate prior background knowledge. In this paper, we first provide a theoretical analysis on the effect of the diversity and quality of the ensemble members. We then propose a unified framework to solve constraint-based clustering ensemble selection problem, where some instance level must-link and cannot-link constraints are given as prior knowledge or background information. We formalize this problem as a combinatorial optimization problem in terms of the consistency under the constraints, the diversity among ensemble members, and the overall quality of ensembles. Our proposed framework brings together two distinct yet interrelated themes from clustering: ensemble clustering and semi-supervised clustering. We study different techniques for searching high-quality solutions. Experiments on benchmark datasets demonstrate the effectiveness of our framework.

The kappa statistic is a widely used as a measure for quantifying agreement between two nominal classifications. The statistic has been extended to the case of two normalized fuzzy classifications. In this paper we define category kappas for quantifying agreement on a particular category of two normalized fuzzy classifications. The overall fuzzy kappa is a weighted average of the proposed category kappas. Since the value of the overall kappa lies between the minimum and maximum values of the category kappas, the overall kappa, in a way, summarizes the agreement reflected in the category kappas. The overall kappa meaningfully reflects the degree of agreement between the fuzzy classifications if the category kappas are approximately equal. If this is not the case, it is more informative to report the category kappas. HighlightsFuzzy kappas for quantifying agreement on a particular category are defined.The fuzzy kappa of Dou et al. is a weighted average of the proposed category kappas.With two categories the category kappas and overall kappa coincide.Adding all other membership functions presents an alternative definition.

This paper presents a fast algorithm and an accelerated toolbox 1 for data visualization. The visualization is stated as an assignment problem between data samples and the same number of given visualization points. The mapping function is approximated by an Extreme Learning Machine, which provides an error for a current assignment. This work presents a new mathematical formulation of the error function based on cosine similarity. It provides a closed form equation for a change of error for exchanging assignments between two random samples (called a swap), and an extreme speed-up over the original method even for a very large corpus like the MNIST Handwritten Digits dataset. The method starts from random assignment, and continues in a greedy optimization algorithm by randomly swapping pairs of samples, keeping the swaps that reduce the error. The toolbox speed reaches a million of swaps per second, and thousands of model updates per second for successful swaps in GPU implementation, even for very large dataset like MNIST Handwritten Digits.

The aim of this paper is to build a tool that able to extract the regions from a brain magnetic resonance image that discriminate healthy controls from subjects with probable dementia of the Alzheimer type. We propose the use of an Extreme Learning Machine method to select the most discriminant regions and thereafter to perform the final classification according to a majority vote decision based strategy. We are selecting the optimal number of votes required to put a subject into the class "Alzheimer" by maximizing the global accuracy and minimizing the number of false positives. The discriminative regions selected in the case study are located in the hippocampus, amygdala, thalamus and putamen, among others. These locations are closely related with a Alzheimer disease according to the medical literature.

In this paper, we consider the theory and the practical implementation of Singular Value Decomposition (SVD) update algorithm. By updating, we mean using previously computed SVD to compute the SVD of a matrix augmented by one column (or row). We compare it with the standard SVD algorithm in terms of computational complexity and accuracy. We show that SVD update algorithm scales better and works faster than SVD computed from scratch. In addition, we analyze errors in singular values after many consecutive updates and verify that they are within reasonable bounds. Finally, we apply SVD update to speed up OP-ELM algorithm and propose new algorithm (Inc)-OP-LEM. In conclusion, we believe that SVD update can be applied to other computational intelligence methods to improve their computational time and scaling.

The l1-regularized least square problem has been considered in diverse fields. However, finding its solution is exacting as its objective function is not differentiable. In this paper, we propose a new one-layer neural network to find the optimal solution of the l1-regularized least squares problem. To solve the problem, we first convert it into a smooth quadratic minimization by splitting the desired variable into its positive and negative parts. Accordingly, a novel neural network is proposed to solve the resulting problem, which is guaranteed to converge to the solution of the problem. Furthermore, the rate of the convergence is dependent on a scaling parameter, not to the size of datasets. The proposed neural network is further adjusted to encompass the total variation regularization. Extensive experiments on the l1 and total variation regularized problems illustrate the reasonable performance of the proposed neural network.

In this paper, we propose a fast manifold learning strategy to estimate the underlying geometrical distribution and develop the relevant mathematical criterion on the basis of the extreme learning machine (ELM) in the high-dimensional space. The local tangent space alignment (LTSA) method has been used to perform the manifold production and the single hidden layer feedforward network (SLFN) is established via ELM to simulate the low-dimensional representation process. The scheme of the ELM ensemble then combines the individual SLFN for the model selection, where the manifold regularization mechanism has been brought into ELM to preserve the local geometrical structure of LTSA. Some developments have been done to evaluate the inherent representation embedding in the ELM learning. The simulation results have shown the excellent performance in the accuracy and efficiency of the developed approach.

In the paper, we examine the general regression problem under the missing data scenario. In order to provide reliable estimates for the regression function (approximation), a novel methodology based on Gaussian Mixture Model and Extreme Learning Machine is developed. Gaussian Mixture Model is used to model the data distribution which is adapted to handle missing values, while Extreme Learning Machine enables to devise a multiple imputation strategy for final estimation. With multiple imputation and ensemble approach over many Extreme Learning Machines, final estimation is improved over the mean imputation performed only once to complete the data. The proposed methodology has longer running times compared to simple methods, but the overall increase in accuracy justifies this trade-off.

In this paper, a novel impulsive networked scheme is investigated to solve the formation control problem of a multi-autonomous underwater vehicle (multi-AUV) system. More specifically, instead of requiring continuous-time communication, in this scheme, information exchanges among the AUVs are conducted via impulse time sequences, which is more practical in the unreliable and narrow banded underwater acoustic communication environment. In particular, the proposed scheme has significant advantages in energy conservation and improved robustness and can be widely applied in many other types of multi-agent systems. Sufficient conditions are derived to guarantee that the desired formation of the multiple AUVs can be achieved under either fixed or switching communication topologies. Finally, two numerical examples are provided to demonstrate the effectiveness of the theoretical results.

We consider supervised dimension reduction (SDR) for problems with discrete inputs. Existing methods are computationally expensive, and often do not take the local structure of data into consideration when searching for a low-dimensional space. In this paper, we propose a novel framework for SDR with the aims that it can inherit scalability of existing unsupervised methods, and that it can exploit well label information and local structure of data when searching for a new space. The way we encode local information in this framework ensures three effects: preserving inner-class local structure, widening inter-class margin, and reducing possible overlap between classes. These effects are vital for success in practice. Such an encoding helps our framework succeed even in cases that data points reside in a nonlinear manifold, for which existing methods fail. The framework is general and flexible so that it can be easily adapted to various unsupervised topic models. We then adapt our framework to three unsupervised models which results in three methods for SDR. Extensive experiments on 10 practical domains demonstrate that our framework can yield scalable and qualitative methods for SDR. In particular, one of the adapted methods can perform consistently better than the state- of-the-art method for SDR while enjoying 30-450 times faster speed.

Autostereoscopic visualization in clinical image-guided interventions and therapy poses constrains on the maximal latency in the visualization chain. In order to use visual feedback in the hand-eye coordinated loop, the latency of the visualization chain should be less than approximately 250–300 ms. This is a challenging constraint for interactive autostereoscopic multi-view volume rendering of large datasets. The various building blocks of such a visualization chain and their latency aspects are explored in this paper. Two complementary strategies to improve the latency of the entire chain are introduced and examined: lowering the resolution of the rendered views and reducing the amount of rendered views by smartly interpolating the missing views at receiver side. These strategies are balanced while optimizing the latency and image quality using a fuzzy logic approach. Furthermore, the optimal view resolution for a multi-view autostereoscopic lenticular display has been determined by investigating the lenticular lattice in the frequency domain. The quantitative aspects of the latency of the proposed building blocks and the resulting image quality have been measured.

In this paper, we present one dynamic model hypothesis to perform fish trajectory tracking in the fish ethology research and develop the relevant mathematical criterion on the basis of the Extreme Learning Machine (ELM). It is shown that the proposed scheme can conduct the non-linear and non Gaussian tracking process by multiple historical cues and current predictions - the state vector motion, the color distribution and the appearance recognition, all of which can be extracted from the single-hidden layer feedforward neural network (SLFN) at diverse levels with ELM. The strategy of the hierarchical hybrid ELM ensemble then combines the individual SLFN of the tracking cues for the performance improvements. The simulation results have shown the excellent performance in both robustness and accuracy of the developed approach.

In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time.

Variational Bayes learning or mean field approximation is widely used in statistical models which are made of mixtures of exponential distributions, for example, normal mixtures, binomial mixtures, and hidden Markov models. To derive variational Bayes learning algorithm, we need to determine the hyperparameters in the a priori distribution; however, the design method of hyperparameters has not yet been established. In the present paper, we propose two different design methods of hyperparameters which are applied to the different purposes. In the former method, the hyperparameter is determined for minimization of the generalization error. In the latter method, it is chosen so that candidates of hidden structure in training data are extracted. It is experimentally shown that the optimal hyperparameters for two purposes are different from each other.

Support vector regression (SVR) is a state-of-the-art method for regression which uses the e‐sensitive loss and produces sparse models. However, non-linear SVRs are difficult to tune because of the additional kernel parameter. In this paper, a new parameter-insensitive kernel inspired from extreme learning is used for non-linear SVR. Hence, the practitioner has only two meta-parameters to optimise. The proposed approach reduces significantly the computational complexity yet experiments show that it yields performances that are very close from the state-of-the-art. Unlike previous works which rely on Monte-Carlo approximation to estimate the kernel, this work also shows that the proposed kernel has an analytic form which is computationally easier to evaluate.

Independent variable group analysis (IVGA) is a method for grouping dependent variables together while keeping mutually independent or weakly dependent variables in separate groups. In this paper two variants of an agglomerative method for learning a hierarchy of IVGA groupings are presented. The method resembles hierarchical clustering, but the choice of clusters to merge is based on variational Bayesian model comparison. This is approximately equivalent to using a distance measure based on a model-based approximation of mutual information between groups of variables. The approach also allows determining optimal cutoff points for the hierarchy. The method is demonstrated to find sensible groupings of variables that can be used for feature selection and ease construction of a predictive model.

This paper introduces a signal classification framework that can be used for brain-computer interface design. The actual classification is performed on sparse autoregressive features. It can use any well-known classification algorithm, such as discriminant analysis, linear logistic regression and support vector machines. The autoregressive coefficients of all signals and channels are simultaneously estimated by the group lasso, and the estimation is guided by the classification performance. Thanks to the variable selection capability of the group lasso, the framework can drop individual autoregressive coefficients that are useless in the prediction stage. Also, the framework is relatively insensitive to the chosen autoregressive order. We devise an efficient algorithm to solve this problem. We test our approach on Keirn and Aunon's data, used for binary classification of electroencephalogram signals, achieving promising results. © 2013.

This paper proposes a general associative memory (GAM) system that combines the functions of other typical associative memory (AM) systems. The GAM is a network consisting of three layers: an input layer, a memory layer, and an associative layer. The input layer accepts key vectors, response vectors, and the associative relationships between these vectors. The memory layer stores the input vectors incrementally to corresponding classes. The associative layer builds associative relationships between classes. The GAM can store and recall binary or non-binary information, learn key vectors and response vectors incrementally, realize many-to-many associations with no predefined conditions, store and recall both static and temporal sequence information, and recall information from incomplete or noise-polluted inputs. Experiments using binary data, real-value data, and temporal sequences show that GAM is an efficient system. The AM experiments using a humanoid robot demonstrates that GAM can accommodate real tasks and build associations between patterns with different dimensions.

Making accurate predictions is a difficult task that is encountered throughout many research domains. In certain cases, the number of available samples is so scarce that providing reliable estimates is a challenging problem. In this paper, we are interested in giving as accurate predictions as possible based on the Extreme Learning Machine type of a neural network in small sample data scenarios. Most of the Extreme Learning Machine literature is focused on choosing a particular model from a pool of candidates, but such approach usually ignores model selection uncertainty and has inferior performance compared to combining methods. We empirically examine several model selection criteria coupled with new model combining approaches that were recently proposed. The results obtained indicate that a careful choice among the combinations must be performed in order to have the most accurate and stable predictions.

We have presented a computational model called selective attention for identification model (SAIM), that can account for a broad range of psychological and neuropsychological phenomena on attention [1]. In this paper we report on work that extends SAIM to model data from visual search tasks. The results show that SAIM can capture important aspects of findings in visual search experiments, including variations of search slopes with the similarity between targets and distractors. SAIM is also capable of simulating experimental findings for redundant targets, including the violation of the Miller inequality. SAIM explains these experimental findings as the consequence of competitive processes involved in object identification. © 2002 Published by Elsevier Science B.V.

The challenge for next generation system identification is to build new flexible models and estimators able to simulate complex systems. This task is especially difficult in the nonlinear setting. In fact, in many real applications the performance of long-term predictors may be severely affected by stability problems arising due to the output feedback. For this purpose, also the use of deep networks, which are having much success to solve classification problems, has not led so far to any significant cross-fertilization with system identification. This paper proposes a novel procedure based on a hierarchical architecture, which we calldeep prediction network, whose flexibility is used to favor the identification of stable systems. In particular, its structure contains layers whose aim is to improve long-term predictions, with complexity controlled by a kernel-based strategy. The usefulness of the new approach is demonstrated through many examples, including important real benchmark problems taken from the system identification literature

In this paper, the Semi-Supervised Subclass Support Vector Data Description is presented, a method that operates in both the supervised and the semi-supervised One-class classification case. The proposed method consists a novel extension of the standard SVDD method, by introducing two additional terms its optimization problem. These two terms correspond to expressing global and local geometric data information respectively, during the classifier optimization process. Global geometric data information is employed by minimizing the global target class variance, assuming that subclasses may have been formed within as well. In addition, by exploiting the semi-supervised learning smoothness assumption, local neighborhood information between all available (labeled and unlabeled) data is preserved, even in the supervised learning case. We show that the adoption of both terms results in a regularized feature space, where low variance directions have been emphasized, while local geometric data information have been preserved. The proposed method has been evaluated in classification problems related to face recognition, human action recognition and generic One-class classification problems, comparing favorably against related One-class classification methods in both the semi-supervised and the supervised learning cases.

A complex nature of big data resources demands new methods for structuring especially for textual content. WordNet is a good knowledge source for comprehensive abstraction of natural language as its good implementations exist for many languages. Since WordNet embeds natural language in the form of a complex network, a transformation mechanism WordNet2Vec is proposed in the paper. It creates vectors for each word from WordNet. These vectors encapsulate general position - role of a given word towards all other words in the natural language. Any list or set of such vectors contains knowledge about the context of its component within the whole language. Such word representation can be easily applied to many analytic tasks like classification or clustering. The usefulness of the WordNet2Vec method was demonstrated in sentiment analysis, i.e. classification with transfer learning for the real Amazon opinion textual dataset., 29 pages, 16 figures, submitted to journal

Recent trends in human–computer interaction (HCI) show a development towards cognitive technical systems (CTS) to provide natural and efficient operating principles. To do so, a CTS has to rely on data from multiple sensors which must be processed and combined by fusion algorithms. Furthermore, additional sources of knowledge have to be integrated, to put the observations made into the correct context. Research in this field often focuses on optimizing the performance of the individual algorithms, rather than reflecting the requirements of CTS. This paper presents the information fusion principles in CTS architectures we developed for Companion Technologies. Combination of information generally goes along with the level of abstractness, time granularity and robustness, such that large CTS architectures must perform fusion gradually on different levels — starting from sensor-based recognitions to highly abstract logical inferences. In our CTS application we sectioned information fusion approaches into three categories: perception-level fusion, knowledge-based fusion and application-level fusion. For each category, we introduce examples of characteristic algorithms. In addition, we provide a detailed protocol on the implementation performed in order to study the interplay of the developed algorithms.

Indoor place recognition is a challenging problem because of the hard representation to complicated intra-class variations and inter-class similarities.This paper presents a new indoor place recognition scheme using deep neural network. Traditional representations of indoor place almost utilize image feature to retain the spatial structure without considering the object’s semantic characteristics. However, we argue that the attributes, state and relationships of objects are much more helpful in indoor place recognition. In particular, we improve the recognition framework by utilizing Place Descriptors (PDs) in text from to connect different types of place information with their categories. Meanwhile, we analyse the ability of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) for classification in natural language, for which we use them to process the indoor place descriptions. In addition, we improve the robustness of the designed deep neural network by combining a number of effective strategies, i.e. L2-regularization, data normalization, and proper calibration of key parameters. Compared with existing state of the art, the proposed approach achieves well performance of 70.73%, 70.08% and 70.16% of accuracy, precision and recall on Visual Genome database respectively. Meanwhile, the accuracy becomes 98.6% after adding voting mechanics.

As a combination of robotic motion planning and Chinese calligraphy culture, robotic calligraphy plays a significant role in the inheritance and education of Chinese calligraphy culture. Most existing calligraphy robots focus on enabling the robots to learn writing through human participation, such as human–robot interactions and manually designed evaluation functions. However, because of the subjectivity of art aesthetics, these existing methods require a large amount of implementation work from human engineers. In addition, the written results cannot be accurately evaluated. To overcome these limitations, in this paper, we propose a robotic calligraphy model that combines a generative adversarial network (GAN) and deep reinforcement learning to enable a calligraphy robot to learn to write Chinese character strokes directly from images captured from Chinese calligraphic textbooks. In our proposed model, to automatically establish an aesthetic evaluation system for Chinese calligraphy, a GAN is first trained to understand and reconstruct stroke images. Then, the discriminator network is independently extracted from the trained GAN and embedded into a variant of the reinforcement learning method, the “actor-critic model”, as a reward function. Thus, a calligraphy robot adopts the improved actor-critic model to learn to write multiple character strokes. The experimental results demonstrate that the proposed model successfully allows a calligraphy robot to write Chinese character strokes based on input stroke images. The performance of our model, compared with the state-of-the-art deep reinforcement learning method, shows the efficacy of the combination approach. In addition, the key technology in this work shows promise as a solution for robotic autonomous assembly.

This paper introduces dynamic kernel convolutional neural networks (DK-CNNs), an enhanced type of CNN, by performing line-by-line scanning regular convolution to generate a latent dimension of kernel weights. The proposed DK-CNN applies regular convolution to the DK weights, which rely on a latent variable, and discretizes the space of the latent variable to extend a new dimension; this process is named “DK convolution”. DK convolution increases the expressive capacity of the convolution operation without increasing the number of parameters by searching for useful patterns within the new extended dimension. In contrast to conventional convolution, which applies a fixed kernel to analyse the changed features, DK convolution employs a DK to analyse fixed features. In addition, DK convolution can replace a standard convolution layer in any CNN network structure. The proposed DK-CNNs were compared with different network structures with and without a latent dimension on the CIFAR and FashionMNIST datasets. The experimental results show that DK-CNNs can achieve better performance than regular CNNs.

Social network has become a very popular way for internet users to communicate and interact online. Users spend plenty of time on famous social networks (e.g., Facebook, Twitter, Sina Weibo, etc.), reading news, discussing events and posting messages. Unfortunately, this popularity also attracts a significant amount of spammers who continuously expose malicious behavior (e.g., post messages containing commercial URLs, following a larger amount of users, etc.), leading to great misunderstanding and inconvenience on users׳ social activities. In this paper, a supervised machine learning based solution is proposed for an effective spammer detection. The main procedure of the work is: first, collect a dataset from Sina Weibo including 30,116 users and more than 16 million messages. Then, construct a labeled dataset of users and manually classify users into spammers and non-spammers. Afterwards, extract a set of feature from message content and users׳ social behavior, and apply into SVM (Support Vector Machines) based spammer detection algorithm. The experiment shows that the proposed solution is capable to provide excellent performance with true positive rate of spammers and non-spammers reaching 99.1% and 99.9% respectively.