Search

The article discusses the artificial intelligence (AI) tools that data mine scientific papers in order to develop new scientific ideas, including the AI tool known as Semantic Scholar developed by the Allen Institute for Artificial Intelligence (AI2) in Seattle, Washington.

A new report from the First Hospital of Hunan University of Chinese Medicine in China provides an overview of research on acupuncture for functional dyspepsia (FD). The study aimed to evaluate the methodological quality of papers that performed meta-analyses and systematic reviews on acupoint selections for FD and to identify the ideal acupoint combinations for FD. The researchers found that acupuncture could alleviate the clinical symptoms of FD, but the quality of methodology in the meta-analysis papers needs improvement. Through data mining, they identified a combination of Neiguan (PC6), Zusanli (ST36), Zhongwan (CV12), and Taichong (LR3) as an essential acupoint combination for the treatment of FD. [Extracted from the article]

To a researcher in a field with high industrial relevance, retrieving and analyzing research papers and patents are important aspects of assessing the scope of the field. Knowledge of the history and effects of the elemental technologies is important for understanding trends. We propose a method for automatically creating a technical trend map from both research papers and patents by focusing on the elemental (underlying) technologies and their effects. We constructed a method that can be used in any research field. To investigate the effectiveness of our method, we conducted an experiment using the data in the NTCIR-8 Workshop Patent Mining Task. The results of our experiment showed recall and precision scores of 0.254 and 0.496, respectively, for the analysis of research papers, and recall and precision scores of 0.455 and 0.507, respectively, for the analysis of patents. Those results indicate that our method for mapping technical trends is both useful and sound. [ABSTRACT FROM AUTHOR]

In this paper, we believe that the mixed selectivity of neuron in the top layer encodes distributed information produced from other neurons to offer a significant computational advantage over recognition accuracy. Thus, this paper proposes a hierarchical network framework that the learning behaviors of features combined from hundreds of midlayers. First, a subnetwork neuron, which itself could be constructed by other nodes, is functional as a subspace features extractor. The top layer of a hierarchical network needs subspace features produced by the subnetwork neurons to get rid of factors that are not relevant, but at the same time, to recast the subspace features into a mapping space so that the hierarchical network can be processed to generate more reliable cognition. Second, this paper shows that with noniterative learning strategy, the proposed method has a wider and shallower structure, providing a significant role in generalization performance improvements. Hence, compared with other state-of-the-art methods, multiple channel features with the proposed method could provide a comparable or even better performance, which dramatically boosts the learning speed. Our experimental results show that our platform can provide a much better generalization performance than 55 other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper proposes a data-driven learning control method for stochastic nonlinear systems under random communication conditions, including data dropouts, communication delays, and packet transmission disordering. A renewal mechanism is added to the buffer to regulate the arrived packets, and a recognition mechanism is introduced to the controller for the selection of suitable update packets. Both intermittent and successive update schemes are proposed based on the conventional P-type iterative learning control algorithm, and are shown to converge to the desired input with probability one. The convergence and effectiveness of the proposed algorithms are verified by means of illustrative simulations. [ABSTRACT FROM AUTHOR]

This paper presents a family of methods for the design of adaptive kernels for tree-structured data that exploits the summarization properties of hidden states of hidden Markov models for trees. We introduce a compact and discriminative feature space based on the concept of hidden states multisets and we discuss different approaches to estimate such hidden state encoding. We show how it can be used to build an efficient and general tree kernel based on Jaccard similarity. Furthermore, we derive an unsupervised convolutional generative kernel using a topology induced on the Markov states by a tree topographic mapping. This paper provides an extensive empirical assessment on a variety of structured data learning tasks, comparing the predictive accuracy and computational efficiency of state-of-the-art generative, adaptive, and syntactical tree kernels. The results show that the proposed generative approach has a good tradeoff between computational complexity and predictive performance, in particular when considering the soft matching introduced by the topographic mapping. [ABSTRACT FROM AUTHOR]

Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The most popular tools for stream data mining are based on decision trees. In previous 15 years, all designed methods, headed by the very fast decision tree algorithm, relayed on Hoeffding’s inequality and hundreds of researchers followed this scheme. Recently, we have demonstrated that although the Hoeffding decision trees are an effective tool for dealing with stream data, they are a purely heuristic procedure; for example, classical decision trees such as ID3 or CART cannot be adopted to data stream mining using Hoeffding’s inequality. Therefore, there is an urgent need to develop new algorithms, which are both mathematically justified and characterized by good performance. In this paper, we address this problem by developing a family of new splitting criteria for classification in stationary data streams and investigating their probabilistic properties. The new criteria, derived using appropriate statistical tools, are based on the misclassification error and the Gini index impurity measures. The general division of splitting criteria into two types is proposed. Attributes chosen based on type- $I$ splitting criteria guarantee, with high probability, the highest expected value of split measure. Type- $II$ criteria ensure that the chosen attribute is the same, with high probability, as it would be chosen based on the whole infinite data stream. Moreover, in this paper, two hybrid splitting criteria are proposed, which are the combinations of single criteria based on the misclassification error and Gini index. [ABSTRACT FROM AUTHOR]

k nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed k value (even though set by experts) to all test samples. Previous solutions assign different $k$ values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal $k$ values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal $k$ values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal $k$ values. In the test stage, the kTree fast outputs the optimal $k$ value for each test sample, and then, the kNN classification can be conducted using the learned optimal $k$ value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed k value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different k values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks. [ABSTRACT FROM PUBLISHER]

This paper concerns a new incremental approach to mining data streams. It is known that patterns in a data stream may evolve over time. In many cases, we need to track and analyze the nature of these changes. In the paper, the probabilistic neural networks are considered as basic models for tracking changes in data streams. We present globally convergent stream data mining algorithms applied to problems of regression, classification, and density estimation in a time-varying (drifting) environment. The algorithms are derived from the Parzen kernel-based probabilistic neural networks working in the online mode. For each problem, a theorem is presented ensuring the L 2 convergence of the algorithm designed for tracking drifting regression, density, or discriminant functions. Illustrative examples explain in detail how to choose the bandwidth of the Parzen kernel and the learning rate of the online algorithm. The performance of all algorithms is shown in exemplary simulations. It should be noted that this paper is one of very few, in the existing literature, presenting mathematically justified stream data mining algorithms. • The incremental version of the Generalized Regression Neural Network (IGRNN) able to track drifting regression functions. • The incremental version of the Probabilistic Neural Network (IPNN) working in non-stationary environments. • Application of IPNN for tracking drifting discriminant functions. • Mathematical proofs of the L 2 convergence of all the proposed estimators. [ABSTRACT FROM AUTHOR]

Most feature selection methods first compute a similarity matrix by assigning a fixed value to pairs of objects in the whole data or to pairs of objects in a class or by computing the similarity between two objects from the original data. The similarity matrix is fixed as a constant in the subsequent feature selection process. However, the similarities computed from the original data may be unreliable, because they are affected by noise features. Moreover, the local structure within classes cannot be recovered if the similarities between the pairs of objects in a class are equal. In this paper, we propose a novel local adaptive projection (LAP) framework. Instead of computing fixed similarities before performing feature selection, LAP simultaneously learns an adaptive similarity matrix $\mathbf{S}$ and a projection matrix $\mathbf{W}$ with an iterative method. In each iteration, $\mathbf{S}$ is computed from the projected distance with the learned $\mathbf{W}$ and W is computed with the learned $\mathbf{S}$. Therefore, LAP can learn better projection matrix $\mathbf{W}$ by weakening the effect of noise features with the adaptive similarity matrix. A supervised feature selection with LAP (SLAP) method and an unsupervised feature selection with LAP (ULAP) method are proposed. Experimental results on eight data sets show the superiority of SLAP compared with seven supervised feature selection methods and the superiority of ULAP compared with five unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach of deriving physical unclonable functions (PUF) from correction circuits measuring and digitizing nonlinearities of data converters. The often digitally available correction data can then be used to generate a fingerprint of the chip. The general concept is presented and then specifically evaluated on an existing Delta-Sigma $(\Delta\Sigma)$ modulator whose outermost feedback DAC mismatches are greatly influencing the overall performance and thus need correction. The applied mixed-signal correction scheme reveals the intrinsic mismatches which are firstly used to linearize the $\Delta\Sigma$ modulator, but which can also be further analyzed. The intra-Hamming distance is initially determined to values less than 6% at nominal conditions and could be further reduced to less than 2% by applying different encodings. Regarding the distinctness of devices, the inter-Hamming distance is highly stable under all circumstances with a value very close to 50%. Though the influence of varying environmental conditions on the stability of repeated PUF readouts is negligible, inevitable deviations in the correction coefficients increase the intra-Hamming distance with respect to nominal conditions. As a result, 80 highly stable identification bits are obtained from the exemplarily used $\Delta\Sigma$ modulator. [ABSTRACT FROM PUBLISHER]

Retinal vessel segmentation is a crucial focus within the realm of medical image analysis, playing a pivotal role in early disease diagnosis, notably retinopathy. Deep learning has exhibited remarkable segmentation capabilities for retinal blood vessels, leveraging the advantages of contextual feature learning. However, there are still some shortcomings in fine retinal vessel segmentation due to the loss of semantic information due to too many pooling operations or limited receptive fields due to fewer pooling operations. In response to the nuanced balance required for expanding the receptive field while preserving information integrity during multiple downsampling operations, this paper introduces DCNet (Dilated Convolution Net), a novel lightweight three-layer dilated-convolution-based network tailored for retinal blood vessel segmentation. This three-layer architecture autonomously extracts crucial segmentation features from various levels of the feature map. Each layer comprises a dilated convolution Positive Sequence Block (PSB) and a dilated convolution Reverse Sequence Block (RSB). The dilated convolution operation is strategically exploited for its capacity to extend the receptive field, facilitating effective feature information extraction. Simultaneously, to alleviate semantic information loss within the deep network's feature map, this paper proposes the Nonlinear Feature Extraction Module (NFEM) to supplement shallow network feature information. Furthermore, to comprehensively leverage information from various scale features, a Feature Fusion Module (FFM) is introduced for multiscale vascular feature extraction, ultimately enhancing segmentation accuracy. DCNet undergoes rigorous evaluation on four publicly accessible retinal vascular datasets – DRIVE, STARE, CHASE_DB1, and HRF. Experimental results unequivocally demonstrate that DCNet achieves superior segmentation performance with fewer model parameters compared to existing state-of-the-art methods. The code for DCNet can be accessed on the following website: https://github.com/ChunhuiYu1/DCNet. [ABSTRACT FROM AUTHOR]

The article discusses research on the use of natural language processing (NLP) to mine data from electronic medical records. The article discusses a paper by Søren Brunak in the journal "Computational Biology" which examined how structured disease definition codes and free-text analysis could be used to discover comorbidities and examine links between diseases. The article discusses the International Classification of Diseases (ICD), a system used for the description of diseases, and looks at a paper by the biomedical informatics professor S. Trent Rosenbloom in the "Journal of the American Medical Informatics Association" on how computer-based documentation can be used by health care providers.

Hashing is emerging as a powerful tool for building highly efficient indices in large-scale search systems. In this paper, we study spectral hashing (SH), which is a classical method of unsupervised hashing. In general, SH solves for the hash codes by minimizing an objective function that tries to preserve the similarity structure of the data given. Although computationally simple, very often SH performs unsatisfactorily and lags distinctly behind the state-of-the-art methods. We observe that the inferior performance of SH is mainly due to its imperfect formulation; that is, the optimization of the minimization problem in SH actually cannot ensure that the similarity structure of the high-dimensional data is really preserved in the low-dimensional hash code space. In this paper, we, therefore, introduce reversed SH (ReSH), which is SH with its input and output interchanged. Unlike SH, which estimates the similarity structure from the given high-dimensional data, our ReSH defines the similarities between data points according to the unknown low-dimensional hash codes. Equipped with such a reversal mechanism, ReSH can seamlessly overcome the drawback of SH. More precisely, the minimization problem in our ReSH can be optimized if and only if similar data points are mapped to adjacent hash codes, and mostly important, dissimilar data points are considerably separated from each other in the code space. Finally, we solve the minimization problem in ReSH by multilayer neural networks and obtain state-of-the-art retrieval results on three benchmark data sets. [ABSTRACT FROM AUTHOR]

Network in network (NiN) is an effective instance and an important extension of deep convolutional neural network consisting of alternating convolutional layers and pooling layers. Instead of using a linear filter for convolution, NiN utilizes shallow multilayer perceptron (MLP), a nonlinear function, to replace the linear filter. Because of the powerfulness of MLP and $ 1\times 1 $ convolutions in spatial domain, NiN has stronger ability of feature representation and hence results in better recognition performance. However, MLP itself consists of fully connected layers that give rise to a large number of parameters. In this paper, we propose to replace dense shallow MLP with sparse shallow MLP. One or more layers of the sparse shallow MLP are sparely connected in the channel dimension or channel–spatial domain. The proposed method is implemented by applying unshared convolution across the channel dimension and applying shared convolution across the spatial dimension in some computational layers. The proposed method is called convolution in convolution (CiC). The experimental results on the CIFAR10 data set, augmented CIFAR10 data set, and CIFAR100 data set demonstrate the effectiveness of the proposed CiC method. [ABSTRACT FROM PUBLISHER]

Classifying binary imbalanced streaming data is a significant task in both machine learning and data mining. Previously, online area under the receiver operating characteristic (ROC) curve (AUC) maximization has been proposed to seek a linear classifier. However, it is not well suited for handling nonlinearity and heterogeneity of the data. In this paper, we propose the kernelized online imbalanced learning (KOIL) algorithm, which produces a nonlinear classifier for the data by maximizing the AUC score while minimizing a functional regularizer. We address four major challenges that arise from our approach. First, to control the number of support vectors without sacrificing the model performance, we introduce two buffers with fixed budgets to capture the global information on the decision boundary by storing the corresponding learned support vectors. Second, to restrict the fluctuation of the learned decision function and achieve smooth updating, we confine the influence on a new support vector to its $k$ -nearest opposite support vectors. Third, to avoid information loss, we propose an effective compensation scheme after the replacement is conducted when either buffer is full. With such a compensation scheme, the performance of the learned model is comparable to the one learned with infinite budgets. Fourth, to determine good kernels for data similarity representation, we exploit the multiple kernel learning framework to automatically learn a set of kernels. Extensive experiments on both synthetic and real-world benchmark data sets demonstrate the efficacy of our proposed approach. [ABSTRACT FROM AUTHOR]

For cryptographic applications based on physical unclonable functions (PUFs), it is very important to estimate the entropy of PUF responses accurately. The upper bound of the entropy estimated by compression algorithms, such as context-tree weighting, is too loose, while the lower bound estimated by the min-entropy calculation is too conservative, especially when the sample size is small. The actual min-entropy is between these bounds but is difficult to estimate accurately. In this paper, two models are proposed to estimate the convergence of the inter-chip min-entropy of static random-access memory (SRAM) PUFs. The basic idea is to find the relation between the expectation of the estimation result and the tested sample size, and then predict the convergence of the min-entropy. Furthermore, an improved Von Neumann extractor is used to increase the entropy per bit while retaining as many responses as possible for error correction. The experimental results demonstrate that the prediction error of the proposed estimation methods is less than 0.01/bit for the tested SRAM chips, and the improved Von Neumann extractor can reduce the number of required responses by approximately 11/16, the amount of helper data by 2/3, and the number of masks by 3/8 compared with the original method. [ABSTRACT FROM PUBLISHER]

Detecting boundary points (including outliers) is often more interesting than detecting normal observations, since they represent valid, interesting, and potentially valuable patterns. Since data representation can uncover the intrinsic data structure, we present an efficient representation-based method for detecting such points, which are generally located around the margin of densely distributed data, such as a cluster. For each point, the negative components in its representation generally correspond to the boundary points among its affine combination of points. In the presented method, the reverse unreachability of a point is proposed to evaluate to what degree this observation is a boundary point. The reverse unreachability can be calculated by counting the number of zero and negative components in the representation. The reverse unreachability explicitly takes into account the global data structure and reveals the disconnectivity between a data point and other points. This paper reveals that the reverse unreachability of points with lower density has a higher score. Note that the score of reverse unreachability of an outlier is greater than that of a boundary point. The top- $m$ ranked points can thus be identified as outliers. The greater the value of the reverse unreachability, the more likely the point is a boundary point. Compared with related methods, our method better reflects the characteristics of the data, and simultaneously detects outliers and boundary points regardless of their distribution and the dimensionality of the space. Experimental results obtained for a number of synthetic and real-world data sets demonstrate the effectiveness and efficiency of our method. [ABSTRACT FROM AUTHOR]

To overcome the energy and bandwidth limitations of traditional IoT systems, “edge computing” or information extraction at the sensor node has become popular. However, now it is important to create very low energy information extraction or pattern recognition systems. In this paper, we present an approximate computing method to reduce the computation energy of a specific type of IoT system used for anomaly detection (e.g. in predictive maintenance, epileptic seizure detection, etc). Termed as Anomaly Detection Based Power Savings (ADEPOS), our proposed method uses low precision computing and low complexity neural networks at the beginning when it is easy to distinguish healthy data. However, on the detection of anomalies, the complexity of the network and computing precision are adaptively increased for accurate predictions. We show that ensemble approaches are well suited for adaptively changing network size. To validate our proposed scheme, a chip has been fabricated in UMC 65nm process that includes an MSP430 microprocessor along with an on-chip switching mode DC-DC converter for dynamic voltage and frequency scaling. Using NASA bearing dataset for machine health monitoring, we show that using ADEPOS we can achieve 8.95X saving of energy along the lifetime without losing any detection accuracy. The energy savings are obtained by reducing the execution time of the neural network on the microprocessor. [ABSTRACT FROM AUTHOR]

Granular computing and three-way decision are two very important methods in the field of knowledge discovery and data mining. In this paper, based on the idea of three-way decision, all attributes in the information table first are divided into three disjoint parts named indispensable attributes, rejected attributes and neutral attributes, respectively. According to the three parts of attributes, many basic and important information granules and granular structures can be induced from the information table. Then a novel granular computing model is proposed by the description operator. On the one hand, many mathematical properties related to the model proposed in this paper are systematically discussed. On the other hand, we make a preliminary and meaningful attempt to deal with network security by using this model. In addition, in order to apply the model more conveniently, two algorithms for computing description set, description degree, attribute reduction and reduction degree are developed. Finally, through numerical experiments, the validity of the algorithms and the related factors that affect the effectiveness of the algorithms are discussed in detail. [ABSTRACT FROM AUTHOR]

A major drawback of nonlinear dimensionality reduction (DR) techniques is their inability to preserve some authentic information from the source domain, leading to projections that are often hard to interpret when it comes to observing anything other than the topological structure of the data. In this paper, we propose a nonlinear DR approach enforcing projection constraints resulting from an a priori knowledge about the structure of the data in multivariate periodical time series. We then propose several ways of exploiting this constrained projection to extract user-relevant information, such as the nominal behavior of a periodical dynamical system or the deviant behaviors which may occur at different time scales. The techniques are demonstrated on both a synthetic dataset composed of simulated multivariate data exhibiting a periodical behavior, and a real dataset corresponding to six months of sensor data acquisitions and recordings inside experimental buildings.1

We would like to thank the Institut National de l'Energie Solaire (INES) and the CEA, LITEN, Laboratoire Energétique du Bâtiment for providing the data. [ABSTRACT FROM AUTHOR]

Feature selection aims to gain relevant features for improved classification performance and remove redundant features for reduced computational cost. How to balance these two factors is a problem especially when the categorical labels are costly to obtain. In this paper, we address this problem using semisupervised learning method and propose a max-relevance and min-redundancy criterion based on Pearson’s correlation (RRPC) coefficient. This new method uses the incremental search technique to select optimal feature subsets. The new selected features have strong relevance to the labels in supervised manner, and avoid redundancy to the selected feature subsets under unsupervised constraints. Comparative studies are performed on binary data and multicategory data from benchmark data sets. The results show that the RRPC can achieve a good balance between relevance and redundancy in semisupervised feature selection. We also compare the RRPC with classic supervised feature selection criteria (such as mRMR and Fisher score), unsupervised feature selection criteria (such as Laplacian score), and semisupervised feature selection criteria (such as sSelect and locality sensitive). Experimental results demonstrate the effectiveness of our method. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel method SOMKE, for kernel density estimation (KDE) over data streams based on sequences of self-organizing map (SOM). In many stream data mining applications, the traditional KDE methods are infeasible because of the high computational cost, processing time, and memory requirement. To reduce the time and space complexity, we propose a SOM structure in this paper to obtain well-defined data clusters to estimate the underlying probability distributions of incoming data streams. The main idea of this paper is to build a series of SOMs over the data streams via two operations, that is, creating and merging the SOM sequences. The creation phase produces the SOM sequence entries for windows of the data, which obtains clustering information of the incoming data streams. The size of the SOM sequences can be further reduced by combining the consecutive entries in the sequence based on the measure of Kullback–Leibler divergence. Finally, the probability density functions over arbitrary time periods along the data streams can be estimated using such SOM sequences. We compare SOMKE with two other KDE methods for data streams, the M-kernel approach and the cluster kernel approach, in terms of accuracy and processing time for various stationary data streams. Furthermore, we also investigate the use of SOMKE over nonstationary (evolving) data streams, including a synthetic nonstationary data stream, a real-world financial data stream and a group of network traffic data streams. The simulation results illustrate the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

For many specific applications in data mining and machine learning, we face explicit or latent size constraint for each cluster that leads to the “balanced clustering” problem. Many existing clustering algorithms perform well in partitioning but fail in producing balanced clusters and preserving the naturally balanced structure of some data. In this paper, we propose a novel balanced clustering framework that flexibly utilizes local and global information of data. First, we propose the global balanced clustering (GBC), in which a global discriminative partitioning model is combined with the minimization of the distribution entropy of data. Then, we show that the proposed GBC can be further used to globally regularize some widely used local clustering models, so as to transform them into balanced clustering that simultaneously capture local and global data. We apply our global balanced regularization to spectral clustering (SC) and local learning (LL)-based clustering, respectively, and propose another two novel balanced clustering models: the local and global balanced SC (LGB-SC) and LGB-LL. Finding the optimal balanced partition is nondeterministic polynomial-time (NP)-hard in general. We adopt the method of augmented Lagrange multipliers to help optimize our model. Comprehensive experiments on several real world benchmarks demonstrate the advantage of our framework to yield balanced clusters while preserving good clustering quality. Our proposed LGB-SC and LGB-LL also outperform SC and LL as well as other classical clustering methods. [ABSTRACT FROM AUTHOR]

The unprecedented increase in data volume has become a severe challenge for conventional patterns of data mining and learning systems tasked with handling big data. The recently introduced Spark platform is a new processing method for big data analysis and related learning systems, which has attracted increasing attention from both the scientific community and industry. In this paper, we propose a shared nearest-neighbor quantum game-based attribute reduction (SNNQGAR) algorithm that incorporates the hierarchical coevolutionary Spark model. We first present a shared coevolutionary nearest-neighbor hierarchy with self-evolving compensation that considers the features of nearest-neighborhood attribute subsets and calculates the similarity between attribute subsets according to the shared neighbor information of attribute sample points. We then present a novel attribute weight tensor model to generate ranking vectors of attributes and apply them to balance the relative contributions of different neighborhood attribute subsets. To optimize the model, we propose an embedded quantum equilibrium game paradigm (QEGP) to ensure that noisy attributes do not degrade the big data reduction results. A combination of the hierarchical coevolutionary Spark model and an improved MapReduce framework is then constructed that it can better parallelize the SNNQGAR to efficiently determine the preferred reduction solutions of the distributed attribute subsets. The experimental comparisons demonstrate the superior performance of the SNNQGAR, which outperforms most of the state-of-the-art attribute reduction algorithms. Moreover, the results indicate that the SNNQGAR can be successfully applied to segment overlapping and interdependent fuzzy cerebral tissues, and it exhibits a stable and consistent segmentation performance for neonatal cerebral cortical surfaces. [ABSTRACT FROM AUTHOR]

The aim of recommender systems is to automatically identify user preferences within collected data, then use those preferences to make recommendations that help with decisions. However, recommender systems suffer from data sparsity problem, which is particularly prevalent in newly launched systems that have not yet had enough time to amass sufficient data. As a solution, cross-domain recommender systems transfer knowledge from a source domain with relatively rich data to assist recommendations in the target domain. These systems usually assume that the entities either fully overlap or do not overlap at all. In practice, it is more common for the entities in the two domains to partially overlap. Moreover, overlapping entities may have different expressions in each domain. Neglecting these two issues reduces prediction accuracy of cross-domain recommender systems in the target domain. To fully exploit partially overlapping entities and improve the accuracy of predictions, this paper presents a cross-domain recommender system based on kernel-induced knowledge transfer, called KerKT. Domain adaptation is used to adjust the feature spaces of overlapping entities, while diffusion kernel completion is used to correlate the non-overlapping entities between the two domains. With this approach, knowledge is effectively transferred through the overlapping entities, thus alleviating data sparsity issues. Experiments conducted on four data sets, each with three sparsity ratios, show that KerKT has 1.13%–20% better prediction accuracy compared with six benchmarks. In addition, the results indicate that transferring knowledge from the source domain to the target domain is both possible and beneficial with even small overlaps. [ABSTRACT FROM AUTHOR]

Multidimensional data (i.e., tensors) with missing entries are common in practice. Extracting features from incomplete tensors is an important yet challenging problem in many fields such as machine learning, pattern recognition, and computer vision. Although the missing entries can be recovered by tensor completion techniques, these completion methods focus only on missing data estimation instead of effective feature extraction. To the best of our knowledge, the problem of feature extraction from incomplete tensors has yet to be well explored in the literature. In this paper, we therefore tackle this problem within the unsupervised learning environment. Specifically, we incorporate low-rank tensor decomposition with feature variance maximization (TDVM) in a unified framework. Based on orthogonal Tucker and CP decompositions, we design two TDVM methods, TDVM-Tucker and TDVM-CP, to learn low-dimensional features viewing the core tensors of the Tucker model as features and viewing the weight vectors of the CP model as features. TDVM explores the relationship among data samples via maximizing feature variance and simultaneously estimates the missing entries via low-rank Tucker/CP approximation, leading to informative features extracted directly from observed entries. Furthermore, we generalize the proposed methods by formulating a general model that incorporates feature regularization into low-rank tensor approximation. In addition, we develop a joint optimization scheme to solve the proposed methods by integrating the alternating direction method of multipliers with the block coordinate descent method. Finally, we evaluate our methods on six real-world image and video data sets under a newly designed multiblock missing setting. The extracted features are evaluated in face recognition, object/action classification, and face/gait clustering. Experimental results demonstrate the superior performance of the proposed methods compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Though quite challenging, leveraging large-scale unlabeled or partially labeled data in learning systems (e.g., model/classifier training) has attracted increasing attentions due to its fundamental importance. To address this problem, many active learning (AL) methods have been proposed that employ up-to-date detectors to retrieve representative minority samples according to predefined confidence or uncertainty thresholds. However, these AL methods cause the detectors to ignore the remaining majority samples (i.e., those with low uncertainty or high prediction confidence). In this paper, by developing a principled active sample mining (ASM) framework, we demonstrate that cost-effective mining samples from these unlabeled majority data are a key to train more powerful object detectors while minimizing user effort. Specifically, our ASM framework involves a switchable sample selection mechanism for determining whether an unlabeled sample should be manually annotated via AL or automatically pseudolabeled via a novel self-learning process. The proposed process can be compatible with mini-batch-based training (i.e., using a batch of unlabeled or partially labeled data as a one-time input) for object detection. In this process, the detector, such as a deep neural network, is first applied to the unlabeled samples (i.e., object proposals) to estimate their labels and output the corresponding prediction confidences. Then, our ASM framework is used to select a number of samples and assign pseudolabels to them. These labels are specific to each learning batch based on the confidence levels and additional constraints introduced by the AL process and will be discarded afterward. Then, these temporarily labeled samples are employed for network fine-tuning. In addition, a few samples with low-confidence predictions are selected and annotated via AL. Notably, our method is suitable for object categories that are not seen in the unlabeled data during the learning process. Extensive experiments on two public benchmarks (i.e., the PASCAL VOC 2007/2012 data sets) clearly demonstrate that our ASM framework can achieve performance comparable to that of the alternative methods but with significantly fewer annotations. [ABSTRACT FROM AUTHOR]

In learning applications, exploring the cluster structures of the high dimensional data is an important task. It requires projecting or visualizing the cluster structures into a low dimensional space. The challenges are: 1) how to perform the projection or visualization with less information loss and 2) how to preserve the interpretability of the original data. Recent methods address these challenges simultaneously by unsupervised feature selection. They learn the cluster indicators based on the $k$ nearest neighbor similarity graph, then select the features highly correlated with these indicators. Under this direction, many techniques, such as local discriminative analysis, nonnegative spectral analysis, nonnegative matrix factorization, etc., have been successfully introduced to make the selection more accurate. In this paper, we focus on enhancing the unsupervised feature selection in another perspective, namely, making the selection exploit the combination effect of the features. Given the expected feature amount, previous works operate on the whole features then select those of high coefficients one by one as the output. Our proposed method, instead, operates on a group of features initially then update the selection when a better group appears. Compared to the previous methods, the proposed method exploits the combination effect of the features by $\ell {}_{2,0}$ norm. It improves the selection accuracy where the cluster structures are strongly related to a group of features. We conduct the experiments on six open access data sets from different domains. The experimental results show that our proposed method is more accurate than the recent methods which do not specially consider the combination effect of the features. [ABSTRACT FROM AUTHOR]

In this paper, a novel learning method calledpostboosting using extended G-mean(PBG) is proposed for online sequential multiclass imbalance learning (OS-MIL) in neural networks. PBG is effective due to three reasons. 1) Through postadjusting a classification boundary under extended G-mean, the challenging issue of imbalanced class distribution for sequentially arriving multiclass data can be effectively resolved. 2) A newly derived update rule for online sequential learning is proposed, which produces a high G-mean for current model and simultaneously possesses almost the same information of its previous models. 3) Adynamic adjustment mechanismprovided by extended G-mean is valid to deal with the unresolved challenging dense-majority problem and two dynamic changing issues, namely, dynamic changing data scarcity (DCDS) and dynamic changing data diversity (DCDD). Compared to other OS-MIL methods, PBG is highly effective on resolving DCDS, while PBG is the only method to resolve dense-majority and DCDD. Furthermore, PBG can directly and effectively handle unscaled data stream. Experiments have been conducted for PBG and two popular OS-MIL methods for neural networks under massive binary and multiclass data sets. Through the analyses of experimental results, PBG is shown to outperform the other compared methods on all data sets in various aspects including the issues of data scarcity, dense-majority, DCDS, DCDD, and unscaled data. [ABSTRACT FROM AUTHOR]

Deep hierarchical representations of the data have been found out to provide better informative features for several machine learning applications. In addition, multilayer neural networks surprisingly tend to achieve better performance when they are subject to an unsupervised pretraining. The booming of deep learning motivates researchers to identify the factors that contribute to its success. One possible reason identified is the flattening of manifold-shaped data in higher layers of neural networks. However, it is not clear how to measure the flattening of such manifold-shaped data and what amount of flattening a deep neural network can achieve. For the first time, this paper provides quantitative evidence to validate the flattening hypothesis. To achieve this, we propose a few quantities for measuring manifold entanglement under certain assumptions and conduct experiments with both synthetic and real-world data. Our experimental results validate the proposition and lead to new insights on deep learning. [ABSTRACT FROM AUTHOR]

Many of the state-of-the-art data mining techniques introduce nonlinearities in their models to cope with complex data relationships effectively. Although such techniques are consistently included among the top classification techniques in terms of predictive power, their lack of transparency renders them useless in any domain where comprehensibility is of importance. Rule-extraction algorithms remedy this by distilling comprehensible rule sets from complex models that explain how the classifications are made. This paper considers a new rule extraction technique, based on active learning. The technique generates artificial data points around training data with low confidence in the output score, after which these are labeled by the black-box model. The main novelty of the proposed method is that it uses a pedagogical approach without making any architectural assumptions of the underlying model. It can therefore be applied to any black-box technique. Furthermore, it can generate any rule format, depending on the chosen underlying rule induction technique. In a large-scale empirical study, we demonstrate the validity of our technique to extract trees and rules from artificial neural networks, support vector machines, and random forests, on 25 data sets of varying size and dimensionality. Our results show that not only do the generated rules explain the black-box models well (thereby facilitating the acceptance of such models), the proposed algorithm also performs significantly better than traditional rule induction techniques in terms of accuracy as well as fidelity. [ABSTRACT FROM PUBLISHER]

Liberty Mutual mines the 389,000 electronic legal invoices it receives each year for detailed data on law firm performance and shady billing practices. In eliminating the paper-based approach, it has also increased productivity and slashed costs. [ABSTRACT FROM AUTHOR]

Blockchain distributed ledger technology (DLT) has widespread applications in society 5.0 because it improves service efficiency and significantly reduces labor costs. However, employing blockchain DLT entails considerable energy consumption in the mining process. This paper proposes a blockchain accelerator (BCA) with ultralow power consumption and a high processing rate to address the problem. The BCA focuses on accelerating the double secure hash algorithm (SHA) 256 function required in the mining process at a system-on-chip (SoC) level. We propose three ideas, namely, multiple local memories (multimem), double-cell processing element (D-PE), and nonce autoupdate (NAU), to reduce the external data transfer time and improve the BCA hardware efficiency. We propose a cascaded multiple BCA chip model to enhance the system throughput by several-fold. Our experiments on an ASIC and FPGA prove that the proposed BCA successfully performs the mining process for multiple blockchain networks with much lower power consumption than that of the state-of-the-art CPUs and GPUs. The BCA is laid out with Renesas 65 nm technology with a chip area of $25~mm^{2}$ and consumes $530~mW$ at 100 MHz. The power efficiency of the layout chip is improved by 2428 and 143 times compared with that of the fastest CPU Intel i9-10940X and GPU RTX 3090, respectively. [ABSTRACT FROM AUTHOR]

Subspace clustering is the task of identifying clusters in subspaces of the input dimensions of a given dataset. Noisy data in certain attributes cause difficulties for traditional clustering algorithms, because the high discrepancies within them can make objects appear too different to be grouped in the same cluster. This requires methods specially designed for subspace clustering. This paper presents our second approach to subspace and projected clustering based on self-organizing maps (SOMs), which is a local adaptive receptive field dimension selective SOM. By introducing a time-variant topology, our method is an improvement in terms of clustering quality, computational cost, and parameterization. This enables the method to identify the correct number of clusters and their respective relevant dimensions, and thus it presents nearly perfect results in synthetic datasets and surpasses our previous method in most of the real-world datasets considered. [ABSTRACT FROM AUTHOR]

Memristor-based circuit architecture for multilayer neural networks is proposed. It is a first of its kind demonstrating successful circuit-based learning for multilayer neural network built with memristors. Though back-propagation algorithm is a powerful learning scheme for multilayer neural networks, its hardware implementation is very difficult due to complexities of the neural synapses and the operations involved in the learning algorithm. In this paper, the circuit of a multilayer neural network is designed with memristor bridge synapses and the learning is realized with a simple learning algorithm called Random Weight Change (RWC). Though RWC algorithm requires more iterations than back-propagation algorithm, we show that a circuit-based learning using RWC is two orders faster than its software counterpart. The method to build a multilayer neural network using memristor bridge synapses and a circuit-based learning architecture of RWC algorithm is proposed. Comparison between software-based and memristor circuit-based learning are presented via simulations. [ABSTRACT FROM AUTHOR]

The article examines the universal themes that are inherent in privacy and confidentiality issues arising within cloud-based conference management systems. Cloud computing requires that data be entrusted to information systems managed by external parties, and conference management systems that use this technology are exposed to potential breaches of privacy via disclosure that would invalidate elements of the academic writing community, such as reviewer anonymity. The author allows that data mining from such sources could also be potentially beneficial by aiding in the detection of fraud and improving the ways in which conferences are administered.

Unsupervised feature extractors are known to perform an efficient and discriminative representation of data. Insight into the mappings they perform and human ability to understand them, however, remain very limited. This is especially prominent when multilayer deep learning architectures are used. This paper demonstrates how to remove these bottlenecks within the architecture of non-negativity constrained autoencoder. It is shown that using both L1 and L2 regularizations that induce non-negativity of weights, most of the weights in the network become constrained to be non-negative, thereby resulting into a more understandable structure with minute deterioration in classification accuracy. Also, this proposed approach extracts features that are more sparse and produces additional output layer sparsification. The method is analyzed for accuracy and feature interpretation on the MNIST data, the NORB normalized uniform object data, and the Reuters text categorization data set. [ABSTRACT FROM AUTHOR]

This paper presents a generalization of self-organizing maps with 1-D neighborhoods (neuron chains) that can be effectively applied to complex cluster analysis problems. The essence of the generalization consists in introducing mechanisms that allow the neuron chain—during learning—to disconnect into subchains, to reconnect some of the subchains again, and to dynamically regulate the overall number of neurons in the system. These features enable the network—working in a fully unsupervised way (i.e., using unlabeled data without a predefined number of clusters)—to automatically generate collections of multiprototypes that are able to represent a broad range of clusters in data sets. First, the operation of the proposed approach is illustrated on some synthetic data sets. Then, this technique is tested using several real-life, complex, and multidimensional benchmark data sets available from the University of California at Irvine (UCI) Machine Learning repository and the Knowledge Extraction based on Evolutionary Learning data set repository. A sensitivity analysis of our approach to changes in control parameters and a comparative analysis with an alternative approach are also performed. [ABSTRACT FROM AUTHOR]

Existing learning models for classification of imbalanced data sets can be grouped as either boundary-based or nonboundary-based depending on whether a decision hyperplane is used in the learning process. The focus of this paper is a new approach that leverages the advantage of both approaches. Specifically, our new model partitions the input space into three parts by creating two additional boundaries in the training process, and then makes the final decision based on a heuristic measurement between the test sample and a subset of selected training samples. Since the original hyperplane used by the underlying original classifier will be eliminated, the proposed model is named the boundary-eliminated (BE) model. Additionally, the pseudoinverse linear discriminant (PILD) is adopted for the BE model so as to obtain a novel classifier abbreviated as BEPILD. Experiments validate both the effectiveness and the efficiency of BEPILD, compared with 13 state-of-the-art classification methods, based on 31 imbalanced and 7 standard data sets. [ABSTRACT FROM AUTHOR]

Traditional classification systems rely heavily on sufficient training data with accurate labels. However, the quality of the collected data depends on the labelers, among which inexperienced labelers may exist and produce unexpected labels that may degrade the performance of a learning system. In this paper, we investigate the multiclass classification problem where a certain amount of training examples are randomly labeled. Specifically, we show that this issue can be formulated as a label noise problem. To perform multiclass classification, we employ the widely used importance reweighting strategy to enable the learning on noisy data to more closely reflect the results on noise-free data. We illustrate the applicability of this strategy to any surrogate loss functions and to different classification settings. The proportion of randomly labeled examples is proved to be upper bounded and can be estimated under a mild condition. The convergence analysis ensures the consistency of the learned classifier to the optimal classifier with respect to clean data. Two instantiations of the proposed strategy are also introduced. Experiments on synthetic and real data verify that our approach yields improvements over the traditional classifiers as well as the robust classifiers. Moreover, we empirically demonstrate that the proposed strategy is effective even on asymmetrically noisy data. [ABSTRACT FROM AUTHOR]

• Solve the imbalance problem in cardiovascular disease data. • A novel method named ADWGKFO is proposed to transform the unbalanced data sets. • A more targeted sampling weight determination method is proposed. • A sample testing method based on Gaussian kernel is proposed to filter new samples. • Empirical analysis shows the proposed method performs better than similar methods. In risk assessment of cardiovascular disease (CVD), the classification error caused by unbalanced data is a significant challenge, which has sparked widespread concern and research upsurge in the field of data mining. Therefore, in view of the imbalance of CVD data sets, an oversampling method via adaptive double weights and Gaussian kernel function (ADWGKFO) is proposed, which converts the unbalanced data sets into balanced data sets. Firstly, clustering algorithm is utilized to cluster minority samples, boundary samples are identified by Borderline-Synthetic Minority Over-sampling Technique (Borderline-SMOTE), K nearest neighbor and support vector machine algorithms, and the number of samples synthesized in each group is calculated based on the double weights of boundary points and majority distribution. Secondly, in order to clearly define the classification boundary, the mutual class potential of new samples in each cluster is calculated by Gaussian kernel function, and new samples are filtered according to the mutual class potential until the data set is balanced. Finally, taking the data sets from Kaggle platform as the research samples, the proposed method is empirically analyzed. In order to validate the efficacy and universality of the proposed method, this paper selects CatBoost that is a new integrated algorithm to test the effect of the ADWGKFO method, and compares it with different sampling methods and different classifiers using performance evaluation indexes such as accuracy, F1-score and area under the curve (AUC). Compared with the combinations of other methods, the accuracy, F1-score and AUC are significantly improved. It is concluded that the ADWGKFO method described in this paper can successfully improve the data quality, and increases the reliability of CVD risk assessment. [ABSTRACT FROM AUTHOR]

In this paper, we discuss a rough set approach to missing attribute values. Among many ways of interpreting missing values, in this paper we focus on two interpretations, lost values and "do not care" conditions. Using these interpretations, global and saturated probabilistic approximations are constructed with two types of granules: characteristic sets and maximal consistent blocks. We compare eight approaches, combining two interpretations of missing attribute values, two types of probabilistic approximations with two types of granules using an error rate that is computed as a result of ten-fold cross-validation. Using a 5% level of statistical significance, we present the experimental results for these eight approaches, showing statistically significant differences between all approaches to mining incomplete data. The results also show that no one method and approach is the best for every data set and that all eight approaches should be attempted. The final section of the paper presents the idea of concept-compatible data sets. We show that for these types of data sets, global and saturated probabilistic approximations for a concept are identical to the concept. We also show that for an incomplete data set with no duplicate rows using the lost interpretation of missing attribute values, the data set is concept-compatible. • Two interpretations of missing attribute values: lost values and "do not care" conditions are considered • Global and saturated probabilistic approximations are constructed from characteristic sets and maximal consistent blocks • Eight approaches to mining incomplete data sets are compared and significant differences between them are indicated • A novel idea of the concept-compatible data sets is introduced [ABSTRACT FROM AUTHOR]

In association rules mining from data that have numeric-valued attributes, automatically adjusting the attribute intervals at the time of the mining process without a preprocess is very critical for preventing data loss and attribute interactions. In this paper, differential evolution and sine cosine algorithm based novel hybrid multi-objective evolutionary optimization methods are proposed for rapidly and directly mining the reduced high-quality numerical association rules by simultaneously adjusting the relevant intervals of related attributes without finding the frequent itemsets. These algorithms perform a global search and find the high-quality rules set in only one execution by modeling the rule mining task as a multi-objective problem that simultaneously meets different conflicting metrics. The algorithms proposed in this paper ensure the discovered rules to have high confidence and support and to be comprehensible. The proposed methods automate the rule mining process by directly finding the minimum intervals for the attributes and eliminating the need for minimum confidence and minimum support determined beforehand for each data set. The performances of new algorithms proposed in this study were tested with those of the state-of-the-art algorithms. The results show superiority of the proposed methods on the data sets that contain fewer attributes and higher number of instances. [ABSTRACT FROM AUTHOR]

Identifying frequent item-sets is a popular data-mining task. It consists of finding sets of items frequently appearing in data. Yet, finding all frequent item-sets in large or dense datasets may be time-consuming, and a user may be interested merely in some specific item-sets rather than all of them. Recently, methods have been proposed for targeted item-set mining; that is to calculate the support of some item-sets of interest. Though this approach is often more suitable for real applications than traditional item-set mining approaches, performance remains an issue. To address that issue, this paper presents a novel algorithm for multitude-targeted mining, named Guided Frequent Pattern-Growth (GFP-Growth). The GFP-Growth algorithm is designed to quickly mine a given set of item-sets using a small amount of memory. This paper proves that GFP-Growth yields the exact frequency-counts for each item-set of interest. It further shows that GFP-Growth can boost the performance for several problems requiring item-set mining. We specifically study the problem of generating minority-class rules from imbalanced data and develop the Minority-Report Algorithm (MRA) that uses GFP-Growth to solve this problem efficiently. We prove several theoretical properties of MRA and present experimental results showing substantial performance gain. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a novel form of association rules (ARs) that do not require discretization of continuous variables or the use of intervals in either sides of the rule. This rule form captures nonlinear relationships among variables, and provides an alternative pattern representation for mining essential relations hidden in a given data set. We refer to the new rule form as a functional AR (FAR). A new neural network-based, co-operative, coevolutionary algorithm is presented for FAR mining. The algorithm is applied to both synthetic and real-world data sets, and its performance is analyzed. The experimental results show that the proposed mining algorithm is able to discover valid and essential underlying relations in the data. Comparison experiments are also carried out with the two state-of-the-art AR mining algorithms that can handle continuous variables to demonstrate the competitive performance of the proposed method. [ABSTRACT FROM PUBLISHER]

One of the main research problems in heterogeneous transfer learning is to determine whether a given source domain is effective in transferring knowledge to a target domain, and then to determine how much of the knowledge should be transferred from a source domain to a target domain. The main objective of this paper is to solve this problem by evaluating the relatedness among given domains through transferred weights. We propose a novel method to learn such transferred weights with the aid of co-occurrence data, which contain the same set of instances but in different feature spaces. Because instances with the same category should have similar features, our method is to compute their principal components in each feature space such that co-occurrence data can be rerepresented by these principal components. The principal component coefficients from different feature spaces for the same instance in the co-occurrence data have the same order of significance for describing the category information. By using these principal component coefficients, the Markov Chain Monte Carlo method is employed to construct a directed cyclic network where each node is a domain and each edge weight is the conditional dependence from one domain to another domain. Here, the edge weight of the network can be employed as the transferred weight from a source domain to a target domain. The weight values can be taken as a prior for setting parameters in the existing heterogeneous transfer learning methods to control the amount of knowledge transferred from a source domain to a target domain. The experimental results on synthetic and real-world data sets are reported to illustrate the effectiveness of the proposed method that can capture strong or weak relations among feature spaces, and enhance the learning performance of heterogeneous transfer learning. [ABSTRACT FROM AUTHOR]