Search

In a modern e-commerce recommender system, it is important to understand the relationships among products. Recognizing product relationships-such as complements or substitutes-accurately is an essential task for generating better recommendation results, as well as improving explainability in recommendation. Products and their associated relationships naturally form a product graph, yet existing efforts do not fully exploit the product graph's topological structure. They usually only consider the information from directly connected products. In fact, the connectivity of products a few hops away also contains rich semantics and could be utilized for improved relationship prediction. In this work, we formulate the problem as a multilabel link prediction task and propose a novel graph neural network-based framework, item relationship graph neural network (IRGNN), for discovering multiple complex relationships simultaneously. We incorporate multihop relationships of products by recursively updating node embeddings using the messages from their neighbors. An edge relational network is designed to effectively capture relational information between products. Extensive experiments are conducted on real-world product data, validating the effectiveness of IRGNN, especially on large and sparse product graphs.

Images are uploaded to the Internet over time which makes concept drifting and distribution change in semantic classes unavoidable. Current hashing methods being trained using a given static database may not be suitable for nonstationary semantic image retrieval problems. Moreover, directly retraining a whole hash table to update knowledge coming from new arriving image data may not be efficient. Therefore, this paper proposes a new incremental hash-bit learning method. At the arrival of new data, hash bits are selected from both existing and newly trained hash bits by an iterative maximization of a 3-component objective function. This objective function is also used to weight selected hash bits to re-rank retrieved images for better semantic image retrieval results. The three components evaluate a hash bit in three different angles: 1) information preservation; 2) partition balancing; and 3) bit angular difference. The proposed method combines knowledge retained from previously trained hash bits and new semantic knowledge learned from the new data by training new hash bits. In comparison to table-based incremental hashing, the proposed method automatically adjusts the number of bits from old data and new data according to the concept drifting in the given data via the maximization of the objective function. Experimental results show that the proposed method outperforms existing stationary hashing methods, table-based incremental hashing, and online hashing methods in 15 different simulated nonstationary data environments.

A very large volume of images is uploaded to the Internet daily. However, current hashing methods for image retrieval are designed for static databases only. They fail to consider the fact that the distribution of images can change when new images are added to the database over time. The changes in the distribution of images include both discovery of a new class and a distribution of images within a class owing to concept drift. Retraining of hash tables using all images in the database requires a large computation effort. This is also biased to old data owing to the huge volume of old images which leads to a poor retrieval performance over time. In this paper, we propose the incremental hashing (ICH) method to deal with the two aforementioned types of changes in the data distribution. The ICH uses a multihashing to retain knowledge coming from images arriving over time and a weight-based ranking to make the retrieval results adaptive to the new data environment. Experimental results show that the proposed method is effective in dealing with changes in the database.

The training of a multilayer perceptron neural network (MLPNN) concerns the selection of its architecture and the connection weights via the minimization of both the training error and a penalty term. Different penalty terms have been proposed to control the smoothness of the MLPNN for better generalization capability. However, controlling its smoothness using, for instance, the norm of weights or the Vapnik-Chervonenkis dimension cannot distinguish individual MLPNNs with the same number of free parameters or the same norm. In this paper, to enhance generalization capabilities, we propose a stochastic sensitivity measure (ST-SM) to realize a new penalty term for MLPNN training. The ST-SM determines the expectation of the squared output differences between the training samples and the unseen samples located within their Q -neighborhoods for a given MLPNN. It provides a direct measurement of the MLPNNs output fluctuations, i.e., smoothness. We adopt a two-phase Pareto-based multiobjective training algorithm for minimizing both the training error and the ST-SM as biobjective functions. Experiments on 20 UCI data sets show that the MLPNNs trained by the proposed algorithm yield better accuracies on testing data than several recent and classical MLPNN training methods.

Pattern recognition and machine learning techniques have been increasingly adopted in adversarial settings such as spam, intrusion, and malware detection, although their security against well-crafted attacks that aim to evade detection by manipulating data at test time has not yet been thoroughly assessed. While previous work has been mainly focused on devising adversary-aware classification algorithms to counter evasion attempts, only few authors have considered the impact of using reduced feature sets on classifier security against the same attacks. An interesting, preliminary result is that classifier security to evasion may be even worsened by the application of feature selection. In this paper, we provide a more detailed investigation of this aspect, shedding some light on the security properties of feature selection against evasion attacks. Inspired by previous work on adversary-aware classifiers, we propose a novel adversary-aware feature selection model that can improve classifier security against evasion attacks, by incorporating specific assumptions on the adversary's data manipulation strategy. We focus on an efficient, wrapper-based implementation of our approach, and experimentally validate its soundness on different application examples, including spam and malware detection.

Undersampling is a widely adopted method to deal with imbalance pattern classification problems. Current methods mainly depend on either random resampling on the majority class or resampling at the decision boundary. Random-based undersampling fails to take into consideration informative samples in the data while resampling at the decision boundary is sensitive to class overlapping. Both techniques ignore the distribution information of the training dataset. In this paper, we propose a diversified sensitivity-based undersampling method. Samples of the majority class are clustered to capture the distribution information and enhance the diversity of the resampling. A stochastic sensitivity measure is applied to select samples from both clusters of the majority class and the minority class. By iteratively clustering and sampling, a balanced set of samples yielding high classifier sensitivity is selected. The proposed method yields a good generalization capability for 14 UCI datasets.

A novel adaptive filter is proposed using a recurrent cerebellar-model-articulation-controller (CMAC). The proposed locally recurrent globally feedforward recurrent CMAC (RCMAC) has favorable properties of small size, good generalization, rapid learning, and dynamic response, thus it is more suitable for high-speed signal processing. To provide fast training, an efficient parameter learning algorithm based on the normalized gradient descent method is presented, in which the learning rates are on-line adapted. Then the Lyapunov function is utilized to derive the conditions of the adaptive learning rates, so the stability of the filtering error can be guaranteed. To demonstrate the performance of the proposed adaptive RCMAC filter, it is applied to a nonlinear channel equalization system and an adaptive noise cancelation system. The advantages of the proposed filter over other adaptive filters are verified through simulations.

The support vector machine (SVM) has been demonstrated to be a very effective classifier in many applications, but its performance is still limited as the data distribution information is underutilized in determining the decision hyperplane. Most of the existing kernels employed in nonlinear SVMs measure the similarity between a pair of pattern images based on the Euclidean inner product or the Euclidean distance of corresponding input patterns, which ignores data distribution tendency and makes the SVM essentially a "local" classifier. In this paper, we provide a step toward a paradigm of kernels by incorporating data specific knowledge into existing kernels. We first find the data structure for each class adaptively in the input space via agglomerative hierarchical clustering (AHC), and then construct the weighted Mahalanobis distance (WMD) kernels using the detected data distribution information. In WMD kernels, the similarity between two pattern images is determined not only by the Mahalanobis distance (MD) between their corresponding input patterns but also by the sizes of the clusters they reside in. Although WMD kernels are not guaranteed to be positive definite (pd) or conditionally positive definite (cpd), satisfactory classification results can still be achieved because regularizers in SVMs with WMD kernels are empirically positive in pseudo-Euclidean (pE) spaces. Experimental results on both synthetic and real-world data sets show the effectiveness of "plugging" data structure into existing kernels.

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machine (SVM), radial basis function neural network (RBFNN), and multilayer perceptron neural network (MLPNN) are local learning machines for solving problems and treat unseen samples near the training samples to be more important. In this paper, we propose a localized generalization error model which bounds from above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure. It is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments using 17 University of California at Irvine (UCI) data sets show that, in comparison with cross validation (CV), sequential learning, and two other ad hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.

The one-class classification problem aims to distinguish a target class from outliers. The spherical one-class classifier (SOCC) solves this problem by finding a hypersphere with minimum volume that contains the target data while keeping outlier samples outside. SOCC achieves satisfactory performance only when the target samples have the same distribution tendency in all orientations. Therefore, the performance of the SOCC is limited in the way that many superfluous outliers might be mistakenly enclosed. The authors propose to exploit target data structures obtained via unsupervised methods such as agglomerative hierarchical clustering and use them in calculating a set of hyperellipsoidal separating boundaries. This method is named the structured one-class classifier (TOCC). The optimization problem in TOCC can be formulated as a series of second-order cone programming problems that can be solved with acceptable efficiency by primal-dual interior-point methods. The experimental results on artificially generated data sets and benchmark data sets demonstrate the advantages of TOCC.

When fuzzy production rules are used to approximate reasoning, interaction exists among rules that have the same consequent. Due to this interaction, the weighted average model frequently used in approximate reasoning does not work well in many real-world problems. In order to model and handle this interaction, this paper proposes to use a nonadditive nonnegative set function to replace the weights assigned to rules having the same consequent, and to draw the reasoning conclusion based on an integral with respect to the nonadditive nonnegative set function, rather than on the weighted average model. Handling interaction in fuzzy production rule reasoning in this way can lead to a good understanding of the rules base and an improvement of reasoning accuracy. This paper also investigates how to determine from data the nonadditive set function that cannot be specified by a domain expert.

This paper introduces a rough set technique for solving the problem of mining Pinyin-to-character (PTC) conversion rules. It first presents a text-structuring method by constructing a language information table from a corpus for each pinyin, which it will then apply to a free-form textual corpus. Data generalization and rule extraction algorithms can then be used to eliminate redundant information and extract consistent PTC conversion rules. The design of our model also addresses a number of important issues such as the long-distance dependency problem, the storage requirements of the rule base, and the consistency of the extracted rules, while the performance of the extracted rules as well as the effects of different model parameters are evaluated experimentally. These results show that by the smoothing method, high precision conversion (0.947) and recall rates (0.84) can be achieved even for rules represented directly by pinyin rather than words. A comparison with the baseline tri-gram model also shows good complement between our method and the tri-gram language model.

Fuzzy production rules (FPRs) have been used for years to capture and represent fuzzy, vague, imprecise and uncertain domain knowledge in many fuzzy systems. There have been a lot of researches on how to generate or obtain FPRs. There exist two methods to obtain FPRs. One is by painstakingly, repeatedly and time-consuming interviewing domain experts to extract the domain knowledge. The other is by using some machine learning techniques to generate and extract FPRs from some training samples. These extracted rules, however, are found to be nonoptimal and sometimes redundant. Furthermore, these generated rules suffer from the problem of low accuracy of classifying or recognizing unseen examples. The reasons for having these problems are 1) the FPRs generated are not powerful enough to represent the domain knowledge, 2) the techniques used to generate FPRs are pre-matured, ad-hoc or may not be suitable for the problem, and 3) further refinement of the extracted rules has not been done. In this paper we look into the solutions of the above problems by 1) enhancing the representation power of FPRs by including local and global weights, 2) developing a fuzzy neural network (FNN) with enhanced learning algorithm, and 3) using this FNN to refine the local and global weights of FPRs. By experimenting our method with some existing benchmark examples, the proposed method is found to have high accuracy in classifying unseen samples without increasing the number of the FPRs extracted and the time required to consult with domain experts is greatly reduced.

Sensitivity analysis on a neural network is mainly investigated after the network has been designed and trained. Very few have considered this as a critical issue prior to network design. Piche's statistical method (1992, 1995) is useful for multilayer perceptron (MLP) design, but too severe limitations are imposed on both input and weight perturbations. This paper attempts to generalize Piche's method by deriving an universal expression of MLP sensitivity for antisymmetric squashing activation functions, without any restriction on input and output perturbations. Experimental results which are based on, a three-layer MLP with 30 nodes per layer agree closely with our theoretical investigations. The effects of the network design parameters such as the number of layers, the number of neurons per layer, and the chosen activation function are analyzed, and they provide useful information for network design decision-making. Based on the sensitivity analysis of MLP, we present a network design method for a given application to determine the network structure and estimate the permitted weight range for network training.

Approximate reasoning in a fuzzy system is concerned with inferring an approximate conclusion from fuzzy and vague inputs. There are many ways in which different forms of conclusions can be drawn. Fuzzy sets are usually represented by fuzzy membership functions. These membership functions are assumed to have a clearly defined base. For other fuzzy sets such as intelligent, smart, or beautiful, etc., it would be difficult to define clearly its base because its base may consist of several other fuzzy sets or unclear nonfuzzy bases. A method to handle this kind of fuzzy set is proposed. A fuzzy neural network (FNN) is also proposed to tune knowledge representation parameters (KRPs). The contributions are that we are able to handle a broader range of fuzzy sets and build more powerful fuzzy systems so that the conclusions drawn are more meaningful, reliable, and accurate. An experiment is presented to demonstrate how our method works.

Fuzzy decision tree induction is an important way of learning from examples with fuzzy representation. Since the construction of optimal fuzzy decision tree is NP-hard, the research on heuristic algorithms is necessary. In this paper, three heuristic algorithms for generating fuzzy decision trees are analyzed and compared. One of them is proposed by the authors. The comparisons are twofold. One is the analytic comparison based on expanded attribute selection and reasoning mechanism; the other is the experimental comparison based on the size of generated trees and learning accuracy. The purpose of this study is to explore comparative strengths and weaknesses of the three heuristics and to show some useful guidelines on how to choose an appropriate heuristic for a particular problem.

An important issue in the design and implementation of a neural network is the sensitivity of its output to input and weight perturbations. In this paper, we discuss the sensitivity of the most popular and general feedforward neural networks--multilayer perceptron (MLP). The sensitivity is defined as the mathematical expectation of the output errors of the MLP due to input and weight perturbations with respect to all input and weight values in a given continuous interval. The sensitivity for a single neuron is discussed first and an analytical expression that is a function of the absolute values of input and weight perturbations is approximately derived. Then an algorithm is given to compute the sensitivity for the entire MLP. As intuitively expected, the sensitivity increases with input and weight perturbations, but the increase has an upper bound that is determined by the structural configuration of the MLP, namely the number of neurons per layer and the number of layers. There exists an optimal value for the number of neurons in a layer, which yields the highest sensitivity value. The effect caused by the number of layers is quite unexpected. The sensitivity of a neural network may decrease at first and then almost keeps constant while the number increases.

The further study on the sensitivity analysis of Neocognitron is discussed in this paper. Fukushima's Neocognitron is capable of recognizing distorted patterns as well as tolerating positional shift. Supervised learning of the Neocognitron is fulfilled by training patterns layer by layer. However, many parameters, such as selectivity and receptive fields are set manually. Furthermore, in Fukushima's original Neocognitron, all the training patterns are designed empirically. In this paper, we use Genetic Algorithms (GAs) to tune the parameters of Neocognitron and search its reasonable training pattern sets. Four contributions are claimed: first, by analyzing the learning mechanism of Fukushima's original Neocognitron, the correlations amongst the training patterns are claimed to affect the performance of Neocognitron, tuning the Neocognitron's number of planes is equivalent to searching reasonable training patterns for its supervised learning; second, a GA-based supervised learning of the Neocognitron is carried out in this way, searching the parameters and training patterns by GAs but specifying the connection weights by training the Neocognitron; third, other than traditional GAs which are unsuitable for the large searching space of training patterns set, the cooperative coevolution is incorporated to play this role; fourth, an effective fitness function is given out when applying the above methodology into numeral recognition. The evolutionary computation in our initial experiments is implemented based on the original training pattern set, e.g. the individuals of the population are generated from Fukushima's original training patterns during initialization of GAs. The results prove that our correlation analysis is reasonable, and show that the performance of a Neocognitron is sensitive to its training patterns, selectivity and receptive fields, especially, the performance is not monotonically increasing with respect to the number of training patterns, and this GA-based supervised learning is able to improve Neocognitron's performance.

We have cloned and characterized a human gene encoding TP2 (telomerase-associated protein 2), a protein with similarity to reverse transcriptases and the catalytic telomerase subunits from Saccharomyces cerevisiae and Euplotes aediculatus. Indirect immunofluorescence revealed that TP2 was localized to the nucleus. Using antibodies to endogenous and epitope-tagged TP2, we found that TP2 was associated specifically with human telomerase activity and the recently identified telomerase-associated protein TP1. Mutation of conserved residues within the reverse transcriptase domain of TP2 severely reduced associated telomerase activity. These results suggest that telomerase is an evolutionarily conserved multisubunit complex composed of both structural and catalytic subunits.

If the given fact for an antecedent in a fuzzy production rule (FPR) does not match exactly with the antecedent of the rule, the consequent can still be drawn by technique such as fuzzy reasoning. Many existing fuzzy reasoning methods are based on Zadeh's Compositional Rule of Inference (CRI) which requires setting up a fuzzy relation between the antecedent and the consequent part. There are some other fuzzy reasoning methods which do not use Zadeh's CRI. Among them, the similarity-based fuzzy reasoning methods, which make use of the degree of similarity between a given fact and the antecedent of the rule to draw the conclusion, are well known. In this paper, six similarity-based fuzzy reasoning methods are compared and analyzed. Two of them are newly proposed by the authors. The comparisons are two-fold. One is to compare the six reasoning methods in drawing appropriate conclusions for a given set of FPRs. The other is to compare them based on five issues: 1) types of FPR handled by these methods; 2) the complexity of the methods; 3) the accuracy of the conclusion drawn; 4) the accuracy of the similarity measure; and 5) the multi-level reasoning capability. The results have shed some lights on how to select an appropriate fuzzy reasoning method under different environments.

A hybrid cognitive system is proposed where a working neocognitron is synthesized with a set of production rules. The knowledge base of a neocognitron is constructed through incorporating production rules into its interlayer connections. Training for prototype patterns is not required. The semantic of interlayer connections is established. The resulting network can now be analyzed according to the rule structure and problematic portions can be corrected. Neocognitrons constructed using this hybrid approach have been tested on the same set of handwritten numerals initiated by Fukushima with scaling and skewing distortions, and with noise contamination. It is found that the performance is comparable to that of Fukushima's network obtained by supervised training.

Rule-embedded neocognitron (REN) is proposed where the knowledge base of a neocognitron is constructed through incorporating production rules into its interlayer connections. Prototype patterns training is not required. The semantic of interlayer connections is established. The resulting network can now be analyzed according to the rule structure and problematic portions can be corrected. We demonstrate the ease with which performance can be improved by applying REN on handwritten numeral recognition. The same set of handwritten numerals initiated by Fukushima is used to test this methodology. It is found that the performance is comparable with that of Fukushima's neocognitron with supervised training.

The heptapeptide toxins produced by the blue-green alga (cyanobacterium) Microcystis aeruginosa are selectively hepatotoxic in mammals. The characteristic post-mortem pathology of the liver is extensive lobular disruption due to sinusoidal breakdown, leakage of blood into the tissue and hepatocyte disintegration. Isolated hepatocytes incubated with toxin show severe structural deformity and surface blebbing. This paper demonstrates the effects of Microcystis toxins on the contraction and aggregation of actin microfilaments, and on the relocation and breakdown of cytokeratin intermediate filaments, in cultured hepatocytes. Earlier work did not show changes in the assembly/disassembly of actin; however, this paper demonstrates the change in cytokeratin from intermediate filaments to distributed granules in the cytoplasm of toxin-affected cells. Acrylamide gel electrophoresis of cytoskeletal fractions from hepatocytes did not show changes in total cytokeratins; however, marked changes in the immunogenicity of cytokeratins at 52 and 58 kDa were seen on toxin exposure of cells. Measurement of 32P-phosphorylation of proteins in toxin-affected cells incubated with [32P]orthophosphate showed a dramatic increase compared to control incubations. This is in agreement with research elsewhere describing phosphatase inhibition in vitro by Microcystis toxins. The data indicate that phosphorylated cytokeratin is a major component of cytoplasmic fraction phosphorylated protein after toxin exposure to hepatocytes. It is concluded that the mechanism of Microcystis toxicity to the hepatocyte is through cytoskeletal damage leading to loss of cell morphology, cell to cell adhesion and finally cellular necrosis. The underlying biochemical lesion is likely to be phosphatase inhibition causing hyperphosphorylation of a number of hepatocyte proteins, including those cytokeratins responsible for microfilament orientation and intermediate filament integrity.