Search

Coupling learning is designed to estimate, discover, and extract the interactions and relationships among learning components. It provides insights into complex interactive data, and has been extensively incorporated into recommender systems to enhance the interpretability of sophisticated relationships between users and items. Coupling learning can be further fostered once the trending collaborative learning can be engaged to take advantage of the cross-platform data. To facilitate this, privacy-preserving solutions are in high demand—it is desired that the collaboration should not expose either the private data of each individual owner or the model parameters trained on their datasets. In this article, we develop a distributed collaborative coupling learning system, which enables differential privacy. The proposed system defends against the adversary who has gained full knowledge of the training mechanism and the access to the model trained collaboratively. It also addresses the privacy-utility tradeoff by a provable tight sensitivity bound. Our experiments demonstrate that the proposed system guarantees favorable privacy gains at a modest cost in recommendation quality, even in scenarios with a large number of training epochs.

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

Existing research exploits the semantic information from reviews to complement user-item interactions for item recommendation. However, as these approaches either defer the user-item interactions until the prediction layer or simply concatenate all the reviews of a user/item into a single review, they fail to capture the complex correlations between each user-item pair or introduce noises. Thus, we propose a novel Hierarchical and Interactive Gate Network (HIGnet) model for rating prediction. Modeling local word informativeness and global review semantics in a hierarchical manner enable us to exploit textual features of users/items and capture complex semantic user-item correlations at different levels of granularities. Experiments on five challenging real-world datasets demonstrate the state-of-the-art performance of the proposed HIGnet model. To facilitate community research, the implementation of the proposed model is made publicly available (https://github.com/uqjwen/higan).

Deep Convolutional Neural Networks (DCNNs) have enhanced the performance of semantic image segmentation but many challenges still remain. Specifically, some details may be lost due to the downsampling operations in DCNNs. Furthermore, objects may appear in an image at different scales, and extracting features using convolutional filters with large sizes is costly in computation. Moreover, in many cases, contextual information, such as global and background features, is potentially useful for semantic segmentation. In this paper, we address these challenges by proposing a Multi-Receptive Atrous Convolutional Network (MRACN) for semantic image segmentation. The proposed MRACN captures the multi-receptive features and the global features at different receptive scales of the input. MRACN can serve as a module easily being integrated into existing models. We adapt the ResNet-101 model as the backbone network and further propose a MRACN segmentation model (MRACN-Seg). The experimental results demonstrate the effectiveness of the proposed model on two datasets: a benchmark dataset (PASCAL VOC 2012) and our industry dataset.

Various aspects of human activity recognition have been researched so far and a variety of methods have been used to address them. Most of this research assumed that the data sources used for the recognition task are static. In real environments, however, sensors can be added or can fail and be replaced by different types of sensors. It is therefore important to create an activity recognition model that is able to leverage dynamically available sensors. To approach this problem, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. In our previous work, we proposed sensor and activity context models to address sensor heterogeneity and also a learning-to-rank method for activity learning and its adaptation based on the proposed context models. However, most of the existing solutions, including our previous work, require labelled data for training. To tackle this problem and further improve the recognition accuracy, in this paper, we propose a knowledge-based method for activity recognition and activity model adaptation with dynamically available contexts in an unsupervised manner. We also propose a semi-supervised data selection method for activity model adaptation, so the activity model can be adapted without labelled data. We use comprehensive datasets to demonstrate effectiveness of the proposed methods, and show their advantage over the conventional machine learning algorithms in terms of recognition accuracy.

Graph learning has been demonstrated as one of the most effective methods for semi-supervised dimension reduction, as it can achieve label propagation between labeled and unlabeled samples to improve the feature projection performance. However, most existing methods perform this important label propagation process on the graph with sub-optimal structure, which will reduce the quality of the learned labels and thus affect the subsequent dimension reduction. To alleviate this problem, in this paper, we propose an effective Label Propagation with Structured Graph Learning (LPSGL) method for semi-supervised dimension reduction. In our model, label propagation, semi-supervised structured graph learning and dimension reduction are simultaneously performed in a unified learning framework. We propose a semi-supervised structured graph learning method to characterize the intrinsic semantic relations of samples more accurately. Further, we assign different importance scores for the given and learned labeled samples to differentiate their effects on learning the feature projection matrix. In our method, the semantic information can be propagated more effectively from labeled samples to the unlabeled samples on the learned structured graph. And a more discriminative feature projection matrix can be learned to perform the dimension reduction. An iterative optimization with the proved convergence is proposed to solve the formulated learning framework. Experiments demonstrate the state-of-the-art performance of the proposed method. The source codes and testing datasets are available at https://github.com/FWang-sdnu/LPSGL-code .

Social media has evolved into one of the most important channels to share micro-videos nowadays. The sheer volume of micro-videos available in social networks often undermines users’ capability to choose the micro-videos that best fit their interests. Recommendation appear as a natural solution to this problem. However, existing video recommendation methods only consider the users’ historical preferences on videos, without exploring any video contents. In this paper, we develop a novel latent genre aware micro-video recommendation model to solve the problem. First, we extract user-item interaction features, and auxiliary features describing both contextual and visual contents of micro-videos. Second, these features are fed into the neural recommendation model that simultaneously learns the latent genres of micro-videos and the optimal recommendation scores. Experiments on real-world dataset demonstrate the effectiveness and the efficiency of our proposed method compared with several state-of-the-art approaches.

Neighborhood-based recommenders are a major class of Collaborative Filtering models. The intuition is to exploit neighbors with similar preferences for bridging unseen user-item pairs and alleviating data sparseness, in other words, learn the sub-graph representation of each user in a user graph. Many existing works propose neural attention networks to aggregate neighbors and place higher weights on the specific subsets of users for recommendation. However, the neighborhood information is not necessarily always informative, and the noises in the neighborhood can negatively affect the model performance. To address this issue, we propose a novel neighborhood-based recommender, where a hybrid gated network is designed to automatically separate similar neighbors from dissimilar (noisy) ones, and aggregate those similar neighbors to comprise neighborhood representations. The confidence in the neighborhood is also addressed by putting higher weights on the neighborhood representations if we are confident with the neighborhood information, and vice versa. In addition, a user-neighbor component is proposed to explicitly regularize user-neighbor proximity in latent space. These two components are combined into a unified model to complement each other for the recommendation task. Extensive experiments on three public datasets demonstrate that the proposed model consistently outperforms the state-of-the-art neighborhood-based recommenders. Furthermore, we study different variants of the proposed model to justify the underlying intuition of the proposed hybrid gated network and user-neighbor modeling components.

Traditional recommender systems encounter several challenges such as data sparsity and unexplained recommendation. To address these challenges, many works propose to exploit semantic information from review data. However, these methods have two major limitations in terms of the way to model textual features and capture textual interaction. For textual modeling, they simply concatenate all the reviews of a user/item into a single review. However, feature extraction at word/phrase level can violate the meaning of the original reviews. As for textual interaction, they defer the interactions to the prediction layer, making them fail to capture complex correlations between users and items. To address those limitations, we propose a novel Hierarchical Text Interaction model (HTI) for rating prediction. In HTI, we propose to model low-level word semantics and high-level review representations hierarchically. The hierarchy allows us to exploit textual features at different granularities. To further capture complex user–item interactions, we propose to exploit semantic correlations between each user–item pair at different hierarchies. At word level, we propose an attention mechanism specialized to each user–item pair, and capture the important words for representing each review. At review level, we mutually propagate textual features between the user and item, and capture the informative reviews. The aggregated review representations are integrated into a collaborative filtering framework for rating prediction. Experiments on five real-world datasets demonstrate that HTI outperforms state-of-the-art models by a large margin. Further case studies provide a deep insight into HTI’s ability to capture semantic correlations at different levels of granularities for rating prediction.

We propose an efficient frequent activity patterns mining in smart environments.We build an accurate activity classifier based on the mined frequent patterns.We distinguish overlapped activities with global and local weights of sensor events.We use publicly available dataset of smart environments to validate our methods. In the past decades, activity recognition has aroused a great interest for the research groups majoring in context-awareness computing and human behaviours monitoring. However, the correlations between the activities and their frequent patterns have never been directly addressed by traditional activity recognition techniques. As a result, activities that trigger the same set of sensors are difficult to differentiate, even though they present different patterns such as different frequencies of the sensor events. In this paper, we propose an efficient association rule mining technique to find the association rules between the activities and their frequent patterns, and build an activity classifier based on these association rules. We also address the classification of overlapped activities by incorporating the global and local weight of the patterns. The experiment results using publicly available dataset demonstrate that our method is able to achieve better performance than traditional recognition methods such as Decision Tree, Naive Bayesian and HMM. Comparison studies show that the proposed association rule mining method is efficient, and we can further improve the activity recognition accuracy by considering global and local weight of frequent patterns of activities.

We propose an activity recognition model balancing accuracy, overhead, data labelling.We propose a similarity measurement method to effectively discover activity patterns.We perform comprehensive experimental and comparison studies to validate our method. In the past decades, activity recognition had aroused great interest for the community of context-awareness computing and human behaviours monitoring. However, most of the previous works focus on supervised methods in which the data labelling is known to be time-consuming and sometimes error-prone. In addition, due to the randomness and erratic nature of human behaviours in realistic environments, supervised models trained with data from certain subject might not be scaled to others. Further more, unsupervised methods, with little knowledge about the activities to be recognised, might result in poor performance and high clustering overhead. To this end, we propose an activity recognition model with labelled and unlabelled data in smart environments. With small amount of labelled data, we discover activity patterns from unlabelled data based on proposed similarity measurement algorithm. Our system does not require large amount of data to be labelled while the proposed similarity measurement method is effective to discover length-varying, disordered and discontinuous activity patterns in smart environments. Therefore, our methods yield comparable performance with much less labelled data when compared with traditional supervised activity recognition, and achieve higher accuracy with lower clustering overhead compared with unsupervised methods. The experiments based on real datasets from the smart environments demonstrate the effectiveness of our method, being able to discover more than 90% of original activities from the unlabelled data, and the comparative experiments show that our methods are capable of providing a better trade-off, regarding the accuracy, overhead and labelling efforts, between the supervised and unsupervised methods.

An activity recognition system essentially processes raw sensor data and maps them into latent activity classes. Most of the previous systems are built with supervised learning techniques and pre-defined data sources, and result in static models. However, in realistic and dynamic environments, original data sources may fail and new data sources become available, a robust activity recognition system should be able to perform evolution automatically with dynamic sensor availability in dynamic environments. In this paper, we propose methods that automatically incorporate dynamically available data sources to adapt and refine the recognition system at run-time. The system is built upon ensemble classifiers which can automatically choose the features with the most discriminative power. Extensive experimental results with publicly available datasets demonstrate the effectiveness of our methods.

Numerous methods have been proposed to address different aspects of human activity recognition. However, most of the previous approaches are static in terms of the data sources used for the recognition task. As sensors can be added or can fail and be replaced by different types of sensors, creating an activity recognition model that is able to leverage dynamically available sensors becomes important. In this paper, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. Specifically, we propose sensor and activity context models to address the problem of sensor heterogeneity, so that sensor readings can be pre-processed and populated into the recognition system properly. Based on those context models, we propose the learning-to-rank method for activity learning and its adaptation. To model the temporal characteristics of the human behaviours, we add temporal regularization into the learning and prediction phases. We use comprehensive datasets to demonstrate effectiveness of the proposed method, and show its advantage over the conventional machine learning algorithms in terms of recognition accuracy. Our method outperforms hybrid models that combine typical machine learning methods with graphical models (i.e. HMM, CRF) for temporal smoothing.

Since people perform activities differently, to avoid overfitting, creating a general model with activity data of various users is required before the deployment for personal use. However, annotating a large amount of activity data is expensive and time-consuming. In this paper, we create a general model for activity recognition with a limited amount of labelled data. We combine Latent Dirichlet Allocation (LDA) and AdaBoost to jointly train a general activity model with partially labelled data. After that, when AdaBoost is used for online prediction, we combine it with graphical models (such as HMM and CRF) to exploit the temporal information in human activities to smooth out accidental misclassifications. Experiments on publicly available datasets show that we are able to obtain the accuracy of more than 90% with 1% labelled data.