Search

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

Many existing solutions perform document-level sentiment classification based on local document only, ignoring other texts that might contribute to better classification accuracy. In this paper, we propose a novel speculative sentiment classification model named SSC. In SSC, we speculate that users with similar rating behaviours are more likely to write documents of similar sentiments toward a product. The motivation of SSC, therefore, is to exploit those speculative similar documents for improving classification accuracy. The proposed SSC model consists of three main components, namely, user-product interaction (UPI) component, document encoding (DE) component, and speculative similar document (SSD) component. The UPI component models user-product interactions, and encodes user/product ratings behaviours into user/product embeddings. The DE component utilizes learned user/product embeddings to capture the informative word vectors for comprising more accurate document representations. The SSD component aggregates documents written by similar users toward the same product for speculative sentiment classification. Because the user similarities are calculated based on user embeddings that encode user rating behaviours, the aggregated documents are more likely to have similar sentiments. The three components are seamlessly integrated into a unified model. In the unified manner, these three components are jointly optimized, and they mutually complement each other to enhance sentiment classification. We conduct extensive experiments on three public datasets, and demonstrate the advantage of the proposed SSC model over state-of-the-art baselines.

Various aspects of human activity recognition have been researched so far and a variety of methods have been used to address them. Most of this research assumed that the data sources used for the recognition task are static. In real environments, however, sensors can be added or can fail and be replaced by different types of sensors. It is therefore important to create an activity recognition model that is able to leverage dynamically available sensors. To approach this problem, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. In our previous work, we proposed sensor and activity context models to address sensor heterogeneity and also a learning-to-rank method for activity learning and its adaptation based on the proposed context models. However, most of the existing solutions, including our previous work, require labelled data for training. To tackle this problem and further improve the recognition accuracy, in this paper, we propose a knowledge-based method for activity recognition and activity model adaptation with dynamically available contexts in an unsupervised manner. We also propose a semi-supervised data selection method for activity model adaptation, so the activity model can be adapted without labelled data. We use comprehensive datasets to demonstrate effectiveness of the proposed methods, and show their advantage over the conventional machine learning algorithms in terms of recognition accuracy.

Common use of smartphones is a compelling reason for performing activity recognition with on-board sensors as it is more practical than other approaches, such as wearable sensors and augmented environments. Many solutions have been proposed by academia, but practical use is limited to experimental settings. Ad hoc solutions exist with different degrees in recognition accuracy and efficiency. To ease the development of activity recognition for the mobile application eco-system, Google released an activity recognition service on their Android platform. In this paper, we present a systematic evaluation of this activity recognition service and share the lesson learnt. Through our experiments, we identified scenarios in which the recognition accuracy was barely acceptable. We analyze the cause of the inaccuracy and propose four practical and light-weight solutions to significantly improve the recognition accuracy and efficiency. Our evaluation confirmed the improvement. As a contribution, we released the proposed solutions as open-source projects for developers who want to incorporate activity recognition into their applications.

We demonstrate that different people perform activities differently.Combine AdaBoost with LDA to build general activity model with minimum labelled data.Hybrid AdaBoost with HMM&CRF for temporal regulatization of human activities.Use publicly available datasets to validate the proposed methods. Activity recognition has been a hot topic for decades, from the scientific research to the development of off-the-shelf commercial products. Since people perform the activities differently, to avoid overfitting, building a general model with activity data of various users is required before the deployment for personal use. However, annotating a large amount of activity data is expensive and time-consuming. In this paper, we build a general model for activity recognition with a limited amount of labelled data. We combine Latent Dirichlet Allocation (LDA) and AdaBoost to jointly train a general activity model with partially labelled data. After that, when AdaBoost is used for online prediction, we combine it with graphical models (such as HMM and CRF) to exploit the temporal information in human activities to smooth out the accidental misclassifications. Experiments with publicly available datasets show that we are able to obtain the accuracy of more than 90% with 1% labelled data.

Aspect Terms Extraction (ATE) and Aspect Categories Detection (ACD) are two fundamental sub-tasks for aspect-based sentiment analysis. Most of the existing works mainly focus on the ATE task or the co-extraction of aspect terms and opinion words, while few attention are paid to the ACD task. In this work, we propose a joint model to seamlessly integrate the ATE and ACD tasks into a multi-task learning framework. Each of the tasks is based on multi-layer Convolutional Neural Networks (CNNs) for computing high-level word representations, and produces a task-specific and a task-share vector. The task-share vector of one task is used to propagate information to the other, and guides the counterpart task to align the informative textual features to produce the task-specific vectors. Finally, a fully-connected layer with a softmax/sigmoid function is applied to the task-specific vectors for the specific information extraction. The rationale underlying the proposed joint model is that, aspect terms and aspect categories are semantically related, and the information propagated between the two tasks can help to capture the semantic alignments between the aspect terms and categories, and produce informative task-specific vectors. Moreover, the ATE task models local semantics at each position of a sentence, while the ACD task extracts global features of the whole sentence. The mutual interactions between local and global features, therefore, can reciprocally capture informative textual features for the information extraction tasks. We validate the effectiveness of the proposed model on two widely used datasets, and show its advantage over the state-of-the-art baselines. We also investigate the effectiveness of the multi-task framework by comparing the proposed model with its variants. Further, we study the robustness of the proposed model by presenting the model performance with respect to different hyperparameters. Finally, we provide visualization examples to gain a better understanding of the advantages the multi-task learning scheme.

An activity recognition system essentially processes raw sensor data and maps them into latent activity classes. Most of the previous systems are built with supervised learning techniques and pre-defined data sources, and result in static models. However, in realistic and dynamic environments, original data sources may fail and new data sources become available, a robust activity recognition system should be able to perform evolution automatically with dynamic sensor availability in dynamic environments. In this paper, we propose methods that automatically incorporate dynamically available data sources to adapt and refine the recognition system at run-time. The system is built upon ensemble classifiers which can automatically choose the features with the most discriminative power. Extensive experimental results with publicly available datasets demonstrate the effectiveness of our methods. HighlightsPropose an activity recognition framework to incorporate dynamically discovered sensors automatically.Propose a method to select the most informative samples for retraining.Propose a novel way of combining AdaBoost with HMM&CRF for temporal regularization.

Neighborhood-based recommenders are a major class of Collaborative Filtering models. The intuition is to exploit neighbors with similar preferences for bridging unseen user-item pairs and alleviating data sparseness, in other words, learn the sub-graph representation of each user in a user graph. Many existing works propose neural attention networks to aggregate neighbors and place higher weights on the specific subsets of users for recommendation. However, the neighborhood information is not necessarily always informative, and the noises in the neighborhood can negatively affect the model performance. To address this issue, we propose a novel neighborhood-based recommender, where a hybrid gated network is designed to automatically separate similar neighbors from dissimilar (noisy) ones, and aggregate those similar neighbors to comprise neighborhood representations. The confidence in the neighborhood is also addressed by putting higher weights on the neighborhood representations if we are confident with the neighborhood information, and vice versa. In addition, a user-neighbor component is proposed to explicitly regularize user-neighbor proximity in latent space. These two components are combined into a unified model to complement each other for the recommendation task. Extensive experiments on three public datasets demonstrate that the proposed model consistently outperforms the state-of-the-art neighborhood-based recommenders. Furthermore, we study different variants of the proposed model to justify the underlying intuition of the proposed hybrid gated network and user-neighbor modeling components.

Traditional recommender systems encounter several challenges such as data sparsity and unexplained recommendation. To address these challenges, many works propose to exploit semantic information from review data. However, these methods have two major limitations in terms of the way to model textual features and capture textual interaction. For textual modeling, they simply concatenate all the reviews of a user/item into a single review. However, feature extraction at word/phrase level can violate the meaning of the original reviews. As for textual interaction, they defer the interactions to the prediction layer, making them fail to capture complex correlations between users and items. To address those limitations, we propose a novel Hierarchical Text Interaction model (HTI) for rating prediction. In HTI, we propose to model low-level word semantics and high-level review representations hierarchically. The hierarchy allows us to exploit textual features at different granularities. To further capture complex user–item interactions, we propose to exploit semantic correlations between each user–item pair at different hierarchies. At word level, we propose an attention mechanism specialized to each user–item pair, and capture the important words for representing each review. At review level, we mutually propagate textual features between the user and item, and capture the informative reviews. The aggregated review representations are integrated into a collaborative filtering framework for rating prediction. Experiments on five real-world datasets demonstrate that HTI outperforms state-of-the-art models by a large margin. Further case studies provide a deep insight into HTI’s ability to capture semantic correlations at different levels of granularities for rating prediction.

We propose an efficient frequent activity patterns mining in smart environments.We build an accurate activity classifier based on the mined frequent patterns.We distinguish overlapped activities with global and local weights of sensor events.We use publicly available dataset of smart environments to validate our methods. In the past decades, activity recognition has aroused a great interest for the research groups majoring in context-awareness computing and human behaviours monitoring. However, the correlations between the activities and their frequent patterns have never been directly addressed by traditional activity recognition techniques. As a result, activities that trigger the same set of sensors are difficult to differentiate, even though they present different patterns such as different frequencies of the sensor events. In this paper, we propose an efficient association rule mining technique to find the association rules between the activities and their frequent patterns, and build an activity classifier based on these association rules. We also address the classification of overlapped activities by incorporating the global and local weight of the patterns. The experiment results using publicly available dataset demonstrate that our method is able to achieve better performance than traditional recognition methods such as Decision Tree, Naive Bayesian and HMM. Comparison studies show that the proposed association rule mining method is efficient, and we can further improve the activity recognition accuracy by considering global and local weight of frequent patterns of activities.

We propose an activity recognition model balancing accuracy, overhead, data labelling.We propose a similarity measurement method to effectively discover activity patterns.We perform comprehensive experimental and comparison studies to validate our method. In the past decades, activity recognition had aroused great interest for the community of context-awareness computing and human behaviours monitoring. However, most of the previous works focus on supervised methods in which the data labelling is known to be time-consuming and sometimes error-prone. In addition, due to the randomness and erratic nature of human behaviours in realistic environments, supervised models trained with data from certain subject might not be scaled to others. Further more, unsupervised methods, with little knowledge about the activities to be recognised, might result in poor performance and high clustering overhead. To this end, we propose an activity recognition model with labelled and unlabelled data in smart environments. With small amount of labelled data, we discover activity patterns from unlabelled data based on proposed similarity measurement algorithm. Our system does not require large amount of data to be labelled while the proposed similarity measurement method is effective to discover length-varying, disordered and discontinuous activity patterns in smart environments. Therefore, our methods yield comparable performance with much less labelled data when compared with traditional supervised activity recognition, and achieve higher accuracy with lower clustering overhead compared with unsupervised methods. The experiments based on real datasets from the smart environments demonstrate the effectiveness of our method, being able to discover more than 90% of original activities from the unlabelled data, and the comparative experiments show that our methods are capable of providing a better trade-off, regarding the accuracy, overhead and labelling efforts, between the supervised and unsupervised methods.

An activity recognition system essentially processes raw sensor data and maps them into latent activity classes. Most of the previous systems are built with supervised learning techniques and pre-defined data sources, and result in static models. However, in realistic and dynamic environments, original data sources may fail and new data sources become available, a robust activity recognition system should be able to perform evolution automatically with dynamic sensor availability in dynamic environments. In this paper, we propose methods that automatically incorporate dynamically available data sources to adapt and refine the recognition system at run-time. The system is built upon ensemble classifiers which can automatically choose the features with the most discriminative power. Extensive experimental results with publicly available datasets demonstrate the effectiveness of our methods.

Activity recognition is being applied in an increasing number of applications. They include health monitoring of the elderly, discovery of frequent behavioural patterns, monitoring of daily life activities (e.g. eating, tooth brushing, sleeping), and analysis of exercise activities (e.g. swimming, running). Current approaches for activity recognition usually use the process of data preprocessing, feature extraction, activity model learning and activity recognition. Most of the previous research pipeline these steps and create static models for processing activity data and recognizing activities. The static models have predefined data sources that are tightly coupled with the models and never change once the models are created. However, the static models are unable to deal with sensor failures and sensor replacements that are quite common in real scenarios. Moreover, additional information provided by newly available data sources from dynamically discovered new sensors may potentially refine the activity model if this information can discriminatively characterize a specific activity class. However, the static models cannot leverage this additional information for self-refinement due to the static assumption of data sources. The primary goal of our research is to design and develop frameworks for activity recognition with dynamically available data sources, and propose and develop algorithms for activity model adaptation with the additional information provided by those data sources. In this thesis, we first provide a critical literature review in the areas of contexts modelling, context management, sensor modelling and sensors in mobile devices, activity recognition, activity model retraining and adaptation, and sensor dynamics in activity recognition. We then present the research on our activity recognition framework that makes the following key contributions. First, we propose a hybrid method that integrates Latent Dirichlet Allocation with conventional classifiers for learning a generic activity model with minimum annotated data. The hybrid method is able to alleviate the problem of data sparsity and requires a little amount of labelled activity data. Furthermore, it can deal with different variants of activity patterns since it is created with activity data of multiple users. The generic activity modelling serves as the starting point of our activity model adaptation with dynamically available sensor data. However, it can also serve as an independent component for other applications such as activity personalization. Second, based on the generic model, we propose a framework for low-level activity (e.g. running, walking) recognition with dynamically available sensors. The components of the framework include a basic classifier, instance selection and smoothing. Firstly, we use AdaBoost as our basic classifier as it is flexible with feature dimensionality and it can automatically select the discriminative features during the learning process. Secondly, we propose to select the most informative instances for activity model adaptation in an unsupervised manner. The instances contain features of the new sensor data, and the information of new sensors are incorporated seamlessly through the adaptation process. Finally, we design smoothing methods by integrating the graphical models such as Hidden Markov Model and Conditional Random Field with the basic classifier AdaBoost. Finally, we propose a framework for high-level activity (e.g, making coffee) recognition with dynamically available contexts. We propose sensor and activity models to address sensor heterogeneity and populating contextual information. Knowledge-driven and data-driven methods are proposed for incorporating the new contexts. The knowledge-driven method specifies the parameters of the new contexts with external knowledge in an unsupervised manner, and the data-driven method learns the parameters of the new contexts with the users' data using the proposed learning-to-rank technique and temporal regularization. Extensive experiments and comprehensive comparisons demonstrate the effectiveness of the proposed frameworks.

Numerous methods have been proposed to address different aspects of human activity recognition. However, most of the previous approaches are static in terms of the data sources used for the recognition task. As sensors can be added or can fail and be replaced by different types of sensors, creating an activity recognition model that is able to leverage dynamically available sensors becomes important. In this paper, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. Specifically, we propose sensor and activity context models to address the problem of sensor heterogeneity, so that sensor readings can be pre-processed and populated into the recognition system properly. Based on those context models, we propose the learning-to-rank method for activity learning and its adaptation. To model the temporal characteristics of the human behaviours, we add temporal regularization into the learning and prediction phases. We use comprehensive datasets to demonstrate effectiveness of the proposed method, and show its advantage over the conventional machine learning algorithms in terms of recognition accuracy. Our method outperforms hybrid models that combine typical machine learning methods with graphical models (i.e. HMM, CRF) for temporal smoothing.

Since people perform activities differently, to avoid overfitting, creating a general model with activity data of various users is required before the deployment for personal use. However, annotating a large amount of activity data is expensive and time-consuming. In this paper, we create a general model for activity recognition with a limited amount of labelled data. We combine Latent Dirichlet Allocation (LDA) and AdaBoost to jointly train a general activity model with partially labelled data. After that, when AdaBoost is used for online prediction, we combine it with graphical models (such as HMM and CRF) to exploit the temporal information in human activities to smooth out accidental misclassifications. Experiments on publicly available datasets show that we are able to obtain the accuracy of more than 90% with 1% labelled data.

Activity recognition is of great importance for a variety of context-aware applications and especially for assistance provided for independent living of the elderly. One of the factors that has impact on the activity recognition accuracy is feature representation. Many approaches have been proposed to select the most discriminative subset of features for activity recognition, but they are data-dependent and related to real numbers gathered from wearable sensors/devices. However, feature representation and transformation in smart environments, in which the data is very often a discrete sensor reading, has not been fully explored. In this paper, we map the activity model in a smart environment to a topic model and leverage the hierarchical Dirichlet process (HDP) to discover hidden structures from the sensor reading sequences. We use a hybrid approach: we combine the discovered latent structures with discriminative classifier such as support vector machines (SVMs), and demonstrate, through comparison studies, its effectiveness in improving activity recognition accuracy and the robustness to sensor noises. The comparison studies are carried out on three publicly available datasets from smart environments.

Applications involve dynamic linked library or legacy code which is difficult to optimize for static compiler, however, can be applied with optimization techniques by run-time system who focuses on binary code instead of source code. In this paper, we present a run-time system which observes the behaviors of the applications and performs optimizations base on the information it collects, and it works on the multi-core platform to boost performance. First, we establish a framework of traditional dynamic optimization system, base on which we propose the ways to reduce the overhead of the system, including different ways of trace construction and basic block linking. Second, we take advantage of hardware resources that multi-core platform provides and exploit parallelism in our system. Finally, we evaluate the performance of our system and have two major results. First, basic block linking can greatly improve the performance of the system, because without it, the system spends much of its time on interpreting and context switching. Second, the benchmarks that running on the system have a speedup of 1.05 on average, depending on the program's behavior.