Search

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

Deep Convolutional Neural Networks (DCNNs) have enhanced the performance of semantic image segmentation but many challenges still remain. Specifically, some details may be lost due to the downsampling operations in DCNNs. Furthermore, objects may appear in an image at different scales, and extracting features using convolutional filters with large sizes is costly in computation. Moreover, in many cases, contextual information, such as global and background features, is potentially useful for semantic segmentation. In this paper, we address these challenges by proposing a Multi-Receptive Atrous Convolutional Network (MRACN) for semantic image segmentation. The proposed MRACN captures the multi-receptive features and the global features at different receptive scales of the input. MRACN can serve as a module easily being integrated into existing models. We adapt the ResNet-101 model as the backbone network and further propose a MRACN segmentation model (MRACN-Seg). The experimental results demonstrate the effectiveness of the proposed model on two datasets: a benchmark dataset (PASCAL VOC 2012) and our industry dataset.

Various aspects of human activity recognition have been researched so far and a variety of methods have been used to address them. Most of this research assumed that the data sources used for the recognition task are static. In real environments, however, sensors can be added or can fail and be replaced by different types of sensors. It is therefore important to create an activity recognition model that is able to leverage dynamically available sensors. To approach this problem, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. In our previous work, we proposed sensor and activity context models to address sensor heterogeneity and also a learning-to-rank method for activity learning and its adaptation based on the proposed context models. However, most of the existing solutions, including our previous work, require labelled data for training. To tackle this problem and further improve the recognition accuracy, in this paper, we propose a knowledge-based method for activity recognition and activity model adaptation with dynamically available contexts in an unsupervised manner. We also propose a semi-supervised data selection method for activity model adaptation, so the activity model can be adapted without labelled data. We use comprehensive datasets to demonstrate effectiveness of the proposed methods, and show their advantage over the conventional machine learning algorithms in terms of recognition accuracy.

Social media has evolved into one of the most important channels to share micro-videos nowadays. The sheer volume of micro-videos available in social networks often undermines users’ capability to choose the micro-videos that best fit their interests. Recommendation appear as a natural solution to this problem. However, existing video recommendation methods only consider the users’ historical preferences on videos, without exploring any video contents. In this paper, we develop a novel latent genre aware micro-video recommendation model to solve the problem. First, we extract user-item interaction features, and auxiliary features describing both contextual and visual contents of micro-videos. Second, these features are fed into the neural recommendation model that simultaneously learns the latent genres of micro-videos and the optimal recommendation scores. Experiments on real-world dataset demonstrate the effectiveness and the efficiency of our proposed method compared with several state-of-the-art approaches.

Graph learning has been demonstrated as one of the most effective methods for semi-supervised dimension reduction, as it can achieve label propagation between labeled and unlabeled samples to improve the feature projection performance. However, most existing methods perform this important label propagation process on the graph with sub-optimal structure, which will reduce the quality of the learned labels and thus affect the subsequent dimension reduction. To alleviate this problem, in this paper, we propose an effective Label Propagation with Structured Graph Learning (LPSGL) method for semi-supervised dimension reduction. In our model, label propagation, semi-supervised structured graph learning and dimension reduction are simultaneously performed in a unified learning framework. We propose a semi-supervised structured graph learning method to characterize the intrinsic semantic relations of samples more accurately. Further, we assign different importance scores for the given and learned labeled samples to differentiate their effects on learning the feature projection matrix. In our method, the semantic information can be propagated more effectively from labeled samples to the unlabeled samples on the learned structured graph. And a more discriminative feature projection matrix can be learned to perform the dimension reduction. An iterative optimization with the proved convergence is proposed to solve the formulated learning framework. Experiments demonstrate the state-of-the-art performance of the proposed method. The source codes and testing datasets are available at https://github.com/FWang-sdnu/LPSGL-code .

Neighborhood-based recommenders are a major class of Collaborative Filtering models. The intuition is to exploit neighbors with similar preferences for bridging unseen user-item pairs and alleviating data sparseness, in other words, learn the sub-graph representation of each user in a user graph. Many existing works propose neural attention networks to aggregate neighbors and place higher weights on the specific subsets of users for recommendation. However, the neighborhood information is not necessarily always informative, and the noises in the neighborhood can negatively affect the model performance. To address this issue, we propose a novel neighborhood-based recommender, where a hybrid gated network is designed to automatically separate similar neighbors from dissimilar (noisy) ones, and aggregate those similar neighbors to comprise neighborhood representations. The confidence in the neighborhood is also addressed by putting higher weights on the neighborhood representations if we are confident with the neighborhood information, and vice versa. In addition, a user-neighbor component is proposed to explicitly regularize user-neighbor proximity in latent space. These two components are combined into a unified model to complement each other for the recommendation task. Extensive experiments on three public datasets demonstrate that the proposed model consistently outperforms the state-of-the-art neighborhood-based recommenders. Furthermore, we study different variants of the proposed model to justify the underlying intuition of the proposed hybrid gated network and user-neighbor modeling components.

Traditional recommender systems encounter several challenges such as data sparsity and unexplained recommendation. To address these challenges, many works propose to exploit semantic information from review data. However, these methods have two major limitations in terms of the way to model textual features and capture textual interaction. For textual modeling, they simply concatenate all the reviews of a user/item into a single review. However, feature extraction at word/phrase level can violate the meaning of the original reviews. As for textual interaction, they defer the interactions to the prediction layer, making them fail to capture complex correlations between users and items. To address those limitations, we propose a novel Hierarchical Text Interaction model (HTI) for rating prediction. In HTI, we propose to model low-level word semantics and high-level review representations hierarchically. The hierarchy allows us to exploit textual features at different granularities. To further capture complex user–item interactions, we propose to exploit semantic correlations between each user–item pair at different hierarchies. At word level, we propose an attention mechanism specialized to each user–item pair, and capture the important words for representing each review. At review level, we mutually propagate textual features between the user and item, and capture the informative reviews. The aggregated review representations are integrated into a collaborative filtering framework for rating prediction. Experiments on five real-world datasets demonstrate that HTI outperforms state-of-the-art models by a large margin. Further case studies provide a deep insight into HTI’s ability to capture semantic correlations at different levels of granularities for rating prediction.

We propose an activity recognition model balancing accuracy, overhead, data labelling.We propose a similarity measurement method to effectively discover activity patterns.We perform comprehensive experimental and comparison studies to validate our method. In the past decades, activity recognition had aroused great interest for the community of context-awareness computing and human behaviours monitoring. However, most of the previous works focus on supervised methods in which the data labelling is known to be time-consuming and sometimes error-prone. In addition, due to the randomness and erratic nature of human behaviours in realistic environments, supervised models trained with data from certain subject might not be scaled to others. Further more, unsupervised methods, with little knowledge about the activities to be recognised, might result in poor performance and high clustering overhead. To this end, we propose an activity recognition model with labelled and unlabelled data in smart environments. With small amount of labelled data, we discover activity patterns from unlabelled data based on proposed similarity measurement algorithm. Our system does not require large amount of data to be labelled while the proposed similarity measurement method is effective to discover length-varying, disordered and discontinuous activity patterns in smart environments. Therefore, our methods yield comparable performance with much less labelled data when compared with traditional supervised activity recognition, and achieve higher accuracy with lower clustering overhead compared with unsupervised methods. The experiments based on real datasets from the smart environments demonstrate the effectiveness of our method, being able to discover more than 90% of original activities from the unlabelled data, and the comparative experiments show that our methods are capable of providing a better trade-off, regarding the accuracy, overhead and labelling efforts, between the supervised and unsupervised methods.

An activity recognition system essentially processes raw sensor data and maps them into latent activity classes. Most of the previous systems are built with supervised learning techniques and pre-defined data sources, and result in static models. However, in realistic and dynamic environments, original data sources may fail and new data sources become available, a robust activity recognition system should be able to perform evolution automatically with dynamic sensor availability in dynamic environments. In this paper, we propose methods that automatically incorporate dynamically available data sources to adapt and refine the recognition system at run-time. The system is built upon ensemble classifiers which can automatically choose the features with the most discriminative power. Extensive experimental results with publicly available datasets demonstrate the effectiveness of our methods.

With the rapid development of theory and practice in Genomics, research on Public Health Genomics, as a new field is beginning to contribute to people's life. A large volume of genomics data is available but not yet readily used in clinical services. A gap exists between genomics research and public healthcare genomics applications. We believe that machine intelligence can play an important role in transferring genomics knowledge to practical use. As a vision of our research, in this paper we present the usefulness of applying machine intelligence to public health genomics.

Numerous methods have been proposed to address different aspects of human activity recognition. However, most of the previous approaches are static in terms of the data sources used for the recognition task. As sensors can be added or can fail and be replaced by different types of sensors, creating an activity recognition model that is able to leverage dynamically available sensors becomes important. In this paper, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. Specifically, we propose sensor and activity context models to address the problem of sensor heterogeneity, so that sensor readings can be pre-processed and populated into the recognition system properly. Based on those context models, we propose the learning-to-rank method for activity learning and its adaptation. To model the temporal characteristics of the human behaviours, we add temporal regularization into the learning and prediction phases. We use comprehensive datasets to demonstrate effectiveness of the proposed method, and show its advantage over the conventional machine learning algorithms in terms of recognition accuracy. Our method outperforms hybrid models that combine typical machine learning methods with graphical models (i.e. HMM, CRF) for temporal smoothing.

Since people perform activities differently, to avoid overfitting, creating a general model with activity data of various users is required before the deployment for personal use. However, annotating a large amount of activity data is expensive and time-consuming. In this paper, we create a general model for activity recognition with a limited amount of labelled data. We combine Latent Dirichlet Allocation (LDA) and AdaBoost to jointly train a general activity model with partially labelled data. After that, when AdaBoost is used for online prediction, we combine it with graphical models (such as HMM and CRF) to exploit the temporal information in human activities to smooth out accidental misclassifications. Experiments on publicly available datasets show that we are able to obtain the accuracy of more than 90% with 1% labelled data.

Advances in the 3G and LTE technologies made video on-demand a very popular entertainment for people on the go. However, uniform coverage of mobile networks is almost impossible and, in addition, we often experience drops in bandwidth due to handovers between cellular stations, interference and admission control. In this paper, we analyse the state-of-the-art approaches to streaming on-demand videos, describe the lesson learnt and develop a solution based on our earlier proposal (Col Stream) to improve the user experience. The new Col Stream solution includes an incentive mechanism that mimics a share market in which users can trade their virtual tokens for bandwidth and vice versa. We also describe the prototype developed for the Android phones.

The rapid adoption of mobile phones and growing number of mobile services (like Whatsapp) have created high volume of mobile data traffic. Compared to the traditional voice and SMS services, mobile data is becoming more and more important. Bandwidth is in high demand and is becoming critical for user experiences with mobile contents. Due to a variety of factors (such as admission control policies, signal fading), bandwidth available to a device cannot be directly mapped to the signal strength. This creates a need for bandwidth estimation on mobile phones, similar to the signal strength indicator, to show the estimated bandwidth at the instant. Due to the complexity of bandwidth measurement and/or limitations of earlier generations of mobile phones, there is a small amount of research on bandwidth estimation for mobile phones. This paper revisits a variety of bandwidth estimation methods for wireless and mobile networks, analyses why most of the solutions fail, studies the accuracy limitation of bandwidth estimation in mobile networks, and finds a feasible solution for dynamically estimating bandwidth on mobile phones. We propose GPing-Pair, a low-cost bandwidth approximation for cellular connections that estimates bandwidth on the mobile device without server support. We implemented the proposed method as an application for Android phones.

The number of mobile users of on demand video is growing rapidly. However, bandwidth fluctuation in the 3G/LTE technologies is an obstacle in providing high quality smooth video playout for users on the go. In this paper, we present ColStream that can aggregate bandwidth from ubiquitous devices to ensure high quality video streaming with minimal stalling time. ColStream dynamically adjusts the set of collaborators and the size of video chunks that the collaborators need to pre-fetch ahead of the video chunks playout time to provide smooth video playout. ColStream uses a multi-objective optimisation method to maximise bandwidth and minimise cost. ColStream requires neither external servers nor proxies to provide its functionality. The paper describes the ColStream functionality, architecture, applied algorithms, ColStream prototype, and evaluation of its suitability for effective video streaming in mobile environments.

MAC protocol is an important topic in Wireless Sensor Network, which plays a decisive role in network performance. The current MAC protocol has many problems such as highly dependent on time synchronization, low throughputs or data delay. With multi-channel allocation strategy, the paper put forward a dynamic route allocation algorithm, which reduced data delay and increased the throughputs. The simulation experiment show that the algorithm improves the throughputs, reduces the collision and data delay.

Mobile Ad hoc network is a multihop and provisional automous networks without support of infrastructure, which is composed of a group of mobile nodes with wireless transceiver. Based on MAODV, the paper proposes NMP-MAODV multicast routing protocol for the link disconnection problem caused by node moving so that the node is out of its upstream node's signal range. The protocol improves the packet delivery ratio and average delay in highly mobile network using node mobility prediction and active-link switch. Simulation results prove the feasibility and effectiveness of NMP-MAODV in Ad hoc networks.