Search

Deep learning has been increasingly applied to improve human activity recognition (HAR) accuracy and reduce the human efforts of handcrafted feature extractions. Federated Learning (FL) is an emerging learning paradigm that enables the collaborative learning of a global model without exposing users' raw data. However, existing FL approaches yield unsatisfactory HAR performance as they fail to dynamically aggregate models according to the statistical diversity of users' data. In this paper, we propose FedDL, a novel federated learning system for HAR that can capture the underlying user relationships and apply them to learn personalized models for different users dynamically. Specifically, we design a dynamic layer sharing scheme that learns the similarity among users' model weights to form the sharing structure and merges models accordingly in an iterative, bottom-up layer-wise manner. FedDL merges local models based on the dynamic sharing scheme, significantly speeding up the convergence while maintaining high accuracy. We have implemented FedDL and evaluated using a new data set we collected using LiDAR and four public real-world datasets involving 178 users in total. The results show that FedDL outperforms several state-of-the-art FL paradigms in terms of model accuracy (by more than 15%), converging rate (by more than 70%), and communication overhead (about 30% reduction). Moreover, the testing results on the datasets of different scales show that FedDL has high scalability and hence can be deployed for large-scale real-world applications.

Recent years have witnessed an emerging class of real-time applications, e.g., autonomous driving, in which resource-constrained edge platforms need to execute a set of real-time mixed Deep Learning (DL) tasks concurrently. Such an application paradigm poses major challenges due to the huge compute workload of deep neural network models, diverse performance requirements of different tasks, and the lack of real-time support from existing DL frameworks. In this paper, we present RT-mDL, a novel framework to support mixed real-time DL tasks on edge platform with heterogeneous CPU and GPU resource. RT-mDL aims to optimize the mixed DL task execution to meet their diverse real-time/accuracy requirements by exploiting unique compute characteristics of DL tasks. RT-mDL employs a novel storage-bounded model scaling method to generate a series of model variants, and systematically optimizes the DL task execution by joint model variants selection and task priority assignment. To improve the CPU/GPU utilization of mixed DL tasks, RT-mDL also includes a new priority-based scheduler which employs a GPU packing mechanism and executes the CPU/GPU tasks independently. Our implementation on an F1/10 autonomous driving testbed shows that, RT-mDL can enable multiple concurrent DL tasks to achieve satisfactory real-time performance in traffic light detection and sign recognition. Moreover, compared to state-of-the-art baselines, RT-mDL can reduce deadline missing rate by 40.12% while only sacrificing 1.7% model accuracy.

Infrastructure-assisted autonomous driving is an emerging paradigm that aims to make affordable autonomous vehicles a reality. A key technology for realizing this vision is real-time point cloud registration which allows a vehicle to fuse the 3D point clouds generated by its own LiDAR and those on roadside infrastructures such as smart lampposts, which can deliver increased sensing range, more robust object detection, and centimeter-level navigation. Unfortunately, the existing methods for point cloud registration assume two clouds to share a similar perspective and large overlap, which result in significant delay and inaccuracy in real-world infrastructure-assisted driving settings. This paper proposes VI-Eye - the first system that can align vehicle-infrastructure point clouds at centimeter accuracy in real-time. Our key idea is to exploit traffic domain knowledge by detecting a set of key semantic objects including road, lane lines, curbs, and traffic signs. Based on the inherent regular geometries of such semantic objects, VI-Eye extracts a small number of saliency points and leverage them to achieve real-time registration of two point clouds. By allowing vehicles and infrastructures to extract the semantic information in parallel, VI-Eye leads to a highly scalable architecture for infrastructure-assisted autonomous driving. To evaluate the performance of VI-Eye, we collect two new multiview LiDAR point cloud datasets on an indoor autonomous driving testbed and a campus smart lamppost testbed, respectively. They contain total 915 point cloud pairs and cover three roads of 1.12km. Experiment results show that VI-Eye achieves centimeter-level accuracy within around 0.2s, and delivers a 5X improvement in accuracy and 2X speedup over state-of-the-art baselines.

Three-degrees-of-freedom (3-DoF) omnidirectional imaging has been widely used in various applications ranging from street maps to 3-DoF VR live broadcasting. Although allowing for navigating viewpoints rotationally inside a virtual world, it does not provide motion parallax key for human 3D perception. Recent research mitigates this problem by introducing 3 transitional degrees of freedom (6-DoF) using multi-sphere images (MSI) which is beginning to show promises in handling occlusions and reflective objects. However, the design of MSI naturally limits the range of authentic 6-DoF experiences, as existing mechanisms for MSI rendering cannot fully utilize multi-layer information when synthesizing novel views between multiple MSIs. To tackle this problem and extend the 6-DoF range, we propose an MSI interpolation pipeline that utilizes adjacent MSIs' 3D information embedded inside their layers. In this work, we describe an MSI projection scheme along with an MSI interpolation network to predict intermediate MSIs in order to facilitate the need for extended range. We demonstrate that our system significantly improves the range of 6-DoF experience compared with other MSI-based methods. With extensive experiments, we show our algorithm outperforms state-of-the-art methods both qualitatively and quantitatively in synthesizing novel view panoramas.

Omnidirectional video is an essential component of Virtual Reality. Although various methods have been proposed to generate content that can be viewed with six degrees of freedom (6-DoF), existing systems usually involve complex depth estimation, image in-painting or stitching pre-processing. In this paper, we propose a system that uses a 3D ConvNet to generate a multi-sphere images (MSI) representation that can be experienced in 6-DoF VR. The system utilizes conventional omnidirectional VR camera footage directly without the need for a depth map or segmentation mask, thereby significantly simplifying the overall complexity of the 6-DoF omnidirectional video composition. By using a newly designed weighted sphere sweep volume (WSSV) fusing technique, our approach is compatible with most panoramic VR camera setups. A ground truth generation approach for high-quality artifact-free 6-DoF contents is proposed and can be used by the research and development community for 6-DoF content generation.

Software fault localization is one of the key activities in software debugging. The program spectrum-based approach is widely used in fault localization. However, lots of program information, for example, the sequence of the execution statement and statement semantics, is missing when such an approach is utilized, which affects the performance. XGBoost is an effective learning algorithm, which can use the characteristics of the training data to build a classification tree during training. In addition, XGBoost can iteratively adjust the information value of the feature, so that the training process retains the importance information of the feature. This paper proposes applying XGBoost into fault localization utilizing information of program execution behaviors. A novel method called XGB-FL is developed, where the program spectrum information is converted into a coverage matrix to train the XGBoost model. We can get the characteristics of the data through the trained model and the importance of the program statement in the classification process. This is also the basis for judging whether the statement is likely to contain a fault. Nine representative data sets have been chosen to evaluate the performance of XGB-FL. The experimental results show that XGB-FL can generally deliver a higher performance in fault localization than those baseline techniques, in terms of precision and efficiency.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

We examine the relationship between eye gaze and typing, focusing on the differences between touch and non-touch typists. To enable typing-based research, we created a 51-participant benchmark dataset for user input across multiple tasks, including user input data, screen recordings, webcam video of the participant's face, and eye tracking positions. There are patterns of eye movements that differ between the two types of typists, representing glances at the keyboard, which can be used to identify touch-.typed strokes with 92% accuracy. Then, we relate eye gaze with cursor activity, aligning both pointing and typing to eye gaze. One demonstrative application of the work is in extending WebGazer, a real-time web-browser-based webcam eye tracker. We show that incorporating typing behavior as a secondary signal improves eye tracking accuracy by 16% for touch typists, and 8% for non-touch typists.

A network testbed with high flexibility plays an important role in testing and evaluating new network technology. Both researchers and engineers always need a network testbed which can customize network experiment on demand. Some of the existing cloud computing platforms lack network customization while some network experimental platform can not provide convenient virtual machine management functions. In this paper, we propose a preliminary implementation of the elastic customized cloud network (ECCN) platform based on Openstack project which provides cloud computing management and supports the Openflow protocol. This testbed can meet the needs of a rapid customized deployment of network resources like virtual machines, load balancers and routers in data center. Simultaneously, it provides the capability to generate and manage the customized network topology and an experimental environment where traditional network can interwork with Software Defined Network (SDN). Experiment results show that the ECCN testbed can help users deploy their own network experiment rapidly and easily with a friendly operation, as well as a flexible network design based on the highly customized experiment. In addition, ECCN guarantees the network isolation between different users based on reuse and sharing of the physical resources.