Search

Representation learning aims to distill useful knowledge from raw data and apply this knowledge to a wide range of applications. This ability to extract information that is useful not only for selected tasks but also generalizes to new settings is a key step towards artificial intelligence. In this thesis, we focus on representations derived through a specific type of generative model, i.e. variational autoencoders (VAEs). VAEs have several desirable properties. Thanks to the use of variational inference and the convenient model assumptions of Gaussian posteriors and a simple prior, VAEs are often easy to train and exhibit fast convergence. The probabilistic modelling formulation allows VAEs to derive a smooth latent representation of the raw data (i.e. semantically similar data samples are likely to be projected to nearby regions in the latent space). VAEs compress the raw data to a much lower dimension latent space. Working with the low-dimensional representations rather than the raw data can significantly reduces costs in memory and computation. With these advantages, VAEs have been widely applied to many applications, including robotics, drug discovery and digital content creation. Despite the widespread application of VAEs, improving the generative modelling of VAEs further remains an active research topic. In this thesis, we focus on two challenges in the VAE training: 1) over-regularized posterior distributions are often encountered in VAEs with Gaussian decoders and simple prior models; 2) the auto-encoding function may cause severe information drift and alter the information in the raw data in successive encodings. We propose solutions to both phenomena. Specifically, we optimize a variance parameter in the Gaussian decoder to balance competing loss terms in the ELBO objective. We adopt a flexible prior model that is implemented as a VAE in the latent space to mitigate the over-regularization effects. To reduce the information drift, we propose to modify the ELBO objective with a consistency loss that penalizes such drift. We show that these proposals can effectively address the challenges identified previously and improve the likelihood score of the VAEs. In addition to the contributions related to improving VAEs, we also demonstrate the power of VAEs' representation learning in two important machine learning applications. Firstly, we show that a VAE's ability to compress complicated, high-dimensional data is key to achieving good performance in anomaly detection. We design a VAE-LSTM anomaly detection system that can accurately identify anomalous effects in a time serious. Secondly, we show that a classifier which incorporates a VAE module can give better calibrated predictions. This is the result of a VAEs' capability of expressing the uncertainty between similar data samples in the spread of the posterior distribution and of identifying out-of-distribution samples.

Location awareness is a fundamental need for intelligent systems, such as self-driving vehicles, delivery drones, and mobile devices. Given their on-board sensors (e.g. camera, inertial sensor and LIDAR), previous researchers have developed a variety of localization systems, by building hand-crafted models and algorithms. Under ideal conditions, these sensors and models are able to accurately estimate system states without time bound. However, in real-world environments, many issues such as imperfect sensor measurements, inaccurate system modelling, complex environmental dynamics and unrealistic constraints, degrade the accuracy and reliability of localization systems. Therefore, this thesis aims to leverage machine learning approaches to overcome the intrinsic problems of the human-designed localization models. This research presents learning methods to estimate self-motion using multimodal sensor data to achieve accurate and robust localization. Firstly, we exploit inertial sensor, a completely ego-centric and relatively robust sensor, to develop Inertial Odometry Neural Network (IONet) that learns motion transformation from raw inertial data, and reconstructs accurate trajectories. This inertial only solution shows impressive performance in locating people and wheeled objects without being influenced by environmental issues. IONet was further refined as L-IONet, a lightweight framework, to reduce the computational burden of model training and testing, and enable real-time inference on low-end devices. As a first trial in this direction, we collected and released Oxford Inertial Odometry Dataset (OxIOD) with a very large amount of inertial motion data collection containing 158 sequences totalling 42 km, to train and comprehensively evaluate our proposed models. Secondly, we present a novel generic framework to learn selective sensor fusion in enabling more robust and accurate odometry estimation and localization in real-world scenarios. Two fusion strategies are proposed: soft fusion, implemented in a deterministic fashion; and hard fusion, which introduces stochastic noise and intuitively learns to keep the most relevant feature representations, while discarding useless or misleading information. Both are trained in an end-to-end fashion, and can be applied to complimentary pairs of sensor modalities, e.g. RGB images, inertial measurements, depth images, and LIDAR point clouds. We offer a visualization and interpretation of fusion masks to give deeper insights into the relative strengths of each stream. Finally, we leverage deep generative models to propose Sequential Invariant Domain Adaptation (SIDA) to mitigate the domain shift problem of the deep neural network based localization models. This framework works well on long continuous sensor data. Its key novelty is to use a shared encoder to convert the input sequence into a domain-invariant hidden representation, to encourage the useful semantic features obtained, whilst discarding the domain specific features. We employ proposed SIDA on deep learning based inertial odometry and human activity recognition to demonstrate its effectiveness in improving the generalization ability in new domains. We show that SIDA is able to transform raw sensor data into an accurate trajectory in new unlabelled domains, benefiting from the knowledge transferred from the labelled source domain. Through extensive experiments, all our proposed methods demonstrate their effectiveness and potential in achieving accurate and robust localization in real-world environments.

A long-term vision within the realm of ubiquitous computing is the creation of smart, digital environments that provide seamless human-computer interaction, allowing computation to recede into the background of everyday life. Key to realizing this vision is the ability for machines to recognize people, so that spaces can become truly personalized. However, the unpredictability of real-world environments impacts robust recognition, limiting usability. In real conditions, human identification systems have to handle issues such as out-of-set subjects and domain deviations, where conventional supervised learning approaches for training and inference are poorly suited. The inability of supervised methods to cope with this inherent diversity could be overcome if equivalently diverse training data were readily available. Unfortunately, obtaining such comprehensive training datasets would incur huge enrolment effort and would be costly to stage. With the rapid development of Internet of Things (IoT), we advocate a new labelling method in this thesis that exploits signals of opportunity hidden in heterogeneous IoT data. The key insight is that one sensor modality can leverage the signals measured by other co-located sensor modalities to improve its own labelling performance. If identity associations between heterogeneous sensor data can be discovered, it is possible to automatically label data, leading to more robust human recognition, without manual labelling or enrolment. We believe that many currently unsolved identification problems could be addressed through our advocated concept. Specifically, this thesis demonstrates that leveraging the signals of opportunity in physical and digital observations of subjects can overcome many obstacles surrounding robust human identification, and we comprehensively tackle this in a number of research threads. Firstly, we propose scan, a general algorithm for cross-modality association, designed to automatically label biometric data sensed in the wild. Secondly, in order to mitigate the errors in the automatically labelled data, we further present autotune, a generic framework that iteratively adapts the biometric model and updates sensor observations. Lastly, we comprehensively investigate the privacy implication of our advocated concept, with an application on smartwatch password inference and countermeasures. We demonstrate snoopy, a password inference framework which is able to accurately intercept passwords entered on the touchscreens of smartwatches of out-of-set victims, just by eavesdropping on motion sensors. To mitigate this attack, we propose a countermeasure deepauth, which is a second-factor authentication system on smartwatches based on behavioural signatures. We prove that the co-located secondary sensor not only can be maliciously used as a leakage channel, but can be effectively employed as a defence channel as well. All the proposed approaches are comprehensively evaluated through large-scale experiments and the results demonstrate their potential impact in a broad spectrum of identification scenarios.

Over the last decade, we have witnessed an unprecedented interest in indoor positioning technologies, with a variety of solutions developed in academic and industrial research labs. Although the field has reached a significant level of maturity, there is still no dominant solution and, as a consequence, positioning services are still lacking in many buildings. In order for a solution to be widely implemented and adopted, two key requirements must be satisfied: low cost and high accuracy. The dichotomy between cost and accuracy has fragmented the technology landscape, leading to a plethora of competing solutions that cannot satisfy both requirements simultaneously. The key objective of this thesis is to investigate how to unify the two disparate camps, providing high positioning accuracy with very low cost. Many approaches have tried to achieve this goal by fusing different sensor modalities. However, the majority of existing work has only investigated how to fuse sequences of measurements for which the associations with the targets are known (i.e. device personal data). Sensor fusion techniques that combine device personal data and anonymous sensor streams (where the associations between the measurements and the targets are not known) remain under-explored as of today. In this thesis, we investigate how to efficiently combine device sensor data and anonymous sensor streams from various sensor modalities in order to build low cost and high accuracy positioning systems. By combining these two types of sensor modalities in one system we see a great potential in designing cost-effective and accurate positioning systems for challenging environments such as for tracking people in highly dynamic industrial settings. Our goal is to design a multi-target multi-sensor tracking framework which will utilise existing sensor infrastructure found in industrial environments and large public buildings (e.g museums) in order to provide reliable positioning services.

Indoor tracking and navigation is a fundamental need for pervasive and context-aware applications. However, no practical and reliable indoor positioning solution is available at present. The major challenge of a practical solution lies in the fact that only the existing devices and infrastructure can be utilized to achieve high positioning accuracy. This thesis presents a robust indoor positioning system with the lifelong learning ability. The typical features of the proposed solution is low-cost, accurate, robust, and scalable. This system only takes the floor plan and the existing devices, e.g. phones, pads, etc. and infrastructure such as WiFi/BLE access points for the sake of practicality. This system has four closely correlated components including, non-line-of-sight identification and mitigation (NIMIT), robust pedestrian dead reckoning (R-PDR), lightweight map matching (MapCraft), and lifelong learning. NIMIT projects the received signal strength (RSS) from WiFi/BLE to locations. The R-PDR component converts the data from inertial measurement unit (IMU) sensors ubiquitous in mobile devices and wearables to the trajectories of the user. Then MapCraft fuses trajectories estimated from the R-PDR and the coarse location information from NIMIT with the floor plan and provides accurate location estimations. The lifelong learning component then learns the various parameters used in all other three components in an unsupervised manner, which continuously improves the the positioning accuracy of the system. Extensive real world experiments in multiple sites show how the proposed system outperforms state-of-the art approaches, demonstrating excellent sub-meter positioning accuracy and accurate reconstruction of tortuous trajectories with zero training effort. As proof of its robustness, we also demonstrate how it is able to accurately track the position regardless of the users, devices, attachments, and environments. We believe that such an accurate and robust approach will enable always-on background localization, enabling a new era of location-aware applications to be developed.

With sensor technology gaining maturity and becoming ubiquitous, we are experiencing an unprecedented wealth of sensor data. In most sensing scenarios, the measurements generated by sensor networks are noisy and usually annotated with some measure of uncertainty. The problem we address in this thesis is how to estimate the accuracy of the sensor systems based on the probabilistic measurements they provide. This problem is increasingly common in many settings, such as multiple sensing services are competing for the same group of users, detecting faults in large scale networks, or establishing trustworthiness of different individuals in social sensing. It is also challenging in many ways, for instance, the ground truth of the monitored states is absent, the users often lack a clear view of the implementation details of the sensor systems, and the reported accuracy can be misleading. To address theses challenges, in this thesis we formulate the problem of estimating the accuracy of sensor systems in a general manner that applies to a broad spectrum of sensing scenarios. We then propose an accuracy estimation framework that breaks the problem into layers, which can be implemented in different ways. We present a novel inference-based accuracy estimation approach, which assesses the accuracy of sensor systems by comparing the reported measurements with the states inferred with the probabilistic measurements from all systems and available prior knowledge. We also propose a new learning-based approach for accuracy estimation, which employs novel parameter learning techniques. The learned parameters are either used to improve estimating the accuracy of sensor measurements, or to derive the accuracy of sensor systems directly in certain cases. We perform a systematic experimental evaluation on two datasets collected from real-world sensor deployments, where an array of different approaches are juxtaposed and compared extensively. We discuss how they trade accuracy for computation cost, and how this trade-off largely depends on the knowledge of the sensing scenarios. We also show that the proposed approaches outperform the competing ones in estimating accuracy and ranking the sensor systems.

Range-based localization is a widely used technique for position estimation where distances are measured to anchors, nodes with known positions, and the position is analytically estimated. It offers the benefits of providing high localization accuracy and involving simple operation over multiple deployments. Examples are the Global Positioning System (GPS) and network-based cellular handset localization. Range-based localization is promising for a range of applications, such as robot deployment in emergency scenarios or monitoring industrial processes. However, the presence of clutter in some of these environments leads to a severe degradation of the localization accuracy due to non-line-of-sight (NLOS) signal propagation. Moreover, current literature in NLOS-mitigation techniques requires that the NLOS distances constitute only a minority of the total number of distances to anchors. The key ideas proposed in the dissertation are: 1) multi-hop localization offers significant advantages over single-hop localization in NLOS-prone environments; and 2) it is possible to further reduce position errors by carefully placing intermediate nodes among the clutter to minimize multi-hop distances between the anchors and the unlocalized node. We demonstrate that shortest path distance (SPD) based multi-hop localization algorithms, namely DV-Distance and MDS-MAP, perform the best among other competing techniques in NLOS-prone settings. However, with random node placement, these algorithms require large node densities to produce high localization accuracy. To tackle this, we show that the strategic placement of a relatively small number of nodes in the clutter can offer significant benefits. We propose two algorithms for node placement: first, the Optimal Placement for DV-Distance (OPDV) focuses on obtaining the optimal positions of the nodes for a known clutter topology; and second, the Adaptive Placement for DV-Distance (APDV) offers a distributed control technique that carefully moves nodes in the monitored area to achieve localization accuracies close to those achieved by OPDV. We evaluate both algorithms via extensive simulations, as well as demonstrate the APDV algorithm on a real robotic hardware platform. We finally demonstrate how the characteristics of the clutter topology influence single-hop and multi-hop distance errors, which in turn, impact the performance of the proposed algorithms.

This thesis describes the development of an offline, cooperative, range-based localization algorithm for use in settings where there is limited or no access to a positioning infrastructure. Motivating applications include underground animal tracking and indoor pedestrian localization. It is assumed that each sensor performs dead reckoning to estimate its current position, relative to a starting point. Each measurement adds error, causing the position estimate to drift further from the truth with time. The key idea behind the proposed algorithm is to use opportunistic radio contacts to mitigate this drift, and hence localize with greater accuracy. The proposed algorithm first fuses radio and motion measurements into a compact graph. This graph encodes key positions along sensor trajectories as vertices, and distance measurements as edges. In so doing, localization is cast as the graph realization problem: assigning coordinates to vertices, in such a way that satisfies the observed distance measurements. The graph is first analysed to certify whether it defines a localization problem with a unique solution. Then, several algorithms are used to estimate the vertex coordinates. These vertex coordinates are then used to apply piecewise corrections to each sensor's dead reckoning trajectory to mitigate drift. Finally, if sufficient anchors are available, the corrected trajectories are then projected into a global coordinate frame. The proposed algorithm is evaluated in simulation for the problem of indoor pedestrian tracking, using realistic error models. The results show firstly that 2D and 3D problems become provably more localizable as more anchors are used, and as the experiment duration increases. Secondly, it is shown that widely-used graph realization algorithms cannot be used for localization, as the complexity of these algorithms scales polynomially or greater with graph vertex count. Thirdly, it is shown a novel piecewise drift correction algorithm typically works well compared to a competing approach from the literature, but rare and identifiable graph configurations may cause the method to underperform.

Collecting data from sensor nodes to designated sinks is a common and challenging task in a wide variety of wireless sensor network (WSN) applications, ranging from animal monitoring to security surveillance. A number of approaches exploiting sink mobility have been proposed in recent years: some are proactive, in that sensor nodes push their read- ings to storage nodes from where they are collected by roaming mobile sinks, whereas others are reactive, in that mobile sinks pull readings from nearby sensor nodes as they traverse the sensor network. In this thesis, we point out that deciding which data collection approach is more energy-efficient depends on application characteristics, includ- ing the mobility patterns of sinks and the desired latency of collected data. We introduce novel adaptive data collection schemes that are able to automatically adjust to changing sink visiting patterns or data requirements, thereby significantly easing the deployment of a WSN. We illustrate cases where combining proactive and reactive modes of data collection is particularly beneficial. This motivates the design of TwinRoute, a novel hybrid algorithm that can flexibly mix the two col- lection modes at appropriate levels depending on the application sce- nario. Our extensive experimental evaluation, which uses synthetic and real-world sink traces, allows us to identify scenario characteristics that suit proactive, reactive or hybrid data collection schemes. It shows that TwinRoute outperforms the pure approaches in most scenarios, achiev- ing desirable tradeoffs between communication cost and timely delivery of sensor data.

This work focuses on the autonomous exploration of unknown areas by a swarm of mobile robots, referred to as agents. When an emergency happens within a building, it is dangerous to send human responders to search the area for hazards and victims. This motivates the need for autonomous agents that are able to coordinate with each other to explore the area as fast as possible. We investigate this problem from an algorithmic, rather than a robotics point of view, and thus abstract away from practical problems, such as obstacle detection and navigation over rough terrain. Our focus is on distributed algorithms that can cope with the following challenges: the topology of the area is typically unknown, communication between agents is intermittent and unreliable, and agents are not aware of their location in indoor environments. In order to address these challenges, we adopt the stigmergy approach, that is, we assume that the area is instrumented with small inexpensive sensors (called tags) and agents coordinate indirectly with each other by reading and updating the state of local tags. We propose three novel distributed algorithms that allow agents to explore unknown areas by coordinating indirectly through a tag-instrumented environment. In addition, we propose two mechanisms for discovering evacuation routes from critical points in the area to emergency exits. Agents are able to combine the tasks of area exploration and evacuation route discovery in a seamless manner. We study the proposed algorithms analytically, and evaluate them empirically in a custom-built simulation environment in a variety of scenarios. We then build a real testbed of agents and tags, and investigate practical mechanisms that allow agents to detect and localise nearby tags, and navigate toward them. Using the real testbed, we derive realistic models of detection, localisation and navigation errors, and investigate how they impact the performance of the proposed exploration algorithms. Finally, we design fault-tolerant exploration algorithms that are robust to these errors and evaluate them extensively in a simulation environment.

This work investigates the problem of efficiently monitoring and disseminating road traffic information in urban settings using fixed and mobile sensor networks. A key challenge in outdoor urban environments is that bandwidth is a scarce resource. It is thus vital to reduce the communication cost of forwarding traffic data from source sensor nodes through the wireless network to the traffic monitoring center. This thesis proposes two distinct approaches to reducing the communication cost of traffic monitoring: 1) in-network data reduction in the context of fixed sensor networks, and 2) efficient data acquisition and routing in the context of mobile sensor networks. In fixed sensor networks, nodes are deployed in fixed locations and are capable of monitoring local traffic at regular intervals. When users can tolerate long delays in traffic updates, we propose Fourier-based compression techniques that exploit spatio-temporal correlations in traffic data and reduce the cost of data delivery. When users require real-time traffic updates, we investigate the use of model-based approaches, in which sensor nodes use a model to predict traffic data, and only report data that deviates from the predicted values. Our evaluation of in-network reduction techniques for fixed sensor networks is based on a real traffic dataset derived from traffic monitoring sensors in the city of Cambridge, UK. In mobile sensor networks, we utilize traveling vehicles as nodes that can sense local traffic and forward it to the monitoring center. The key challenge in vehicular networks is to minimize the communication cost of traffic monitoring by jointly optimizing the processes of data acquisition and routing. Given user requirements for data freshness, we devise a traffic data acquisition scheme, and propose two routing algorithms, D-Greedy and D-MinCost, that carefully alternate between the multi- hop forwarding and data muling strategies. The proposed algorithms are compared with existing approaches in a simulation environment using realistic vehicular traces from the city of Zurich.