Search

A presentation at the the Ohio State University Imageomics Institute. The work describes collaboration between Drexel University’s Metadata Research Center and the Tulane University Biodiversity Research Institute to redesign database structures and computational workflows for the NSF supported Harnessing the Data Revolution, Biology Guided Neural Networks (HDR-BGNN) project. The work makes database management easier and improves computationally derived metadata for over 300,000 fish specimen images. This work provides proof-of-concept for resource description framework (RDF) in metadata pipelines for other members of the Imagenomics Institute.

Privacy policies are long, complex documents that end-users seldom read. Privacy labels aim to ameliorate these issues by providing succinct summaries of salient data practices. In December 2020, Apple began requiring that app developers submit privacy labels describing their apps’ data practices. Yet, research suggests that app developers often struggle to do so. In this paper, we automatically identify possible discrepancies between mobile app privacy policies and their privacy labels. Such discrepancies could be indicators of potential privacy compliance issues. We introduce the Automated Privacy Label Analysis System (ATLAS). ATLAS includes three components: a pipeline to systematically retrieve iOS App Store listings and privacy policies; an ensemble-based classifier capable of predicting privacy labels from the text of privacy policies with 91.3% accuracy using state-of-the-art NLP techniques; and a discrepancy analysis mechanism that enables a large-scale privacy analysis of the iOS App Store. Our system has enabled us to analyze 354,725 iOS apps. We find several interesting trends. For example, only 40.3% of apps in the App Store provide easily accessible privacy policies, and only 29.6% of apps provide both accessible privacy policies and privacy labels. Among apps that provide both, 88.0% have at least one possible discrepancy between the text of their privacy policy and their privacy label, which could be indicative of a potential compliance issue. We find that, on average, apps have 5.32 such potential compliance issues. We hope that ATLAS will help app developers, researchers, regulators, and mobile app stores alike. For example, app developers could use our classifier to check for discrepancies between their privacy policies and privacy labels, and regulators could use our system to help review apps at scale for potential compliance issues.

Cyber-physical systems (CPSs), engineered systems which include sensing, processing, control, and communication in physical spaces, are ubiquitous in modern critical infrastructures including manufacturing, transportation systems, energy delivery, health care, water management, and the smart grid. The presence of heterogeneous components and devices creates numerous attack surfaces in these large scale, highly connected systems. Consequently, these systems are attractive targets for adversaries and are essential to protect in today’s society. In this dissertation, we provide a set of mechanisms and tools that can be used to achieve resilient CPSs, where safety is preserved while functionality is restored in the presence of attacks. More specifically, we focus on two necessary components in designing resilient CPSs: detection and response. The recognition and detection of attacks is the first and foremost step in achieving resilience. Once an attack is detected, a number of forms of active response can be implemented to ensure system resilience. We first present a number of tools which leverage both cyber theory and system theory to detect powerful stealthy attacks. Specifically, we set forth the moving target defense as an active detection mechanism for detecting what would otherwise be stealthy covert attacks. We then introduce a number of response mechanisms which leverage both cyber theory and system theory to ensure safety and security against these attacks. Specifically, we set forth software rejuvenation and overlay networks as response mechanisms that provide resilience against attacks on the CPS control software and communication network, respectively. We then set forth some general design and analysis tools for achieving resilient CPSs, providing a framework that minimizes an adversary’s window of opportunity when attacking decentralized systems. We conclude by providing a general tool that analyzes the resilience of any response mechanism against stealthy attacks.

We humans today are dual-citizens inhabiting both the digital and physical worlds; Both offer us unique advantages. The digital world offers us advantages such as unlimited access to information and expansion of human capabilities, while the physical world has long been providing us structural affordances, functional mechanisms, rich sensations, and much more. Currently, there exists a gap between the two. We constantly find ourselves having to frequently switch contexts to interact with either. Within this thesis, I propose building bi-direction interactions between the two worlds so that we can leverage advantages of both worlds simultaneously. I demonstrate this vision with an interactive system and a device each seamlessly bring advantages of one world into another, leveraging computational techniques and inspirations from design.

Soft robots have emerged as a new branch of robotics with deformable bodies to achieve adaptability to dynamically changing unstructured environments and safe interaction with life forms ranging from cells to humans. The miniaturization efforts in soft robotics for operation in enclosed, small, and remote spaces led to the development of stimuli-responsive soft robots where the actuation mechanism relies on the encoded response of the robot body to the external stimuli at the material level. Among the wide range of proposed external stimuli of temperature, light, chemical, and electric and magnetic fields, magnetic fields are exceptionally promising due to their safe and transparent interaction around biological tissues. The magnetically responsive soft robots or, in short, magnetic soft robots present untethered, fast, and reversible actuation at small scales within confined environments, making them ideal candidates for minimally invasive clinical operations within the human body. While the anticipated applications and impact of magnetic soft robots are exciting, various challenges are associated with the programming of the magnetic response, the prediction of the resultant magnetic response, and the design of the robot structure and magnetic encoding, requiring the development of novel strategies. In this thesis, a magnetic programming method, simulation approaches for predicting magnetic responses, and a data-driven strategy for the design of robot structure and magnetic encoding are introduced to develop the fundamentals of magnetic soft robots, addressing the pressing issues. A novel magnetic programming strategy is presented based on heating magnetic soft materials above the Curie temperature of the embedded ferromagnetic particles and aligning the magnetic domains by applying magnetic fields in the desired direction. The proposed method comprehensively demonstrates discrete, three-dimensional, high-throughput, and reprogrammable magnetization at high spatial resolution addressing the limitations of the existing magnetic programming approaches. To address the non-intuitive design challenge, a systematic and experience-free data-driven design approach is proposed to spatially program morphology and 3D magnetic profile of magnetic soft robots for desired behaviors. Our design strategy relying on the developed computationally low-cost simulation engine reveals complex magnetic soft robot behaviors that were unattainable. The best-performing designs are experimentally realized via the introduced magnetic programming method, showcasing the sim2real transfer. In addition, a dynamic faster-than-real-time simulation framework based on the discrete elastic rod method is introduced to accelerate the design and control endeavors in magnetic soft robots. The developed framework is validated rigorously for 2D and 3D quasi-static and dynamic conditions and captures the fundamental bending, twisting, and buckling behavior of the magnetic soft robots.

Wearable Cognitive Assistance (WCA) applications run on wearable mobile de- vices, to provide guidance for real world tasks. Physical assembly tasks have been a significant focus of research on WCA. We introduce new techniques to support the development of WCA applications for more complex assembly tasks than previous techniques supported. In addition, our work reduces the load on developers creating WCA applications by eliminating the need to collect and label real training images. We accomplish this by training computer vision models on synthetically generated images. This dissertation investigates escalation to human experts in cases when a user is not satisfied with the automated guidance from an application. Lastly, we develop a new version of a software framework for WCA applications, and evaluate ways in which WCA applications can benefit from running computations directly on mobile devices.

This paper provides a review of publications on the topic of stylometry, including both successful and unsuccessful attempts to apply the concept and real-world use cases. We then conducted our own experiments to attempt to conduct stylometry, using a variety of platforms for training and employing ML models. The purpose of this experimentation will be to identify where the barrier to entry lies in applying stylometry for average users. To do these, we conducted tests using techniques with varying levels of difficulty in application and varying levels of expected and actual success. Finally, based on both the literature review and our experiments, we will comment on what effects stylometry and machine learning may have on intrusions to privacy for those using anonymous digital platforms, now and in the future.

Independent workers—such as gig workers, online freelancers, or micro-entrepreneurs—take on heightened uncertainty in pursuit of flexible working arrangements. While workers may be independent from organizations’ directive control, a decade of ethnographic studies has highlighted how independent workers—who are digitally distributed in space and time—are in fact interdependent on each other for social, emotional, and material support. To augment workers’ quests for peer support, scholars and practitioners have designed dozens of intricate sociotechnical systems which foster large-scale, online peer support networks. Yet, solely sociotechnical approaches to building peer support systems have failed to create systems which provide inclusive support for this rapidly growing and diverse workforce. In pursuit of universal user adoption, such approaches often overlook existing peer networks which are entirely offline, and the resulting systems are rarely accessible, or desirable, to workers with limited trust in technology or technology literacy. This dissertation presents an approach to building community-based peer support systems for work which bridges two disparate bodies of work: sociotechnical system design and participatory action research. To do so, I followed a participatory action protocol to work with community partners who already fostered networks of peer workers to understand if technological interventions could provide supplemental support. In the case that community partners decided to explore technological supplements for peer support, I followed a co-design software protocol to build peer support systems which were driven by local community needs. The outcomes of this approach included not just peer support systems but also educational materials, in-person workshops, and a novel model of on-demand technical support for system on-boarding and maintenance. I illustrated this approach across two multi-year community partnerships with local hubs for independent workers in Pittsburgh, PA. The resulting three peer systems—Hirepeer, Peerdea and Tech Help Desk—facilitated career, professional, and skill development among independent workers.

Face recognition has emerged as one of the most prominent tasks for deep learning algorithms over the past decade. With a carefully designed pipeline, the face recognition system has improved dramatically in terms of accuracy and robustness. Unfortunately, the face recognition system still fails under challenging conditions in the real world. For example, a face recognition system can fail under extreme environmental conditions such as poor lighting, large pose variations, extreme facial expressions, or low resolution. We argue, however, that these challenging cases do not eliminate identity information because humans are capable of recognizing a person under a variety of challenging conditions. The poor performance of face recognition systems under such conditions can therefore be improved by design that aims at solving these cases. Often, challenging cases are a result of data imbalances in the training data, where face recognition systems see a very limited number of rare cases, thus not being able to recognize them. We demonstrate multiple directions for solving data imbalance issues in this dissertation. In the first part of this dissertation, we present an overview of our face recognition system and describe the efforts we made to deal with data imbalances. We demonstrate the major components of our face recognition system, namely our robust face detector for data collection, and facial feature extractor for face matching. The systems are designed in a way that improves the capability of capturing as much identity information from an imbalanced dataset as possible. Specifically, we introduce a multi-scale deep feature extraction module for robust face detection. During the process, deep facial features are extracted from multiple layers of the backbone neural network. This brings stable predictions regardless of a variety of challenging conditions, such as extreme resolution, blurry images, and occlusion. With our face detection system, we prevent data imbalances from entering the pre-processing pipeline. Next, we introduce and apply Ring loss to our facial feature extractor. Ring loss is a smooth feature normalization method to improve face recognition accuracy in rare cases. Observations suggest that faces with poor matching accuracy are not always difficult examples, but rather rare examples in training. Thus, we propose to perform a smooth L2 normalization of the deep facial features to a common magnitude. This will improve the robustness of deep features and make them more distinguishable based on their directions. Our experimental results validate the effectiveness of our proposed methods in tackling imbalanced datasets without incorporating any data augmentation. In the second part of the dissertation, we introduce a series of face synthesis algorithms as a data augmentation tool that aims at reducing existing data imbalances. A key focus of our research is to develop unsupervised deep generative models that maximize data augmentation while minimizing human intervention. We first examine the capability of traditional 2D Generative Adversarial Networks (GAN) to synthesize and manipulate realistic faces toward identity-preserving data augmentation. To be specific, we use linear manipulation of Style-GAN latent space to perform guided synthesis of 2D face images. We discovered the innate defects of traditional GANs during the development of this method, namely their inability to maintain consistency throughout 3D transformations, such as facial poses. As a result, we further propose and describe the design of an unsupervised symmetry-aware 3D face generator to perform smooth and realistic 3D pose manipulation of human faces while keeping the 3D geometry and generic identity information unchanged. The 3D generator is trained by combining an efficient 3D-aware GAN backbone with the prevailing Neural Radiance Fields (NeRF) module. We compared our newly developed method with other existing 3D GAN architectures and the result reveals a remarkable improvement in terms of pose manipulation accuracy and identity preservation. Meanwhile, we incorporate a simple but highly efficient background disentangle module to decouple the faces and the background during the synthetic process. Overall, our face synthesis methods yield promising results. We hope to inspire future researchers in the face recognition community to continue tackling the challenging data imbalance issue with generative models.

Third party tracking is common across the Internet and is used for tailored advertisements towards individuals. These practices threaten the privacy of people, especially those seeking an abortion. The tracking of abortion-related data is especially important in the aftermath of the Supreme Court’s ruling in Dobbs v Jackson Women Health Organization, which abolished the federal right to abortion that was previously established in Roe v Wade. The right to abortions is now decided by each state, many of which have severely limited or completely banned abortions. As laws and restrictions are changing regarding abortions, privacy is an increasing concern for women seeking abortions as well as individuals assisting in or providing abortions. This research examines third party tracking across abortion websites by analyzing advertisement content after visiting abortion websites. We found that while the overall distribution of advertisements were unaffected by abortion-seeking behavior, the ratio of health/pregnancy related ads were significantly higher for the conditions that visited abortion related websites. This study suggests many ideas for future research and demonstrates the challenges and importance of measuring third party tracking requests for individuals seeking abortions.

As software becomes more extensive and complex, identifying and remitting potential vulnerabilities is increasingly challenging. Fuzzing is an automated technique to discover bugs by repeatedly supplying the program-under-test (PUT) with generated inputs intended to trigger unknown bugs in the PUT. In 2016, B¨ohme et al. introduced the concept of power schedules and an improved search strategy to the then state-of-the-art fuzzer AFL. Using their implementation, which they dubbed AFLFast, they found that these changes resulted in significantly faster discovery of more crashes than AFL. In independent work, researchers at Siemens have been investigating how to take advantage of data center scale infrastructure best when fuzzing. To encourage adoption and facilitate academic research, they have opensourced their own distributed fuzzing system, FLUFFI, in September 2019. This thesis investigates the application of the power schedule and search strategy in AFLFast to FLUFFI. Specifically, we have implemented AFLFast’s power schedule and search strategy as well as a round-robin search strategy on top of the upstream version of FLUFFI. To evaluate the effectiveness of these changes, we have chosen 10 binaries with known bugs from Google’s FuzzBench and measured the differences in code and bug coverage between different combinations of power schedules and search strategies. Our findings include: (i) the ideas of B¨ohme et al. can be applied to FLUFFI to improve the fuzzing outcomes in a manner similar to how AFLFast improved upon AFL; and (ii) despite its simplicity, round-robin can be a desirable search strategy earlier in a fuzzing campaign.

Powerful Machine Learning (ML) models trained on large, annotated datasets have driven impressive advances in fields including natural language processing and computer vision. In turn, such developments have led to impactful applications of ML in areas such as healthcare, e-commerce, and predictive maintenance. However, obtaining annotated datasets at the scale required for training high capacity ML models is frequently a bottleneck for promising applications of ML. In this thesis, I study alternative pathways for acquiring domain knowledge and develop methodologies to enable learning from weak supervision, i.e., imperfect and indirect forms of supervision. I cover three forms of weak supervision: pairwise linkage feedback, programmatic weak supervision, and paired multi-modal data. These forms of information are often easy to obtain at scale, and the methods I develop reduce–and in some cases eliminate–the need for pointillistic ground truth annotations. I begin by studying the utility of pairwise supervision. I introduce a new constrained clustering method which uses small amounts of pairwise constraints to simultaneously learn a kernel and cluster data. The method outperforms related approaches on a large and diverse group of publicly available datasets. Next, I introduce imperfect pairwise supervision to programmatic weak supervision label models. I show empirically that just one source of weak pairwise feedback can lead to significantly improved downstream performance. I then further the study of programmatic data labeling methods by introducing approaches that model the distribution of inputs in concert with weak labels. I first introduce a framework for joint learning of a label and end model on the basis of observed weak labels, showing improvements over prior work in terms of end model performance on downstream test sets. Next, I introduce a method that fuses generative adversarial networks and programmatic weak supervision label models to the benefit of both, measured by label model performance and data generation quality. In the last part of this thesis, I tackle a central challenge in programmatic weak supervision: the need for experts to provide labeling rules. First, I introduce an interactive learning framework that aids users in discovering weak supervision sources to capture subject matter experts’ knowledge of the application domain in an efficient fashion. I then study the opportunity of dispensing with labeling functions altogether by learning from unstructured natural language descriptions directly. In particular, I study how biomedical text paired with images can be exploited for self-supervised vision–language processing, yielding data-efficient representations and enabling zero-shot classification, without requiring experts to define rules on the text or images. Together, these works provide novel methodologies and frameworks to encode and use expert domain knowledge more efficiently in ML models, reducing the bottleneck created by the need for manual ground truth annotations.

Estimating the 6-DoF camera pose and the 3D human pose lies at the core of many computer vision tasks, such as virtual reality, augmented reality, and human-robot interaction. Existing efforts either rely on large amounts of 3D training data for each new scene or require strong prior knowledge, e.g., known camera poses, only available in laboratory environments. Despite the improvements in the numbers on a few public datasets, the gap between laboratory research and real-world applications remains. The objective of this thesis is to develop camera and human pose estimation methods that can bridge this gap. This thesis includes two parts. The first part focuses on camera pose estimation using human information. We first introduce a single-view camera pose estimation method that uses a lightweight network trained only on synthetic 2D human trajectory data to directly regress the camera pose at test using real human trajectories. After that, we present a wide-baseline multi-view camera pose estimation method that treats humans as key points and uses a re-ID network pre-trained on public datasets to embed human features for solving cross-view matching. We show that both methods do not require 3D data collection and annotation and generalize to new scenarios without extra effort. The second part of this thesis concentrates on multi-view multi-person 3D human pose estimation targeting the challenging setting where the camera poses are unknown. We present a method that follows the detection-matching-reconstruction process and treats the cross-view matching as a clustering problem with the number of humans and cameras as constraints. Compared with existing methods, ours is one of the first that does not require camera poses, 3D data collection, or model training for each specific dataset. Next, we further improve the method by introducing a multi-step clustering mechanism and leveraging short-term single-view tracking to boost cross-view matching performance. Our method shows excellent generalization ability across various in-the-wild settings.

Search-based planning algorithms enable robots to come up with well- reasoned long-horizon plans to achieve a given task objective. They for- mulate the problem as a shortest path problem on a graph embedded in the state space of the domain. Much research has been dedicated to achieving greater planning speeds to enable robots to respond quickly to changes in the environment. Additionally, as the task complexity in- creases, it becomes important to incorporate more sophisticated models like simulators in the planning loop. However, these complex models are expensive to compute and prohibitively reduce planning speed. Because of the plateau in CPU clock speed, single-threaded planning algorithms have hit a performance plateau. On the other hand, the number of CPU cores has grown significantly, a trend that is likely to continue. This calls for the need for planning algorithms that exploit paralleliza- tion. However, unlike sampling-based planning algorithms, parallelizing search-based planning algorithms is not trivial if optimality or bounded sub-optimality is to be maintained due to their sequential nature. A key feature of robotics domains is that the major chunk of computational effort during planning is spent on computing the outcome of an action and the cost of the resulting edge instead of searching the graph. In this thesis, we exploit this insight and develop several parallel search-based planning algorithms that harness the multithreading capability of mod- ern processors to parallelize edge computations. We show that these novel algorithms drastically improve planning times across several domains. Our first contribution is a parallelized lazy search algorithm, Massively Parallelized Lazy Planning (MPLP). The existing lazy search algorithms are designed to run as a single process and achieve faster planning by in- telligently balancing computational effort between searching the graph and evaluating edges. The key idea that MPLP exploits is that search- ing the graph and evaluating edges can be performed asynchronously in parallel. On the theoretical front, we show that MPLP provides rigorous guarantees of completeness and bounded suboptimality. As with all lazy search algorithms, MPLP assumes that successor states can be generated without evaluating edges, which allows the algorithm to defer edge evaluations and lazily proceed with the search. However, this assumption does not always hold, for example, in the case of simulation- in-the-loop planning, which uses a computationally expensive simulator to generate successors. To that end, our second contribution is Edge- Based Parallel A* for Slow Evaluations (ePA*SE) which interleaves plan- ning with the parallel evaluation of edges while guaranteeing optimality. We also present its bounded suboptimal variant that trades off optimality for planning speed. For its applicability in real-time robotics, ePA*SE must compute plans under a time budget and therefore have anytime performance. Though lower solution cost is desired, it is not the first priority in such settings. Our third contribution is Anytime Edge-Based Parallel A* for Slow Eval- uations (A-ePA*SE), which brings the anytime property to ePA*SE. ePA*SE targets domains with expensive but similar edge computation times. However, in several robotics domains, the action space is heteroge- nous in the computational effort required to evaluate the outcome of an action and its cost. Therefore, our fourth contribution is Generalized Edge-Based Parallel A* for Slow Evaluations (GePA*SE), which gener- alizes ePA*SE to domains where edge computations vary significantly. We show that GePA*SE outperforms ePA*SE and other baselines in do- mains with heterogenous actions by employing a parallelization strategy that explicitly reasons about the computational effort required for their evaluation. Finally, we demonstrate the utility of parallelization in an algorithm that integrates graph search techniques with trajectory optimization (INSAT). Since trajectory optimization is computationally expensive, running INSAT on a single thread limits its practical use. The proposed parallelized version Parallelized Interleaving of Search and Trajectory Optimization (PINSAT) achieves several multiple increases in planning speed and sig- nificantly higher success rates.

The human brain is a marvel of complexity, with billions of neurons and trillions of connections that allow us to perform an astounding array of behaviors, from basic movements like walking and grasping to complex cognitive processes like decision- making and language. However, despite decades of research, much about how the brain learns and remembers remains a mystery. One challenge in understanding the neural basis of learning and memory is that seemingly simple acts, like taking a sip of water, are in fact immensely complex processes that require an intricate coordination of neural activity. Yet humans are able to learn and remember how to perform a vast array of new skills, suggesting that the brain has the capacity to produce neural activity appropriate for a wide variety of tasks. To better understand how the brain learns and remembers new behaviors, it is important to investigate the interplay between learning and memory. However, this relationship is not well understood, and a better understanding of it could shed light on how the brain is able to learn new tasks without forgetting familiar ones. A difficulty in probing how the brain learns and remembers different tasks is that the relationship between neural activity and behavior is very complex and difficult to estimate. To circumvent this obstacle, we employ a brain-computer interface (BCI). A BCI allows the experimenters to specify the causal relationship between neural activity and behavior, which can provide insights into the underlying mechanisms of learning and memory. This is in contrast to traditional arm-reaching experiments, where the relationship between neural activity and behavior is largely unknown. The focus of this thesis is to explore the geometry of neural population activity during motor learning and memory. By leveraging the causal relationship between neural activity and behavior, we aim to shed light on the complex processes that underlie motor memories, and to provide insights into how the brain is able to learn and retain new behaviors. The pinnacle result of this thesis is that learning alters neural activity to simultaneously support both memory and action - i.e. learning leaves a “memory trace” in neural population activity.

Micro-hypervisors are used in many research projects to improve the security of computer systems. For example, some micro-hypervisors can separate securitysensitive programs from commodity operating systems, which typically consist of millions of lines of code. Thus, the security-sensitive programs are secure even if the operating systems are compromised. XMHF is a micro-hypervisor framework for the x86 micro-architecture that allows developers to extend it into customized micro-hypervisors. Unfortunately, XMHF does not support the latest commodity operating systems and hardware. This thesis presents an enhancement of XMHF, called XMHF+, which addresses the compatibility issues mentioned above and introduces new features. XMHF+ extends its support to 64-bit modern operating systems such as Windows 10 and Debian 11, as well as modern chipsets with TPM 2.0. Moreover, XMHF+ virtualizes the hardware virtualization extension, enabling popular hypervisors such as KVM, VMware, VirtualBox, and Hyper-V to run on top of it. XMHF+ maintains the design principles of XMHF, making it possible to verify its memory integrity as future work.

The renderer process in a Firefox browser depends on numerous third-party libraries to render media – audio, images, video and other content – exposing it to web attacks. The attackers use the vulnerabilities in the third-party library code to trigger browser crashes or even achieve code execution within the browser. To protect the renderer from such vulnerabilities, the RLBox library focuses on hardening the renderer-library boundary by sandboxing a third-party library in its own process and providing a dedicated tainted type system to identify data that interacts with the sandboxed library’s functions. RLBox disallows any computation on the tainted data in the renderer as it could affect the control and/or data flow. For the developer to be able to use the tainted data within the renderer code, they need to eliminate the taint type. To remove the taint RLBox has functions, also known as validation methods, where developers add in checks for the values returned from the library function. This urges the developers to think through all cases thoroughly, specifically the corner cases, which can be very challenging since the unwrapped value might not have immediate apparent consequences but could cause memory violations – out-of-bounds reads or writes –upon later uses. In order to spot faulty validators, we propose a formal verification-based approach that will give users additional guidance through counter examples to construct conditions on the return values and enables the user to make informed decisions about their validators. In this work, we use DIVINE, an explicit-state LTL model checker, to formally verify the validators exhaustively. We developed C++ applications that sandbox external libraries using RLBox. We tested the validators written in the application to identify incorrect conditions and provide counterexamples. Overall, our work emphasizes the benefit of employing formal verification to strengthen the validators, which are crucial in the renderer-library boundary hardening.

Unmanned ground vehicles are being deployed in increasingly diverse and complex environments. With modern developments in sensing and planning, the field of ground vehicle mobility presents rich possibilities for mechanical innovations that may be especially relevant for unmanned systems. In particular, reconfigurability may enable vehicles to traverse a wider set of terrains with greater efficiency by allowing them the benefits of multiple configurations. However, reconfigurability is not without its costs including increased size, weight, cost, and complexity. In this work, we present a method for evaluating the positive and negative impacts of reconfigurability to enable the optimization of unmanned vehicle design. We start with the formation of definitions and metrics for reconfigurability, mobility, and complexity, drawing from a wide range of robotic applications. Next, we analyze the combination and optimization of these functions to find ideal physical parameters for a given objective. After that, we delve into the application side of this topic with a case study in reconfigurable vehicles and the design of a novel manually reconfigurable tracked vehicle. Finally, we evaluate this vehicle and validate the optimization method experimentally and through mission scenarios.

As our society’s reliance on automations increases, so does the importance of the security of the embedded systems that control these automations. Unfortunately, there is currently a scarcity in educational material for embedded systems security due to the area’s relative nascency. In this thesis, we explore how to design Capture The Flag (CTF) problems that showcase common beginner mistakes in embedded systems programming. We started by designing a storyline that involves a protagonist and her companion robot, which runs on a low-resource real-time embedded system. As it happens, the programming of this robot contains multiple subtle mistakes that would allow the antagonists in our story to become saboteurs. The goal of the learners is to role-play as the antagonists and discover how to generate malicious inputs that trigger these mistakes. Our effort to date has generated 5 CTF problems, each featuring a FreeRTOS program that implements the corresponding scenario. Our programs utilize the ocial FreeRTOS POSIX/Linux Simulator so that they can run on any traditional Linux server. This approach eliminates the need to access embedded hardware when playing our CTF problems, thus allowing our effort to scale to a large number of learners and support remote learning.

Minimalist Data Wrangling with Python is envisaged as a student's first introduction to data science, providing a high-level overview as well as discussing key concepts in detail. We explore methods for cleaning data gathered from different sources, transforming, selecting, and extracting features, performing exploratory data analysis and dimensionality reduction, identifying naturally occurring data clusters, modelling patterns in data, comparing data between groups, and reporting the results. This textbook is a non-profit project. Its online and PDF versions are freely available at ttps://datawranglingpy.gagolewski.com/.

Computer vision models have proven to be tremendously capable of recognizing and detecting several real-world objects: cars, people, pets. However, the best performing classes have abundant examples in large-scale datasets today and obscured or small objects are still challenging. In short, computer vision perception still falls tremendously short of its gold standard - human perception. Humans are capable of learning novel categories quickly regardless of the amount of data and can classify objects from far away, obscured, or small. This thesis aims to bridge the gap between human and computer vision through two main parts. In the first part of this thesis, we focus on closing the gap between human and computer vision by improving computer vision models’ performance on datasets with real-world data distributions. Since real-world object distribution is often imbalanced, where some categories are seen frequently while others are seen rarely, models struggle to perform well on under represented classes. Contrastly, humans are remarkably good at learning new objects even if rarely seen. Thus, we aim to improve standard vision tasks on long-tailed distributed datasets which resemble a real-world distribution. Our first approach starts in visual classification task where we aim to increase performance on rarer classes. In this work, we create new stronger classifiers for rarer classes by leveraging the representations and classifiers learnt for common classes. Our simple method can be applied on top of any existing set of classifiers, thus showcasing that learning better classifiers does not require extensive or complicated approaches. Our second approach ventures into visual detection and segmentation, where the additional localization task makes it difficult to train better rare detectors. We take a closer look at the basic resampling approach used widely in detection for long-tailed datasets. Notably, we showcase that the fundamental resampling strategy in detection can be improved by not only resampling whole images but also resampling just objects. Successful real-world models depend heavily on the quality of training and testing data. In part two of this thesis, we close the gap between human and computer vision by developing a large-scale neuro-imaging dataset and identifing and exploring a large challenge facing visual dataset curation. First, we build the first large-scale visual fMRI dataset, BOLD5000. In an effort to bridge the gap between computer vision and human vision, we design a dataset with 5,000 images taken from computer vision benchmark datasets. Through this effort, we identified a crucial and time-consuming component of dataset curation: creating labeling instructions for annotators and participants. Labeling instructions for a typical visual dataset will include detailed i definitions and visual category examples provided to annotators. These labeling instructions through both text description and visual exemplars provide thorough and high-quality category definitions. Unfortunately, current datasets typically do not release their labeling instructions (LIs). We introduce a new task, labeling instruction generation, to reverse engineer LIs from existing datasets. Our method leverages existing large visual and language models (VLMs) to generate LIs that provide visually meaningful exemplars and significantly outperforms all baselines for image retrieval.

Minimalist Data Wrangling with Python is envisaged as a student's first introduction to data science, providing a high-level overview as well as discussing key concepts in detail. We explore methods for cleaning data gathered from different sources, transforming, selecting, and extracting features, performing exploratory data analysis and dimensionality reduction, identifying naturally occurring data clusters, modelling patterns in data, comparing data between groups, and reporting the results. This textbook is a non-profit project. Its online and PDF versions are freely available at https://datawranglingpy.gagolewski.com/. A printed version (the same as the aforementioned PDF one) can be ordered from Amazon. Dr Marek Gagolewski is currently a Senior Lecturer in Applied AI at Deakin University in Melbourne, Australia and an Associate Professor in Data Science (on leave) at the Faculty of Mathematics and Information Science, Warsaw University of Technology, Poland. His research interests are related to data science, in particular: modelling complex phenomena, developing usable, general purpose algorithms, studying their analytical properties, and finding out how people use, misuse, understand, and misunderstand methods of data analysis in research, commercial, and decision making settings. In his spare time, he writes books for his students and develops free (libre) data analysis software, such as stringi – one of the most often downloaded R packages, and genieclust – a fast and robust clustering algorithm in both Python and R. See also: Deep R Programming at https://deepr.gagolewski.com/., Please cite this book as: Gagolewski M. (2023), Minimalist Data Wrangling with Python, Zenodo, Melbourne, DOI: 10.5281/zenodo.6451068, ISBN: 978-0-6455719-1-2, URL: https://datawranglingpy.gagolewski.com/

Submitted in total fulfilment for the degree of Doctor of Philosophy (PhD) to the Department of Computer Science and Information Technology, School of Engineering and Mathematical Sciences, College of Science, Health and Engineering, La Trobe University, Victoria.

IoT is an ever-progressing area of research. Nowadays every company is looking for ways of integrating their products with IoT for reasons of feasibility and innovation. As the number of devices connecting into the Internet grows every day, the security of these wireless devices grows in complexity. This is due to the wireless properties and nature of these networks and the low energy consumption from these battery-based devices joining the network. The risk pertaining to the insecurity of these networks is also heightened by the fact that the physical aspect of users canbe affected by attacks to the network. The motivation of this project is based on the idea that any vulnerability and any security issue need to be treated with persistent concern. In their 2020 paper, Akestoridis et al. show their work of the open source Zigbee Network security analysis tool called Zigator. Through this work they found a stream of reconnaissance attacks targeting Zigbee's lower layers, stated by the IEEE 802.15.4 standard, where it was observed that by pairing the short MAC addresses between source and destination packets, an attacker can infer the network topology. IEEE 802.15.4 standard provides a security feature that does not encrypt the MAC packet headers, thus what propelled us to find an approach with obfuscation. The most common way for obfuscation in networks is to make use of dummy packets, though the majority of work uses these specifically for spatial obfuscation, source/destination anonymization and packet route obfuscation. Thus, there is not much work done targeting network topology obfuscation for battery-based devices in ad hoc networks. Part of the methodology consisted in understanding and using the network simulator NS3. Unfortunately, the implemented NS3 model for IEEE802.15.4 protocol was incomplete, and other research papers that solved this problem did not provide means for acquiring their complete implementations of the model.The contribution of this project takes two aspects. The simulator has been important for algorithm implementation and correctness evaluation in this work. In order to reach this part, essential primitives specific to this standard were implemented in the open-source tool. Thus, this is one of the contributions of this work: providing the research community with a more realistic simulation of IEEE 802.15.4 networkswithin NS3. Second, I derived two algorithms to obfuscate the network topology by keeping in mind the amount of energy consumption. Both algorithms follow an aliasing method, where devices are provided different MAC addresses in order to obfuscate their traffic visibility. The algorithms differ in how the devices receive theirAliases. For Mac Layer Determined Aliasing (MLDA), it is assumed devices have set MAC Addresses, while Network Layer Determined Aliasing (NeLDA) expects the MAC addresses to be transmitted as payload during the association process. We show that we achieved device obfuscation though with increased overhead, thus weprovide an analysis on the trade-off between overhead and obfuscation level for both algorithms. We use theoretical analysis to consider device polling in our overheadcalculations.

Cyber attacks are an increasing threat to victims ranging from individuals to businesses and nation-states. Underpinning these attacks is an ecosystem of adversaries that compete with each other in profitable nefarious activities such as advertising illegal pharmacies, manipulatingbusiness and product reviews, or engaging in illegal financial activities. Preventing these attacks in general is intractable as it involves solving problems such as creating perfectly secure software, authenticating product reviews, and verifying the intentions of financial market participants. Inthis work we approach these problems through the lens of economic rationality and postulate that understanding the economics of the attacker unlocks empirically effective solutions that make attacks unattractive or impractical rather than impossible.

Cyber-physical systems (CPSs) combining discrete control and continuous physical dynamics are pervasive in modern society: examples include driver assistance in cars, industrial robotics, airborne collision avoidance systems, and the electrical grid. Many of these systems are safety-critical because they operate in close proximity to humans. Formal safety verification of these systems is important because it is a key tool for attaining the strongest possible safety guarantees. Hybrid systems models, in particular, are a successful formalism for CPS. Hybrid systems theorem-proving in differential dynamic logic (dL) and its generalization differential game logic (dGL) are notable for strong logical foundations and successful application in case studies using the theorem provers KeYmaera and KeYmaera X. However, safety verification of models does not imply safety of implementations, which might not be faithful to the model. Moreover, a machine-checked proof is only as trustworthy as the software which checks it, thus correctness of proof-checkers is crucial. This thesis addresses implementation and soundness gaps by using constructive logic and programming languages as the foundation of an end-to-end verification toolchain. That is: Constructive Differential Game Logic (CdGL) enables practical, end-to-end verification of cyber-physical systems. CdGL enables synthesis of implementations with bulletproof theoretical foundations. Logic is the keystone of the end-to-end connection from high-level proofs and foundations to implementations. Our pursuit of practical proving includes innovations in the proof language itself. CdGL proofs, in contrast to dGL, are suitable for synthesizing controllers which determine safe actions for a CPS and monitors which check the compliance of the external environment with model assumptions. The synthesized code is automatically proven correct down to machine-code level. The foundations are also strengthened by our formalization of classical dL’s soundness in Isabelle/HOL, allowing hybrid systems proofs in dL to be exported and rechecked. We evaluate the toolchain on a 2D robot which follows arcs. The model and implementation cross-validate each other: monitors catch incorrect code and assumptions, while testing with monitors enabled allows us to assess the realism of the model.

GNNQ paper accepted to ISWC22 + supplemental material.

In 2021 the Centre for eResearch helped over 2,650 researchers, working on over 1,300 research projects. This is an increase of 540 researchers (up 25%) and 279 projects (up 26%) over 2020. A major highlight of 2021 was the Research Data Maturity work that CeR and ORSI-led across the University, interviewing and surveying hundreds of researchers to gain deep insights into their research data needs, practices, frustrations and opportunities.

Research on graph representation learning (GRL) has made major strides over the past decade, with widespread applications in domains such as e-commerce, personalization, fraud & abuse, life sciences, and social network analysis. Despite its widespread success, fundamental questions on practices employed in modern day GRL have remained unanswered. Unraveling and advancing two such fundamental questions on the practices in modern day GRL forms the overarching theme of my thesis. The first part of my thesis deals with the mathematical foundations of GRL. GRL is used to solve tasks such as node classification, link prediction, clustering, graph classification, and so on, albeit with seemingly different frameworks (e.g. Graph neural networks for node/graph classification, (implicit) matrix factorization for link prediction/ clustering, etc.). The existence of very distinct frameworks for different graph tasks has puzzled researchers and practitioners alike. In my thesis, using group theory, I provide a theoretical blueprint that connects these seemingly different frameworks, bridging methods like matrix factorization and graph neural networks. With this renewed understanding, I then provide guidelines to better realize the full capabilities of these methods in a multitude of tasks. The second part of my thesis deals with cases where modeling real-world objects as a graph is an oversimplified description of the underlying data. Specifically, I look at two such objects (i) modeling hypergraphs (where edges encompass two or more vertices) and (ii) using GRL for predicting protein properties. Towards (i) hypergraphs, I develop a hypergraph neural network which takes advantage of the inherent sparsity of real world hypergraphs, without unduly sacrificing on its ability to distinguish non isomorphic hypergraphs. The designed hypergraph neural network is then leveraged to learn expressive representations of hyperedges for two tasks, namely hyperedge classification and hyperedge expansion. Experiments show that using our network results in improved performance over the current approach of converting the hypergraph into a dyadic graph and using (dyadic) GRL frameworks. Towards (ii) proteins, I introduce the concept of conditional invariances and leverage it to model the inherent flexibility present in proteins. Using conditional invariances, I provide a new framework for GRL which can capture protein-dependent conformations and ensures that all viable conformers of a protein obtain the same representation. Experiments show that endowing existing GRL models with my framework shows noticeable improvements on multiple different protein datasets and tasks.

Human movement in urban areas is a complex phenomenon to analyze and understand. The geographical spread of human movement, described by locations between which individuals tend to travel, varies from city to city. The geographical spread of such locations within a city would also vary with time. This thesis explores how to formally analyze and understand human mobility using large sets of realworld data from ride-sharing services for more than a dozen cities and to derive succinct characterizations for large urban areas which account for both geographical and temporal changes. A wide range of machine learning problems require immense amounts of data. To overcome this issue for human mobility, we propose a framework which includes a stochastic graph model, and adversarial networks to generate synthetic humanmobility data which conforms with the geographical and temporal characteristics observed in real data.Deriving interesting insights from large sets of data and applying those to realworld applications can be challenging. Here we also highlight how formal characterizations can be applied to applications like ride pooling and vehicle placement. We also derive performance bounds for online algorithms using city level characterizations. Finally, we provide an open-source framework to understand properties of humanmovement, generate synthetic human mobility data, and apply it to different what-if scenarios for real-world applications. Using such a framework would help publicentities and researchers alike to thoroughly understand a complex and highly relevant phenomenon.Throughout, we link our problem formulations with other domains like social network graphs and online algorithms along with extensive empirical evaluations toincrease our understanding of not just human mobility but established techniques in the linked domains.

Semidefinite programming has long been a theoretically powerful tool for solving relaxations of challenging, often NP-hard optimization problems. However, it has typically not been practical for most large-scale tasks, owing to the high memory and computational cost of typical solvers for solving SDPs. In this thesis, we aim to break the barrier and bring SDP’s power back to large-scale machine learning problems. To achieve this, we introduce a series of optimization solvers, operating on the low-rank or low-cardinality manifolds of the semidefinite variables. We find that in many domains, these methods allow SDP relaxations to exceed the state of the art in terms of both computational cost and the relevant performance metrics. First, we proposed the Mixing method, a low-rank SDP solver aimed at the classical MAXCUT SDP relaxation. We also show that the Mixing method can accurately estimate the mode and partition function of the pairwise Markov Random Fields, and scales to millions of variables. Further, we show how to learn the parameters inside SDPs by analytically differentiating through the optimization problem with implicit differentiation and the mixing methods, which leads to a differentiable SAT solver that can be integrated within the loop of larger deep learning systems. For nonnegative constraints, we propose a separate variant aimed at low cardinality SDPs, and demonstrate how to apply the method to community detection on finding clusters within large-scale networks. Finally, we show that the technique can also be applied to more generic problems, such as a generic linear programming problems (with arbitrarily structured constraints), and we use this approach to develop a scalable sparse linear programming solver that improves solution speed over existing state-of-the-art commercial solvers

Large-scale deep learning models have reached previously unattainable performance for various tasks. However, the ever-growing resource consumption of neural networks generates large carbon footprint, brings difficulty for academics to engage in research and stops emerging economies from enjoying growing Artificial Intelligence (AI) benefits. To further scale AI to bring more benefits, two major challenges need to be solved. Firstly, even though large-scale deep learning models achieved remarkable success, their performance is still not satisfactory when fine-tuning with only a handful of examples, thereby hindering widespread adoption in real-world applications where a large scale of labeled data is difficult to obtain. Secondly, current machine learning models are still mainly designed for tasks in closed environments where testing datasets are highly similar to training datasets. When the deployed datasets have distribution shift relative to collected training data, we generally observe degraded performance of developed models. How to build adaptable models becomes another critical challenge. To address those challenges, in this dissertation, we focus on two topics: few-shot learning and domain adaptation, where few-shot learning aims to learn tasks with limited labeled data and domain adaption address the discrepancy between training data and testing data. In Part 1, we show our few-shot learning studies. The proposed few-shot solutions are built upon large-scale language models with evolutionary explorations from improving supervision signals, incorporating unlabeled data and improving few-shot learning abilities with lightweight fine-tuning design to reduce deployment costs. In Part 2, domain adaptation studies are introduced. We develop a progressive series of domain adaption approaches to transfer knowledge across domains efficiently to handle distribution shifts, including capturing common patterns across domains, adaptation with weak supervision and adaption to thousands of domains with limited labeled data and unlabeled data.

Submitted in total fulfilment of the requirements for the degree of Doctor of Philosophy to the School of Computing, Engineering and Mathematical Sciences. La Trobe University, Victoria.

Deep learning enables effective computer vision without hand crafting feature extractors. It has great potential if applied to specialized domains such as ecology, military, and medical science. However, the laborious task of creating labeled training sets of rare targets is a major deterrent to achieving its goal. A domain expert’s time and attention is precious. We address this problem by designing, implementing, and evaluating Eureka, a system for human-efficient discovery of rare phenomena from unlabeled visual data. Eureka’s central idea is interactive contentbased search of visual data based on early-discard and machine learning. We first demonstrate its effectiveness for curating training sets of rare objects. By analyzing contributing factors to human efficiency, we identify and evaluate important systemlevel optimizations that utilize edge computing and intelligent storage. Lastly, we extend Eureka to the task of discovering temporal events from video data.

Large-scale infrastructures across various domains (e.g., Internet services, sensor farms, operations monitoring etc.) produce ever-increasing amounts of streaming data. As these data streams contain invaluable operational insights, operators invest heavily on telemetry frameworks to extract these insights and use them towards ensuring their infrastructure’s reliability and growth. In this dissertation, we focus on telemetry for streaming video infrastructures. In particular, this work is motivated by a previously unexplored aspect of video telemetry, namely viewership analytics. That is, detecting and diagnosing video viewership anomalies, simultaneously, across multiple subpopulations of viewers. This dissertation aims at enhancing video operators’ toolbox with novel telemetry capabilities for viewership analytics. Nevertheless, designing telemetry workflows for viewership analytics proves challenging on multiple fronts. First, increases in volume and dimensionality of incoming data streams result in a combinatorial explosion of data subpopulations to monitor and, as a result, in prohibitive cost and resource overheads for operators. Second, the contextual and non stationary nature of viewership complicates the detection and diagnosis of viewership anomalies. Last, the need to simultaneously monitor ever-increasing numbers of subpopulations of viewers complicates extracting the few critical, and often highly localized, events of interest needed to provide actionable insights to operators. Our work addresses these challenges through the design and implementation of a suite of practical tools for video viewership analytics. First, we introduce Hydra, a novel sketch-based analytics framework for efficient and general analytics over multidimensional data streams. We show that HYDRA offers robust accuracy guarantees at one tenth (or less) of the operational cost of exact analytics frameworks and does so with query latencies that are up to 20× lower than existing alternatives. In Proteas, our second contribution, we leverage key structural insights of viewership in order to enable accurate detection and insightful diagnosis of viewership anomalies. We show that our approach ensures low numbers of false positives and outperforms the closest state-of-the-art alternatives. Last, we illustrate how these insights can be combined in the design of an end-to-end telemetry framework. Through extensive analysis driven by real-world datasets, we demonstrate that our findings can yield substantial cost and resource benefits over existing solutions. Additionally, we discuss their potential applicability in different domains, in addition to video.

An exegesis submitted to fulfil the requirements for the degree of Doctor of Philosophy (Communications) to the School of Politics, Media and Philosophy, La Trobe University.

With the rise in availability of data for language learning, the role of linguistic structure is under scrutiny. The underlying syntactic structure of language allows for composition of simple elements into more complex ones in innumerable ways; generalization to new examples hinges on this structure. We define a syntactic inductive bias as a signal that steers the learning algorithm towards a syntactically robust solution, over others. This thesis explores the need for incorporation of such biases into already powerful neural models of language. We describe three general approaches for incorporating syntactic inductive biases into task-specific models, under different levels of supervision. The first method calls for joint learning of entire syntactic dependency trees with semantic dependency graphs through direct supervision, to facilitate better semantic dependency parsing. Second, we introduce the paradigm of scaffolded learning, which enables us to leverage inductive biases from syntactic sources to predict a related semantic structure, using only as much supervision as is necessary. The third approach yields general-purpose contextualized representations conditioned on large amounts of data along with their shallow syntactic structures, obtained automatically. The linguistic representations learned as a result of syntactic inductive biases are shown to be effective across a range of downstream tasks, but their usefulness is especially pronounced for semantic tasks.

There is currently an ongoing policy discussion regarding the impact of the observed polarization online, how it affects the spread of false information, and what if anything should be done to curtail it. To design and implement effective and efficient interventions in this area, requires a detailed understanding of how the members of polarized communities interact with each other and with outsiders holding opposing views. The focus of this dissertation is the study of polarized Twitter communities, and the spread of disinformation through them, during contentious events. Due to its large number of users, Twitter has become one of the primary social media platforms for acquiring, sharing, and spreading information. However, it has also become a source for misinformation spread and polarization. The effect might not be crucial when the subject in hand is a trivial one, however, during globally concerning events, it gains an undeniable importance. A significant amount of research on information diffusion through this medium has focused on retweeting, despite it being only one potential reaction to information found on Twitter. As shown in this work, this can be misleading, particularly when characterizing the spread of disinformation or when identifying polarized communities. To address this, we explore two different subareas of the identification of stance in Twitter conversations, one that seeks to identify a user’s stance towards a pre-defined target (target stance classification) and one that focuses on the stance to messages from other users (conversation stance classification). Analyzing such conversations is difficult and requires complex natural language processing models that often rely on copious amounts of labeled data. These issues are amplified when working in languages other than English, as labeled resources are scarcer. In the pursuit of the objectives set forth in this work, we developed a weak-labeling methodology for target stance detection which requires minimal labeling effort and constructed one of the first labeled datasets in Spanish for the identification of stance in conversations. This dataset was constructed seeking to provide a unified benchmark for the detection of both polarized online discussions and rumors. These resources are then used for the development of state-of-the-art stance classifiers to explore polarized Twitter communities during a major political event that shocked the South American Region at the end of 2019. For example, results show that a user’s tendency to share information consistent to their views of the government is not consistent to the “filter bubble” explanation for polarization. That is, we show that users from both sides of the ideological spectrum actively engaged with each other (mostly negatively). This implies that the observed phenomenon is more consistent with polarized social media practices consistent with confirmation bias on part of the users.

From autonomous cars to online banking, software nowadays is widely used in safety and security-critical settings. As the complexity and size of software grows, ensuring that it behaves as a programmer intended becomes increasingly difficult, raising concernsabout software���s reliability. To tackle this problem, we wish to provide strong, formal guarantees about the security and correctness of real-world critical software. In this thesis, we therefore advocate for the adoption of a proof-oriented programming paradigm in high-assurance software development. We argue that co-developing programs and proofs yields several benefits: the program structure can simplify the proofs, while proofs can simplify programming and improve the software quality too by, for instance, eliminating unneeded checks and cases. To validate this thesis, we rely on two case studies, which we describenext. Our first case study targets high-performance cryptography, the cornerstone of Internet security. Relying on proof-oriented programming, we develop EverCrypt, a comprehensive collection of verified, high-performance cryptographic functionalities available via a carefully designed API. We first propose a methodology to composeand verify C and assembly cryptographic implementations against shared specifications. We then demonstrate how abstraction and zero-cost generic programming can simplifyverification without sacrificing performance, leading to verified cryptographic code whose performance matches or exceeds the best unverified implementations. EverCrypthas been deployed in several high-profile open-source projects such as Mozilla Firefox and the Linux Kernel.Our second case study investigates the use of proof-oriented programming to develop concurrent, low-level systems. To this end, we present Steel, a novel verificationframework based on a higher-order, impredicative concurrent separation logic shallowly embedded in the F? proof assistant. We show how designing Steel with proofs inmind enables us to automatically separate verification conditions between separationlogic predicates and first-order logic encodeable predicates, allowing us to providepractical automation through a mixture of efficient reflective tactics that focus on the former, and SMT solving for the latter. We finally demonstrate the expressiveness andprogrammability of Steel on a variety of examples, including sequential, self-balancing trees; standard, concurrent data structures such as the Owicki-Gries monotonic counterand Michael and Scott���s 2-lock queue; various synchronization primitives such as libraries for spin locks and fork/join parallelism; and a library for message-passing concurrency on dependently typed channels.

Socially interactive robots present numerous unique programming challenges for interaction developers. While modern authoring tools succeed at making the authoring experience approachable and convenient for developers from a wide variety of backgrounds, they are less successful at targeting assistance to developers based on the specific task or interaction being authored. We propose interaction templates, a data-driven solution for (1) matching in-progress robot programs to candidate task or interaction models and then (2) providing assistance to developers by using the matched models to generate modifications to in-progress programs. In this paper, we present the various dimensions that define first how interaction templates might be used, then how interaction templates may be represented, and finally how they might be collected.

In this thesis, we analyze new theories of clustering, one of the most fundamental tasks in machine learning. We use methods drawing from multiple disciplines, including metric embeddings, spectral algorithms, and group representation theory. 1.We propose a metric that adapts to the shape of data, and show how to quickly compute it. These metrics may be useful for improving k-means clustering methods. 2. We build a spectral partition method with provable theoretical guarantees. This may lead to more theoretically principled spectral clustering methods, as existing methods do not have any such guarantees. Spectral clustering is one of the most popular methods of clustering. 3. We classify all Manhattan distance kernels. Kernel methods are one of the oldest and most established methods of clustering data. This result is a Manhattan distance analog of one of the fundamental results on machine learning kernels. Each of these contributions answers natural questions in machine learning theory. We develop multidisciplinary tools from disciplines ranging from linear algebra to group theory, and combine these with key ideas from metric embeddings and computational geometry.

Submitted in total fulfilment for the degree of Doctor of Philosophy to the Department of Computer Science and Information Technology, School of Engineering and Mathematical Sciences, La Trobe University, Victoria.

In recent years with the fast progress made in neural networks research, supervised machine learning approaches have become increasingly powerful in finding flexible functions to predict target variable Y from input features X. However, most of these complex models require a large amounts of data to train, and often work under the assumption that the data points are i.i.d. In reality these assumptions are very likely to be violated. A simplified notion of this violation is when the training and the test datasets come from different joint distributions (i.e. P train(X, Y ) 6= P test(X, Y )). In this setting, where the training and test datasets are also known as source and target domains respectively, domain adaptation is required to obtain good performance. In particular, when only unlabeled features are observed in the target domain, this setting is referred to as unsupervised domain adaptation, and it will be the main focus of this thesis. Domain adaptation is a wide sub-field of machine learning with the task of designing algorithms to account for this distributional difference under specific assumptions, for the purpose of better prediction performance in the target domain. In this thesis we make use of the data-generating process to address several subproblems of unsupervised domain adaptation. Namely, we first address the problem of unsupervised domain adaptation with multiple labeled source domains and an unlabeled target domain under the conditional-target shift setting, and we present an approach to capture the low-dimensional changes of the joint distribution across domains in order to perform prediction in the target domain. Secondly, we introduce an algorithm to reduce the dimensionality of the data when performing domain adaptation under the covariate shift setting. In particular, we make use of the particular properties of the covariate shift setting in order to reduce the dimensionality of the data such that we preserve relevant predictive information about the target variable Y . We further investigate domain adaptation from the perspective of the data-generating process when addressing the problem using neural networks. Deep neural architectures are commonly used to extract domain-invariant representations from the observed features. However, without labels in the target domain, there is no guarantee that these representations will have relevant predictive information for the target domain data. In this thesis, we investigate techniques to regularize this invariant representation in order to enforce that it has non-trivial structure which contains information which is relevant for predicting Y in the target domain. The first of these techniques is based on mutual information, and the second technique makes use of a novel criterion of distortion of the marginal distribution when transforming it from the source to the target domain.

Nowadays many important applications such as hiring, university admissions, and scientific peer review rely on the collective efforts of a large number of individuals. These applications often operate at an extremely large scale which creates both opportunities and challenges. On the opportunity side, the large amount of data generated in these applications enables a novel data science perspective on the classical problem of decision-making. On the challenge side, in many of these applications, human decision-makers need to interact with various interfaces and algorithms, and follow various policies. When not carefully designed, such interfaces, algorithms, and policies may lead to unintended consequences. Identifying and overcoming such unintended consequences is an important research problem. In this thesis, we explore these opportunities and tackle these challenges with a general goal of understanding and improving distributed human decision-making in a principled manner. One application where the need for improvement is especially strong is scientific peer review. On the one hand, peer review is the backbone of academia, and scientific community agrees on the importance of improvement of the system. On the other hand, peer review is a microcosm of distributed decision-making that features a complex interplay between noise, bias, and incentives. Thus, insights learned from this specific domain apply to many other areas where similar problems arise. All in all, in this thesis, we aim at developing a principled approach towards scientific peer review—an important prerequisite for fair, equitable, and efficient progression of science. The three broad challenges that arise in peer review are noise, bias, and incentives. In this thesis, we work on each of these challenges: Noise and reviewer assignment. A suitable choice of reviewers is a cornerstone of peer review: poor assignment of reviewers to submissions may result in a large amount of noise in decisions. Nowadays, the scale of many publication venues makes it infeasible to manually assign reviewers to submissions. Thus, stakeholders rely on algorithmic support to automate this task. Our work demonstrates that when such algorithmic support is not designed with application-specific constraints in mind, it can result in unintended consequences, compromising fairness and accuracy of the process. More importantly, we make progress in developing better algorithms by (i) designing an assignment algorithm with strong theoretical guarantees and reliable practical performance, and (ii) collecting a dataset that enables other researchers to develop better algorithms for estimating expertise of reviewers in reviewing submissions. Bias and policies. Human decision-making is susceptible to various biases, including identity-related biases (e.g., race and gender) and policy-related biases (e.g., primacy effect). To counteract these biases in peer review, it is crucial to design peer-review policies in an evidence-based manner. With this motivation, we conduct a series of real-world experiments to collect evidence that informs stakeholders in their policy decisions. Our work reveals that while some of the commonplace biases (e.g., herding) are not present in peer review, there are other application-specific biases (e.g., resubmission bias) that significantly impact decisions. Additionally, we demonstrate that reliable testing for biases in peer review often requires novel statistical tools as off-the-shelf techniques may result in false conclusions. Incentives and reviewing. Honesty is a core value of science and peer review is built on the assumption of honesty of everyone involved in the process. However, fierce competition in the academic job market and the large power a single reviewer has over an outcome of a submission create incentives for reviewers to consciously or subconsciously deviate from honest behavior. Our work offers (i) tools to test for such deviations, (ii) empirical evidence of the presence of wrong incentives, and (iii) potential solutions on how to incentivize reviewers to put more effort in writing high-quality reviews.

The exponential growth of Internet of Things (IoT) and cyber-physical systems is resulting in complex environments comprising of various devices interacting with each other and with users. In addition, the rapid advances in Artificial Intelligence are making those devices able to autonomously modify their behaviors through the use of techniques such as reinforcement learning (RL). There is thus the need for an intelligent monitoring system on the network edge with a global view of the environment to autonomously predict optimal device actions. However, it is clear however that ensuring safety and security in such environments is critical. To this effect, we develop a constrained RL framework for IoT environments that determines optimal devices actions with respect to user-defined goals or required functionalities using deep Q learning. We use anomaly based intrusion detection on the network edge to dynamically generate security and safety policies to constrain the RL agent in the framework. We analyze the balance required between ���safety/security��� and ���functionality��� in IoT environments by manipulating the exploration of safe and unsafe benefit state spaces in the RL framework. We instantiate the framework for testing on application layer control in smart home environments, and network layer control including network functionalities like rate control and routing, for SDN based environments.

Emerging automated vehicle (AV) systems are increasingly deploying various types of perception applications to satisfythe requirements of safety and convenience. Platform cost and power consumption concerns also drive automotivesystem designers to engineer better autonomous systems that share minimum system resources. These new trends lead to many challenges for designing resource sharing and scheduling that provide predictable performance for multipleheterogeneous applications. In short, perception, resource management and fault-tolerance support have to run concurrently and work together well. Many prior studies have focused on very specific layers, perception alone, or only fault-tolerant needs ignoring the rest of the system. This thesis studies each of these three inter-related subsystems and proposes frameworks to ensure that perception tasks work well together in coordinated, real-time and fault-tolerantfashion. We analyze the perception needs for AVs and provide practical insight into the real-time resource usage patterns of several software platforms for various applications on an autonomous vehicle. We also introduce detailed methodologies to analyze the computational workloads of the heterogeneous perception tasks. To meet the real-time requirements of AVs, hardware platforms typically include a variety of computing resources ranging from multi-core processors to hardware accelerators such as Graphics-Processing Units (GPUs). We, therefore, introduce novel analytical and systems techniques for running multiple heterogeneous perception applications together. Specifically, we focus on the issues of memory contention, synchronization, and access control for hardware accelerators. In conjunction with analyzable real-time scheduling techniques, we also study the different failure modes of these high-performance computing platforms and develop software strategies to ensure fault-tolerant operations. Our solutions are readily applicableto commodity hardware not only for migrating existing perception applications to single GPU-based embeddedplatforms but also for developing new software and hardware perception systems for AVs.

Ordinary differential equations (ODEs) are quintessential models of real-world continuous behavior in the physical and engineering sciences. They also feature prominently in hybrid system models that combine discrete and continuous dynamics, and interactions thereof. Formal verification of ODEs and hybrid systems is of increasing practical importance because the real-world systems they model, such as control systems and cyber-physical systems, are often required to operate in safetyand mission-critical settings—obtaining comprehensive and trustworthy verification results for continuous and hybrid systems gives a strong measure of confidence that the real-world systems they model operate correctly. This thesis studies deductive verification for ordinary dierential equations with a focus on proofs of their i) safety, ii) liveness, and iii) stability properties. These proofs are compositionally extended to obtain proofs of iv) stability for hybrid (switched) systems. The combination of safety, liveness, and stability is crucial for comprehensive correctness of real-world systems: i) safety of a system model ensures that it always stays within a prescribed set of safe states throughout its operation, ii) liveness ensures that the modeled system will eventually reach its speci ed goal or complete its mission, and iii) & iv) stability ensures that the idealized models are robust to real-world perturbations, which is important for control system designs. The overarching thesis insight is the use of deductive reasoning as a basis for understanding the aforementioned properties and for developing their proofs. Specifically, this thesis uses differential dynamic logic (dL), a logic for deductive verification of hybrid systems, as a trustworthy logical foundation upon which all reasoning principles for safety, liveness, and stability are rigorously derived. The thesis first shows how ODE invariance, a key ingredient in proofs of ODE safety, can be completely axiomatized and reasoned about syntactically in dL. Often, ODE liveness and existence properties are formally proved through reenement-based reasoning in dL, where each reenement step is justified by proving an ODE safety property. Finally, stability properties for ODEs and hybrid systems are specied using dL’s ability to nest safety and liveness modalities with first-order quanti cation. Proofs of those stability specifications build on ODE safety and liveness (sub-)proofs by compositionally adding dL reasoning for the first-order quantifiers and hybrid systems. Formal dL specifications elucidate the logical relationships between the properties studied in this thesis. Indeed, these relationships are re ected in the thesis structure outlined above because they yield chapter-by-chapter identification, buildup, and generalization of the deductive building blocks underlying proof methods for the respective properties. The deductive approach enables such generalizations while retaining utmost confidence in the correctness of the resulting proofs because every step is soundly and syntactically justified using dL’s parsimonious axiomatization. The derived proof principles and insights are put into practice by implementing them in the KeYmaera X theorem prover for hybrid systems based on dL.

Submitted in total fulfilment of the requirements for the degree of Doctor of Philosophy to the School of Computing, Engineering and Mathematical Sciences, La Trobe University, Victoria.