Search

The use of reinforcement learning (RL) to learn policies for just-in-time adaptive interventions (JITAIs) is of significant interest in many behavioral intervention domains including improving levels of physical activity. In a messaging-based physical activity JITAI, a mobile health app is typically used to send messages to a participant to encourage engagement in physical activity. In this setting, RL methods can be used to learn what intervention options to provide to a participant in different contexts. However, deploying RL methods in real physical activity adaptive interventions comes with challenges: the cost and time constraints of real intervention studies result in limited data to learn adaptive intervention policies. Further, commonly used RL simulation environments have dynamics that are of limited relevance to physical activity adaptive interventions and thus shed little light on what RL methods may be optimal for this challenging application domain. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods that address the significant challenges of policy learning for adaptive behavioral interventions., Comment: Accepted at NeurIPS 2024 workshop on Behavioral ML

Home repair and installation services require technicians to visit customers and resolve tasks of different complexity. Technicians often have heterogeneous skills and working experiences. The geographical spread of customers makes achieving only perfect matches between technician skills and task requirements impractical. Additionally, technicians are regularly absent due to sickness. With non-perfect assignments regarding task requirement and technician skill, some tasks may remain unresolved and require a revisit and rework. Companies seek to minimize customer inconvenience due to delay. We model the problem as a sequential decision process where, over a number of service days, customers request service while heterogeneously skilled technicians are routed to serve customers in the system. Each day, our policy iteratively builds tours by adding "important" customers. The importance bases on analytical considerations and is measured by respecting routing efficiency, urgency of service, and risk of rework in an integrated fashion. We propose a state-dependent balance of these factors via reinforcement learning. A comprehensive study shows that taking a few non-perfect assignments can be quite beneficial for the overall service quality. We further demonstrate the value provided by a state-dependent parametrization.

Restaurant meal delivery has been rapidly growing in the last few years. The main challenges in operating it are the temporally and spatially dispersed stochastic demand that arrives from customers all over town as well as the customers' expectation of timely and fresh delivery. To overcome these challenges a new business concept emerged, "Ghost kitchens". This concept proposes synchronized food preparation of several restaurants in a central complex, exploiting consolidation benefits. However, dynamically scheduling food preparation and delivery is challenging and we propose operational strategies for the effective operations of ghost kitchens. We model the problem as a sequential decision process. For the complex, combinatorial decision space of scheduling order preparations, consolidating orders to trips, and scheduling trip departures, we propose a large neighborhood search procedure based on partial decisions and driven by analytical properties. Within the large neighborhood search, decisions are evaluated via a value function approximation, enabling anticipatory and real-time decision making. We show the effectiveness of our method and demonstrate the value of ghost kitchens compared to conventional meal delivery systems. We show that both integrated optimization of cook scheduling and vehicle dispatching, as well as anticipation of future demand and decisions, are essential for successful operations. We further derive several managerial insights, amongst others, that companies should carefully consider the trade-off between fast delivery and fresh food.

Wearable sensors enable health researchers to continuously collect data pertaining to the physiological state of individuals in real-world settings. However, such data can be subject to extensive missingness due to a complex combination of factors. In this work, we study the problem of imputation of missing step count data, one of the most ubiquitous forms of wearable sensor data. We construct a novel and large scale data set consisting of a training set with over 3 million hourly step count observations and a test set with over 2.5 million hourly step count observations. We propose a domain knowledge-informed sparse self-attention model for this task that captures the temporal multi-scale nature of step-count data. We assess the performance of the model relative to baselines and conduct ablation studies to verify our specific model designs., Comment: Accepted by Conference on Health, Inference, and Learning (CHIL) 2024

Localization is a critical technology for various applications ranging from navigation and surveillance to assisted living. Localization systems typically fuse information from sensors viewing the scene from different perspectives to estimate the target location while also employing multiple modalities for enhanced robustness and accuracy. Recently, such systems have employed end-to-end deep neural models trained on large datasets due to their superior performance and ability to handle data from diverse sensor modalities. However, such neural models are often trained on data collected from a particular set of sensor poses (i.e., locations and orientations). During real-world deployments, slight deviations from these sensor poses can result in extreme inaccuracies. To address this challenge, we introduce FlexLoc, which employs conditional neural networks to inject node perspective information to adapt the localization pipeline. Specifically, a small subset of model weights are derived from node poses at run time, enabling accurate generalization to unseen perspectives with minimal additional overhead. Our evaluations on a multimodal, multiview indoor tracking dataset showcase that FlexLoc improves the localization accuracy by almost 50% in the zero-shot case (no calibration data available) compared to the baselines. The source code of FlexLoc is available at https://github.com/nesl/FlexLoc.

Constantly locating moving objects, i.e., geospatial tracking, is essential for autonomous building infrastructure. Accurate and robust geospatial tracking often leverages multimodal sensor fusion algorithms, which require large datasets with time-aligned, synchronized data from various sensor types. However, such datasets are not readily available. Hence, we propose GDTM, a nine-hour dataset for multimodal object tracking with distributed multimodal sensors and reconfigurable sensor node placements. Our dataset enables the exploration of several research problems, such as optimizing architectures for processing multimodal data, and investigating models' robustness to adverse sensing conditions and sensor placement variances. A GitHub repository containing the code, sample data, and checkpoints of this work is available at https://github.com/nesl/GDTM.

The success of self-supervised contrastive learning hinges on identifying positive data pairs, such that when they are pushed together in embedding space, the space encodes useful information for subsequent downstream tasks. Constructing positive pairs is non-trivial as the pairing must be similar enough to reflect a shared semantic meaning, but different enough to capture within-class variation. Classical approaches in vision use augmentations to exploit well-established invariances to construct positive pairs, but invariances in the time-series domain are much less obvious. In our work, we propose a novel method of using a learned measure for identifying positive pairs. Our Retrieval-Based Reconstruction (REBAR) measure measures the similarity between two sequences as the reconstruction error that results from reconstructing one sequence with retrieved information from the other. Then, if the two sequences have high REBAR similarity, we label them as a positive pair. Through validation experiments, we show that the REBAR error is a predictor of mutual class membership. Once integrated into a contrastive learning framework, our REBAR method learns an embedding that achieves state-of-the-art performance on downstream tasks across various modalities., Comment: ICLR 2024 | Code available at: https://github.com/maxxu05/rebar

In manufacturing sectors such as textiles and electronics, manual processes are a fundamental part of production. The analysis and monitoring of the processes is necessary for efficient production design. Traditional methods for analyzing manual processes are complex, expensive, and inflexible. Compared to established approaches such as Methods-Time-Measurement (MTM), machine learning (ML) methods promise: Higher flexibility, self-sufficient & permanent use, lower costs. In this work, based on a video stream, the current motion class in a manual assembly process is detected. With information on the current motion, Key-Performance-Indicators (KPIs) can be derived easily. A skeleton-based action recognition approach is taken, as this field recently shows major success in machine vision tasks. For skeleton-based action recognition in manual assembly, no sufficient pre-work could be found. Therefore, a ML pipeline is developed, to enable extensive research on different (pre-) processing methods and neural nets. Suitable well generalizing approaches are found, proving the potential of ML to enhance analyzation of manual processes. Models detect the current motion, performed by an operator in manual assembly, but the results can be transferred to all kinds of manual processes., Comment: 16 pages, 6 figures. Find peer-reviewed version in Proceedings of IntelliSys 2023

Visual object tracking has seen significant progress in recent years. However, the vast majority of this work focuses on tracking objects within the image plane of a single camera and ignores the uncertainty associated with predicted object locations. In this work, we focus on the geospatial object tracking problem using data from a distributed camera network. The goal is to predict an object's track in geospatial coordinates along with uncertainty over the object's location while respecting communication constraints that prohibit centralizing raw image data. We present a novel single-object geospatial tracking data set that includes high-accuracy ground truth object locations and video data from a network of four cameras. We present a modeling framework for addressing this task including a novel backbone model and explore how uncertainty calibration and fine-tuning through a differentiable tracker affect performance.

Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual's time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information., Comment: Accepted at UAI 2023

Quaoar is a classical Trans-Neptunian Object (TNO) with an area equivalent diameter of 1,100 km and an orbital semi-major axis of 43.3 astronomical units. Based on stellar occultations observed between 2018 and 2021, an inhomogeneous ring (Q1R, Quaoar's first ring) was detected around this body. Aims. A new stellar occultation by Quaoar was observed on August 9th, 2022 aiming to improve Quaoar's shape models and the physical parameters of Q1R while searching for additional material around the body. Methods. The occultation provided nine effective chords across Quaoar, pinning down its size, shape, and astrometric position. Large facilities, such as Gemini North and the Canada-France-Hawaii Telescope (CFHT), were used to obtain high acquisition rates and signal-to-noise ratios. The light curves were also used to characterize the Q1R ring (radial profiles and orbital elements). Results. Quaoar's elliptical fit to the occultation chords yields the limb with an apparent semi-major axis of $579.5\pm4.0$ km, apparent oblateness of $0.12\pm0.01$, and area-equivalent radius of $543\pm2$ km. Quaoar's limb orientation is consistent with Q1R and Weywot orbiting in Quaoar's equatorial plane. The orbital radius of Q1R is refined to a value of $4,057\pm6$ km. The radial opacity profile of the more opaque ring profile follows a Lorentzian shape that extends over 60 km, with a full width at half maximum (FWHM) of $\sim5$ km and a peak normal optical depth of 0.4. Besides the secondary events related to the already reported rings, new secondary events detected during the August 2022 occultation in three different data sets are consistent with another ring around Quaoar with a radius of $2,520\pm20$ km, assuming the ring is circular and co-planar with Q1R. This new ring has a typical width of 10 km and a normal optical depth of $\sim$0.004. Like Q1R, it also lies outside Quaoar's classical Roche limit., Comment: Accepted for publication in Astronomy & Astrophysics (17-April-2023). 18 pages, 12 figures

Metallic pipes and other containers are widely used to store and transport toxic gases and liquids. Various sensors have been designed to monitor the environment inside metallic pipes and containers, such as pressure, liquid-level, and chemical sensors. Moreover, sensors are also used to inspect and detect pipe leakages. However, sensors are usually placed outside of metallic pipes and containers and use ultrasound to monitor the internal unseen environment. This is mainly due to the fact that internal sensors cannot communicate with external data sinks without cables, but using cables can dramatically affect the metal-sealed structure. Wireless communication is desirable to communicate with internal sensors, but it experiences high attenuation losses since metal can block wireless signals due to its high conductivity. This paper develops a low-cost through-metal communication system prototype using off-the-shelf electronic components. The system is fully reconfigurable, and arbitrary modulation and coding schemes can be implemented. We design the transmit module which includes a signal processing microcontroller, an amplifier, and a transmit coil, and the receive module which includes a receive coil, an amplifier, and a microcontroller with demodulation algorithms and bit-error-rate (BER) calculations. The performance of the prototype is evaluated using various symbol rates, distances, and transmission power. The results show that the communication system can achieve a 500 bps data rate with 0.01 BER and 3.4 cm communication range when penetrating an Aluminum pipe with 7 mm thickness.

This policy brief is one in a series which examines selected topics related to Wisconsin's educator workforce. This brief presents a descriptive profile of Wisconsin's principal and assistant principal labor force over the past 20 years in terms of key attributes such as counts of both types of administrators, selected demographic characteristics (including gender, race/ethnicity, age, and years of experience), and distribution across locale type (urban, suburban, town, rural) and region of the state. Specific questions this brief addresses are as follows: (1) How has the number of principals and assistant principals working in Wisconsin public schools changed over the past two decades, and how does this compare to trends in student enrollment; (2) How has the distribution of Wisconsin principals and assistant principals by locale type and region of the state (including the state's largest districts) changed over the past two decades; and (3) How have selected characteristics of Wisconsin's principals and assistant principals changed over time? [For "Selected Characteristics of Teachers in Wisconsin Public Schools, 1999-00 to 2020-21," see ED624420.]

This policy brief, produced by the Wisconsin Center for Education Research (WCER) at the University of Wisconsin-Madison as part of the Region 10 Comprehensive Center, is one in a series examining selected topics related to Wisconsin's educator workforce. The goal of this brief is to summarize key trends in the use of emergency credentials within Wisconsin's public schools in recent years, which is a topic of interest for at least two reasons. A first reason is that increased reliance by Wisconsin school districts on emergency credentials as a strategy for filling vacant positions would serve as a meaningful indicator that widely-reported shortages of educators are worsening. A second policy implication associated with emergency credentials is a potential equity issue. To the extent that emergency credentials are being used by some districts and schools more than others, particularly for teaching positions and by districts and schools with higher shares of traditionally marginalized students, state and local policymakers should question whether additional supports and resources may be necessary to ensure that all students have access to highly-qualified educators.

Individual participant data (IPD) meta-analyses of randomised trials are considered a reliable way to assess participant-level treatment effect modifiers but may not make the best use of the available data. Traditionally, effect modifiers are explored one covariate at a time, which gives rise to the possibility that evidence of treatment-covariate interaction may be due to confounding from a different, related covariate. We aimed to evaluate current practice when estimating treatment-covariate interactions in IPD meta-analysis, specifically focusing on involvement of additional covariates in the models. We reviewed 100 IPD meta-analyses of randomised trials, published between 2015 and 2020, that assessed at least one treatment-covariate interaction. We identified four approaches to handling additional covariates: (1) Single interaction model (unadjusted): No additional covariates included (57/100 studies); (2) Single interaction model (adjusted): Adjustment for the main effect of at least one additional covariate (35/100); (3) Multiple interactions model: Adjustment for at least one two-way interaction between treatment and an additional covariate (3/100); and (4) Three-way interaction model: Three-way interaction formed between treatment, the additional covariate and the potential effect modifier (5/100). IPD is not being utilised to its fullest extent. In an exemplar dataset, we demonstrate how these approaches can lead to different conclusions. Researchers should adjust for additional covariates when estimating interactions in IPD meta-analysis providing they adjust their main effects, which is already widely recommended. Further, they should consider whether more complex approaches could provide better information on who might benefit most from treatments, improving patient choice and treatment policy and practice., Comment: 23 pages, 3 figures, 4 tables

High-level autonomy requires discrete and continuous reasoning to decide both what actions to take and how to execute them. Integrated Task and Motion Planning (TMP) algorithms solve these hybrid problems jointly to consider constraints between the discrete symbolic actions (i.e., the task plan) and their continuous geometric realization (i.e., motion plans). This joint approach solves more difficult problems than approaches that address the task and motion subproblems independently. TMP algorithms combine and extend results from both task and motion planning. TMP has mainly focused on computational performance and completeness and less on solution optimality. Optimal TMP is difficult because the independent optima of the subproblems may not be the optimal integrated solution, which can only be found by jointly optimizing both plans. This paper presents Task and Motion Informed Trees (TMIT*), an optimal TMP algorithm that combines results from makespan-optimal task planning and almost-surely asymptotically optimal motion planning. TMIT* interleaves asymmetric forward and reverse searches to delay computationally expensive operations until necessary and perform an efficient informed search directly in the problem's hybrid state space. This allows it to solve problems quickly and then converge towards the optimal solution with additional computational time, as demonstrated on the evaluated robotic-manipulation benchmark problems., Comment: 9 pages, 5 figures. Accepted to IEEE RA-L and IROS 2022

We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. The final of the 4 provided datasets contained real noise from the O3a observing run and signals up to a duration of 20 seconds with the inclusion of precession effects and higher order modes. We present the average sensitivity distance and runtime for the 6 entered algorithms derived from 1 month of test data unknown to the participants prior to submission. Of these, 4 are machine learning algorithms. We find that the best machine learning based algorithms are able to achieve up to 95% of the sensitive distance of matched-filtering based production analyses for simulated Gaussian noise at a false-alarm rate (FAR) of one per month. In contrast, for real noise, the leading machine learning search achieved 70%. For higher FARs the differences in sensitive distance shrink to the point where select machine learning submissions outperform traditional search algorithms at FARs $\geq 200$ per month on some datasets. Our results show that current machine learning search algorithms may already be sensitive enough in limited parameter regions to be useful for some production settings. To improve the state-of-the-art, machine learning algorithms need to reduce the false-alarm rates at which they are capable of detecting signals and extend their validity to regions of parameter space where modeled searches are computationally expensive to run. Based on our findings we compile a list of research areas that we believe are the most important to elevate machine learning searches to an invaluable tool in gravitational-wave signal detection., Comment: 25 pages, 6 figures, 4 tables, additional material available at https://github.com/gwastro/ml-mock-data-challenge-1

In this paper we present BayesLDM, a system for Bayesian longitudinal data modeling consisting of a high-level modeling language with specific features for modeling complex multivariate time series data coupled with a compiler that can produce optimized probabilistic program code for performing inference in the specified model. BayesLDM supports modeling of Bayesian network models with a specific focus on the efficient, declarative specification of dynamic Bayesian Networks (DBNs). The BayesLDM compiler combines a model specification with inspection of available data and outputs code for performing Bayesian inference for unknown model parameters while simultaneously handling missing data. These capabilities have the potential to significantly accelerate iterative modeling workflows in domains that involve the analysis of complex longitudinal data by abstracting away the process of producing computationally efficient probabilistic inference code. We describe the BayesLDM system components, evaluate the efficiency of representation and inference optimizations and provide an illustrative example of the application of the system to analyzing heterogeneous and partially observed mobile health data., Comment: Accepted at IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) 2022

Multiquery planning algorithms find paths between various different starts and goals in a single search space. They are designed to do so efficiently by reusing information across planning queries. This information may be computed before or during the search and often includes knowledge of valid paths. Using known valid paths to solve an individual planning query takes less computational effort than finding a completely new solution. This allows multiquery algorithms, such as PRM*, to outperform single-query algorithms, such as RRT*, on many problems but their relative performance depends on how much information is reused. Despite this, few multiquery planners explicitly seek to maximize path reuse and, as a result, many do not consistently outperform single-query alternatives. This paper presents Effort Informed Roadmaps (EIRM*), an almost-surely asymptotically optimal multiquery planning algorithm that explicitly prioritizes reusing computational effort. EIRM* uses an asymmetric bidirectional search to identify existing paths that may help solve an individual planning query and then uses this information to order its search and reduce computational effort. This allows it to find initial solutions up to an order-of-magnitude faster than state-of-the-art planning algorithms on the tested abstract and robotic multiquery planning problems., Comment: 16 pages, 7 figures, 1 table. Video and code available at https://robotic-esp.com/code/eirmstar/

Bayesian methods hold significant promise for improving the uncertainty quantification ability and robustness of deep neural network models. Recent research has seen the investigation of a number of approximate Bayesian inference methods for deep neural networks, building on both the variational Bayesian and Markov chain Monte Carlo (MCMC) frameworks. A fundamental issue with MCMC methods is that the improvements they enable are obtained at the expense of increased computation time and model storage costs. In this paper, we investigate the potential of sparse network structures to flexibly trade-off model storage costs and inference run time against predictive performance and uncertainty quantification ability. We use stochastic gradient MCMC methods as the core Bayesian inference method and consider a variety of approaches for selecting sparse network structures. Surprisingly, our results show that certain classes of randomly selected substructures can perform as well as substructures derived from state-of-the-art iterative pruning methods while drastically reducing model training times., Comment: Preprint. Work in progress

We report the fabrication and measurement of top gated epitaxial graphene p-n junctions where exfoliated hexagonal boron nitride (h-BN) is used as the gate dielectric. The four-terminal longitudinal resistance across a single junction is well quantized at the von Klitzing constant R_K with a relative uncertainty of 10-7. After the exploration of numerous parameter spaces, we summarize the conditions upon which these devices could function as potential resistance standards. Furthermore, we offer designs of programmable electrical resistance standards over six orders of magnitude by using external gating.

We present the fourth Open Gravitational-wave Catalog (4-OGC) of binary neutron star (BNS), binary black hole (BBH) and neutron star-black hole (NSBH) mergers. The catalog includes observations from 2015-2020 covering the first through third observing runs (O1, O2, O3a, O3b) of Advanced LIGO and Advanced Virgo. The updated catalog includes 7 BBH mergers which were not previously reported with high significance during O3b for a total of 94 observations: 90 BBHs, 2 NSBHs, and 2 BNSs. The most confident new detection, GW200318_191337, has component masses $49.1^{+16.4}_{-12.0}\textrm{M}_\odot$ and $31.6^{+12.0}_{-11.6}\textrm{M}_\odot$; its redshift of $0.84^{+0.4}_{-0.35}$ ($90\%$ credible interval) may make it the most distant merger so far. We estimate the merger rate of BBH sources, assuming a powerlaw mass distribution containing an additive Gaussian peak, to be $16.5_{-6.2}^{+10.4} (25.0_{-8.0}^{+12.6})$ Gpc$^{-3}$ yr$^{-1}$ at a redshift of $z=0$ ($0.2$). For BNS and NSBH sources, we estimate a merger rate of $200^{+309}_{-148}$ Gpc$^{-3}$ yr$^{-1}$ and $19^{+30}_{-14}$ Gpc$^{-3}$ yr$^{-1}$, respectively, assuming the known sources are representative of the total population. We provide reference parameter estimates for each of these sources using an up-to-date model accounting for instrumental calibration uncertainty. The corresponding data release also includes our full set of sub-threshold candidates., Comment: v2, 22 pages, 7 figures, 5 tables, data release at https://github.com/gwastro/4-ogc, submitted to ApJ, comments welcome

Approximate Bayesian deep learning methods hold significant promise for addressing several issues that occur when deploying deep learning components in intelligent systems, including mitigating the occurrence of over-confident errors and providing enhanced robustness to out of distribution examples. However, the computational requirements of existing approximate Bayesian inference methods can make them ill-suited for deployment in intelligent IoT systems that include lower-powered edge devices. In this paper, we present a range of approximate Bayesian inference methods for supervised deep learning and highlight the challenges and opportunities when applying these methods on current edge hardware. We highlight several potential solutions to decreasing model storage requirements and improving computational scalability, including model pruning and distillation methods.

Optimal path planning is the problem of finding a valid sequence of states between a start and goal that optimizes an objective. Informed path planning algorithms order their search with problem-specific knowledge expressed as heuristics and can be orders of magnitude more efficient than uninformed algorithms. Heuristics are most effective when they are both accurate and computationally inexpensive to evaluate, but these are often conflicting characteristics. This makes the selection of appropriate heuristics difficult for many problems. This paper presents two almost-surely asymptotically optimal sampling-based path planning algorithms to address this challenge, Adaptively Informed Trees (AIT*) and Effort Informed Trees (EIT*). These algorithms use an asymmetric bidirectional search in which both searches continuously inform each other. This allows AIT* and EIT* to improve planning performance by simultaneously calculating and exploiting increasingly accurate, problem-specific heuristics. The benefits of AIT* and EIT* relative to other sampling-based algorithms are demonstrated on twelve problems in abstract, robotic, and biomedical domains optimizing path length and obstacle clearance. The experiments show that AIT* and EIT* outperform other algorithms on problems optimizing obstacle clearance, where a priori cost heuristics are often ineffective, and still perform well on problems minimizing path length, where such heuristics are often effective., Comment: 25 pages, 12 figures, 2 tables. Videos available at https://www.youtube.com/watch?v=aC2xBwOcDoo and https://www.youtube.com/playlist?list=PLbaQBz4TuPczfN6PN6NkfmlnXpcf79Aq_

Local delivery platforms are collaborative undertakings where local stores offer instant-delivery to local customers ordering their products online. Offering such delivery services both cost-efficiently and reliably is one of the main challenges for local delivery platforms, as they face a complex, dynamic, stochastic dynamic pickup-and-delivery problem. Orders need to be consolidated to increase the efficiency of the delivery operations and thereby enable a high service guarantee towards the customer and stores. But, waiting for consolidation opportunities may jeopardize delivery service reliability in the future, and thus requires anticipating future demand. This paper introduces a generic approach to balance the consolidation potential and delivery urgency of orders. Specifically, it presents a newly developed parameterized Cost-Function Approximation (CFA) approach that modifies a set-packing formulation with two parameters. This CFA approach not only anticipates future demand but also utilizes column generation to search the large decision space related to pickup-and-delivery problems fast. Inspired by a motivating application in the city of Groningen, the Netherlands, numerical experiments show that our CFA approach strongly increases perceived customer satisfaction while lowering the total travel time of the vehicles compared to various benchmark policies. Furthermore, our CFA also reduces the percentage of late deliveries, and their lateness, to a minimum. Finally, our approach may assist managers in practice to manage the non-trivial balance between consolidation opportunity and delivery urgency.

Gravitational waves from the coalescence of compact-binary sources are now routinely observed by Earth bound detectors. The most sensitive search algorithms convolve many different pre-calculated gravitational waveforms with the detector data and look for coincident matches between different detectors. Machine learning is being explored as an alternative approach to building a search algorithm that has the prospect to reduce computational costs and target more complex signals. In this work we construct a two-detector search for gravitational waves from binary black hole mergers using neural networks trained on non-spinning binary black hole data from a single detector. The network is applied to the data from both observatories independently and we check for events coincident in time between the two. This enables the efficient analysis of large quantities of background data by time-shifting the independent detector data. We find that while for a single detector the network retains $91.5\%$ of the sensitivity matched filtering can achieve, this number drops to $83.9\%$ for two observatories. To enable the network to check for signal consistency in the detectors, we then construct a set of simple networks that operate directly on data from both detectors. We find that none of these simple two-detector networks are capable of improving the sensitivity over applying networks individually to the data from the detectors and searching for time coincidences., Comment: 11 pages, 4 figures, 2 tables

Irregularly sampled time series commonly occur in several domains where they present a significant challenge to standard deep learning models. In this paper, we propose a new deep learning framework for probabilistic interpolation of irregularly sampled time series that we call the Heteroscedastic Temporal Variational Autoencoder (HeTVAE). HeTVAE includes a novel input layer to encode information about input observation sparsity, a temporal VAE architecture to propagate uncertainty due to input sparsity, and a heteroscedastic output layer to enable variable uncertainty in output interpolations. Our results show that the proposed architecture is better able to reflect variable uncertainty through time due to sparse and irregular sampling than a range of baseline and traditional models, as well as recently proposed deep latent variable models that use homoscedastic output layers., Comment: Accepted at International Conference on Learning Representations (ICLR) 2022

Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posteriors that are approximated with, potentially crude, surrogates. This difficulty has engendered loss-calibrated techniques that aim to learn posterior approximations that favor high-utility decisions. In this paper, focusing on Bayesian neural networks, we develop methods for correcting approximate posterior predictive distributions encouraging them to prefer high-utility decisions. In contrast to previous work, our approach is agnostic to the choice of the approximate inference algorithm, allows for efficient test time decision making through amortization, and empirically produces higher quality decisions. We demonstrate the effectiveness of our approach through controlled experiments spanning a diversity of tasks and datasets., Comment: Accepted to Conference on Uncertainty in AI (UAI) '21

Compact binary systems emit gravitational radiation which is potentially detectable by current Earth bound detectors. Extracting these signals from the instruments' background noise is a complex problem and the computational cost of most current searches depends on the complexity of the source model. Deep learning may be capable of finding signals where current algorithms hit computational limits. Here we restrict our analysis to signals from non-spinning binary black holes and systematically test different strategies by which training data is presented to the networks. To assess the impact of the training strategies, we re-analyze the first published networks and directly compare them to an equivalent matched-filter search. We find that the deep learning algorithms can generalize low signal-to-noise ratio (SNR) signals to high SNR ones but not vice versa. As such, it is not beneficial to provide high SNR signals during training, and fastest convergence is achieved when low SNR samples are provided early on. During testing we found that the networks are sometimes unable to recover any signals when a false alarm probability $<10^{-3}$ is required. We resolve this restriction by applying a modification we call unbounded Softmax replacement (USR) after training. With this alteration we find that the machine learning search retains $\geq 91.5\%$ of the sensitivity of the matched-filter search down to a false-alarm rate of 1 per month., Comment: 18 pages, 12 figures, 3 tables, supplemental materials at https://github.com/gwastro/ml-training-strategies

We present the third Open Gravitational-wave Catalog (3-OGC) of compact-binary coalescences, based on the analysis of the public LIGO and Virgo data from 2015 through 2019 (O1, O2, O3a). Our updated catalog includes a population of 57 observations, including four binary black hole mergers that had not previously been reported. This consists of 55 binary black hole mergers and the two binary neutron star mergers GW170817 and GW190425. We find no additional significant binary neutron star or neutron star--black hole merger events. The most confident new detection is the binary black hole merger GW190925\_232845 which was observed by the LIGO Hanford and Virgo observatories with $\mathcal{P}_{\textrm{astro}} > 0.99$; its primary and secondary component masses are $20.2^{+3.9}_{-2.5} M_{\odot}$ and $15.6^{+2.1}_{-2.6} M_{\odot}$, respectively. We estimate the parameters of all binary black hole events using an up-to-date waveform model that includes both sub-dominant harmonics and precession effects. To enable deep follow-up as our understanding of the underlying populations evolves, we make available our comprehensive catalog of events, including the sub-threshold population of candidates, and the posterior samples of our source parameter estimates., Comment: 17 pages, 5 figures, 4 tables, v3 updated to match ApJ accepted version. Supplementary material at https://github.com/gwastro/3-ogc

Obstacle clearance in state space is an important optimization objective in path planning because it can result in safe paths. This technical report presents admissible solution- and path-cost heuristics for this objective, which can be used to improve the performance of informed path planning algorithms., Comment: 6 pages, 3 figures

Buildings rarely perform as designed/simulated and and there are numerous tangible benefits if this gap is reconciled. A new scientific yet pragmatic methodology - called Enhanced Parameter Estimation (EPE) - is proposed that allows physically relevant parameter estimation rather than a blind force-fit to energy use data. It calibrates a rapidly and inexpensively created simulation model of the building in two stages: (a) building shell calibration with the HVAC system is replaced by an by an ideal system that meets the loads. EPE identifies a small number of high-level heat flows in the energy balance, calculates them with specifically tailored individual driving functions, introduces physically significant parameters to best accomplish energy balance, and, estimates the parameters and their uncertainty bounds. Calibration is thus done with corrective heat flows without any arbitrary tuning of input parameters, (b) HVAC system calibration with the building shell replaced by a box with only process loads; as many parameters as the data allows are estimated. Calibration accuracy is enhanced by machine learning of the residual errors. The EPE methodology is demonstrated through: a synthetic building and one an actual 75,000 Sq.Ft. building in Pennsylvania. A subsequent paper will provide further details and applications., Comment: 25 pages, 13 figures, 3 tables

There has been a paradigm-shift in urban logistic services in the last years; demand for real-time, instant mobility and delivery services grows. This poses new challenges to logistic service providers as the underlying stochastic dynamic vehicle routing problems (SDVRPs) require anticipatory real-time routing actions. Searching the combinatorial action space for efficient routing actions is by itself a complex task of mixed-integer programming (MIP) well-known by the operations research community. This complexity is now multiplied by the challenge of evaluating such actions with respect to their effectiveness given future dynamism and uncertainty, a potentially ideal case for reinforcement learning (RL) well-known by the computer science community. For solving SDVRPs, joint work of both communities is needed, but as we show, essentially non-existing. Both communities focus on their individual strengths leaving potential for improvement. Our survey paper highlights this potential in research originating from both communities. We point out current obstacles in SDVRPs and guide towards joint approaches to overcome them.

Irregular sampling occurs in many time series modeling applications where it presents a significant challenge to standard deep learning models. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. In this paper, we propose a new deep learning framework for this setting that we call Multi-Time Attention Networks. Multi-Time Attention Networks learn an embedding of continuous-time values and use an attention mechanism to produce a fixed-length representation of a time series containing a variable number of observations. We investigate the performance of this framework on interpolation and classification tasks using multiple datasets. Our results show that the proposed approach performs as well or better than a range of baseline and recently proposed models while offering significantly faster training times than current state-of-the-art methods., Comment: Accepted at International Conference on Learning Representations (ICLR) 2021

Major teacher evaluation changes have occurred across the nation during the past decade. Pressed by federal education initiatives and encouraged by research on promising teacher evaluation practices, states and school districts have implemented evaluation systems using new measures of educator practice and student learning. A growing body of research has examined teacher evaluation policies and related impacts. Relatively less research focuses on the roles of principals, teachers, coaches, and peers. Wisconsin adopted a system for educator evaluation that includes common measures, similar training requirements, and an emphasis on educator growth and development. The evaluation of the Wisconsin Educator Effectiveness System has provided informative state, district and school reports on teacher and principal perceptions of the system. There is much to learn, however, about how schools are carrying out evaluation practices to support educator improvement. This report presents findings from the Learning-Centered Evaluation Study carried out during the 2017-2018 school year. We examine school-level educator effectiveness practices and outcomes within Wisconsin districts focusing evaluation efforts on educator improvement rather than accountability. The report provides background on the system development in Wisconsin, summarizes our study design, and presents findings on learning-centered practices and observations on how evaluation affects teaching. The report includes a policy response from the study sponsor and thoughts on future studies. [The study is part of the evaluation and research agenda of the Wisconsin Educator Effectiveness Research Partnership.]

Irregularly sampled time series data arise naturally in many application domains including biology, ecology, climate science, astronomy, and health. Such data represent fundamental challenges to many classical models from machine learning and statistics due to the presence of non-uniform intervals between observations. However, there has been significant progress within the machine learning community over the last decade on developing specialized models and architectures for learning from irregularly sampled univariate and multivariate time series data. In this survey, we first describe several axes along which approaches to learning from irregularly sampled time series differ including what data representations they are based on, what modeling primitives they leverage to deal with the fundamental problem of irregular sampling, and what inference tasks they are designed to perform. We then survey the recent literature organized primarily along the axis of modeling primitives. We describe approaches based on temporal discretization, interpolation, recurrence, attention and structural invariance. We discuss similarities and differences between approaches and highlight primary strengths and weaknesses., Comment: Presented at NeurIPS 2020 Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA)

Motion planning is a fundamental problem in autonomous robotics that requires finding a path to a specified goal that avoids obstacles and takes into account a robot's limitations and constraints. It is often desirable for this path to also optimize a cost function, such as path length. Formal path-quality guarantees for continuously valued search spaces are an active area of research interest. Recent results have proven that some sampling-based planning methods probabilistically converge toward the optimal solution as computational effort approaches infinity. This survey summarizes the assumptions behind these popular asymptotically optimal techniques and provides an introduction to the significant ongoing research on this topic., Comment: Posted with permission from the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4. Copyright 2021 by Annual Reviews, https://www.annualreviews.org/. 25 pages. 2 figures

We present the prospects for the early (pre-merger) detection and localization of compact-binary coalescences using gravitational waves over the next 10 years. Early warning can enable the direct observation of the prompt and early electromagnetic emission of a neutron star merger. We examine the capabilities of the ground based detectors at their "Design" sensitivity (2021-2022), the planned "A+" upgrade (2024-2026), and the envisioned "Voyager" concept (late 2020's). We find that for a fiducial rate of binary neutron star mergers of $1000 ~\mathrm{Gpc}^{-3} \mathrm{yr}^{-1}$, the Design, A+, and Voyager era networks can provide 18, 54, and 195s of warning for one source per year of observing, respectively, with a sky localization area $<$100 deg$^2$ at a $90\%$ credible level. At the same rate, the A+ and Voyager era networks will be able to provide 9 and 43s of warning, respectively, for a source with $<$10 deg$^2$ localization area. We compare the idealized search sensitivity to that achieved by the PyCBC Live search tuned for pre-merger detection. The gravitational-wave community will be prepared to produce pre-merger alerts. Our results motivate the operation of observatories with wide fields-of-view, automation, and the capability for fast slewing to observe simultaneously with the gravitational-wave network., Comment: 11 pages, 4 figures, data release at https://github.com/gwastro/gw-merger-forecasting, updated to match ApJL version

Irregularly-sampled time series occur in many domains including healthcare. They can be challenging to model because they do not naturally yield a fixed-dimensional representation as required by many standard machine learning models. In this paper, we consider irregular sampling from the perspective of missing data. We model observed irregularly-sampled time series data as a sequence of index-value pairs sampled from a continuous but unobserved function. We introduce an encoder-decoder framework for learning from such generic indexed sequences. We propose learning methods for this framework based on variational autoencoders and generative adversarial networks. For continuous irregularly-sampled time series, we introduce continuous convolutional layers that can efficiently interface with existing neural network architectures. Experiments show that our models are able to achieve competitive or better classification results on irregularly-sampled multivariate time series compared to recent RNN models while offering significantly faster training times.

The demand for same-day delivery (SDD) has increased rapidly in the last few years and has particularly boomed during the COVID-19 pandemic. The fast growth is not without its challenge. In 2016, due to low concentrations of memberships and far distance from the depot, certain minority neighborhoods were excluded from receiving Amazon's SDD service, raising concerns about fairness. In this paper, we study the problem of offering fair SDD-service to customers. The service area is partitioned into different regions. Over the course of a day, customers request for SDD service, and the timing of requests and delivery locations are not known in advance. The dispatcher dynamically assigns vehicles to make deliveries to accepted customers before their delivery deadline. In addition to the overall service rate (utility), we maximize the minimal regional service rate across all regions (fairness). We model the problem as a multi-objective Markov decision process and develop a deep Q-learning solution approach. We introduce a novel transformation of learning from rates to actual services, which creates a stable and efficient learning process. Computational results demonstrate the effectiveness of our approach in alleviating unfairness both spatially and temporally in different customer geographies. We also show this effectiveness is valid with different depot locations, providing businesses with an opportunity to achieve better fairness from any location. Further, we consider the impact of ignoring fairness in service, and results show that our policies eventually outperform the utility-driven baseline when customers have a high expectation on service level.

While deep learning methods continue to improve in predictive accuracy on a wide range of application domains, significant issues remain with other aspects of their performance including their ability to quantify uncertainty and their robustness. Recent advances in approximate Bayesian inference hold significant promise for addressing these concerns, but the computational scalability of these methods can be problematic when applied to large-scale models. In this paper, we describe initial work on the development ofURSABench(the Uncertainty, Robustness, Scalability, and Accu-racy Benchmark), an open-source suite of bench-marking tools for comprehensive assessment of approximate Bayesian inference methods with a focus on deep learning-based classification tasks, Comment: Presented at the ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning

As two neutron stars merge, they emit gravitational waves that can potentially be detected by earth bound detectors. Matched-filtering based algorithms have traditionally been used to extract quiet signals embedded in noise. We introduce a novel neural-network based machine learning algorithm that uses time series strain data from gravitational-wave detectors to detect signals from non-spinning binary neutron star mergers. For the Advanced LIGO design sensitivity, our network has an average sensitive distance of 130 Mpc at a false-alarm rate of 10 per month. Compared to other state-of-the-art machine learning algorithms, we find an improvement by a factor of 6 in sensitivity to signals with signal-to-noise ratio below 25. However, this approach is not yet competitive with traditional matched-filtering based methods. A conservative estimate indicates that our algorithm introduces on average 10.2 s of latency between signal arrival and generating an alert. We give an exact description of our testing procedure, which can not only be applied to machine learning based algorithms but all other search algorithms as well. We thereby improve the ability to compare machine learning and classical searches., Comment: 14 pages, 6 figures, 1 table, supplemental materials at https://github.com/gwastro/bns-machine-learning-search

In this paper, we present a general framework for distilling expectations with respect to the Bayesian posterior distribution of a deep neural network classifier, extending prior work on the Bayesian Dark Knowledge framework. The proposed framework takes as input "teacher" and student model architectures and a general posterior expectation of interest. The distillation method performs an online compression of the selected posterior expectation using iteratively generated Monte Carlo samples. We focus on the posterior predictive distribution and expected entropy as distillation targets. We investigate several aspects of this framework including the impact of uncertainty and the choice of student model architecture. We study methods for student model architecture search from a speed-storage-accuracy perspective and evaluate down-stream tasks leveraging entropy distillation including uncertainty ranking and out-of-distribution detection., Comment: Accepted at UAI '20

Intensive Care Unit Electronic Health Records (ICU EHRs) store multimodal data about patients including clinical notes, sparse and irregularly sampled physiological time series, lab results, and more. To date, most methods designed to learn predictive models from ICU EHR data have focused on a single modality. In this paper, we leverage the recently proposed interpolation-prediction deep learning architecture(Shukla and Marlin 2019) as a basis for exploring how physiological time series data and clinical notes can be integrated into a unified mortality prediction model. We study both early and late fusion approaches and demonstrate how the relative predictive value of clinical text and physiological data change over time. Our results show that a late fusion approach can provide a statistically significant improvement in mortality prediction performance over using individual modalities in isolation., Comment: Presented at ACM Conference on Health, Inference and Learning (Workshop Track), 2020

Path planning is an active area of research essential for many applications in robotics. Popular techniques include graph-based searches and sampling-based planners. These approaches are powerful but have limitations. This paper continues work to combine their strengths and mitigate their limitations using a unified planning paradigm. It does this by viewing the path planning problem as the two subproblems of search and approximation and using advanced graph-search techniques on a sampling-based approximation. This perspective leads to Advanced BIT*. ABIT* combines truncated anytime graph-based searches, such as ATD*, with anytime almost-surely asymptotically optimal sampling-based planners, such as RRT*. This allows it to quickly find initial solutions and then converge towards the optimum in an anytime manner. ABIT* outperforms existing single-query, sampling-based planners on the tested problems in $\mathbb{R}^{4}$ and $\mathbb{R}^{8}$, and was demonstrated on real-world problems with NASA/JPL-Caltech., Comment: IEEE International Conference on Robotics and Automation (ICRA) 2020, 6 + 1 pages, 3 figures, video available at https://youtu.be/VFdihv8Lq2A

Informed sampling-based planning algorithms exploit problem knowledge for better search performance. This knowledge is often expressed as heuristic estimates of solution cost and used to order the search. The practical improvement of this informed search depends on the accuracy of the heuristic. Selecting an appropriate heuristic is difficult. Heuristics applicable to an entire problem domain are often simple to define and inexpensive to evaluate but may not be beneficial for a specific problem instance. Heuristics specific to a problem instance are often difficult to define or expensive to evaluate but can make the search itself trivial. This paper presents Adaptively Informed Trees (AIT*), an almost-surely asymptotically optimal sampling-based planner based on BIT*. AIT* adapts its search to each problem instance by using an asymmetric bidirectional search to simultaneously estimate and exploit a problem-specific heuristic. This allows it to quickly find initial solutions and converge towards the optimum. AIT* solves the tested problems as fast as RRT-Connect while also converging towards the optimum., Comment: IEEE International Conference on Robotics and Automation (ICRA) 2020, 6 + 2 pages, 5 figures, video available at https://youtu.be/twM723QM9TQ

In this paper, we consider the problem of assessing the adversarial robustness of deep neural network models under both Markov chain Monte Carlo (MCMC) and Bayesian Dark Knowledge (BDK) inference approximations. We characterize the robustness of each method to two types of adversarial attacks: the fast gradient sign method (FGSM) and projected gradient descent (PGD). We show that full MCMC-based inference has excellent robustness, significantly outperforming standard point estimation-based learning. On the other hand, BDK provides marginal improvements. As an additional contribution, we present a storage-efficient approach to computing adversarial examples for large Monte Carlo ensembles using both the FGSM and PGD attacks., Comment: Presented at SafeAI Workshop, AAAI 2020

In this paper, we consider same-day delivery with vehicles and drones. Customers make delivery requests over the course of the day, and the dispatcher dynamically dispatches vehicles and drones to deliver the goods to customers before their delivery deadline. Vehicles can deliver multiple packages in one route but travel relatively slowly due to the urban traffic. Drones travel faster, but they have limited capacity and require charging or battery swaps. To exploit the different strengths of the fleets, we propose a deep Q-learning approach. Our method learns the value of assigning a new customer to either drones or vehicles as well as the option to not offer service at all. In a systematic computational analysis, we show the superiority of our policy compared to benchmark policies and the effectiveness of our deep Q-learning approach. We also show that our policy can maintain effectiveness when the fleet size changes moderately. Experiments on data drawn from varied spatial/temporal distributions demonstrate that our trained policies can cope with changes in the input data.

This paper proposes an upgraded electro-magnetic side-channel attack that automatically reconstructs the intercepted data. A novel system is introduced, running in parallel with leakage signal interception and catching compromising data in real-time. Based on deep learning and character recognition the proposed system retrieves more than 57% of characters present in intercepted signals regardless of signal type: analog or digital. The approach is also extended to a protection system that triggers an alarm if the system is compromised, demonstrating a success rate over 95%. Based on software-defined radio and graphics processing unit architectures, this solution can be easily deployed onto existing information systems where information shall be kept secret.

In this paper, we present a new deep learning architecture for addressing the problem of supervised learning with sparse and irregularly sampled multivariate time series. The architecture is based on the use of a semi-parametric interpolation network followed by the application of a prediction network. The interpolation network allows for information to be shared across multiple dimensions of a multivariate time series during the interpolation stage, while any standard deep learning model can be used for the prediction network. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. We investigate the performance of this architecture on both classification and regression tasks, showing that our approach outperforms a range of baseline and recently proposed models., Comment: International Conference on Learning Representations. arXiv admin note: substantial text overlap with arXiv:1812.00531