Search

In industry, online randomized controlled experiment (a.k.a A/B experiment) is a standard approach to measure the impact of a causal change. These experiments have small treatment effect to reduce the potential blast radius. As a result, these experiments often lack statistical significance due to low signal-to-noise ratio. To improve the precision (or reduce standard error), we introduce the idea of trigger observations where the output of the treatment and the control model are different. We show that the evaluation with full information about trigger observations (full knowledge) improves the precision in comparison to a baseline method. However, detecting all such trigger observations is a costly affair, hence we propose a sampling based evaluation method (partial knowledge) to reduce the cost. The randomness of sampling introduces bias in the estimated outcome. We theoretically analyze this bias and show that the bias is inversely proportional to the number of observations used for sampling. We also compare the proposed evaluation methods using simulation and empirical data. In simulation, evaluation with full knowledge reduces the standard error as much as 85%. In empirical setup, evaluation with partial knowledge reduces the standard error by 36.48%., Comment: 11 pages, 3 page appendix, 6 figures

Recently, pre-trained language models (PLMs) have been increasingly adopted in spoken language understanding (SLU). However, automatic speech recognition (ASR) systems frequently produce inaccurate transcriptions, leading to noisy inputs for SLU models, which can significantly degrade their performance. To address this, our objective is to train SLU models to withstand ASR errors by exposing them to noises commonly observed in ASR systems, referred to as ASR-plausible noises. Speech noise injection (SNI) methods have pursued this objective by introducing ASR-plausible noises, but we argue that these methods are inherently biased towards specific ASR systems, or ASR-specific noises. In this work, we propose a novel and less biased augmentation method of introducing the noises that are plausible to any ASR system, by cutting off the non-causal effect of noises. Experimental results and analyses demonstrate the effectiveness of our proposed methods in enhancing the robustness and generalizability of SLU models against unseen ASR systems by introducing more diverse and plausible ASR noises in advance., Comment: 9 pages, 3 figures

The Motsch-Tadmor (MT) model is a variant of the Cucker-Smale model with a normalized communication weight function. The normalization poses technical challenges in analyzing the collective behavior due to the absence of conservation of momentum. We study three quantitative estimates for the discrete-time MT model considering the first-order Euler discretization. First, we provide a sufficient framework leading to the asymptotic mono-cluster flocking. The proposed framework is given in terms of coupling strength, communication weight function, and initial data. Second, we show that the continuous transition from the discrete MT model to the continuous MT model can be made uniformly in time using the finite-time convergence result and asymptotic flocking estimate. Third, we present uniform-in-time stability estimates for the discrete MT model. We also provide several numerical examples and compare them with analytical results.

Bradykinesia, characterized by involuntary slowing or decrement of movement, is a fundamental symptom of Parkinson's Disease (PD) and is vital for its clinical diagnosis. Despite various methodologies explored to quantify bradykinesia, computer vision-based approaches have shown promising results. However, these methods often fall short in adequately addressing key bradykinesia characteristics in repetitive limb movements: "occasional arrest" and "decrement in amplitude." This research advances vision-based quantification of bradykinesia by introducing nuanced numerical analysis to capture decrement in amplitudes and employing a simple deep learning technique, LSTM-FCN, for precise classification of occasional arrests. Our approach structures the classification process hierarchically, tailoring it to the unique dynamics of bradykinesia in PD. Statistical analysis of the extracted features, including those representing arrest and fatigue, has demonstrated their statistical significance in most cases. This finding underscores the importance of considering "occasional arrest" and "decrement in amplitude" in bradykinesia quantification of limb movement. Our enhanced diagnostic tool has been rigorously tested on an extensive dataset comprising 1396 motion videos from 310 PD patients, achieving an accuracy of 80.3%. The results confirm the robustness and reliability of our method.

The colorful Helly theorem and Tverberg's theorem are fundamental results in discrete geometry. We prove a theorem which interpolates between the two. In particular, we show the following for any integers $d \geq m \geq 1$ and $k$ a prime power. Suppose $F_1, F_2, \dots, F_m$ are families of convex sets in $\mathbb{R}^d$, each of size $n > (\frac{d}{m}+1)(k-1)$, such that for any choice $C_i\in F_i$ we have $\bigcap_{i=1}^mC_i\neq \emptyset$. Then, one of the families $F_i$ admits a Tverberg $k$-partition. That is, one of the $F_i$ can be partitioned into $k$ nonempty parts such that the convex hulls of the parts have nonempty intersection. As a corollary, we also obtain a result concerning $r$-dimensional transversals to families of convex sets in $\mathbb{R}^d$ that satisfy the colorful Helly hypothesis, which extends the work of Karasev and Montejano., Comment: accepted to SOCG2024

Realtime face identification (FID) from a video feed is highly computation-intensive, and may exhaust computation resources if performed on a device with a limited amount of resources (e.g., a mobile device). In general, FID performs better when images are sampled at a higher rate, minimizing false negatives. However, performing it at an overwhelmingly high rate exposes the system to the risk of a queue overflow that hampers the system's reliability. This paper proposes a novel, queue-aware FID framework that adapts the sampling rate to maximize the FID performance while avoiding a queue overflow by implementing the Lyapunov optimization. A preliminary evaluation via a trace-based simulation confirms the effectiveness of the framework., Comment: This paper was presented at ACM Symposium on Operating Systems Principles (SOSP) Workshop on AI Systems (AISys), Shanghai, China, October 2017

This paper studies the bias problem of multi-hop question answering models, of answering correctly without correct reasoning. One way to robustify these models is by supervising to not only answer right, but also with right reasoning chains. An existing direction is to annotate reasoning chains to train models, requiring expensive additional annotations. In contrast, we propose a new approach to learn evidentiality, deciding whether the answer prediction is supported by correct evidences, without such annotations. Instead, we compare counterfactual changes in answer confidence with and without evidence sentences, to generate "pseudo-evidentiality" annotations. We validate our proposed model on an original set and challenge set in HotpotQA, showing that our method is accurate and robust in multi-hop reasoning., Comment: Accepted to ACL2021 (main conference)