Search

Recent progress on large language models (LLMs) has enabled dialogue agents to generate highly naturalistic and plausible text. However, current LLM language generation focuses on responding accurately to questions and requests with a single effective response. In reality, many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. Accounting for how an agent can effectively steer a conversation is a crucial ability in many dialogue tasks, from healthcare to preference elicitation. Existing methods for fine-tuning dialogue agents to accomplish such tasks would rely on curating some amount of expert data. However, doing so often requires understanding the underlying cognitive processes of the conversational partner, which is a skill neither humans nor LLMs trained on human data can reliably do. Our key insight is that while LLMs may not be adept at identifying effective strategies for steering conversations a priori, or in the middle of an ongoing conversation, they can do so post-hoc, or in hindsight, after seeing how their conversational partner responds. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. We apply our approach to two domains that require understanding human mental state, intelligent interaction, and persuasion: mental health support, and soliciting charitable donations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents., Comment: 23 pages, 5 figures

Work on shallow discourse parsing in English has focused on the Wall Street Journal corpus, the only large-scale dataset for the language in the PDTB framework. However, the data is not openly available, is restricted to the news domain, and is by now 35 years old. In this paper, we present and evaluate a new open-access, multi-genre benchmark for PDTB-style shallow discourse parsing, based on the existing UD English GUM corpus, for which discourse relation annotations in other frameworks already exist. In a series of experiments on cross-domain relation classification, we show that while our dataset is compatible with PDTB, substantial out-of-domain degradation is observed, which can be alleviated by joint training on both datasets., Comment: Accepted to EMNLP 2024 (main, long); camera-ready version

Neural networks are widely used in machine learning and data mining. Typically, these networks need to be trained, implying the adjustment of weights (parameters) within the network based on the input data. In this work, we propose a novel approach, RandomNet, that employs untrained deep neural networks to cluster time series. RandomNet uses different sets of random weights to extract diverse representations of time series and then ensembles the clustering relationships derived from these different representations to build the final clustering results. By extracting diverse representations, our model can effectively handle time series with different characteristics. Since all parameters are randomly generated, no training is required during the process. We provide a theoretical analysis of the effectiveness of the method. To validate its performance, we conduct extensive experiments on all of the 128 datasets in the well-known UCR time series archive and perform statistical analysis of the results. These datasets have different sizes, sequence lengths, and they are from diverse fields. The experimental results show that the proposed method is competitive compared with existing state-of-the-art methods., Comment: 25 pages, 10 figures

We present a general framework which can be used to prove that, in an annealed sense, rescaled spatial stochastic population models converge to generalized propagating fronts. Our work is motivated by recent results of Etheridge, Freeman, and Penington [EFP2017] and Huang and Durrett [HD2021], who proved convergence to classical mean curvature flow (MCF) for certain spatial stochastic processes, up until the first time when singularities of MCF form. Our arguments rely on the level-set method and the abstract approach to front propagation introduced by Barles and Souganidis [BS1998]. This approach is amenable to stochastic models equipped with moment duals which satisfy certain general and verifiable properties. Our main results improve the existing results in several ways, first by removing regularity conditions on the initial data, and second by establishing convergence beyond the formation of singularities of MCF. In particular, we obtain a general convergence theorem which holds globally in time. This is then applied to all of the models considered in [EFP2017] and [HD2021]., Comment: 65 pages

As NLP models become increasingly capable of understanding documents in terms of coherent entities rather than strings, obtaining the most salient entities for each document is not only an important end task in itself but also vital for Information Retrieval (IR) and other downstream applications such as controllable summarization. In this paper, we present and evaluate GUMsley, the first entity salience dataset covering all named and non-named salient entities for 12 genres of English text, aligned with entity types, Wikification links and full coreference resolution annotations. We promote a strict definition of salience using human summaries and demonstrate high inter-annotator agreement for salience based on whether a source entity is mentioned in the summary. Our evaluation shows poor performance by pre-trained SOTA summarization models and zero-shot LLM prompting in capturing salient entities in generated summaries. We also show that predicting or providing salient entities to several model architectures enhances performance and helps derive higher-quality summaries by alleviating the entity hallucination problem in existing abstractive summarization., Comment: Camera-ready for EACL 2024

We present GENTLE, a new mixed-genre English challenge corpus totaling 17K tokens and consisting of 8 unusual text types for out-of domain evaluation: dictionary entries, esports commentaries, legal documents, medical notes, poetry, mathematical proofs, syllabuses, and threat letters. GENTLE is manually annotated for a variety of popular NLP tasks, including syntactic dependency parsing, entity recognition, coreference resolution, and discourse parsing. We evaluate state-of-the-art NLP systems on GENTLE and find severe degradation for at least some genres in their performance on all tasks, which indicates GENTLE's utility as an evaluation dataset for NLP systems., Comment: Camera-ready for LAW-XVII collocated with ACL 2023

We describe a system for deep reinforcement learning of robotic manipulation skills applied to a large-scale real-world task: sorting recyclables and trash in office buildings. Real-world deployment of deep RL policies requires not only effective training algorithms, but the ability to bootstrap real-world training and enable broad generalization. To this end, our system combines scalable deep RL from real-world data with bootstrapping from training in simulation, and incorporates auxiliary inputs from existing computer vision systems as a way to boost generalization to novel objects, while retaining the benefits of end-to-end training. We analyze the tradeoffs of different design decisions in our system, and present a large-scale empirical validation that includes training on real-world data gathered over the course of 24 months of experimentation, across a fleet of 23 robots in three office buildings, with a total training set of 9527 hours of robotic experience. Our final validation also consists of 4800 evaluation trials across 240 waste station configurations, in order to evaluate in detail the impact of the design decisions in our system, the scaling effects of including more real-world data, and the performance of the method on novel objects. The projects website and videos can be found at \href{http://rl-at-scale.github.io}{rl-at-scale.github.io}., Comment: Published at Robotics: Science and Systems 2023

Time series classification is an important data mining task that has received a lot of interest in the past two decades. Due to the label scarcity in practice, semi-supervised time series classification with only a few labeled samples has become popular. Recently, Similarity-aware Time Series Classification (SimTSC) is proposed to address this problem by using a graph neural network classification model on the graph generated from pairwise Dynamic Time Warping (DTW) distance of batch data. It shows excellent accuracy and outperforms state-of-the-art deep learning models in several few-label settings. However, since SimTSC relies on pairwise DTW distances, the quadratic complexity of DTW limits its usability to only reasonably sized datasets. To address this challenge, we propose a new efficient semi-supervised time series classification technique, LB-SimTSC, with a new graph construction module. Instead of using DTW, we propose to utilize a lower bound of DTW, LB_Keogh, to approximate the dissimilarity between instances in linear time, while retaining the relative proximity relationships one would have obtained via computing DTW. We construct the pairwise distance matrix using LB_Keogh and build a graph for the graph neural network. We apply this approach to the ten largest datasets from the well-known UCR time series classification archive. The results demonstrate that this approach can be up to 104x faster than SimTSC when constructing the graph on large datasets without significantly decreasing classification accuracy., Comment: Accpeted by DLG-AAAI'23

Recent rapid development of sensor technology has allowed massive fine-grained time series (TS) data to be collected and set the foundation for the development of data-driven services and applications. During the process, data sharing is often involved to allow the third-party modelers to perform specific time series data mining (TSDM) tasks based on the need of data owner. The high resolution of TS brings new challenges in protecting privacy. While meaningful information in high-resolution TS shifts from concrete point values to local shape-based segments, numerous research have found that long shape-based patterns could contain more sensitive information and may potentially be extracted and misused by a malicious third party. However, the privacy issue for TS patterns is surprisingly seldom explored in privacy-preserving literature. In this work, we consider a new privacy-preserving problem: preventing malicious inference on long shape-based patterns while preserving short segment information for the utility task performance. To mitigate the challenge, we investigate an alternative approach by sharing Matrix Profile (MP), which is a non-linear transformation of original data and a versatile data structure that supports many data mining tasks. We found that while MP can prevent concrete shape leakage, the canonical correlation in MP index can still reveal the location of sensitive long pattern. Based on this observation, we design two attacks named Location Attack and Entropy Attack to extract the pattern location from MP. To further protect MP from these two attacks, we propose a Privacy-Aware Matrix Profile (PMP) via perturbing the local correlation and breaking the canonical correlation in MP index vector. We evaluate our proposed PMP against baseline noise-adding methods through quantitative analysis and real-world case studies to show the effectiveness of the proposed method., Comment: This is a preprint. The paper has been accepted by SIAM SDM2023

While much attention has been paid to identifying explicit hate speech, implicit hateful expressions that are disguised in coded or indirect language are pervasive and remain a major challenge for existing hate speech detection systems. This paper presents the first attempt to apply Entity Linking (EL) techniques to both explicit and implicit hate speech detection, where we show that such real world knowledge about entity mentions in a text does help models better detect hate speech, and the benefit of adding it into the model is more pronounced when explicit entity triggers (e.g., rally, KKK) are present. We also discuss cases where real world knowledge does not add value to hate speech detection, which provides more insights into understanding and modeling the subtleties of hate speech.

We give a new, self-contained proof of the multidimensional central limit theorem using the technique of ``doubling variables," which is traditionally used to prove uniqueness of solutions of partial differential equations (PDEs). Our technique also yields quantitative bounds for random variables with finite $2+\gamma$ moment for some $\gamma \in (0,1]$; when $\gamma=1$, this proves a version of the Berry--Esseen theorem in $\mathbb{R}^d$., Comment: 15 pages

We prove quantitative estimates on the the parabolic Green function and the stationary invariant measure in the context of stochasic homogenization of elliptic equations in nondivergence form. We consequently obtain a quenched, local CLT for the corresponding diffusion process and a quantitative ergodicity estimate for the environmental process. Each of these results are characterized by deterministic (in terms of the environment) estimates which are valid above a random, ``minimal'' length scale, the stochastic moments of which we estimate sharply., Comment: 60 pages

Time series motif discovery has been a fundamental task to identify meaningful repeated patterns in time series. Recently, time series chains were introduced as an expansion of time series motifs to identify the continuous evolving patterns in time series data. Informally, a time series chain (TSC) is a temporally ordered set of time series subsequences, in which every subsequence is similar to the one that precedes it, but the last and the first can be arbitrarily dissimilar. TSCs are shown to be able to reveal latent continuous evolving trends in the time series, and identify precursors of unusual events in complex systems. Despite its promising interpretability, unfortunately, we have observed that existing TSC definitions lack the ability to accurately cover the evolving part of a time series: the discovered chains can be easily cut by noise and can include non-evolving patterns, making them impractical in real-world applications. Inspired by a recent work that tracks how the nearest neighbor of a time series subsequence changes over time, we introduce a new TSC definition which is much more robust to noise in the data, in the sense that they can better locate the evolving patterns while excluding the non-evolving ones. We further propose two new quality metrics to rank the discovered chains. With extensive empirical evaluations, we demonstrate that the proposed TSC definition is significantly more robust to noise than the state of the art, and the top ranked chains discovered can reveal meaningful regularities in a variety of real world datasets., Comment: Accepted to ICDM 2022. This is an extended version of the paper

We explore the relationship between recursive distributional equations and convergence results for finite difference schemes of parabolic partial differential equations (PDEs). We focus on a family of random processes called symmetric cooperative motions, which generalize the symmetric simple random walk and the symmetric hipster random walk introduced in [Addario-Berry, Cairns, Devroye, Kerriou and Mitchell, arXiv:1909.07367]. We obtain a distributional convergence result for symmetric cooperative motions and, along the way, obtain a novel proof of the Bernoulli central limit theorem. In addition, we prove a PDE result relating distributional solutions and viscosity solutions of the porous medium equation and the parabolic $p$-Laplace equation, respectively, in one dimension., Comment: 33 pages, 0 figures

Previous work on Entity Linking has focused on resources targeting non-nested proper named entity mentions, often in data from Wikipedia, i.e. Wikification. In this paper, we present and evaluate WikiGUM, a fully wikified dataset, covering all mentions of named entities, including their non-named and pronominal mentions, as well as mentions nested within other mentions. The dataset covers a broad range of 12 written and spoken genres, most of which have not been included in Entity Linking efforts to date, leading to poor performance by a pretrained SOTA system in our evaluation. The availability of a variety of other annotations for the same data also enables further research on entities in context.

We prove distributional convergence for a family of random processes on $\mathbb{Z}$, which we call asymmetric cooperative motions. The model generalizes the "totally asymmetric hipster random walk" introduced in [Addario-Berry, Cairns, Devroye, Kerriou and Mitchell, 2020]. We present a novel approach based on connecting a temporal recurrence relation satisfied by the cumulative distribution functions of the process to the theory of finite difference schemes for Hamilton-Jacobi equations [Crandall and Lyons, 1984]. We also point out some surprising lattice effects that can persist in the distributional limit, and propose several generalizations and directions for future research., Comment: 28 pages, to appear in Transactions of the AMS

The controllable handling of an arbitrary single particle of matter with sub-100 nanometer (nm) dimensions is an essential but unsolved scientific challenge. We demonstrate nanoparticle-seeded glancing angle deposition using 10-100 nm diameter nanoparticle seeds (Er2O3, Fe@C, and Fe). The products are nanoparticle-nanowire heterostructures composed of arbitrary nanoscale tips attached to micron-length nanowire handles. Optical micromanipulation of the micron-scale handles enables concurrent manipulation of the attached nanoscale particles of matter., Comment: 8 pages, 2 figures, 3 tables, updated reference DOIs and fixed typos and formatting issues

This paper establishes bounds on the homogenized surface tension for a heterogeneous Allen-Cahn energy functional in a periodic medium. The approach is based on relating the homogenized energy to a purely geometric variational problem involving the large scale behaviour of the signed distance function to a hyperplane in periodic media. Motivated by this, a homogenization result for the signed distance function to a hyperplane in both periodic and almost periodic media is proven., Comment: Fixed an error in an earlier version and other revisions. 40 pages

We establish an invariance principle for the barycenter of a Brunet-Derrida particle system in $d$ dimensions. The model consists of $N$ particles undergoing dyadic branching Brownian motion with rate $1$. At a branching event, the number of particles is kept equal to $N$ by removing the particle located furthest away from the barycenter. To prove the invariance principle, a key step is to establish Harris recurrence for the process viewed from its barycenter., Comment: 36 pages. This version incorporates useful feedback from an anonymous referee. To appear in Annals of Applied Probability

Finding anomalous subsequence in a long time series is a very important but difficult problem. Existing state-of-the-art methods have been focusing on searching for the subsequence that is the most dissimilar to the rest of the subsequences; however, they do not take into account the background patterns that contain the anomalous candidates. As a result, such approaches are likely to miss local anomalies. We introduce a new definition named \textit{semantic discord}, which incorporates the context information from larger subsequences containing the anomaly candidates. We propose an efficient algorithm with a derived lower bound that is up to 3 orders of magnitude faster than the brute force algorithm in real world data. We demonstrate that our method significantly outperforms the state-of-the-art methods in locating anomalies by extensive experiments. We further explain the interpretability of semantic discord., Comment: Accepted by SDM 2020

Time series anomaly detection is an important task, with applications in a broad variety of domains. Many approaches have been proposed in recent years, but often they require that the length of the anomalies be known in advance and provided as an input parameter. This limits the practicality of the algorithms, as such information is often unknown in advance, or anomalies with different lengths might co-exist in the data. To address this limitation, previously, a linear time anomaly detection algorithm based on grammar induction has been proposed. While the algorithm can find variable-length patterns, it still requires preselecting values for at least two parameters at the discretization step. How to choose these parameter values properly is still an open problem. In this paper, we introduce a grammar-induction-based anomaly detection method utilizing ensemble learning. Instead of using a particular choice of parameter values for anomaly detection, the method generates the final result based on a set of results obtained using different parameter values. We demonstrate that the proposed ensemble approach can outperform existing grammar-induction-based approaches with different criteria for selection of parameter values. We also show that the proposed approach can achieve performance similar to that of the state-of-the-art distance-based anomaly detection algorithm., Comment: 12 pages