Search

In this paper we explore the use of symbolic knowledge and machine teaching to reduce human data labeling efforts in building neural task bots. We propose SYNERGY, a hybrid learning framework where a task bot is developed in two steps: (i) Symbolic knowledge to neural networks: Large amounts of simulated dialog sessions are generated based on task-specific symbolic knowledge which is represented as a task schema consisting of dialog flows and task-oriented databases. Then a pre-trained neural dialog model, SOLOIST, is fine-tuned on the simulated dialogs to build a bot for the task. (ii) Neural learning: The fine-tuned neural dialog model is continually refined with a handful of real task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the task bot. We validate SYNERGY on four dialog tasks. Experimental results show that SYNERGY maps task-specific knowledge into neural dialog models achieving greater diversity and coverage of dialog flows, and continually improves model performance with machine teaching, thus demonstrating strong synergistic effects of symbolic knowledge and machine teaching., Comment: 8 pages

In this paper, we study the instability induced by the Tollmien-Schlichting wave governed by the MHD system in the Prandtl-Hartmann regime. The interaction of the inviscid mode and viscous mode that leads to the instability is analyzed by the introduction of a new decomposition of the Orr-Sommerfeld operator on the velocity and magnetic fields. The critical Gevrey index for the instability is justified by constructing the growing mode in the essential frequency and it is shown to be the same as the incompressible Navier-Stokes equations in the Prandtl regime. This result justifies rigorously the physical understanding that the transverse magnetic field to the boundary in the Prandtl-Hartmann regime has no extra stabilizing effect on the Tollmien-Schlichting wave., Comment: 38 pages

We study $\bar{Q}Q\bar{q}q$ and $\bar{Q}qQ\bar{q}$ states as mixed states in QCD sum rules. By calculating the two-point correlation functions of pure states of their corresponding currents, we review the mass and coupling constant predictions of $J^{PC}=1^{++}$, $1^{--}$, $1^{-+}$ states. By calculating the two-point mixed correlation functions of $\bar{Q}Q\bar{q}q$ and $\bar{Q}qQ\bar{q}$ currents, and we estimate the mass and coupling constants of the corresponding `"physical state" that couples to both $\bar{Q}Q\bar{q}q$ and $\bar{Q}qQ\bar{q}$ currents. Our results suggest that $1^{++}$ states are more likely mixing from $\bar{Q}Q\bar{q}q$ and $\bar{Q}qQ\bar{q}$ components, while for $1^{--}$ and $1^{-+}$ states, there is less mixing between $\bar{Q}Q\bar{q}q$ and $\bar{Q}qQ\bar{q}$. Our results suggest the $Y$ series of states have more complicated components., Comment: 14 pages,3 figs, 7 tables

Existing reasoning tasks often have an important assumption that the input contents can be always accessed while reasoning, requiring unlimited storage resources and suffering from severe time delay on long sequences. To achieve efficient reasoning on long sequences with limited storage resources, memory augmented neural networks introduce a human-like write-read memory to compress and memorize the long input sequence in one pass, trying to answer subsequent queries only based on the memory. But they have two serious drawbacks: 1) they continually update the memory from current information and inevitably forget the early contents; 2) they do not distinguish what information is important and treat all contents equally. In this paper, we propose the Rehearsal Memory (RM) to enhance long-sequence memorization by self-supervised rehearsal with a history sampler. To alleviate the gradual forgetting of early information, we design self-supervised rehearsal training with recollection and familiarity tasks. Further, we design a history sampler to select informative fragments for rehearsal training, making the memory focus on the crucial information. We evaluate the performance of our rehearsal memory by the synthetic bAbI task and several downstream tasks, including text/video question answering and recommendation on long sequences., Comment: Accepted by ICML 2021

Vehicle search is one basic task for the efficient traffic management in terms of the AI City. Most existing practices focus on the image-based vehicle matching, including vehicle re-identification and vehicle tracking. In this paper, we apply one new modality, i.e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario. The natural language-based vehicle search poses one new challenge of fine-grained understanding of both vision and language modalities. To connect language and vision, we propose to jointly train the state-of-the-art vision models with the transformer-based language model in an end-to-end manner. Except for the network structure design and the training strategy, several optimization objectives are also re-visited in this work. The qualitative and quantitative experiments verify the effectiveness of the proposed method. Our proposed method has achieved the 1st place on the 5th AI City Challenge, yielding competitive performance 18.69% MRR accuracy on the private test set. We hope this work can pave the way for the future study on using language description effectively and efficiently for real-world vehicle retrieval systems. The code will be available at https://github.com/ShuaiBai623/AIC2021-T5-CLV., Comment: CVPR 2021 AI CITY CHALLENGE Natural Language-Based Vehicle Retrieval Top 1

Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations. In this paper, instead of investigating these control signals separately, we propose a new two-stage architecture, M6-UFC, to unify any number of multi-modal controls. In M6-UFC, both the diverse control signals and the synthesized image are uniformly represented as a sequence of discrete tokens to be processed by Transformer. Different from existing two-stage autoregressive approaches such as DALL-E and VQGAN, M6-UFC adopts non-autoregressive generation (NAR) at the second stage to enhance the holistic consistency of the synthesized image, to support preserving specified image blocks, and to improve the synthesis speed. Further, we design a progressive algorithm that iteratively improves the non-autoregressively generated image, with the help of two estimators developed for evaluating the compliance with the controls and evaluating the fidelity of the synthesized image, respectively. Extensive experiments on a newly collected large-scale clothing dataset M2C-Fashion and a facial dataset Multi-Modal CelebA-HQ verify that M6-UFC can synthesize high-fidelity images that comply with flexible multi-modal controls., Comment: Accepted by NeurIPS21

Modulating the electric potential on a conducting electrode is presented to generate an optical contrast for scattering microscopy that is sensitive to both surface charge and local topography. We dub this method Electric-Double-Layer-Modulation microscopy. We numerically compute the change in the local ion concentration that is the origin of this optical contrast for three experimentally relevant geometries: nanosphere, nanowire, and nanohole. In absence of plasmonic effects and physical absorption, the observable optical contrast is proportional to the derivative of the ion concentration with respect to the modulated potential. We demonstrate that this derivative depends on the size of the object and, less intuitively, also on its surface charge. This dependence is key to measuring the surface charge, in an absolute way, using this method. Our results help to identify the experimental conditions such as dynamic range and sensitivity that will be necessary for detecting the elementary charge jumps. We conclude that the nanohole is the most suitable geometry among these three for achieving elementary charge sensitivity., Comment: 17 pages, 10 figures

This paper is concerned with the vanishing viscosity and magnetic resistivity limit for the two-dimensional steady incompressible MHD system on the half plane with no-slip boundary condition on velocity field and perfectly conducting wall condition on magnetic field. We prove the nonlinear stability of shear flows of Prandtl type with nondegenerate tangential magnetic field, but without any positivity or monotonicity assumption on the velocity field. It is in sharp contrast to the steady Navier-Stokes equations and reflects the stabilization effect of magnetic field. Unlike the unsteady MHD system, we manage the degeneracy on the boundary caused by no-slip boundary condition and obtain the estimates of solutions by introducing an intrinsic weight function and some good auxiliary functions., Comment: 28 pages

Unrestricted particle transport through microfluidic channels is of paramount importance to a wide range of applications, including lab-on-a-chip devices. In this article, we study using video microscopy the electro-osmotic aggregation of colloidal particles at the opening of a micrometer-sized silica channel in presence of a salt gradient. Particle aggregation eventually leads to clogging of the channel, which may be undone by a time-adjusted reversal of the applied electric potential. We numerically model our system via the Stokes-Poisson-Nernst-Planck equations in a geometry that approximates the real sample. This allows us to identify the transport processes induced by the electric field and salt gradient and to provide evidence that a balance thereof leads to aggregation. We further demonstrate experimentally that a net flow of colloids through the channel may be achieved by applying a square-waveform electric potential with an appropriately tuned duty cycle. Our results serve to guide the design of microfluidic and nanofluidic pumps that allow for controlled particle transport and provide new insights for anti-fouling in ultra-filtration., Comment: 9 pages, 7 figures

We interpret the $X_1(2900)$ as an $S$-wave $\bar{D}_1K$ molecular state in the Bethe-Salpeter equation approach with the ladder and instantaneous approximations for the kernel. By solving the Bethe-Salpeter equation numerically with the kernel containing one-particle-exchange diagrams and introducing three different form factors (monopole, dipole, and exponential form factors) in the verties, we find the bound state exists. We also study the decay width of the decay $X_1(2900)$ to $D^-K^+$., Comment: 13 pages, 4 figures. arXiv admin note: text overlap with arXiv:2004.14085, arXiv:1906.09002

For task-oriented dialog systems to be maximally useful, it must be able to process conversations in a way that is (1) generalizable with a small number of training examples for new task domains, and (2) robust to user input in various styles, modalities or domains. In pursuit of these goals, we introduce the RADDLE benchmark, a collection of corpora and tools for evaluating the performance of models across a diverse set of domains. By including tasks with limited training data, RADDLE is designed to favor and encourage models with a strong generalization ability. RADDLE also includes a diagnostic checklist that facilitates detailed robustness analysis in aspects such as language variations, speech errors, unseen entities, and out-of-domain utterances. We evaluate recent state-of-the-art systems based on pre-training and fine-tuning, and find that grounded pre-training on heterogeneous dialog corpora performs better than training a separate model per domain. Overall, existing models are less than satisfactory in robustness evaluation, which suggests opportunities for future improvement., Comment: 12 pages; Project website at aka.ms/raddle

Video moment retrieval aims to localize the target moment in an video according to the given sentence. The weak-supervised setting only provides the video-level sentence annotations during training. Most existing weak-supervised methods apply a MIL-based framework to develop inter-sample confrontment, but ignore the intra-sample confrontment between moments with semantically similar contents. Thus, these methods fail to distinguish the target moment from plausible negative moments. In this paper, we propose a novel Regularized Two-Branch Proposal Network to simultaneously consider the inter-sample and intra-sample confrontments. Concretely, we first devise a language-aware filter to generate an enhanced video stream and a suppressed video stream. We then design the sharable two-branch proposal module to generate positive proposals from the enhanced stream and plausible negative proposals from the suppressed one for sufficient confrontment. Further, we apply the proposal regularization to stabilize the training process and improve model performance. The extensive experiments show the effectiveness of our method. Our code is released at here., Comment: ACM MM 2020

Within the quasi-two-body decay model, we study the localized $CP$ violation and branching fraction of the four-body decay $\bar{B}^0\rightarrow [K^-\pi^+]_{S/V}[\pi^+\pi^-]_{V/S} \rightarrow K^-\pi^+\pi^-\pi^+$ when $K^-\pi^+$ and $\pi^-\pi^+$ pair invariant masses are $0.35

Spatio-temporal video grounding aims to retrieve the spatio-temporal tube of a queried object according to the given sentence. Currently, most existing grounding methods are restricted to well-aligned segment-sentence pairs. In this paper, we explore spatio-temporal video grounding on unaligned data and multi-form sentences. This challenging task requires to capture critical object relations to identify the queried target. However, existing approaches cannot distinguish notable objects and remain in ineffective relation modeling between unnecessary objects. Thus, we propose a novel object-aware multi-branch relation network for object-aware relation discovery. Concretely, we first devise multiple branches to develop object-aware region modeling, where each branch focuses on a crucial object mentioned in the sentence. We then propose multi-branch relation reasoning to capture critical object relationships between the main branch and auxiliary branches. Moreover, we apply a diversity loss to make each branch only pay attention to its corresponding object and boost multi-branch learning. The extensive experiments show the effectiveness of our proposed method., Comment: Accepted by IJCAI 2020

In this paper, we consider a novel task, Spatio-Temporal Video Grounding for Multi-Form Sentences (STVG). Given an untrimmed video and a declarative/interrogative sentence depicting an object, STVG aims to localize the spatio-temporal tube of the queried object. STVG has two challenging settings: (1) We need to localize spatio-temporal object tubes from untrimmed videos, where the object may only exist in a very small segment of the video; (2) We deal with multi-form sentences, including the declarative sentences with explicit objects and interrogative sentences with unknown objects. Existing methods cannot tackle the STVG task due to the ineffective tube pre-generation and the lack of object relationship modeling. Thus, we then propose a novel Spatio-Temporal Graph Reasoning Network (STGRN) for this task. First, we build a spatio-temporal region graph to capture the region relationships with temporal object dynamics, which involves the implicit and explicit spatial subgraphs in each frame and the temporal dynamic subgraph across frames. We then incorporate textual clues into the graph and develop the multi-step cross-modal graph reasoning. Next, we introduce a spatio-temporal localizer with a dynamic selection method to directly retrieve the spatio-temporal tubes without tube pre-generation. Moreover, we contribute a large-scale video grounding dataset VidSTG based on video relation dataset VidOR. The extensive experiments demonstrate the effectiveness of our method., Comment: The camera ready version for CVPR 2020