Search

In this paper, we study the sensor selection problem for remote state estimation under the Quality-of-Service (QoS) requirement constraints. Multiple sensors are employed to observe a linear time-invariant system, and their measurements should be transmitted to a remote estimator for state estimation. However, due to the limited communication resources and the QoS requirement constraints, only some of the sensors can be allowed to transmit their measurements. To estimate the system state as accurately as possible, it is essential to select sensors for transmission appropriately. We formulate the sensor selection problem as a non-convex optimization problem. It is difficult to solve such a problem and even to find a feasible solution. To obtain a solution which can achieve good estimation performance, we first reformulate and relax the formulated problem. Then, we propose an algorithm based on successive convex approximation (SCA) to solve the relaxed problem. By utilizing the solution of the relaxed problem, we propose a heuristic sensor selection algorithm which can provide a good suboptimal solution. Simulation results are presented to show the effectiveness of the proposed heuristic.

We propose a logic-informed knowledge-driven modeling framework for human movements by analyzing their trajectories. Our approach is inspired by the fact that human actions are usually driven by their intentions or desires, and are influenced by environmental factors such as the spatial relationships with surrounding objects. In this paper, we introduce a set of spatial-temporal logic rules as knowledge to explain human actions. These rules will be automatically discovered from observational data. To learn the model parameters and the rule content, we design an expectation-maximization (EM) algorithm, which treats the rule content as latent variables. The EM algorithm alternates between the E-step and M-step: in the E-step, the posterior distribution over the latent rule content is evaluated; in the M-step, the rule generator and model parameters are jointly optimized by maximizing the current expected log-likelihood. Our model may have a wide range of applications in areas such as sports analytics, robotics, and autonomous cars, where understanding human movements are essential. We demonstrate the model's superior interpretability and prediction performance on pedestrian and NBA basketball player datasets, both achieving promising results., Comment: There are missing descriptions of the results in section 5.6, and the coordinates have an offset

We train an equivariant machine learning model to predict energies and forces for a real-world study of hydrogen combustion under conditions of finite temperature and pressure. This challenging case for reactive chemistry illustrates that ML learned potential energy surfaces (PESs) are always incomplete as they are overly reliant on chemical intuition of what data is important for training, i.e. stable or metastable energy states. Instead we show here that a negative design data acquisition strategy is necessary to create a more complete ML model of the PES, since it must also learn avoidance of unforeseen high energy intermediates or even unphysical energy configurations. Because this type of data is unintuitive to create, we introduce an active learning workflow based on metadynamics that samples a lower dimensional manifold within collective variables that efficiently creates highly variable energy configurations for further ML training. This strategy more rapidly completes the ML PES such that deviations among query by committee ML models helps to now signal occasional calls to the external ab initio data source to further molecular dynamics in time without need for retraining the ML model. With the hybrid ML-physics model we predict the change in transition state and/or reaction mechanism at finite temperature and pressure for hydrogen combustion, thereby delivering on the promise of real application work using ML trained models of an ab initio PES with two orders of magnitude reduction in cost.

The Wide Field Survey Telescope (WFST) is a dedicated photometric surveying facility being built jointly by the University of Science and Technology of China and the Purple Mountain Observatory. It is equipped with a 2.5-meter diameter primary mirror, an active optics system, and a mosaic CCD camera with 0.73 gigapixels on the primary focal plane for high-quality image capture over an FOV of 6.5-square-degree. It is anticipated that WFST will be set up at the Lenghu site in the summer of 2023 and begin to observe the northern sky in four optical bands (u, g, r, and i) with a range of cadences, from hourly/daily in the Deep High-Cadence Survey (DHS) program to semiweekly in the Wide-Field Survey (WFS) program, three months later. During a photometric night, a nominal 30 s exposure in the WFS program will reach a depth of 22.27, 23.32, 22.84, and 22.31 (AB magnitudes) in these four bands, respectively, allowing for the detection of a tremendous amount of transients in the low-z universe and a systematic investigation of the variability of Galactic and extragalactic objects. In the DHS program, intranight 90 s exposures as deep as 23 (u) and 24 mag (g), in combination with target of opportunity follow-ups, will provide a unique opportunity to explore energetic transients in demand for high sensitivities, including the electromagnetic counterparts of gravitational wave events, supernovae within a few hours of their explosions, tidal disruption events and fast, luminous optical transients even beyond a redshift of unity. In addition, the final 6-year co-added images, anticipated to reach g=25.8 mag in WFS or 1.5 mags deeper in DHS, will be of fundamental importance to general Galactic and extragalactic science. The highly uniform legacy surveys of WFST will serve as an indispensable complement to those of LSST that monitor the southern sky., Comment: 48 pages

The generalized Kadanoff-Baym ansatz (GKBA) is an approximation to the Kadanoff-Baym equations (KBE), that neglects certain memory effects that contribute to the Green's function at non-equal times. Here we present arguments and numerical results to demonstrate the practical insignificance of the quantities neglected when deriving the GKBA at conditions at which KBE and GKBA are appropriate. We provide a mathematical proof that places a scaling bound on the neglected terms, further reinforcing that these terms are typically small in comparison to terms that are kept in the GKBA. We perform calculations in a range of models, including different system sizes and filling fractions, as well as experimentally relevant non-equilibrium excitations. We find that both the GKBA and KBE capture the dynamics of interacting systems with moderate and even strong interactions well. We explicitly compute terms neglected in the GKBA approximation and show, in the scenarios tested here, that they are orders of magnitude smaller than the terms that are accounted for, i.e., they offer only a small correction when included in the full Kadanoff-Baym equations., Comment: 14 pages, 3 figures, Supplemental information with 10 figures

We propose a multi-dimensional structured state space (S4) approach to speech enhancement. To better capture the spectral dependencies across the frequency axis, we focus on modifying the multi-dimensional S4 layer with whitening transformation to build new small-footprint models that also achieve good performance. We explore several S4-based deep architectures in time (T) and time-frequency (TF) domains. The 2-D S4 layer can be considered a particular convolutional layer with an infinite receptive field although it utilizes fewer parameters than a conventional convolutional layer. Evaluated on the VoiceBank-DEMAND data set, when compared with the conventional U-net model based on convolutional layers, the proposed TF-domain S4-based model is 78.6% smaller in size, yet it still achieves competitive results with a PESQ score of 3.15 with data augmentation. By increasing the model size, we can even reach a PESQ score of 3.18., Comment: Accepted to Interspeech 2023. Code will be released at https://github.com/Kuray107/S4ND-U-Net_speech_enhancement

In this work, we introduce a "score-based assessment" framework for estimating the transferability of pre-trained speech models (PSMs) for fine-tuning target tasks. We leverage upon two representation theories, Bayesian likelihood estimation and optimal transport, to generate rank scores for the PSM candidates using the extracted representations. Our framework efficiently computes transferability scores without actual fine-tuning of candidate models or layers by making a temporal independent hypothesis. We evaluate some popular supervised speech models (e.g., Conformer RNN-Transducer) and self-supervised speech models (e.g., HuBERT) in cross-layer and cross-model settings using public data. Experimental results show a high Spearman's rank correlation and low $p$-value between our estimation framework and fine-tuning ground truth. Our proposed transferability framework requires less computational time and resources, making it a resource-saving and time-efficient approach for tuning speech foundation models., Comment: Accepted to Interspeech. Code is available at: https://github.com/virginiakm1988/LogME-CTC. Fixed a typo

Solving partial differential equations (PDEs) is a central task in scientific computing. Recently, neural network approximation of PDEs has received increasing attention due to its flexible meshless discretization and its potential for high-dimensional problems. One fundamental numerical difficulty is that random samples in the training set introduce statistical errors into the discretization of loss functional which may become the dominant error in the final approximation, and therefore overshadow the modeling capability of the neural network. In this work, we propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set, provided by a deep generative model. The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile when it is being minimized. Such an idea is achieved by implicitly embedding the Wasserstein distance between the residual-induced distribution and the uniform distribution into the loss, which is then minimized together with the residual. A nearly uniform residual profile means that its variance is small for any normalized weight function such that the Monte Carlo approximation error of the loss functional is reduced significantly for a certain sample size. The adversarial adaptive sampling (AAS) approach proposed in this work is the first attempt to formulate two essential components, minimizing the residual and seeking the optimal training set, into one minmax objective functional for the neural network approximation of PDEs., Comment: ICLR, 2024

In this work, we introduce S4M, a new efficient speech separation framework based on neural state-space models (SSM). Motivated by linear time-invariant systems for sequence modeling, our SSM-based approach can efficiently model input signals into a format of linear ordinary differential equations (ODEs) for representation learning. To extend the SSM technique into speech separation tasks, we first decompose the input mixture into multi-scale representations with different resolutions. This mechanism enables S4M to learn globally coherent separation and reconstruction. The experimental results show that S4M performs comparably to other separation backbones in terms of SI-SDRi, while having a much lower model complexity with significantly fewer trainable parameters. In addition, our S4M-tiny model (1.8M parameters) even surpasses attention-based Sepformer (26.0M parameters) in noisy conditions with only 9.2 of multiply-accumulate operation (MACs)., Comment: Accepted by InterSpeech 2023

In this work, we explore Parameter-Efficient-Learning (PEL) techniques to repurpose a General-Purpose-Speech (GSM) model for Arabic dialect identification (ADI). Specifically, we investigate different setups to incorporate trainable features into a multi-layer encoder-decoder GSM formulation under frozen pre-trained settings. Our architecture includes residual adapter and model reprogramming (input-prompting). We design a token-level label mapping to condition the GSM for Arabic Dialect Identification (ADI). This is challenging due to the high variation in vocabulary and pronunciation among the numerous regional dialects. We achieve new state-of-the-art accuracy on the ADI-17 dataset by vanilla fine-tuning. We further reduce the training budgets with the PEL method, which performs within 1.86% accuracy to fine-tuning using only 2.5% of (extra) network trainable parameters. Our study demonstrates how to identify Arabic dialects using a small dataset and limited computation with open source code and pre-trained models., Comment: Accepted to Interspeech 2023, 5 pages. Code is available at: https://github.com/Srijith-rkr/KAUST-Whisper-Adapter under MIT license

In this work, we devise a parameter-efficient solution to bring differential privacy (DP) guarantees into adaptation of a cross-lingual speech classifier. We investigate a new frozen pre-trained adaptation framework for DP-preserving speech modeling without full model fine-tuning. First, we introduce a noisy teacher-student ensemble into a conventional adaptation scheme leveraging a frozen pre-trained acoustic model and attain superior performance than DP-based stochastic gradient descent (DPSGD). Next, we insert residual adapters (RA) between layers of the frozen pre-trained acoustic model. The RAs reduce training cost and time significantly with a negligible performance drop. Evaluated on the open-access Multilingual Spoken Words (MLSW) dataset, our solution reduces the number of trainable parameters by 97.5% using the RAs with only a 4% performance drop with respect to fine-tuning the cross-lingual speech classifier while preserving DP guarantees., Comment: Accepted to Interspeech 2023. Code will be available at: https://github.com/Chun-wei-Ho/Private-Speech-Adapter. The authors would like to express their gratitude to Prof. Chin-Hui Lee from Georgia Tech for providing helpful insights and suggestions

This paper presents a parameter-efficient learning (PEL) to develop a low-resource accent adaptation for text-to-speech (TTS). A resource-efficient adaptation from a frozen pre-trained TTS model is developed by using only 1.2\% to 0.8\% of original trainable parameters to achieve competitive performance in voice synthesis. Motivated by a theoretical foundation of optimal transport (OT), this study carries out PEL for TTS where an auxiliary unsupervised loss based on OT is introduced to maximize a difference between the pre-trained source domain and the (unseen) target domain, in addition to its supervised training loss. Further, we leverage upon this unsupervised loss refinement to boost system performance via either sliced Wasserstein distance or maximum mean discrepancy. The merit of this work is demonstrated by fulfilling PEL solutions based on residual adapter learning, and model reprogramming when evaluating the Mandarin accent adaptation. Experiment results show that the proposed methods can achieve competitive naturalness with parameter-efficient decoder fine-tuning, and the auxiliary unsupervised loss improves model performance empirically., Comment: Accepted to Interspeech 2023

Variational quantum circuits (VQCs) hold promise for quantum machine learning on noisy intermediate-scale quantum (NISQ) devices. While tensor-train networks (TTNs) can enhance VQC representation and generalization, the resulting hybrid model, TTN-VQC, faces optimization challenges due to the Polyak-Lojasiewicz (PL) condition. To mitigate this challenge, we introduce Pre+TTN-VQC, a pre-trained TTN model combined with a VQC. Our theoretical analysis, grounded in two-stage empirical risk minimization, provides an upper bound on the transfer learning risk. It demonstrates the approach's advantages in overcoming the optimization challenge while maintaining TTN-VQC's generalization capability. We validate our findings through experiments on quantum dot and handwritten digit classification using simulated and actual NISQ environments., Comment: In submission

With growing demand for time-domain simulations of correlated many-body systems, the development of efficient and stable integration schemes for the time-dependent Schr\"odinger equation is of keen interest in modern electronic structure theory. In the present work, we present two novel approaches for the formation of the quantum propagator for time-dependent equation-of-motion coupled cluster theory (TD-EOM-CC) based on the Chebyshev and Arnoldi expansions of the complex, non-hermitian matrix exponential, respectively. The proposed algorithms are compared with the short-iterative Lanczos method of Cooper, et al [J. Phys. Chem. A. 2021 125, 5438-5447], the fourth-order Runge-Kutta method (RK4), and exact dynamics for a set of small but challenging test problems. For each of the cases studied, both of the proposed integration schemes demonstrate superior accuracy and efficiency relative to the reference simulations., Comment: 28 pages, 4 figures

The goal of video summarization is to automatically shorten videos such that it conveys the overall story without losing relevant information. In many application scenarios, improper video summarization can have a large impact. For example in forensics, the quality of the generated video summary will affect an investigator's judgment while in journalism it might yield undesired bias. Because of this, modeling explainability is a key concern. One of the best ways to address the explainability challenge is to uncover the causal relations that steer the process and lead to the result. Current machine learning-based video summarization algorithms learn optimal parameters but do not uncover causal relationships. Hence, they suffer from a relative lack of explainability. In this work, a Causal Explainer, dubbed Causalainer, is proposed to address this issue. Multiple meaningful random variables and their joint distributions are introduced to characterize the behaviors of key components in the problem of video summarization. In addition, helper distributions are introduced to enhance the effectiveness of model training. In visual-textual input scenarios, the extra input can decrease the model performance. A causal semantics extractor is designed to tackle this issue by effectively distilling the mutual information from the visual and textual inputs. Experimental results on commonly used benchmarks demonstrate that the proposed method achieves state-of-the-art performance while being more explainable., Comment: The paper has been accepted by the CVPR Workshop on New Frontiers in Visual Language Reasoning: Compositionality, Prompts, and Causality, 2023

We use local diffusion maps to assess the quality of two types of collective variables (CVs) for a recently published hydrogen combustion benchmark dataset~\cite{guan2022benchmark} that contains ab initio molecular dynamics trajectories and normal modes along minimum energy paths. This approach was recently advocated in~\cite{tlldiffmap20} for assessing CVs and analyzing reactions modeled by classical molecular dynamics simulations. We report the effectiveness of this approach to molecular systems modeled by quantum ab initio molecular dynamics. In addition to assessing the quality of CVs, we also use global diffusion maps to perform committor analysis as proposed in~\cite{tlldiffmap20}. We show that the committor function obtained from the global diffusion map allows us to identify transition regions of interest in several hydrogen combustion reaction channels.

[Abridged] Aims. We provide an important step toward a better understanding of the magnetorotational instability (MRI)-dust coevolution in protoplanetary disks by presenting a proof of concept that dust evolution ultimately plays a crucial role in the MRI activity. Methods. First, we study how a fixed power-law dust size distribution with varying parameters impacts the MRI activity, especially the steady-state MRI-driven accretion, by employing and improving our previous 1+1D MRI-driven turbulence model. Second, we relax the steady-state accretion assumption in this disk accretion model, and partially couple it to a dust evolution model in order to investigate how the evolution of dust (dynamics and grain growth processes combined) and MRI-driven accretion are intertwined on million-year timescales. Results. Dust coagulation and settling lead to a higher gas ionization degree in the protoplanetary disk, resulting in stronger MRI-driven turbulence as well as a more compact dead zone. On the other hand, fragmentation has an opposite effect because it replenishes the disk in small dust particles. Since the dust content of the disk decreases over million years of evolution due to radial drift, the MRI-driven turbulence overall becomes stronger and the dead zone more compact until the disk dust-gas mixture eventually behaves as a grain-free plasma. Furthermore, our results show that dust evolution alone does not lead to a complete reactivation of the dead zone. Conclusions. The MRI activity evolution (hence the temporal evolution of the MRI-induced $\alpha$-parameter) is controlled by dust evolution and occurs on a timescale of local dust growth, as long as there is enough dust particles in the disk to dominate the recombination process for the ionization chemistry. Once it is no longer the case, it is expected to be controlled by gas evolution and occurs on a viscous evolution timescale., Comment: 23 pages, 13 figures, Accepted for publication in A&A

Spiral arms are observed in numerous protoplanetary discs. These spiral arms can be excited by companions, either on bound or unbound orbits. We simulate a scenario where an unbound perturber, i.e. a flyby, excites spiral arms during a periastron passage. We run three-dimensional hydrodynamical simulations of a parabolic flyby encountering a gaseous protoplanetary disc. The perturber mass ranges from $10\, \rm M_J$ to $1\, \rm M_{\odot}$. The perturber excites a two-armed spiral structure, with a more prominent spiral feature for higher mass perturbers. The two arms evolve over time, eventually winding up, consistent with previous works. We focus on analysing the pattern speed and pitch angle of these spirals during the whole process. The initial pattern speed of the two arms are close to the angular velocity of the perturber at periastron, and then it decreases over time. The pitch angle also decreases over time as the spiral winds up. The spirals disappear after several local orbital times. An inclined prograde orbit flyby induces similar disc substructures as a coplanar flyby. A solar-mass flyby event causes increased eccentricity growth in the protoplanetary disc, leading to an eccentric disc structure which dampens over time. The spirals' morphology and the disc eccentricity can be used to search for potential unbound stars or planets around discs where a flyby is suspected. Future disc observations at high resolution and dedicated surveys will help to constrain the frequency of such stellar encounters in nearby star-forming regions., Comment: 17 pages, 18 figures, accepted to MNRAS

In this paper, we study four dimensional hypersurface M^4_r with proper mean curvature vector field (i.e. \Delta\vec{H} is proportional to \vec{H}) in pseudo-Riemannian space form N^5_s(c), and show that it has constant mean curvature, and give the range of this constant. As an application, we get that biharmonic hypersurfaces in N^5_s(c) are minimal in some specific cases, which partially confirms B.-Y. Chen's conjecture.

Sparsely gated Mixture-of-Expert (MoE) has demonstrated its effectiveness in scaling up deep neural networks to an extreme scale. Despite that numerous efforts have been made to improve the performance of MoE from the model design or system optimization perspective, existing MoE dispatch patterns are still not able to fully exploit the underlying heterogeneous network environments. In this paper, we propose TA-MoE, a topology-aware routing strategy for large-scale MoE trainging, from a model-system co-design perspective, which can dynamically adjust the MoE dispatch pattern according to the network topology. Based on communication modeling, we abstract the dispatch problem into an optimization objective and obtain the approximate dispatch pattern under different topologies. On top of that, we design a topology-aware auxiliary loss, which can adaptively route the data to fit in the underlying topology without sacrificing the model accuracy. Experiments show that TA-MoE can substantially outperform its counterparts on various hardware and model configurations, with roughly 1.01x-1.61x, 1.01x-4.77x, 1.25x-1.54x improvements over the popular DeepSpeed-MoE, FastMoE and FasterMoE.

Let $V$ be a simple vertex algebra of countable dimension, $G$ be a finite automorphism group of $V$ and $\sigma$ be a central element of $G$. Assume that ${\cal S}$ is a finite set of inequivalent irreducible $\sigma$-twisted $V$-modules such that ${\cal S}$ is invariant under the action of $G$. Then there is a finite dimensional semisimple associative algebra ${\cal A}_{\alpha}(G,{\cal S})$ for a suitable $2$-cocycle $\alpha$ naturally determined by the $G$-action on ${\cal S}$ such that $({\cal A}_{\alpha}(G,{\cal S}),V^G)$ form a dual pair on the sum $\cal M$ of $\sigma$-twisted $V$-modules in ${\cal S}$ in the sense that (1) the actions of ${\cal A}_{\alpha}(G,{\cal S})$ and $V^G$ on $\cal M$ commute, (2) each irreducible ${\cal A}_{\alpha}(G,{\cal S})$-module appears in $\cal M,$ (3) the multiplicity space of each irreducible ${\cal A}_{\alpha}(G,{\cal S})$-module is an irreducible $V^G$-module, (4) the multiplicitiy spaces of different irreducible ${\cal A}_{\alpha}(G,{\cal S})$-modules are inequivalent $V^G$-modules. As applications, every irreducible $\sigma$-twisted $V$-module is a direct sum of finitely many irreducible $V^G$-modules and irreducible $V^G$-modules appearing in different $G$-orbits are inequivalent. This result generalizes many previous ones. We also establish a bijection between subgroups of $G$ and subalgebras of $V$ containing $V^G.$, Comment: 24 pages

Parametric optimal control problems governed by partial differential equations (PDEs) are widely found in scientific and engineering applications. Traditional grid-based numerical methods for such problems generally require repeated solutions of PDEs with different parameter settings, which is computationally prohibitive especially for problems with high-dimensional parameter spaces. Although recently proposed neural network methods make it possible to obtain the optimal solutions simultaneously for different parameters, challenges still remain when dealing with problems with complex constraints. In this paper, we propose AONN, an adjoint-oriented neural network method, to overcome the limitations of existing approaches in solving parametric optimal control problems. In AONN, the neural networks are served as parametric surrogate models for the control, adjoint and state functions to get the optimal solutions all at once. In order to reduce the training difficulty and handle complex constraints, we introduce an iterative training framework inspired by the classical direct-adjoint looping (DAL) method so that penalty terms arising from the Karush-Kuhn-Tucker (KKT) system can be avoided. Once the training is done, parameter-specific optimal solutions can be quickly computed through the forward propagation of the neural networks, which may be further used for analyzing the parametric properties of the optimal solutions. The validity and efficiency of AONN is demonstrated through a series of numerical experiments with problems involving various types of parameters., Comment: 19 pages, 14 figures

In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement that, for the first time, empowers model reprogramming on ASR. Specifically, we investigate how to select trainable components (i.e., encoder) of a conformer-based RNN-Transducer, as a frozen pre-trained backbone. Experiments on a seven-language multilingual LibriSpeech speech (MLS) task show that model reprogramming only requires 4.2% (11M out of 270M) to 6.8% (45M out of 660M) of its original trainable parameters from a full ASR model to perform competitive results in a range of 11.9% to 8.1% WER averaged across different languages. In addition, we discover different setups to make large-scale pre-trained ASR succeed in both monolingual and multilingual speech recognition. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses (e.g., w2v-bert) in terms of lower WER and better training efficiency., Comment: Submitted to ICASSP 2023. The project was initiated in May 2022 during a research internship at Google Research