Search

This paper concerns the tilt stability for the minimization of the sum of a twice continuously differentiable matrix-valued function and the Ky-Fan $\kappa$-norm. By using the expression of second subderivative of the Ky-Fan $\kappa$-norm, we derive a verifiable criterion to identify the tilt stability of a local minimum for this class of nonconvex and nonsmooth problems. As a byproduct, a practical criterion is achieved for the tilt stable solution of the nuclear-norm regularized minimization., Comment: 30pages

Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires learning to ground language in perception and embodied actions, in order to accomplish complex tasks. The Scalable, Instructable, Multiworld Agent (SIMA) project tackles this by training agents to follow free-form instructions across a diverse range of virtual 3D environments, including curated research environments as well as open-ended, commercial video games. Our goal is to develop an instructable agent that can accomplish anything a human can do in any simulated 3D environment. Our approach focuses on language-driven generality while imposing minimal assumptions. Our agents interact with environments in real-time using a generic, human-like interface: the inputs are image observations and language instructions and the outputs are keyboard-and-mouse actions. This general approach is challenging, but it allows agents to ground language across many visually complex and semantically rich environments while also allowing us to readily run agents in new environments. In this paper we describe our motivation and goal, the initial progress we have made, and promising preliminary results on several diverse research environments and a variety of commercial video games.

Computing the proximal operator of the sparsity-promoting piece-wise exponential (PiE) penalty $1-e^{-|x|/\sigma}$ with a given shape parameter $\sigma>0$, which is treated as a popular nonconvex surrogate of $\ell_0$-norm, is fundamental in feature selection via support vector machines, image reconstruction, zero-one programming problems, compressed sensing, etc. Due to the nonconvexity of PiE, for a long time, its proximal operator is frequently evaluated via an iteratively reweighted $\ell_1$ algorithm, which substitutes PiE with its first-order approximation, however, the obtained solutions only are the critical point. Based on the exact characterization of the proximal operator of PiE, we explore how the iteratively reweighted $\ell_1$ solution deviates from the true proximal operator in certain regions, which can be explicitly identified in terms of $\sigma$, the initial value and the regularization parameter in the definition of the proximal operator. Moreover, the initial value can be adaptively and simply chosen to ensure that the iteratively reweighted $\ell_1$ solution belongs to the proximal operator of PiE.

Annual variations of interstellar scintillation can be modelled to constrain parameters of the ionized interstellar medium. If a pulsar is in a binary system, then investigating the orbital parameters is possible through analysis of the orbital variation of scintillation. In observations carried out from 2011 January to 2020 August by the European Pulsar Timing Array radio telescopes, PSRs~J0613$-$0200 and J0636+5128 show strong annual variations in their scintillation velocity, while the former additionally exhibits an orbital fluctuation. Bayesian theory and Markov-chain-Monte-Carlo methods are used to interpret these periodic variations. We assume a thin and anisotropic scattering screen model, and discuss the mildly and extremely anisotropic scattering cases. PSR~J0613$-$0200 is best described by mildly anisotropic scattering, while PSR~J0636+5128 exhibits extremely anisotropic scattering. We measure the distance, velocity and degree of anisotropy of the scattering screen for our two pulsars, finding that scattering screen distances from Earth for PSRs~J0613$-$0200 and J0636+5128 are 316$^{+28}_{-20}$\,pc and 262$^{+96}_{-38}$\,pc, respectively. The positions of these scattering screens are coincident with the shell of the Local Bubble towards both pulsars. These associations add to the growing evidence of the Local Bubble shell as a dominant region of scattering along many sightlines., Comment: Accepted by SCIENCE CHINA Physics, Mechanics & Astronomy ( SCPMA)

Support Vector Machine (SVM) has been one of the most successful machine learning techniques for binary classification problems. The key idea is to maximize the margin from the data to the hyperplane subject to correct classification on training samples. The commonly used hinge loss and its variations are sensitive to label noise, and unstable for resampling due to its unboundedness. This paper is concentrated on the kernel SVM with the $\ell_0$-norm hinge loss (referred as $\ell_0$-KSVM), which is a composite function of hinge loss and $\ell_0$-norm and then could overcome the difficulties mentioned above. In consideration of the nonconvexity and nonsmoothness of $\ell_0$-norm hinge loss, we first characterize the limiting subdifferential of the $\ell_0$-norm hinge loss and then derive the equivalent relationship among the proximal stationary point, the Karush-Kuhn-Tucker point, and the local optimal solution of $\ell_0$-KSVM. Secondly, we develop an ADMM algorithm for $\ell_0$-KSVM, and obtain that any limit point of the sequence generated by the proposed algorithm is a locally optimal solution. Lastly, some experiments on the synthetic and real datasets are illuminated to show that $\ell_0$-KSVM can achieve comparable accuracy compared with the standard KSVM while the former generally enjoys fewer support vectors.

This paper characterizes the proximal operator of the piece-wise exponential function $1\!-\!e^{-|x|/\sigma}$ with a given shape parameter $\sigma\!>\!0$, which is a popular nonconvex surrogate of $\ell_0$-norm in support vector machines, zero-one programming problems, and compressed sensing, etc. Although Malek-Mohammadi et al. [IEEE Transactions on Signal Processing, 64(21):5657--5671, 2016] once worked on this problem, the expressions they derived were regrettably inaccurate. In a sense, it was lacking a case. Using the Lambert W function and an extensive study of the piece-wise exponential function, we have rectified the formulation of the proximal operator of the piece-wise exponential function in light of their work. We have also undertaken a thorough analysis of this operator. Finally, as an application in compressed sensing, an iterative shrinkage and thresholding algorithm (ISTA) for the piece-wise exponential function regularization problem is developed and fully investigated. A comparative study of ISTA with nine popular non-convex penalties in compressed sensing demonstrates the advantage of the piece-wise exponential penalty.

Interstellar scattering (ISS) of radio pulsar emission can be used as a probe of the ionised interstellar medium (IISM) and causes corruptions in pulsar timing experiments. Two types of ISS phenomena (intensity scintillation and pulse broadening) are caused by electron density fluctuations on small scales (< 0.01 AU). Theory predicts that these are related, and both have been widely employed to study the properties of the IISM. Larger scales ($\sim$1-100\,AU) cause measurable changes in dispersion and these can be correlated with ISS observations to estimate the fluctuation spectrum over a very wide scale range. IISM measurements can often be modeled by a homogeneous power-law spatial spectrum of electron density with the Kolmogorov ($-11/3$) spectral exponent. Here we aim to test the validity of using the Kolmogorov exponent with PSR~J0826+2637. We do so using observations of intensity scintillation, pulse broadening and dispersion variations across a wide fractional bandwidth (20 -- 180\,MHz). We present that the frequency dependence of the intensity scintillation in the high frequency band matches the expectations of a Kolmogorov spectral exponent but the pulse broadening in the low frequency band does not change as rapidly as predicted with this assumption. We show that this behavior is due to an inhomogeneity in the scattering region, specifically that the scattering is dominated by a region of transverse size $\sim$40\,AU. The power spectrum of the electron density, however, maintains the Kolmogorov spectral exponent from spatial scales of 5$\times10^{-6}$\,AU to $\sim$100\,AU., Comment: Accepted for publication in MNRAS, typo fixed

This paper concerns the twice epi-differentiability and parabolic regularity of a class of non-amenable functions, the composition of a piecewise twice differentiable (PWTD) function and a parabolically semidifferentiable mapping. Such composite functions often appear in composite optimization problems, disjunctive optimization problems, and low-rank and/or sparsity optimization problems. By establishing the proper twice epi-differentiability and parabolic epi-differentiability of PWTD functions, we prove the parabolic epi-differentiability of this class of composite functions, and its twice epi-differentiability under the parabolic regularity assumption. Then, we identify a condition to ensure its parabolic regularity with the help of an upper and lower estimate of its second subderivative, and demonstrate that this condition holds for several classes of specific non-amenable functions., Comment: 32pages

Interstellar scintillation analysis of pulsars allows us to probe the small-scale distribution and inhomogeneities of the ionized interstellar medium. Our priority is to present the data set and the basic measurements of scintillation parameters of pulsars employing long-term scintillation observations carried out from 2011 January to 2020 August by the European Pulsar Timing Array radio telescopes in the 21-cm and 11-cm bands. Additionally, we aim to identify future possible lines of study using this long-term scintillation dataset. We present the long-term time series of $\nu_{\rm d}$ and $\tau_{\rm d}$ for 13 pulsars. Sanity-checks and comparisons indicate that the scintillation parameters of our work and previously published works are mostly consistent. For two pulsars, PSRs~J1857+0943 and J1939+2134, we were able to obtain measurements of the $\nu_{\rm d}$ at both bands, which allows us to derive the time series of frequency scaling indices with a mean and a standard deviation of 2.82$\pm$1.95 and 3.18$\pm$0.60, respectively. We found some interesting features which will be studied in more detail in subsequent papers in this series: (i) in the time series of PSR~J1939+2134, where the scintillation bandwidth sharply increases or decreases associated with a sharp change of dispersion measure; (ii) PSR~J0613$-$0200 and PSR~J0636+5126 show a strong annual variation in the time series of the $\tau_{\rm d}$; (iii) PSR~J1939+2134 shows a weak anti-correlation between scintillation timescale and dispersion in WSRT data., Comment: 18 pages, 6 figures, 4 tables, accepted for publication in A&A

Context. Interstellar scintillation (ISS) of pulsar emission can be used both as a probe of the ionised interstellar medium (IISM) and cause corruptions in pulsar timing experiments. Of particular interest are so-called scintillation arcs which can be used to measure time-variable interstellar scattering delays directly, potentially allowing high-precision improvements to timing precision. Aims. The primary aim of this study is to carry out the first sizeable and self-consistent census of diffractive pulsar scintillation and scintillation-arc detectability at low frequencies, as a primer for larger-scale IISM studies and pulsar-timing related propagation studies with the LOw-Frequency ARray (LOFAR) High Band Antennae (HBA). Results. In this initial set of 31 sources, 15 allow full determination of the scintillation properties; nine of these show detectable scintillation arcs at 120-180 MHz. Eight of the observed sources show unresolved scintillation; and the final eight don't display diffractive scintillation. Some correlation between scintillation detectability and pulsar brightness and dispersion measure is apparent, although no clear cut-off values can be determined. Our measurements across a large fractional bandwidth allow a meaningful test of the frequency scaling of scintillation parameters, uncorrupted by influences from refractive scintillation variations. Conclusions. Our results indicate the powerful advantage and great potential of ISS studies at low frequencies and the complex dependence of scintillation detectability on parameters like pulsar brightness and interstellar dispersion. This work provides the first installment of a larger-scale census and longer-term monitoring of interstellar scintillation effects at low frequencies., Comment: 16 pages, 6 figures, accepted for publication in A&A

Automatic speech recognition (ASR) of single channel far-field recordings with an unknown number of speakers is traditionally tackled by cascaded modules. Recent research shows that end-to-end (E2E) multi-speaker ASR models can achieve superior recognition accuracy compared to modular systems. However, these models do not ensure real-time applicability due to their dependency on full audio context. This work takes real-time applicability as the first priority in model design and addresses a few challenges in previous work on multi-speaker recurrent neural network transducer (MS-RNN-T). First, we introduce on-the-fly overlapping speech simulation during training, yielding 14% relative word error rate (WER) improvement on LibriSpeechMix test set. Second, we propose a novel multi-turn RNN-T (MT-RNN-T) model with an overlap-based target arrangement strategy that generalizes to an arbitrary number of speakers without changes in the model architecture. We investigate the impact of the maximum number of speakers seen during training on MT-RNN-T performance on LibriCSS test set, and report 28% relative WER improvement over the two-speaker MS-RNN-T. Third, we experiment with a rich transcription strategy for joint recognition and segmentation of multi-party speech. Through an in-depth analysis, we discuss potential pitfalls of the proposed system as well as promising future research directions., Comment: Accepted by ICASSP 2022

End-to-end (E2E) automatic speech recognition (ASR) models have recently demonstrated superior performance over the traditional hybrid ASR models. Training an E2E ASR model requires a large amount of data which is not only expensive but may also raise dependency on production data. At the same time, synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines has advanced to near-human naturalness. In this work, we propose to utilize synthetic speech for ASR training (SynthASR) in applications where data is sparse or hard to get for ASR model training. In addition, we apply continual learning with a novel multi-stage training strategy to address catastrophic forgetting, achieved by a mix of weighted multi-style training, data augmentation, encoder freezing, and parameter regularization. In our experiments conducted on in-house datasets for a new application of recognizing medication names, training ASR RNN-T models with synthetic audio via the proposed multi-stage training improved the recognition performance on new application by more than 65% relative, without degradation on existing general applications. Our observations show that SynthASR holds great promise in training the state-of-the-art large-scale E2E ASR models for new applications while reducing the costs and dependency on production data., Comment: Accepted to Interspeech 2021

This paper is concerned with the calmness of a partial perturbation to the composite rank constraint system, an intersection of the rank constraint set and a general closed set, which is shown to be equivalent to a local Lipschitz-type error bound and also a global Lipschitz-type error bound under a certain compactness. Based on its lifted formulation, we derive two criteria for identifying those closed sets such that the associated partial perturbation possesses the calmness, and provide a collection of examples to demonstrate that the criteria are satisfied by common nonnegative and positive semidefinite rank constraint sets. Then, we use the calmness of this perturbation to obtain several global exact penalties for rank constrained optimization problems, and a family of equivalent DC surrogates for rank regularized problems.

Bootstrapping speech recognition on limited data resources has been an area of active research for long. The recent transition to all-neural models and end-to-end (E2E) training brought along particular challenges as these models are known to be data hungry, but also came with opportunities around language-agnostic representations derived from multilingual data as well as shared word-piece output representations across languages that share script and roots. We investigate here the effectiveness of different strategies to bootstrap an RNN-Transducer (RNN-T) based automatic speech recognition (ASR) system in the low resource regime, while exploiting the abundant resources available in other languages as well as the synthetic audio from a text-to-speech (TTS) engine. Our experiments demonstrate that transfer learning from a multilingual model, using a post-ASR text-to-text mapping and synthetic audio deliver additive improvements, allowing us to bootstrap a model for a new language with a fraction of the data that would otherwise be needed. The best system achieved a 46% relative word error rate (WER) reduction compared to the monolingual baseline, among which 25% relative WER improvement is attributed to the post-ASR text-to-text mappings and the TTS synthetic data.

Recent research shows end-to-end ASR systems can recognize overlapped speech from multiple speakers. However, all published works have assumed no latency constraints during inference, which does not hold for most voice assistant interactions. This work focuses on multi-speaker speech recognition based on a recurrent neural network transducer (RNN-T) that has been shown to provide high recognition accuracy at a low latency online recognition regime. We investigate two approaches to multi-speaker model training of the RNN-T: deterministic output-target assignment and permutation invariant training. We show that guiding separation with speaker order labels in the former case enhances the high-level speaker tracking capability of RNN-T. Apart from that, with multistyle training on single- and multi-speaker utterances, the resulting models gain robustness against ambiguous numbers of speakers during inference. Our best model achieves a WER of 10.2% on simulated 2-speaker LibriSpeech data, which is competitive with the previously reported state-of-the-art nonstreaming model (10.3%), while the proposed model could be directly applied for streaming applications., Comment: Accepted at ICASSP2021

Today, many state-of-the-art automatic speech recognition (ASR) systems apply all-neural models that map audio to word sequences trained end-to-end along one global optimisation criterion in a fully data driven fashion. These models allow high precision ASR for domains and words represented in the training material but have difficulties recognising words that are rarely or not at all represented during training, i.e. trending words and new named entities. In this paper, we use a text-to-speech (TTS) engine to provide synthetic audio for out-of-vocabulary (OOV) words. We aim to boost the recognition accuracy of a recurrent neural network transducer (RNN-T) on OOV words by using the extra audio-text pairs, while maintaining the performance on the non-OOV words. Different regularisation techniques are explored and the best performance is achieved by fine-tuning the RNN-T on both original training data and extra synthetic data with elastic weight consolidation (EWC) applied on the encoder. This yields a 57% relative word error rate (WER) reduction on utterances containing OOV words without any degradation on the whole test set., Comment: To appear in Proc. ICASSP2021, June 06-11, 2021, Toronto, Ontario, Canada

This paper is concerned with second-order optimality conditions for the mathematical program with semidefinite cone complementarity constraints (SDCMPCC).To achieve this goal, we first provide an exact characterization on the second-order tangent set to the semidefinite cone complementarity (SDCC) set in terms of the second-order directional derivative of the projection operator onto the SDCC set, and then use the second-order tangent set to the SDCC set to derive second-order necessary and sufficient conditions for SDCMPCC under suitable subregularity constraint qualifications. An application is also illustrated in characterizing the second-order necessary optimality condition for the semidefinite rank regularized problem., Comment: 26pages

For the rank regularized minimization problem, we introduce several kinds of stationary points by the problem itself and its equivalent reformulations including the mathematical program with an equilibrium constraint (MPEC), the global exact penalty of the MPEC,the surrogate yielded by eliminating the dual part in the exact penalty. A clear relation chart is established for these stationary points, which guides the user to choose an appropriate reformulation for seeking a low-rank solution. As a byproduct, we also provide a weaker condition for a local minimizer of the MPEC to be the M-stationary point by characterizing the directional limiting normal cone to the graph of the normal cone mapping of the positive semidefinite (PSD) cone., Comment: 28 pages

For a proper extended real-valued function, this work focuses on the relationship between the subregularity of its subdifferential mapping relative to the critical set and its KL property of exponent 1/2. When the function is lsc convex, we establish the equivalence between them under the continuous assumption on the critical set. Then, for the uniformly prox-regular function, under its continuity on the local minimum set, the KL property of exponent 1/2 on the local minimum set is shown to be equivalent to the subregularity of its subdifferential relative to this set. Moreover, for this class of nonconvex functions, under a separation assumption of stationary values, we show that the subregularity of its subdifferential relative to the critical set also implies its KL property of exponent $1/2$. These results provide a bridge for the two kinds of regularity, and their application is illustrated by examples., Comment: 18 pages, 2 figures