Search

Streaming automatic speech recognition (ASR) is very important for many real-world ASR applications. However, a notable challenge for streaming ASR systems lies in balancing operational performance against latency constraint. Recently, a method of chunking, simulating future context and decoding, called CUSIDE, has been proposed for connectionist temporal classification (CTC) based streaming ASR, which obtains a good balance between reduced latency and high recognition accuracy. In this paper, we present CUSIDE-T, which successfully adapts the CUSIDE method over the recurrent neural network transducer (RNN-T) ASR architecture, instead of being based on the CTC architecture. We also incorporate language model rescoring in CUSIDE-T to further enhance accuracy, while only bringing a small additional latency. Extensive experiments are conducted over the AISHELL-1, WenetSpeech and SpeechIO datasets, comparing CUSIDE-T and U2++ (both based on RNN-T). U2++ is an existing counterpart of chunk based streaming ASR method. It is shown that CUSIDE-T achieves superior accuracy performance for streaming ASR, with equal settings of latency.

In weak-lensing cosmological studies, peak statistics is sensitive to nonlinear structures and thus complementary to cosmic shear two-point correlations. In this paper, we explore a new approach, namely, the peak steepness statistics, with the overall goal to understand the cosmological information embedded there in comparison with the commonly used peak height statistics. We perform the analyses with ray-tracing simulations considering different sets of cosmological parameters $\Omega_{\rm m}$ and $\sigma_8$. A theoretical model to calculate the abundance of high peaks based on steepness is also presented, which can well describe the main trend of the peak distribution from simulations. We employ $\Delta\chi^2$ and Fisher analyses to study the cosmological dependence of the two peak statistics using our limited sets of simulations as well as our theoretical model. Within our considerations without including potential systematic effects, the results show that the steepness statistics tends to have higher sensitivities to the cosmological parameters than the peak height statistics and this advantage is diluted with the increase of the shape noise. Using the theoretical model, we investigate the physical reasons accounting for the different cosmological information embedded in the two statistics. Our analyses indicate that the projection effect from large-scale structures plays an important role to enhance the gain from the steepness statistics. The redshift and cosmology dependence of dark matter halo density profiles also contributes to the differences between the two statistics., Comment: 20 pages, 15 figures. Accepted for publication in MNRAS

Remote sensing of the Earth's surface water is critical in a wide range of environmental studies, from evaluating the societal impacts of seasonal droughts and floods to the large-scale implications of climate change. Consequently, a large literature exists on the classification of water from satellite imagery. Yet, previous methods have been limited by 1) the spatial resolution of public satellite imagery, 2) classification schemes that operate at the pixel level, and 3) the need for multiple spectral bands. We advance the state-of-the-art by 1) using commercial imagery with panchromatic and multispectral resolutions of 30 cm and 1.2 m, respectively, 2) developing multiple fully convolutional neural networks (FCN) that can learn the morphological features of water bodies in addition to their spectral properties, and 3) FCN that can classify water even from panchromatic imagery. This study focuses on rivers in the Arctic, using images from the Quickbird, WorldView, and GeoEye satellites. Because no training data are available at such high resolutions, we construct those manually. First, we use the RGB, and NIR bands of the 8-band multispectral sensors. Those trained models all achieve excellent precision and recall over 90% on validation data, aided by on-the-fly preprocessing of the training data specific to satellite imagery. In a novel approach, we then use results from the multispectral model to generate training data for FCN that only require panchromatic imagery, of which considerably more is available. Despite the smaller feature space, these models still achieve a precision and recall of over 85%. We provide our open-source codes and trained model parameters to the remote sensing community, which paves the way to a wide range of environmental hydrology applications at vastly superior accuracies and 2 orders of magnitude higher spatial resolution than previously possible., Comment: 21 pages, 10 figures, 3 tables

In this article, we establish a characterization of the set $M(B^{0,b}_{p,\infty}(\mathbb{R}^n))$ of all pointwise multipliers of Besov spaces $B^{0,b}_{p,\infty}(\mathbb{R}^n)$ with only logarithmic smoothness $b\in\mathbb{R}$ in the special cases $p=1$ and $p=\infty$. As applications of these two characterizations, we clarify whether or not the three concrete examples, namely characteristic functions of open sets, continuous functions defined by differences, and the functions $e^{ik\cdot x}$ with $k\in\mathbb{Z}^n$ and $x\in\mathbb{R}^n$, are pointwise multipliers of $B^{0,b}_{1,\infty}(\mathbb{R}^n)$ and $B^{0,b}_{\infty,\infty}(\mathbb{R}^n)$, respectively; furthermore, we obtain the explicit estimates of $\|e^{ik \cdot x}\|_{M(B^{0,b}_{1,\infty}(\mathbb{R}^n))}$ and $\|e^{ik \cdot x}\|_{M(B^{0,b}_{\infty,\infty}(\mathbb{R}^n))}$. In the case that $p\in(1,\infty)$, we give some sufficient conditions and some necessary conditions of the pointwise multipliers of $B^{0,b}_{p,\infty}(\mathbb{R}^n)$ and a complete characterization of $M(B^{0,b}_{p,\infty}(\mathbb{R}^n))$ is still open. However, via a different method, we are still able to accurately calculate $\|e^{ik \cdot x}\|_{M(B^{0,b}_{p,\infty}(\mathbb{R}^n))}$, $k\in\mathbb{Z}^n$, in this situation. The novelty of this article is that most of the proofs are constructive and these constructions strongly depend on the logarithmic structure of Besov spaces under consideration., Comment: 64 pages; Submitted

Pre-training is prevalent in nowadays deep learning to improve the learned model's performance. However, in the literature on federated learning (FL), neural networks are mostly initialized with random weights. These attract our interest in conducting a systematic study to explore pre-training for FL. Across multiple visual recognition benchmarks, we found that pre-training can not only improve FL, but also close its accuracy gap to the counterpart centralized learning, especially in the challenging cases of non-IID clients' data. To make our findings applicable to situations where pre-trained models are not directly available, we explore pre-training with synthetic data or even with clients' data in a decentralized manner, and found that they can already improve FL notably. Interestingly, many of the techniques we explore are complementary to each other to further boost the performance, and we view this as a critical result toward scaling up deep FL for real-world applications. We conclude our paper with an attempt to understand the effect of pre-training on FL. We found that pre-training enables the learned global models under different clients' data conditions to converge to the same loss basin, and makes global aggregation in FL more stable. Nevertheless, pre-training seems to not alleviate local model drifting, a fundamental problem in FL under non-IID data., Comment: Accepted to ICLR 2023

We propose and study an efficient photonic CDE method using MZI-based coherent optical matrix multiplication. It improves the compensation performance by about 60% when the tap-length is limited, and only 50% taps of the theoretical value is needed for photonic CDE with 1-dB penalty., Comment: 4 pages,5 figures, Conference paper, submitted to European Conference on Optical Communication (ECOC)

Tropical precipitation clusters exhibit power-law frequency distributions in area and volume (integrated precipitation), implying a lack of characteristic scale in tropical convective organization. However, it remains unknown what gives rise to the power laws and how the power-law exponents for area and volume are related to one another. Here, we explore the perspective that precipitation clusters are islands above a convective threshold on a rough column-water-vapor (CWV) topography. This perspective is supported by the agreement between the precipitation clusters and CWV islands in their frequency distributions as well as fractal dimensions. Power laws exist for CWV islands at different thresholds through the CWV topography, suggesting that the existence of power-laws is not specifically related to local precipitation dynamics, but is rather a general feature of CWV islands. Furthermore, the frequency distributions and fractal dimensions of the clusters can be reproduced when the CWV field is modeled to be self-affine with a roughness exponent of 0.3. Self-affine scaling theory relates the statistics of precipitation clusters to the roughness exponent; it also relates the power-law slopes for area and volume without involving the roughness exponent. Thus, the perspective of precipitation clusters as CWV islands provides a useful framework to consider many statistical properties of the precipitation clusters, particularly given that CWV is well-observed over a wide range of length scales in the tropics. However, the statistics of CWV islands at the convective threshold imply a smaller roughness than is inferred from the power spectrum of the bulk CWV field, and further work is needed to understand the scaling of the CWV field., Comment: 22 pages, 9 figures

The optimal design of neural networks is a critical problem in many applications. Here, we investigate how dynamical systems with polynomial nonlinearities can inform the design of neural systems that seek to emulate them. We propose a Learnability metric and its associated features to quantify the near-equilibrium behavior of learning dynamics. Equating the Learnability of neural systems with equivalent parameter estimation metric of the reference system establishes bounds on network structure. In this way, norms from theory provide a good first guess for neural structure, which may then further adapt with data. The proposed approach neither requires training nor training data. It reveals exact sizing for a class of neural networks with multiplicative nodes that mimic continuous- or discrete-time polynomial dynamics. It also provides relatively tight lower size bounds for classical feed-forward networks that is consistent with simulated assessments., Comment: 7 pages

The Super Tau-Charm Facility (STCF) is a future electron-positron collider proposed in China with a peak luminosity of above 0.5$\times$10$^{35}$ cm$^{-2}$s$^{-1}$ and center-of-mass energy ranging from 2 to 7 GeV. An excellent particle identification (PID) capability is one of the most important requirements for the detector at the STCF. A 3$\sigma$ $\pi$/K separation power at the momentum of up to 2 GeV/c is required within the detector acceptance. A DIRC-like time-of-flight (DTOF) detector is proposed to meet the PID requirement for the endcap region of the STCF. The conceptual design of the DTOF detector and its geometry optimization is herein presented. The PID performance of the detector is studied using Geant4 simulation. With a proper reconstruction algorithm, an overall time resolution of ~50 ps is achieved for the detector with an optimum geometry when convoluting contributions from all other sources, including the transit time spread (TTS) of the photodetector, electronic timing accuracy, and an assumed precision (~40 ps) of the event start time. A $\pi$/K separation power of better than 4$\sigma$ at the momentum of 2 GeV/c is achieved over the entire sensitive area of the DTOF detector, thereby fulfilling the physics requirement of the PID detector for the experiment at the STCF., Comment: accepted for Journal of Instrumentation

The Large High-Altitude Air Shower Observatory (LHAASO) is being built at Haizi Mountain, Sichuan province of China at an altitude of 4410 meters. One of its main goals is to survey the northern sky for very-high-energy gamma ray sources via its ground-based water Cherenkov detector array (WCDA). 900 8-inch photomultiplier tubes (PMTs) CR365-02-1 from Beijing Hamamatsu Photon Techniques INC. (BHP) are installed in the WCDA, collecting Cherenkov photons produced by air shower particles crossing water. The design of the PMT base with a high dynamic range for CR365-02-1, the PMT batch test system, and the test results of 997 PMTs are presented in this paper., Comment: 23 pages, 17 figures

Convolutional Neutral Networks have been successfully applied in searching for strong lensing systems, leading to discoveries of new candidates from large surveys. On the other hand, systematic investigations about their robustness are still lacking. In this paper, we first construct a neutral network, and apply it to $r$-band images of Luminous Red Galaxies (LRGs) of the Kilo Degree Survey (KiDS) Data Release 3 to search for strong lensing systems. We build two sets of training samples, one fully from simulations, and the other one using the LRG stamps from KiDS observations as the foreground lens images. With the former training sample, we find 48 high probability candidates after human-inspection, and among them, 27 are newly identified. Using the latter training set, about 67\% of the aforementioned 48 candidates are also found, and there are 11 more new strong lensing candidates identified. We then carry out tests on the robustness of the network performance with respect to the variation of PSF. With the testing samples constructed using PSF in the range of 0.4 to 2 times of the median PSF of the training sample, we find that our network performs rather stable, and the degradation is small. We also investigate how the volume of the training set can affect our network performance by varying it from 0.1 millions to 0.8 millions. The output results are rather stable showing that within the considered range, our network performance is not very sensitive to the volume size., Comment: 17 pages, 13 figures. Accepted by MNRAS. Comments welcome

The use of artificial neural networks as models of chaotic dynamics has been rapidly expanding. Still, a theoretical understanding of how neural networks learn chaos is lacking. Here, we employ a geometric perspective to show that neural networks can efficiently model chaotic dynamics by becoming structurally chaotic themselves. We first confirm neural network's efficiency in emulating chaos by showing that a parsimonious neural network trained only on few data points can reconstruct strange attractors, extrapolate outside training data boundaries, and accurately predict local divergence rates. We then posit that the trained network's map comprises sequential geometric stretching, rotation, and compression operations. These geometric operations indicate topological mixing and chaos, explaining why neural networks are naturally suitable to emulate chaotic dynamics., Comment: in IEEE Transactions on Neural Networks and Learning Systems

Precipitation extremes intensify in most regions in climate-model projections. Changes in vertical velocities contribute to the changes in intensity of precipitation extremes but remain poorly understood. Here, we find that mid-tropospheric vertical velocities in extratropical precipitation extremes strengthen overall in simulations of 21st-century climate change. For each extreme event, we solve the quasi-geostrophic omega equation to decompose this strengthening into different physical contributions. We first consider a dry decomposition in which latent heating is treated as an external forcing of upward motion. Much of the positive contribution to upward motion from increased latent heating is offset by negative contributions from increases in dry static stability and changes in the horizontal length scale of vertical velocities. However, taking changes in latent heating as given is a limitation when the aim is to understand changes in precipitation, since latent heating and precipitation are closely linked. Therefore, we also perform a moist decomposition of the changes in vertical velocities in which latent heating is represented through a moist static stability. In the moist decomposition, changes in moist static stability play a key role and contributions from other factors such as changes in the depth of the upward motion increase in importance. While both dry and moist decompositions are self-consistent, the moist dynamical perspective has greater potential to give insights into the causes of the dynamical contributions to changes in precipitation extremes in different regions.

Two-dimensional semiconductors host excitons with very large oscillator strengths and binding energies due to significantly reduced carrier screening. Two-dimensional semiconductors integrated with optical cavities are emerging as a promising platform for studying strong light-matter interactions as a route to explore a variety of exotic many-body effects. Here, in few-layered WS2 coupled with plasmonic nanoparticle lattices, we observe the formation of a collective polaritonic mode near the exciton energy and the formation of a complete polariton band gap with energy scale comparable to the exciton-plasmon coupling strength. A coupled oscillator model reveals that the collective mode arises from the cooperative coupling of the excitons to the plasmonic lattice diffraction orders via exciton-exciton interactions. The emergence of the collective mode is accompanied by a superlinear increase of the polariton mode splitting as a function of the square root of the exciton oscillator strength. The presence of these many body effects, which are enhanced in systems which lack bulk polarization, not only allows the formation of a collective mode with periodically varying field profiles, but also further enhances the exciton-plasmon coupling. By integrating the hybrid WS2-plasmonic lattice device with a field-effect transistor, we demonstrate active tuning of the collective mode and the polariton band gap. These systems provide new opportunities for obtaining a deeper and systematic understanding of many body cooperative phenomena in periodic photonic systems and for designing more complex and actively controllable polaritonic devices including switchable polariton lasers, waveguides, and optical logical elements.

This paper presents a method to reconstruct dense semantic trajectory stream of human interactions in 3D from synchronized multiple videos. The interactions inherently introduce self-occlusion and illumination/appearance/shape changes, resulting in highly fragmented trajectory reconstruction with noisy and coarse semantic labels. Our conjecture is that among many views, there exists a set of views that can confidently recognize the visual semantic label of a 3D trajectory. We introduce a new representation called 3D semantic map---a probability distribution over the semantic labels per trajectory. We construct the 3D semantic map by reasoning about visibility and 2D recognition confidence based on view-pooling, i.e., finding the view that best represents the semantics of the trajectory. Using the 3D semantic map, we precisely infer all trajectory labels jointly by considering the affinity between long range trajectories via estimating their local rigid transformations. This inference quantitatively outperforms the baseline approaches in terms of predictive validity, representation robustness, and affinity effectiveness. We demonstrate that our algorithm can robustly compute the semantic labels of a large scale trajectory set involving real-world human interactions with object, scenes, and people.

Current multispectral imagers suffer from low photon efficiency and limited spectrum range. These limitations are partially due to the technological limitations from array sensors (CCD or CMOS), and also caused by separative measurement of the entries/slices of a spatial-spectral data cube. Besides, they are mostly expensive and bulky. To address above issues, this paper proposes to image the 3D multispectral data with a single bucket detector in a multiplexing way. Under the single pixel imaging scheme, we project spatial-spectral modulated illumination onto the target scene to encode the scene's 3D information into a 1D measurement sequence. Conventional spatial modulation is used to resolve the scene's spatial information. To avoid increasing requisite acquisition time for 2D to 3D extension of the latent data, we conduct spectral modulation in a frequency-division multiplexing manner in the speed gap between slow spatial light modulation and fast detector response. Then the sequential reconstruction falls into a simple Fourier decomposition and standard compressive sensing problem. A proof-of-concept setup is built to capture the multispectral data (64 pixels $\times$ 64 pixels $\times$ 10 wavelength bands) in the visible wavelength range (450nm-650nm) with acquisition time being 1 minute. The imaging scheme is of high flexibility for different spectrum ranges and resolutions. It holds great potentials for various low light and airborne applications, and can be easily manufactured production-volume portable multispectral imagers.