Search

For ultra-wideband and high-rate wireless communication systems, wideband spectrum sensing (WSS) is critical, since it empowers secondary users (SUs) to capture the spectrum holes for opportunistic transmission. However, WSS encounters challenges such as excessive costs of hardware and computation due to the high sampling rate, as well as robustness issues arising from scenario mismatch. In this paper, a WSS neural network (WSSNet) is proposed by exploiting multicoset preprocessing to enable the sub-Nyquist sampling, with the two dimensional convolution design specifically tailored to work with the preprocessed samples. A federated transfer learning (FTL) based framework mobilizing multiple SUs is further developed to achieve a robust model adaptable to various scenarios, which is paved by the selective weight pruning for the fast model adaptation and inference. Simulation results demonstrate that the proposed FTL-WSSNet achieves the fairly good performance in different target scenarios even without local adaptation samples.

The StutteringSpeech Challenge focuses on advancing speech technologies for people who stutter, specifically targeting Stuttering Event Detection (SED) and Automatic Speech Recognition (ASR) in Mandarin. The challenge comprises three tracks: (1) SED, which aims to develop systems for detection of stuttering events; (2) ASR, which focuses on creating robust systems for recognizing stuttered speech; and (3) Research track for innovative approaches utilizing the provided dataset. We utilizes an open-source Mandarin stuttering dataset AS-70, which has been split into new training and test sets for the challenge. This paper presents the dataset, details the challenge tracks, and analyzes the performance of the top systems, highlighting improvements in detection accuracy and reductions in recognition error rates. Our findings underscore the potential of specialized models and augmentation strategies in developing stuttered speech technologies., Comment: 8 pages, 2 figures, accepted by SLT 2024

Non-classical light sources emitting bundles of N-photons with strong correlation represent versatile resources of interdisciplinary importance with applications ranging from fundamental tests of quantum mechanics to quantum information processing. Yet, high-order correlations, gN(0),quantifying photon correlation, are still limited to hundreds. Here, we report the generation of a super-bunching light source in photonic crystal fiber with g2(0) reaching 5.86*104 and g5(0) up to 2.72*108, through measuring its photon number probability distributions. under giant g2(0) values, the super-bunching light source presents upturned-tail photon distributions and ubiquitous extreme multi-photon events, where 31 photons from a single light pulse at a mean of 1.99*10-4 photons per pulse have been determined. The probability of this extreme event has been enhanced by 10139 folds compared to a coherent laser with Poissonian distribution. By varying the power of the pumping laser, both photon number distributions and corresponding high-order correlations of this light source can be substantially tailored from Poissonian to super-bunching distributions. These phenomena are attributed to the synchronized nonlinear interactions in photonic crystal fibers pumping by bright squeezed light, and the theoretical simulations agree well with the experimental results. Our research showcases the ability to achieve non-classical light sources with giant high-order correlations and extreme multi-photon events, paving the way for high-order correlation imaging, extreme nonlinear optical effects, quantum information processing, and exploring light-matter interactions with multi-photon physics.

In recent years, dynamic parameterization of acoustic environments has garnered attention in audio processing. This focus includes room volume and reverberation time (RT60), which define local acoustics independent of sound source and receiver orientation. Previous studies show that purely attention-based models can achieve advanced results in room parameter estimation. However, their success relies on supervised pretrainings that require a large amount of labeled true values for room parameters and complex training pipelines. In light of this, we propose a novel Self-Supervised Blind Room Parameter Estimation (SS-BRPE) system. This system combines a purely attention-based model with self-supervised learning to estimate room acoustic parameters, from single-channel noisy speech signals. By utilizing unlabeled audio data for pretraining, the proposed system significantly reduces dependencies on costly labeled datasets. Our model also incorporates dynamic feature augmentation during fine-tuning to enhance adaptability and generalizability. Experimental results demonstrate that the SS-BRPE system not only achieves more superior performance in estimating room parameters than state-of-the-art (SOTA) methods but also effectively maintains high accuracy under conditions with limited labeled data. Code available at https://github.com/bjut-chunxiwang/SS-BRPE., Comment: 5 pages, 3 figures, submitted to ICASSP 2025

Document Structured Extraction (DSE) aims to extract structured content from raw documents. Despite the emergence of numerous DSE systems, their unified evaluation remains inadequate, significantly hindering the field's advancement. This problem is largely attributed to existing benchmark paradigms, which exhibit fragmented and localized characteristics. To address these limitations and offer a thorough evaluation of DSE systems, we introduce a novel benchmark named READoc, which defines DSE as a realistic task of converting unstructured PDFs into semantically rich Markdown. The READoc dataset is derived from 2,233 diverse and real-world documents from arXiv and GitHub. In addition, we develop a DSE Evaluation S$^3$uite comprising Standardization, Segmentation and Scoring modules, to conduct a unified evaluation of state-of-the-art DSE approaches. By evaluating a range of pipeline tools, expert visual models, and general VLMs, we identify the gap between current work and the unified, realistic DSE objective for the first time. We aspire that READoc will catalyze future research in DSE, fostering more comprehensive and practical solutions.

We extract the leading Fock-state light front wave functions (LF-LFWFs) of heavy flavor-asymmetric pseudoscalar mesons $D$, $B$ and $B_c$ from their Bethe-Salpeter wave functions based on Dyson-Schwinger equations approach, and study their leading twist parton distribution amplitudes, generalized parton distribution functions and transverse momentum dependent parton distributions. The spatial distributions of the quark and antiquark on the transverse plane are given, along with their charge and energy distributions on the light front. We find that in the considered mesons, the heavier quarks carry most longitudinal momentum fraction and yield narrow $x$-distributions, while the lighter quarks play an active role in shaping the transverse distributions within both spatial and momentum space, exhibiting a duality embodying characteristics from both light mesons and heavy quarkonium., Comment: 10 pages,9 figures

Pre-explored Semantic Maps, constructed through prior exploration using visual language models (VLMs), have proven effective as foundational elements for training-free robotic applications. However, existing approaches assume the map's accuracy and do not provide effective mechanisms for revising decisions based on incorrect maps. To address this, we introduce Context-Aware Replanning (CARe), which estimates map uncertainty through confidence scores and multi-view consistency, enabling the agent to revise erroneous decisions stemming from inaccurate maps without requiring additional labels. We demonstrate the effectiveness of our proposed method by integrating it with two modern mapping backbones, VLMaps and OpenMask3D, and observe significant performance improvements in object navigation tasks. More details can be found on the project page: https://carmaps.github.io/supplements/., Comment: CoRL 2024. The first three authors contributed equally, and their order of authorship is interchangeable. Project page: https://carmaps.github.io/supplements/

Maintaining the coherence in quantum systems is interesting in both fundamental physics and quantum information processing. In particular, suppressing the dephasing caused by thermal fluctuations in quantum systems can potentially enable functional quantum devices. Techniques to reduce motional dephasing of quantum superpositions include spin echo and bang-bang. In this paper, we demonstrate the effectiveness of a novel protocol on a collective quantum superposition state known as a Rydberg polariton. These collective states are potentially important in the context of single photon sources, optical transistor, all-optical quantum gates and fast read-out of quantum information. However progress in Rydberg polariton quantum technology has been hindered by fast motional dephasing on which no effective methods exist for undoing it. Here, we show how our protocol via optical dressing using Raman lasers cancels dephasing and enhances coherence times by more than an order of magnitude., Comment: 9 pages, 7 figures

Introducing superconductivity in topological materials can lead to innovative electronic phases and device functionalities. Here, we present a new strategy for quantum engineering of superconducting junctions in moire materials through direct, on-chip, and fully encapsulated 2D crystal growth. We achieve robust and designable superconductivity in Pd-metalized twisted bilayer molybdenum ditelluride (MoTe2) and observe anomalous superconducting effects in high-quality junctions across ~ 20 moire cells. Surprisingly, the junction develops enhanced, instead of weakened, superconducting behaviors, exhibiting fluctuations to a higher critical magnetic field compared to its adjacent Pd7MoTe2 superconductor. Additionally, the critical current further exhibits a striking V-shaped minimum at zero magnetic field. These features are unexpected in conventional Josephson junctions and indeed absent in junctions of natural bilayer MoTe2 created using the same approach. We discuss implications of these observations, including the possible formation of mixed even- and odd-parity superconductivity at the moire junctions. Our results also demonstrate a pathway to engineer and investigate superconductivity in fractional Chern insulators., Comment: 25 pages, 5 main article figures and 10 supplementary figures

Using particle data from the ELFIN satellites, we present a statistical study of 284 proton isotropy boundary events on the nightside magnetosphere, characterizing their occurrence and distribution in local time, latitude (L-shell), energy, and precipitating energy flux, as a function of geomagnetic activity. For a given charged particle species and energy, its isotropy boundary (IB) is the magnetic latitude poleward of which persistently isotropic pitch-angle distributions ($J_{prec}/J_{perp}\sim 1$) are first observed to occur. This isotropization is interpreted as resulting from magnetic field-line curvature (FLC) scattering in the equatorial magnetosphere. We find that proton IBs are observed under all observed activity levels, spanning 16 to 05 MLT with $\sim$100% occurrence between 19 and 03 MLT, trending toward 60% at dawn/dusk. These results are also compared with electron IB properties observed using ELFIN, where we find similar trends across local time and activity, with the onset in $\geq$50 keV proton IB occurring on average 2 L-shells lower, and providing between 3 and 10 times as much precipitating power. Proton IBs typically span $64^\circ$-$66^\circ$ in magnetic latitude (5-6 in L-shell), corresponding to the outer edge of the ring current, tending toward lower IGRF latitudes as geomagnetic activity increases. The IBs were found to commonly occur 0.3-2.1 Re beyond the plasmapause. Proton IBs typically span $<$50 keV to $\sim$1 MeV in energy, maximizing near 22 MLT, and decreasing to a typical upper limit of 300-400 keV toward dawn and dusk, with peak observed isotropic energy increasing by $\sim$500 keV during active intervals. These results suggest that FLC in the vicinity of IBs can provide a substantial depletion mechanism for energetic protons, with the total nightside precipitating power from FLC-scattering found to be on the order of 100 MW, at times $\geq$10 GW.

We continue to study a quintic Z2-equivariant Li\'enard system $\dot x=y,\dot y=-(a_0x+a_1x^3+a_2x^5)-(b_0+b_1x^2)y$ with $a_2b_1\ne 0$, arising from the complex Ginzburg-Landau equation. Global dynamics of the system have been studied in [{\it SIAM J. Math. Anal.}, {\bf 55}(2023) 5993-6038] when the sum of the indices of all equilibria is $-1$, i.e., $a_2<0$. The aim of this paper is to study the global dynamics of this quintic Li\'enard system when the sum of the indices of all equilibria is $1$, i.e., $a_2>0$., Comment: 51 pages

In the single-field model, the preheating process occurs through self-resonance of inflaton field. We study the nonlinear structures generated during preheating in the $\alpha$-attractor models and monodromy models. The potentials have a power law form $\propto\left|\phi\right|^{2n}$ near the origin and a flat region away from bottom, which are consistent with current cosmological observations. The Floquet analysis shows that potential parameters in monodromy model have a significant influence on the region of resonance bands. The analytical oscillons solution for the $\alpha$-attractor T model with parameter $n=1$ is derived using the small amplitude analysis method. Besides we investigate the formation of nonlinear structures, the equation of state and the energy transfer through the (3+1) dimensional lattice simulation. We find that the symmetric T potential and the asymmetric E potential in the $\alpha$-attractor models have similar nonlinear dynamics. And the potential parameter $n$ in monodromy model significantly influences the lifetime of transients, whereas the parameter $q$ exerts minimal impact on the nonlinear dynamics., Comment: 14 pages, 12 figures, 2 tables

Social interactions form the foundation of human societies. Artificial intelligence has made significant progress in certain areas, but enabling machines to seamlessly understand social interactions remains an open challenge. It is important to address this gap by endowing machines with social capabilities. We identify three key capabilities needed for effective social understanding: 1) understanding multimodal social cues, 2) understanding multi-party dynamics, and 3) understanding beliefs. Building upon these foundations, we classify and review existing machine learning works on social understanding from the perspectives of verbal, non-verbal, and multimodal social cues. The verbal branch focuses on understanding linguistic signals such as speaker intent, dialogue sentiment, and commonsense reasoning. The non-verbal branch addresses techniques for perceiving social meaning from visual behaviors such as body gestures, gaze patterns, and facial expressions. The multimodal branch covers approaches that integrate verbal and non-verbal multimodal cues to holistically interpret social interactions such as recognizing emotions, conversational dynamics, and social situations. By reviewing the scope and limitations of current approaches and benchmarks, we aim to clarify the development trajectory and illuminate the path towards more comprehensive intelligence for social understanding. We hope this survey will spur further research interest and insights into this area.

Automatic Medical Imaging Narrative generation aims to alleviate the workload of radiologists by producing accurate clinical descriptions directly from radiological images. However, the subtle visual nuances and domain-specific terminology in medical images pose significant challenges compared to generic image captioning tasks. Existing approaches often neglect the vital distinction between normal and abnormal findings, leading to suboptimal performance. In this work, we propose FODA-PG, a novel Fine-grained Organ-Disease Adaptive Partitioning Graph framework that addresses these limitations through domain-adaptive learning. FODA-PG constructs a granular graphical representation of radiological findings by separating disease-related attributes into distinct "disease-specific" and "disease-free" categories based on their clinical significance and location. This adaptive partitioning enables our model to capture the nuanced differences between normal and pathological states, mitigating the impact of data biases. By integrating this fine-grained semantic knowledge into a powerful transformer-based architecture and providing rigorous mathematical justifications for its effectiveness, FODA-PG generates precise and clinically coherent reports with enhanced generalization capabilities. Extensive experiments on the IU-Xray and MIMIC-CXR benchmarks demonstrate the superiority of our approach over state-of-the-art methods, highlighting the importance of domain adaptation in medical report generation.

Ground-roll wave is a common coherent noise in land field seismic data. This Rayleigh-type surface wave usually has low frequency, low apparent velocity, and high amplitude, therefore obscures the reflection events of seismic shot gathers. Commonly used techniques focus on the differences of ground-roll and reflection in transformed domain such as $f-k$ domain, wavelet domain, or curvelet domain. These approaches use a series of fixed atoms or bases to transform the data in time-space domain into transformed domain to separate different waveforms, thus tend to suffer from the complexity for a delicate design of the parameters of the transform domain filter. To deal with these problems, a novel way is proposed to separate ground-roll from reflections using convolutional neural network (CNN) model based method to learn to extract the features of ground-roll and reflections automatically based on training data. In the proposed method, low-pass filtered seismic data which is contaminated by ground-roll wave is used as input of CNN, and then outputs both ground-roll component and low-frequency part of reflection component simultaneously. Discriminative loss is applied together with similarity loss in the training process to enhance the similarity to their train labels as well as the difference between the two outputs. Experiments are conducted on both synthetic and real data, showing that CNN based method can separate ground roll from reflections effectively, and has generalization ability to a certain extent.

Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms along with their optimization analyses have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention. This lack of investigation can be attributed to the complex interplay between data heterogeneity and infrequent communication due to the local updates within the FL framework. This motivates us to investigate a fundamental question in FL: Can we quantify the impact of data heterogeneity and local updates on the generalization performance for FL as the learning process evolves? To this end, we conduct a comprehensive theoretical study of FL's generalization performance using a linear model as the first step, where the data heterogeneity is considered for both the stationary and online/non-stationary cases. By providing closed-form expressions of the model error, we rigorously quantify the impact of the number of the local updates (denoted as $K$) under three settings ($K=1$, $K<\infty$, and $K=\infty$) and show how the generalization performance evolves with the number of rounds $t$. Our investigation also provides a comprehensive understanding of how different configurations (including the number of model parameters $p$ and the number of training samples $n$) contribute to the overall generalization performance, thus shedding new insights (such as benign overfitting) for implementing FL over networks., Comment: Published in MobiHoc 2024

We report new pion electroproduction measurements in the $\Delta(1232)$ resonance, utilizing the SHMS - HMS magnetic spectrometers of Hall C at Jefferson Lab. The data focus on a region that exhibits a strong and rapidly changing interplay of the mesonic cloud and quark-gluon dynamics in the nucleon. The results are in reasonable agreement with models that employ pion cloud effects and chiral effective field theory calculations, but at the same time they suggest that an improvement is required to the theoretical calculations and provide valuable input that will allow their refinements. The data illustrate the potential of the magnetic spectrometers setup in Hall C towards the study the $\Delta(1232)$ resonance. These first reported results will be followed by a series of measurements in Hall C, that will expand the studies of the $\Delta(1232)$ resonance offering a high precision insight within a wide kinematic range from low to high momentum transfers.

The interplay between topology and soliton is a central topic in nonlinear topological physics. So far, most studies have been confined to conservative settings. Here, we explore Thouless pumping of dissipative temporal solitons in a nonconservative one-dimensional optical system with gain and spectral filtering, described by the paradigmatic complex Ginzburg-Landau equation. Two dissipatively induced nonlinear topological phase transitions are identified. First, when varying dissipative parameters across a threshold, the soliton transitions from being trapped in time to quantized drifting. This quantized temporal drift remains robust, even as the system evolves from a single-soliton state into multi-soliton state. Second, a dynamically emergent phase transition is found: the soliton is arrested until a critical point of its evolution, where a transition to topological drift occurs. Both phenomena uniquely arise from the dynamical interplay of dissipation, nonlinearity and topology., Comment: 6 pages, 3 figures

The human skin exhibits remarkable capability to perceive contact forces and environmental temperatures, providing intricate information essential for nuanced manipulation. Despite recent advancements in soft tactile sensors, a significant challenge remains in accurately decoupling signals - specifically, separating force from directional orientation and temperature - resulting in fail to meet the advanced application requirements of robots. This research proposes a multi-layered soft sensor unit (F3T) designed to achieve isolated measurements and mathematical decoupling of normal pressure, omnidirectional tangential forces, and temperature. We developed a circular coaxial magnetic film featuring a floating-mountain multi-layer capacitor, facilitating the physical decoupling of normal and tangential forces in all directions. Additionally, we incorporated an ion gel-based temperature sensing film atop the tactile sensor. This sensor is resilient to external pressure and deformation, enabling it to measure temperature and, crucially, eliminate capacitor errors induced by environmental temperature changes. This innovative design allows for the decoupled measurement of multiple signals, paving the way for advancements in higher-level robot motion control, autonomous decision-making, and task planning.

In this paper we prove the following Willmore-type inequality: On an unbounded closed convex set $K\subset\mathbb{R}^{n+1}$ $(n\ge 2)$, for any embedded hypersurface $\Sigma\subset K$ with boundary $\partial\Sigma\subset \partial K$ satisfying certain contact angle condition, there holds $$\frac1{n+1}\int_{\Sigma}\vert{H}\vert^n{\rm d}A\ge{\rm AVR}(K)\vert\mathbb{B}^{n+1}\vert.$$ Moreover, equality holds if and only if $\Sigma$ is a part of a sphere and $K\setminus\Omega$ is a part of the solid cone determined by $\Sigma$. Here $\Omega$ is the bounded domain enclosed by $\Sigma$ and $\partial K$, $H$ is the normalized mean curvature of $\Sigma$, and ${\rm AVR}(K)$ is the asymptotic volume ratio of $K$. We also prove an anisotropic version of this Willmore-type inequality. As a special case, we obtain a Willmore-type inequality for anisotropic capillary hypersurfaces in a half-space.

Progress in gravitational-wave astronomy depends upon having sensitive detectors with good data quality. Since the end of the LIGO-Virgo-KAGRA third Observing run in March 2020, detector-characterization efforts have lead to increased sensitivity of the detectors, swifter validation of gravitational-wave candidates and improved tools used for data-quality products. In this article, we discuss these efforts in detail and their impact on our ability to detect and study gravitational-waves. These include the multiple instrumental investigations that led to reduction in transient noise, along with the work to improve software tools used to examine the detectors data-quality. We end with a brief discussion on the role and requirements of detector characterization as the sensitivity of our detectors further improves in the future Observing runs., Comment: 35 pages, 18 figures

The $CP$ violation (CPV) in the baryon system has not yet been definitively established. We demonstrate that individual partial-wave CPV in the $\Lambda_b\to p\pi^-,pK^-$ decays can exceed $10\%$, but the destruction between different partial waves results in small net direct CPV as observed in current experiments. There is thus high possibility of identifying CPV in $b$-baryon decays through measurements of partial-wave CPV. The above observation is supported by the first full QCD calculation of two-body hadronic $\Lambda_b$ baryon decays with controllable uncertainties in the perturbative QCD formalism., Comment: 5 pages, 2 figures,4 tables

At the dynamic nexus of artificial intelligence and quantum technology, quantum neural networks (QNNs) play an important role as an emerging technology in the rapidly developing field of quantum machine learning. This development is set to revolutionize the applications of quantum computing. This article reviews the concept of QNNs and their physical realizations, particularly implementations based on quantum optics . We first examine the integration of quantum principles with classical neural network architectures to create QNNs. Some specific examples, such as the quantum perceptron, quantum convolutional neural networks, and quantum Boltzmann machines are discussed. Subsequently, we analyze the feasibility of implementing QNNs through photonics. The key challenge here lies in achieving the required non-linear gates, and measurement-induced approaches, among others, seem promising. To unlock the computational potential of QNNs, addressing the challenge of scaling their complexity through quantum optics is crucial. Progress in controlling quantum states of light is continuously advancing the field. Additionally, we have discovered that different QNN architectures can be unified through non-Gaussian operations. This insight will aid in better understanding and developing more complex QNN circuits.

The echo signals of frequency diverse array multiple-input multiple-output (FDA-MIMO) feature angle-range coupling, enabling simultaneous discrimination and estimation of multiple targets at different locations. In light of this, based on FDA-MIMO, this paper explores an sensing-centric integrated sensing and communication (ISAC) system for multi-target sensing. On the transmitter side, the complex coefficients index modulation (CCIM) scheme is designed, which carries extra bits by selecting complex coefficients from the coefficient vector. At the sensing receiver, we propose the FDA-MIMO-based spatial spectrum multi-target estimation (SSMTE) method, which first jointly estimates the angle and distance of targets and then estimates the velocities. To reduce the sensing computational complexity, the low-complexity spatial spectrum estimation (LCSSE) algorithm is proposed. LCSSE reduces the complexity without degrading the sensing performance by converting the joint angle-range search into two one-dimensional searches. To address the range ambiguity caused by frequency offset, a frequency offset design criterion (FODC) is proposed. It designs the integer and fractional components of the frequency offset to ensure the ambiguity distance exceeds the maximum sensing range, thereby alleviating parameters pairing errors. Moreover, the closed-form expressions for the bit error rate (BER) tight upper bound and the Cram\'er-Rao bound (CRB) are derived. Simulation results show that the proposed system excels in multi-target sensing and communications.

This technical report outlines our submission system for the CHiME-8 NOTSOFAR-1 Challenge. The primary difficulty of this challenge is the dataset recorded across various conference rooms, which captures real-world complexities such as high overlap rates, background noises, a variable number of speakers, and natural conversation styles. To address these issues, we optimized the system in several aspects: For front-end speech signal processing, we introduced a data-driven joint training method for diarization and separation (JDS) to enhance audio quality. Additionally, we also integrated traditional guided source separation (GSS) for multi-channel track to provide complementary information for the JDS. For back-end speech recognition, we enhanced Whisper with WavLM, ConvNeXt, and Transformer innovations, applying multi-task training and Noise KLD augmentation, to significantly advance ASR robustness and accuracy. Our system attained a Time-Constrained minimum Permutation Word Error Rate (tcpWER) of 14.265% and 22.989% on the CHiME-8 NOTSOFAR-1 Dev-set-2 multi-channel and single-channel tracks, respectively.

The rapid advancement of drone technology has significantly impacted various sectors, including search and rescue, environmental surveillance, and industrial inspection. Multidrone systems offer notable advantages such as enhanced efficiency, scalability, and redundancy over single-drone operations. Despite these benefits, ensuring resilient formation control in dynamic and adversarial environments, such as under communication loss or cyberattacks, remains a significant challenge. Classical approaches to resilient formation control, while effective in certain scenarios, often struggle with complex modeling and the curse of dimensionality, particularly as the number of agents increases. This paper proposes a novel, learning-based formation control for enhancing the adaptability and resilience of multidrone formations using graph attention networks (GATs). By leveraging GAT's dynamic capabilities to extract internode relationships based on the attention mechanism, this GAT-based formation controller significantly improves the robustness of drone formations against various threats, such as Denial of Service (DoS) attacks. Our approach not only improves formation performance in normal conditions but also ensures the resilience of multidrone systems in variable and adversarial environments. Extensive simulation results demonstrate the superior performance of our method over baseline formation controllers. Furthermore, the physical experiments validate the effectiveness of the trained control policy in real-world flights., Comment: This work has been submitted to the IEEE for possible publication

The success of Large Language Models (LLMs) relies heavily on the huge amount of pre-training data learned in the pre-training phase. The opacity of the pre-training process and the training data causes the results of many benchmark tests to become unreliable. If any model has been trained on a benchmark test set, it can seriously hinder the health of the field. In order to automate and efficiently test the capabilities of large language models, numerous mainstream benchmarks adopt a multiple-choice format. As the swapping of the contents of multiple-choice options does not affect the meaning of the question itself, we propose a simple and effective data leakage detection method based on this property. Specifically, we shuffle the contents of the options in the data to generate the corresponding derived data sets, and then detect data leakage based on the model's log probability distribution over the derived data sets. If there is a maximum and outlier in the set of log probabilities, it indicates that the data is leaked. Our method is able to work under black-box conditions without access to model training data or weights, effectively identifying data leakage from benchmark test sets in model pre-training data, including both normal scenarios and complex scenarios where options may have been shuffled intentionally or unintentionally. Through experiments based on two LLMs and benchmark designs, we demonstrate the effectiveness of our method. In addition, we evaluate the degree of data leakage of 31 mainstream open-source LLMs on four benchmark datasets and give a ranking of the leaked LLMs for each benchmark, and we find that the Qwen family of LLMs has the highest degree of data leakage.

Traditional OCR systems (OCR-1.0) are increasingly unable to meet people's usage due to the growing demand for intelligent processing of man-made optical characters. In this paper, we collectively refer to all artificial optical signals (e.g., plain texts, math/molecular formulas, tables, charts, sheet music, and even geometric shapes) as "characters" and propose the General OCR Theory along with an excellent model, namely GOT, to promote the arrival of OCR-2.0. The GOT, with 580M parameters, is a unified, elegant, and end-to-end model, consisting of a high-compression encoder and a long-contexts decoder. As an OCR-2.0 model, GOT can handle all the above "characters" under various OCR tasks. On the input side, the model supports commonly used scene- and document-style images in slice and whole-page styles. On the output side, GOT can generate plain or formatted results (markdown/tikz/smiles/kern) via an easy prompt. Besides, the model enjoys interactive OCR features, i.e., region-level recognition guided by coordinates or colors. Furthermore, we also adapt dynamic resolution and multi-page OCR technologies to GOT for better practicality. In experiments, we provide sufficient results to prove the superiority of our model.

One-shot voice conversion(VC) aims to change the timbre of any source speech to match that of the target speaker with only one speech sample. Existing style transfer-based VC methods relied on speech representation disentanglement and suffered from accurately and independently encoding each speech component and recomposing back to converted speech effectively. To tackle this, we proposed Pureformer-VC, which utilizes Conformer blocks to build a disentangled encoder, and Zipformer blocks to build a style transfer decoder as the generator. In the decoder, we used effective styleformer blocks to integrate speaker characteristics effectively into the generated speech. The models used the generative VAE loss for encoding components and triplet loss for unsupervised discriminative training. We applied the styleformer method to Zipformer's shared weights for style transfer. The experimental results show that the proposed model achieves comparable subjective scores and exhibits improvements in objective metrics compared to existing methods in a one-shot voice conversion scenario., Comment: submmited to ICASSP 2025

Generating high-fidelity, temporally consistent videos in autonomous driving scenarios faces a significant challenge, e.g. problematic maneuvers in corner cases. Despite recent video generation works are proposed to tackcle the mentioned problem, i.e. models built on top of Diffusion Transformers (DiT), works are still missing which are targeted on exploring the potential for multi-view videos generation scenarios. Noticeably, we propose the first DiT-based framework specifically designed for generating temporally and multi-view consistent videos which precisely match the given bird's-eye view layouts control. Specifically, the proposed framework leverages a parameter-free spatial view-inflated attention mechanism to guarantee the cross-view consistency, where joint cross-attention modules and ControlNet-Transformer are integrated to further improve the precision of control. To demonstrate our advantages, we extensively investigate the qualitative comparisons on nuScenes dataset, particularly in some most challenging corner cases. In summary, the effectiveness of our proposed method in producing long, controllable, and highly consistent videos under difficult conditions is proven to be effective.

In recent years, significant academic advancements have been made in the field of autonomous vehicles, with Local maps emerging as a crucial component of autonomous driving technology. Local maps not only provide intricate details of road networks but also serve as fundamental inputs for critical tasks such as vehicle localization, navigation, and decision-making. Given the characteristics of SD map (Standard Definition Map), which include low cost, ease of acquisition, and high versatility, perception methods that integrate SD map as prior information have demonstrated significant potential in the field of Local map perception. The purpose of this paper is to provide researchers with a comprehensive overview and summary of the latest advancements in the integration of SD map as prior information for Local map perception methods. This review begins by introducing the task definition and general pipeline of local map perception methods that incorporate SD maps as prior information, along with relevant public datasets. And then it focuses on the representation and encoding methods of multi-source information, as well as the methods for fusing multi-source information. In response to this burgeoning trend, this article presents a comprehensive and meticulous overview of the diverse research efforts in this particular field. Finally, the article addresses pertinent issues and future challenges with the aim of guiding researchers in understanding the current trends and methodologies prevalent in the field., Comment: 14 pages, 11 figures

Situation awareness is essential for understanding and reasoning about 3D scenes in embodied AI agents. However, existing datasets and benchmarks for situated understanding are limited in data modality, diversity, scale, and task scope. To address these limitations, we propose Multi-modal Situated Question Answering (MSQA), a large-scale multi-modal situated reasoning dataset, scalably collected leveraging 3D scene graphs and vision-language models (VLMs) across a diverse range of real-world 3D scenes. MSQA includes 251K situated question-answering pairs across 9 distinct question categories, covering complex scenarios within 3D scenes. We introduce a novel interleaved multi-modal input setting in our benchmark to provide text, image, and point cloud for situation and question description, resolving ambiguity in previous single-modality convention (e.g., text). Additionally, we devise the Multi-modal Situated Next-step Navigation (MSNN) benchmark to evaluate models' situated reasoning for navigation. Comprehensive evaluations on MSQA and MSNN highlight the limitations of existing vision-language models and underscore the importance of handling multi-modal interleaved inputs and situation modeling. Experiments on data scaling and cross-domain transfer further demonstrate the efficacy of leveraging MSQA as a pre-training dataset for developing more powerful situated reasoning models., Comment: Project page: https://msr3d.github.io/

After shipment to the Daresbury Lab and return to Fermilab, the prototype HB650 cryomodule underwent another phase of 2K RF testing to ascertain any performance issues that may have arisen from the transport of the cryomodule. While measurements taken at room temperature after the conclusion of shipment indicated that there were no negative impacts on cavity alignment, beamline vacuum, or cavity frequency, testing at 2K was required to validate other aspects such as tuner operation, cavity coupling, cryogenic system integrity, and cavity performance. Results of this latest round of limited 2K testing will be presented., Comment: 32nd Linear Accelerator Conference (LINAC 2024)

Ensuring timing guarantees for every individual tasks is critical in real-time systems. Even for periodic tasks, providing timing guarantees for tasks with segmented self-suspending behavior is challenging due to timing anomalies, i.e., the reduction of execution or suspension time of some jobs increases the response time of another job. The release jitter of tasks can add further complexity to the situation, affecting the predictability and timing guarantees of real-time systems. The existing worst-case response time analyses for sporadic self-suspending tasks are only over-approximations and lead to overly pessimistic results. In this work, we address timing anomalies without compromising the worst-case response time (WCRT) analysis when scheduling periodic segmented self-suspending tasks with release jitter. We propose two treatments: segment release time enforcement and segment priority modification, and prove their effectiveness in eliminating timing anomalies. Our evaluation demonstrates that the proposed treatments achieve higher acceptance ratios in terms of schedulability compared to state-of-the-art scheduling algorithms. Additionally, we implement the segment-level fixed-priority scheduling mechanism on RTEMS and verify the validity of our segment priority modification treatment. This work expands our previous conference publication at the 29th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2023), which considers only periodic segmented self-suspending tasks without release jitter., Comment: This is an extension from a previous conference publication at the 29th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2023)

We present a robust refinement method for estimating oriented normals from unstructured point clouds. In contrast to previous approaches that either suffer from high computational complexity or fail to achieve desirable accuracy, our novel framework incorporates sign orientation and data augmentation in the feature space to refine the initial oriented normals, striking a balance between efficiency and accuracy. To address the issue of noise-caused direction inconsistency existing in previous approaches, we introduce a new metric called the Chamfer Normal Distance, which faithfully minimizes the estimation error by correcting the annotated normal with the closest point found on the potentially clean point cloud. This metric not only tackles the challenge but also aids in network training and significantly enhances network robustness against noise. Moreover, we propose an innovative dual-parallel architecture that integrates Multi-scale Local Feature Aggregation and Hierarchical Geometric Information Fusion, which enables the network to capture intricate geometric details more effectively and notably reduces ambiguity in scale selection. Extensive experiments demonstrate the superiority and versatility of our method in both unoriented and oriented normal estimation tasks across synthetic and real-world datasets among indoor and outdoor scenarios. The code is available at https://github.com/YingruiWoo/OCMG-Net.git., Comment: 18 pages, 16 figures

The Integrated Process Planning and Scheduling (IPPS) problem combines process route planning and shop scheduling to achieve high efficiency in manufacturing and maximize resource utilization, which is crucial for modern manufacturing systems. Traditional methods using Mixed Integer Linear Programming (MILP) and heuristic algorithms can not well balance solution quality and speed when solving IPPS. In this paper, we propose a novel end-to-end Deep Reinforcement Learning (DRL) method. We model the IPPS problem as a Markov Decision Process (MDP) and employ a Heterogeneous Graph Neural Network (GNN) to capture the complex relationships among operations, machines, and jobs. To optimize the scheduling strategy, we use Proximal Policy Optimization (PPO). Experimental results show that, compared to traditional methods, our approach significantly improves solution efficiency and quality in large-scale IPPS instances, providing superior scheduling strategies for modern intelligent manufacturing systems., Comment: 24 pages, 13 figures

We prove that a $T_0$ topological space is $\omega$-well-filtered if and only if it does not admit either the natural numbers with the cofinite topology or with the Scott topology as its closed subsets in the strong topology. Based on this, we offer a refined topological characterization for the $\omega$-well-filterification of $T_0$-spaces and solve a problem posed by Xiaoquan Xu. In the setting of second countable spaces, we also characterise sobriety by convergences of certain $\Pi^0_2$-Cauchy subsets of the spaces.

The Ho-Zhao problem asks whether any two dcpo's with isomorphic Scott closed set lattices are themselves isomorphic, that is, whether the category $\mathbf{DCPO}$ of dcpo's and Scott-continuous maps is $\Gamma$-faithful. In 2018, Ho, Goubault-Larrecq, Jung and Xi answered this question in the negative, and they introduced the category $\mathbf{DOMI}$ of dominated dcpo's and proved that it is {$\Gamma$-faithful}. Dominated dcpo's subsume many familiar families of dcpo's in domain theory, such as the category of bounded-complete dcpo's and that of sober dcpo's, among others. However, it is unknown whether the category of dominated dcpo's dominates all well-filtered dcpo's, a class strictly larger than that of bounded-complete lattices and that of sober dcpo's. In this paper, we address this very natural question and show that the category $\mathbf{WF}$ of well-filtered dcpo's is not $\Gamma$-faithful, and as a result of it, well-filtered dcpo's need not be dominated in general. Since not all dcpo's are well-filtered, our work refines the results of Ho, Goubault-Larrecq, Jung and Xi. As a second contribution, we confirm that the Lawson's category of $\Omega^{*}$-compact dcpo's is $\Gamma$-faithful. Moreover, we locate a class of dcpo's which we call weakly dominated dcpo's, and show that this class is $\Gamma$-faithful and strictly larger than $\mathbf{DOMI}$.

The non-Hermitian skin effect (NHSE) is a distinctive topological phenomenon observed in nonHermitian systems. Recently, there has been considerable interest in exploring higher-order NHSE occurrences in two and three dimensions. In such systems, topological edge states collapse into a corner while bulk states remain delocalized. Through a Hermitian-non-Hermitian correspondence, this study predicts and experimentally observes the higher-order NHSE in an acoustic Kagome lattice possessing nonreciprocal hoppings. By rotating the frequency spectrum and employing complexfrequency excitation techniques, we observe the localization of acoustic energy towards a corner of the lattice in the topologically nontrivial phase, even when the source is located far from that corner. In contrast, the acoustic energy spreads out when excited at the frequencies hosting the bulk states. These observations are unequivocal evidence of the higher-order NHSE.

We employ the nuclear lattice effective field theory (NLEFT), an efficient tool for nuclear ab initio calculations, to solve the asymmetric multi-hadron systems. We take the $DD^*K$ three-body system as an illustration to demonstrate the capability of the method. Here the two-body chiral interactions between $D$, $D^*$ and $K$ are regulated with a soft lattice regulator and calibrated with the binding energies of the $T_{cc}^+$, $D^{*}_{s0}(2317)$ and $D_{s1}(2460)$ molecular states. We then calculate the three-body binding energy using the NLEFT and analyze the systematic uncertainties due to the finite volume effects, the sliding cutoff and the leading-order three-body forces. Even when the three-body interaction is repulsive (even as large as the infinite repulsive interaction), the three-body system has a bound state unambiguously with binding energy no larger than the $D_{s1}(2460)D$ threshold. To check the renormalization group invariance of our framework, we extract the first excited state. We find that when the ground state is fixed, the first excited states with various cutoffs coincide with each other when the cubic size goes larger. In addition, the standard angular momentum and parity projection technique is implemented for the quantum numbers of the ground and excited states. We find that both of them are S-wave states with quantum number $J^{P}=1^-$. Because the three-body state contains two charm quarks, it is easier to be detected in the Large Hadron Collider., Comment: 17 pages, 6 figures

We present cosmological constraints from weak lensing with the Subaru Hyper Suprime-Cam (HSC) first-year (Y1) data, using a simulation-based inference (SBI) method. % We explore the performance of a set of higher-order statistics (HOS) including the Minkowski functionals, counts of peaks and minima, and the probability distribution function and compare them to the traditional two-point statistics. The HOS, also known as non-Gaussian statistics, can extract additional non-Gaussian information that is inaccessible to the two-point statistics. We use a neural network to compress the summary statistics, followed by an SBI approach to infer the posterior distribution of the cosmological parameters. We apply cuts on angular scales and redshift bins to mitigate the impact of systematic effects. Combining two-point and non-Gaussian statistics, we obtain $S_8 \equiv \sigma_8 \sqrt{\Omega_m/0.3} = 0.804_{-0.040}^{+0.041}$ and $\Omega_m = 0.344_{-0.090}^{+0.083}$, similar to that from non-Gaussian statistics alone. These results are consistent with previous HSC analyses and Planck 2018 cosmology. Our constraints from non-Gaussian statistics are $\sim 25\%$ tighter in $S_8$ than two-point statistics, where the main improvement lies in $\Omega_m$, with $\sim 40$\% tighter error bar compared to using the angular power spectrum alone ($S_8 = 0.766_{-0.056}^{+0.054}$ and $\Omega_m = 0.365_{-0.141}^{+0.148}$). We find that, among the non-Gaussian statistics we studied, the Minkowski functionals are the primary driver for this improvement. Our analyses confirm the SBI as a powerful approach for cosmological constraints, avoiding any assumptions about the functional form of the data's likelihood., Comment: 14 pages, 7 figures

We propose a scheme to generate low driving threshold quantum correlations in Brillouin optomechanical system based on synthetic magnetism. Our proposal consists of a mechanical (acoustic) resonator coupled to two optical modes through the standard optomechanical radiation pressure (an electrostrictive force). The electrostrictive force that couples the acoustic mode to the optical ones striggers Backward Stimulated Brillouin Scattering (BSBS) process in the system. Moreover, the mechanical and acoustic resonators are mechanically coupled through the coupling rate $J_m$, which is $\theta$-phase modulated. Without a mechanical coupling, the generated quantum correlations require a strong driving field. By accounting phonon hopping coupling, the synthetic magnetism is induced and the quantum correlations are generated for low coupling strengths. The generated quantum correlations display sudden death and revival phenonmena, and are robust against thermal noise. Our results suggest a way for low threshold quantum correlations generation, and are useful for quantum communications, quantum sensors, and quantum computational tasks., Comment: 10 pages, 7 figures. Comments are welcome!

UHD images, typically with resolutions equal to or higher than 4K, pose a significant challenge for efficient image quality assessment (IQA) algorithms, as adopting full-resolution images as inputs leads to overwhelming computational complexity and commonly used pre-processing methods like resizing or cropping may cause substantial loss of detail. To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception. Specifically, aesthetic features are extracted from low-resolution images downsampled from the UHD ones, which lose high-frequency texture information but still preserve the global aesthetics characteristics. Technical distortions are measured using a fragment image composed of mini-patches cropped from UHD images based on the grid mini-patch sampling strategy. The salient content of UHD images is detected and cropped to extract quality-aware features from the salient regions. We adopt the Swin Transformer Tiny as the backbone networks to extract features from these three perspectives. The extracted features are concatenated and regressed into quality scores by a two-layer multi-layer perceptron (MLP) network. We employ the mean square error (MSE) loss to optimize prediction accuracy and the fidelity loss to optimize prediction monotonicity. Experimental results show that the proposed model achieves the best performance on the UHD-IQA dataset while maintaining the lowest computational complexity, demonstrating its effectiveness and efficiency. Moreover, the proposed model won first prize in ECCV AIM 2024 UHD-IQA Challenge. The code is available at https://github.com/sunwei925/UIQA., Comment: The proposed model won first prize in ECCV AIM 2024 Pushing the Boundaries of Blind Photo Quality Assessment Challenge

Ultra-Wide-Field Fluorescein Angiography (UWF-FA) enables precise identification of ocular diseases using sodium fluorescein, which can be potentially harmful. Existing research has developed methods to generate UWF-FA from Ultra-Wide-Field Scanning Laser Ophthalmoscopy (UWF-SLO) to reduce the adverse reactions associated with injections. However, these methods have been less effective in producing high-quality late-phase UWF-FA, particularly in lesion areas and fine details. Two primary challenges hinder the generation of high-quality late-phase UWF-FA: the scarcity of paired UWF-SLO and early/late-phase UWF-FA datasets, and the need for realistic generation at lesion sites and potential blood leakage regions. This study introduces an improved latent diffusion model framework to generate high-quality late-phase UWF-FA from limited paired UWF images. To address the challenges as mentioned earlier, our approach employs a module utilizing Cross-temporal Regional Difference Loss, which encourages the model to focus on the differences between early and late phases. Additionally, we introduce a low-frequency enhanced noise strategy in the diffusion forward process to improve the realism of medical images. To further enhance the mapping capability of the variational autoencoder module, especially with limited datasets, we implement a Gated Convolutional Encoder to extract additional information from conditional images. Our Latent Diffusion Model for Ultra-Wide-Field Late-Phase Fluorescein Angiography (LPUWF-LDM) effectively reconstructs fine details in late-phase UWF-FA and achieves state-of-the-art results compared to other existing methods when working with limited datasets. Our source code is available at: https://github.com/Tinysqua/****., Comment: 13 pages, 7 figures

The research of the properties of neutron stars with dark energy is a particularly interesting yet unresolved problem in astrophysics. We analyze the influence of dark energy on the equation of state, the maximum mass, the surface gravitational redshift, and the Keplerian frequency for the traditional neutron star and the hyperon star matter within the relativistic mean field theory, using the GM1 and TM1 parameter sets by considering the two flavor symmetries of SU(6) and SU(3) combined with the observations of PSR J1614-2230, PSR J0348+0432, PSR J0030+0451, RX J0720.4-3125, and 1E 1207.4-5209. It is found that the existence of dark energy leads to the softened equations of state of the traditional neutron star and the hyperon star. The radius of a fixed-mass traditional neutron star (or hyperon star) with dark energy becomes smaller, which leads to increased compactness. The existence of dark energy can also enhance the surface gravitational redshift and the Keplerian frequency of the traditional neutron stars and the hyperon stars. The growth of the Keplerian frequency may cause speeding up of the spin rate, which may provide a possible way to understand and explain the pulsar glitch phenomenon. Specifically, we infer that the mass and the surface gravitational redshift of PSR J1748-2446ad without dark energy for the GM1 (TM1) parameter set are 1.141 $M_\odot$ (1.309 $M_\odot$) and 0.095 (0.105), respectively. The corresponding values for the GM1 (TM1) parameter set are 0.901 $M_\odot$ (1.072 $M_\odot$) and 0.079 (0.091) if PSR J1748-2446ad contains dark energy with $\alpha=0.05$. PSR J1748-2446ad may be a low-mass pulsar with a lower surface gravitational redshift under our selected models., Comment: 19 pages,8 figures, submitted to Commun. Theor. Phys

Backdoor attacks, in which a model behaves maliciously when given an attacker-specified trigger, pose a major security risk for practitioners who depend on publicly released language models. Backdoor detection methods aim to detect whether a released model contains a backdoor, so that practitioners can avoid such vulnerabilities. While existing backdoor detection methods have high accuracy in detecting backdoored models on standard benchmarks, it is unclear whether they can robustly identify backdoors in the wild. In this paper, we examine the robustness of backdoor detectors by manipulating different factors during backdoor planting. We find that the success of existing methods highly depends on how intensely the model is trained on poisoned data during backdoor planting. Specifically, backdoors planted with either more aggressive or more conservative training are significantly more difficult to detect than the default ones. Our results highlight a lack of robustness of existing backdoor detectors and the limitations in current benchmark construction.

Existing research often treats long-form videos as extended short videos, leading to several limitations: inadequate capture of long-range dependencies, inefficient processing of redundant information, and failure to extract high-level semantic concepts. To address these issues, we propose a novel approach that more accurately reflects human cognition. This paper introduces HERMES: temporal-coHERent long-forM understanding with Episodes and Semantics, a model that simulates episodic memory accumulation to capture action sequences and reinforces them with semantic knowledge dispersed throughout the video. Our work makes two key contributions: First, we develop an Episodic COmpressor (ECO) that efficiently aggregates crucial representations from micro to semi-macro levels, overcoming the challenge of long-range dependencies. Second, we propose a Semantics ReTRiever (SeTR) that enhances these aggregated representations with semantic information by focusing on the broader context, dramatically reducing feature dimensionality while preserving relevant macro-level information. This addresses the issues of redundancy and lack of high-level concept extraction. Extensive experiments demonstrate that HERMES achieves state-of-the-art performance across multiple long-video understanding benchmarks in both zero-shot and fully-supervised settings., Comment: This is an improved and expanded version of our EVAL-FoMo Workshop at ECCV'24 (v1 of this paper). Project page: https://joslefaure.github.io/assets/html/hermes.html

Earth's magnetotail, a night-side region characterized by stretched magnetic field lines and strong plasma currents, is the primary site for the release of magnetic field energy and its transformation into plasma heating and kinetic energy plus charged particle acceleration during magnetic reconnection. In this study, we demonstrate that the efficiency of this acceleration can be sufficiently high to produce populations of relativistic and ultra-relativistic electrons, with energies up to several MeV, which exceeds all previous theoretical and simulation estimates. Using data from the low altitude ELFIN and CIRBE CubeSats, we show multiple events of relativistic electron bursts within the magnetotail, far poleward of the outer radiation belt. These bursts are characterized by power-law energy spectra and can be detected during even moderate substorms.

Parsing of eye components (i.e. pupil, iris and sclera) is fundamental for eye tracking and gaze estimation for AR/VR products. Mainstream approaches tackle this problem as a multi-class segmentation task, providing only visible part of pupil/iris, other methods regress elliptical parameters using human-annotated full pupil/iris parameters. In this paper, we consider two priors: projected full pupil/iris circle can be modelled with ellipses (ellipse prior), and the visibility of pupil/iris is controlled by openness of eye-region (condition prior), and design a novel method CondSeg to estimate elliptical parameters of pupil/iris directly from segmentation labels, without explicitly annotating full ellipses, and use eye-region mask to control the visibility of estimated pupil/iris ellipses. Conditioned segmentation loss is used to optimize the parameters by transforming parameterized ellipses into pixel-wise soft masks in a differentiable way. Our method is tested on public datasets (OpenEDS-2019/-2020) and shows competitive results on segmentation metrics, and provides accurate elliptical parameters for further applications of eye tracking simultaneously.

Recently, visual grounding and multi-sensors setting have been incorporated into perception system for terrestrial autonomous driving systems and Unmanned Surface Vehicles (USVs), yet the high complexity of modern learning-based visual grounding model using multi-sensors prevents such model to be deployed on USVs in the real-life. To this end, we design a low-power multi-task model named NanoMVG for waterway embodied perception, guiding both camera and 4D millimeter-wave radar to locate specific object(s) through natural language. NanoMVG can perform both box-level and mask-level visual grounding tasks simultaneously. Compared to other visual grounding models, NanoMVG achieves highly competitive performance on the WaterVG dataset, particularly in harsh environments and boasts ultra-low power consumption for long endurance., Comment: 8 pages, 6 figures