Search

Investigating the signals of dark matter annihilation is one of the most popular ways to understand the nature of dark matter. In particular, many recent studies are focussing on using radio data to examine the possible signals of dark matter revealed in galaxies and galaxy clusters. In this article, we investigate on the spectral data of the central radio halo of the cool-core cluster RX J1720.1+2638. We show that the radio spectral data can be best accounted by the synchrotron emission due to dark matter annihilation via $\tau$ lepton channel (with dark matter mass $m=15$ GeV) or $b$ quark channel (with dark matter mass $m=110$ GeV), although using the very coarse spectral data with notable errors. Despite the fact that cosmic-ray emission can also provide a good explanation for the observed radio spectrum, our results suggest a possible positive evidence for dark matter annihilation revealed in the form of radio emission in RX J1720.1+2638 cluster., Comment: Accepted in MNRAS

Spatiotemporal optical vortices (STOVs), as a kind of structured light pulses carrying transverse orbital angular momentum (OAM), have recently attracted significant research interest due to their unique photonic properties. However, general STOV pulses typically exhibit an annular intensity profile in the spatiotemporal plane, with a radius that scales with the topological charge, limiting their potential in many applications. Here, to address this limitation, we introduce the concept of perfect spatiotemporal optical vortices (PSTOVs). Unlike STOV pulses, the intensity distribution of PSTOV wavepackets is nearly independent of the topological charge. We show that such wavepackets can be generated by applying the spatiotemporal Fourier transform to a Bessel-Gaussian mode in the spatiotemporal frequency domain. More importantly, the mode distribution of PSTOV wavepackets can be freely controlled by introducing azimuthal-dependent phase modulation, enabling conversion from a standard annular profile to arbitrary polygonal shapes. Finally, experimental results confirm the successful generation of these wavepackets. Our findings will expand the study of STOV pulses and explore their potential applications in optical communications, information processing, topological photonics, and ultrafast control of light-matter interactions.

Semantic segmentation plays a crucial role in remote sensing applications, where the accurate extraction and representation of features are essential for high-quality results. Despite the widespread use of encoder-decoder architectures, existing methods often struggle with fully utilizing the high-dimensional features extracted by the encoder and efficiently recovering detailed information during decoding. To address these problems, we propose a novel semantic segmentation network, namely DeepKANSeg, including two key innovations based on the emerging Kolmogorov Arnold Network (KAN). Notably, the advantage of KAN lies in its ability to decompose high-dimensional complex functions into univariate transformations, enabling efficient and flexible representation of intricate relationships in data. First, we introduce a KAN-based deep feature refinement module, namely DeepKAN to effectively capture complex spatial and rich semantic relationships from high-dimensional features. Second, we replace the traditional multi-layer perceptron (MLP) layers in the global-local combined decoder with KAN-based linear layers, namely GLKAN. This module enhances the decoder's ability to capture fine-grained details during decoding. To evaluate the effectiveness of the proposed method, experiments are conducted on two well-known fine-resolution remote sensing benchmark datasets, namely ISPRS Vaihingen and ISPRS Potsdam. The results demonstrate that the KAN-enhanced segmentation model achieves superior performance in terms of accuracy compared to state-of-the-art methods. They highlight the potential of KANs as a powerful alternative to traditional architectures in semantic segmentation tasks. Moreover, the explicit univariate decomposition provides improved interpretability, which is particularly beneficial for applications requiring explainable learning in remote sensing., Comment: 13 pages, 8 figures

We investigate nuclear modification and the universality of short-range correlation (SRC) in pion-induced Drell-Yan process. Employing nuclear parton distribution functions (nPDFs) and pion PDFs, the ratio of differential cross sections of different nuclei relative to the free nucleon is presented. A kind of universal modification function was proposed which would provide nontrivial tests of SRC universality on the platform of pion-induced Drell-Yan. This work improves our understanding of nuclear structure and strong interactions., Comment: 6 pages, 3 figures

Nonconvex-concave (NC-C) finite-sum minimax problems have broad applications in decentralized optimization and various machine learning tasks. However, the nonsmooth nature of NC-C problems makes it challenging to design effective variance reduction techniques. Existing vanilla stochastic algorithms using uniform samples for gradient estimation often exhibit slow convergence rates and require bounded variance assumptions. In this paper, we develop a novel probabilistic variance reduction updating scheme and propose a single-loop algorithm called the probabilistic variance-reduced smoothed gradient descent-ascent (PVR-SGDA) algorithm. The proposed algorithm achieves an iteration complexity of $O(\epsilon^{-4})$, surpassing the best-known rates of stochastic algorithms for NC-C minimax problems and matching the performance of the best deterministic algorithms in this context. Finally, we demonstrate the effectiveness of the proposed algorithm through numerical simulations., Comment: The conference version of this paper has been accepted by ICASSP 2025

In this paper, we study self-normalized moderate deviations for degenerate { $U$}-statistics of order $2$. Let $\{X_i, i \geq 1\}$ be i.i.d. random variables and consider symmetric and degenerate kernel functions in the form $h(x,y)=\sum_{l=1}^{\infty} \lambda_l g_l (x) g_l(y)$, where $\lambda_l > 0$, $E g_l(X_1)=0$, and $g_l (X_1)$ is in the domain of attraction of a normal law for all $l \geq 1$. Under the condition $\sum_{l=1}^{\infty}\lambda_l<\infty$ and some truncated conditions for $\{g_l(X_1): l \geq 1\}$, we show that $ \text{log} P({\frac{\sum_{1 \leq i \neq j \leq n}h(X_{i}, X_{j})} {\max_{1\le l<\infty}\lambda_l V^2_{n,l} }} \geq x_n^2) \sim - { \frac {x_n^2}{ 2}}$ for $x_n \to \infty$ and $x_n =o(\sqrt{n})$, where $V^2_{n,l}=\sum_{i=1}^n g_l^2(X_i)$. As application, a law of the iterated logarithm is also obtained., Comment: 32 pages

Previous research has shown that the principal singular vectors of a pre-trained model's weight matrices capture critical knowledge. In contrast, those associated with small singular values may contain noise or less reliable information. As a result, the LoRA-based parameter-efficient fine-tuning (PEFT) approach, which does not constrain the use of the spectral space, may not be effective for tasks that demand high representation capacity. In this study, we enhance existing PEFT techniques by incorporating the spectral information of pre-trained weight matrices into the fine-tuning process. We investigate spectral adaptation strategies with a particular focus on the additive adjustment of top singular vectors. This is accomplished by applying singular value decomposition (SVD) to the pre-trained weight matrices and restricting the fine-tuning within the top spectral space. Extensive speaker verification experiments on VoxCeleb1 and CN-Celeb1 demonstrate enhanced tuning performance with the proposed approach. Code is released at https://github.com/lizhepolyu/SpectralFT., Comment: Accepted by ICASSP 2025

While ViTs have achieved across machine learning tasks, deploying them in real-world scenarios faces a critical challenge: generalizing under OoD shifts. A crucial research gap exists in understanding how to design ViT architectures, both manually and automatically, for better OoD generalization. To this end, we introduce OoD-ViT-NAS, the first systematic benchmark for ViTs NAS focused on OoD generalization. This benchmark includes 3000 ViT architectures of varying computational budgets evaluated on 8 common OoD datasets. Using this benchmark, we analyze factors contributing to OoD generalization. Our findings reveal key insights. First, ViT architecture designs significantly affect OoD generalization. Second, ID accuracy is often a poor indicator of OoD accuracy, highlighting the risk of optimizing ViT architectures solely for ID performance. Third, we perform the first study of NAS for ViTs OoD robustness, analyzing 9 Training-free NAS methods. We find that existing Training-free NAS methods are largely ineffective in predicting OoD accuracy despite excelling at ID accuracy. Simple proxies like Param or Flop surprisingly outperform complex Training-free NAS methods in predicting OoD accuracy. Finally, we study how ViT architectural attributes impact OoD generalization and discover that increasing embedding dimensions generally enhances performance. Our benchmark shows that ViT architectures exhibit a wide range of OoD accuracy, with up to 11.85% improvement for some OoD shifts. This underscores the importance of studying ViT architecture design for OoD. We believe OoD-ViT-NAS can catalyze further research into how ViT designs influence OoD generalization., Comment: Accepted in NeurIPS 2024

Early detection of neurocognitive disorders (NCDs) is crucial for timely intervention and disease management. Speech analysis offers a non-intrusive and scalable screening method, particularly through narrative tasks in neuropsychological assessment tools. Traditional narrative analysis often focuses on local indicators in microstructure, such as word usage and syntax. While these features provide insights into language production abilities, they often fail to capture global narrative patterns, or microstructures. Macrostructures include coherence, thematic organization, and logical progressions, reflecting essential cognitive skills potentially critical for recognizing NCDs. Addressing this gap, we propose to investigate specific cognitive and linguistic challenges by analyzing topical shifts, temporal dynamics, and the coherence of narratives over time, aiming to reveal cognitive deficits by identifying narrative impairments, and exploring their impact on communication and cognition. The investigation is based on the CU-MARVEL Rabbit Story corpus, which comprises recordings of a story-telling task from 758 older adults. We developed two approaches: the Dynamic Topic Models (DTM)-based temporal analysis to examine the evolution of topics over time, and the Text-Image Temporal Alignment Network (TITAN) to evaluate the coherence between spoken narratives and visual stimuli. DTM-based approach validated the effectiveness of dynamic topic consistency as a macrostructural metric (F1=0.61, AUC=0.78). The TITAN approach achieved the highest performance (F1=0.72, AUC=0.81), surpassing established microstructural and macrostructural feature sets. Cross-comparison and regression tasks further demonstrated the effectiveness of proposed dynamic macrostructural modeling approaches for NCD detection., Comment: 12 pages, 8 figures

The effective frequency of a mechanical resonator can be tuned via the spring effect induced by quadratic optomechanical (QOM) coupling, and both spontaneous symmetry breaking and anti-parity-time phase transition were predicted in the QOM systems. Here, we show that the mechanical susceptibility can be enhanced significantly by driving the QOM system with a strong external optical field, and divergence will happen as the driving strength approaches the critical point (CP) for spontaneous symmetry breaking. Based on the CP, we propose a highly sensitive temperature sensor with a mechanical resonator quadratically coupled to an optical mode. We find that the sensitivity of the temperature sensor can be enhanced by several orders of magnitude as the driving strength approaches the CP, and the sensitivity of the temperature sensor remains high in the low-temperature limit. Our work provides an effective way to realize highly sensitive temperature sensing at ultra-low temperature in the QOM systems., Comment: 9 pages, 4 figures

We study the basic computational problem of detecting approximate stationary points for continuous piecewise affine (PA) functions. Our contributions span multiple aspects, including complexity, regularity, and algorithms. Specifically, we show that testing first-order approximate stationarity concepts, as defined by commonly used generalized subdifferentials, is computationally intractable unless P=NP. To facilitate computability, we consider a polynomial-time solvable relaxation by abusing the convex subdifferential sum rule and establish a tight characterization of its exactness. Furthermore, addressing an open issue motivated by the need to terminate the subgradient method in finite time, we introduce the first oracle-polynomial-time algorithm to detect so-called near-approximate stationary points for PA functions. A notable byproduct of our development in regularity is the first necessary and sufficient condition for the validity of an equality-type (Clarke) subdifferential sum rule. Our techniques revolve around two new geometric notions for convex polytopes and may be of independent interest in nonsmooth analysis. Moreover, some corollaries of our work on complexity and algorithms for stationarity testing address open questions in the literature. To demonstrate the versatility of our results, we complement our findings with applications to a series of structured piecewise smooth functions, including $\rho$-margin-loss SVM, piecewise affine regression, and nonsmooth neural networks., Comment: 77 pages, 6 figures. This work supersedes arXiv:2302.12261 and will be presented in part at SODA 2025

Underwater imaging grapples with challenges from light-water interactions, leading to color distortions and reduced clarity. In response to these challenges, we propose a novel Color Balance Prior \textbf{Guided} \textbf{Hyb}rid \textbf{Sens}e \textbf{U}nderwater \textbf{I}mage \textbf{R}estoration framework (\textbf{GuidedHybSensUIR}). This framework operates on multiple scales, employing the proposed \textbf{Detail Restorer} module to restore low-level detailed features at finer scales and utilizing the proposed \textbf{Feature Contextualizer} module to capture long-range contextual relations of high-level general features at a broader scale. The hybridization of these different scales of sensing results effectively addresses color casts and restores blurry details. In order to effectively point out the evolutionary direction for the model, we propose a novel \textbf{Color Balance Prior} as a strong guide in the feature contextualization step and as a weak guide in the final decoding phase. We construct a comprehensive benchmark using paired training data from three real-world underwater datasets and evaluate on six test sets, including three paired and three unpaired, sourced from four real-world underwater datasets. Subsequently, we tested 14 traditional and retrained 23 deep learning existing underwater image restoration methods on this benchmark, obtaining metric results for each approach. This effort aims to furnish a valuable benchmarking dataset for standard basis for comparison. The extensive experiment results demonstrate that our method outperforms 37 other state-of-the-art methods overall on various benchmark datasets and metrics, despite not achieving the best results in certain individual cases. The code and dataset are available at \href{https://github.com/CXH-Research/GuidedHybSensUIR}{https://github.com/CXH-Research/GuidedHybSensUIR}., Comment: Accepted by IEEE TCSVT

3D object detection is crucial for Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADAS). However, most 3D detectors prioritize detection accuracy, often overlooking network inference speed in practical applications. In this paper, we propose RadarNeXt, a real-time and reliable 3D object detector based on the 4D mmWave radar point clouds. It leverages the re-parameterizable neural networks to catch multi-scale features, reduce memory cost and accelerate the inference. Moreover, to highlight the irregular foreground features of radar point clouds and suppress background clutter, we propose a Multi-path Deformable Foreground Enhancement Network (MDFEN), ensuring detection accuracy while minimizing the sacrifice of speed and excessive number of parameters. Experimental results on View-of-Delft and TJ4DRadSet datasets validate the exceptional performance and efficiency of RadarNeXt, achieving 50.48 and 32.30 mAPs with the variant using our proposed MDFEN. Notably, our RadarNeXt variants achieve inference speeds of over 67.10 FPS on the RTX A4000 GPU and 28.40 FPS on the Jetson AGX Orin. This research demonstrates that RadarNeXt brings a novel and effective paradigm for 3D perception based on 4D mmWave radar., Comment: 8 pages, 5 figures, 3 tables. Code: https://github.com/Pay246-git468/RadarNeXt

As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e.g., LoRA) will become essential due to their efficiency in reducing computation costs. However, recent studies have raised alarming concerns that LoRA fine-tuning could potentially compromise the safety alignment in LLMs, posing significant risks for the model owner. In this paper, we first investigate the underlying mechanism by analyzing the changes in safety alignment related features before and after fine-tuning. Then, we propose a fixed safety module calculated by safety data and a task-specific initialization for trainable parameters in low-rank adaptations, termed Safety-alignment preserved Low-Rank Adaptation (SaLoRA). Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments. Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.

The kagome lattice has garnered significant attention due to its ability to host quantum spin Fermi liquid states. Recently, the combination of unique lattice geometry, electron-electron correlations, and adjustable magnetism in solid kagome materials has led to the discovery of numerous fascinating quantum properties. These include unconventional superconductivity, charge and spin density waves (CDW/SDW), pair density waves (PDW), and Chern insulator phases. These emergent states are closely associated with the distinctive characteristics of the kagome lattice's electronic structure, such as van Hove singularities, Dirac fermions, and flat bands, which can exhibit exotic quasi-particle excitations under different symmetries and magnetic conditions. Recently, various quantum kagome materials have been developed, typically consisting of kagome layers stacked along the $z$-axis with atoms either filling the geometric centers of the kagome lattice or embedded between the layers. In this topical review, we begin by introducing the fundamental properties of several kagome materials. To gain an in-depth understanding of the relationship between topology and correlation, we then discuss the complex phenomena observed in these systems. These include the simplest kagome metal $T_3X$, kagome intercalation metal $TX$, and the ternary compounds $AT_6X_6$ and $RT_3X_5$ ($A$ = Li, Mg, Ca, or rare earth; $T$ = V, Cr, Mn, Fe, Co, Ni; $X$ = Sn, Ge; $R$ = K, Rb, Cs). Finally, we provide a perspective on future experimental work in this field., Comment: Invited Review

In this paper, we show that the value functions of mean field control problems with common noise are the unique viscosity solutions to fully second-order Hamilton-Jacobi-Bellman equations, in a Crandall-Lions-like framework. We allow the second-order derivative in measure to be state-dependent and thus infinite-dimensional, rather than derived from a finite-dimensional operator, hence the term ''fully''. Our argument leverages the construction of smooth approximations from particle systems developed by Cosso, Gozzi, Kharroubi, Pham, and Rosestolato [Trans. Amer. Math. Soc., 2023], and the compactness argument via penalization of measure moments in Soner and Yan [Appl. Math. Optim., 2024]. Our work addresses unbounded dynamics and state-dependent common noise volatility, and to our knowledge, this is the first result of its kind in the literature., Comment: 32 pages

Continuous gravitational waves (CWs) emission from neutron stars carries information about their internal structure and equation of state, and it can provide tests of General Relativity. We present a search for CWs from a set of 45 known pulsars in the first part of the fourth LIGO--Virgo--KAGRA observing run, known as O4a. We conducted a targeted search for each pulsar using three independent analysis methods considering the single-harmonic and the dual-harmonic emission models. We find no evidence of a CW signal in O4a data for both models and set upper limits on the signal amplitude and on the ellipticity, which quantifies the asymmetry in the neutron star mass distribution. For the single-harmonic emission model, 29 targets have the upper limit on the amplitude below the theoretical spin-down limit. The lowest upper limit on the amplitude is $6.4\!\times\!10^{-27}$ for the young energetic pulsar J0537-6910, while the lowest constraint on the ellipticity is $8.8\!\times\!10^{-9}$ for the bright nearby millisecond pulsar J0437-4715. Additionally, for a subset of 16 targets we performed a narrowband search that is more robust regarding the emission model, with no evidence of a signal. We also found no evidence of non-standard polarizations as predicted by the Brans-Dicke theory., Comment: main paper: 12 pages, 6 figures, 4 tables

Recent advancements in large language models (LLMs) based embedding models have established new state-of-the-art benchmarks for text embedding tasks, particularly in dense vector-based retrieval. However, these models predominantly focus on English, leaving multilingual embedding capabilities largely unexplored. To address this limitation, we present LUSIFER, a novel zero-shot approach that adapts LLM-based embedding models for multilingual tasks without requiring multilingual supervision. LUSIFER's architecture combines a multilingual encoder, serving as a language-universal learner, with an LLM-based embedding model optimized for embedding-specific tasks. These components are seamlessly integrated through a minimal set of trainable parameters that act as a connector, effectively transferring the multilingual encoder's language understanding capabilities to the specialized embedding model. Additionally, to comprehensively evaluate multilingual embedding performance, we introduce a new benchmark encompassing 5 primary embedding tasks, 123 diverse datasets, and coverage across 14 languages. Extensive experimental results demonstrate that LUSIFER significantly enhances the multilingual performance across various embedding tasks, particularly for medium and low-resource languages, without requiring explicit multilingual training data.

In this paper, we present the Global Multimedia Deepfake Detection held concurrently with the Inclusion 2024. Our Multimedia Deepfake Detection aims to detect automatic image and audio-video manipulations including but not limited to editing, synthesis, generation, Photoshop,etc. Our challenge has attracted 1500 teams from all over the world, with about 5000 valid result submission counts. We invite the top 20 teams to present their solutions to the challenge, from which the top 3 teams are awarded prizes in the grand finale. In this paper, we present the solutions from the top 3 teams of the two tracks, to boost the research work in the field of image and audio-video forgery detection. The methodologies developed through the challenge will contribute to the development of next-generation deepfake detection systems and we encourage participants to open source their methods., Comment: Inclusion 2024 Global Multimedia Deepfake Detection Competition Top Team Technical Report

Large-scale Vision-Language Models (VLMs) have advanced by aligning vision inputs with text, significantly improving performance in computer vision tasks. Moreover, for VLMs to be effectively utilized in real-world applications, an understanding of diverse multi-vision sensor data, such as thermal, depth, and X-ray information, is essential. However, we find that current VLMs process multi-vision sensor images without deep understanding of sensor information, disregarding each sensor's unique physical properties. This limitation restricts their capacity to interpret and respond to complex questions requiring multi-vision sensor reasoning. To address this, we propose a novel Multi-vision Sensor Perception and Reasoning (MS-PR) benchmark, assessing VLMs on their capacity for sensor-specific reasoning. Moreover, we introduce Diverse Negative Attributes (DNA) optimization to enable VLMs to perform deep reasoning on multi-vision sensor tasks, helping to bridge the core information gap between images and sensor data. Extensive experimental results validate that the proposed DNA method can significantly improve the multi-vision sensor reasoning for VLMs., Comment: https://github.com/top-yun/MS-PR. arXiv admin note: text overlap with arXiv:2408.12114

Finger vein recognition (FVR) has emerged as a secure biometric technique because of the confidentiality of vascular bio-information. Recently, deep learning-based FVR has gained increased popularity and achieved promising performance. However, the limited size of public vein datasets has caused overfitting issues and greatly limits the recognition performance. Although traditional data augmentation can partially alleviate this data shortage issue, it cannot capture the real finger posture variations due to the rigid label-preserving image transformations, bringing limited performance improvement. To address this issue, we propose a novel motion transfer (MT) model for finger vein image data augmentation via modeling the actual finger posture and rotational movements. The proposed model first utilizes a key point detector to extract the key point and pose map of the source and drive finger vein images. We then utilize a dense motion module to estimate the motion optical flow, which is fed to an image generation module for generating the image with the target pose. Experiments conducted on three public finger vein databases demonstrate that the proposed motion transfer model can effectively improve recognition accuracy. Code is available at: https://github.com/kevinhuangxf/FingerVeinRecognition., Comment: 5 Pages

Weakly supervised temporal action localization (WS-TAL) is a task of targeting at localizing complete action instances and categorizing them with video-level labels. Action-background ambiguity, primarily caused by background noise resulting from aggregation and intra-action variation, is a significant challenge for existing WS-TAL methods. In this paper, we introduce a hybrid multi-head attention (HMHA) module and generalized uncertainty-based evidential fusion (GUEF) module to address the problem. The proposed HMHA effectively enhances RGB and optical flow features by filtering redundant information and adjusting their feature distribution to better align with the WS-TAL task. Additionally, the proposed GUEF adaptively eliminates the interference of background noise by fusing snippet-level evidences to refine uncertainty measurement and select superior foreground feature information, which enables the model to concentrate on integral action instances to achieve better action localization and classification performance. Experimental results conducted on the THUMOS14 dataset demonstrate that our method outperforms state-of-the-art methods. Our code is available in \url{https://github.com/heyuanpengpku/GUEF/tree/main}.

Fairly allocating indivisible goods is a frequently occurring task in everyday life. Given an initial allocation of the goods, we consider the problem of reforming it via a sequence of exchanges to attain fairness in the form of envy-freeness up to one good (EF1). We present a vast array of results on the complexity of determining whether it is possible to reach an EF1 allocation from the initial allocation and, if so, the minimum number of exchanges required. In particular, we uncover several distinctions based on the number of agents involved and their utility functions. Furthermore, we derive essentially tight bounds on the worst-case number of exchanges needed to achieve EF1.

Tripartite entanglement as a remarkable resource in quantum information science has been extensively investigated in hybrid quantum systems, whereas it is generally weak and prone to be suppressed by noise, restricting its practical application in quantum technologies. Here, we propose how to enhance the tripartite entanglement among magnons, photons and phonons in a hybrid cavity-magnon optomechanical system by exploiting a synergistic effect of the optical parametric amplification (OPA) and mechanical parametric amplification (MPA). We find that in the case of individually applying the OPA or MPA, the tripartite entanglement can be enhanced by several folds. Remarkably, upon suitably tuning the phase matching of the two parametric fields presented simultaneously, the strengths of the entanglement can be further enhanced due to the constructive interference between the OPA and MPA. We also show that our method can improve the robustness of the entanglement against thermal noise. Our work provides a promising method for manipulating the entanglement with easy tunability and may serve as a useful tool for the enhancement and protection of fragile quantum resources., Comment: 12 pages, 4 figures, accepted for publication in PRA

We consider the generative modeling of speech over multiple minutes, a requirement for long-form multimedia generation and audio-native voice assistants. However, current spoken language models struggle to generate plausible speech past tens of seconds, from high temporal resolution of speech tokens causing loss of coherence, to architectural issues with long-sequence training or extrapolation, to memory costs at inference time. With these considerations we propose SpeechSSM, the first speech language model to learn from and sample long-form spoken audio (e.g., 16 minutes of read or extemporaneous speech) in a single decoding session without text intermediates, based on recent advances in linear-time sequence modeling. Furthermore, to address growing challenges in spoken language evaluation, especially in this new long-form setting, we propose: new embedding-based and LLM-judged metrics; quality measurements over length and time; and a new benchmark for long-form speech processing and generation, LibriSpeech-Long. Speech samples and the dataset are released at https://google.github.io/tacotron/publications/speechssm/

Underwater Image Restoration (UIR) remains a challenging task in computer vision due to the complex degradation of images in underwater environments. While recent approaches have leveraged various deep learning techniques, including Transformers and complex, parameter-heavy models to achieve significant improvements in restoration effects, we demonstrate that pure CNN architectures with lightweight parameters can achieve comparable results. In this paper, we introduce UIR-PolyKernel, a novel method for underwater image restoration that leverages Polymorphic Large Kernel CNNs. Our approach uniquely combines large kernel convolutions of diverse sizes and shapes to effectively capture long-range dependencies within underwater imagery. Additionally, we introduce a Hybrid Domain Attention module that integrates frequency and spatial domain attention mechanisms to enhance feature importance. By leveraging the frequency domain, we can capture hidden features that may not be perceptible to humans but are crucial for identifying patterns in both underwater and on-air images. This approach enhances the generalization and robustness of our UIR model. Extensive experiments on benchmark datasets demonstrate that UIR-PolyKernel achieves state-of-the-art performance in underwater image restoration tasks, both quantitatively and qualitatively. Our results show that well-designed pure CNN architectures can effectively compete with more complex models, offering a balance between performance and computational efficiency. This work provides new insights into the potential of CNN-based approaches for challenging image restoration tasks in underwater environments. The code is available at \href{https://github.com/CXH-Research/UIR-PolyKernel}{https://github.com/CXH-Research/UIR-PolyKernel}., Comment: Accepted by ICASSP2025

Chatbot research is advancing with the growing importance of chatbots in fields that require human interactions, such as customer support and mental health care. Despite these advancements, chatbots still face significant challenges in understanding subtle nuances and managing long conversation histories. To address these issues, our study introduces a dual approach: firstly, we employ Emotional Preference Optimization (EPO) to train chatbots not only with correct responses but also with counter-emotional responses-those that are contextually similar but emotionally divergent. This training enables the model to discern fine nuance distinctions between correct and counter-emotional responses, thereby enhancing the quality of its responses. Secondly, we introduce MambaCompressor to effectively compress and manage extensive conversation histories, significantly reducing time and memory complexities while improving the chatbot's contextual understanding. Our comprehensive experiments across multiple datasets demonstrate that our model significantly outperforms existing models in generating empathetic responses and efficiently managing lengthy dialogues.

Multimodal large language models (MLLMs) have significantly advanced tasks like caption generation and visual question answering by integrating visual and textual data. However, they sometimes produce misleading or hallucinate content due to discrepancies between their pre-training data and real user prompts. Existing approaches using Direct Preference Optimization (DPO) in vision-language tasks often rely on strong models like GPT-4 or CLIP to determine positive and negative responses. Here, we propose a new framework in generating synthetic data using a reward model as a proxy of human preference for effective multimodal alignment with DPO training. The resulting DPO dataset ranges from 2K to 9K image-text pairs, was evaluated on LLaVA-v1.5-7B, where our approach demonstrated substantial improvements in both the trustworthiness and reasoning capabilities of the base model across multiple hallucination and vision-language benchmark. The experiment results indicate that integrating selected synthetic data, such as from generative and rewards models can effectively reduce reliance on human-annotated data while enhancing MLLMs' alignment capability, offering a scalable solution for safer deployment., Comment: Project Page: https://pds-dpo.github.io/

In human communication, both verbal and non-verbal cues play a crucial role in conveying emotions, intentions, and meaning beyond words alone. These non-linguistic information, such as facial expressions, eye contact, voice tone, and pitch, are fundamental elements of effective interactions, enriching conversations by adding emotional and contextual depth. Recognizing the importance of non-linguistic content in communication, we present AV-EmoDialog, a dialogue system designed to exploit verbal and non-verbal information from users' audio-visual inputs to generate more responsive and empathetic interactions. AV-EmoDialog systematically exploits the emotional cues in audio-visual dialogues; extracting speech content and emotional tones from speech, analyzing fine-grained facial expressions from visuals, and integrating these cues to generate emotionally aware responses in an end-to-end manner. Through extensive experiments, we validate that the proposed AV-EmoDialog outperforms existing multimodal LLMs in generating not only emotionally appropriate but also contextually appropriate responses.

As recursion has been underlying any linguistic work for the last 60 years, the acquisition of recursive structures by children during language learning has become a focal point of inquiry. This study delves into the developmental trajectory of Mandarin-speaking children's acquisition of recursive possessives and locatives, assessing the impact of structural diversity on language acquisition. The research contrasts the comprehension of two-level recursive structures among children aged 3 to 7 years, employing answering question while seeing a picture task to elicit responses. The findings indicate that children do not attain adult-like proficiency in two-level recursion until the age of 6, and there exists a notable asymmetry in the acquisition of recursive possessives versus locatives. These results underscore the primacy of structural complexity and cognitive factors in the acquisition process, enhancing our comprehension of the cognitive foundations of language development and the pivotal role of recursion in child language acquisition.

Magnetic fields are ubiquitous across different physical systems of current interest; from the early Universe, compact astrophysical objects and heavy-ion collisions to condensed matter systems. A proper treatment of the effects produced by magnetic fields during the dynamical evolution of these systems, can help to understand observables that otherwise show a puzzling behavior. Furthermore, when these fields are comparable to or stronger than \Lambda_QCD, they serve as excellent probes to help elucidate the physics of strongly interacting matter under extreme conditions of temperature and density. In this work we provide a comprehensive review of recent developments on the description of QED and QCD systems where magnetic field driven effects are important. These include the modification of meson static properties such as masses and form factors, the chiral magnetic effect, the description of anomalous transport coefficients, superconductivity in extreme magnetic fields, the properties of neutron stars, the evolution of heavy-ion collisions, as well as effects on the QCD phase diagram. We describe recent theory and phenomenological developments using effective models as well as LQCD methods. The work represents a state-of-the-art review of the field, motivated by presentations and discussions during the "Workshop on Strongly Interacting Matter in Strong Electromagnetic Fields" that took place in the European Centre for Theoretical Studies in Nuclear Physics and Related Areas (ECT*) in the city of Trento, Italy, September 25-29, 2023., Comment: 325 pages-long review of recent topics of interest in the field of magnetic field effects on QED and QCD matter. To be susbmitted to PNPP

Help-seeking is a critical aspect of learning and problem-solving for computing students. Recent research has shown that many students are aware of generative AI (GenAI) tools; however, there are gaps in the extent and effectiveness of how students use them. With over two years of widespread GenAI usage, it is crucial to understand whether students' help-seeking behaviors with these tools have evolved and how. This paper presents findings from a repeated cross-sectional survey conducted among computing students across North American universities (n=95). Our results indicate shifts in GenAI usage patterns. In 2023, 34.1% of students (n=47) reported never using ChatGPT for help, ranking it fourth after online searches, peer support, and class forums. By 2024, this figure dropped sharply to 6.3% (n=48), with ChatGPT nearly matching online search as the most commonly used help resource. Despite this growing prevalence, there has been a decline in students' hourly and daily usage of GenAI tools, which may be attributed to a common tendency to underestimate usage frequency. These findings offer new insights into the evolving role of GenAI in computing education, highlighting its increasing acceptance and solidifying its position as a key help resource., Comment: 2 pages, 1 figure, to be published in SIGCSE 2025

Sessile ciliates attach to substrates and generate feeding currents to capture passing particulates and dissolved nutrients. Optimal ciliary activity that maximizes nutrient flux at the cell surface while minimizing the rate of hydrodynamic energy dissipation is well characterized in uniform nutrient fields. However, it is unclear how ciliary motion should change when nutrients are non-uniform or patchy. To address this question, we modeled the sessile ciliate and feeding currents using the spherical envelope model, and used an unsteady advection-diffusion equation to describe the nutrient scalar field. In the absence of flows, we calculated the diffusive nutrient uptake analytically in linear nutrient gradients and found no advantage over uptake in uniform fields. With ciliary activity driving feeding currents, we used a spectral method to solve for the unsteady nutrient concentration. We found that, when the axis of symmetry of the ciliary motion is aligned with the nutrient gradient, nutrient uptake at the cell surface increases steadily over time, with highest uptake achieved by the treadmill ciliary motion which is optimal in uniform fields as well. The associated nutrient uptake in concentration gradients scales with the square root of the product of time and P\'eclet number. In patchy environments, optimal ciliary activity depends on the nature of the patchiness. Our findings highlight strategies that enable sessile ciliates to thrive in environments with fluctuating nutrient availability., Comment: 30 pages, 5 figures

Large Vision Language Models (LVLMs) have achieved significant progress in integrating visual and textual inputs for multimodal reasoning. However, a recurring challenge is ensuring these models utilize visual information as effectively as linguistic content when both modalities are necessary to formulate an accurate answer. We hypothesize that hallucinations arise due to the lack of effective visual grounding in current LVLMs. This issue extends to vision-language benchmarks, where it is difficult to make the image indispensable for accurate answer generation, particularly in vision question-answering tasks. In this work, we introduce FiVL, a novel method for constructing datasets designed to train LVLMs for enhanced visual grounding and to evaluate their effectiveness in achieving it. These datasets can be utilized for both training and assessing an LVLM's ability to use image content as substantive evidence rather than relying solely on linguistic priors, providing insights into the model's reliance on visual information. To demonstrate the utility of our dataset, we introduce an innovative training task that outperforms baselines alongside a validation method and application for explainability. The code is available at https://github.com/IntelLabs/fivl.

Spiking Neural Networks (SNNs) have a low-power advantage but perform poorly in image segmentation tasks. The reason is that directly converting neural networks with complex architectural designs for segmentation tasks into spiking versions leads to performance degradation and non-convergence. To address this challenge, we first identify the modules in the architecture design that lead to the severe reduction in spike firing, make targeted improvements, and propose Spike2Former architecture. Second, we propose normalized integer spiking neurons to solve the training stability problem of SNNs with complex architectures. We set a new state-of-the-art for SNNs in various semantic segmentation datasets, with a significant improvement of +12.7% mIoU and 5.0 efficiency on ADE20K, +14.3% mIoU and 5.2 efficiency on VOC2012, and +9.1% mIoU and 6.6 efficiency on CityScapes., Comment: This work has been accepted on Association for the Advancement of Artificial Intelligence 2025

Partial label learning (PLL) is a complicated weakly supervised multi-classification task compounded by class imbalance. Currently, existing methods only rely on inter-class pseudo-labeling from inter-class features, often overlooking the significant impact of the intra-class imbalanced features combined with the inter-class. To address these limitations, we introduce Granular Ball Representation for Imbalanced PLL (GBRIP), a novel framework for imbalanced PLL. GBRIP utilizes coarse-grained granular ball representation and multi-center loss to construct a granular ball-based nfeature space through unsupervised learning, effectively capturing the feature distribution within each class. GBRIP mitigates the impact of confusing features by systematically refining label disambiguation and estimating imbalance distributions. The novel multi-center loss function enhances learning by emphasizing the relationships between samples and their respective centers within the granular balls. Extensive experiments on standard benchmarks demonstrate that GBRIP outperforms existing state-of-the-art methods, offering a robust solution to the challenges of imbalanced PLL., Comment: AAAI25

Allocating indivisible goods is a ubiquitous task in fair division. We study additive welfarist rules, an important class of rules which choose an allocation that maximizes the sum of some function of the agents' utilities. Prior work has shown that the maximum Nash welfare (MNW) rule is the unique additive welfarist rule that guarantees envy-freeness up to one good (EF1). We strengthen this result by showing that MNW remains the only additive welfarist rule that ensures EF1 for identical-good instances, two-value instances, as well as normalized instances with three or more agents. On the other hand, if the agents' utilities are integers, we demonstrate that several other rules offer the EF1 guarantee, and provide characterizations of these rules for various classes of instances., Comment: Appears in the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2025

This paper introduces an algorithm designed to approximate quantum transformation matrix with a restricted number of gates by using the block decomposition technique. Addressing challenges posed by numerous gates in handling large qubit transformations, the algorithm provides a solution by optimizing gate usage while maintaining computational accuracy. Inspired by the Block Decompose algorithm, our approach processes transformation matrices in a block-wise manner, enabling users to specify the desired gate count for flexibility in resource allocation. Simulations validate the effectiveness of the algorithm in approximating transformations with significantly fewer gates, enhancing quantum computing efficiency for complex calculations. This work contributes a practical tool for efficient quantum computation, bridging the gap for scalable and effective quantum information processing applications., Comment: 12 pages, 7 figs+tables

For fractals on Riemannian manifolds, the theory of iterated function systems often does not apply well directly, as fractal sets are often defined by relations that are multivalued or non-contractive. To overcome this difficulty, we introduce the notion of iterated relation systems. We study the attractor of an iterated relation system and formulate a condition under which such an attractor can be identified with that of an associated graph-directed iterated function system. Using this method, we obtain dimension formulas for the attractor of an iterated relation system under the graph open set condition or the graph finite type condition. This method improves the one in [Ngai-Xu, J. Geom. Anal. {\bf 33} (2023), 262], which relies on knowing the specific structure of the attractor.

The mesoscopic level serves as a bridge between the macroscopic and microscopic worlds, addressing gaps overlooked by both. Image manipulation localization (IML), a crucial technique to pursue truth from fake images, has long relied on low-level (microscopic-level) traces. However, in practice, most tampering aims to deceive the audience by altering image semantics. As a result, manipulation commonly occurs at the object level (macroscopic level), which is equally important as microscopic traces. Therefore, integrating these two levels into the mesoscopic level presents a new perspective for IML research. Inspired by this, our paper explores how to simultaneously construct mesoscopic representations of micro and macro information for IML and introduces the Mesorch architecture to orchestrate both. Specifically, this architecture i) combines Transformers and CNNs in parallel, with Transformers extracting macro information and CNNs capturing micro details, and ii) explores across different scales, assessing micro and macro information seamlessly. Additionally, based on the Mesorch architecture, the paper introduces two baseline models aimed at solving IML tasks through mesoscopic representation. Extensive experiments across four datasets have demonstrated that our models surpass the current state-of-the-art in terms of performance, computational complexity, and robustness., Comment: AAAI 2025. Code: $\href{https://github.com/scu-zjz/Mesorch}{this~url}$

Urbanization as a global trend has led to many environmental challenges, including the urban heat island (UHI) effect. The increase in temperature has a significant impact on the well-being of urban residents. Air temperature ($T_a$) at 2m above the surface is a key indicator of the UHI effect. How land use land cover (LULC) affects $T_a$ is a critical research question which requires high-resolution (HR) $T_a$ data at neighborhood scale. However, weather stations providing $T_a$ measurements are sparsely distributed e.g. more than 10km apart; and numerical models are impractically slow and computationally expensive. In this work, we propose a novel method to predict HR $T_a$ at 100m ground separation distance (gsd) using land surface temperature (LST) and other LULC related features which can be easily obtained from satellite imagery. Our method leverages diffusion models for the first time to generate accurate and visually realistic HR $T_a$ maps, which outperforms prior methods. We pave the way for meteorological research using computer vision techniques by providing a dataset of an extended spatial and temporal coverage, and a high spatial resolution as a benchmark for future research. Furthermore, we show that our model can be applied to urban planning by simulating the impact of different urban designs on $T_a$.

Current Large Language Models (LLMs) for understanding proteins primarily treats amino acid sequences as a text modality. Meanwhile, Protein Language Models (PLMs), such as ESM-2, have learned massive sequential evolutionary knowledge from the universe of natural protein sequences. Furthermore, structure-based encoders like ProteinMPNN learn the structural information of proteins through Graph Neural Networks. However, whether the incorporation of protein encoders can enhance the protein understanding of LLMs has not been explored. To bridge this gap, we propose EvoLlama, a multimodal framework that connects a structure-based encoder, a sequence-based protein encoder and an LLM for protein understanding. EvoLlama consists of a ProteinMPNN structure encoder, an ESM-2 protein sequence encoder, a multimodal projector to align protein and text representations and a Llama-3 text decoder. To train EvoLlama, we fine-tune it on protein-oriented instructions and protein property prediction datasets verbalized via natural language instruction templates. Our experiments show that EvoLlama's protein understanding capabilities have been significantly enhanced, outperforming other fine-tuned protein-oriented LLMs in zero-shot settings by an average of 1%-8% and surpassing the state-of-the-art baseline with supervised fine-tuning by an average of 6%. On protein property prediction datasets, our approach achieves promising results that are competitive with state-of-the-art task-specific baselines. We will release our code in a future version.

Recent advancements in generative AI systems have raised concerns about academic integrity among educators. Beyond excelling at solving programming problems and text-based multiple-choice questions, recent research has also found that large multimodal models (LMMs) can solve Parsons problems based only on an image. However, such problems are still inherently text-based and rely on the capabilities of the models to convert the images of code blocks to their corresponding text. In this paper, we further investigate the capabilities of LMMs to solve graph and tree data structure problems based only on images. To achieve this, we computationally construct and evaluate a novel benchmark dataset comprising 9,072 samples of diverse graph and tree data structure tasks to assess the performance of the GPT-4o, GPT-4v, Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 1.0 Pro Vision, and Claude 3 model families. GPT-4o and Gemini 1.5 Flash performed best on trees and graphs respectively. GPT-4o achieved 87.6% accuracy on tree samples, while Gemini 1.5 Flash, achieved 56.2% accuracy on graph samples. Our findings highlight the influence of structural and visual variations on model performance. This research not only introduces an LMM benchmark to facilitate replication and further exploration but also underscores the potential of LMMs in solving complex computing problems, with important implications for pedagogy and assessment practices., Comment: 14 pages, 4 figures, to be published in ACE 2025

In this paper, we solve the complete Salpeter equation and use the obtained relativistic wave function to calculate the strong and radiative electromagnetic decays of ${D^*}$ meson. The focus of this study is on the relativistic corrections. In our method, the wave function of $D$ meson is not pure $S$-wave, but includes both nonrelativistic $S$-wave and relativistic $P$-wave, while the wave function of $D^*$ meson includes nonrelativistic $S$-wave, as well as relativistic $P$-wave and $D$-wave. So in this case, the decay ${D^{*}\rightarrow{D}\gamma}$ is not a nonrelativistic $M1$ transition, but rather an $M1+E2+M3+E4$ decay in our method. We find that in strong decay $D^{*}\rightarrow{D}{\pi}$, the nonrelativistic contribution is dominant, while in electromagnetic decay ${D^{*}\rightarrow{D}\gamma}$, the relativistic correction is dominant., Comment: 12 pages, 2 figures

Let $d\geq1$, $\Omega$ be a bounded domain of a smooth complete Riemannian d-manifold M, and $\mu$ be a positive finite Borel measure with compact support in $\overline{\Omega}$. We prove the Courant nodal domain theorem for the eigenfunctions of Kre\u{i}n-Feller operator $\Delta_{\mu}$ under the assumption that such eigenfunctions are continuous on $\overline{\Omega}$. For $d\geq2$, We prove that on a bounded domain $\Omega\subset M$ with smooth boundary and on which the Green's function of the Laplace-Beltrami operator exists, the eigenfunctions of $\Delta_{\mu}$ are continuous on $\Omega$. We also prove that if M is compact and $\partial M=\emptyset$, then the eigenfuctions of $\Delta_{\mu}$ are continuous on M.

Finding optimal solutions to combinatorial optimization problems is pivotal in both scientific and technological domains, within academic research and industrial applications. A considerable amount of effort has been invested in the development of accelerated methods that leverage sophisticated models and harness the power of advanced computational hardware. Despite the advancements, a critical challenge persists, the dual demand for both high efficiency and broad generality in solving problems. In this work, we propose a general method, Free-Energy Machine (FEM), based on the ideas of free-energy minimization in statistical physics, combined with automatic differentiation and gradient-based optimization in machine learning. The algorithm is flexible, solving various combinatorial optimization problems using a unified framework, and is efficient, naturally utilizing massive parallel computational devices such as graph processing units (GPUs) and field-programmable gate arrays (FPGAs). We benchmark our algorithm on various problems including the maximum cut problems, balanced minimum cut problems, and maximum $k$-satisfiability problems, scaled to millions of variables, across both synthetic, real-world, and competition problem instances. The findings indicate that our algorithm not only exhibits exceptional speed but also surpasses the performance of state-of-the-art algorithms tailored for individual problems. This highlights that the interdisciplinary fusion of statistical physics and machine learning opens the door to delivering cutting-edge methodologies that will have broad implications across various scientific and industrial landscapes., Comment: Comments are welcome!

Reconstructing desired objects and scenes has long been a primary goal in 3D computer vision. Single-view point cloud reconstruction has become a popular technique due to its low cost and accurate results. However, single-view reconstruction methods often rely on expensive CAD models and complex geometric priors. Effectively utilizing prior knowledge about the data remains a challenge. In this paper, we introduce hyperbolic space to 3D point cloud reconstruction, enabling the model to represent and understand complex hierarchical structures in point clouds with low distortion. We build upon previous methods by proposing a hyperbolic Chamfer distance and a regularized triplet loss to enhance the relationship between partial and complete point clouds. Additionally, we design adaptive boundary conditions to improve the model's understanding and reconstruction of 3D structures. Our model outperforms most existing models, and ablation studies demonstrate the significance of our model and its components. Experimental results show that our method significantly improves feature extraction capabilities. Our model achieves outstanding performance in 3D reconstruction tasks., Comment: Accepted by AAAI25

Deep-learning based traffic prediction models require vast amounts of data to learn embedded spatial and temporal dependencies. The inherent privacy and commercial sensitivity of such data has encouraged a shift towards decentralised data-driven methods, such as Federated Learning (FL). Under a traditional Machine Learning paradigm, traffic flow prediction models can capture spatial and temporal relationships within centralised data. In reality, traffic data is likely distributed across separate data silos owned by multiple stakeholders. In this work, a cross-silo FL setting is motivated to facilitate stakeholder collaboration for optimal traffic flow prediction applications. This work introduces an FL framework, referred to as FedTPS, to generate synthetic data to augment each client's local dataset by training a diffusion-based trajectory generation model through FL. The proposed framework is evaluated on a large-scale real world ride-sharing dataset using various FL methods and Traffic Flow Prediction models, including a novel prediction model we introduce, which leverages Temporal and Graph Attention mechanisms to learn the Spatio-Temporal dependencies embedded within regional traffic flow data. Experimental results show that FedTPS outperforms multiple other FL baselines with respect to global model performance., Comment: 11 pages, 7 figures, 6 tables, ACM format