Search

Quantum error correction (QEC) is essential for protecting quantum information against noise, yet understanding the structure of the Knill-Laflamme (KL) coefficients $\lambda_{ij}$ from the condition $PE_i^\dagger E_j P = \lambda_{ij} P$ remains challenging, particularly for nonadditive codes. In this work, we introduce the signature vector $\vec{\lambda}(P)$, composed of the off-diagonal KL coefficients $\lambda_{ij}$, where each coefficient corresponds to equivalence classes of errors counted only once. We define its Euclidean norm $\lambda^*(P)$ as a scalar measure representing the total strength of error correlations within the code subspace defined by the projector $P$. We parameterize $P$ on a Stiefel manifold and formulate an optimization problem based on the KL conditions to systematically explore possible values of $\lambda^*$. Moreover, we show that, for $((n,K,d))$ codes, $\lambda^*$ is invariant under local unitary transformations. Applying our approach to the $((6, 2, 3))$ quantum code, we find that $\lambda^*_{\text{min}} = \sqrt{0.6}$ and $\lambda^*_{\text{max}} = 1$, with $\lambda^* = 1$ corresponding to a known degenerate stabilizer code. We construct continuous families of new nonadditive codes parameterized by vectors in $\mathbb{R}^5$, with $\lambda^*$ varying over the interval $[\sqrt{0.6}, 1]$. For the $((7, 2, 3))$ code, we identify $\lambda^*_{\text{min}} = 0$ (corresponding to the non-degenerate Steane code) and $\lambda^*_{\text{max}} = \sqrt{7}$ (corresponding to the permutation-invariant code by Pollatsek and Ruskai), and we demonstrate continuous paths connecting these extremes via cyclic codes characterized solely by $\lambda^*$. Our findings provide new insights into the structure of quantum codes, advance the theoretical foundations of QEC, and open new avenues for investigating intricate relationships between code subspaces and error correlations., Comment: 18 pages, 2 figures

Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual large language model (LLM) with low-rank adaptation (LoRA) designed for enhanced video (with paired audio) captioning through directed preference optimization (DPO). We propose new metrics to evaluate the completeness and accuracy of video descriptions, which are optimized using DPO. To further improve training, we introduce a novel multi-round DPO (mrDPO) approach, which involves periodically updating the DPO reference model, merging and re-initializing the LoRA module as a proxy for parameter updates after each training round (1,000 steps), and incorporating guidance from ground-truth video captions to stabilize the process. To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels. Experiments show that mrDPO significantly enhances video-SALMONN 2's captioning accuracy, reducing global and local error rates by 40\% and 20\%, respectively, while decreasing the repetition rate by 35\%. The final video-SALMONN 2 model, with just 7 billion parameters, surpasses leading models such as GPT-4o and Gemini-1.5-Pro in video captioning tasks, while maintaining competitive performance to the state-of-the-art on widely used video question-answering benchmark among models of similar size. Upon acceptance, we will release the code, model checkpoints, and training and test data. Demos are available at \href{https://video-salmonn-2.github.io}{https://video-salmonn-2.github.io}.

The inversion of diffusion model sampling, which aims to find the corresponding initial noise of a sample, plays a critical role in various tasks. Recently, several heuristic exact inversion samplers have been proposed to address the inexact inversion issue in a training-free manner. However, the theoretical properties of these heuristic samplers remain unknown and they often exhibit mediocre sampling quality. In this paper, we introduce a generic formulation, \emph{Bidirectional Explicit Linear Multi-step} (BELM) samplers, of the exact inversion samplers, which includes all previously proposed heuristic exact inversion samplers as special cases. The BELM formulation is derived from the variable-stepsize-variable-formula linear multi-step method via integrating a bidirectional explicit constraint. We highlight this bidirectional explicit constraint is the key of mathematically exact inversion. We systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE. Consequently, we propose the Optimal BELM (O-BELM) sampler through the LTE minimization approach. We conduct additional analysis to substantiate the theoretical stability and global convergence property of the proposed optimal sampler. Comprehensive experiments demonstrate our O-BELM sampler establishes the exact inversion property while achieving high-quality sampling. Additional experiments in image editing and image interpolation highlight the extensive potential of applying O-BELM in varying applications., Comment: accepted paper by NeurIPS

The slowing down of Moore's law has driven the development of application-specific processors for deep learning. Analog photonic processors offer a promising solution for accelerating matrix-vector multiplications (MVMs) in deep learning by leveraging parallel computations in the optical domain. Intensity-based photonic MVM processors, which do not utilize the phase information of light, are appealing due to their simplified operations. However, existing intensity-based schemes for such processors often employ wavelength multiplexing or mode multiplexing, both of which have limited scalability due to high insertion loss or wavelength crosstalk. In this work, we present a scalable intensity-based photonic MVM processor based on the concept of waveguide multiplexing. This scheme employs multiport photodetectors (PDs) to sum the intensities of multiple optical signals, eliminating the need for multiple wavelengths or modes. A 16-port Ge PD with a 3 dB bandwidth of 11.8 GHz at a bias voltage of -3 V is demonstrated, and it can be further scaled up to handle 250 ports while maintaining a 6.1 GHz operation bandwidth. A 4 $\times$ 4 circuit fabricated on a Si-on-insulator (SOI) platform is used to perform MVMs in a 3-layer neural network designed for classifying Iris flowers, achieving a classification accuracy of 93.3%. Furthermore, the performance of large-scale circuits in a convolutional neural network (CNN) for Fashion-MNIST is simulated, resulting in a classification accuracy of 90.53%. This work provides a simplified and scalable approach to photonic MVM, laying a foundation for large-scale and multi-dimensional photonic matrix-matrix multiplication in optical neural networks.

Despite the significant progress in controllable music generation and editing, challenges remain in the quality and length of generated music due to the use of Mel-spectrogram representations and UNet-based model structures. To address these limitations, we propose a novel approach using a Diffusion Transformer (DiT) augmented with an additional control branch using ControlNet. This allows for long-form and variable-length music generation and editing controlled by text and melody prompts. For more precise and fine-grained melody control, we introduce a novel top-$k$ constant-Q Transform representation as the melody prompt, reducing ambiguity compared to previous representations (e.g., chroma), particularly for music with multiple tracks or a wide range of pitch values. To effectively balance the control signals from text and melody prompts, we adopt a curriculum learning strategy that progressively masks the melody prompt, resulting in a more stable training process. Experiments have been performed on text-to-music generation and music-style transfer tasks using open-source instrumental recording data. The results demonstrate that by extending StableAudio, a pre-trained text-controlled DiT model, our approach enables superior melody-controlled editing while retaining good text-to-music generation performance. These results outperform a strong MusicGen baseline in terms of both text-based generation and melody preservation for editing. Audio examples can be found at https://stable-audio-control.github.io/web/., Comment: 5 pages, 1 figure

Liquid Argon Time Projection Chamber (LArTPC) is an exceptional dual calorimeter capable of estimating the energy of incident particles through both the ionization charge and the scintillation light. Our studies show that due to the mechanisms of charge recombination and light generation involved in the energy dissipation in liquid argon, light calorimetry in LArTPCs is inherently self-compensating: the missing energy in the hadronic component is compensated for by the extra recombination luminescence compared to the electromagnetic component. Good compensation of the electron-to-hadron response ratio (e/h) around unity can be achieved across a broad range of drift electric fields from 0.2 to 1.8 kV/cm.This inherent self-compensation enhances the appeal of light calorimetry in LArTPCs, complementing the well-established charge calorimetry. Using GeV neutrinos as a case study, we show that light calorimetry can achieve an energy resolution comparable to the more sophisticated charge imaging calorimetry. The synergy between light and charge calorimetry offers a novel approach to evaluating and mitigating systematic uncertainties in energy measurements with LArTPCs., Comment: 15 pages, 11 figures

The Key-Value (KV) cache is a crucial component in serving transformer-based autoregressive large language models (LLMs), enabling faster inference by storing previously computed KV vectors. However, its memory consumption scales linearly with sequence length and batch size, posing a significant bottleneck in LLM deployment. Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages, which requires extensive parameter tuning thus unsuitable for pre-trained LLMs; (2) KV cache compression at test time, primarily through token eviction policies, which often overlook inter-layer dependencies and can be task-specific. This paper introduces an orthogonal approach to KV cache compression. We propose a low-rank approximation of KV weight matrices, allowing for plug-in integration with existing transformer-based LLMs without model retraining. To effectively compress KV cache at the weight level, we adjust for layerwise sensitivity and introduce a progressive compression strategy, which is supported by our theoretical analysis on how compression errors accumulate in deep networks. Our method is designed to function without model tuning in upcycling stages or task-specific profiling in test stages. Extensive experiments with LLaMA models ranging from 8B to 70B parameters across various tasks show that our approach significantly reduces the GPU memory footprint while maintaining performance., Comment: 15 pages, 4 figures

Van der Waals (vdW) heterostructures have attracted much attention due to their distinctive optical, electrical, and thermal properties, demonstrating promising potential in areas such as photocatalysis, ultrafast photonics, and free electron radiation devices. Particularly, they are promising platforms for studying thermionic emission. Here, we illustrate that using vdW heterostructure-based thermionic emission can enhance heat transfer in vacuum devices. As a proof of concept, we demonstrate that this approach offers a promising solution to the long-standing overheating issue in X-ray tubes. Specifically, we show that the saturated target temperature of a 2000 W X-ray tube can be reduced from around 1200 celsius to 490 celsius. Additionally, our study demonstrates that by reducing the height of the Schottky barrier formed in the vdW heterostructures, the thermionic cooling performance can be enhanced. Our findings pave the way for the development of high-power X-ray tubes., Comment: 4 figures, 11 pages

Connectivity queries, which check whether vertices belong to the same connected component, are fundamental in graph computations. Sliding window connectivity processes these queries over sliding windows, facilitating real-time streaming graph analytics. However, existing methods struggle with low-latency processing due to the significant overhead of continuously updating index structures as edges are inserted and deleted. We introduce a novel approach that leverages spanning trees to efficiently process queries. The novelty of this method lies in its ability to maintain spanning trees efficiently as window updates occur. Notably, our approach completely eliminates the need for replacement edge searches, a traditional bottleneck in managing spanning trees during edge deletions. We also present several optimizations to maximize the potential of spanning-tree-based indexes. Our comprehensive experimental evaluation shows that index update latency in spanning trees can be reduced by up to 458x while maintaining query performance, leading to an 8x improvement in throughput. Our approach also significantly outperforms the state-of-the-art in both query processing and index updates. Additionally, our methods use significantly less memory and demonstrate consistent efficiency across various settings.

In complex auditory environments, the human auditory system possesses the remarkable ability to focus on a specific speaker while disregarding others. In this study, a new model named SWIM, a short-window convolution neural network (CNN) integrated with Mamba, is proposed for identifying the locus of auditory attention (left or right) from electroencephalography (EEG) signals without relying on speech envelopes. SWIM consists of two parts. The first is a short-window CNN (SW$_\text{CNN}$), which acts as a short-term EEG feature extractor and achieves a final accuracy of 84.9% in the leave-one-speaker-out setup on the widely used KUL dataset. This improvement is due to the use of an improved CNN structure, data augmentation, multitask training, and model combination. The second part, Mamba, is a sequence model first applied to auditory spatial attention decoding to leverage the long-term dependency from previous SW$_\text{CNN}$ time steps. By joint training SW$_\text{CNN}$ and Mamba, the proposed SWIM structure uses both short-term and long-term information and achieves an accuracy of 86.2%, which reduces the classification errors by a relative 31.0% compared to the previous state-of-the-art result. The source code is available at https://github.com/windowso/SWIM-ASAD., Comment: accepted by SLT 2024

Continual Learning (CL) aims to learn in non-stationary scenarios, progressively acquiring and maintaining knowledge from sequential tasks. Recent Prompt-based Continual Learning (PCL) has achieved remarkable performance with Pre-Trained Models (PTMs). These approaches grow a prompt sets pool by adding a new set of prompts when learning each new task (\emph{prompt learning}) and adopt a matching mechanism to select the correct set for each testing sample (\emph{prompt retrieval}). Previous studies focus on the latter stage by improving the matching mechanism to enhance Prompt Retrieval Accuracy (PRA). To promote cross-task knowledge facilitation and form an effective and efficient prompt sets pool, we propose a plug-in module in the former stage to \textbf{Learn Whether to Grow (LW2G)} based on the disparities between tasks. Specifically, a shared set of prompts is utilized when several tasks share certain commonalities, and a new set is added when there are significant differences between the new task and previous tasks. Inspired by Gradient Projection Continual Learning, our LW2G develops a metric called Hinder Forward Capability (HFC) to measure the hindrance imposed on learning new tasks by surgically modifying the original gradient onto the orthogonal complement of the old feature space. With HFC, an automated scheme Dynamic Growing Approach adaptively learns whether to grow with a dynamic threshold. Furthermore, we design a gradient-based constraint to ensure the consistency between the updating prompts and pre-trained knowledge, and a prompts weights reusing strategy to enhance forward transfer. Extensive experiments show the effectiveness of our method. The source codes are available at \url{https://github.com/RAIAN08/LW2G}., Comment: submit to neurips2024

With the advances in deep learning, the performance of end-to-end (E2E) single-task models for speech and audio processing has been constantly improving. However, it is still challenging to build a general-purpose model with high performance on multiple tasks, since different speech and audio processing tasks usually require different training data, input features, or model architectures to achieve optimal performance. In this work, MT2KD, a novel two-stage multi-task learning framework is proposed to build a general-purpose speech and audio encoder that jointly performs three fundamental tasks: automatic speech recognition (ASR), audio tagging (AT) and speaker verification (SV). In the first stage, multi-teacher knowledge distillation (KD) is applied to align the feature spaces of three single-task high-performance teacher encoders into a single student encoder using the same unlabelled data. In the second stage, multi-task supervised fine-tuning is carried out by initialising the model from the first stage and training on the separate labelled data of each single task. Experiments demonstrate that the proposed multi-task training pipeline significantly outperforms a baseline model trained with multi-task learning from scratch. The final system achieves good performance on ASR, AT and SV: with less than 4% relative word-error-rate increase on ASR, only 1.9 lower mean averaged precision on AT and 0.23% absolute higher equal error rate on SV compared to the best-performing single-task encoders, using only a 66M total model parameters.

Speech quality assessment typically requires evaluating audio from multiple aspects, such as mean opinion score (MOS) and speaker similarity (SIM) etc., which can be challenging to cover using one small model designed for a single task. In this paper, we propose leveraging recently introduced auditory large language models (LLMs) for automatic speech quality assessment. By employing task-specific prompts, auditory LLMs are finetuned to predict MOS, SIM and A/B testing results, which are commonly used for evaluating text-to-speech systems. Additionally, the finetuned auditory LLM is able to generate natural language descriptions assessing aspects like noisiness, distortion, discontinuity, and overall quality, providing more interpretable outputs. Extensive experiments have been performed on the NISQA, BVCC, SOMOS and VoxSim speech quality datasets, using open-source auditory LLMs such as SALMONN, Qwen-Audio, and Qwen2-Audio. For the natural language descriptions task, a commercial model Google Gemini 1.5 Pro is also evaluated. The results demonstrate that auditory LLMs achieve competitive performance compared to state-of-the-art task-specific small models in predicting MOS and SIM, while also delivering promising results in A/B testing and natural language descriptions. Our data processing scripts and finetuned model checkpoints will be released upon acceptance., Comment: submitted to ICASSP 2025

Cardinality estimation (CE) plays a crucial role in many database-related tasks such as query generation, cost estimation, and join ordering. Lately, we have witnessed the emergence of numerous learned CE models. However, no single CE model is invincible when it comes to the datasets with various data distributions. To facilitate data-intensive applications with accurate and efficient cardinality estimation, it is important to have an approach that can judiciously and efficiently select the most suitable CE model for an arbitrary dataset. In this paper, we study a new problem of selecting the best CE models for a variety of datasets. This problem is rather challenging as it is hard to capture the relationship from various datasets to the performance of disparate models. To address this problem, we propose a model advisor, named AutoCE, which can adaptively select the best model for a dataset. The main contribution of AutoCE is the learning-based model selection, where deep metric learning is used to learn a recommendation model and incremental learning is proposed to reduce the labeling overhead and improve the model robustness. We have integrated AutoCE into PostgreSQL and evaluated its impact on query optimization. The results showed that AutoCE achieved the best performance (27% better) and outperformed the baselines concerning accuracy (2.1 times better) and efficacy (4.2 times better).

Cardinality estimation (CE) plays a crucial role in database optimizer. We have witnessed the emergence of numerous learned CE models recently which can outperform traditional methods such as histograms and samplings. However, learned models also bring many security risks. For example, a query-driven learned CE model learns a query-to-cardinality mapping based on the historical workload. Such a learned model could be attacked by poisoning queries, which are crafted by malicious attackers and woven into the historical workload, leading to performance degradation of CE. In this paper, we explore the potential security risks in learned CE and study a new problem of poisoning attacks on learned CE in a black-box setting. Experiments show that PACE reduces the accuracy of the learned CE models by 178 times, leading to a 10 times decrease in the end-to-end performance of the target database.

We employ the diagrammatic Monte Carlo method based on lattice path-integral representaion of the particle sector and real-space diagrammatics of the phonon sector to study effects of optical phonon dispersion on Bose-Einstein condensation of bipolarons in the bond model. For dispersionless phonons with frequency $\omega$ this model was recently shown to give rise to small-size, light-mass bipolarons that undergo a superfluid transition at high values of the $T_c/\omega$ ratio. We find that for dispersive phonons, $T_c$ remains relatively high over a broader range of the coupling strength and even keeps increasing in the deep adiabatic regime both in two and three dimensions. This result implies that phonon dispersion is not a limiting factor in the search for new materials with high superconducting temperatures., Comment: 7 pages 6 figures

Sulfur-bearing molecules are commonly detected in dense cores within star-forming regions, but the total sulfur budget is significantly low, when compared to the interstellar medium (ISM) value. The properties of sulfur-bearing molecules are not well understood due to the absence of large sample studies with uniform observational configurations. To deepen our understanding of this subject, we conducted a study using ALMA 870 \micron~observations of 11 massive protoclusters. By checking the spectra of 248 dense cores in 11 massive protoclusters, a total of 10 sulfur-bearing species (CS, SO, \htcs, NS, \sot, \ttso, \tfsot, \ttsot, \seoo, \octfs) were identified. The parameters including systemic velocities, line widths, gas temperatures, column densities, and abundances were derived. Our results indicate that SO appears to be more easily detected in a wider range of physical environments than \htcs, despite these two species show similarities in gas distributions and abundances. \tfsot~and \htcs~are good tracers of the temperature of sulfur-bearing species, in which \htcs~traces the outer warm envelope and \tfsot~is associated with high-temperature central-regions. High-mass star-forming feedback (outflow and other non-thermal motions) significantly elevates the sulfur-bearing molecular abundances and detection rates specifically for \sot~and SO. A positive correlation between the \sot~abundance increasing factor ($F$) and temperatures suggests that \sot~could serve as a sulfur reservoir on the grain mantles of dense cores and then can be desorbed from dust to gas phase as the temperature rises. This work shows the importance of a large and unbiased survey to understand the sulfur depletion in dense cores.

Personalized search has been extensively studied in various applications, including web search, e-commerce, social networks, etc. With the soaring popularity of short-video platforms, exemplified by TikTok and Kuaishou, the question arises: can personalization elevate the realm of short-video search, and if so, which techniques hold the key? In this work, we introduce $\text{PR}^2$, a novel and comprehensive solution for personalizing short-video search, where $\text{PR}^2$ stands for the Personalized Retrieval and Ranking augmented search system. Specifically, $\text{PR}^2$ leverages query-relevant collaborative filtering and personalized dense retrieval to extract relevant and individually tailored content from a large-scale video corpus. Furthermore, it utilizes the QIN (Query-Dominate User Interest Network) ranking model, to effectively harness user long-term preferences and real-time behaviors, and efficiently learn from user various implicit feedback through a multi-task learning framework. By deploying the $\text{PR}^2$ in production system, we have achieved the most remarkable user engagement improvements in recent years: a 10.2% increase in CTR@10, a notable 20% surge in video watch time, and a 1.6% uplift of search DAU. We believe the practical insights presented in this work are valuable especially for building and improving personalized search systems for the short video platforms.

Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model. To explore new capabilities in language modeling for speech processing, we introduce the generative speech transcription error correction (GenSEC) challenge. This challenge comprises three post-ASR language modeling tasks: (i) post-ASR transcription correction, (ii) speaker tagging, and (iii) emotion recognition. These tasks aim to emulate future LLM-based agents handling voice-based interfaces while remaining accessible to a broad audience by utilizing open pretrained language models or agent-based APIs. We also discuss insights from baseline evaluations, as well as lessons learned for designing future evaluations., Comment: IEEE SLT 2024. The initial draft version has been done in December 2023. Post-ASR Text Processing and Understanding Community: https://huggingface.co/GenSEC-LLM

Diffusion-based generative models have recently achieved remarkable results in speech and vocal enhancement due to their ability to model complex speech data distributions. While these models generalize well to unseen acoustic environments, they may not achieve the same level of fidelity as the discriminative models specifically trained to enhance particular acoustic conditions. In this paper, we propose Ex-Diff, a novel score-based diffusion model that integrates the latent representations produced by a discriminative model to improve speech and vocal enhancement, which combines the strengths of both generative and discriminative models. Experimental results on the widely used MUSDB dataset show relative improvements of 3.7% in SI-SDR and 10.0% in SI-SIR compared to the baseline diffusion model for speech and vocal enhancement tasks, respectively. Additionally, case studies are provided to further illustrate and analyze the complementary nature of generative and discriminative models in this context.

PiNNAcLe is an implementation of our adaptive learn-on-the-fly algorithm for running machine-learning potential (MLP)-based molecular dynamics (MD) simulations -- an emerging approach to simulate the large-scale and long-time dynamics of systems where empirical forms of the PES are difficult to obtain. The algorithm aims to solve the challenge of parameterizing MLPs for large-time-scale MD simulations, by validating simulation results at adaptive time intervals. This approach eliminates the need of uncertainty quantification methods for labelling new data, and thus avoids the additional computational cost and arbitrariness thereof. The algorithm is implemented in the NextFlow workflow language (Di Tommaso et al., 2017). Components such as MD simulation and MLP engines are designed in a modular fashion, and the workflows are agnostic to the implementation of such modules. This makes it easy to apply the same algorithm to different references, as well as scaling the workflow to a variety of computational resources. The code is published under BSD 3-Clause License, the source code and documentation are hosted on Github. It currently supports MLP generation with the atomistic machine learning package PiNN (Shao et al., 2020), electronic structure calculations with CP2K (K\"uhne et al., 2020) and DFTB+ (Hourahine et al., 2020), and MD simulation with ASE (Larsen et al., 2017).

In this paper, we investigate the impact of loop quantum gravity (LQG) on extreme mass-ratio inspirals (EMRIs), and the results indicate that LQG effects cause the orbital decay to occur faster compared to the Schwarzschild case. Furthermore, we use the augmented analytic kludge approach to generate EMRI waveforms and study the LISA's capability to detect the LQG effect with faithfulness. Additionally, employing the Fisher information matrix method for parameter estimation, we estimate that after one year of observation, the uncertainty in $r_0$ reduces to approximately $6.59\times 10^{-4}$ with a signal-to-noise ratio of $49$., Comment: 22 pages,7 figures

Instruction fine-tuning (IFT) elicits instruction following capabilities and steers the behavior of large language models (LLMs) via supervised learning. However, existing models trained on open-source IFT datasets only have the ability to follow instructions from users, and often fail to follow complex role and rules specified by developers, a.k.a. system prompts. The ability to follow these roles and rules is essential for deployment, as it ensures that the model safely interacts with users within developer defined guidelines. To improve such role and rule following ability, we propose \model, an automated data generation pipeline that generates diverse roles and rules from existing IFT instructions, along with corresponding responses. This data can then be used to train models that follow complex system prompts. The models are evaluated on our newly created benchmarks for role and rule following ability, as well as standard instruction-following benchmarks and general NLP tasks. Our framework significantly improves role and rule following capability in LLMs, as evidenced by over 25% increase in pass-rate on rule adherence, i.e. following all requirements, in our experiments with the Alpaca and Ultrachat datasets. Moreover, our models achieves this increase without any regression on popular instruction following benchmarks.

Tactile sensation plays a crucial role in the development of multi-modal large models and embodied intelligence. To collect tactile data with minimal cost as possible, a series of studies have attempted to generate tactile images by vision-to-touch image translation. However, compared to text modality, visual modality-driven tactile generation cannot accurately depict human tactile sensation. In this work, we analyze the characteristics of tactile images in detail from two granularities: object-level (tactile texture, tactile shape), and sensor-level (gel status). We model these granularities of information through text descriptions and propose a fine-grained Text-to-Touch generation method (TextToucher) to generate high-quality tactile samples. Specifically, we introduce a multimodal large language model to build the text sentences about object-level tactile information and employ a set of learnable text prompts to represent the sensor-level tactile information. To better guide the tactile generation process with the built text information, we fuse the dual grains of text information and explore various dual-grain text conditioning methods within the diffusion transformer architecture. Furthermore, we propose a Contrastive Text-Touch Pre-training (CTTP) metric to precisely evaluate the quality of text-driven generated tactile data. Extensive experiments demonstrate the superiority of our TextToucher method. The source codes will be available at \url{https://github.com/TtuHamg/TextToucher}.

This is an index of the contributions by the Giant Radio Array for Neutrino Detection (GRAND) Collaboration to the 10th International Workshop on Acoustic and Radio EeV Neutrino Detection Activities (ARENA 2024, University of Chicago, June 11-14, 2024). The contributions include an overview of GRAND in its present and future incarnations, methods of radio-detection that are being developed for them, and ongoing joint work between the GRAND and BEACON experiments., Comment: Note: To access the list of contributions, please follow the "HTML" link that can be found on the arXiv page

The manipulation and transformation of quantum resources are key parts of quantum mechanics. Among them, asymmetry is one of the most useful operational resources, which is widely used in quantum clocks, quantum metrology, and other tasks. Recent studies have shown that the asymmetry of quantum states can be significantly amplified with the assistance of correlating catalysts which are finite-dimensional auxiliaries. In the experiment, we perform translationally invariant operations, ensuring that the asymmetric resources of the entire system remain non-increasing, on a composite system composed of a catalytic system and a quantum system. The experimental results demonstrate an asymmetry amplification of 0.0172\pm0.0022 in the system following the catalytic process. Our work showcases the potential of quantum catalytic processes and is expected to inspire further research in the field of quantum resource theories., Comment: 17pages,7figures

The rapid advancement in point cloud processing technologies has significantly increased the demand for efficient and compact models that achieve high-accuracy classification. Knowledge distillation has emerged as a potent model compression technique. However, traditional KD often requires extensive computational resources for forward inference of large teacher models, thereby reducing training efficiency for student models and increasing resource demands. To address these challenges, we introduce an innovative offline recording strategy that avoids the simultaneous loading of both teacher and student models, thereby reducing hardware demands. This approach feeds a multitude of augmented samples into the teacher model, recording both the data augmentation parameters and the corresponding logit outputs. By applying shape-level augmentation operations such as random scaling and translation, while excluding point-level operations like random jittering, the size of the records is significantly reduced. Additionally, to mitigate the issue of small student model over-imitating the teacher model's outputs and converging to suboptimal solutions, we incorporate a negative-weight self-distillation strategy. Experimental results demonstrate that the proposed distillation strategy enables the student model to achieve performance comparable to state-of-the-art models while maintaining lower parameter count. This approach strikes an optimal balance between performance and complexity. This study highlights the potential of our method to optimize knowledge distillation for point cloud classification tasks, particularly in resource-constrained environments, providing a novel solution for efficient point cloud analysis.

Advances in self-supervised learning are essential for enhancing feature extraction and understanding in point cloud processing. This paper introduces PMT-MAE (Point MLP-Transformer Masked Autoencoder), a novel self-supervised learning framework for point cloud classification. PMT-MAE features a dual-branch architecture that integrates Transformer and MLP components to capture rich features. The Transformer branch leverages global self-attention for intricate feature interactions, while the parallel MLP branch processes tokens through shared fully connected layers, offering a complementary feature transformation pathway. A fusion mechanism then combines these features, enhancing the model's capacity to learn comprehensive 3D representations. Guided by the sophisticated teacher model Point-M2AE, PMT-MAE employs a distillation strategy that includes feature distillation during pre-training and logit distillation during fine-tuning, ensuring effective knowledge transfer. On the ModelNet40 classification task, achieving an accuracy of 93.6\% without employing voting strategy, PMT-MAE surpasses the baseline Point-MAE (93.2\%) and the teacher Point-M2AE (93.4\%), underscoring its ability to learn discriminative 3D point cloud representations. Additionally, this framework demonstrates high efficiency, requiring only 40 epochs for both pre-training and fine-tuning. PMT-MAE's effectiveness and efficiency render it well-suited for scenarios with limited computational resources, positioning it as a promising solution for practical point cloud analysis.

This study addresses the computational inefficiencies in point cloud classification by introducing novel MLP-based architectures inspired by recent advances in CNN optimization. Traditional neural networks heavily rely on multiplication operations, which are computationally expensive. To tackle this, we propose Add-MLP and Shift-MLP, which replace multiplications with addition and shift operations, respectively, significantly enhancing computational efficiency. Building on this, we introduce SA-MLP, a hybrid model that intermixes alternately distributed shift and adder layers to replace MLP layers, maintaining the original number of layers without freezing shift layer weights. This design contrasts with the ShiftAddNet model from previous literature, which replaces convolutional layers with shift and adder layers, leading to a doubling of the number of layers and limited representational capacity due to frozen shift weights. Moreover, SA-MLP optimizes learning by setting distinct learning rates and optimizers specifically for the adder and shift layers, fully leveraging their complementary strengths. Extensive experiments demonstrate that while Add-MLP and Shift-MLP achieve competitive performance, SA-MLP significantly surpasses the multiplication-based baseline MLP model and achieves performance comparable to state-of-the-art MLP-based models. This study offers an efficient and effective solution for point cloud classification, balancing performance with computational efficiency.

In this paper, Whisper, a large-scale pre-trained model for automatic speech recognition, is proposed to apply to speaker verification. A partial multi-scale feature aggregation (PMFA) approach is proposed based on a subset of Whisper encoder blocks to derive highly discriminative speaker embeddings.Experimental results demonstrate that using the middle to later blocks of the Whisper encoder keeps more speaker information. On the VoxCeleb1 and CN-Celeb1 datasets, our system achieves 1.42% and 8.23% equal error rates (EERs) respectively, receiving 0.58% and 1.81% absolute EER reductions over the ECAPA-TDNN baseline, and 0.46% and 0.97% over the ResNet34 baseline. Furthermore, our results indicate that using Whisper models trained on multilingual data can effectively enhance the model's robustness across languages. Finally, the low-rank adaptation approach is evaluated, which reduces the trainable model parameters by approximately 45 times while only slightly increasing EER by 0.2%., Comment: Accepted by Interspeech 2024

We consider a wide class of fully nonlinear integro-differential equations that degenerate when the gradient of the solution vanishes. By using compactness and perturbation arguments, we give a complete characterization of the regularity of viscosity solutions according to different diffusion orders. More precisely, when the order of the fractional diffusion is sufficiently close to 2, we obtain H\"{o}lder continuity for the gradient of any viscosity solutions and further derive an improved gradient regularity estimate at the origin. For the order of the fractional diffusion in the interval $(1, 2)$, we prove that there is at least one solution of class $C^{1, \alpha}_{\rm loc}$. Additionally, if the order of the fractional diffusion is in the interval $(0,1]$, the local H\"{o}lder continuity of solutions is inferred.

The present work is motivated by the mixing mechanism in inertial confinement fusion (ICF), which remarkably degrades the ignition performance. The mixing is a direct result of velocity disequilibrium and affected by temperature disequilibrium.To deal with these disequilibria, we propose a fully-disequilibrium hydrodynamic model (termed as Baer-Nunziato-Zeldovich model or BNZ model) with 9 equations, 4 temperatures (two for ions, and two for electrons), 4 pressures (two for ions, and two for electrons) and 2 ion velocities for two-component dense plasma flows. The model can be used for describing both grain and atomic mixing by choosing corresponding relaxation mechanism. The derivation starts from a multi-component conservative entropy-dissipative Bhatnagar-Gross-Krook (BGK) model to obtain a 14-equation model. It is then reduced to more practical 9-equation BNZ model and further to simpler 8-equation model, 6-equation model (the Kapila-Zeldovich model or the KZ model), and the 5-equation model. The reductions are performed on the basis of various relaxation time evaluations, the plasma neutrality and the smallness of the electron mass. The models are thermodynamics-compatible, and capable of dealing with mechanical and thermal disequilibria at both atomic and grain scales. Moreover, the corresponding hydrodynamic subsystems are shown to be hyperbolic and solved with the Godunov finite volume method. The BNZ model and solution methods are verified and validated by some benchmark problems and 1D-2D comparisons. We then consider the velocity, pressure and temperature disequilibria during the passage of an ablation shock in ICF.

The low-energy effective action of String Theory introduces corrections to the dilaton-graviton sector, resulting in deformed black holes beyond general relativity. We analyze extreme mass-ratio inspiral systems (EMRIs), where a stellar-mass object spirals into a slowly rotating supermassive black hole including a distinct deviation parameter. This study examines the effects of this deformation on gravitational wave fluxes, orbital evolution, and phase dynamics, incorporating leading-order post-Newtonian corrections. With one-year observations of EMRIs, we employ the Fisher information matrix method to evaluate the potential for detecting deviations from general relativity through space-based gravitational wave detectors that utilize time-delay interferometry to suppress laser noise. The constraint on modified black holes, $\Delta\alpha \preceq 10^{-5}$, is almost the same with and without the time-delay interferometry combination. This analysis enhances our understanding and underscores the crucial role of observations in advancing gravitational phenomena within String Theory., Comment: 19 pages, 4 figures; Added some references and revised some sentences; Comments are welcome

Significant advancements in Large Multimodal Models (LMMs) have enabled them to tackle complex problems involving visual-mathematical reasoning. However, their ability to identify geometric elements remains understudied. To bridge this gap, we introduce Tangram, a novel benchmark designed to evaluate the performance of LMMs on geometric element recognition. Tangram includes 1,080 diverse geometric diagrams sourced from primary and secondary school exams, competitions, and textbooks, covering from simple basic geometric shapes to complex combinations. Each diagram is associated with four questions, resulting in a total of 4,320 visual-question-answer pairs. Unlike existing benchmarks that seek higher-level cognition and reasoning, Tangram focuses on the understanding of geometric elements, requiring models to perform a "simple but interesting" counting task. Systematic evaluation of 10 prominent LMMs, such as GPT-4o and Claude 3.5 Sonnet, shows that even in the seemingly simple task, these models still face significant challenges. Notably, the overall accuracy of the top performer across all tested models is only 56.8%, marking a significant gap when compared to human performance. These findings highlight the limitations of current multimodal artificial intelligence systems in handling basic perception tasks, and will inspire the development of the next generation of expert-level multimodal foundational models. The Tangram and evaluation code will be available soon., Comment: 12 pages, 7 figures

Oxygen transport membranes(OTMs)have provided great opportunities in the last decades but are suffering from the trade-off effect between stability and oxygen permeability. Here, we report a group of new planar dual-phase mixed ionic-electronic conducting (MIEC) OTMs consisting of CO$_2$$_{0.85}$Nd$_{0.1}$Cu$_{0.05}$O$_2$ (CNCO) and Nd$_x$Sr$_{1-x}$Fe$_{1-y}$Cu$_y$O$_3$(NSFCO; $x = 0.4, 0.6$; $y = 0.05, 0.1$) phases, showing excellent oxygen permeability while comparable CO$_2$-resistant stability. The substitution of Cu as a bifunctional additive decreases the sintering temperature and enhances bulk diffusion and oxygen permeability with the co-doping of Nd.The oxygen permeation fluxes reached 2.62 and 1.52 mL min$^{-1}$ cm$^{-2}$ at 1000$^\circ$C through the optimal 60wt%Ce0.85Nd0.1Cu0.05O2-40wt%Nd0.4Sr0.6Fe0.9Cu0.1O3 (CNCO-NSFCO41) composition with He and CO$_2$ sweeping, respectively, higher than all reported dense dual-phase OTMs. Such excellent CO$_2$-tolerant permeability meets the needs of potential industrial applications. Analysis with Zhu's oxygen permeation model shows lower bulk diffusion resistance of CNCO-NSFCO41 than that of reported 60wt%Ce0.85Pr0.1Cu0.05O2-40wt%Pr0.4Sr0.6Fe0.9Cu0.1O3(CPCO-PSFCO41)and more limitation by the interfacial exchange at high temperature. All the prepared OTMs also show good long-term stability over 100 hours in both atmospheres. Our results confirm the excellent oxygen permeability and stability under a high-concentration CO2 atmosphere, providing a material candidate for CO2 capture in oxyfuel combustion., Comment: 36 pages, 6 figures

Exploring high-efficiency and low-cost electrocatalysts is valuable for water-splitting technologies. Recently, Si-group compounds have attracted increasing attention in electrocatalysis, considering the abundant Si-group elements on Earth. However, Si-group compounds for HER electrocatalysis have not been systematically studied. In this study, we unveil the activity trends of non-noble metal catalyst A15-type V3M (i.e., V3Si, V3Ge, and V3Sn) superconductors and show that V3Si is the most efficient HER catalyst because of the high electronic conductivity and suitable d-band center. Among them, the V3Si only requires 33.4 mV to reach 10 mA cm-2, and only 57.6 mV and 114.6 mV are required to attain a high current density of 100 mA cm-2 and 500 mA cm-2, respectively. These low overpotentials are close to the 34.3 mV at 10 mA cm-2 of state-of-art Pt/C (20 %) but superior to 168.5 mV of Pt/C (20 %) at 100 mA cm-2. Furthermore, the V3Si illustrates exceptional durability with no obvious decay in the 120 h at the different current densities (i.e., 10 - 250 mA cm-2). The excellent HER activity of V3Si alloy can be ascribed to the synergies of superior electronic conductivity and suitable d-band center. Moreover, DFT calculations reveal that the absolute hydrogen adsorption Gibbs free energy is decreased after introducing the V to Si. Beyond offering a stable and high-performance electrocatalyst in an acidic medium, this work inspires the rational design of desirable silicide electrocatalysts., Comment: 25 pages, 5 figures

In this paper, we discuss the impact of tellurium (Te) doping on the spinel superconductor CuRh2Se4. We conducted a comprehensive evaluation of the structural and superconducting properties of the system using various techniques, including X-ray diffraction (XRD), resistivity, magnetization, and specific heat measurements. Based on our XRD analysis, we found that the spinel superconductor CuRh2Se4-xTex crystallizes in the space group Fd3m(227) with x in the region of 0 to 0.28, while the layered compound CuRh2Se4-xTex crystallizes in the space group P3m1 (164) with x in the region of 2.8 to 4.0. The upper critical magnetic field can be increased from 0.95(2) T for CuRh2Se4 to 3.44(1) T for CuRh2Se3.72Te0.28 by doping with elemental Te. However, the layered compound CuRh2Se4-xTex did not exhibit superconducting properties. Besides, the specific heat measurements of CuRh2Se4-xTex (x = 0, 0.1, 0.28) indicate that the Te element doping affects the electronic structure and interactions of the material and breaks the stability of the superconducting pairing, which leads to a decrease in the Tc. Finally, we show the electronic phase diagram of Tc with Te doping to summarise our findings., Comment: 25 pages, 6 figures, 1 table

Topological semimetals with protected surface states mark a new paradigm of research beyond the early landmarks of band-structure engineering, allowing fabrication of efficient catalyst to harness the rich metallic surface states to activate specific chemical processes. Herein, we demonstrate a facile solid-phase method for in-situ doping of Ir at the Os sites in the Os3Sn7, an alloy with topological states, which significantly improves the photocatalytic performance for the reduction of CO2 to CO and CH4. Experimental evidence combined with theoretical calculations reveal that the nontrivial topological surface states greatly accelerate charge-separation/electron-enrichment and adsorption/activation of CO2 molecules, rendering highly efficient reaction channels to stimulate the formation of *COOH and *CO, as well CHO*. This work shows the promise of achieving high photocatalytic performances with synthesizing topological catalysts and provides hints on the design of novel topological catalysts with superior photoactivity towards the CO2 reduction reaction., Comment: 33 Pages, 6 Figures, 1 Table

Despite ongoing research, the rational design of nontrivial topological semimetal surface states for the selective photocatalytic CO$_2$ conversion into valuable products remains full of challenges. Herein, we present the synthesis of 1T-OsCoTe$_2$ for the photoreduction upgrading of CO$_2$ to tricarbon alkane C$_3$H$_8$,by the integration of experimental work and theory calculation. Experimental studies suggested a high electron based selectivity of 71.2% for C$_3$H$_8$ and an internal quantum efficiency of 54.6% at 380 nm. In-situ X-ray photoelectron spectroscopy and X-ray absorption fine structure spectroscopy demonstrated that Co and Os atoms coordinated with Te atoms enable an efficient Os-Te-Co electron transfer to activate the generation of *CH$_3$,*CHOCO and *CH$_2$OCOCO. Density functional theory calculations further confirmed Os-Te-Co electron bridging on the improved CO$_2$ conversion kinetics. To our knowledge, this is the first report suggesting the role of Os atoms in accelerating the photocatalytic CO$_2$ conversion activity of the topological semimetal 1T-OsCoTe$_2$., Comment: 31 pages, 6 Figures

The operation of upcoming ultra-high-energy cosmic-ray, gamma-ray, and neutrino radio-detection experiments, like the Giant Radio Array for Neutrino Detection (GRAND), poses significant computational challenges involving the production of numerous simulations of particle showers and their detection, and a high data throughput. GRANDlib is an open-source software tool designed to meet these challenges. Its primary goal is to perform end-to-end simulations of the detector operation, from the interaction of ultra-high-energy particles, through -- by interfacing with external air-shower simulations -- the ensuing particle shower development and its radio emission, to its detection by antenna arrays and its processing by data-acquisition systems. Additionally, GRANDlib manages the visualization, storage, and retrieval of experimental and simulated data. We present an overview of GRANDlib to serve as the basis of future GRAND analyses., Comment: 11 pages, 9 figures, plus appendices