Search

Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes, which is important for the reliable deployment of machine learning models in the open world. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. However, existing methods generally focus on excavating the discriminative information from a single input, which implicitly limits its representation dimension. In this work, we introduce a novel perspective, i.e., employing different common corruptions on the input space, to expand that. We reveal an interesting phenomenon termed confidence mutation, where the confidence of OOD data can decrease significantly under the corruptions, while the ID data shows a higher confidence expectation considering the resistance of semantic features. Based on that, we formalize a new scoring method, namely, Confidence aVerage (CoVer), which can capture the dynamic differences by simply averaging the scores obtained from different corrupted inputs and the original ones, making the OOD and ID distributions more separable in detection tasks. Extensive experiments and analyses have been conducted to understand and verify the effectiveness of CoVer. The code is publicly available at: https://github.com/tmlr-group/CoVer., Comment: accepted by NeurIPS 2024

Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many papers have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this paper, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out the through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically., Comment: Accepted in SCIENCE CHINA Information Sciences

Realizing large-scale quantum networks requires the generation of high-fidelity quantum entanglement states between remote quantum nodes, a key resource for quantum communication, distributed computation and sensing applications. However, entanglement distribution between quantum network nodes is hindered by optical transmission loss and local operation errors. Here, we propose a novel quantum repeater architecture that synergistically integrates Rydberg atom quantum processors with optical cavities to overcome these challenges. Our scheme leverages cavity-mediated interactions for efficient remote entanglement generation, followed by Rydberg interaction-based entanglement purification and swapping. Numerical simulations, incorporating realistic experimental parameters, demonstrate the generation of Bell states with 99\% fidelity at rates of 1.1\,kHz between two nodes in local-area network (distance $0.1\,\mathrm{km}$), and can be extend to metropolitan-area ($25\,\mathrm{km}$) or intercity ($\mathrm{250\,\mathrm{km}}$, with the assitance of frequency converters) network with a rate of 0.1\,kHz. This scalable approach opens up near-term opportunities for exploring quantum network applications and investigating the advantages of distributed quantum information processing., Comment: 3 figures

A number of experiments show that the decay lifetimes of muons bound to atomic nuclei are longer than the decay lifetimes of free muons. In this paper, a scheme of extending quantum mechanics (EQM) is proposed to resolve this problem. The Schr$\ddot{\text{o}}$dinger's equation is obtained to prove the validation of this attempt. The decay ratio of bound muons is also calculated in EQM, and the result is in good agreement with the experimental data., Comment: 5 pages, 1 figure, 2 tables

Hypomimia is a non-motor symptom of Parkinson's disease that manifests as delayed facial movements and expressions, along with challenges in articulation and emotion. Currently, subjective evaluation by neurologists is the primary method for hypomimia detection, and conventional rehabilitation approaches heavily rely on verbal prompts from rehabilitation physicians. There remains a deficiency in accessible, user-friendly and scientifically rigorous assistive tools for hypomimia treatments. To investigate this, we developed HypomimaCoach, an Action Unit (AU)-based digital therapy system for hypomimia detection and rehabilitation in Parkinson's disease. The HypomimaCoach system was designed to facilitate engagement through the incorporation of both relaxed and controlled rehabilitation exercises, while also stimulating initiative through the integration of digital therapies that incorporated traditional face training methods. We extract action unit(AU) features and their relationship for hypomimia detection. In order to facilitate rehabilitation, a series of training programmes have been devised based on the Action Units (AUs) and patients are provided with real-time feedback through an additional AU recognition model, which guides them through their training routines. A pilot study was conducted with seven participants in China, all of whom exhibited symptoms of Parkinson's disease hypomimia. The results of the pilot study demonstrated a positive impact on participants' self-efficacy, with favourable feedback received. Furthermore, physician evaluations validated the system's applicability in a therapeutic setting for patients with Parkinson's disease, as well as its potential value in clinical applications.

In this paper, we study the thermodynamics of Schwarzschild-anti-de Sitter black holes within the framework of non-commutative geometry. By solving the Einstein's equations, we derive the corrected Schwarzschild-AdS black hole with Lorentzian distribution and analyze the thermodynamics. Our results confirm that if the energy-momentum tensor outside the event horizon is related to the mass of the black hole, the conventional first law of thermodynamics will be violated. The study of criticality reveals that the black hole undergoes a small black hole-large black hole phase transition similar to that of the Van der Waals system, with a critical point and a critical ratio slightly smaller than that of the Van der Waals fluid. As the non-commutative parameter increases, the phase transition process shortens, leading to a critical point, and ultimately to the disappearance of the phase transition. The violation of the conventional first law results in a discontinuity of the Gibbs free energy during the phase transition, indicating the occurrence of zeroth-order phase transition. Moreover, we investigate the Joule-Thomson expansion, obtaining the minimum inversion temperature and the minimum inversion mass., Comment: 37pages, 11figures

The importance of Very Long Baseline Interferometry (VLBI) for pulsar research is becoming increasingly prominent and receiving more and more attention. In this paper, we present pathfinding pulsar observation results with the Chinese VLBI Network (CVN) incorporating the Five-hundred-meter Aperture Spherical radio Telescope (FAST). On MJD 60045 (April 11th, 2023), PSRs B0919+06 and B1133+16 were observed with the phase-referencing mode in the L-band using four radio telescopes (FAST, TianMa, Haoping and Nanshan) and correlated with the pulsar binning mode of the distributed FX-style software correlator in Shanghai. After further data processing with the NRAO Astronomical Image Processing System (AIPS), we detected these two pulsars and fitted their current positions with accuracy at the milliarcsecond level. By comparison, our results show significantly better agreement with predicted values based on historical VLBI observations than that with previous timing observations, as pulsar astrometry with the VLBI provides a more direct and model-independent method for accurately obtaining related parameters., Comment: Accepted by the Chinese Physics Letters

The switching and control of optical fields based on nonlinear optical effects are often limited to relatively weak nonlinear susceptibility and strong optical pump fields. Here, an optical medium with programmable susceptibility tensor based on polarizable atoms is proposed. Under a structured optical pump, the ground state population of atoms could be efficiently controlled by tuning the chirality and intensity of optical fields, and thus the optical response of the medium is programmable in both space and time. We demonstrate the potential of this approach by engineering the spatial distribution of the complex susceptibility tensor of the medium in photonic structures to realize nonreciprocal optical effects. Specifically, we investigate the advantages of chiral interaction between atoms and photons in an atom-cladded waveguide, theoretically showing that reconfigurable, strong, and fastly switchable isolation of optical signals in a selected optical mode is possible. The susceptibility-programmable medium provides a promising way to efficiently control the optical field, opening up a wide range of applications for integrated photonic devices and structured optics., Comment: 7 pages, 4 figures

Network optimization is a fundamental challenge in the Internet of Things (IoT) network, often characterized by complex features that make it difficult to solve these problems. Recently, generative diffusion models (GDMs) have emerged as a promising new approach to network optimization, with the potential to directly address these optimization problems. However, the application of GDMs in this field is still in its early stages, and there is a noticeable lack of theoretical research and empirical findings. In this study, we first explore the intrinsic characteristics of generative models. Next, we provide a concise theoretical proof and intuitive demonstration of the advantages of generative models over discriminative models in network optimization. Based on this exploration, we implement GDMs as optimizers aimed at learning high-quality solution distributions for given inputs, sampling from these distributions during inference to approximate or achieve optimal solutions. Specifically, we utilize denoising diffusion probabilistic models (DDPMs) and employ a classifier-free guidance mechanism to manage conditional guidance based on input parameters. We conduct extensive experiments across three challenging network optimization problems. By investigating various model configurations and the principles of GDMs as optimizers, we demonstrate the ability to overcome prediction errors and validate the convergence of generated solutions to optimal solutions.

We study the null geodesics in the extremal Kerr-Newman exterior. We clarify the roots of the radial potential and obtain the parameter space of the azimuthal angular momentum and the Carter constant of the light rays for varieties of the orbits. It is known that one of the unique features of extremal black holes for the null geodesics is the existence of the stable double root at the horizon, giving rise to the stable spherical motion. For the black hole's spin $a

Ultraluminous X-ray sources (ULXs) are non-nuclear point-like objects observed with extremely high X-ray luminosity that exceeds the Eddington limit of a $\rm10\,M_\odot$ black hole. A fraction of ULXs has been confirmed to contain neutron star (NS) accretors due to the discovery of their X-ray pulsations. The donors detected in NS ULXs are usually luminous massive stars because of the observational biases. Recently, the He donor star in NGC 247 ULX-1 has been identified, which is the first evidence of a He donor star in ULXs. In this paper, we employed the stellar evolution code MESA to investigate the formation of ULXs through the NS+He star channel, in which a He star transfers its He-rich material onto the surface of a NS via Roche-lobe overflow. We evolved a large number of NS+He star systems and provided the parameter space for the production of ULXs. We found that the initial NS+He star systems should have $\rm\sim 0.7-2.6 \, M_\odot$ He star and $\rm \sim 0.1-2500\, d$ orbital period for producing ULXs, eventually evolving into intermediate-mass binary pulsars. According to binary population synthesis calculations, we estimated that the Galactic rate of NS ULXs with He donor stars is in the range of $\sim1.6-4.0\times10^{-4}\,{\rm yr}^{-1}$, and that there exist $\sim7-20$ detectable NS ULXs with He donor stars in the Galaxy., Comment: 19 pages, 6 figures, 2 tables

Recent large language models (LLMs) have demonstrated promising capabilities in modeling real-world knowledge and enhancing knowledge-based generation tasks. In this paper, we further explore the potential of using LLMs to aid in the design of soft modular robots, taking into account both user instructions and physical laws, to reduce the reliance on extensive trial-and-error experiments typically needed to achieve robot designs that meet specific structural or task requirements. Specifically, we formulate the robot design process as a sequence generation task and find that LLMs are able to capture key requirements expressed in natural language and reflect them in the construction sequences of robots. To simplify, rather than conducting real-world experiments to assess design quality, we utilize a simulation tool to provide feedback to the generative model, allowing for iterative improvements without requiring extensive human annotations. Furthermore, we introduce five evaluation metrics to assess the quality of robot designs from multiple angles including task completion and adherence to instructions, supporting an automatic evaluation process. Our model performs well in evaluations for designing soft modular robots with uni- and bi-directional locomotion and stair-descending capabilities, highlighting the potential of using natural language and LLMs for robot design. However, we also observe certain limitations that suggest areas for further improvement., Comment: 8 pages, 7 figures

Traditional methods for optimizing light source emissions rely on a time-consuming trial-and-error approach. While in-situ optimization of light source gain media emission during growth is ideal, it has yet to be realized. In this work, we integrate in-situ reflection high-energy electron diffraction (RHEED) with machine learning (ML) to correlate the surface reconstruction with the photoluminescence (PL) of InAs/GaAs quantum dots (QDs), which serve as the active region of lasers. A lightweight ResNet-GLAM model is employed for the real-time processing of RHEED data as input, enabling effective identification of optical performance. This approach guides the dynamic optimization of growth parameters, allowing real-time feedback control to adjust the QDs emission for lasers. We successfully optimized InAs QDs on GaAs substrates, with a 3.2-fold increase in PL intensity and a reduction in full width at half maximum (FWHM) from 36.69 meV to 28.17 meV under initially suboptimal growth conditions. Our automated, in-situ self-optimized lasers with 5-layer InAs QDs achieved electrically pumped continuous-wave operation at 1240 nm with a low threshold current of 150 A/cm2 at room temperature, an excellent performance comparable to samples grown through traditional manual multi-parameter optimization methods. These results mark a significant step toward intelligent, low-cost, and reproductive light emitters production., Comment: 5 figures

Benefiting from improved stability due to stronger interlayer van der Waals interactions, few-layer fullerene networks are experimentally more accessible compared to monolayer polymeric C$_{60}$. However, there is a lack of systematic theoretical studies on the material properties of few-layer C$_{60}$ networks. Here, we compare the structural, electronic and optical properties of bilayer and monolayer fullerene networks. The band gap and band-edge positions remain mostly unchanged after stacking two layers into a bilayer, enabling the bilayer to be almost as efficient a photocatalyst as the monolayer. The effective mass ratio along different directions is varied for conduction band states due to interlayer interactions,leading to enhanced anisotropy in carrier transport. Additionally, stronger exciton absorption is found in the bilayer than that in the monolayer over the entire visible light range, rendering the bilayer a more promising candidate for photovoltaics. Moreoever, the polarisation dependence of optical absorption in the bilayer is increased in the red-yellow light range, offering unique opportunities in photonics and display technologies with tailored optical properties over specific directions. Our study provides strategies to tune electronic and optical properties of 2D polymeric C$_{60}$ via the introduction of stacking degrees of freedom., Comment: 7 pages, 4 figures

Large language models (LLMs) have revolutionized numerous applications, yet their deployment remains challenged by memory constraints on local devices. While scaling laws have enhanced LLM capabilities, the primary bottleneck has shifted from \textit{capability} to \textit{availability}, emphasizing the need for efficient memory management. Traditional compression methods, such as quantization, often require predefined compression ratios and separate compression processes for each setting, complicating deployment in variable memory environments. In this paper, we introduce \textbf{BitStack}, a novel, training-free weight compression approach that enables megabyte-level trade-offs between memory usage and model performance. By leveraging weight decomposition, BitStack can dynamically adjust the model size with minimal transmission between running memory and storage devices. Our approach iteratively decomposes weight matrices while considering the significance of each parameter, resulting in an approximately 1-bit per parameter residual block in each decomposition iteration. These blocks are sorted and stacked in storage as basic transmission units, with different quantities loaded based on current memory availability. Extensive experiments across a wide range of tasks demonstrate that, despite offering fine-grained size control, BitStack consistently matches or surpasses strong quantization baselines, particularly at extreme compression ratios. To the best of our knowledge, this is the first decomposition-based method that effectively bridges the gap to practical compression techniques like quantization. Code is available at https://github.com/xinghaow99/BitStack.

Detecting Human-Object Interactions (HOI) in zero-shot settings, where models must handle unseen classes, poses significant challenges. Existing methods that rely on aligning visual encoders with large Vision-Language Models (VLMs) to tap into the extensive knowledge of VLMs, require large, computationally expensive models and encounter training difficulties. Adapting VLMs with prompt learning offers an alternative to direct alignment. However, fine-tuning on task-specific datasets often leads to overfitting to seen classes and suboptimal performance on unseen classes, due to the absence of unseen class labels. To address these challenges, we introduce a novel prompt learning-based framework for Efficient Zero-Shot HOI detection (EZ-HOI). First, we introduce Large Language Model (LLM) and VLM guidance for learnable prompts, integrating detailed HOI descriptions and visual semantics to adapt VLMs to HOI tasks. However, because training datasets contain seen-class labels alone, fine-tuning VLMs on such datasets tends to optimize learnable prompts for seen classes instead of unseen ones. Therefore, we design prompt learning for unseen classes using information from related seen classes, with LLMs utilized to highlight the differences between unseen and related seen classes. Quantitative evaluations on benchmark datasets demonstrate that our EZ-HOI achieves state-of-the-art performance across various zero-shot settings with only 10.35% to 33.95% of the trainable parameters compared to existing methods. Code is available at https://github.com/ChelsieLei/EZ-HOI., Comment: Accepted by NeurIPS 2024

This paper investigates an under-explored challenge in large language models (LLMs): chain-of-thought prompting with noisy rationales, which include irrelevant or inaccurate reasoning thoughts within examples used for in-context learning. We construct NoRa dataset that is tailored to evaluate the robustness of reasoning in the presence of noisy rationales. Our findings on NoRa dataset reveal a prevalent vulnerability to such noise among current LLMs, with existing robust methods like self-correction and self-consistency showing limited efficacy. Notably, compared to prompting with clean rationales, base LLM drops by 1.4%-19.8% in accuracy with irrelevant thoughts and more drastically by 2.2%-40.4% with inaccurate thoughts. Addressing this challenge necessitates external supervision that should be accessible in practice. Here, we propose the method of contrastive denoising with noisy chain-of-thought (CD-CoT). It enhances LLMs' denoising-reasoning capabilities by contrasting noisy rationales with only one clean rationale, which can be the minimal requirement for denoising-purpose prompting. This method follows a principle of exploration and exploitation: (1) rephrasing and selecting rationales in the input space to achieve explicit denoising and (2) exploring diverse reasoning paths and voting on answers in the output space. Empirically, CD-CoT demonstrates an average improvement of 17.8% in accuracy over the base model and shows significantly stronger denoising capabilities than baseline methods. The source code is publicly available at: https://github.com/tmlr-group/NoisyRationales., Comment: Accepted by NeurIPS 2024

Muons, which play a crucial role in both fundamental and applied physics, have traditionally been generated through proton accelerators or from cosmic rays. With the advent of ultra-short high-intensity lasers capable of accelerating electrons to GeV levels, it has become possible to generate muons in laser laboratories. In this work, we show the first proof of principle experiment for novel muon production with an ultra-short, high-intensity laser device through GeV electron beam bombardment on a lead converter target. The muon physical signal is confirmed by measuring its lifetime which is the first clear demonstration of laser-produced muons. Geant4 simulations were employed to investigate the photo-production, electro-production, and Bethe-Heitler processes response for muon generation and their subsequent detection. The results show that the dominant contributions of muons are attributed to the photo-production/electro-production and a significant yield of muons up to 0.01 $\mu$/$e^-$ out of the converter target could be achieved. This laser muon source features compact, ultra-short pulse and high flux. Moreover, its implementation in a small laser laboratory is relatively straightforward, significantly reducing the barriers to entry for research in areas such as muonic X-ray elemental analysis, muon spin spectroscopy and so on.

We explore the gravitational-wave emission from head-on collisions of equal-mass solitonic boson-star binaries from simulations spanning a two-dimensional parameter space, consisting of the central scalar-field amplitude of the stars and the solitonic potential parameter. We report the gravitational-wave energies emitted by boson-star binaries which, due to their combination of moderately high compactness with significant deformability, we often find to be louder by up to an order of magnitude than analogous black-hole collisions. The dependence of the radiated energy on the boson-star parameters exhibits striking needle-sharp features and discontinuous jumps to the value emitted by black-hole binaries. We explain these features in terms of the solitonic potential and the stability properties of the respective individual stars., Comment: 17 pages, 10 figures

The topology of a pure state of two entangled photons is leveraged to provide a discretization of quantum information. Since discrete signals are inherently more resilient to the effects of perturbations, this discrete class of entanglement observables may offer an advantage against noise. Establishing this is the primary objective of this paper. We develop a noise model that exploits the specific form of such topological wave functions - an entangled state of two photons with one in an orbital angular momentum state and the other in a polarization state. We show that noise affecting both photons can be recast as a position-dependent perturbation affecting only the photon in the polarization state. This approach allows us to utilize both the language and concepts used in studying noisy qubits, as well as recent advances in quantum polarimetry. By adding noise to a finite-dimensional Hilbert space of polarization states, we can describe the noise using quantum operations expressed through appropriate Krauss operators, whose structure is determined by quantum polarimetry. For non-depolarizing noise, we provide an argument based on homotopic maps that demonstrates the topology's resilience to noise. For depolarizing noise, numerical studies using the quantum channel description show that the discrete entanglement signal remains completely resilient. Finally, we identify sources of local noise that can destabilize the topology. This foundational work establishes a framework for understanding how topology enhances the resilience of quantum information, directly impacting the distribution of information through entanglement in noisy environments, such as quantum computers and quantum networks., Comment: 24 pages, 15 figures

Cross-modal text-molecule retrieval model aims to learn a shared feature space of the text and molecule modalities for accurate similarity calculation, which facilitates the rapid screening of molecules with specific properties and activities in drug design. However, previous works have two main defects. First, they are inadequate in capturing modality-shared features considering the significant gap between text sequences and molecule graphs. Second, they mainly rely on contrastive learning and adversarial training for cross-modality alignment, both of which mainly focus on the first-order similarity, ignoring the second-order similarity that can capture more structural information in the embedding space. To address these issues, we propose a novel cross-modal text-molecule retrieval model with two-fold improvements. Specifically, on the top of two modality-specific encoders, we stack a memory bank based feature projector that contain learnable memory vectors to extract modality-shared features better. More importantly, during the model training, we calculate four kinds of similarity distributions (text-to-text, text-to-molecule, molecule-to-molecule, and molecule-to-text similarity distributions) for each instance, and then minimize the distance between these similarity distributions (namely second-order similarity losses) to enhance cross-modal alignment. Experimental results and analysis strongly demonstrate the effectiveness of our model. Particularly, our model achieves SOTA performance, outperforming the previously-reported best result by 6.4%., Comment: BIBM 2024 regular paper

Wire bending is a technique used in manufacturing to mass-produce items such as clips, mounts, and braces. Wire bending machines like the DIWire by Pensalabs have made this process accessible for personal fabrication. However, such machines are controlled using Computer Aided Manufacturing (CAM) software which is hard to use, making custom design challenging. We present Y-AR, a Computer Aided Design (CAD) interface for 3D wire bending. Y-AR uses mixed reality to let designers create structures that physically connect to objects in the environment. The interface incorporates springs as design primitives which (1) apply forces to hold objects, and (2) counter-act dimensional inaccuracies inherently caused by mid air modeling and measurement errors in AR. We demonstrate design workflows to design and fabricate a range of mechanisms designed in Y-AR as well as structures made using free-hand design tools. In our usability evaluation, all 12 participants successfully designed and fabricated a functional bottle holder with Y-AR. 10 out of 12 participants felt that the system aided their design process, rating it above 7 out of 10.

The discovery and identification of molecules in biological and environmental samples is crucial for advancing biomedical and chemical sciences. Tandem mass spectrometry (MS/MS) is the leading technique for high-throughput elucidation of molecular structures. However, decoding a molecular structure from its mass spectrum is exceptionally challenging, even when performed by human experts. As a result, the vast majority of acquired MS/MS spectra remain uninterpreted, thereby limiting our understanding of the underlying (bio)chemical processes. Despite decades of progress in machine learning applications for predicting molecular structures from MS/MS spectra, the development of new methods is severely hindered by the lack of standard datasets and evaluation protocols. To address this problem, we propose MassSpecGym -- the first comprehensive benchmark for the discovery and identification of molecules from MS/MS data. Our benchmark comprises the largest publicly available collection of high-quality labeled MS/MS spectra and defines three MS/MS annotation challenges: \textit{de novo} molecular structure generation, molecule retrieval, and spectrum simulation. It includes new evaluation metrics and a generalization-demanding data split, therefore standardizing the MS/MS annotation tasks and rendering the problem accessible to the broad machine learning community. MassSpecGym is publicly available at \url{https://github.com/pluskal-lab/MassSpecGym}.

Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been proven effective as a succinct representation for capturing essential object features, and for establishing a reference frame in action prediction, enabling data-efficient learning of robot skills. However, their manual design nature and reliance on additional human labels limit their scalability. In this paper, we propose KALM, a framework that leverages large pre-trained vision-language models (LMs) to automatically generate task-relevant and cross-instance consistent keypoints. KALM distills robust and consistent keypoints across views and objects by generating proposals using LMs and verifies them against a small set of robot demonstration data. Based on the generated keypoints, we can train keypoint-conditioned policy models that predict actions in keypoint-centric frames, enabling robots to generalize effectively across varying object poses, camera views, and object instances with similar functional shapes. Our method demonstrates strong performance in the real world, adapting to different tasks and environments from only a handful of demonstrations while requiring no additional labels. Website: https://kalm-il.github.io/, Comment: CoRL LangRob Workshop, 2024

Large language models (LLMs) achieve good performance on challenging reasoning benchmarks, yet could also make basic reasoning mistakes. This contrasting behavior is puzzling when it comes to understanding the mechanisms behind LLMs' reasoning capabilities. One hypothesis is that the increasingly high and nearly saturated performance on common reasoning benchmarks could be due to the memorization of similar problems. In this paper, we systematically investigate this hypothesis with a quantitative measurement of memorization in reasoning tasks, using a dynamically generated logical reasoning benchmark based on Knights and Knaves (K&K) puzzles. We found that LLMs could interpolate the training puzzles (achieving near-perfect accuracy) after fine-tuning, yet fail when those puzzles are slightly perturbed, suggesting that the models heavily rely on memorization to solve those training puzzles. On the other hand, we show that while fine-tuning leads to heavy memorization, it also consistently improves generalization performance. In-depth analyses with perturbation tests, cross difficulty-level transferability, probing model internals, and fine-tuning with wrong answers suggest that the LLMs learn to reason on K&K puzzles despite training data memorization. This phenomenon indicates that LLMs exhibit a complex interplay between memorization and genuine reasoning abilities. Finally, our analysis with per-sample memorization score sheds light on how LLMs switch between reasoning and memorization in solving logical puzzles. Our code and data are available at https://memkklogic.github.io.

Many properties of soft contact interfaces are controlled by the contact area (e.g. friction, contact stiffness and surface charge generation). The contact area increases with the contact age at rest. In contrast, it usually reduces under unidirectional shear loading. Although the physical origin of such a reduction is still debated, it always happens in an anisotropic way because the reduction mainly occurs along the shearing direction. Whether such anisotropy is a necessary condition for shear-induced area reduction remains an open question. Here, we investigate the contact area evolution of elastomer-based sphere-plane contacts under an isotropic shear loading, i.e. torsional loading. We find that, when macroscopic sliding is reached, the contact area has undergone a net area reduction. However, the area evolves non-monotonically as the twisting angle increases, with an initial rise up to a maximum before dropping to the value during macroscopic sliding. The ratio of maximum to initial contact area is found weakly dependent on the normal load, angular velocity and dwell time (time interval between the instants when the normal load and twist motion are first applied) within the investigated ranges. We show that non-monotonic area evolution can also be found under unidirectional shear loading conditions under large normal force. These observations challenge the current descriptions of shear-induced contact area evolution and are expected to serve as a benchmark for future modelling attempts in the field.

Space-based gravitational wave observatories, such as LISA, Taiji, and TianQin, employ long-baseline laser interferometry, necessitating displacement measurement sensitivity at 1 pm/$\sqrt{Hz}$ level. A significant challenge in achieving this precision is the coupling noise arising from far-field wavefront errors (WFE) and laser pointing jitter. This paper presents a comprehensive noise model that incorporates three critical factors: transmitted WFE, static pointing angle, and laser beam jitter. Utilizing the Nijboer-Zernike diffraction theory, we derive an approximate expression for far-field WFE, ensuring minimal error and efficient computational performance. The approximate expression has convincing physical interpretability and reveals how various Zernike aberrations and their coupling impact far-field WFE. Furthermore, the study identifies that correcting optical axis deviations induced by $Z_3^{\pm1}$ through beam tilt exacerbates far-field WFE, underscoring the necessity for active suppression of $Z_3^{\pm1}$. The proposed model facilitates detailed system simulations of the laser link, evaluates Tilt-to-Length (TTL) noise, and offers theoretical insights for system optimization., Comment: 25 pages, 13 figures

Time-varying data with irregular structures can be described by finite time-vertex graph signals (FTVGS), which represent potential temporal and spatial relationships among multiple sources. While sampling and corresponding reconstruction of FTVGS with known spectral support are well investigated, methods for the case of unknown spectral support remain underdeveloped. Existing random sampling schemes may acquire samples from any vertex at any time, which is uncommon in practical applications where sampling typically involves only a subset of vertices and time instants. In sight of this requirement, this paper proposes a subset random sampling scheme for FTVGS. We first randomly select some rows and columns of the FTVGS to form a submatrix, and then randomly sample within the submatrix. Theoretically, we prove sufficient conditions to ensure that the original FTVGS is reconstructed with high probability. Also, we validate the feasibility of reconstructing the original FTVGS by experiments., Comment: 6 pages, 4 figures, conference

Using $(10087\pm44)\times 10^{6}$ $J/\psi$ decays collected by the BESIII detector at the BEPCII collider, we search for baryon number violation via $\Lambda-\bar{\Lambda}$ oscillation in the decay $J/\psi \to \Lambda \bar{\Lambda}$. No evidence for $\Lambda-\bar\Lambda$ oscillation is observed. The upper limit on the time-integrated probability of $\Lambda-\bar{\Lambda}$ oscillation is estimated to be $1.4\times 10^{-6}$, corresponding to an oscillation parameter less than $2.1\times 10^{-18}~\mathrm{GeV}$ at $90\%$ confidence level., Comment: 8 pages, 2 figures

Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates that merging the parameters of independently fine-tuned models can effectively achieve MTL. However, existing merging methods primarily seek a static optimal solution within the original model parameter space, which often results in performance degradation due to the inherent diversity among tasks and potential interferences. To address this challenge, in this paper, we propose a Weight-Ensembling Mixture of Experts (WEMoE) method for multi-task model merging. Specifically, we first identify critical (or sensitive) modules by analyzing parameter variations in core modules of Transformer-based models before and after finetuning. Then, our WEMoE statically merges non-critical modules while transforming critical modules into a mixture-of-experts (MoE) structure. During inference, expert modules in the MoE are dynamically merged based on input samples, enabling a more flexible and adaptive merging approach. Building on WEMoE, we further introduce an efficient-and-effective WEMoE (E-WEMoE) method, whose core mechanism involves eliminating non-essential elements in the critical modules of WEMoE and implementing shared routing across multiple MoE modules, thereby significantly reducing both the trainable parameters, the overall parameter count, and computational overhead of the merged model by WEMoE. Experimental results across various architectures and tasks demonstrate that both WEMoE and E-WEMoE outperform state-of-the-art (SOTA) model merging methods in terms of MTL performance, generalization, and robustness.

Recently, graph prompt learning has garnered increasing attention in adapting pre-trained GNN models for downstream graph learning tasks. However, existing works generally conduct prompting over all graph elements (e.g., nodes, edges, node attributes, etc.), which is suboptimal and obviously redundant. To address this issue, we propose exploiting sparse representation theory for graph prompting and present Graph Sparse Prompting (GSP). GSP aims to adaptively and sparsely select the optimal elements (e.g., certain node attributes) to achieve compact prompting for downstream tasks. Specifically, we propose two kinds of GSP models, termed Graph Sparse Feature Prompting (GSFP) and Graph Sparse multi-Feature Prompting (GSmFP). Both GSFP and GSmFP provide a general scheme for tuning any specific pre-trained GNNs that can achieve attribute selection and compact prompt learning simultaneously. A simple yet effective algorithm has been designed for solving GSFP and GSmFP models. Experiments on 16 widely-used benchmark datasets validate the effectiveness and advantages of the proposed GSFPs.