Search

With the rapidly increasing capabilities and adoption of code agents for AI-assisted coding, safety concerns, such as generating or executing risky code, have become significant barriers to the real-world deployment of these agents. To provide comprehensive and practical evaluations on the safety of code agents, we propose RedCode, a benchmark for risky code execution and generation: (1) RedCode-Exec provides challenging prompts that could lead to risky code execution, aiming to evaluate code agents' ability to recognize and handle unsafe code. We provide a total of 4,050 risky test cases in Python and Bash tasks with diverse input formats including code snippets and natural text. They covers 25 types of critical vulnerabilities spanning 8 domains (e.g., websites, file systems). We provide Docker environments and design corresponding evaluation metrics to assess their execution results. (2) RedCode-Gen provides 160 prompts with function signatures and docstrings as input to assess whether code agents will follow instructions to generate harmful code or software. Our empirical findings, derived from evaluating three agent frameworks based on 19 LLMs, provide insights into code agents' vulnerabilities. For instance, evaluations on RedCode-Exec show that agents are more likely to reject executing risky operations on the operating system, but are less likely to reject executing technically buggy code, indicating high risks. Risky operations described in natural text lead to a lower rejection rate than those in code format. Additionally, evaluations on RedCode-Gen show that more capable base models and agents with stronger overall coding abilities, such as GPT4, tend to produce more sophisticated and effective harmful software. Our findings highlight the need for stringent safety evaluations for diverse code agents. Our dataset and code are available at https://github.com/AI-secure/RedCode., Comment: Accepted by NeurIPS 2024 Datasets and Benchmarks Track

Using 7.93 ${\rm fb^{-1}}$ of $e^+e^-$ collision data collected at a center-of-mass energy of 3.773 ${\rm GeV}$ with the BESIII detector, we present an analysis of the decay $D^{0} \to \eta \pi^- e^+ \nu_{e}$. The branching fraction of the decay $D^{0} \to a_{0}(980)^{-} e^+ \nu_{e}$ with $a_{0}(980)^{-} \to \eta \pi^{-}$ is measured to be $(0.86\pm0.17_{\text{stat}}\pm0.05_{\text{syst}})\times 10^{-4}$. The decay dynamics of this process is studied with a single-pole parameterization of the hadronic form factor and the Flatt\'e formula describing the $a_0(980)$ line shape in the differential decay rate. The product of the form factor $f^{ a_0}_{+}(0)$ and the Cabibbo-Kobayashi-Maskawa matrix element $|V_{cd}|$ is determined for the first time with the result $f^{ a_0}_+(0)|V_{cd}|=0.126\pm0.013_{\rm stat}\pm0.003_{\rm syst}$.

Multi-instance point cloud registration aims to estimate the pose of all instances of a model point cloud in the whole scene. Existing methods all adopt the strategy of first obtaining the global correspondence and then clustering to obtain the pose of each instance. However, due to the cluttered and occluded objects in the scene, it is difficult to obtain an accurate correspondence between the model point cloud and all instances in the scene. To this end, we propose a simple yet powerful 3D focusing-and-matching network for multi-instance point cloud registration by learning the multiple pair-wise point cloud registration. Specifically, we first present a 3D multi-object focusing module to locate the center of each object and generate object proposals. By using self-attention and cross-attention to associate the model point cloud with structurally similar objects, we can locate potential matching instances by regressing object centers. Then, we propose a 3D dual masking instance matching module to estimate the pose between the model point cloud and each object proposal. It performs instance mask and overlap mask masks to accurately predict the pair-wise correspondence. Extensive experiments on two public benchmarks, Scan2CAD and ROBI, show that our method achieves a new state-of-the-art performance on the multi-instance point cloud registration task. Code is available at https://github.com/zlynpu/3DFMNet., Comment: Accepted to NeurIPS 2024

To realize ferromagnetic semiconductors with high Curie temperature TC is still a challenge in spintronics. Recent experiments have obtained two-dimensional (2D) room temperature ferromagnetic metals, such as monolayers MnSe2 and Cr3Te6. In this paper, by the density functional theory (DFT) calculations, we proposed a way to obtain 2D high TC ferromagnetic semiconductors through element replacement in these ferromagnetic metals. High TC ferromagnetic semiconductors are predicted in the monolayers (Mn, D)Se2 and (Cr, D)3Te6, where element D is taken as vacancy, 3d, 4d and 5d transition metal elements. For the concentrations of D from 1/9 to 1/3, there are about 10 ferromagnetic semiconductors with TC above 200 K, including (Cr5/6, W1/6)3Te6 and (Cr4/6, Mo2/6)3Te6 with TC above 300 K. In addition, Mn(Se6/8, Sb2/8)2 is also predicted to be a 2D ferromagnetic semiconductor with TC above 300 K. Considering the fast developments on fabrication and manipulation of 2D materials, our theoretical results propose a way to explore the high temperature ferromagnetic semiconductors from experimentally obtained 2D high temperature ferromagnetic metals through element replacement approach.

Characterizing the express power of the Transformer architecture is critical to understanding its capacity limits and scaling law. Recent works provide the circuit complexity bounds to Transformer-like architecture. On the other hand, Rotary Position Embedding ($\mathsf{RoPE}$) has emerged as a crucial technique in modern large language models, offering superior performance in capturing positional information compared to traditional position embeddings, which shows great potential in application prospects, particularly for the long context scenario. Empirical evidence also suggests that $\mathsf{RoPE}$-based Transformer architectures demonstrate greater generalization capabilities compared to conventional Transformer models. In this work, we establish a tighter circuit complexity bound for Transformers with $\mathsf{RoPE}$ attention. Our key contribution is that we show that unless $\mathsf{TC}^0 = \mathsf{NC}^1$, a $\mathsf{RoPE}$-based Transformer with $\mathrm{poly}(n)$-precision, $O(1)$ layers, hidden dimension $d \leq O(n)$ cannot solve the arithmetic problem or the Boolean formula value problem. This result significantly demonstrates the fundamental limitation of the expressivity of the $\mathsf{RoPE}$-based Transformer architecture, although it achieves giant empirical success. Our theoretical framework not only establishes tighter complexity bounds but also may instruct further work on the $\mathsf{RoPE}$-based Transformer.

Compute Express Link (CXL) serves as a rising industry standard, delivering high-speed cache-coherent links to a variety of devices, including host CPUs, computational accelerators, and memory devices. It is designed to promote system scalability, enable peer-to-peer exchanges, and accelerate data transmissions. To achieve these objectives, the most recent CXL protocol has brought forth several innovative features, such as port-focused routing, device-handled coherence, and PCIe 6.0 compatibility. However, due to the limited availability of hardware prototypes and simulators compatible with CXL, earlier CXL research has largely depended on emulating CXL devices using remote NUMA nodes. Unfortunately, these NUMA-based emulators have difficulties in accurately representing the new features due to fundamental differences in hardware and protocols. Moreover, the absence of support for non-tree topology and PCIe links makes it complex to merely adapt existing simulators for CXL simulation. To overcome these problems, we introduce ESF, a simulation framework specifically designed for CXL systems. ESF has been developed to accurately reflect the unique features of the latest CXL protocol from the ground up. It uses a specialized interconnect layer to facilitate connections within a wide range of system topologies and also includes key components to carry out specific functions required by these features. By utilizing ESF, we thoroughly investigate various aspects of CXL systems, including system topology, device-handled coherence, and the effects of PCIe characteristics, leading to important findings that can guide the creation of high-performance CXL systems. The ESF source codes are fully open-source and can be accessed at https://anonymous.4open.science/r/ESF-1CE3.

While 3D content generation has advanced significantly, existing methods still face challenges with input formats, latent space design, and output representations. This paper introduces a novel 3D generation framework that addresses these challenges, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space. Our framework employs a Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information, and incorporates a cascaded latent diffusion model for improved shape-texture disentanglement. The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single/multi-view image inputs. Notably, the newly proposed latent space naturally enables geometry-texture disentanglement, thus allowing 3D-aware editing. Experimental results demonstrate the effectiveness of our approach on multiple datasets, outperforming existing methods in both text- and image-conditioned 3D generation., Comment: project page: https://nirvanalan.github.io/projects/GA/

The MUltiplexed Survey Telescope (MUST) is a 6.5-meter telescope under development. Dedicated to highly-multiplexed, wide-field spectroscopic surveys, MUST observes over 20,000 targets simultaneously using 6.2-mm pitch positioning robots within a ~5 deg2 field of view. MUST aims to carry out the first Stage-V spectroscopic survey in the 2030s to map the 3D Universe with over 100 million galaxies and quasars, spanning from the nearby Universe to redshift z~5.5, corresponding to around 1 billion years after the Big Bang. To cover this extensive redshift range, we present an initial conceptual target selection algorithm for different types of galaxies, from local bright galaxies, luminous red galaxies, and emission line galaxies to high-redshift (2 < z < 5.5) Lyman-break galaxies. Using Fisher forecasts, we demonstrate that MUST can address fundamental questions in cosmology, including the nature of dark energy, test of gravity theories, and investigations into primordial physics. This is the first paper in the series of science white papers for MUST, with subsequent developments focusing on additional scientific cases such as galaxy and quasar evolution, Milky Way physics, and dynamic phenomena in the time-domain Universe., Comment: To be submitted to SCPMA

In Partially Observable Markov Decision Processes, integrating an agent's history into memory poses a significant challenge for decision-making. Traditional imitation learning, relying on observation-action pairs for expert demonstrations, fails to capture the expert's memory mechanisms used in decision-making. To capture memory processes as demonstrations, we introduce the concept of memory dependency pairs $(p, q)$ indicating that events at time $p$ are recalled for decision-making at time $q$. We introduce AttentionTuner to leverage memory dependency pairs in Transformers and find significant improvements across several tasks compared to standard Transformers when evaluated on Memory Gym and the Long-term Memory Benchmark. Code is available at https://github.com/WilliamYue37/AttentionTuner.

A unified definition for the rotation angle and rotation angular speed of general beams, including those with orbital angular momentum (OAM), has been lacking until now. The rotation of a general beam is characterized by observing the rotational behavior of the directions of the extreme spot sizes during propagation. We introduce the beam quality $M^2(\psi)$ factor to characterize the unique beam quality of a general beam across all directions, not limited to the $x$- or $y$-axes. Besides that, we present the beam center $s_{\psi}(\psi,z)$, spot size $w_{\psi}(\psi,z)$, waist position, waist radius, and divergence angle along the direction that forms an angle $\psi$ with the $x$-axis in the plane perpendicular to the $z$-axis for the general beam. Furthermore, this paper presents rapid calculation formulas for these parameters, utilizing the mode expansion method (MEM). Subsequently, we prove that only two extreme spot sizes exist in a given detection plane and the angle between the maximum and minimum spot angles is consistently $90^{\circ}$ during the propagation. We also prove the spot rotation angles converge as $z$ approaches either positive or negative infinity. We first show the extreme spot sizes, spot rotation angle, and angular speed for the vortex beam. Our formulas efficiently differentiate between vortex OAM beams and asymmetry OAM beams., Comment: 22 pages, 6 figures

Discovering governing equations of complex network dynamics is a fundamental challenge in contemporary science with rich data, which can uncover the mysterious patterns and mechanisms of the formation and evolution of complex phenomena in various fields and assist in decision-making. In this work, we develop a universal computational tool that can automatically, efficiently, and accurately learn the symbolic changing patterns of complex system states by combining the excellent fitting ability from deep learning and the equation inference ability from pre-trained symbolic regression. We conduct intensive experimental verifications on more than ten representative scenarios from physics, biochemistry, ecology, epidemiology, etc. Results demonstrate the outstanding effectiveness and efficiency of our tool by comparing with the state-of-the-art symbolic regression techniques for network dynamics. The application to real-world systems including global epidemic transmission and pedestrian movements has verified its practical applicability. We believe that our tool can serve as a universal solution to dispel the fog of hidden mechanisms of changes in complex phenomena, advance toward interpretability, and inspire more scientific discoveries., Comment: preprint

Nonreciprocal interaction between two spatially separated subsystems plays a crucial role in signal processing and quantum networks. Here, we propose an efficient scheme to achieve nonreciprocal interaction and entanglement between two qubits by combining coherent and dissipative couplings in a superconducting platform, where two coherently coupled transmon qubits simultaneously interact with a transmission line waveguide. The coherent interaction between the transmon qubits can be achieved via capacitive coupling or via an intermediary cavity mode, while the dissipative interaction is induced by the transmission line via reservoir engineering. With high tunability of superconducting qubits, their positions along the transmission line can be adjusted to tune the dissipative coupling, enabling to tailor reciprocal and nonreciprocal interactions between the qubits. A fully nonreciprocal interaction can be achieved when the separation between the two qubits is $(4n+3)\lambda_{0} /4$, where $n$ is an integer and $\lambda_{0}$ is the photon wavelength. This nonreciprocal interaction enables the generation of nonreciprocal entanglement between the two transmon qubits. Furthermore, applying a drive field to one of the qubit can stabilize the system into a nonreciprocal steady-state entangled state. Remarkably, the nonreciprocal interaction in this work does not rely on the presence of nonlinearity or complex configurations, which has more potential applications in designing nonreciprocal quantum devices, processing quantum information, and building quantum networks., Comment: 11 pages, 7 figures

Generative image composition aims to regenerate the given foreground object in the background image to produce a realistic composite image. In this work, we propose an effective finetuning strategy for generative image composition model, in which we finetune a pretrained model using one or more images containing the same foreground object. Moreover, we propose a multi-reference strategy, which allows the model to take in multiple reference images of the foreground object. The experiments on MureCOM dataset verify the effectiveness of our method.

In this contribution, we introduce the concept of Instance Performance Difference (IPD), a metric designed to measure the gap in performance that a robotics perception task experiences when working with real vs. synthetic pictures. By pairing synthetic and real instances in the pictures and evaluating their performance similarity using perception algorithms, IPD provides a targeted metric that closely aligns with the needs of real-world applications. We explain and demonstrate this metric through a rock detection task in lunar terrain images, highlighting the IPD's effectiveness in identifying the most realistic image synthesis method. The metric is thus instrumental in creating synthetic image datasets that perform in perception tasks like real-world photo counterparts. In turn, this supports robust sim-to-real transfer for perception algorithms in real-world robotics applications., Comment: 4 pages, 3 figures, 1 table

The evaluation of mathematical reasoning capabilities is essential for advancing Artificial General Intelligence (AGI). While Large Language Models (LLMs) have shown impressive performance in solving mathematical problems, existing benchmarks such as GSM8K and MATH present limitations, including narrow problem definitions with specific numbers and reliance on predetermined rules that hinder accurate assessments of reasoning and adaptability. This paper introduces the UTMath Benchmark, which robustly evaluates the models through extensive unit tests. It consists of 1,053 problems across 9 mathematical domains, with over 68 test cases per problem. We propose an innovative evaluation framework inspired by unit testing in software development, focusing on both accuracy and reliability of results. Furthermore, we introduce the Reasoning-to-Coding of Thoughts (RCoT) approach, which encourages LLMs to perform explicit reasoning before generating code, leading to generating more advanced solution and improved performance. Furthermore, we are releasing not only the UTMath benchmark but also the UTMath-Train training dataset (more than 70k samples), to support the community in further exploring mathematical reasoning.

New LLM evaluation benchmarks are important to align with the rapid development of Large Language Models (LLMs). In this work, we present Chinese SimpleQA, the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions, and Chinese SimpleQA mainly has five properties (i.e., Chinese, Diverse, High-quality, Static, Easy-to-evaluate). Specifically, first, we focus on the Chinese language over 6 major topics with 99 diverse subtopics. Second, we conduct a comprehensive quality control process to achieve high-quality questions and answers, where the reference answers are static and cannot be changed over time. Third, following SimpleQA, the questions and answers are very short, and the grading process is easy-to-evaluate based on OpenAI API. Based on Chinese SimpleQA, we perform a comprehensive evaluation on the factuality abilities of existing LLMs. Finally, we hope that Chinese SimpleQA could guide the developers to better understand the Chinese factuality abilities of their models and facilitate the growth of foundation models.

Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG, which first extracts relevant subgraphs based on pre-defined rules and then proceeds to design various strategies aiming to improve the representations and fusion of the extracted structural knowledge. Despite their effectiveness, there are still two challenges. On one hand, subgraphs extracted by rule-based methods may have the potential to overlook critical nodes and result in uncontrollable subgraph size. On the other hand, the misalignment between graph and text modalities undermines the effectiveness of knowledge fusion, ultimately impacting the task performance. To deal with the problems above, we propose a novel framework: \textbf{S}ubgraph R\textbf{E}trieval Enhanced by Gra\textbf{P}h-\textbf{T}ext \textbf{A}lignment, named \textbf{SEPTA}. Firstly, we transform the knowledge graph into a database of subgraph vectors and propose a BFS-style subgraph sampling strategy to avoid information loss, leveraging the analogy between BFS and the message-passing mechanism. In addition, we propose a bidirectional contrastive learning approach for graph-text alignment, which effectively enhances both subgraph retrieval and knowledge fusion. Finally, all the retrieved information is combined for reasoning in the prediction module. Extensive experiments on five datasets demonstrate the effectiveness and robustness of our framework., Comment: Accepted by ECML PKDD 2024

This paper studies a distributed estimation problem for time-varying/time-invariant large-scale interconnected systems (LISs). A fully distributed estimator is presented by recursively solving a distributed modified Riccati equation (DMRE) with decoupling variables. By partitioning the LIS based on the transition matrix's block structure, it turns out that the stability of the subsystem is independent of the global LIS if the decoupling variable is selected as the number of out-neighbors. Additionally, it is revealed that any LIS can be equivalently represented by a Markov system. Based on this insight, we show that the stability decoupling above can also be achieved if the decoupling variable equals the number of in-neighbors. Then, the distributed estimator is proved to be stable if the DMRE remains uniformly bounded. When the LIS is considered time-invariant, and by analyzing the spectral radius of a linear operator, it is proved that the DMRE is uniformly bounded if and only if a linear matrix inequality is feasible. Based on the boundedness result, we also show that the distributed estimator converges to a unique steady state for any initial condition. Finally, simulations verify the effectiveness of the proposed methods., Comment: 15 pages, 4 figures

The density dependence of nuclear symmetry energy $E_{\rm sym}(\rho)$ remains the most uncertain aspect of the equation of state (EOS) of supradense neutron-rich nucleonic matter. Implications of observational crustal fraction of neutron star (NS) moment of inertia $\Delta I/I$ on the $E_{\rm sym}(\rho)$ are examined in the present work, utilizing an isospin-dependent parameterization of NS EOS. We find that symmetry energy parameters significantly influence the $\Delta I/I$, while the EOS of symmetric nuclear matter has a negligible effect. An increase in the slope $L$ and skewness $J_{\rm sym}$ of symmetry energy results in a larger $\Delta I/I$, whereas an increase in the curvature $K_{\rm sym}$ leads to a reduction in $\Delta I/I$. Additionally, we discuss the imprints of observational $\Delta I/I$ on the symmetry energy for NSs with masses of 1.0 M$_\odot$ or 1.4 M$_\odot$. Our results indicate that the $\Delta I/I$ has the potential to set a lower limit of symmetry energy at densities exceeding $3\rho_0$, particularly when $L$ is constrained to values less than $60$ MeV, thereby enhancing our understanding of supradense NS matter., Comment: Submitted to Chinese Physics C

In the field of non-invasive medical imaging, radiomic features are utilized to measure tumor characteristics. However, these features can be affected by the techniques used to discretize the images, ultimately impacting the accuracy of diagnosis. To investigate the influence of various image discretization methods on diagnosis, it is common practice to evaluate multiple discretization strategies individually. This approach often leads to redundant and time-consuming tasks such as training predictive models and fine-tuning hyperparameters separately. This study examines the feasibility of employing multi-task Bayesian optimization to accelerate the hyperparameters search for classifying benign and malignant pulmonary nodules using RBF SVM. Our findings suggest that multi-task Bayesian optimization significantly accelerates the search for hyperparameters in comparison to a single-task approach. To the best of our knowledge, this is the first investigation to utilize multi-task Bayesian optimization in a critical medical context., Comment: 12 pages, 4 figures, 37 references

In the digital age, users store personal data in corporate databases, making data security central to enterprise management. Given the extensive attack surface, assets face challenges like weak authentication, vulnerabilities, and malware. Attackers may exploit vulnerabilities to gain unauthorized access, masquerading as legitimate users. Such attacks can lead to privacy breaches, business disruption, financial losses, and reputational damage. Complex attack vectors blur lines between insider and external threats. To address this, we introduce the IDU-Detector, integrating Intrusion Detection Systems (IDS) with User and Entity Behavior Analytics (UEBA). This integration monitors unauthorized access, bridges system gaps, ensures continuous monitoring, and enhances threat identification. Existing insider threat datasets lack depth and coverage of diverse attack vectors. This hinders detection technologies from addressing complex attack surfaces. We propose new, diverse datasets covering more attack scenarios, enhancing detection technologies. Testing our framework, the IDU-Detector achieved average accuracies of 98.96% and 99.12%. These results show effectiveness in detecting attacks, improving security and response speed, and providing higher asset safety assurance.

Precise control and manipulation of neutral atoms are essential for quantum technologies but largely dependent on conventional bulky optical setups. Here, we demonstrate a multifunctional metalens that integrates an achromatic lens with large numerical aperture, a quarter-wave plate, and a polarizer for trapping and characterizing single Rubidium atoms. The metalens simultaneously focuses a trapping beam at 852\,nm and collects single-photon fluorescence at 780\,nm. We observe a strong dependence of the trapping lifetime on an external bias magnetic field, suggests a complex interplay between the circularly polarized trapping light and the atom's internal states. Our work showcases the potential of metasurfaces in realizing compact and integrated quantum systems based on cold atoms, opening up new possibilities for studying quantum control and manipulation at the nanoscale., Comment: 10 pages, 5 figures

Differential privacy (DP) is a formal notion that restricts the privacy leakage of an algorithm when running on sensitive data, in which privacy-utility trade-off is one of the central problems in private data analysis. In this work, we investigate the fundamental limits of differential privacy in online learning algorithms and present evidence that separates three types of constraints: no DP, pure DP, and approximate DP. We first describe a hypothesis class that is online learnable under approximate DP but not online learnable under pure DP under the adaptive adversarial setting. This indicates that approximate DP must be adopted when dealing with adaptive adversaries. We then prove that any private online learner must make an infinite number of mistakes for almost all hypothesis classes. This essentially generalizes previous results and shows a strong separation between private and non-private settings since a finite mistake bound is always attainable (as long as the class is online learnable) when there is no privacy requirement.

Offboard perception aims to automatically generate high-quality 3D labels for autonomous driving (AD) scenes. Existing offboard methods focus on 3D object detection with closed-set taxonomy and fail to match human-level recognition capability on the rapidly evolving perception tasks. Due to heavy reliance on human labels and the prevalence of data imbalance and sparsity, a unified framework for offboard auto-labeling various elements in AD scenes that meets the distinct needs of perception tasks is not being fully explored. In this paper, we propose a novel multi-modal Zero-shot Offboard Panoptic Perception (ZOPP) framework for autonomous driving scenes. ZOPP integrates the powerful zero-shot recognition capabilities of vision foundation models and 3D representations derived from point clouds. To the best of our knowledge, ZOPP represents a pioneering effort in the domain of multi-modal panoptic perception and auto labeling for autonomous driving scenes. We conduct comprehensive empirical studies and evaluations on Waymo open dataset to validate the proposed ZOPP on various perception tasks. To further explore the usability and extensibility of our proposed ZOPP, we also conduct experiments in downstream applications. The results further demonstrate the great potential of our ZOPP for real-world scenarios., Comment: Accepted by NeurIPS 2024

The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want. Nevertheless, the availability of these models is still limited when we expect to generate images that fall into a specific domain either hard to describe or just unseen to the models. In this work, we propose DomainGallery, a few-shot domain-driven image generation method which aims at finetuning pretrained Stable Diffusion on few-shot target datasets in an attribute-centric manner. Specifically, DomainGallery features prior attribute erasure, attribute disentanglement, regularization and enhancement. These techniques are tailored to few-shot domain-driven generation in order to solve key issues that previous works have failed to settle. Extensive experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios. Codes are available at https://github.com/Ldhlwh/DomainGallery., Comment: NeurIPS 2024

We present here a complete microscopic theory of a family of neutral excitations in the fractional quantum Hall fluids, related to the geometric fluctuations of the quantum Hall ground states. Many of the physical properties of such geometric modes can be inferred analytically. These include the chirality, multiplicity and energy of these geometric modes, as well as the relationship to the density modulation of the ground states of both incompressible and compressible fluids, with or without translational symmetry (e.g. the bubble and stripe phases). With a particular focus on the recently experimentally measured graviton modes as the special case, we elucidate both the universal aspects of the geometric modes and the non-universal aspects dependent on the details of the microscopic Hamiltonians. The microscopic theory explains some of the phenomenological components in the effective field theory and composite fermion theory. It predicts how geometric or graviton modes of both chiralities can be measured in experiments for any topological or compressible phases at different energy scales. In particular we show gapped geometric modes can exist even for compressible FQH phases, though translational symmetry of the ground state is important for such modes to couple to external probes (e.g. Raman scattering) in the long wavelength limit., Comment: 16 pages, 3 figures, comments very welcome

Recent advances in large language models (LLMs), make it potentially feasible to automatically refactor source code with LLMs. However, it remains unclear how well LLMs perform compared to human experts in conducting refactorings automatically and accurately. To fill this gap, in this paper, we conduct an empirical study to investigate the potential of LLMs in automated software refactoring, focusing on the identification of refactoring opportunities and the recommendation of refactoring solutions. We first construct a high-quality refactoring dataset comprising 180 real-world refactorings from 20 projects, and conduct the empirical study on the dataset. With the to-be-refactored Java documents as input, ChatGPT and Gemini identified only 28 and 7 respectively out of the 180 refactoring opportunities. However, explaining the expected refactoring subcategories and narrowing the search space in the prompts substantially increased the success rate of ChatGPT from 15.6% to 86.7%. Concerning the recommendation of refactoring solutions, ChatGPT recommended 176 refactoring solutions for the 180 refactorings, and 63.6% of the recommended solutions were comparable to (even better than) those constructed by human experts. However, 13 out of the 176 solutions suggested by ChatGPT and 9 out of the 137 solutions suggested by Gemini were unsafe in that they either changed the functionality of the source code or introduced syntax errors, which indicate the risk of LLM-based refactoring. To this end, we propose a detect-and-reapply tactic, called RefactoringMirror, to avoid such unsafe refactorings. By reapplying the identified refactorings to the original code using thoroughly tested refactoring engines, we can effectively mitigate the risks associated with LLM-based automated refactoring while still leveraging LLM's intelligence to obtain valuable refactoring recommendations.

The origin of the binary black hole mergers observed by LIGO-Virgo-KAGRA (LVK) remains an open question. We calculate the merger rate from primordial black holes (PBHs) within the density spike around supermassive black holes (SMBHs) at the center of galaxies. We show that the merger rate within the spike is comparable to that within the wider dark matter halo. We also calculate the extreme mass ratio inspiral (EMRI) signal from PBHs hosted within the density spike spiralling into their host SMBHs due to GW emission. We predict that LISA may detect $\sim10^4$ of these EMRIs with signal-to-noise ratio of 5 within a 4-year observation run, if all dark matter is made up of PBHs. Uncertainties in our rates come from the uncertain mass fraction of PBHs within the dark matter spike, relative to the host central SMBHs, which defines the parameter space LISA can constrain., Comment: 6 pages, 3 figures, plus Appendix (3 figures, 1 table)

Differential absorption Lidar (DIAL) in the ultraviolet (UV) region is an effective approach for monitoring tropospheric ozone. 4H-SiC single-photon detectors (SPDs) are emergent devices for UV single-photon detection. Here, we demonstrate a 4H-SiC SPD-based ozone DIAL. We design and fabricate the 4H-SiC single-photon avalanche diode with a beveled mesa structure and optimized layer thickness. An active quenching circuit with a quenching time of 1.03 ns is developed to significantly mitigate the afterpulsing effect while enhancing the maximum count rate. After characterization, the SPD exhibits excellent performance with a photon detection efficiency of 16.6% at 266 nm, a dark count rate of 138 kcps, a maximum count rate of 13 Mcps, and an afterpulse probability of 2.7% at room temperature. Then, we apply two 4H-SiC SPDs in an ozone DIAL. The measured ozone concentrations at altitudes of 1-3.5 km agree well with the results of a commercial ozone DIAL. Our work provides an alternative solution for general UV Lidar applications., Comment: Accepted by Applied Physics Letters

The gravitational deflection of light signals restricted in the polar-axis plane of a moving Kerr-Newman (KN) black hole with a constant velocity along the polar axis is studied within the second post-Minkowskian (PM) approximation. For this purpose, the Lorentz boosting technique is adopted to obtain the exact metric of a moving KN black hole with an arbitrary constant velocity in Kerr-Schild coordinates for the first time. Based on the weak field limit of the exact metric, we then derive the equations of motion of test particles constrained in the polar-axis plane of a moving KN source whose velocity is along the polar axis and collinear with its angular momentum. An iterative technique is utilized subsequently in the calculations of the null deflection angle up to the 2PM order caused by the moving lens, and this deflection angle is found to be spin-independent. Finally, we discuss the influence of the motion of the lens on the gravitational deflection and estimate the possibility to detect this kinematical effect. Our work might be helpful for future astronomical observations., Comment: 10 pages, 2 figures

How can we test AI performance? This question seems trivial, but it isn't. Standard benchmarks often have problems such as in-distribution and small-size test sets, oversimplified metrics, unfair comparisons, and short-term outcome pressure. As a consequence, good performance on standard benchmarks does not guarantee success in real-world scenarios. To address these problems, we present Touchstone, a large-scale collaborative segmentation benchmark of 9 types of abdominal organs. This benchmark is based on 5,195 training CT scans from 76 hospitals around the world and 5,903 testing CT scans from 11 additional hospitals. This diverse test set enhances the statistical significance of benchmark results and rigorously evaluates AI algorithms across various out-of-distribution scenarios. We invited 14 inventors of 19 AI algorithms to train their algorithms, while our team, as a third party, independently evaluated these algorithms on three test sets. In addition, we also evaluated pre-existing AI frameworks--which, differing from algorithms, are more flexible and can support different algorithms--including MONAI from NVIDIA, nnU-Net from DKFZ, and numerous other open-source frameworks. We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain., Comment: Accepted to NeurIPS-2024

Machine unlearning has emerged as an effective strategy for forgetting specific information in the training data. However, with the increasing integration of visual data, privacy concerns in Vision Language Models (VLMs) remain underexplored. To address this, we introduce Facial Identity Unlearning Benchmark (FIUBench), a novel VLM unlearning benchmark designed to robustly evaluate the effectiveness of unlearning algorithms under the Right to be Forgotten setting. Specifically, we formulate the VLM unlearning task via constructing the Fictitious Facial Identity VQA dataset and apply a two-stage evaluation pipeline that is designed to precisely control the sources of information and their exposure levels. In terms of evaluation, since VLM supports various forms of ways to ask questions with the same semantic meaning, we also provide robust evaluation metrics including membership inference attacks and carefully designed adversarial privacy attacks to evaluate the performance of algorithms. Through the evaluation of four baseline VLM unlearning algorithms within FIUBench, we find that all methods remain limited in their unlearning performance, with significant trade-offs between model utility and forget quality. Furthermore, our findings also highlight the importance of privacy attacks for robust evaluations. We hope FIUBench will drive progress in developing more effective VLM unlearning algorithms.

In this paper, we give the analytic expression for homogeneous part of solutions of arbitrary tree-level cosmological correlators, including massive propagators and time-derivative interactions cases. The solutions are given in the form of multivariate hypergeometric functions. It is achieved by two step. Firstly, we indicate the factorization of the homogeneous part of solutions, i.e., the homogeneous part of solutions of multiple vertices is the product of the solutions of the single vertex. Secondly, we give the solution to the $\text{d} \log$-form differential equations of arbitrary single vertex integral family. We also show how to determine the boundary conditions for the differential equations. There are two techniques we developed for the computation. Firstly, we analytically solve $\text{d} \log$-form differential equations via power series expansion. Secondly, we handle degenerate multivariate poles in power series expansion of differential equations by blow-up. They could also be useful in the evaluation of multi-loop Feynman integrals in flat spacetime., Comment: 33 pages, 1 figure

Implementing and tuning chirality is fundamental in physics, chemistry, and material science. Chiral charge density waves (CDWs), where chirality arises from correlated charge orders, are attracting intense interest due to their exotic transport and optical properties. However, a general framework for predicting chiral CDW materials is lacking, primarily because the underlying mechanisms remain elusive. Here, we address this challenge by developing the first comprehensive predictive framework, systematically identifying chiral CDW materials via first-principles calculations. The key lies in the previously overlooked phase difference of the CDW Q-vectors between layers, which is linked to opposite collective atomic displacements across different layers. This phase difference induces a spiral arrangement of the Q-vectors, ultimately giving rise to a chiral structure in real space. We validate our framework by applying it to the kagome lattice AV$_{3}$Sb$_{5}$ (A = K, Rb, Cs), successfully predicting emergent structural chirality. To demonstrate the generality of our approach, we extend it to predict chiral CDWs in the triangular-lattice NbSe$_{2}$. Beyond material predictions, our theory uncovers a universal and unprecedented Hall effect in chiral CDW materials, occurring without external magnetic fields or intrinsic magnetization. Our experiments on CsV$_{3}$Sb$_{5}$ confirm this prediction, observing a unique signature where the Hall conductivity's sign reverses when the input current is reversed, a phenomenon distinct from known Hall effects. Our findings elucidate the mechanisms behind chiral CDWs and open new avenues for discovering materials with unconventional quantum properties, with potential applications in next-generation electronic and spintronic devices.

We accomplish for the first time the two-loop computation of the leading-twist contribution to the pion electromagnetic form factor by employing the effective field theory formalism rigorously. The next-to-next-to-leading-order short-distance matching coefficient is determined by evaluating the appropriate $5$-point QCD amplitude with the modern multi-loop technique and subsequently by implementing the ultraviolet renormalization and infrared subtractions with the inclusion of evanescent operators. The renormalization/factorization scale independence of the obtained form factor is then validated explicitly at ${\cal O}(\alpha_s^3)$. The yielding two-loop QCD correction to this fundamental quantity turns out to be numerically significant at experimentally accessible momentum transfers. We further demonstrate that the newly computed two-loop radiative correction is highly beneficial for an improved determination of the leading-twist pion distribution amplitude., Comment: 7+6 pages, 4 figures

The field of bioinformatics has seen significant progress, making the cross-modal text-molecule retrieval task increasingly vital. This task focuses on accurately retrieving molecule structures based on textual descriptions, by effectively aligning textual descriptions and molecules to assist researchers in identifying suitable molecular candidates. However, many existing approaches overlook the details inherent in molecule sub-structures. In this work, we introduce the Optimal TRansport-based Multi-grained Alignments model (ORMA), a novel approach that facilitates multi-grained alignments between textual descriptions and molecules. Our model features a text encoder and a molecule encoder. The text encoder processes textual descriptions to generate both token-level and sentence-level representations, while molecules are modeled as hierarchical heterogeneous graphs, encompassing atom, motif, and molecule nodes to extract representations at these three levels. A key innovation in ORMA is the application of Optimal Transport (OT) to align tokens with motifs, creating multi-token representations that integrate multiple token alignments with their corresponding motifs. Additionally, we employ contrastive learning to refine cross-modal alignments at three distinct scales: token-atom, multitoken-motif, and sentence-molecule, ensuring that the similarities between correctly matched text-molecule pairs are maximized while those of unmatched pairs are minimized. To our knowledge, this is the first attempt to explore alignments at both the motif and multi-token levels. Experimental results on the ChEBI-20 and PCdes datasets demonstrate that ORMA significantly outperforms existing state-of-the-art (SOTA) models., Comment: BIBM 2024 Regular Paper

Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt tuning with OOD features extracted from ID data. However, the irrelevant context mined from ID data can be spurious due to the inaccurate foreground-background decomposition, thus limiting the OOD detection performance. In this work, we propose a novel framework, namely, Self-Calibrated Tuning (SCT), to mitigate this problem for effective OOD detection with only the given few-shot ID data. Specifically, SCT introduces modulating factors respectively on the two components of the original learning objective. It adaptively directs the optimization process between the two tasks during training on data with different prediction uncertainty to calibrate the influence of OOD regularization, which is compatible with many prompt tuning based OOD detection methods. Extensive experiments and analyses have been conducted to characterize and demonstrate the effectiveness of the proposed SCT. The code is publicly available., Comment: accepted by NeurIPS 2024

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large, Comment: 17 pages, 4 Figures

We explore optimally training protein language models, an area of significant interest in biological research where guidance on best practices is limited. Most models are trained with extensive compute resources until performance gains plateau, focusing primarily on increasing model sizes rather than optimizing the efficient compute frontier that balances performance and compute budgets. Our investigation is grounded in a massive dataset consisting of 939 million protein sequences. We trained over 300 models ranging from 3.5 million to 10.7 billion parameters on 5 to 200 billion unique tokens, to investigate the relations between model sizes, training token numbers, and objectives. First, we observed the effect of diminishing returns for the Causal Language Model (CLM) and that of overfitting for the Masked Language Model~(MLM) when repeating the commonly used Uniref database. To address this, we included metagenomic protein sequences in the training set to increase the diversity and avoid the plateau or overfitting effects. Second, we obtained the scaling laws of CLM and MLM on Transformer, tailored to the specific characteristics of protein sequence data. Third, we observe a transfer scaling phenomenon from CLM to MLM, further demonstrating the effectiveness of transfer through scaling behaviors based on estimated Effectively Transferred Tokens. Finally, to validate our scaling laws, we compare the large-scale versions of ESM-2 and PROGEN2 on downstream tasks, encompassing evaluations of protein generation as well as structure- and function-related tasks, all within less or equivalent pre-training compute budgets., Comment: NeurIPS 2024 (Spotlight); Code: https://github.com/cxysteven/ScalingProteinLM. Additional resources are available here

Transition Edge Sensors (TESs) are amongst the most sensitive cryogenic detectors and can be easily optimized for the detection of massive particles or photons ranging from X-rays all the way down to millimetre radiation. Furthermore, TESs exhibit unmatched energy resolution while being easily frequency domain multiplexed in arrays of several hundred pixels. Such great performance, along with rather simple and sturdy readout and amplification chains make TESs extremely compelling for applications in many fields of scientific endeavour. While the first part of this article is an in-depth discussion on the working principles of Transition Edge Sensors, the remainder of this review article focuses on the applications of Transition Edge Sensors in advanced scientific instrumentation serving as an accessible and thorough list of possible starting points for more comprehensive literature research.

Federated continual learning (FCL) aims to learn from sequential data stream in the decentralized federated learning setting, while simultaneously mitigating the catastrophic forgetting issue in classical continual learning. Existing FCL methods usually employ typical rehearsal mechanisms, which could result in privacy violations or additional onerous storage and computational burdens. In this work, an efficient and non-IID robust federated continual learning framework, called Federated Prototype-Augmented Prompt Learning (FPPL), is proposed. The FPPL can collaboratively learn lightweight prompts augmented by prototypes without rehearsal. On the client side, a fusion function is employed to fully leverage the knowledge contained in task-specific prompts for alleviating catastrophic forgetting. Additionally, global prototypes aggregated from the server are used to obtain unified representation through contrastive learning, mitigating the impact of non-IID-derived data heterogeneity. On the server side, locally uploaded prototypes are utilized to perform debiasing on the classifier, further alleviating the performance degradation caused by both non-IID and catastrophic forgetting. Empirical evaluations demonstrate the effectiveness of FPPL, achieving notable performance with an efficient design while remaining robust to diverse non-IID degrees. Code is available at: https://github.com/ycheoo/FPPL.

Advanced Persistent Threats (APTs) are continuously evolving, leveraging their stealthiness and persistence to put increasing pressure on current provenance-based Intrusion Detection Systems (IDS). This evolution exposes several critical issues: (1) The dense interaction between malicious and benign nodes within provenance graphs introduces neighbor noise, hindering effective detection; (2) The complex prediction mechanisms of existing APTs detection models lead to the insufficient utilization of prior knowledge embedded in the data; (3) The high computational cost makes detection impractical. To address these challenges, we propose Winemaking, a lightweight threat detection system built on a knowledge distillation framework, capable of node-level detection within audit log provenance graphs. Specifically, Winemaking applies graph Laplacian regularization to reduce neighbor noise, obtaining smoothed and denoised graph signals. Subsequently, Winemaking employs a teacher model based on GNNs to extract knowledge, which is then distilled into a lightweight student model. The student model is designed as a trainable combination of a feature transformation module and a personalized PageRank random walk label propagation module, with the former capturing feature knowledge and the latter learning label and structural knowledge. After distillation, the student model benefits from the knowledge of the teacher model to perform precise threat detection. We evaluate Winemaking through extensive experiments on three public datasets and compare its performance against several state-of-the-art IDS solutions. The results demonstrate that Winemaking achieves outstanding detection accuracy across all scenarios and the detection time is 1.4 to 5.2 times faster than the current state-of-the-art methods., Comment: 8 pages body, 11 pages total(without authors)

Semantic segmentation models, like mask2former, often demand a substantial amount of manually annotated data, which is time-consuming and inefficient to acquire. Leveraging state-of-the-art text-to-image models like Midjourney and Stable Diffusion has emerged as an effective strategy for automatically generating synthetic data instead of human annotations. However, prior approaches have been constrained to synthesizing single-instance images due to the instability inherent in generating multiple instances with Stable Diffusion. To expand the domains and diversity of synthetic datasets, this paper introduces a novel paradigm named DiffuMask-Editor, which combines the Diffusion Model for Segmentation with Image Editing. By integrating multiple objects into images using Text2Image models, our method facilitates the creation of more realistic datasets that closely resemble open-world settings while simultaneously generating accurate masks. Our approach significantly reduces the laborious effort associated with manual annotation while ensuring precise mask generation. Experimental results demonstrate that synthetic data generated by DiffuMask-Editor enable segmentation methods to achieve superior performance compared to real data. Particularly in zero-shot backgrounds, DiffuMask-Editor achieves new state-of-the-art results on Unseen classes of VOC 2012. The code and models will be publicly available soon., Comment: 13 pages,4 figures

We consider Josephson junctions formed by coupling two conventional superconductors via an unconventional magnet and investigate the formation of Andreev bound states, their impact on the Josephson effect, and the emergent superconducting correlations. We focus on unconventional magnets known as $d$-wave altermagnets and $p$-wave magnets. We find that the Andreev bound states in $d$-wave altermagnet and $p_y$-wave magnet Josephson junctions strongly depend on the transverse momentum, with a spin splitting and low-energy minima as a function of the superconducting phase difference $\varphi$. In contrast, the Andreev bound states for $p_{x}$-wave magnets are insensitive to the transverse momentum. We show that the Andreev bound states can be probed by the local density of states in the middle of the junction, which also reveals that $d_{x^{2}-y^{2}}$- and $p$-wave magnet junctions are prone to host zero energy peaks. While the zero-energy peak in $d_{x^{2}-y^{2}}$-wave altermagnet junctions tends to oscillate with the magnetic order, it remains robust in $p$-wave magnet junctions. We also demonstrate that the critical currents in $d$-wave altermagnet Josephson junctions exhibit an oscillatory decay with the increase of the magnetic order, while the oscillations are absent in $p$-wave magnet junctions albeit the currents exhibit a slow decay. Furthermore, we also demonstrate that the interplay of the Josephson effect and unconventional magnetic order of $d$-wave altermagnets and $p$-wave magnets originates from odd-frequency spin-triplet $s$-wave superconducting correlations that are otherwise absent. Our results can serve as a guide to pursue the new functionality of Josephson junctions based on unconventional magnets., Comment: 15 pages, 8 figures

In industrial recommendation systems, multi-task learning (learning multiple tasks simultaneously on a single model) is a predominant approach to save training/serving resources and improve recommendation performance via knowledge transfer between the joint learning tasks. However, multi-task learning often suffers from negative transfer: one or several tasks are less optimized than training them separately. To carefully balance the optimization, we propose a gradient balancing approach called MultiBalance, which is suitable for industrial-scale multi-task recommendation systems. It balances the per-task gradients to alleviate the negative transfer, while saving the huge cost for grid search or manual explorations for appropriate task weights. Moreover, compared with prior work that normally balance the per-task gradients of shared parameters, MultiBalance is more efficient since only requiring to access per-task gradients with respect to the shared feature representations. We conduct experiments on Meta's large-scale ads and feeds multi-task recommendation system, and observe that MultiBalance achieves significant gains (e.g., 0.738% improvement for normalized entropy (NE)) with neutral training cost in Queries Per Second (QPS), which is significantly more efficient than prior methods that balance per-task gradients of shared parameters with 70~80% QPS degradation.

Quantum information processing (QIP) based on high-dimensional quantum systems provides unique advantages and new potentials where high-dimensional quantum correlations (QCs) play vital roles. Exploring the resistance of QCs against noises is crucial as QCs are fragile due to complex and unavoidable system-environment interactions. In this study, we investigate the performance of high-dimensional QCs under local dephasing noise using a single nitrogen-vacancy center in diamond. A freezing phenomenon in the high-dimensional quantum discord dynamics was observed, showing discord is robust against local dephasing noise. Utilizing a robustness metric known as freezing index, we found that the discord of qutrits outperforms their qubits counterpart when confronted with dephasing noise. Furthermore, we developed a geometric picture to explain this intriguing freezing phenomenon phenomenon. Our findings highlight the potential of utilizing discord as a physical resource for advancing QIP in high-dimensional quantum settings.

Course evaluation is a critical component in higher education pedagogy. It not only serves to identify limitations in existing course designs and provide a basis for curricular innovation, but also to offer quantitative insights for university administrative decision-making. Traditional evaluation methods, primarily comprising student surveys, instructor self-assessments, and expert reviews, often encounter challenges, including inherent subjectivity, feedback delays, inefficiencies, and limitations in addressing innovative teaching approaches. Recent advancements in large language models (LLMs) within artificial intelligence (AI) present promising new avenues for enhancing course evaluation processes. This study explores the application of LLMs in automated course evaluation from multiple perspectives and conducts rigorous experiments across 100 courses at a major university in China. The findings indicate that: (1) LLMs can be an effective tool for course evaluation; (2) their effectiveness is contingent upon appropriate fine-tuning and prompt engineering; and (3) LLM-generated evaluation results demonstrate a notable level of rationality and interpretability.

Large-scale distributed model training requires simultaneous training on up to thousands of machines. Faulty machine detection is critical when an unexpected fault occurs in a machine. From our experience, a training task can encounter two faults per day on average, possibly leading to a halt for hours. To address the drawbacks of the time-consuming and labor-intensive manual scrutiny, we propose Minder, an automatic faulty machine detector for distributed training tasks. The key idea of Minder is to automatically and efficiently detect faulty distinctive monitoring metric patterns, which could last for a period before the entire training task comes to a halt. Minder has been deployed in our production environment for over one year, monitoring daily distributed training tasks where each involves up to thousands of machines. In our real-world fault detection scenarios, Minder can accurately and efficiently react to faults within 3.6 seconds on average, with a precision of 0.904 and F1-score of 0.893.

LLMs have demonstrated impressive proficiency in generating coherent and high-quality text, making them valuable across a range of text-generation tasks. However, rigorous evaluation of this generated content is crucial, as ensuring its quality remains a significant challenge due to persistent issues such as factual inaccuracies and hallucinations. This paper introduces two fine-tuned general-purpose LLM autoevaluators, REC-12B and REC-70B, specifically designed to evaluate generated text across several dimensions: faithfulness, instruction following, coherence, and completeness. These models not only provide ratings for these metrics but also offer detailed explanations and verifiable citations, thereby enhancing trust in the content. Moreover, the models support various citation modes, accommodating different requirements for latency and granularity. Extensive evaluations on diverse benchmarks demonstrate that our general-purpose LLM auto-evaluator, REC-70B, outperforms state-of-the-art LLMs, excelling in content evaluation by delivering better quality explanations and citations with minimal bias. It achieves Rank \#1 as a generative model on the RewardBench leaderboard\footnote{\url{https://huggingface.co/spaces/allenai/reward-bench}} under the model name \texttt{TextEval-Llama3.1-70B}. Our REC dataset and models are released at \url{https://github.com/adelaidehsu/REC}.

Aligning large language models (LLMs) with human intent is critical for enhancing their performance across a variety of tasks. Standard alignment techniques, such as Direct Preference Optimization (DPO), often rely on the binary Bradley-Terry (BT) model, which can struggle to capture the complexities of human preferences -- particularly in the presence of noisy or inconsistent labels and frequent ties. To address these limitations, we introduce the Tie-rank Oriented Bradley-Terry model (TOBT), an extension of the BT model that explicitly incorporates ties, enabling more nuanced preference representation. Building on this, we propose Tie-rank Oriented Direct Preference Optimization (TODO), a novel alignment algorithm that leverages TOBT's ternary ranking system to improve preference alignment. In evaluations on Mistral-7B and Llama 3-8B models, TODO consistently outperforms DPO in modeling preferences across both in-distribution and out-of-distribution datasets. Additional assessments using MT Bench and benchmarks such as Piqa, ARC-c, and MMLU further demonstrate TODO's superior alignment performance. Notably, TODO also shows strong results in binary preference alignment, highlighting its versatility and potential for broader integration into LLM alignment. The implementation details can be found in https://github.com/XXares/TODO.