Search

The training process of large language models (LLMs) often involves varying degrees of test data contamination. Although current LLMs are achieving increasingly better performance on various benchmarks, their performance in practical applications does not always match their benchmark results. Leakage of benchmarks can prevent the accurate assessment of LLMs' true performance. However, constructing new benchmarks is costly, labor-intensive and still carries the risk of leakage. Therefore, in this paper, we ask the question, Can we reuse these leaked benchmarks for LLM evaluation? We propose Inference-Time Decontamination (ITD) to address this issue by detecting and rewriting leaked samples without altering their difficulties. ITD can mitigate performance inflation caused by memorizing leaked benchmarks. Our proof-of-concept experiments demonstrate that ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU. On MMLU, using Inference-time Decontamination can lead to a decrease in the results of Phi3 and Mistral by 6.7% and 3.6% respectively. We hope that ITD can provide more truthful evaluation results for large language models.

In Retrieval-Augmented Generation (RAG), retrieval is not always helpful and applying it to every instruction is sub-optimal. Therefore, determining whether to retrieve is crucial for RAG, which is usually referred to as Active Retrieval. However, existing active retrieval methods face two challenges: 1. They usually rely on a single criterion, which struggles with handling various types of instructions. 2. They depend on specialized and highly differentiated procedures, and thus combining them makes the RAG system more complicated and leads to higher response latency. To address these challenges, we propose Unified Active Retrieval (UAR). UAR contains four orthogonal criteria and casts them into plug-and-play classification tasks, which achieves multifaceted retrieval timing judgements with negligible extra inference cost. We further introduce the Unified Active Retrieval Criteria (UAR-Criteria), designed to process diverse active retrieval scenarios through a standardized procedure. Experiments on four representative types of user instructions show that UAR significantly outperforms existing work on the retrieval timing judgement and the performance of downstream tasks, which shows the effectiveness of UAR and its helpfulness to downstream tasks.

Leveraging quantum properties to enhance complex learning tasks has been proven feasible, with excellent recent achievements in the field of unsupervised learning. However, current quantum schemes neglect adaptive adjustments for unsupervised task scenarios. This work proposes a novel quantum unsupervised similarity learning method, QUSL. Firstly, QUSL uses similarity triplets for unsupervised learning, generating positive samples by perturbing anchor images, achieving a learning process independent of classical algorithms. Subsequently, combining the feature interweaving of triplets, QUSL employs metaheuristic algorithms to systematically explore high-performance mapping processes, obtaining quantum circuit architectures more suitable for unsupervised image similarity tasks. Ultimately, QUSL realizes feature learning with lower quantum resource costs. Comprehensive numerical simulations and experiments on quantum computers demonstrate that QUSL outperforms state-of-the-art quantum methods. QUSL achieves over 50% reduction in critical quantum resource utilization. QUSL improves similarity detection correlation by up to 19.5% across multiple datasets, exhibiting robustness in NISQ environments. While using fewer quantum resources, QUSL shows potential for large-scale unsupervised tasks.

Cells not only can have self-motility by crawling, but also are capable of other types of active motions like periodic contraction or pulsation. In this work, by using the Voronoi cell model, we show how this non-motility activity affects the structure, dynamic and density fluctuations of cellular monolayers. We found that random cell pulsation can fluidize solid epithelial tissues, resulting in a \emph{hyperuniform} fluid state, while pulsation synchronization inhibits the fluidity, causes a reversed solidification. We prove this solidification is BKT-type transition, characterized by strong density/dynamic heterogeneity arising from the creation of topological defects in the pulsating phase space. The magnitude and length scale of density heterogeneity diverge as the pulsating period increases, resulting in an opposite \emph{giant} density fluctuation. Our findings accord with recent experimental observations and can be quantitatively explained by a proposed fluctuating hydrodynamic theory., Comment: 7pages, 3 figures

Background. Joint range of motion (ROM) is an important quantitative measure for physical therapy. Commonly relying on a goniometer, accurate and reliable ROM measurement requires extensive training and practice. This, in turn, imposes a significant barrier for those who have limited in-person access to healthcare. Objective. The current study presents and evaluates an alternative machine learning-based ROM evaluation method that could be remotely accessed via a webcam. Methods. To evaluate its reliability, the ROM measurements for a diverse set of joints (neck, spine, and upper and lower extremities) derived using this method were compared to those obtained from a marker-based optical motion capture system. Results. Data collected from 25 healthy adults demonstrated that the webcam solution exhibited high test-retest reliability, with substantial to almost perfect intraclass correlation coefficients for most joints. Compared with the marker-based system, the webcam-based system demonstrated substantial to almost perfect inter-rater reliability for some joints, and lower inter-rater reliability for other joints (e.g., shoulder flexion and elbow flexion), which could be attributed to the reduced sensitivity to joint locations at the apex of the movement. Conclusions. The proposed webcam-based method exhibited high test-retest and inter-rater reliability, making it a versatile alternative for existing ROM evaluation methods in clinical practice and the tele-implementation of physical therapy and rehabilitation.

Theoretical studies predicted possible superconductivity in electron-doped chromium pnictides isostructural to their iron counterparts. Here, we report the synthesis and characterization of a new ZrCuSiAs-type Cr-based compound ThCrAsN, as well as its oxygen-doped variants. All samples of ThCrAsN$_{1-x}$O$_x$ show metallic conduction, but no superconductivity is observed above 30 mK even though the oxygen substitution reaches 75\%. The magnetic structure of ThCrAsN is determined to be G-type antiferromagnetic by magnetization measurements and first-principles calculations jointly. The calculations also indicate that the in-plane Cr--Cr direct interaction of ThCrAsN is robust against the heavy electron doping. The calculated density of states of the orbital occupations of Cr for ThCrAs(N,O) is strongly spin-polarized. Our results suggest the similarities between chromium pnictides and iron-based superconductors shouldn't be overestimated.

The Hidden Quantum Markov Model (HQMM) has significant potential for analyzing time-series data and studying stochastic processes in the quantum domain as an upgrading option with potential advantages over classical Markov models. In this paper, we introduced the split HQMM (SHQMM) for implementing the hidden quantum Markov process, utilizing the conditional master equation with a fine balance condition to demonstrate the interconnections among the internal states of the quantum system. The experimental results suggest that our model outperforms previous models in terms of scope of applications and robustness. Additionally, we establish a new learning algorithm to solve parameters in HQMM by relating the quantum conditional master equation to the HQMM. Finally, our study provides clear evidence that the quantum transport system can be considered a physical representation of HQMM. The SHQMM with accompanying algorithms present a novel method to analyze quantum systems and time series grounded in physical implementation.

Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. Although prompt tuning has been shown to match the performance of full model tuning when training data is sufficient, it tends to struggle in few-shot learning settings. In this paper, we present Multi-task Pre-trained Modular Prompt (MP2) to boost prompt tuning for few-shot learning. MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks. On downstream tasks, the pre-trained prompts are selectively activated and combined, leading to strong compositional generalization to unseen tasks. To bridge the gap between pre-training and fine-tuning, we formulate upstream and downstream tasks into a unified machine reading comprehension task. Extensive experiments under two learning paradigms, i.e., gradient descent and black-box tuning, show that MP2 significantly outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot settings. In addition, we demonstrate that MP2 can achieve surprisingly fast and strong adaptation to downstream tasks by merely learning 8 parameters to combine the pre-trained modular prompts., Comment: Accepted to ACL 2023 (main conference). Code and data are publicly available at https://github.com/Hzfinfdu/MPMP

Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives. However, the output summary is always evaluated from summary-level which leads to the inconsistency in training and evaluation. In this paper, we propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. By modeling a contrastive objective, we show that the summarization model is able to directly generate summaries according to the summary-level score without additional modules and parameters. Extensive experiments demonstrate that COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score while preserving the parameter efficiency and inference efficiency. Compared with state-of-the-art multi-stage systems, we save more than 100 GPU training hours and obtaining 3~8 speed-up ratio during inference while maintaining comparable results., Comment: Accepted by COLING 2022

Therapeutic modulation of immune states is central to the treatment of human disease. However, how drugs and drug combinations impact the diverse cell types in the human immune system remains poorly understood at the transcriptome scale. Here, we apply single-cell mRNA-seq to profile the response of human immune cells to 502 immunomodulatory drugs alone and in combination. We develop a unified mathematical model that quantitatively describes the transcriptome scale response of myeloid and lymphoid cell types to individual drugs and drug combinations through a single inferred regulatory network. The mathematical model reveals how drug combinations generate novel, macrophage and T-cell states by recruiting combinations of gene expression programs through both additive and non-additive drug interactions. A simplified drug response algebra allows us to predict the continuous modulation of immune cell populations between activated, resting and hyper-inhibited states through combinatorial drug dose titrations. Our results suggest that transcriptome-scale mathematical models could enable the design of therapeutic strategies for programming the human immune system using combinations of therapeutics., Comment: 19 pages, 5 figures

The quest for quantum materials with diverse symmetry-protected topological states has been the focus of recent research interest, primarily due to their fascinating physical properties and the potential technological utility. In this work, we report on the magnetotransport, de Haas-van Alphen (dHvA) oscillations, and the first-principles calculations of the stannide Cu$_3$Sn that is isostructural with the recently reported topological semimetal Ag$_3$Sn. The magnetoresistance was found to vary quasi-linearly in field. Clear dHvA oscillations were observed under a field as low as 1 Tesla at 2 K, with three major oscillation frequencies $F_{\alpha}$=8.74 T, $F_{\beta}$=150.19 T and $F_{\gamma}$=229.66 T and extremely small effective masses. The analysis of dHvA quantum oscillations revealed a possible nonzero Berry phase, suggestive of the nontrivial band topology. The corroborating evidence for the nontrivial electronic topology also comes from the first-principles calculations which yield a nonzero $\mathbb{Z}_2$ topological index. These results collectively suggest that Cu$_3$Sn, in analogy to its homologue Ag$_3$Sn, may be another intermetallic stannide hosting topological Dirac fermions., Comment: 4 figures, 1 table