Search

Reward models (RMs) are crucial for aligning large language models (LLMs) with human preferences. They are trained using preference datasets where each example consists of one input prompt, two responses, and a preference label. As curating a high-quality human labeled preference dataset is both time-consuming and expensive, people often rely on existing powerful LLMs for preference label generation. This can potentially introduce noise and impede RM training. In this work, we present RMBoost, a novel synthetic preference data generation paradigm to boost reward model quality. Unlike traditional methods, which generate two responses before obtaining the preference label, RMBoost first generates one response and selects a preference label, followed by generating the second more (or less) preferred response conditioned on the pre-selected preference label and the first response. This approach offers two main advantages. First, RMBoost reduces labeling noise since preference pairs are constructed intentionally. Second, RMBoost facilitates the creation of more diverse responses by incorporating various quality aspects (e.g., helpfulness, relevance, completeness) into the prompts. We conduct extensive experiments across three diverse datasets and demonstrate that RMBoost outperforms other synthetic preference data generation techniques and significantly boosts the performance of four distinct reward models.

In literature it is shown that bi-Lipschitz maps between self-similar sets or self-affine sets enjoy a locally measure preserving property, namely, if $f:(E,\mu)\to (F,\nu)$ is a bi-Lipschitz map, then the Radon-Nykodym derivative $df^*\nu/d\mu$ is a constant function on a subset $E'\subset E$ with $\mu(E')>0$, where $f^*\nu(\cdot)=\nu(f(\cdot))$. Indeed, this measure preserving property plays an important role in Lipschitz classification of fractal sets. In this paper, we show that such measure preserving property also holds for bi-Lipschitz maps between two Moran sets in a certain class.

Continuous learning from an immense volume of data streams becomes exceptionally critical in the internet era. However, data streams often do not conform to the same distribution over time, leading to a phenomenon called concept drift. Since a fixed static model is unreliable for inferring concept-drifted data streams, establishing an adaptive mechanism for detecting concept drift is crucial. Current methods for concept drift detection primarily assume that the labels or error rates of downstream models are given and/or underlying statistical properties exist in data streams. These approaches, however, struggle to address high-dimensional data streams with intricate irregular distribution shifts, which are more prevalent in real-world scenarios. In this paper, we propose MCD-DD, a novel concept drift detection method based on maximum concept discrepancy, inspired by the maximum mean discrepancy. Our method can adaptively identify varying forms of concept drift by contrastive learning of concept embeddings without relying on labels or statistical properties. With thorough experiments under synthetic and real-world scenarios, we demonstrate that the proposed method outperforms existing baselines in identifying concept drifts and enables qualitative analysis with high explainability.

Atom interferometers provide a powerful tool for measuring physical constants and testifying fundamental physics with unprecedented precision. Conventional atom interferometry focuses on the phase difference between two paths and utilizes matter waves with fixed coherence. Here, we report on realizing a Ramsey-Bord\'e interferometer of coherent matter waves dressed by a moving optical lattice in the gravity direction, and explore the resulting interference along multiple paths with tunable coherence. We investigate spatial correlations of atoms both within the lattice and between two arms by interferometry, and observe the emerging multiple interference peaks owing to the long-range coherence nature of the Bose-Einstein condensate. Our findings agree well with theoretical simulations, paving the way for high-precision interferometry with ultracold atoms.

The escalating complexity of micro-services architecture in cloud-native technologies poses significant challenges for maintaining system stability and efficiency. To conduct root cause analysis (RCA) and resolution of alert events, we propose a pioneering framework, multi-Agent Blockchain-inspired Collaboration for root cause analysis in micro-services architecture (mABC), to revolutionize the AI for IT operations (AIOps) domain, where multiple agents based on the powerful large language models (LLMs) perform blockchain-inspired voting to reach a final agreement following a standardized process for processing tasks and queries provided by Agent Workflow. Specifically, seven specialized agents derived from Agent Workflow each provide valuable insights towards root cause analysis based on their expertise and the intrinsic software knowledge of LLMs collaborating within a decentralized chain. To avoid potential instability issues in LLMs and fully leverage the transparent and egalitarian advantages inherent in a decentralized structure, mABC adopts a decision-making process inspired by blockchain governance principles while considering the contribution index and expertise index of each agent. Experimental results on the public benchmark AIOps challenge dataset and our created train-ticket dataset demonstrate superior performance in accurately identifying root causes and formulating effective solutions, compared to previous strong baselines. The ablation study further highlights the significance of each component within mABC, with Agent Workflow, multi-agent, and blockchain-inspired voting being crucial for achieving optimal performance. mABC offers a comprehensive automated root cause analysis and resolution in micro-services architecture and achieves a significant improvement in the AIOps domain compared to existing baselines

Since the discovery of pulsars, the rotating-lighthouse model has been the choice of model to explain the radiation pulsation of pulsars. After discovering that some main sequence stars (e.g., CU Virginis) and ultracool dwarfs (e.g., TVLM 513-46546) also behave like pulsars, the lighthouse model was again adopted to explain their pulse signals. Our research found that if we use the magnetic-field oscillation (MO) model, we can explain the pulse radiation results better regardless of whether its source is a neutron star, a main-sequence star or an ultracool dwarf. We propose a verifiable prediction that can be used to evaluate the MO model. Our prediction is that there is a 90 degree phase lag between the magnetic field and radio signal of TVLM 513-46546, and the zero-crossing point of the magnetic field is the moment when the direction of the light's circular polarization is reversed. No new observations are needed to check this prediction, but certain existing data needs to be re-mined., Comment: 8 pages, 4 figures, accepted for publication in MNRAS

Continual Few-shot Relation Extraction (CFRE) is a practical problem that requires the model to continuously learn novel relations while avoiding forgetting old ones with few labeled training data. The primary challenges are catastrophic forgetting and overfitting. This paper harnesses prompt learning to explore the implicit capabilities of pre-trained language models to address the above two challenges, thereby making language models better continual few-shot relation extractors. Specifically, we propose a Contrastive Prompt Learning framework, which designs prompt representation to acquire more generalized knowledge that can be easily adapted to old and new categories, and margin-based contrastive learning to focus more on hard samples, therefore alleviating catastrophic forgetting and overfitting issues. To further remedy overfitting in low-resource scenarios, we introduce an effective memory augmentation strategy that employs well-crafted prompts to guide ChatGPT in generating diverse samples. Extensive experiments demonstrate that our method outperforms state-of-the-art methods by a large margin and significantly mitigates catastrophic forgetting and overfitting in low-resource scenarios., Comment: Accepted as COLING2024

Advancements in the experimental toolbox of cold atoms have enabled the meticulous control of atomic Bloch oscillation within optical lattices, thereby enhancing the capabilities of gravity interferometers. This work delves into the impact of thermal effects on Bloch oscillation in 1D accelerated optical lattices aligned with gravity by varying the system's initial temperature. Through the application of Raman cooling, we effectively reduce the longitudinal thermal effect, stabilizing the longitudinal coherence length over the timescale of its lifetime. The atomic losses over multiple Bloch oscillation is measured, which are primarily attributed to transverse excitation. Furthermore, we identify two distinct inverse scaling behaviors in the oscillation lifetime scaled by the corresponding density with respect to temperatures, implying diverse equilibrium processes within or outside the Bose-Einstein condensate regime. The competition between the system's coherence and atomic density leads to a relatively smooth variation in the actual lifetime versus temperature. Our findings provide valuable insights into the interaction between thermal effects and Bloch oscillation, offering avenues for the refinement of quantum measurement technologies., Comment: 8 pages, 7 figures

Conventionally, intrinsic orbital angular momentum (OAM) is associated with phase vortices. However, our investigation into the propagation dynamics of 2D superimposed catastrophe beams, termed cyclone catastrophe beams (CCBs), reveals that these beams inherently exhibit rotation and possess OAM, distinct from the typical connection to phase vortices. Our observations clearly show these beams rotating during autofocusing propagation and particle manipulation, confirming the presence of OAM. Theoretical calculations affirm that the OAM of these beams is intrinsic and can be adjusted by varying the number of superimposed beams. Furthermore, our interference and phase studies indicate that, although CCBs exhibit phase vortices, they do not rotate around the singularities of phase vortices and their total topological charges are zero. This implies that the manifestation of OAM within CCBs does not rely on nonzero topological charge of the presented phase vortices within CCBs. Especially, eigenstates decomposition analysis illustrates that CCBs can be decomposed as a composite of Laguerre-Gaussian (LG) modes with uneven fidelity, where the topological charges of LG modes align with multiples of the superimposed catastrophe beams but do not equal to the value of the OAM per photon within CCBs, emphasizing the intrinsic OAM within CCBs and the absence of a connection to phase vortices. Our findings not only advance the understanding of the relationship between OAM and phase vortices but also pave the way for different applications of OAM waves, catalyzing their development in optics and other domains., Comment: 8pages, 5 figures

Large language models (LLMs) and vision language models (VLMs) demonstrate excellent performance on a wide range of tasks by scaling up parameter counts from O(10^9) to O(10^{12}) levels and further beyond. These large scales make it impossible to adapt and deploy fully specialized models given a task of interest. Parameter-efficient fine-tuning (PEFT) emerges as a promising direction to tackle the adaptation and serving challenges for such large models. We categorize PEFT techniques into two types: intrusive and non-intrusive. Intrusive PEFT techniques directly change a model's internal architecture. Though more flexible, they introduce significant complexities for training and serving. Non-intrusive PEFT techniques leave the internal architecture unchanged and only adapt model-external parameters, such as embeddings for input. In this work, we describe AdaLink as a non-intrusive PEFT technique that achieves competitive performance compared to SoTA intrusive PEFT (LoRA) and full model fine-tuning (FT) on various tasks. We evaluate using both text-only and multimodal tasks, with experiments that account for both parameter-count scaling and training regime (with and without instruction tuning).

Personalized text generation presents a specialized mechanism for delivering content that is specific to a user's personal context. While the research progress in this area has been rapid, evaluation still presents a challenge. Traditional automated metrics such as BLEU and ROUGE primarily measure lexical similarity to human-written references, and are not able to distinguish personalization from other subtle semantic aspects, thus falling short of capturing the nuances of personalized generated content quality. On the other hand, human judgments are costly to obtain, especially in the realm of personalized evaluation. Inspired by these challenges, we explore the use of large language models (LLMs) for evaluating personalized text generation, and examine their ability to understand nuanced user context. We present AuPEL, a novel evaluation method that distills three major semantic aspects of the generated text: personalization, quality and relevance, and automatically measures these aspects. To validate the effectiveness of AuPEL, we design carefully controlled experiments and compare the accuracy of the evaluation judgments made by LLMs versus that of judgements made by human annotators, and conduct rigorous analyses of the consistency and sensitivity of the proposed metric. We find that, compared to existing evaluation metrics, AuPEL not only distinguishes and ranks models based on their personalization abilities more accurately, but also presents commendable consistency and efficiency for this task. Our work suggests that using LLMs as the evaluators of personalized text generation is superior to traditional text similarity metrics, even though interesting new challenges still remain.

Large Language Models (LLMs), renowned for their remarkable performance across diverse domains, present a challenge when it comes to practical deployment due to their colossal model size. In response to this challenge, efforts have been directed toward the application of traditional network pruning techniques to LLMs, uncovering a massive number of parameters that can be pruned in one-shot without hurting performance. Prevailing LLM pruning strategies have consistently adhered to the practice of uniformly pruning all layers at equivalent sparsity, resulting in robust performance. However, this observation stands in contrast to the prevailing trends observed in the field of vision models, where non-uniform layerwise sparsity typically yields stronger results. To understand the underlying reasons for this disparity, we conduct a comprehensive study and discover a strong correlation with the emergence of activation outliers in LLMs. Inspired by this finding, we introduce a novel LLM pruning methodology that incorporates a tailored set of non-uniform layerwise sparsity ratios, termed as Outlier Weighed Layerwise sparsity (OWL). The sparsity ratio of OWL is proportional to the outlier ratio observed within each layer, facilitating a more effective alignment between layerwise weight sparsity and outlier ratios. Our empirical evaluation, conducted across the LLaMA-V1 family and OPT, spanning various benchmarks, demonstrates the distinct advantages offered by OWL over previous methods. For instance, OWL exhibits a remarkable performance gain, surpassing the state-of-the-art Wanda and SparseGPT by 61.22 and 6.80 perplexity at a high sparsity level of 70%, respectively, while delivering 2.6x end-to-end inference speed-up in the DeepSparse inference engine. Codes are available at https://github.com/luuyin/OWL.

Diffusion models achieve great success in generating diverse and high-fidelity images, yet their widespread application, especially in real-time scenarios, is hampered by their inherently slow generation speed. The slow generation stems from the necessity of multi-step network inference. While some certain predictions benefit from the full computation of the model in each sampling iteration, not every iteration requires the same amount of computation, potentially leading to inefficient computation. Unlike typical adaptive computation challenges that deal with single-step generation problems, diffusion processes with a multi-step generation need to dynamically adjust their computational resource allocation based on the ongoing assessment of each step's importance to the final image output, presenting a unique set of challenges. In this work, we propose AdaDiff, an adaptive framework that dynamically allocates computation resources in each sampling step to improve the generation efficiency of diffusion models. To assess the effects of changes in computational effort on image quality, we present a timestep-aware uncertainty estimation module (UEM). Integrated at each intermediate layer, the UEM evaluates the predictive uncertainty. This uncertainty measurement serves as an indicator for determining whether to terminate the inference process. Additionally, we introduce an uncertainty-aware layer-wise loss aimed at bridging the performance gap between full models and their adaptive counterparts.

We conducted a comprehensive study on the robust propagation of the same spot-size Gaussian beam (SSGB), same envelope Gaussian beam (SEGB), Circular Airy beam (CAB) and Circular Pearcey beam (CPB) in complex environments. Our findings clearly demonstrate that autofocusing beams exhibit higher stability in propagation compared with the Gaussian beams. To validate our results, we statistically analyze the intensity fluctuation of autofocusing beams and Gaussian beams. The analysis reveals that the intensity fluctuations of autofocusing beams are significantly smaller than that of Gaussian beams. Furthermore, we study the variation of the coherence factor and find that the coherence of autofocusing beams is better than that of Gaussian beams under turbulence. Additionally, we observe the change of the scintillation index (SI) with the propagation distance z, and our results show that autofocusing beams exhibit less oscillation than Gaussian beams, indicating that autofocusing beams propagate in complex environments with less distortion and fewer intensity fluctuation. Overall, our results suggest that autofocusing beams are promising for applications such as optical trapping and manipulation in complex environments. Our study provides valuable insights into the selection of stable beams that exhibit high intensity and high field gradient at the focal position in complex environments., Comment: 10pages, 6figures

We demonstrate that Schwarzschild spacetime has a conformal extension and that, beyond null infinity, there is a black hole with a timelike singularity. In conformal extended spacetime, every null infinity is a killing horizon with vanishing surface gravity. When a quantized massless scalar field is taken into this spacetime and different vacuums for the field are defined, thermal radiation coming from the extended black hole could be observed. This makes sense, much like the thermal radiation coming from the white hole. The Unruh effect is therefore plausible in conformal extended Schwarzschild spacetime. It is shown that the thermal radiation coming from past null infinity in Schwarzschild spacetime, which is difficult to imagine as the result of any physical process, is the result of the reduction of the thermal radiation passing through past null infinity in conformal extended spacetime. We also present a conformal extension of Kerr spacetime for the first time. Then, by examining a quantized massless scalar field on this spacetime, we get the meaningful conclusion that there is thermal radiation coming from a different rotating black hole passing through past null infinity. Similarly, the result of the Kerr black hole is consistent with that of the Schwarzschild black hole., Comment: 6 pages, 4 figures

Personalized text generation is an emerging research area that has attracted much attention in recent years. Most studies in this direction focus on a particular domain by designing bespoke features or models. In this work, we propose a general approach for personalized text generation using large language models (LLMs). Inspired by the practice of writing education, we develop a multistage and multitask framework to teach LLMs for personalized generation. In writing instruction, the task of writing from sources is often decomposed into multiple steps that involve finding, evaluating, summarizing, synthesizing, and integrating information. Analogously, our approach to personalized text generation consists of multiple stages: retrieval, ranking, summarization, synthesis, and generation. In addition, we introduce a multitask setting that helps the model improve its generation ability further, which is inspired by the observation in education that a student's reading proficiency and writing ability are often correlated. We evaluate our approach on three public datasets, each of which covers a different and representative domain. Our results show significant improvements over a variety of baselines.

We employed caustic theory to analyze the propagation dynamics of partially coherent self-accelerating beams such as self-healing of partially coherent Airy beams. Our findings revealed that as the spatial coherence decreases, the self-healing ability of beams increases. This result have been demonstrated both in simulation and experiment. This is an innovative application of the caustic theory to the field of partially coherent structured beams, and provides a comprehensive understanding of self-healing property. Our results have significant implications for practical applications of partially coherent beams in fields such as optical communication, encryption, and imaging., Comment: 9 pages, 4 figures

Circular swallowtail beams (CSBs) with their remarkable autofocusing capability have garnered significant interests due to their potential applications in optical trapping. This study delves into a comprehensive investigation of the trapping force properties of CSBs. Through a combination of experimental observations and theoretical analysis, we systematically explore the quantitative manipulation of trapping forces by adjusting specific parameters. This detailed investigation provides insights into the trapping force performance and stability of CSBs. Furthermore, the experimental validation of particle trapping using CSBs underscores their effectiveness, emphasizing their significant potential for optical manipulation and trapping applications., Comment: 6 pages, 4 figures

Learning new programming skills requires tailored guidance. With the emergence of advanced Natural Language Generation models like the ChatGPT API, there is now a possibility of creating a convenient and personalized tutoring system with AI for computer science education. This paper presents GPTutor, a ChatGPT-powered programming tool, which is a Visual Studio Code extension using the ChatGPT API to provide programming code explanations. By integrating Visual Studio Code API, GPTutor can comprehensively analyze the provided code by referencing the relevant source codes. As a result, GPTutor can use designed prompts to explain the selected code with a pop-up message. GPTutor is now published at the Visual Studio Code Extension Marketplace, and its source code is openly accessible on GitHub. Preliminary evaluation indicates that GPTutor delivers the most concise and accurate explanations compared to vanilla ChatGPT and GitHub Copilot. Moreover, the feedback from students and teachers indicated that GPTutor is user-friendly and can explain given codes satisfactorily. Finally, we discuss possible future research directions for GPTutor. This includes enhancing its performance and personalization via further prompt programming, as well as evaluating the effectiveness of GPTutor with real users., Comment: 6 pages. International Conference on Artificial Intelligence in Education 2023

The self-accelerating beams such as the Airy beam show great potentials in many applications including optical manipulation, imaging and communication. However, their superior features during linear propagation could be easily corrupted by optical nonlinearity or spatial incoherence individually. Here we investigate how the interaction of spatial incoherence and nonlinear propagation affect the beam quality of Airy beam, and find that the two destroying factors can in fact balance each other. Our results show that the influence of coherence and nonlinearity on the propagation of partially incoherent Airy beams (PIABs) can be formulated as two exponential functions that have factors of opposite signs. With appropriate spatial coherence length, the PIABs not only resist the corruption of beam profile caused by self-focusing nonlinearity, but also exhibits less anomalous diffraction caused by the self-defocusing nonlinearity. Our work provides deep insight into how to maintain the beam quality of self-accelerating Airy beams by exploiting the interaction between partially incoherence and optical nonlinearity. Our results may bring about new possibilities for optimizing partially incoherent structured field and developing related applications such as optical communication, incoherent imaging and optical manipulations., Comment: 12pages,6 figures

Polarized upconversion luminescence (PUCL) of lanthanide (Ln3+) ions has been widely used in single particle tracking, microfluidics detection, three-dimensional displays, and so on. However, no effective strategy has been developed for modulating PUCL. Here, we report a strategy to regulate PUCL in Ho3+-doped NaYF4 single nanorods based on the number of upconversion photons. By constructing a multiphoton upconversion system for Ho3+, we regulate the degree of polarization (DOP) of PUCL from 0.6 for two-photon luminescence to 0.929 for three-photon upconversion luminescence (UCL). Furthermore, our strategy is verified by cross-relaxation between Ho3+ and Yb3+, excitation wavelength, excitation power density, and local site symmetry. And this regulation strategy of PUCL has also been achieved in Tm3+, where DOP is ranged from 0.233 for two-photon luminescence to 0.925 for four-photon UCL. Besides, multi-dimensional anti-counterfeiting display has been explored with PUCL. This work provides an effective strategy for the regulation of PUCL and also provides more opportunities for the development of polarization display, optical encoding, anti-counterfeiting, and integrated optical devices.

Photonic analogs of the moir\'e superlattices mediated by interlayer electromagnetic coupling are expected to give rise to rich phenomena such as nontrivial flatband topology. Here, we propose and demonstrate a scheme to tune the flatbands in a bilayer moir\'e superlattice by employing the band offset. The band offset is changed by fixing the bands of one slab while shifting those of the other slab, which is accomplished by modifying the thickness of the latter slab. Our results show that the band-offset tuning not only makes some flatbands emerge and disappear, but also leads to two sets of flatbands that are robustly formed even with the change of band offset over a broad range. These robust flatbands form either at the AA-stack site or at the AB-stack site, and as a result, a single-cell superlattice can support a pair of high-quality localized modes with tunable frequencies. Moreover, we develop a diagrammatic model to provide an intuitive insight into the formation of the robust flatbands. Our work demonstrates a simple yet efficient way to design and control complex moir\'e flatbands, providing new opportunities to utilize photonic moir\'e superlattices for advanced light-matter interaction including lasing and nonlinear harmonic generation., Comment: 5 pages, 5 figures

Large-scale Transformer models bring significant improvements for various downstream vision language tasks with a unified architecture. The performance improvements come with increasing model size, resulting in slow inference speed and increased cost for severing. While some certain predictions benefit from the full complexity of the large-scale model, not all of inputs need the same amount of computation to conduct, potentially leading to computation resource waste. To handle this challenge, early exiting is proposed to adaptively allocate computational power in term of input complexity to improve inference efficiency. The existing early exiting strategies usually adopt output confidence based on intermediate layers as a proxy of input complexity to incur the decision of skipping following layers. However, such strategies cannot apply to encoder in the widely-used unified architecture with both encoder and decoder due to difficulty of output confidence estimation in the encoder. It is suboptimal in term of saving computation power to ignore the early exiting in encoder component. To handle this challenge, we propose a novel early exiting strategy for unified visual language models, which allows dynamically skip the layers in encoder and decoder simultaneously in term of input layer-wise similarities with multiple times of early exiting, namely \textbf{MuE}. By decomposing the image and text modalities in the encoder, MuE is flexible and can skip different layers in term of modalities, advancing the inference efficiency while minimizing performance drop. Experiments on the SNLI-VE and MS COCO datasets show that the proposed approach MuE can reduce expected inference time by up to 50\% and 40\% while maintaining 99\% and 96\% performance respectively.

Self-supervised pre-training recently demonstrates success on large-scale multimodal data, and state-of-the-art contrastive learning methods often enforce the feature consistency from cross-modality inputs, such as video/audio or video/text pairs. Despite its convenience to formulate and leverage in practice, such cross-modality alignment (CMA) is only a weak and noisy supervision, since two modalities can be semantically misaligned even they are temporally aligned. For example, even in the commonly adopted instructional videos, a speaker can sometimes refer to something that is not visually present in the current frame; and the semantic misalignment would only be more unpredictable for the raw videos from the internet. We conjecture that might cause conflicts and biases among modalities, and may hence prohibit CMA from scaling up to training with larger and more heterogeneous data. This paper first verifies our conjecture by observing that, even in the latest VATT pre-training using only instructional videos, there exist strong gradient conflicts between different CMA losses within the same video, audio, text triplet, indicating them as the noisy source of supervision. We then propose to harmonize such gradients, via two techniques: (i) cross-modality gradient realignment: modifying different CMA loss gradients for each sample triplet, so that their gradient directions are more aligned; and (ii) gradient-based curriculum learning: leveraging the gradient conflict information on an indicator of sample noisiness, to develop a curriculum learning strategy to prioritize training on less noisy sample triplets. Applying those techniques to pre-training VATT on the HowTo100M dataset, we consistently improve its performance on different downstream tasks. Moreover, we are able to scale VATT pre-training to more complicated non-narrative Youtube8M dataset to further improve the state-of-the-arts., Comment: Accepted at NeurIPS 2022