Search

Increasing educational attainment is one of the most important and effective tools for health and economic improvements. The extent to which extreme climate events disrupt education, resulting in fewer years of schooling and reduced educational attainment, remains under-studied. Children in low- and middle-income countries may be uniquely vulnerable to loss of schooling after such disasters due to the poor physical condition of schools and the lack of resources to rebuild and mitigate unexpected household shocks. Our analysis assesses this overlooked social cost of tropical cyclones on schooling attainment. We study the education records of nearly 5.1 million people living in 13 low- and middle-income countries that were exposed to tropical cyclones between 1954-2010. We find that exposure to tropical cyclones during preschool age is associated with a 2.7 percentage point decrease in primary school enrollment on average (14.2% decrease), with larger effects from more intense storms (up to 28% decrease for the most intense storms). These effects are more pronounced among school-age girls compared to boys and are greater in areas less accustomed to experiencing tropical cyclones. We estimate that, across all LMICs, tropical cyclone exposure has resulted in more than 410,000 children not attending primary school in the last 20 years, leading to a reduction of more than 4.1 million total years of schooling. These impacts, identified among some of the world's poorest populations, may grow in importance as exposure to severe tropical cyclones is projected to increase with climate change.

The Stellar Abundances and Galactic Evolution Survey (SAGES) is a multi-band survey that covers the northern sky area of ~12000 deg2. Nanshan One-meter Wide-field Telescope (NOWT) of Xinjiang Astronomical Observatory (XAO) carried out observations on g/r/i bands. We present here the survey strategy, data processing, catalog construction, and database schema. The observations of NOWT started in 2016 August and was completed in 2018 January, total 17827 frames were obtained and ~4600 deg2 sky areas were covered. In this paper, we released the catalog of the data in the g/r/i bands observed with NOWT. In total, there are 109,197,578 items of the source records. The catalog is the supplement for the SDSS for the bright end, and the combination of our catalog and these catalogs could be helpful for source selections for other surveys and the Milky Way sciences, e.g., white dwarf candidates and stellar flares., Comment: 12 pages, 8 figures, accepted for publication in RAA

Modern radar typically employs a chain architecture that consists of radio-frequency (RF) and intermediate frequency (IF) units, baseband digital signal processor, and information display. However, this architecture often results in high costs, significant hardware demands, and integration challenges. Here we propose a simplified radar architecture based on space-time-coding (STC) information metasurfaces. With their powerful capabilities to generate multiple harmonic frequencies and customize their phases, the STC metasurfaces play a key role in chirp signal generation, transmission, and echo reception. Remarkably, the receiving STC metasurface can implement dechirp processing directly on the RF level and realize the digital information outputs, which are beneficial to lower the hardware requirement at the receiving end while potentially shortening the time needed for conventional digital processing. As a proof of concept, the proposed metasurface radar is tested in a series of experiments for target detection and range/speed measurement, yielding results comparable to those obtained by conventional methods. This study provides valuable inspiration for a new radar system paradigm to combine the RF front ends and signal processors on the information metasurface platform that offers essential functionalities while significantly reducing the system complexity and cost., Comment: 25 pages, 10 figures

The performance of Large Language Models (LLMs) is substantially influenced by the pretraining corpus, which consists of vast quantities of unsupervised data processed by the models. Despite its critical role in model performance, ensuring the quality of this data is challenging due to its sheer volume and the absence of sample-level quality annotations and enhancements. In this paper, we introduce DecorateLM, a data engineering method designed to refine the pretraining corpus through data rating, tagging and editing. Specifically, DecorateLM rates texts against quality criteria, tags texts with hierarchical labels, and edits texts into a more formalized format. Due to the massive size of the pretraining corpus, adopting an LLM for decorating the entire corpus is less efficient. Therefore, to balance performance with efficiency, we curate a meticulously annotated training corpus for DecorateLM using a large language model and distill data engineering expertise into a compact 1.2 billion parameter small language model (SLM). We then apply DecorateLM to enhance 100 billion tokens of the training corpus, selecting 45 billion tokens that exemplify high quality and diversity for the further training of another 1.2 billion parameter LLM. Our results demonstrate that employing such high-quality data can significantly boost model performance, showcasing a powerful approach to enhance the quality of the pretraining corpus.

Quantum collision model provides a promising tool for investigating system-bath dynamics. Most of the studies on quantum collision models work in the resonant regime. In quantum dynamics, the off-resonant interaction often brings in exciting ffects. It is thereby attractive to investigate quantum collision models in the off-resonant regime. On the other hand, a bath with a negative absolute temperature is anticipated to be instrumental in developing thermal devices. The design of an effective bath with negative absolute temperature coupled to a qubit is significant for developing such thermal devices. We establish an effective negative-absolute-temperature bath coupled to a qubit with a quantum collision model in a far-off-resonant regime. We conduct a detailed and systematic investigation on the off-resonant collision model. There is an additional constraint on the collision duration resulting from the far-off resonant collision. The dynamics of the collision model in the far-off-resonant regime are different from the one beyond the far-off-resonant regime. Numerical simulations confirm the validity of the proposed approach., Comment: 7 pages, 3 figures

Distinct from the $\text{SU}(2)$ case, the fermionic systems with $\text{SU}(N)$ symmetry are expected to exhibit novel physics, such as exotic singlet formation. Using the density matrix renormalization group technique, we obtain the ground state phase diagram of the $\text{SU}(3)$ $t$-$J$ chain for density $n<1$. The ground state phase diagram includes the Luttinger liquid, the extended Luther-Emery liquid characterized by a spin gap, and the phase separation state. We quantitatively assess the characteristics of the three phases by measuring spin gap, compressibility, various correlation functions and structure factors. We further study the extended Luther-Emery liquid phase and discover molecular superfluid quasi-long-range order. The mechanism of the molecular superfluid is the combination of three $\text{SU}(3)$ fermions on sites that are not completely connected. Accordingly, we can speculate the behavior of the $\text{SU}(N)$ $t$-$J$ chain model with larger $N$ values, operating within the same filling regime., Comment: 8 pages, 8 figures

The [O III] $\lambda\lambda\ 4960,5008$ emission lines in the optical spectra of galaxies and quasars have been widely used to investigate the possible variation of the fine-structure constant $\alpha$ over cosmic time. In this work, we utilize the Large Sky Area Multi-object Fiber Spectroscopic Telescope (LAMOST) quasar survey, for the first time, to measure the relative $\alpha$ variation $\Delta\alpha/\alpha$ in time through the [O III] doublet method. From the LAMOST Data Release 9 quasar catalog, we refine a sample of 209 quasar spectra with strong and narrow [O III] emission lines over a redshift range of $0

Navigating efficiently to an object in an unexplored environment is a critical skill for general-purpose intelligent robots. Recent approaches to this object goal navigation problem have embraced a modular strategy, integrating classical exploration algorithms-notably frontier exploration-with a learned semantic mapping/exploration module. This paper introduces a novel informative path planning and 3D object probability mapping approach. The mapping module computes the probability of the object of interest through semantic segmentation and a Bayes filter. Additionally, it stores probabilities for common objects, which semantically guides the exploration based on common sense priors from a large language model. The planner terminates when the current viewpoint captures enough voxels identified with high confidence as the object of interest. Although our planner follows a zero-shot approach, it achieves state-of-the-art performance as measured by the Success weighted by Path Length (SPL) and Soft SPL in the Habitat ObjectNav Challenge 2023, outperforming other works by more than 20%. Furthermore, we validate its effectiveness on real robots. Project webpage: https://ippon-paper.github.io/

We study non-perturbative effects of torus partition function of the $T\bar{T}$-deformed 2d CFTs by resurgence. The deformed partition function can be written as an infinite series of the deformation parameter $\lambda$. We develop highly efficient methods to compute perturbative coefficients in the $\lambda$ expansion. To exemplify, the first 600 coefficients for the $T\bar{T}$-deformed free boson and free fermion are computed. Equipped with the large order perturbative data, we provide convincing numerical evidence that the $\lambda$ expansion series is asymptotic and not Borel resummable. We extract the non-perturbative contribution by resurgence and propose that they originate from new complex saddle points after analytically continuing the modular parameters in the integral representation of the partition function. The proposal is checked by comparing the predicted asymptotic behavior of the coefficients and large order perturbative data, which match nicely. The implications of these non-perturbative contributions for the Stokes phenomenon, which relates the positive and negative signs of $\lambda$, is also discussed., Comment: 20 pages, 6 figures

The fabrication of well-defined, low-dimensional diamondoid-based materials is a promising approach for tailoring diamond properties such as superconductivity. On-surface self-assembly of halogenated diamondoids under ultrahigh vacuum conditions represents an effective strategy in this direction, enabling reactivity exploration and on-surface synthesis approaches. Here we demonstrate through scanning probe microscopy, time-of-flight mass spectrometry and photoelectron spectroscopy, that self-assembled layers of dibromodiamantanes on gold can be debrominated at atomic wavelengths (Al K$\alpha$ at 8.87 $\mathring{A}$ and Mg K$\alpha$ at 9.89 $\mathring{A}$) and low temperatures without affecting their well-defined arrangement. The resulting 'in-architecture' debromination enables the fabrication of diamantane chains from self-assembled precursors in close proximity, which is otherwise inaccessible through annealing on metal surfaces. Our work introduces a novel approach for the fabrication of nanodiamond chains, with significant implications for in-architecture and layer-by-layer synthesis.

The Hubble tension refers to the significant discrepancy in the Hubble constant $H_{0}$ obtained from two different measurement methods in cosmology. One method derives data from the Cosmic Microwave Background (CMB) observations by the Planck satellite, yielding a value of $67.4\pm{0.5} \ \mathrm{km\ s^{-1}} \mathrm{Mpc^{-1}} $, while the other method relies on direct measurements of Type Ia supernovae, producing a value $73.04\pm{1.04} \ \mathrm{km\ s^{-1}} \mathrm{Mpc^{-1}} $. This issue has persisted for several years. To theoretically explore potential solutions to this problem, this paper examines a model within the framework of Einstein-Cartan (EC) theory, where torsion is introduced with spin as the corresponding entity, allowing for the assumption $H = -\alpha \phi$. By employing the Markov Chain Monte Carlo (MCMC) algorithm and utilizing Cosmic Chronometers (CC) data, we impose parameter constraints on various parameters in the Friedmann equations, particularly focusing on the curvature density parameter $\Omega_k$, to assess whether the model remains stable under this assumption and whether the estimated parameters align more closely with either of the observational results. In conclusion, we find that the parameter constraints in the model incorporating torsion ($ H_0 = 67.6^{+2.1}_{-2.7} \ \mathrm{km\ s^{-1}\ Mpc^{-1}}$, obtained under the Big Bang Nucleosynthesis (BBN) constraint with $\Omega_{k}=0$; $ H_0 = 66.2^{+4.4}_{-2.9} \ \mathrm{km\ s^{-1}\ Mpc^{-1}}$, obtained under same constraint but set $\Omega_{k}$ as a free variable; $ H_0 = 68.8^{+2.9}_{-4.2} \ \mathrm{km\ s^{-1}\ Mpc^{-1}}$, obtained under the Planck constraint) are more consistent with the value derived from CMB data, favoring the lower $H_0$ value., Comment: 13 pages,5 figures

Superluminal propagation is an intrinsic problem in the diffusion equation and has not been effectively addressed for a long time. In this work, a rigorous solution to this issue is obtained under the assumption that particles undergo a random flight process, where they move isotropically at a constant speed while experiencing random scatterings. We validate this solution by comparing it with comprehensive simulations of the random flight process and find that it significantly deviates from the solution derived from the J\"{u}ttner propagator. This solution is broadly applicable to various diffusion phenomena, such as cosmic-ray propagation. We emphasize that our rigorous solution is particularly crucial in scenarios involving burst-like particle injection, where previous phenomenological approaches to the superluminal diffusion problem may not yield accurate results., Comment: 8 pages, 4 figures

The interplay between crystal electric field splitting of d states and Hund's rule exchange energy in cobalt-based perovskites offers a promising avenue for inducing spin-state transitions. This study reports a new body-centered tetragonal (BCT) phase of BaCoO$_3$ (BCT-BaCoO$_3$), synthesized under high pressure (15 GPa) and high temperature (1200 {\deg}C) conditions. BCT-BaCoO$_3$ adopts a double perovskite structure of EuTiO$_3$-type (space group I4/mcm, #140), confirmed by high-resolution scanning transmission electron microscopy. X-ray photoelectron spectroscopy reveals a rare Co$^{4+}$ valence state. Magnetization and X-ray absorption measurements reveal a low-spin to high-spin transition that takes place between 200 and 300 K. While spin crossovers are relatively common among common oxides, the one observed in BCT-BaCoO$_3$ is remarkable in that it proceeds in the opposite direction from conventional spin transitions. BCT-BaCoO$_3$ exhibits a low-spin (S = 1/2) state at high temperatures and transitions to a high-spin (S = 5/2) state at low temperatures. Within the high-spin state, hard ferromagnetic order onsets at T$_C$ = 107 K. Electrical resistivity indicates weak magnetoresistance and insulating behavior. Overall, BCT-BaCoO$_3$ presents an exceptional model for the exploration of spin-state transitions and the study of Co spin states in cobalt-based perovskites., Comment: 22+14 pages, 5+7 figures

The task of macro- and micro-expression spotting aims to precisely localize and categorize temporal expression instances within untrimmed videos. Given the sparse distribution and varying durations of expressions, existing anchor-based methods often represent instances by encoding their deviations from predefined anchors. Additionally, these methods typically slice the untrimmed videos into fixed-length sliding windows. However, anchor-based encoding often fails to capture all training intervals, and slicing the original video as sliding windows can result in valuable training intervals being discarded. To overcome these limitations, we introduce PESFormer, a simple yet effective model based on the vision transformer architecture to achieve point-to-interval expression spotting. PESFormer employs a direct timestamp encoding (DTE) approach to replace anchors, enabling binary classification of each timestamp instead of optimizing entire ground truths. Thus, all training intervals are retained in the form of discrete timestamps. To maximize the utilization of training intervals, we enhance the preprocessing process by replacing the short videos produced through the sliding window method.Instead, we implement a strategy that involves zero-padding the untrimmed training videos to create uniform, longer videos of a predetermined duration. This operation efficiently preserves the original training intervals and eliminates video slice enhancement.Extensive qualitative and quantitative evaluations on three datasets -- CAS(ME)^2, CAS(ME)^3 and SAMM-LV -- demonstrate that our PESFormer outperforms existing techniques, achieving the best performance.

Cardiovascular diseases (CVD) remain a leading health concern and contribute significantly to global mortality rates. While clinical advancements have led to a decline in CVD mortality, accurately identifying individuals who could benefit from preventive interventions remains an unsolved challenge in preventive cardiology. Current CVD risk prediction models, recommended by guidelines, are based on limited traditional risk factors or use CT imaging to acquire quantitative biomarkers, and still have limitations in predictive accuracy and applicability. On the other hand, end-to-end trained CVD risk prediction methods leveraging deep learning on CT images often fail to provide transparent and explainable decision grounds for assisting physicians. In this work, we proposed a novel joint representation that integrates discrete quantitative biomarkers and continuous deep features extracted from chest CT scans. Our approach initiated with a deep CVD risk classification model by capturing comprehensive continuous deep learning features while jointly obtaining currently clinical-established quantitative biomarkers via segmentation models. In the feature joint representation stage, we use an instance-wise feature-gated mechanism to align the continuous and discrete features, followed by a soft instance-wise feature interaction mechanism fostering independent and effective feature interaction for the final CVD risk prediction. Our method substantially improves CVD risk predictive performance and offers individual contribution analysis of each biomarker, which is important in assisting physicians' decision-making processes. We validated our method on a public chest low-dose CT dataset and a private external chest standard-dose CT patient cohort of 17,207 CT volumes from 6,393 unique subjects, and demonstrated superior predictive performance, achieving AUCs of 0.875 and 0.843, respectively., Comment: 23 pages, 9 figures

Graphs are an intuitive way to represent relationships between variables in fields such as finance and neuroscience. However, these graphs often need to be inferred from data. In this paper, we propose a novel framework to infer a latent graph by treating the observed multidimensional data as graph-referenced stationary signals. Specifically, we introduce the graphical autoregressive model (GAR), where the inverse covariance matrix of the observed signals is expressed as a second-order polynomial of the normalized graph Laplacian of the latent graph. The GAR model extends the autoregressive model from time series analysis to general undirected graphs, offering a new approach to graph inference. To estimate the latent graph, we develop a three-step procedure based on penalized maximum likelihood, supported by theoretical analysis and numerical experiments. Simulation studies and an application to S&P 500 stock price data show that the GAR model can outperform Gaussian graphical models when it fits the observed data well. Our results suggest that the GAR model offers a promising new direction for inferring latent graphs across diverse applications. Codes and example scripts are available at https://github.com/jed-harwood/SGM .

Two-dimensional (2D) van der Waals (vdW) magnets with layer-dependent magnetic states and/or diverse magnetic interactions and anisotropies have attracted extensive research interest. Despite the advances, a notable challenge persists in effectively manipulating the tunneling anisotropic magnetoresistance (TAMR) of 2D vdW magnet-based magnetic tunnel junctions (MTJs). Here, we report the novel and anomalous tunneling magnetoresistance (TMR) oscillations and pioneering demonstration of bias and gate voltage controllable TAMR in 2D vdw MTJs, utilizing few-layer CrPS4. This material, inherently an antiferromagnet, transitions to a canted magnetic order upon application of external magnetic fields. Through TMR measurements, we unveil the novel, layer-dependent oscillations in the tunneling resistance for few-layer CrPS4 devices under both out-of-plane and in-plane magnetic fields, with a pronounced controllability via gate voltage. Intriguingly, we demonstrate that both the polarity and magnitude of TAMR in CrPS4 can be effectively tuned through either a bias or gate voltage. We further elucidate the mechanism behind this electrically tunable TAMR through first-principles calculations. The implications of our findings are far-reaching, providing new insights into 2D magnetism and opening avenues for the development of innovative spintronic devices based on 2D vdW magnets., Comment: 22 pages, 4 figures

Ensuring that generative AI systems align with human values is essential but challenging, especially when considering multiple human values and their potential trade-offs. Since human values can be personalized and dynamically change over time, the desirable levels of value alignment vary across different ethnic groups, industry sectors, and user cohorts. Within existing frameworks, it is hard to define human values and align AI systems accordingly across different directions simultaneously, such as harmlessness, helpfulness, and positiveness. To address this, we develop a novel, first-principle approach called Multi-Human-Value Alignment Palette (MAP), which navigates the alignment across multiple human values in a structured and reliable way. MAP formulates the alignment problem as an optimization task with user-defined constraints, which define human value targets. It can be efficiently solved via a primal-dual approach, which determines whether a user-defined alignment target is achievable and how to achieve it. We conduct a detailed theoretical analysis of MAP by quantifying the trade-offs between values, the sensitivity to constraints, the fundamental connection between multi-value alignment and sequential alignment, and proving that linear weighted rewards are sufficient for multi-value alignment. Extensive experiments demonstrate MAP's ability to align multiple values in a principled manner while delivering strong empirical performance across various tasks.

Magnetoresistance (MR) oscillations serve as a hallmark of intrinsic quantum behavior, traditionally observed only in conducting systems. Here we report the discovery of MR oscillations in an insulating system, the vertical junctions of CrPS$_4$ which is a two dimensional (2D) A-type antiferromagnetic semiconductor. Systematic investigations of MR peaks under varying conditions, including electrode materials, magnetic field direction, temperature, voltage bias and layer number, elucidate a correlation between MR oscillations and spin-canted states in CrPS$_4$. Experimental data and analysis point out the important role of the in-gap electronic states in generating MR oscillations, and we proposed that spin selected interlayer hopping of localized states may be responsible for it. Our findings not only illuminate the unusual electronic transport in CrPS$_4$ but also underscore the potential of van der Waals magnets for exploring interesting phenomena.

Long-Context Question Answering (LCQA), a challenging task, aims to reason over long-context documents to yield accurate answers to questions. Existing long-context Large Language Models (LLMs) for LCQA often struggle with the "lost in the middle" issue. Retrieval-Augmented Generation (RAG) mitigates this issue by providing external factual evidence. However, its chunking strategy disrupts the global long-context information, and its low-quality retrieval in long contexts hinders LLMs from identifying effective factual details due to substantial noise. To this end, we propose LongRAG, a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG's understanding of complex long-context knowledge (i.e., global information and factual details). We design LongRAG as a plug-and-play paradigm, facilitating adaptation to various domains and LLMs. Extensive experiments on three multi-hop datasets demonstrate that LongRAG significantly outperforms long-context LLMs (up by 6.94%), advanced RAG (up by 6.16%), and Vanilla RAG (up by 17.25%). Furthermore, we conduct quantitative ablation studies and multi-dimensional analyses, highlighting the effectiveness of the system's components and fine-tuning strategies. Data and code are available at https://github.com/QingFei1/LongRAG., Comment: EMNLP 2024 Main

Inter-generational communication is essential for bridging generational gaps and fostering mutual understanding. However, maintaining it is complex due to cultural, communicative, and geographical differences. Recent research indicated that while Virtual Reality (VR) creates a relaxed atmosphere and promotes companionship, it inadequately addresses the complexities of inter-generational dialogue, including variations in values and relational dynamics. To address this gap, we explored the opportunities of Artificial Intelligence (AI) in supporting inter-generational communication in VR. We developed three technology probes (e.g., Content Generator, Communication Facilitator, and Info Assistant) in VR and employed them in a probe-based participatory design study with twelve inter-generational pairs. Our results show that AI-powered VR facilitates inter-generational communication by enhancing mutual understanding, fostering conversation fluency, and promoting active participation. We also introduce several challenges when using AI-powered VR in supporting inter-generational communication and derive design implications for future VR platforms, aiming to improve inter-generational communication.

Since the discovery of superconductor one hundred years ago, tremendous theoretical and technological progresses have been achieved. The zero resistance and complete diamagnetism of superconducting materials promise many possibilities in diverse fields. However, the complexity and expensive manufacturing costs associated with the time-consuming superconductor fabrication process may retard their practices in a large extent. Here, via liquid metal printing we proposed to quickly fabricate superconducting electronics which can work at the prescribed cryogenic temperatures. By way of the room temperature fluidity of liquid metal composite inks, such one-step printing allows to pattern various superconducting circuits on the desired substrate. As the first-ever conceptual trial, the most easily available gallium-based liquid alloy inks were particularly adopted to composite with copper particles to achieve superconductivity under specific temperatures around 6.4K. Further, a series of liquid metal alloy and particles loaded composites were screened out and comparatively interpreted regarding their superconducting properties and potential values as printable inks in fabricating superconducting devices. The cost-effective feature and straightforward adaptability of the fabrication principle were evaluated. This work suggests an easy-going way for fabricating ending user superconducting devices, which may warrant more promising investigations and practices in the coming time., Comment: 14 pages, 4 figures, 2 tables

In recent years, Facial Expression Recognition (FER) has gained increasing attention. Most current work focuses on supervised learning, which requires a large amount of labeled and diverse images, while FER suffers from the scarcity of large, diverse datasets and annotation difficulty. To address these problems, we focus on utilizing large unlabeled Face Recognition (FR) datasets to boost semi-supervised FER. Specifically, we first perform face reconstruction pre-training on large-scale facial images without annotations to learn features of facial geometry and expression regions, followed by two-stage fine-tuning on FER datasets with limited labels. In addition, to further alleviate the scarcity of labeled and diverse images, we propose a Mixup-based data augmentation strategy tailored for facial images, and the loss weights of real and virtual images are determined according to the intersection-over-union (IoU) of the faces in the two images. Experiments on RAF-DB, AffectNet, and FERPlus show that our method outperforms existing semi-supervised FER methods and achieves new state-of-the-art performance. Remarkably, with only 5%, 25% training sets,our method achieves 64.02% on AffectNet,and 88.23% on RAF-DB, which is comparable to fully supervised state-of-the-art methods. Codes will be made publicly available at https://github.com/zhelishisongjie/SSFER.

The autonomous driving industry is rapidly advancing, with Vehicle-to-Vehicle (V2V) communication systems highlighting as a key component of enhanced road safety and traffic efficiency. This paper introduces a novel Real-time Vehicle-to-Vehicle Communication Based Network Cooperative Control System (VVCCS), designed to revolutionize macro-scope traffic planning and collision avoidance in autonomous driving. Implemented on Quanser Car (Qcar) hardware platform, our system integrates the distributed databases into individual autonomous vehicles and an optional central server. We also developed a comprehensive multi-modal perception system with multi-objective tracking and radar sensing. Through a demonstration within a physical crossroad environment, our system showcases its potential to be applied in congested and complex urban environments., Comment: ICICT 2024, 18 pages

Ensuring awareness of fairness and privacy in Large Language Models (LLMs) is critical. Interestingly, we discover a counter-intuitive trade-off phenomenon that enhancing an LLM's privacy awareness through Supervised Fine-Tuning (SFT) methods significantly decreases its fairness awareness with thousands of samples. To address this issue, inspired by the information theory, we introduce a training-free method to \textbf{DEA}ctivate the fairness and privacy coupled \textbf{N}eurons (\textbf{DEAN}), which theoretically and empirically decrease the mutual information between fairness and privacy awareness. Extensive experimental results demonstrate that DEAN eliminates the trade-off phenomenon and significantly improves LLMs' fairness and privacy awareness simultaneously, \eg improving Qwen-2-7B-Instruct's fairness awareness by 12.2\% and privacy awareness by 14.0\%. More crucially, DEAN remains robust and effective with limited annotated data or even when only malicious fine-tuning data is available, whereas SFT methods may fail to perform properly in such scenarios. We hope this study provides valuable insights into concurrently addressing fairness and privacy concerns in LLMs and can be integrated into comprehensive frameworks to develop more ethical and responsible AI systems. Our code is available at \url{https://github.com/ChnQ/DEAN}.

Companies like Amazon and UPS are heavily invested in last-mile delivery problems. Optimizing last-delivery operations not only creates tremendous cost savings for these companies but also generate broader societal and environmental benefits in terms of better delivery service and reduced air pollutants and greenhouse gas emissions. Last-mile delivery is readily formulated as the Travelling Salesman Problem (TSP), where a salesperson must visit several cities and return to the origin with the least cost. A solution to this problem is a Hamiltonian circuit in an undirected graph. Many methods exist for solving the TSP, but they often assume the travel costs are fixed. In practice, travel costs between delivery zones are random quantities, as they are subject to variation from traffic, weather, and other factors. Innovations such as truck-drone last-mile delivery creates even more uncertainties due to scarce data. A Bayesian D-optimal experimental design in conjunction with a regression model are proposed to estimate these unknown travel costs, and subsequently search for a highly efficient solution to the TSP. This framework can naturally be extended to incorporate the use of drones and any other emerging technology that has use in last-mile delivery., Comment: 22 Pages, 2 Figures with 4 subfigure panels each, To be submitted to Quality Engineering

We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to accurately represent specular surfaces. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy that significantly enhances both scene geometry and specular color prediction. We demonstrate that our model outperforms prior methods for view synthesis of scenes containing dynamic specular objects and that it is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes., Comment: Project page: https://cdfan0627.github.io/spectromotion/

Knowledge distillation (KD) is widely used to train small, high-performing student language models (LMs) using large teacher LMs. While effective in fine-tuning, KD during pre-training faces challenges in efficiency, flexibility, and effectiveness. Existing methods either incur high computational costs due to online teacher inference, require tokenization matching between teacher and student LMs, or risk losing the difficulty and diversity of the teacher-generated training data. To address these issues, we propose MiniPLM, a KD framework for pre-training LMs by refining the training data distribution with the teacher's knowledge. For efficiency, MiniPLM performs offline teacher LM inference, allowing KD for multiple student LMs without adding training-time costs. For flexibility, MiniPLM operates solely on the training corpus, enabling KD across model families. For effectiveness, MiniPLM leverages the differences between large and small LMs to enhance the difficulty and diversity of the training data, helping student LMs acquire versatile and sophisticated knowledge. Extensive experiments demonstrate that MiniPLM boosts the student LMs' performance on 9 widely used downstream tasks, improves the language modeling capabilities, and reduces pre-training computation. The benefit of MiniPLM extends to large pre-training scales, evidenced by the extrapolation of the scaling curves. Further analysis reveals that MiniPLM supports KD across model families and enhances the utilization of pre-training data. Our model, code, and data are available at https://github.com/thu-coai/MiniPLM.

Text sanitization, which employs differential privacy to replace sensitive tokens with new ones, represents a significant technique for privacy protection. Typically, its performance in preserving privacy is evaluated by measuring the attack success rate (ASR) of reconstruction attacks, where attackers attempt to recover the original tokens from the sanitized ones. However, current reconstruction attacks on text sanitization are developed empirically, making it challenging to accurately assess the effectiveness of sanitization. In this paper, we aim to provide a more accurate evaluation of sanitization effectiveness. Inspired by the works of Palamidessi et al., we implement theoretically optimal reconstruction attacks targeting text sanitization. We derive their bounds on ASR as benchmarks for evaluating sanitization performance. For real-world applications, we propose two practical reconstruction attacks based on these theoretical findings. Our experimental results underscore the necessity of reassessing these overlooked risks. Notably, one of our attacks achieves a 46.4% improvement in ASR over the state-of-the-art baseline, with a privacy budget of epsilon=4.0 on the SST-2 dataset. Our code is available at: https://github.com/mengtong0110/On-the-Vulnerability-of-Text-Sanitization.

Non-Abelian braiding is a key property of Majorana zero modes (MZMs) that can be utilized for topological quantum computation. However, the presence of trivial Andreev bound states (ABSs) in topological superconductors can hinder the non-Abelian braiding of MZMs. We systematically investigate the braiding properties of ABSs induced by various inhomogeneous potentials in nanowires and quantify the main obstacles to non-Abelian braiding. We find that if a trivial ABSs is present at zero energy with a tiny energy fluctuation, their non-Abelian braiding property can be sustained for a longer braiding time cost, since the undesired dynamic phase is suppressed. Under certain conditions, the non-Abelian braiding of ABSs can even surpass that of MZMs in realistic systems, suggesting that ABSs might also be suitable for topological quantum computation., Comment: 13 pages, 9 figures

The integration of ferroelectrics with semiconductors is crucial for developing functional devices, such as field-effect transistors, tunnel junctions, and nonvolatile memories. However, the synthesis of high-quality single-crystalline ferroelectric nitride perovskites has been limited, hindering a comprehensive understanding of their switching dynamics and potential applications. Here we report the synthesis and characterizations of epitaxial single-phase ferroelectric cerium tantalum nitride (CeTaN3) on both oxides and semiconductors. The polar symmetry of CeTaN3 was confirmed by observing the atomic displacement of central ions relative to the center of the TaN6 octahedra, as well as through optical second harmonic generation. We observed switchable ferroelectric domains in CeTaN3 films using piezo-response force microscopy, complemented by the characterization of square-like polarization-electric field hysteresis loops. The remanent polarization of CeTaN3 reaches approximately 20 uC/cm2 at room temperature, consistent with theoretical calculations. This work establishes a vital link between ferroelectric nitride perovskites and their practical applications, paving the way for next-generation information and energy-storage devices with enhanced performance, scalability, and manufacturability., Comment: 47 pages, 4 figures

Camouflaged Object Segmentation (COS) faces significant challenges due to the scarcity of annotated data, where meticulous pixel-level annotation is both labor-intensive and costly, primarily due to the intricate object-background boundaries. Addressing the core question, "Can COS be effectively achieved in a zero-shot manner without manual annotations for any camouflaged object?" we affirmatively respond and introduce a robust zero-shot COS framework. This framework leverages the inherent local pattern bias of COS and employs a broad semantic feature space derived from salient object segmentation (SOS) for efficient zero-shot transfer. We incorporate an Masked Image Modeling (MIM) based image encoder optimized for Parameter-Efficient Fine-Tuning (PEFT), a Multimodal Large Language Model (M-LLM), and a Multi-scale Fine-grained Alignment (MFA) mechanism. The MIM pre-trained image encoder focuses on capturing essential low-level features, while the M-LLM generates caption embeddings processed alongside these visual cues. These embeddings are precisely aligned using MFA, enabling our framework to accurately interpret and navigate complex semantic contexts. To optimize operational efficiency, we introduce a learnable codebook that represents the M-LLM during inference, significantly reducing computational overhead. Our framework demonstrates its versatility and efficacy through rigorous experimentation, achieving state-of-the-art performance in zero-shot COS with $F_{\beta}^w$ scores of 72.9\% on CAMO and 71.7\% on COD10K. By removing the M-LLM during inference, we achieve an inference speed comparable to that of traditional end-to-end models, reaching 18.1 FPS. Code: https://github.com/R-LEI360725/ZSCOS-CaMF

Auctions, a fundamental economic mechanism, encompass the valuation of goods or services and the competitive bidding algorithms within a specific framework, serving to uncover the true market value. However, current research predominantly focuses on the bidding algorithms within a given auction mechanism, often overlooking the advantages of incorporating individual bidders' unique preferences and the semantic information related to the items into the valuation process. Our analysis, both theoretical and empirical, shows that imprecise or noisy valuations can significantly affect the overall utility for participants. To bridge this gap, we propose a personalized valuation framework, namely \textbf{S}emantic-enhanced \textbf{P}ersonalized \textbf{V}aluation in \textbf{A}uction (\ours), which integrates Large Language Models (LLMs) to incorporate semantic information into each bidder's unique valuation process. Specifically, SPVA employs a two-stage approach: it first fine-tunes LLMs to encode bidder preferences in personalized valuations, and then constructs a Vickrey auction environment integrated with a bidding algorithm to demonstrate that SPVA's more accurate valuations result in higher profits. Additionally, we have developed a semantic-enhanced dataset comprising over 23,000 samples and introduced new personalized evaluation metrics that reflect both bidder preferences and profit. Through simulations of various auction scenarios, our method demonstrates its ability to provide accurate valuations and capture bidder preferences, affirming the method's effectiveness in real-world auction settings., Comment: 14 pages, 5 figures

Let $M$ be a closed orientable $(n-1)$-connected $(2n+2)$-manifold, $n\geq 2$. In this paper we combine the Postnikov tower of spheres and the homotopy decomposition of the reduced suspension space $\Sigma M$ to investigate the cohomotopy sets $\pi^\ast(M)$ for $n=2,3,4$, under the assumption that $M$ has $2$-torsion-free homology. All cohomotopy sets $\pi^i(M)$ of such manifolds $M$ are characterized except $\pi^4(M)$ for $n=3,4$., Comment: 31 pages

We explored the Patrol Security Game (PSG), a robotic patrolling problem modeled as an extensive-form Stackelberg game, where the attacker determines the timing, location, and duration of their attack. Our objective is to devise a patrolling schedule with an infinite time horizon that minimizes the attacker's payoff. We demonstrated that PSG can be transformed into a combinatorial minimax problem with a closed-form objective function. By constraining the defender's strategy to a time-homogeneous first-order Markov chain (i.e., the patroller's next move depends solely on their current location), we proved that the optimal solution in cases of zero penalty involves either minimizing the expected hitting time or return time, depending on the attacker model, and that these solutions can be computed efficiently. Additionally, we observed that increasing the randomness in the patrol schedule reduces the attacker's expected payoff in high-penalty cases. However, the minimax problem becomes non-convex in other scenarios. To address this, we formulated a bi-criteria optimization problem incorporating two objectives: expected maximum reward and entropy. We proposed three graph-based algorithms and one deep reinforcement learning model, designed to efficiently balance the trade-off between these two objectives. Notably, the third algorithm can identify the optimal deterministic patrol schedule, though its runtime grows exponentially with the number of patrol spots. Experimental results validate the effectiveness and scalability of our solutions, demonstrating that our approaches outperform state-of-the-art baselines on both synthetic and real-world crime datasets., Comment: Under review of TCPS

We present MMCS, a system capable of recognizing medical images and patient facial details, and providing professional medical diagnoses. The system consists of two core components: The first component is the analysis of medical images and videos. We trained a specialized multimodal medical model capable of interpreting medical images and accurately analyzing patients' facial emotions and facial paralysis conditions. The model achieved an accuracy of 72.59% on the FER2013 facial emotion recognition dataset, with a 91.1% accuracy in recognizing the happy emotion. In facial paralysis recognition, the model reached an accuracy of 92%, which is 30% higher than that of GPT-4o. Based on this model, we developed a parser for analyzing facial movement videos of patients with facial paralysis, achieving precise grading of the paralysis severity. In tests on 30 videos of facial paralysis patients, the system demonstrated a grading accuracy of 83.3%.The second component is the generation of professional medical responses. We employed a large language model, integrated with a medical knowledge base, to generate professional diagnoses based on the analysis of medical images or videos. The core innovation lies in our development of a department-specific knowledge base routing management mechanism, in which the large language model categorizes data by medical departments and, during the retrieval process, determines the appropriate knowledge base to query. This significantly improves retrieval accuracy in the RAG (retrieval-augmented generation) process. This mechanism led to an average increase of 4 percentage points in accuracy for various large language models on the MedQA dataset.Our code is open-sourced and available at: https://github.com/renllll/MMCS.

The antiferromagnetism in transition metal compounds is mostly mediated by the bridging anions through a so-called superexchange mechanism. However, in materials like normal spinels $AB_2X_4$ with local moments only at the $A$ site, such an anion-mediated superexchange needs to be modified. Here we report a new spinel compound Co$_{1+x}$Ir$_{2-x}$S$_4$ ($x$ = 0.3). The physical property measurements strongly suggest an antiferromagnetic-like transition at 292 K in the Co($A$) diamond sublattice. The first-principle calculations reveal that the nearest-neighbor Co($A$) spins align antiferromagnetically with an ordered magnetic moment of 1.67 $\mu_\mathrm{B}$, smaller than the expected $S = 3/2$ for Co$^{2+}$. In the antiferromagnetic state, there exists an inter-cation charge-transfer gap between the non-bonding Ir-$t_\mathrm{2g}$ orbitals at the valence band maximum and the Co-S antibonding molecular orbitals at the conduction band minimum. The small charge transfer energy significantly enhances the virtual hopping between these two states, facilitating a robust long-range superexchange interaction between two neighboring CoS$_4$ complexes, which accounts for the high N\'{e}el temperature in Co$_{1+x}$Ir$_{2-x}$S$_4$. This inter-cation charge transfer mediated magnetic interaction expands the traditional superexchange theory, which could be applicable in complex magnetic materials with multiple cations., Comment: 10 pages, 7 figures

Introducing uniform magnetic order in two-dimensional topological insulators (2D TIs) by constructing heterostructures of TI and magnet is a promising way to realize the high-temperature Quantum Anomalous Hall effect. However, the topological properties of 2D materials are susceptible to several factors that make them difficult to maintain, and whether topological interfacial states (TISs) can exist at magnetic-topological heterostructure interfaces is largely unknown. Here, we experimentally show that TISs in a lateral heterostructure of CrTe_{2}/Bi(110) are robust against disorder, defects, high magnetic fields (time-reversal symmetry breaking perturbations), and elevated temperature (77 K). The lateral heterostructure is realized by lateral epitaxial growth of bilayer (BL) Bi to monolayer CrTe_{2} grown on HOPG. Scanning Tunneling Microscopy and non-contact Atomic Force Microscopy demonstrate a black phosphorus-like structure with low atomic buckling (less than 0.1 {\AA}) of the BL Bi(110), indicating the presence of its topological properties. Scanning tunneling spectroscopy and energy-dependent dI/dV mapping further confirm the existence of topologically induced one-dimensional in-gap states localized at the interface. These results demonstrate the robustness of TISs in lateral magnetic-topological heterostructures, which is competitive with those in vertically stacked magnetic-topological heterostructures, and provides a promising route for constructing planar high-density non-dissipative devices using TISs.

Recently, it has been revealed that a variety of novel phenomena emerge in hyperbolic spaces, while non-Hermitian physics has significantly enriched the landscape of condensed matter physics. Building on these developments, we construct a geodesic-based method to study the non-Hermitian skin effect (NHSE) in non-reciprocal hyperbolic lattices. Additionally, we develop a geodesic-periodic boundary condition (geodesic-PBC), akin to the Euclidean periodic boundary condition (PBC), that complements its open boundary condition. Importantly, we find that the non-reciprocal directionality within a hyperbolic regular polygon and the geodesic-based boundary determine the spectral sensitivity, and hence, the NHSE. Unlike in Euclidean models, however, we must utilize boundary localization to distinguish non-trivial skin modes from their trivial boundary counterpart due to the extensive boundary volume of hyperbolic lattices. We also relate the spatial density profile with the finite-size scaling of hyperbolic lattices. These aspects highlight the profound impact of hyperbolic geometry on non-Hermitian systems and offer new perspectives on the intricate relationship between the geometric characteristics of hyperbolic lattices and non-Hermitian physics., Comment: 13 pages and 10 figures

We propose Counterfactual Analysis Quadratic Unconstrained Binary Optimization (CAQUBO) to solve QUBO problems for feature selection in recommender systems. CAQUBO leverages counterfactual analysis to measure the impact of individual features and feature combinations on model performance and employs the measurements to construct the coefficient matrix for a quantum annealer to select the optimal feature combinations for recommender systems, thereby improving their final recommendation performance. By establishing explicit connections between features and the recommendation performance, the proposed approach demonstrates superior performance compared to the state-of-the-art quantum annealing methods. Extensive experiments indicate that integrating quantum computing with counterfactual analysis holds great promise for addressing these challenges.

Accurate time-series forecasting is essential across a multitude of scientific and industrial domains, yet deep learning models often struggle with challenges such as capturing long-term dependencies and adapting to drift in data distributions over time. We introduce Future-Guided Learning, an approach that enhances time-series event forecasting through a dynamic feedback mechanism inspired by predictive coding. Our approach involves two models: a detection model that analyzes future data to identify critical events and a forecasting model that predicts these events based on present data. When discrepancies arise between the forecasting and detection models, the forecasting model undergoes more substantial updates, effectively minimizing surprise and adapting to shifts in the data distribution by aligning its predictions with actual future outcomes. This feedback loop, drawing upon principles of predictive coding, enables the forecasting model to dynamically adjust its parameters, improving accuracy by focusing on features that remain relevant despite changes in the underlying data. We validate our method on a variety of tasks such as seizure prediction in biomedical signal analysis and forecasting in dynamical systems, achieving a 40\% increase in the area under the receiver operating characteristic curve (AUC-ROC) and a 10\% reduction in mean absolute error (MAE), respectively. By incorporating a predictive feedback mechanism that adapts to data distribution drift, Future-Guided Learning offers a promising avenue for advancing time-series forecasting with deep learning.

Ultra-long gamma-ray bursts (ULGRBs) are characterized by exceptionally long-duration central engine activities, with characteristic timescales exceeding 1000 seconds. We present ground-based optical afterglow observations of the ultra-long gamma-ray burst GRB 211024B, detected by \textit{Swift}. Its X-ray light curve exhibits a characteristic ``internal plateau" with a shallow decay phase lasting approximately $\sim 15$ ks, followed by a steep decline ($\alpha_{\rm drop}\sim-7.5$). Moreover, the early optical emission predicted by the late r-band optical afterglow is significantly higher than the observed value, indicating an external shock with energy injection. To explain these observations, we propose a magnetar central engine model. The magnetar collapse into a black hole due to spin-down or hyperaccretion, leading to the observed steep break in the X-ray light curve. The afterglow model fitting reveals that the afterglow injection luminosity varies with different assumptions of the circumburst medium density, implying different potential energy sources. For the interstellar medium (ISM) case with a fixed injection end time, the energy may originate from the magnetar's dipole radiation. However, in other scenarios, relativistic jets produced by the magnetar/black hole system could be the primary energy source., Comment: 18 pages, 7 figures, accepted by ApJ

Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM) -- which enhances models with up-to-date knowledge -- emerges as a promising method to reduce hallucination. However, existing RALMs may instead exacerbate hallucination when retrieving lengthy contexts. To address this challenge, we propose COFT, a novel \textbf{CO}arse-to-\textbf{F}ine highligh\textbf{T}ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts. Specifically, COFT consists of three components: \textit{recaller}, \textit{scorer}, and \textit{selector}. First, \textit{recaller} applies a knowledge graph to extract potential key entities in a given context. Second, \textit{scorer} measures the importance of each entity by calculating its contextual weight. Finally, \textit{selector} selects high contextual weight entities with a dynamic threshold algorithm and highlights the corresponding paragraphs, sentences, or words in a coarse-to-fine manner. Extensive experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30\%$ in the F1 score metric. Moreover, COFT also exhibits remarkable versatility across various long-form tasks, such as reading comprehension and question answering.

This paper proposes a Generative Face Video Compression (GFVC) approach using Supplemental Enhancement Information (SEI), where a series of compact spatial and temporal representations of a face video signal (i.e., 2D/3D key-points, facial semantics and compact features) can be coded using SEI message and inserted into the coded video bitstream. At the time of writing, the proposed GFVC approach is an official "technology under consideration" (TuC) for standardization by the Joint Video Experts Team (JVET) of ISO/IEC JVT 1/SC 29 and ITU-T SG16. To the best of the authors' knowledge, the JVET work on the proposed SEI-based GFVC approach is the first standardization activity for generative video compression. The proposed SEI approach has not only advanced the reconstruction quality of early-day Model-Based Coding (MBC) via the state-of-the-art generative technique, but also established a new SEI definition for future GFVC applications and deployment. Experimental results illustrate that the proposed SEI-based GFVC approach can achieve remarkable rate-distortion performance compared with the latest Versatile Video Coding (VVC) standard, whilst also potentially enabling a wide variety of functionalities including user-specified animation/filtering and metaverse-related applications.

For superconducting quantum processors, stable high-fidelity two-qubit operations depend on precise flux control of the tunable coupler. However, the pulse distortion poses a significant challenge to the control precision. Current calibration methods, which often rely on microwave crosstalk or additional readout resonators for coupler excitation and readout, tend to be cumbersome and inefficient, especially when couplers only have flux control. Here, we introduce and experimentally validate a novel pulse calibration scheme that exploits the strong coupling between qubits and couplers, eliminating the need for extra coupler readout and excitation. Our method directly measures the short-time and long-time step responses of the coupler flux pulse transient, enabling us to apply predistortion to subsequent signals using fast Fourier transformation and deconvolution. This approach not only simplifies the calibration process but also significantly improves the precision and stability of the flux control. We demonstrate the efficacy of our method through the implementation of diabatic CZ and iSWAP gates with fidelities of $99.61\pm0.04\%$ and $99.82\pm0.02\%$, respectively, as well as a series of diabatic CPhase gates with high fidelities characterized by cross-entropy benchmarking. The consistency and robustness of our technique are further validated by the reduction in pulse distortion and phase error observed across multilayer CZ gates. These results underscore the potential of our calibration and predistortion method to enhance the performance of two-qubit gates in superconducting quantum processors., Comment: 16 pages, 11 figures, 7 tables

Nearly 900 million people live in low-lying coastal zones around the world and bear the brunt of impacts from more frequent and severe hurricanes and storm surges. Oceanographers simulate ocean current circulation along the coasts to develop early warning systems that save lives and prevent loss and damage to property from coastal hazards. Traditionally, such simulations are conducted using coastal ocean circulation models such as the Regional Ocean Modeling System (ROMS), which usually runs on an HPC cluster with multiple CPU cores. However, the process is time-consuming and energy expensive. While coarse-grained ROMS simulations offer faster alternatives, they sacrifice detail and accuracy, particularly in complex coastal environments. Recent advances in deep learning and GPU architecture have enabled the development of faster AI (neural network) surrogates. This paper introduces an AI surrogate based on a 4D Swin Transformer to simulate coastal tidal wave propagation in an estuary for both hindcast and forecast (up to 12 days). Our approach not only accelerates simulations but also incorporates a physics-based constraint to detect and correct inaccurate results, ensuring reliability while minimizing manual intervention. We develop a fully GPU-accelerated workflow, optimizing the model training and inference pipeline on NVIDIA DGX-2 A100 GPUs. Our experiments demonstrate that our AI surrogate reduces the time cost of 12-day forecasting of traditional ROMS simulations from 9,908 seconds (on 512 CPU cores) to 22 seconds (on one A100 GPU), achieving over 450$\times$ speedup while maintaining high-quality simulation results. This work contributes to oceanographic modeling by offering a fast, accurate, and physically consistent alternative to traditional simulation models, particularly for real-time forecasting in rapid disaster response.

Multimodal Chain of Thought (MCoT) is a popular prompting strategy for improving the performance of multimodal large language models (MLLMs) across a range of complex reasoning tasks. Despite its popularity, there is a notable absence of automated methods for evaluating the quality of reasoning steps in MCoT. To address this gap, we propose Multimodal Chain-of-Thought Evaluation (MiCEval), a framework designed to assess the correctness of reasoning chains by evaluating the quality of both the description and each reasoning step. The evaluation of the description component focuses on the accuracy of the image descriptions, while the reasoning step evaluates the quality of each step as it is conditionally generated based on the preceding steps. MiCEval is built upon a fine-grained dataset with annotations that rate each step according to correctness, relevance, and informativeness. Extensive experiments on four state-of-the-art MLLMs show that step-wise evaluations using MiCEval align more closely with human judgments compared to existing methods based on cosine similarity or fine-tuning approaches. MiCEval datasets and code can be found in https://github.com/alenai97/MiCEval., Comment: 40 pages

Model merging-based multitask learning (MTL) offers a promising approach for performing MTL by merging multiple expert models without requiring access to raw training data. However, in this paper, we examine the merged model's representation distribution and uncover a critical issue of "representation bias". This bias arises from a significant distribution gap between the representations of the merged and expert models, leading to the suboptimal performance of the merged MTL model. To address this challenge, we first propose a representation surgery solution called Surgery. Surgery is a lightweight, task-specific module that aligns the final layer representations of the merged model with those of the expert models, effectively alleviating bias and improving the merged model's performance. Despite these improvements, a performance gap remains compared to the traditional MTL method. Further analysis reveals that representation bias phenomena exist at each layer of the merged model, and aligning representations only in the last layer is insufficient for fully reducing systemic bias because biases introduced at each layer can accumulate and interact in complex ways. To tackle this, we then propose a more comprehensive solution, deep representation surgery (also called SurgeryV2), which mitigates representation bias across all layers, and thus bridges the performance gap between model merging-based MTL and traditional MTL. Finally, we design an unsupervised optimization objective to optimize both the Surgery and SurgeryV2 modules. Our experimental results show that incorporating these modules into state-of-the-art (SOTA) model merging schemes leads to significant performance gains. Notably, our SurgeryV2 scheme reaches almost the same level as individual expert models or the traditional MTL model. The code is available at \url{https://github.com/EnnengYang/SurgeryV2}., Comment: This paper is an extended version of our previous work [arXiv:2402.02705] presented at ICML 2024

Protecting the intellectual property of open-source Large Language Models (LLMs) is very important, because training LLMs costs extensive computational resources and data. Therefore, model owners and third parties need to identify whether a suspect model is a subsequent development of the victim model. To this end, we propose a training-free REEF to identify the relationship between the suspect and victim models from the perspective of LLMs' feature representations. Specifically, REEF computes and compares the centered kernel alignment similarity between the representations of a suspect model and a victim model on the same samples. This training-free REEF does not impair the model's general capabilities and is robust to sequential fine-tuning, pruning, model merging, and permutations. In this way, REEF provides a simple and effective way for third parties and models' owners to protect LLMs' intellectual property together. The code is available at https://github.com/tmylla/REEF.

Although Large Language Models (LLMs) have demonstrated remarkable capabilities, their massive parameter counts and associated extensive computing make LLMs' deployment the main part of carbon emission from nowadays AI applications. Compared to modern GPUs like H$100$, it would be significantly carbon-sustainable if we could leverage old-fashioned GPUs such as M$40$ (as shown in Figure 1, M$40$ only has one third carbon emission of H$100$'s) for LLM servings. However, the limited High Bandwidth Memory (HBM) available on such GPU often cannot support the loading of LLMs due to the gigantic model size and intermediate activation data, making their serving challenging. For instance, a LLaMA2 model with $70$B parameters typically requires $128$GB for inference, which substantially surpasses $24$GB HBM in a $3090$ GPU and remains infeasible even considering the additional $64$GB DRAM. To address this challenge, this paper proposes a mixed-precision with a model modularization algorithm to enable LLM inference on outdated hardware with resource constraints. (The precision denotes the numerical precision like FP16, INT8, INT4) and multi-level caching (M2Cache).) Specifically, our M2Cache first modulizes neurons in LLM and creates their importance ranking. Then, it adopts a dynamic sparse mixed-precision quantization mechanism in weight space to reduce computational demands and communication overhead at each decoding step. It collectively lowers the operational carbon emissions associated with LLM inference. Moreover, M2Cache introduces a three-level cache management system with HBM, DRAM, and SSDs that complements the dynamic sparse mixed-precision inference. To enhance communication efficiency, M2Cache maintains a neuron-level mixed-precision LRU cache in HBM, a larger layer-aware cache in DRAM, and a full model in SSD., Comment: 24 pages, 13 figures