Search

Grain boundaries (GBs) trigger structure-specific chemical segregation of solute atoms. According to the three-dimensional (3D) topology of grains, GBs - although defined as planar defects - cannot be free of curvature. This implies formation of topologically-necessary arrays of secondary GB dislocations. We report here that these pattern-forming secondary GB dislocations can have an additional and, in some cases, even a much stronger effect on GB segregation than defect-free GBs. Using nanoscale correlative tomography combining crystallography and chemical analysis, we quantified the relationship between secondary GB dislocations and their segregation energy spectra for a model Fe-W alloy. This discovery unlocks new design opportunities for advanced materials, leveraging the additional degrees of freedom provided by topologically-necessary secondary GB dislocations to modulate segregation.

Large language models (LLMs) can solve an increasing number of complex reasoning tasks while making surprising mistakes in basic numerical understanding and processing (such as 9.11 > 9.9). The latter ability is essential for tackling complex arithmetic and mathematical problems and serves as a foundation for most reasoning tasks, but previous work paid little attention to it or only discussed several restricted tasks (like integer addition). In this paper, we comprehensively investigate the numerical understanding and processing ability (NUPA) of LLMs. Firstly, we introduce a benchmark covering four common numerical representations and 17 distinct numerical tasks in four major categories, resulting in 41 meaningful combinations in total. These tasks are derived from primary and secondary education curricula, encompassing nearly all everyday numerical understanding and processing scenarios, and the rules of these tasks are very simple and clear. Through the benchmark, we find that current LLMs fail frequently in many of the tasks. To study the problem, we train small models with existing and potential techniques for enhancing NUPA (such as special tokenizers, PEs, and number formats), comprehensively evaluating their effectiveness using our testbed. We also finetune practical-scale LLMs on our proposed NUPA tasks and find that 1) naive finetuning can improve NUPA a lot on many but not all tasks, and 2) surprisingly, techniques designed to enhance NUPA prove ineffective for finetuning pretrained models. We further explore the impact of chain-of-thought techniques on NUPA. Our work takes a preliminary step towards understanding and improving NUPA of LLMs. Our benchmark and code are released at https://github.com/GraphPKU/number_cookbook.

Massive black hole binaries are one of the important sources for the TianQin project. Our research has revealed that, for TianQin, the signal-to-noise ratio squared during the inspiral phase of massive black hole binaries exhibits a direct proportionality to the ratio of the observation duration to the time remaining until coalescence. This finding is expected to greatly simplify the estimation of detection capabilities for massive black hole binaries. In this paper, we demonstrated this relationship under both all-sky average and non-average conditions. The latter introduces only an additional term, which we refer to as the response factor. Although this term is not easily calculated analytically, we provide a simple estimation method with an error margin of within 2%., Comment: 12 pages, 5 figures, comments welcome

Extreme-mass-ratio inspirals (EMRIs), consisting of a massive black hole and a stellar compact object, are one of the most important sources for space-borne gravitational wave detectors like TianQin. Their population study can be used to constrain astrophysical models that interpret the EMRI formation mechanisms. In this paper, as a first step, we employ a parametrization method to describe the EMRI population model in the loss cone formation channel. This approach, however, can be extended to other models such as the accretion disc driven formation channel. We present the phenomenological characteristic of the MBH mass, spin, and redshift distributions. Then, we investigate the posterior distribution of the hyper-parameters that describe this population model. Our results show that TianQin could recover almost all the posterior of the hyper-parameters within $1\sigma$ confidence interval. With one hundred detectable EMRI events, the hyper-parameters $\alpha_1, \alpha_2, b$, which describe the MBH mass distribution, could be measured with an accuracy of $37\%$, $24\%$, and $3\%$, respectively. The hyper-parameters $\mu_z$, and $\sigma_z$, which describe the redshift distribution, have $\mu_z$ above the detectable range of TianQin, and $\sigma_z$ measured with an accuracy of $14.5\%$. With this estimation accuracy, the EMRI population characteristics can be effectively demonstrated, potentially serving as evidence for EMRI formation in the future studies. Furthermore, with an increasing number of detectable events, the parameter estimation for the hyper-parameters will improve and the confidence intervals will be narrowed., Comment: 11 pages, 5 figures

The distributed telescope array offers promise for conducting large-sky-area, high-frequency time domain surveys. Multiple telescopes can be deployed at each observation site, so intra-site observation task scheduling is crucial for enhancing observation efficiency and quality. Efficient use of observable time and rapid response to special situations are critical to maximize scientific discovery in time domain surveys. Besides, the competing scientific priorities, time-varying observation conditions, and capabilities of observation equipment, lead to a vast search space of the scheduling. So with the increasing number of telescopes and observation fields, balancing computational time with solution quality in observation scheduling poses a significant challenge. Informed by the seminal contributions of earlier studies on a multilevel scheduling model and global scheduler for time domain telescope array, this study is devoted to further exploring the site scheduler. Formulating the observation scheduling of multiple telescopes at the site as a cooperative decision-making problem, this paper proposes GRRIS, a real-time intra-site observation scheduling scheme for telescope array using graph and reinforcement learning. It employs a graph neural network to learn node features that can embed the spatial structure of the observation scheduling. An algorithm based on multi-agent reinforcement learning is designed to efficiently learn the optimum allocation policy of telescope agents to field nodes. Through numerical simulations with real-world scenarios, GRRIS can achieve up to a 22% solution improvement over the most competitive scheme. It offers better scalability and sub-second decision speed, meeting the needs of observation scheduling control for future distributed telescope arrays., Comment: Astronomical Journal

The opening of the gravitational wave window has significantly enhanced our capacity to explore the universe's most extreme and dynamic sector. In the mHz frequency range, a diverse range of compact objects, from the most massive black holes at the farthest reaches of the Universe to the lightest white dwarfs in our cosmic backyard, generate a complex and dynamic symphony of gravitational wave signals. Once recorded by gravitational wave detectors, these unique fingerprints have the potential to decipher the birth and growth of cosmic structures over a wide range of scales, from stellar binaries and stellar clusters to galaxies and large-scale structures. The TianQin space-borne gravitational wave mission is scheduled for launch in the 2030s, with an operational lifespan of five years. It will facilitate pivotal insights into the history of our universe. This document presents a concise overview of the detectable sources of TianQin, outlining their characteristics, the challenges they present, and the expected impact of the TianQin observatory on our understanding of them., Comment: TianQin Gravitational Wave Whitepaper, 72 pages, 30 figures

As a space-borne gravitational wave observatory, TianQin can observe a large variety of gravitational wave sources. The rich signals can be composed by different types of astronomical systems, like Galactic compact binaries, inspiral of stellar mass black holes, merger of massive black holes, and extreme mass ratio inspirals. The incoherent summation of these signals can also form a stochastic gravitational wave background. Using the future TianQin observation, it is possible to put stringent constraints on fundamental science, like the expansion history of our universe. In this work, we provide a brief review of these topics that contributes to the 7th International Workshop on the TianQin Science Mission., Comment: Prepared for the proceeding of the 7th International Workshop on the TianQin Science Mission. 12 pages, 2 figures

Even in non-interacting limit, electrons on different Bloch bands of a multiband system do not move as if they are oblivious to the presence of one another. Instead, they move in concert by virtue of a non-Abelian interband Berry connection. While the impact of this quantum geometric attribute manifests most famously through the Hall response of topological bands, the geometric effects in superconductors have attracted significant recent attention. In particular, much has been discussed about the quantum-metric-induced superfluid weight (SW) in flatband superconductors. In this study, we revisit the geometric SW in generic multiband superconductors and trace its origin to a series of microscopic processes. We separately derive the SW of models containing only intraband Cooper pairing and those involving interband pairing. Two classes of processes enabled by the so-called interband velocity (or the closely related interband Berry connection) are identified: one resembles the transfer of Cooper pairs between different bands, and the other corresponds to virtual single-electron back-and-forth tunneling between the bands. The former contribution manifests as effective Josephson coupling between the multiband superconducting order parameters, while the latter constitutes the only source of SW for a superconducting flatband well isolated from other bands. We further numerically evaluate the SW of a simple two-band superconductor with trivial band topology, showcasing how the geometric contribution is sensitive to the details of the multiband pairing configuration. In particular, we highlight an intriguing scenario of negative SW, which may pave way for the formation of novel pair density wave order. Our study provides deeper and more intuitive insight into the origin and nature of the SW induced by the quantum geometry of paired Bloch states., Comment: 12 pages, 4 figures

In the milli-Hertz frequency band, stochastic gravitational-wave background can be composed of both astronomical and cosmological sources, both can be anisotropic. Numerically depicting these anisotropies can be critical in revealing the underlying properties of their origins. For the first time, we perform a theoretical analysis of the constraining ability of TianQin on multiple moments of the stochastic background. First, we find that with a one-year operation, for a background with a signal-to-noise ratio of 16, TianQin can recover the multiple moments up to $l=4$. We also identified a unique feature of the stochastic background sky map, which is the mirror symmetry along the fixed orbital plane of TianQin. Thirdly, we explain the difference in anisotropy recovering ability between TianQin and LISA, by employing the criteria of the singularity of the covariance matrix (which is the condition number). Finally, we find that since the different data channel combinations correspond to different singularities, certain combinations might have an advantage in stochastic background map-making. We believe that the findings of this work can provide an important reference to future stochastic background analysis pipelines. It can also serve as a guideline for designing better gravitational-wave detectors aiming to decipher anisotropies in the stochastic background., Comment: 12 pages, 6 figures

Space-borne detectors, including TianQin and Laser Interferometry Space Antenna (LISA), are tasked with the simultaneous observation of Galactic foreground, astrophysical and cosmological stochastic gravitational-wave backgrounds (SGWBs). For the first time, we employ a space-borne detector network to identify the multi-component SGWB. Specifically, we develop a tailored likelihood for cross-correlation detection with such networks. Combined with the likelihood, we use the simulated datasets of the TianQin + LISA network to conduct model selection and parameter estimation. Our results indicate that, after 4 years of operation, the network could detect a single SGWB from either astrophysical or cosmological origins, with an energy density $\Omega_{\rm ast/cos}$ (10 mHz) on the order of $10^{-12}$, despite the presence of a Galactic foreground. Furthermore, to distinguish the cosmological background from both a Galactic foreground and the expected astrophysical background, the energy density $\Omega_{\rm cos}$ should reach around $2\times 10^{-11}$., Comment: 17 pages, 9 figures

Critical single-particle fluctuations associated with particle displacements are inherent to simple glass-forming liquids in the limit of large dimensions and leave a pseudo-critical trace across all finite dimensions. This characteristic could serve as a crucial test for distinguishing between theories of glass formation. We here examine these critical fluctuations, as captured by the well-established non-Gaussian parameter, within both mode-coupling theory (MCT) and dynamical mean-field theory (DMFT) across dimensions for hard sphere liquids and for the minimally structured Mari--Kurchan model. We establish general scaling laws relevant to any liquid dynamics theory in large dimensions and show that the dimensional scalings predicted by MCT are inconsistent with those from DMFT. Simulation results for hard spheres in moderately high dimensions align with the DMFT scenario, reinforcing the relevance of mean-field theory for capturing glass physics in finite dimensions. We identify potential adjustments to MCT to account for certain mean-field physics. Our findings also highlight that local structure and spatial dimensionality can affect single-particle critical fluctuations in non-trivial ways., Comment: 42 pages, 9 figures

Recent advancements in digital pathology have led to the development of numerous foundational models that utilize self-supervised learning on patches extracted from gigapixel whole slide images (WSIs). While this approach leverages vast amounts of unlabeled data, we have discovered a significant issue: features extracted from these self-supervised models tend to cluster by individual WSIs, a phenomenon we term WSI-specific feature collapse. This problem can potentially limit the model's generalization ability and performance on various downstream tasks. To address this issue, we introduce EXAONEPath, a novel foundational model trained on patches that have undergone stain normalization. Stain normalization helps reduce color variability arising from different laboratories and scanners, enabling the model to learn more consistent features. EXAONEPath is trained using 285,153,903 patches extracted from a total of 34,795 WSIs. Our experiments demonstrate that EXAONEPath significantly mitigates the feature collapse problem, indicating that the model has learned more generalized features rather than overfitting to individual WSI characteristics. We compared EXAONEPath with state-of-the-art models across six downstream task datasets, and our results show that EXAONEPath achieves superior performance relative to the number of WSIs used and the model's parameter count. This suggests that the application of stain normalization has substantially improved the model's efficiency and generalization capabilities., Comment: License updated

The early inspiral from stellar-mass binary black holes can emit milli-Hertz gravitational wave signals, making them detectable sources for space-borne gravitational wave missions like TianQin. However, the traditional matched filtering technique poses a significant challenge for analyzing this kind of signals, as it requires an impractically high number of templates ranging from $10^{31}$ to $10^{40}$. We propose a search strategy that involves two main parts: initially, we reduce the dimensionality of the simulated signals using incremental principal component analysis (IPCA). Subsequently we train the convolutional neural networks (CNNs) based on the compressed TianQin data obtained from IPCA, aiming to develop both a detection model and a point parameter estimation model. The compression efficiency for the trained IPCA model achieves a cumulative variance ratio of 95.6% when applied to $10^6$ simulated signals. To evaluate the performance of CNN we generate the receiver operating characteristic curve for the detection model which is applied to the test data with varying signal-to-noise ratios. At a false alarm probability of 5% the corresponding true alarm probability for signals with a signal-to-noise ratio of 50 is 86.5%. Subsequently, we introduce the point estimation model to evaluate the value of the chirp mass of corresponding sBBH signals with an error. For signals with a signal-to-noise ratio of 50, the trained point estimation CNN model can estimate the chirp mass of most test events, with a standard deviation error of 2.48 $M_{\odot}$ and a relative error precision of 0.12., Comment: 12 pages, 10 figures

Silicon strip detectors have been widely utilized in space experiments for gamma-ray and cosmic-ray detections thanks to their high spatial resolution and stable performance. For a silicon micro-strip detector, the Monte Carlo simulation is recognized as a practical and cost-effective approach to verify the detector performance. In this study, a technique for the simulation of the silicon micro-strip detector with the $\rm Allpix^{2}$ framework is developed. By incorporating the electric field into the particle transport simulation based on Geant4, this framework could precisely emulate the carrier drift in the silicon micro-strip detector. The simulation results are validated using the beam test data as well as the flight data of the DAMPE experiment, which suggests that the $\rm Allpix^{2}$ framework is a powerful tool to obtain the performance of the silicon micro-strip detector.

This paper presents a novel method for utilizing fine-tuned Large Language Models (LLMs) to minimize data requirements in load profile analysis, demonstrated through the restoration of missing data in power system load profiles. A two-stage fine-tuning strategy is proposed to adapt a pre-trained LLMs, i.e., GPT-3.5, for missing data restoration tasks. Through empirical evaluation, we demonstrate the effectiveness of the fine-tuned model in accurately restoring missing data, achieving comparable performance to state-of-the-art specifically designed models such as BERT-PIN. Key findings include the importance of prompt engineering and the optimal utilization of fine-tuning samples, highlighting the efficiency of few-shot learning in transferring knowledge from general user cases to specific target users. Furthermore, the proposed approach demonstrates notable cost-effectiveness and time efficiency compared to training models from scratch, making it a practical solution for scenarios with limited data availability and computing resources. This research has significant potential for application to other power system load profile analysis tasks. Consequently, it advances the use of LLMs in power system analytics, offering promising implications for enhancing the resilience and efficiency of power distribution systems.

Space-borne gravitational wave detectors like TianQin might encounter data gaps due to factors like micro-meteoroid collisions or hardware failures. Such glitches will cause discontinuity in the data and have been observed in the LISA Pathfinder. The existence of such data gaps presents challenges to the data analysis for TianQin, especially for massive black hole binary mergers, since its signal-to-noise ratio (SNR) accumulates in a non-linear way, a gap near the merger could lead to significant loss of SNR. It could introduce bias in the estimate of noise properties, and furthermore the results of the parameter estimation. In this work, using simulated TianQin data with injected a massive black hole binary merger, we study the window function method, and for the first time, the inpainting method to cope with the data gap, and an iterative estimate scheme is designed to properly estimate the noise spectrum. We find that both methods can properly estimate noise and signal parameters. The easy-to-implement window function method can already perform well, except that it will sacrifice some SNR due to the adoption of the window. The inpainting method is slower, but it can minimize the impact of the data gap., Comment: 12 pages, 5 figures, comments welcome

Superconductivity and long-range ferromagnetism hardly coexist in a uniform manner. The counter-example has been observed, in uranium-based superconductors for instance, with a coexisting temperature limited to about 1 K. Here, we report the coexistence of high temperature superconductivity and itinerant ferromagnetism in lithium intercalated FeSe flakes. In superconducting samples with transition temperature around 40 K, we observe the anomalous Hall effect with a hysteresis loop in transverse resistivity and a butterfly-like pattern of magneto-resistance. Intriguingly, such ferromagnetism persists down to a temperature at which the zero-field resistance fully vanishes. Furthermore, the superconductivity is enhanced under an in-plane magnetic field, suggestive of the participation of spin-polarized Cooper pairs. The surprising finding underscores a uniform coexistence of the two antagonistic phenomena on a record-high energy scale.

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that enables distributed clients to collaboratively learn a shared global model while keeping the data on the original device, thereby preserving privacy. A central challenge in FL is the effective aggregation of local model weights from disparate and potentially unbalanced participating clients. Existing methods often treat each client indiscriminately, applying a single proportion to the entire local model. However, it is empirically advantageous for each weight to be assigned a specific proportion. This paper introduces an innovative Element-Wise Weights Aggregation Method for Federated Learning (EWWA-FL) aimed at optimizing learning performance and accelerating convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local weights to the global model at the level of individual elements, thereby allowing each participating client to make element-wise contributions to the learning process. By taking into account the unique dataset characteristics of each client, EWWA-FL enhances the robustness of the global model to different datasets while also achieving rapid convergence. The method is flexible enough to employ various weighting strategies. Through comprehensive experiments, we demonstrate the advanced capabilities of EWWA-FL, showing significant improvements in both accuracy and convergence speed across a range of backbones and benchmarks., Comment: 2023 IEEE International Conference on Data Mining Workshops (ICDMW)

This paper introduces ViT4LPA, an innovative Vision Transformer (ViT) based approach for Load Profile Analysis (LPA). We transform time-series load profiles into load images. This allows us to leverage the ViT architecture, originally designed for image processing, as a pre-trained image encoder to uncover latent patterns within load data. ViT is pre-trained using an extensive load image dataset, comprising 1M load images derived from smart meter data collected over a two-year period from 2,000 residential users. The training methodology is self-supervised, masked image modeling, wherein masked load images are restored to reveal hidden relationships among image patches. The pre-trained ViT encoder is then applied to various downstream tasks, including the identification of electric vehicle (EV) charging loads and behind-the-meter solar photovoltaic (PV) systems and load disaggregation. Simulation results illustrate ViT4LPA's superior performance compared to existing neural network models in downstream tasks. Additionally, we conduct an in-depth analysis of the attention weights within the ViT4LPA model to gain insights into its information flow mechanisms.

Weak-signal limit is often used in estimating stochastic gravitational-wave background (SGWB) intensities. This approximation fails and the signal-to-noise ratio (SNR) can be much weaker when background signals are loud compared to the detector noise. In this work, we highlight this limitation for the SGWB detection using space-borne detector networks. For the TianQin + LISA network, the SNR estimated under the weak-signal limit might be off by as large as an order of magnitude. Contour plots of SNR over the parameter spaces are also presented to indicate regions susceptible to this discrepancy. Our results suggest that DA and DB type extragalactic double white dwarfs may yield an SGWB with SNR surpassing 100 after 1 year of operation in the weak-signal-limit scenario, with a redshift-independent merger rate of about $500\,\,{\rm Mpc^{-3}\,Myr^{-1}}$. In fact, this value falls significantly below the necessary threshold. Similar influences arise for first-order phase transitions, yet pinning down parameter regions remains formidable due to model uncertainties., Comment: 11 pages, 7 figures

Foundation models (FMs) emerge as a promising solution to harness distributed and diverse environmental data by leveraging prior knowledge to understand the complicated temporal and spatial correlations within heterogeneous datasets. Unlike distributed learning frameworks such as federated learning, which often struggle with multimodal data, FMs can transform diverse inputs into embeddings. This process facilitates the integration of information from various modalities and the application of prior learning to new domains. However, deploying FMs in resource-constrained edge systems poses significant challenges. To this end, we introduce CoRAST, a novel learning framework that utilizes FMs for enhanced analysis of distributed, correlated heterogeneous data. Utilizing a server-based FM, CoRAST can exploit existing environment information to extract temporal, spatial, and cross-modal correlations among sensor data. This enables CoRAST to offer context-aware insights for localized client tasks through FM-powered global representation learning. Our evaluation on real-world weather dataset demonstrates CoRAST's ability to exploit correlated heterogeneous data through environmental representation learning to reduce the forecast errors by up to 50.3% compared to the baselines., Comment: accepted and to be published in 2024 IEEE International Workshop on Foundation Models for Cyber-Physical Systems & Internet of Things (FMSys)

Extreme-mass-ratio inspirals (EMRIs) could be detected by space-borne gravitational-wave (GW) detectors, such as the Laser Interferometer Space Antenna (LISA), TianQin and Taiji. Localizing EMRIs by GW detectors can help us select candidate host galaxies, which can be used to infer the cosmic expansion history. In this paper, we demonstrate that the localization information can also be used to infer the formation channel of EMRIs, and hence allow us to extract more precisely the redshift probability distributions. By conducting mock observations of the EMRIs which can be detected by TianQin and LISA, as well as the galaxies which can be provided by the future Chinese Space Station Telescope, we find that TianQin can constrain the Hubble-Lema\^itre constant $H_0$ to a precision of $\sim3\%-8\%$ and the dark energy equation of state parameter $w_0$ to $\sim10\%-40\%$. The TianQin+LISA network, by increasing the localization accuracy, can improve the precisions of $H_0$ and $w_0$ to $\sim0.4\%-7\%$ and $\sim4\%-20\%$, respectively. Then, considering an illustrative case in which all EMRIs originate in AGNs, and combining the mock EMRI observation with a mock AGN catalog, we show that TianQin can recognize the EMRI-AGN correlation with $\sim 1300$ detections. The TianQin+LISA network can reduce this required number to $\sim 30$. Additionally, we propose a statistical method to directly estimate the fraction of EMRIs produced in AGNs, $f_{\rm agn}$, and show that observationally deriving this value could significantly improve the constraints on the cosmological parameters. These results demonstrate the potentials of using EMRIs as well as galaxy and AGN surveys to improve the constraints on cosmological parameters and the formation channel of EMRIs., Comment: 21 pages, 10 figures, and 1 table; accepted for publication in ApJS

The cosmic distance duality relation (CDDR), expressed as DL(z) = (1 + z)2DA(z), plays an important role in modern cosmology. In this paper, we propose a new method of testing CDDR using strongly lensed gravitational wave (SLGW) signals. Under the geometric optics approximation, we calculate the gravitational lens effects of two lens models, the point mass and singular isothermal sphere. We use functions of {\eta}1(z) = 1 + {\eta}0z and {\eta}2(z) = 1 + {\eta}0z=(1 + z) to parameterize the deviation of CDDR. By reparameterizing the SLGW waveform with CDDR and the distance-redshift relation, we include the deviation parameters {\eta}0 of CDDR as waveform parameters. We evaluate the ability of this method by calculating the parameter estimation of simulated SLGW signals from massive binary black holes. We apply the Fisher information matrix and Markov Chain Monte Carlo methods to calculate parameter estimation. We find that with only one SLGW signal, the measurement precision of {\eta}0 can reach a considerable level of 0.5-1.3% for {\eta}1(z) and 1.1-2.6% for {\eta}2(z), depending on the lens model and parameters., Comment: 15 pages, 7 figures

The Silicon Charge Detector (SCD) is a subdetector of the High Energy Cosmic Radiation Detection payload. The dynamic range of the silicon microstrip detector can be extended by the capacitive coupling effect, which is related to the interstrip capacitance and the coupling capacitance. A detector prototype with several sets of parameters was designed and tested in the ion beams at the CERN Super Proton Synchrotron. The capacitive coupling fractions with readout strip and floating strip incidences were studied using the beam test data and SPICE simulation.

Despite the impressive performance in a variety of complex tasks, modern large language models (LLMs) still have trouble dealing with some math problems that are simple and intuitive for humans, such as addition. While we can easily learn basic rules of addition and apply them to new problems of any length, LLMs struggle to do the same. Instead, they may rely on similar cases seen in the training corpus for help. We define these two different reasoning mechanisms as "rule-based reasoning" and "case-based reasoning". Since rule-based reasoning is essential for acquiring systematic generalization ability, we aim to explore exactly whether transformers use rule-based or case-based reasoning for math problems. Through carefully designed intervention experiments on five math tasks, we confirm that transformers are performing case-based reasoning, no matter whether scratchpad is used, which aligns with the previous observations that transformers use subgraph matching/shortcut learning to reason. To mitigate such problems, we propose a Rule-Following Fine-Tuning (RFFT) technique to teach transformers to perform rule-based reasoning. Specifically, we provide explicit rules in the input and then instruct transformers to recite and follow the rules step by step. Through RFFT, we successfully enable LLMs fine-tuned on 1-5 digit addition to generalize to up to 12-digit addition with over 95% accuracy, which is over 40% higher than scratchpad. The significant improvement demonstrates that teaching LLMs to use rules explicitly helps them learn rule-based reasoning and generalize better in length.

Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approaches encounter issues with limited generalizability across tasks, necessitating one distinct editor for each task, significantly hindering the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets are available in https://github.com/zjunlp/EasyEdit., Comment: IJCAI 2024; the project website is at https://www.zjukg.org/project/InstructEdit/

The power of large language models (LLMs) has been demonstrated through numerous data and computing resources. However, the application of language models on mobile devices is facing huge challenge on the computation and memory costs, that is, tiny language models with high performance are urgently required. Limited by the highly complex training process, there are many details for optimizing language models that are seldom studied carefully. In this study, based on a tiny language model with 1B parameters, we carefully design a series of empirical study to analyze the effect of each component. Three perspectives are mainly discussed, \ie, neural architecture, parameter initialization, and optimization strategy. Several design formulas are empirically proved especially effective for tiny language models, including tokenizer compression, architecture tweaking, parameter inheritance and multiple-round training. Then we train PanGu-$\pi$-1B Pro and PanGu-$\pi$-1.5B Pro on 1.6T multilingual corpora, following the established formulas. Experimental results demonstrate the improved optimization and architecture yield a notable average improvement of 8.87 on benchmark evaluation sets for PanGu-$\pi$-1B Pro. Besides, PanGu-$\pi$-1.5B Pro surpasses a range of SOTA models with larger model sizes, validating its superior performance. The code is available at https://github.com/YuchuanTian/RethinkTinyLM.

This paper analyses the impact of various parameter changes on the estimation of parameters for massive black hole binary (MBHB) systems using a Bayesian inference technique. Several designed MBHB systems were chosen for comparison with a fiducial system to explore the influence of parameters such as sky location, inclination angle, anti-spin, large mass ratio and light mass. And the two reported MBHB candidates named OJ287 and Tick-Tock are also considered. The study found that the network of TianQin and LISA can break certain degeneracies among different parameters, improving the estimation of parameters, particularly for extrinsic parameters. Meanwhile, the degeneracies between different intrinsic parameters are highly sensitive to the value of the parameters. Additionally, the small inclination angles and limited detection of the inspiral phase can introduce significant bias in the estimation of parameters. The presence of instrument noise will also introduce bias and worsen the precision. The paper concludes that the network of TianQin and LISA can significantly improve the estimation of extrinsic parameters by about one order of magnitude while yielding slight improvements in the intrinsic parameters. Moreover, parameter estimation can still be subject to biases even with a sufficiently high signal-to-noise ratio if the detected signal does not encompass all stages of the inspiral, merger, and ringdown., Comment: 17 pages, 10 figures

Few-shot prompting elicits the remarkable abilities of large language models by equipping them with a few demonstration examples in the input. However, the traditional method of providing large language models with all demonstration input-output pairs at once may not effectively guide large language models to learn the specific input-output mapping relationship. In this paper, inspired by the regulatory and supportive role of metacognition in students' learning, we propose a novel metacognition-enhanced few-shot prompting, which guides large language models to reflect on their thought processes to comprehensively learn the given demonstration examples. Furthermore, considering that positive reinforcement can improve students' learning motivation, we introduce positive reinforcement into our metacognition-enhanced few-shot prompting to promote the few-shot learning of large language models by providing response-based positive feedback. The experimental results on two real-world datasets show that our metacognition-enhanced few-shot prompting with positive reinforcement surpasses traditional few-shot prompting in classification accuracy and macro F1., Comment: 5 pages, 4 figures, 2 tables

The mean-field theory (MFT) of simple glasses, which is exact in the limit of infinite spatial dimensions, $d\rightarrow\infty$, offers theoretical insight as well as quantitative predictions about certain features of $d=3$ systems. In order to more systematically relate the behavior of physical systems to MFT, however, various finite-$d$ corrections need to be accounted for. Although some efforts along this direction have already been undertaken, theoretical and technical challenges hinder progress. A general approach to sidestep many of these difficulties consists of simulating minimally-structured models whose behavior smoothly converges to that described by the MFT as $d$ increases, so as to permit a dimensional extrapolation. Using this approach, we here extract the small fluctuations around the dynamical MFT captured by a standard liquid-state observable the non-Gaussian parameter $\alpha_2$. The results provide insight into the physical origin of these fluctuations as well as a quantitative reference with which to compare observations in more realistic glass formers., Comment: 10 pages, 7 figures

Theoretical predictions suggest that very massive stars have the potential to form through multiple collisions and eventually evolve into intermediate-mass black holes (IMBHs) within Population III star clusters embedded in mini dark matter haloes. In this study, we investigate the long-term evolution of Population III star clusters, including models with a primordial binary fraction of $f_{\rm b}=0$ and 1, using the $N$-body simulation code PETAR. We comprehensively examine the phenomenon of hierarchical triple black holes in the clusters, specifically focusing on their merging inner binary black holes (BBHs), with post-Newtonian correction, by using the TSUNAMI code. Our findings suggest a high likelihood of the inner BBHs containing IMBHs with masses on the order of $\mathcal{O}(100)M_{\odot}$, and as a result, their merger rate could be up to $0.1{\rm Gpc}^{-3}{\rm yr}^{-3}$. The orbital eccentricities of some merging inner BBHs oscillate over time periodically, known as the Kozai-Lidov oscillation, due to dynamical perturbations. Detectable merging inner BBHs for mHz GW detectors LISA/TianQin/Taiji concentrate within $z<3$. More distant sources would be detectable for CE/ET/LIGO/KAGRA/DECIGO, which are sensitive from $\mathcal{O}(0.1)$Hz to $\mathcal{O}(100)$Hz. Furthermore, compared with merging isolated BBHs, merging inner BBHs affected by dynamical perturbations from tertiary BHs tend to have higher eccentricities, with a significant fraction of sources with eccentricities closing to 1 at mHz bands. GW observations would help constrain formation channels of merging BBHs, whether through isolated evolution or dynamical interaction, by examining eccentricities., Comment: 19 pages, 22 figures, and 3 tables, accepted by MNRAS

Inspired by the success of the Transformer model in natural language processing and computer vision, this paper introduces BERT-PIN, a Bidirectional Encoder Representations from Transformers (BERT) powered Profile Inpainting Network. BERT-PIN recovers multiple missing data segments (MDSs) using load and temperature time-series profiles as inputs. To adopt a standard Transformer model structure for profile inpainting, we segment the load and temperature profiles into line segments, treating each segment as a word and the entire profile as a sentence. We incorporate a top candidates selection process in BERT-PIN, enabling it to produce a sequence of probability distributions, based on which users can generate multiple plausible imputed data sets, each reflecting different confidence levels. We develop and evaluate BERT-PIN using real-world dataset for two applications: multiple MDSs recovery and demand response baseline estimation. Simulation results show that BERT-PIN outperforms the existing methods in accuracy while is capable of restoring multiple MDSs within a longer window. BERT-PIN, served as a pre-trained model, can be fine-tuned for conducting many downstream tasks, such as classification and super resolution.

Space-based gravitational wave detectors like TianQin or LISA could observe extreme-mass-ratio-inspirals (EMRIs) at millihertz frequencies. The accurate identification of these EMRI signals from the data plays a crucial role in enabling in-depth study of astronomy and physics. We aim at the identification stage of the data analysis, with the aim to extract key features of the signal from the data, such as the evolution of the orbital frequency, as well as to pinpoint the parameter range that can fit the data well for the subsequent parameter inference stage. In this manuscript, we demonstrated the identification of EMRI signals without any additional prior information on physical parameters. High-precision measurements of EMRI signals have been achieved, using a hierarchical search. It combines the search for physical parameters that guide the subsequent parameter inference, and a semi-coherent search with phenomenological waveforms that reaches precision levels down to $10^{-4}$ for the phenomenological waveform parameters $\omega_{0}$, $\dot{\omega}_{0}$, and $\ddot{\omega}_{0}$. As a result, we obtain measurement relative errors of less than 4% for the mass of the massive black hole, while keeping the relative errors of the other parameters within as small as 0.5%., Comment: 16 pages, 7 figures, comments welcom

The field of two-dimensional (2D) ferromagnetism has been proliferating over the past few years, with ongoing interests in basic science and potential applications in spintronic technology. However, a high-resolution spectroscopic study of the 2D ferromagnet is still lacking due to the small size and air sensitivity of the exfoliated nanoflakes. Here, we report a thickness-dependent ferromagnetism in epitaxially grown Cr2Te3 thin films and investigate the evolution of the underlying electronic structure by synergistic angle-resolved photoemission spectroscopy, scanning tunneling microscopy, x-ray absorption spectroscopy, and first-principle calculations. A conspicuous ferromagnetic transition from Stoner to Heisenberg-type is directly observed in the atomically thin limit, indicating that dimensionality is a powerful tuning knob to manipulate the novel properties of 2D magnetism. Monolayer Cr2Te3 retains robust ferromagnetism, but with a suppressed Curie temperature, due to the drastic drop in the density of states near the Fermi level. Our results establish atomically thin Cr2Te3 as an excellent platform to explore the dual nature of localized and itinerant ferromagnetism in 2D magnets., Comment: 32 pages, 4 + 10 figures