Search

Named entity recognition (NER) is a crucial task that aims to identify structured information, which is often replete with complex, technical terms and a high degree of variability. Accurate and reliable NER can facilitate the extraction and analysis of important information. However, NER for other than English is challenging due to limited data availability, as the high expertise, time, and expenses are required to annotate its data. In this paper, by using the limited data, we explore various factors including model structure, corpus annotation scheme and data augmentation techniques to improve the performance of a NER model for French. Our experiments demonstrate that these approaches can significantly improve the model's F1 score from original CRF score of 62.41 to 79.39. Our findings suggest that considering different extrinsic factors and combining these techniques is a promising approach for improving NER performance where the size of data is limited.

Field-theoretical calculations predict that, at the upper critical dimension $d_c=4$, the finite-size scaling (FSS) behaviors of the Ising model would be modified by multiplicative logarithmic corrections with thermal and magnetic correction exponents $(\hat{y}_t, \hat{y}_h)=(1/6,1/4)$. Using high-efficient cluster algorithms and the lifted worm algorithm, we present a systematic study of the FSS of the four-dimensional Ising model in the Fortuin-Kasteleyn (FK) bond and loop representations. Our numerical results reveal the FSS behaviors of various geometric and physical quantities in the three representations, offering robust evidence for the logarithmic correction form conjectured by the field theory. In particular, clear evidence is obtained for the existence of $\hat{y}_t=1/6$ in the loop representation, while it is difficult to extract in the spin representations, because of mixing with the Gaussian-fixed-point asymptotics. In the FK-bond representation, the multiplicative logarithmic correction for the second-largest cluster is also numerically observed to be governed by an exponent $\hat{y}_{h2} = -1/4$ with its exact value unknown yet., Comment: 7 pages, 3 figures

The Robust Satisficing (RS) model is an emerging approach to robust optimization, offering streamlined procedures and robust generalization across various applications. However, the statistical theory of RS remains unexplored in the literature. This paper fills in the gap by comprehensively analyzing the theoretical properties of the RS model. Notably, the RS structure offers a more straightforward path to deriving statistical guarantees compared to the seminal Distributionally Robust Optimization (DRO), resulting in a richer set of results. In particular, we establish two-sided confidence intervals for the optimal loss without the need to solve a minimax optimization problem explicitly. We further provide finite-sample generalization error bounds for the RS optimizer. Importantly, our results extend to scenarios involving distribution shifts, where discrepancies exist between the sampling and target distributions. Our numerical experiments show that the RS model consistently outperforms the baseline empirical risk minimization in small-sample regimes and under distribution shifts. Furthermore, compared to the DRO model, the RS model exhibits lower sensitivity to hyperparameter tuning, highlighting its practicability for robustness considerations.

The spin Coulomb drag effect, arising from the exchange of momentum between electrons of opposite spins, plays a crucial role in the spin transport of interacting electron systems. This effect leads to the emergence of a spin mass and the finite lifetime of spin currents, posing challenges for the accurate description of spin dynamics. Using the state-of-the-art Variational Diagrammatic Monte Carlo approach, we investigate the spin-resolved exchange-correlation (XC) kernel in the three-dimensional uniform electron gas. Our analysis reveals a distinct nature in the spin response, characterized by a $1/q^2$ divergence in the spin XC kernel at finite frequencies. This so-called ultranonlocal behavior, stemming from the spin Coulomb drag effect, is absent in the charge channel. By extracting precise values for the spin mass enhancement factor, we observe a trend of increasing enhancement with decreasing electron density. Furthermore, we find compelling evidence for the suppression of spin diffusion at low temperatures, characterized by vanishing inverse relaxation times. These findings deepen our understanding of the intricate relation between the Coulomb interaction and the spin transport, providing valuable insights for the development of accurate density functional approximations and the advancement of spintronics and quantum technologies., Comment: 8 pages, 5 figures

Considering widely dispersed uncertain renewable energy sources (RESs), scenario-based stochastic optimization is an effective method for the economic dispatch of renewables-rich power systems. However, on classic computers, to simulate RES uncertainties with high accuracy, the massive scenario generation is very time-consuming, and the pertinent optimization problem is high-dimensional NP-hard mixed-integer programming. To this end, we design a quantum-assisted scheme to accelerate the stochastic optimization for power system economic dispatch without losing accuracy. We first propose the unified quantum amplitude estimation to characterize RES uncertainties, thereby generating massive scenarios by a few qubits to reduce state variables. Then, strong Benders cuts corresponding to some specific scenarios are selected to control the solution scale of Benders master problem in the iterative process, all of which are implemented by customized quantum approximation optimization algorithms. Finally, we perform numerical experiments on the modified IEEE 6-bus system to test the designed scheme.

To satisfy dynamic requirements of power systems, it is imperative for grid-tied inverters to ensure good disturbance rejection performance (DRP) under variable grid conditions. This letter discovers and theoretically proves that for general networks, synthesizing grid-following (GFL) inverters and grid-forming (GFM) inverters can always more effectively ensure the DRP of multiple inverters, as compared to homogeneous inverter-based systems that solely utilize either GFL or GFM inverters. The synthesis of GFL inverters and GFM inverters can concurrently increase the smallest eigenvalue and decrease the largest eigenvalue of the network grounded Laplacian matrix. This can be equivalent to rematching the proper short-circuit ratio (SCR) for GFL and GFM inverters, thereby ensuring the DRP of inverters both in weak and strong grids. The results reveal the necessity of synthesizing diverse inverter control schemes from the network-based perspective. Sensitivity function-based tests and real-time simulations validate our results., Comment: 6 pages

We propose a computational graph representation of high-order Feynman diagrams in Quantum Field Theory (QFT), applicable to any combination of spatial, temporal, momentum, and frequency domains. Utilizing the Dyson-Schwinger and parquet equations, our approach effectively organizes these diagrams into a fractal structure of tensor operations, significantly reducing computational redundancy. This approach not only streamlines the evaluation of complex diagrams but also facilitates an efficient implementation of the field-theoretic renormalization scheme, crucial for enhancing perturbative QFT calculations. Key to this advancement is the integration of Taylor-mode automatic differentiation, a key technique employed in machine learning packages to compute higher-order derivatives efficiently on computational graphs. To operationalize these concepts, we develop a Feynman diagram compiler that optimizes diagrams for various computational platforms, utilizing machine learning frameworks. Demonstrating this methodology's effectiveness, we apply it to the three-dimensional uniform electron gas problem, achieving unprecedented accuracy in calculating the quasiparticle effective mass at metal density. Our work demonstrates the synergy between QFT and machine learning, establishing a new avenue for applying AI techniques to complex quantum many-body problems.

Small-signal synchronization instability (SSI) may be triggered when a grid-connected converter is operated in weak grids. This problem is highly related to the active and reactive power (referred to as complex power) generation or consumption of the converter. Such an instability phenomenon manifests as power oscillations within the bandwidth frequency of phase-locked loop (PLL). However, in a multi-converter power system (MCPS), it is challenging to characterize the role of each converter's complex power in SSI issues due to the high dimension of system dynamics. In this paper, we focus on the impact regularity of complex power on the small-signal synchronization stability of the MCPS. Firstly, we formulate the modeling of the MCPS and develop a systematical stability indicator to explicitly quantify the stability margin under various operating complex power. Then, based on the proposed stability indicator, the "resultant force" of complex power on the stability margin of the MCPS can be analytically characterized into a weighted summation form. Furthermore, we elaborate on the impact of each converter's active power and reactive power, respectively. Notably, we demonstrate the cancellation effect between the active power generation and consumption on the stability margin, which ultimately helps prevent SSI issues in the MCPS. The correctness of theoretical findings is well verified by simulations results., Comment: 11 pages

Reconfigurable intelligent surfaces (RISs) are widely considered a promising technology for future wireless communication systems. As an important indicator of RIS-assisted communication systems in green wireless communications, energy efficiency (EE) has recently received intensive research interest as an optimization target. However, most previous works have ignored the different power consumption between ON and OFF states of the PIN diodes attached to each RIS element. This oversight results in extensive unnecessary power consumption and reduction of actual EE due to the inaccurate power model. To address this issue, in this paper, we first utilize a practical power model for a RIS-assisted multi-user multiple-input single-output (MU-MISO) communication system, which takes into account the difference in power dissipation caused by ON-OFF states of RIS's PIN diodes. Based on this model, we formulate a more accurate EE optimization problem. However, this problem is non-convex and has mixed-integer properties, which poses a challenge for optimization. To solve the problem, an effective alternating optimization (AO) algorithm framework is utilized to optimize the base station and RIS beamforming precoder separately. To obtain the essential RIS beamforming precoder, we develop two effective methods based on maximum gradient search and SDP relaxation respectively. Theoretical analysis shows the exponential complexity of the original problem has been reduced to polynomial complexity. Simulation results demonstrate that the proposed algorithm outperforms the existing ones, leading to a significant increase in EE across a diverse set of scenarios., Comment: Reconfigurable intelligent surface is a promising 6G technology. However, RIS power models are inaccurate. In this paper, we construct a practical power model for RIS communication systems with an SDP-relaxation algorithm, achieving optimal energy efficiency

Besides its original spin representation, the Ising model is known to have the Fortuin-Kasteleyn (FK) bond and loop representations, of which the former was recently shown to exhibit two upper critical dimensions $(d_c=4,d_p=6)$. Using a lifted worm algorithm, we determine the critical coupling as $K_c = 0.077\,708\,91(4)$ for $d=7$, which significantly improves over the previous results, and then study critical geometric properties of the loop-Ising clusters on tori for spatial dimensions $d=5$ to 7. We show that, as the spin representation, the loop Ising model has only one upper critical dimension at $d_c=4$. However, sophisticated finite-size scaling (FSS) behaviors, like two length scales, two configuration sectors and two scaling windows, still exist as the interplay effect of the Gaussian fixed point and complete-graph asymptotics. Moreover, using the Loop-Cluster algorithm, we provide an intuitive understanding of the emergence of the percolation-like upper critical dimension $d_p=6$ in the FK-Ising model. As a consequence, a unified physical picture is established for the FSS behaviors in all the three representations of the Ising model above $d_c=4$., Comment: 12 pages, 12 figures

Recently, pre-trained foundation models have enabled significant advancements in multiple fields. In molecular machine learning, however, where datasets are often hand-curated, and hence typically small, the lack of datasets with labeled features, and codebases to manage those datasets, has hindered the development of foundation models. In this work, we present seven novel datasets categorized by size into three distinct categories: ToyMix, LargeMix and UltraLarge. These datasets push the boundaries in both the scale and the diversity of supervised labels for molecular learning. They cover nearly 100 million molecules and over 3000 sparsely defined tasks, totaling more than 13 billion individual labels of both quantum and biological nature. In comparison, our datasets contain 300 times more data points than the widely used OGB-LSC PCQM4Mv2 dataset, and 13 times more than the quantum-only QM1B dataset. In addition, to support the development of foundational models based on our proposed datasets, we present the Graphium graph machine learning library which simplifies the process of building and training molecular machine learning models for multi-task and multi-level molecular datasets. Finally, we present a range of baseline results as a starting point of multi-task and multi-level training on these datasets. Empirically, we observe that performance on low-resource biological datasets show improvement by also training on large amounts of quantum data. This indicates that there may be potential in multi-task and multi-level training of a foundation model and fine-tuning it to resource-constrained downstream tasks.

Biomedical named entity recognition (NER) is a critial task that aims to identify structured information in clinical text, which is often replete with complex, technical terms and a high degree of variability. Accurate and reliable NER can facilitate the extraction and analysis of important biomedical information, which can be used to improve downstream applications including the healthcare system. However, NER in the biomedical domain is challenging due to limited data availability, as the high expertise, time, and expenses are required to annotate its data. In this paper, by using the limited data, we explore various extrinsic factors including the corpus annotation scheme, data augmentation techniques, semi-supervised learning and Brill transformation, to improve the performance of a NER model on a clinical text dataset (i2b2 2012, \citet{sun-rumshisky-uzuner:2013}). Our experiments demonstrate that these approaches can significantly improve the model's F1 score from original 73.74 to 77.55. Our findings suggest that considering different extrinsic factors and combining these techniques is a promising approach for improving NER performance in the biomedical domain where the size of data is limited.

Reducing the computational cost of running large scale neural networks using sparsity has attracted great attention in the deep learning community. While much success has been achieved in reducing FLOP and parameter counts while maintaining acceptable task performance, achieving actual speed improvements has typically been much more difficult, particularly on general purpose accelerators (GPAs) such as NVIDIA GPUs using low precision number formats. In this work we introduce PopSparse, a library that enables fast sparse operations on Graphcore IPUs by leveraging both the unique hardware characteristics of IPUs as well as any block structure defined in the data. We target two different types of sparsity: static, where the sparsity pattern is fixed at compile-time; and dynamic, where it can change each time the model is run. We present benchmark results for matrix multiplication for both of these modes on IPU with a range of block sizes, matrix sizes and densities. Results indicate that the PopSparse implementations are faster than dense matrix multiplications on IPU at a range of sparsity levels with large matrix size and block size. Furthermore, static sparsity in general outperforms dynamic sparsity. While previous work on GPAs has shown speedups only for very high sparsity (typically 99\% and above), the present work demonstrates that our static sparse implementation outperforms equivalent dense calculations in FP16 at lower sparsity (around 90%). IPU code is available to view and run at ipu.dev/sparsity-benchmarks, GPU code will be made available shortly.

The exact solution of the Ising model on the complete graph (CG) provides an important, though mean-field, insight for the theory of continuous phase transitions. Besides the original spin, the Ising model can be formulated in the Fortuin-Kasteleyn random-cluster and the loop representation, in which many geometric quantities have no correspondence in the spin representations. Using a lifted-worm irreversible algorithm, we study the CG-Ising model in the loop representation, and, based on theoretical and numerical analyses, obtain a number of exact results including volume fractal dimensions and scaling forms. Moreover, by combining with the Loop-Cluster algorithm, we demonstrate how the loop representation can provide an intuitive understanding to the recently observed rich geometric phenomena in the random-cluster representation, including the emergence of two configuration sectors, two length scales and two scaling windows., Comment: 10 pages, 10 figures

We present GPS++, a hybrid Message Passing Neural Network / Graph Transformer model for molecular property prediction. Our model integrates a well-tuned local message passing component and biased global attention with other key ideas from prior literature to achieve state-of-the-art results on large-scale molecular dataset PCQM4Mv2. Through a thorough ablation study we highlight the impact of individual components and find that nearly all of the model's performance can be maintained without any use of global self-attention, showing that message passing is still a competitive approach for 3D molecular property prediction despite the recent dominance of graph transformers. We also find that our approach is significantly more accurate than prior art when 3D positional information is not available., Comment: arXiv admin note: text overlap with arXiv:2212.02229

This technical report presents GPS++, the first-place solution to the Open Graph Benchmark Large-Scale Challenge (OGB-LSC 2022) for the PCQM4Mv2 molecular property prediction task. Our approach implements several key principles from the prior literature. At its core our GPS++ method is a hybrid MPNN/Transformer model that incorporates 3D atom positions and an auxiliary denoising task. The effectiveness of GPS++ is demonstrated by achieving 0.0719 mean absolute error on the independent test-challenge PCQM4Mv2 split. Thanks to Graphcore IPU acceleration, GPS++ scales to deep architectures (16 layers), training at 3 minutes per epoch, and large ensemble (112 models), completing the final predictions in 1 hour 32 minutes, well under the 4 hour inference budget allocated. Our implementation is publicly available at: https://github.com/graphcore/ogb-lsc-pcqm4mv2.