Search

Activity cliffs, which refer to pairs of molecules that are structurally similar but show significant differences in their potency, can lead to model representation collapse and make the model challenging to distinguish them. Our research indicates that as molecular similarity increases, graph-based methods struggle to capture these nuances, whereas image-based approaches effectively retain the distinctions. Thus, we developed MaskMol, a knowledge-guided molecular image self-supervised learning framework. MaskMol accurately learns the representation of molecular images by considering multiple levels of molecular knowledge, such as atoms, bonds, and substructures. By utilizing pixel masking tasks, MaskMol extracts fine-grained information from molecular images, overcoming the limitations of existing deep learning models in identifying subtle structural changes. Experimental results demonstrate MaskMol's high accuracy and transferability in activity cliff estimation and compound potency prediction across 20 different macromolecular targets, outperforming 25 state-of-the-art deep learning and machine learning approaches. Visualization analyses reveal MaskMol's high biological interpretability in identifying activity cliff-relevant molecular substructures. Notably, through MaskMol, we identified candidate EP4 inhibitors that could be used to treat tumors. This study not only raises awareness about activity cliffs but also introduces a novel method for molecular image representation learning and virtual screening, advancing drug discovery and providing new insights into structure-activity relationships (SAR)., Comment: 33 pages, 5 figures

Mobile technology (mobile phones and wearable devices) generates continuous data streams encompassing outcomes, exposures and covariates, presented as intensive longitudinal or multivariate time series data. The high frequency of measurements enables granular and dynamic evaluation of treatment effect, revealing their persistence and accumulation over time. Existing methods predominantly focus on the contemporaneous effect, temporal-average, or population-average effects, assuming stationarity or invariance of treatment effects over time, which are inadequate both conceptually and statistically to capture dynamic treatment effects in personalized mobile health data. We here propose new causal estimands for multivariate time series in N-of-1 studies. These estimands summarize how time-varying exposures impact outcomes in both short- and long-term. We propose identifiability assumptions and a g-formula estimator that accounts for exposure-outcome and outcome-covariate feedback. The g-formula employs a state space model framework innovatively to accommodate time-varying behavior of treatment effects in non-stationary time series. We apply the proposed method to a multi-year smartphone observational study of bipolar patients and estimate the dynamic effect of phone-based communication on mood of patients with bipolar disorder in an N-of-1 setting. Our approach reveals substantial heterogeneity in treatment effects over time and across individuals. A simulation-based strategy is also proposed for the development of a short-term, dynamic, and personalized treatment recommendation based on patient's past information, in combination with a novel positivity diagnostics plot, validating proper causal inference in time series data.

Attosecond x-ray pulses play a crucial role in the study of ultrafast phenomena occurring within inner and valence electrons. Especially isolated attosecond pulses with high photon energy and high peak power are of great significance in single-shot imaging in the soft x-ray region, life sciences, and attosecond pump-probe experiments. In modern accelerators, laser manipulation of electrons can be used to tailor the ultrafast properties of free-electron laser (FEL) pulses. In this paper, we propose a novel laser manipulation technique that makes use of two laser beams with mutual delays and tilted wavefronts to synthesize microbunching rotation on the scale of infrared laser wavelengths within the electron bunch for generating isolated attosecond soft x-ray pulses. This microbunching rotation ultimately leads to an enhanced current contrast ratio between the main peak and the surrounding satellite peaks within the bunch. By properly accounting for the longitudinal space charge fields within the FEL undulator, a tapered undulator can further suppress the side peaks in the radiation pulse and enable the selection of an isolated, hundred-attosecond, GW-level soft x-ray pulse.

Mixture-of-Experts (MoE) architectures have emerged as a paradigm-shifting approach for large language models (LLMs), offering unprecedented computational efficiency. However, these architectures grapple with challenges of token distribution imbalance and expert homogenization, impeding optimal semantic generalization. We introduce a novel framework that redefines MoE routing through affinity-driven active selection. The innovations for the framework encompass: (1) A rigorous formulation of expert-token affinity metrics. (2) An adaptive bidirectional selection mechanism leveraging resonance between experts and tokens. (3) Theoretical derivation and experimental evidence of reduced expert capacity bounds under dynamic token distribution evolution. It is also integrated with orthogonal feature extraction module and an optimized loss function for expert localization. Our theoretical analysis demonstrates that this approach mitigates expert homogenization while enabling substantial capacity boundary reduction. Experimental validation corroborates these findings: it achieves a 40% reduction in token processed by each expert without compromising model convergence or efficacy. When coupled with communication optimizations, the training efficiency improvements of 5.4% to 46.6% can be observed. After supervised fine-tuning, it exhibits performance gains of 9.7% to 14.1% across GDAD, C-Eval, and TeleQnA benchmarks.

In this paper, we extend financial sentiment analysis~(FSA) to event-level since events usually serve as the subject of the sentiment in financial text. Though extracting events from the financial text may be conducive to accurate sentiment predictions, it has specialized challenges due to the lengthy and discontinuity of events in a financial text. To this end, we reconceptualize the event extraction as a classification task by designing a categorization comprising coarse-grained and fine-grained event categories. Under this setting, we formulate the \textbf{E}vent-Level \textbf{F}inancial \textbf{S}entiment \textbf{A}nalysis~(\textbf{EFSA} for short) task that outputs quintuples consisting of (company, industry, coarse-grained event, fine-grained event, sentiment) from financial text. A large-scale Chinese dataset containing $12,160$ news articles and $13,725$ quintuples is publicized as a brand new testbed for our task. A four-hop Chain-of-Thought LLM-based approach is devised for this task. Systematically investigations are conducted on our dataset, and the empirical results demonstrate the benchmarking scores of existing methods and our proposed method can reach the current state-of-the-art. Our dataset and framework implementation are available at https://anonymous.4open.science/r/EFSA-645E

Many complex problems encountered in both production and daily life can be conceptualized as combinatorial optimization problems (COPs) over graphs. Recent years, reinforcement learning (RL) based models have emerged as a promising direction, which treat the COPs solving as a heuristic learning problem. However, current finite-horizon-MDP based RL models have inherent limitations. They are not allowed to explore adquately for improving solutions at test time, which may be necessary given the complexity of NP-hard optimization tasks. Some recent attempts solve this issue by focusing on reward design and state feature engineering, which are tedious and ad-hoc. In this work, we instead propose a much simpler but more effective technique, named gauge transformation (GT). The technique is originated from physics, but is very effective in enabling RL agents to explore to continuously improve the solutions during test. Morever, GT is very simple, which can be implemented with less than 10 lines of Python codes, and can be applied to a vast majority of RL models. Experimentally, we show that traditional RL models with GT technique produce the state-of-the-art performances on the MaxCut problem. Furthermore, since GT is independent of any RL models, it can be seamlessly integrated into various RL frameworks, paving the way of these models for more effective explorations in the solving of general COPs.

The proliferation of low-earth-orbit (LEO) satellite networks leads to the generation of vast volumes of remote sensing data which is traditionally transferred to the ground server for centralized processing, raising privacy and bandwidth concerns. Federated edge learning (FEEL), as a distributed machine learning approach, has the potential to address these challenges by sharing only model parameters instead of raw data. Although promising, the dynamics of LEO networks, characterized by the high mobility of satellites and short ground-to-satellite link (GSL) duration, pose unique challenges for FEEL. Notably, frequent model transmission between the satellites and ground incurs prolonged waiting time and large transmission latency. This paper introduces a novel FEEL algorithm, named FEDMEGA, tailored to LEO mega-constellation networks. By integrating inter-satellite links (ISL) for intra-orbit model aggregation, the proposed algorithm significantly reduces the usage of low data rate and intermittent GSL. Our proposed method includes a ring all-reduce based intra-orbit aggregation mechanism, coupled with a network flow-based transmission scheme for global model aggregation, which enhances transmission efficiency. Theoretical convergence analysis is provided to characterize the algorithm performance. Extensive simulations show that our FEDMEGA algorithm outperforms existing satellite FEEL algorithms, exhibiting an approximate 30% improvement in convergence rate., Comment: 16 pages, 15 figures

Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large graphs. However, existing works fail to balance the computation cost and communication cost on machines with different power (including computing capability, network bandwidth and memory size), as they only consider replication factor and neglect the difference of machines in realistic data centers. In this paper, we propose a general graph partitioning algorithm WindGP, which can support fast and high-quality edge partitioning on heterogeneous machines. WindGP designs novel preprocessing techniques to simplify the metric and balance the computation cost according to the characteristics of graphs and machines. Also, best-first search is proposed instead of BFS and DFS, in order to generate clusters with high cohesion. Furthermore, WindGP adaptively tunes the partition results by sophisticated local search methods. Extensive experiments show that WindGP outperforms all state-of-the-art partition methods by 1.35 - 27 times on both dense and sparse distributed graph algorithms, and has good scalability with graph size and machine number., Comment: 19 pages, 15 figures, 18 tables

The Mixtures-of-Experts (MoE) model is a widespread distributed and integrated learning method for large language models (LLM), which is favored due to its ability to sparsify and expand models efficiently. However, the performance of MoE is limited by load imbalance and high latency of All-to-All communication, along with relatively redundant computation owing to large expert capacity. Load imbalance may result from existing routing policies that consistently tend to select certain experts. The frequent inter-node communication in the All-to-All procedure also significantly prolongs the training time. To alleviate the above performance problems, we propose a novel routing strategy that combines load balance and locality by converting partial inter-node communication to that of intra-node. Notably, we elucidate that there is a minimum threshold for expert capacity, calculated through the maximal angular deviation between the gating weights of the experts and the assigned tokens. We port these modifications on the PanGu-Sigma model based on the MindSpore framework with multi-level routing and conduct experiments on Ascend clusters. The experiment results demonstrate that the proposed LocMoE reduces training time per epoch by 12.68% to 22.24% compared to classical routers, such as hash router and switch router, without impacting the model accuracy., Comment: 1. Update the font size of all figures. 2. Update the name of the proposed layer Grouped Average Pooling (GrAP). 3. Change the order of the Section Contribution Statement

Graph partitioning (GP), a.k.a. community detection, is a classic problem that divides the node set of a graph into densely-connected blocks. Following prior work on the IEEE HPEC Graph Challenge benchmark and recent advances in graph machine learning, we propose a novel RAndom FasT Graph Partitioning (RaftGP) method based on an efficient graph embedding scheme. It uses the Gaussian random projection to extract community-preserving features from classic GP objectives. These features are fed into a graph neural network (GNN) to derive low-dimensional node embeddings. Surprisingly, our experiments demonstrate that a randomly initialized GNN even without training is enough for RaftGP to derive informative community-preserving embeddings and support high-quality GP. To enable the derived embeddings to tackle GP, we introduce a hierarchical model selection algorithm that simultaneously determines the number of blocks and the corresponding GP result. We evaluate RaftGP on the Graph Challenge benchmark and compare the performance with five baselines, where our method can achieve a better trade-off between quality and efficiency. In particular, compared to the baseline algorithm of the IEEE HPEC Graph Challenge, our method is 6.68x -- 23.9x faster on graphs with 1E3 -- 5E4 nodes and at least 64.5x faster on larger (1E5 node) graphs on which the baseline takes more than 1E4 seconds. Our method achieves better accuracy on all test cases. We also develop a new graph generator to address some limitations of the original generator in the benchmark.

Gene mining is an important topic in the field of life sciences, but traditional machine learning methods cannot consider the regulatory relationships between genes. Deep learning methods perform poorly in small sample sizes. This study proposed a deep learning method, called TransGeneSelector, that can mine critical regulatory genes involved in certain life processes using a small-sample transcriptome dataset. The method combines a WGAN-GP data augmentation network, a sample filtering network, and a Transformer classifier network, which successfully classified the state (germinating or dry seeds) of Arabidopsis thaliana seed in a dataset of 79 samples, showing performance comparable to that of Random Forests. Further, through the use of SHapley Additive exPlanations method, TransGeneSelector successfully mined genes involved in seed germination. Through the construction of gene regulatory networks and the enrichment analysis of KEGG, as well as RT-qPCR quantitative analysis, it was confirmed that these genes are at a more upstream regulatory level than those Random Forests mined, and the top 11 genes that were uniquely mined by TransGeneSelector were found to be related to the KAI2 signaling pathway, which is of great regulatory importance for germination-related genes. This study provides a practical tool for life science researchers to mine key genes from transcriptome data., Comment: 34 pages,6 figures

Recent advances and achievements of artificial intelligence (AI) as well as deep and graph learning models have established their usefulness in biomedical applications, especially in drug-drug interactions (DDIs). DDIs refer to a change in the effect of one drug to the presence of another drug in the human body, which plays an essential role in drug discovery and clinical research. DDIs prediction through traditional clinical trials and experiments is an expensive and time-consuming process. To correctly apply the advanced AI and deep learning, the developer and user meet various challenges such as the availability and encoding of data resources, and the design of computational methods. This review summarizes chemical structure based, network based, NLP based and hybrid methods, providing an updated and accessible guide to the broad researchers and development community with different domain knowledge. We introduce widely-used molecular representation and describe the theoretical frameworks of graph neural network models for representing molecular structures. We present the advantages and disadvantages of deep and graph learning methods by performing comparative experiments. We discuss the potential technical challenges and highlight future directions of deep and graph learning models for accelerating DDIs prediction., Comment: Accepted by Briefings in Bioinformatics

External seeded free-electron lasers (FELs) are compelling tools for generating fully coherent EUV and soft X-ray radiations. Echo-enabled harmonic generation (EEHG), the most typical representative of external seeded FELs, has witnessed a remarkable growth of fully coherent FELs in the last decade, continuously evolving towards higher harmonic conversions and shorter wavelengths. Ultra-high harmonic generation is imperative in the field of FELs. This paper presents a novel method for generating FEL radiation with ultra-high harmonic conversion, utilizing harmonic optical klystron in combination with EEHG. This method can effectively increase the harmonic conversion order to about 90. Theoretical analysis and numerical simulations show that intense and almost fully coherent FEL pulses can be generated with a wavelength of 3 nm. At the same time, the seed laser intensity required by this scheme is lower compared to nominal EEHG, thus facilitating the generation of high-repetition-rate seeded FELs.

In this paper, we develop an invertible mapping, called B-KRnet, on a bounded domain and apply it to density estimation/approximation for data or the solutions of PDEs such as the Fokker-Planck equation and the Keller-Segel equation. Similar to KRnet, the structure of B-KRnet adapts the triangular form of the Knothe-Rosenblatt rearrangement into a normalizing flow model. The main difference between B-KRnet and KRnet is that B-KRnet is defined on a hypercube while KRnet is defined on the whole space, in other words, we introduce a new mechanism in B-KRnet to maintain the exact invertibility. Using B-KRnet as a transport map, we obtain an explicit probability density function (PDF) model that corresponds to the pushforward of a prior (uniform) distribution on the hypercube. To approximate PDFs defined on a bounded computational domain, B-KRnet is more effective than KRnet. By coupling KRnet and B-KRnet, we can also define a deep generative model on a high-dimensional domain where some dimensions are bounded and other dimensions are unbounded. A typical case is the solution of the stationary kinetic Fokker-Planck equation, which is a PDF of position and momentum. Based on B-KRnet, we develop an adaptive learning approach to approximate partial differential equations whose solutions are PDFs or can be regarded as a PDF. In addition, we apply B-KRnet to density estimation when only data are available. A variety of numerical experiments is presented to demonstrate the effectiveness of B-KRnet., Comment: 26 pages, 13 figures

With the development of computed tomography (CT) imaging technology, it is possible to acquire multi-energy data by spectral CT. Being different from conventional CT, the X-ray energy spectrum of spectral CT is cutting into several narrow bins which leads to the result that only a part of photon can be collected in each individual energy channel, which cause the image qualities to be severely degraded by noise and artifacts. To address this problem, we propose a spectral CT reconstruction algorithm based on low-rank representation and structure preserving regularization in this paper. To make full use of the prior knowledge about both the inter-channel correlation and the sparsity in gradient domain of inner-channel data, this paper combines a low-rank correlation descriptor with a structure extraction operator as priori regularization terms for spectral CT reconstruction. Furthermore, a split-Bregman based iterative algorithm is developed to solve the reconstruction model. Finally, we propose a multi-channel adaptive parameters generation strategy according to CT values of each individual energy channel. Experimental results on numerical simulations and real mouse data indicate that the proposed algorithm achieves higher accuracy on both reconstruction and material decomposition than the methods based on simultaneous algebraic reconstruction technique (SART), total variation minimization (TVM), total variation with low-rank (LRTV), and spatial-spectral cube matching frame (SSCMF). Compared with SART, our algorithm improves the feature similarity (FSIM) by 40.4% on average for numerical simulation reconstruction, whereas TVM, LRTV, and SSCMF correspond to 26.1%, 28.2%, and 29.5%, respectively., Comment: 21 pages, 9 figures

The exoplanet field now abounds with new discoveries of planets and planetary systems. It is of great interest for the scientific community to understand the compositions and internal structures of these new planets outside our own solar system, and then infer their formation scenarios. In particular, the proper implementation of the equation-of-states of various matters into the calculation of internal structure is crucial. Thus, we have come up with this write-up that provides the detailed steps to turn an equation-of-state (EOS) into a mass-radius relation of exoplanets, and highlight the physics and chemistry involved in these calculations. It must be visualized!, Comment: For pedagogical purpose only (non-profit). Comments and suggestions welcome!

We explain the overall equilibrium-temperature-dependent trend in the exoplanet mass-radius diagram, using the escape mechanisms of hydrogen and relevant volatiles, and the chemical equilibrium calculation of molecular hydrogen (H$_2$) break-up into atomic hydrogen (H). We identify two Cosmic Hydrogen and Ice Loss Lines (CHILLs) in the mass-radius diagram. Gas disks are well known to disperse in ten million years. However, gas-rich planets may lose some or almost all gas on a much longer timescale. We thus hypothesize that most planets that are born out of a hydrogen-gas-dominated nebular disk begin by possessing a primordial H$_2$-envelope. This envelope is gradually lost due to escape processes caused by host-stellar radiation., Comment: (Version 3) Published in Icarus, Volume 414, 116033, 2024

We propose in this work the gradient-enhanced deep neural networks (DNNs) approach for function approximations and uncertainty quantification. More precisely, the proposed approach adopts both the function evaluations and the associated gradient information to yield enhanced approximation accuracy. In particular, the gradient information is included as a regularization term in the gradient-enhanced DNNs approach, for which we present similar posterior estimates (by the two-layer neural networks) as those in the path-norm regularized DNNs approximations. We also discuss the application of this approach to gradient-enhanced uncertainty quantification, and present several numerical experiments to show that the proposed approach can outperform the traditional DNNs approach in many cases of interests., Comment: 14 pages, 3 figures

In this work, we propose adaptive deep learning approaches based on normalizing flows for solving fractional Fokker-Planck equations (FPEs). The solution of a FPE is a probability density function (PDF). Traditional mesh-based methods are ineffective because of the unbounded computation domain, a large number of dimensions and the nonlocal fractional operator. To this end, we represent the solution with an explicit PDF model induced by a flow-based deep generative model, simplified KRnet, which constructs a transport map from a simple distribution to the target distribution. We consider two methods to approximate the fractional Laplacian. One method is the Monte Carlo approximation. The other method is to construct an auxiliary model with Gaussian radial basis functions (GRBFs) to approximate the solution such that we may take advantage of the fact that the fractional Laplacian of a Gaussian is known analytically. Based on these two different ways for the approximation of the fractional Laplacian, we propose two models, MCNF and GRBFNF, to approximate stationary FPEs and MCTNF to approximate time-dependent FPEs. To further improve the accuracy, we refine the training set and the approximate solution alternately. A variety of numerical examples is presented to demonstrate the effectiveness of our adaptive deep density approaches., Comment: 25 pages, 22 figures

The squeezed cat state, an essential quantum resource, can be used for quantum error correction and slowing decoherence of the optical cat state. However, preparing a squeezed cat state with high generation rate, and effectively manipulating it, remain challenging. In this work, a high-performance all-optical in-line squeezer is developed to prepare a squeezed cat state and manipulate the phase of the quadrature squeezing. This scheme has the advantages that the phase of the quadrature squeezing of the squeezed cat state can be manipulated by changing the working condition of the squeezer, and that a higher generation rate can be achieved via the deterministic squeezing operation of the in-line squeezer. The generation rate of squeezed cat states reaches 2 kHz, the same as that of the initial cat state. The all-optical in-line squeezer proposed here removes the requirements of electro-optic and opto-electric conversions necessary for an off-line squeezer, thus enabling high-bandwidth squeezing operations on non-Gaussian states. These results provide an efficient method to prepare and manipulate optical squeezed cat states, which makes a step closer to their applications in all-optical quantum information processing., Comment: Published online in Laser & Photonics Reviews