Search

We derive explicit restriction and continuation formulas between n-dimensional (Anti)-de Sitter spaces and the (n + 1)-dimensional Minkowskian ambient space for the codifferential and Laplace-de Rham operators acting on p-forms., Comment: 9 pages, v2: typos corrected

Entity Linking (EL) and Relation Extraction (RE) are fundamental tasks in Natural Language Processing, serving as critical components in a wide range of applications. In this paper, we propose ReLiK, a Retriever-Reader architecture for both EL and RE, where, given an input text, the Retriever module undertakes the identification of candidate entities or relations that could potentially appear within the text. Subsequently, the Reader module is tasked to discern the pertinent retrieved entities or relations and establish their alignment with the corresponding textual spans. Notably, we put forward an innovative input representation that incorporates the candidate entities or relations alongside the text, making it possible to link entities or extract relations in a single forward pass and to fully leverage pre-trained language models contextualization capabilities, in contrast with previous Retriever-Reader-based methods, which require a forward pass for each candidate. Our formulation of EL and RE achieves state-of-the-art performance in both in-domain and out-of-domain benchmarks while using academic budget training and with up to 40x inference speed compared to competitors. Finally, we show how our architecture can be used seamlessly for Information Extraction (cIE), i.e. EL + RE, and setting a new state of the art by employing a shared Reader that simultaneously extracts entities and relations., Comment: To be presented at ACL 2024

The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spatial information within the images. To overcome these challenges, we introduce ImageFlowNet, a novel framework that learns latent-space flow fields that evolve multiscale representations in joint embedding spaces using neural ODEs and SDEs to model disease progression in the image domain. Notably, ImageFlowNet learns multiscale joint representation spaces by combining cohorts of patients together so that information can be transferred between the patient samples. The dynamics then provide plausible trajectories of progression, with the SDE providing alternative trajectories from the same starting point. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We then demonstrate ImageFlowNet's effectiveness through empirical evaluations on three longitudinal medical image datasets depicting progression in retinal geographic atrophy, multiple sclerosis, and glioblastoma., Comment: Fixed some typos. Merged multibib

We investigate spatially resolved variations in the bandgap energy across multiple InxGa1-xAs quantum wells (QWs) on a GaAs substrate within a metamorphic laser structure. Using high resolution scanning transmission electron microscopy and low-loss electron energy loss spectroscopy, we present a detailed analysis of the local bandgap energy, indium concentration, and strain distribution within the QWs. Our findings reveal significant inhomogeneities, particularly near the interfaces, in both the strain and indium content, and a bandgap variability across QWs. These results are correlated with density functional theory simulations to further elucidate the interplay between strain, composition, and bandgap energy. This work underscores the importance of spatially resolved analysis in understanding, and optimising, the electronic and optical properties of semiconductor devices. The study suggests that the collective impact of individual QWs might affect the emission and performance of the final device, providing insights for the design of next-generation metamorphic lasers with multiple QWs as the active region.

Signals from interacting brain regions display transient synchronization of phases and amplitudes in different frequencies. Commonly, the interaction between regions of the brain is quantitatively described by either analyzing the correlations of amplitudes of the measured signals or by calculating phase-synchronization measures. However, for a complete picture of the interactions it is important to analyze the dynamics of both the amplitude and the phase. In this work, we present a new method for finding the coupling between brain regions by reconstructing the phase-amplitude dynamics directly from the measured electrophysiological signals. For this purpose, we use the recent advances in the field of phase-amplitude reduction of oscillatory systems, which allow the representation of an uncoupled oscillatory system as a phase-amplitude oscillator in a unique form using transformations (parametrizations) related to the eigenfunctions of the Koopman operator. By combining the parametrization method and the Fourier-Laplace averaging method of finding the eigenfunctions of the Koopman operator, we developed a novel method of assessing the transformation functions from the signals of the interacting oscillatory systems. The resulting reconstructed dynamical system is a network of phase-amplitude oscillators with the interactions between them represented as coupling functions in phase and amplitude coordinates. Using synthetic signals generated from several models with known and unknown theoretical phase-amplitude reduced forms, we demonstrate that our method is capable of finding the proper unique dynamic form of these oscillatory systems in the reduced phase-amplitude space. Our method can be applied to describe any network of interacting oscillators as a dynamical system using signals of the network elements.

Identities pertaining to the de Rham codifferential {\delta} in differential geometry are scattered in the literature. This article gathers such formulas involving usual differential operators (Lie derivative, Schouten-Nijenhuis bracket, ...), while adding a few new ones using a natural extension of the interior product, to provide a compact handy summary., Comment: Revtex 8 pages, v2: typo corrected

Proteins are essential for almost all biological processes and derive their diverse functions from complex 3D structures, which are in turn determined by their amino acid sequences. In this paper, we exploit the rich biological inductive bias of amino acid sequences and introduce FoldFlow-2, a novel sequence-conditioned SE(3)-equivariant flow matching model for protein structure generation. FoldFlow-2 presents substantial new architectural features over the previous FoldFlow family of models including a protein large language model to encode sequence, a new multi-modal fusion trunk that combines structure and sequence representations, and a geometric transformer based decoder. To increase diversity and novelty of generated samples -- crucial for de-novo drug design -- we train FoldFlow-2 at scale on a new dataset that is an order of magnitude larger than PDB datasets of prior works, containing both known proteins in PDB and high-quality synthetic structures achieved through filtering. We further demonstrate the ability to align FoldFlow-2 to arbitrary rewards, e.g. increasing secondary structures diversity, by introducing a Reinforced Finetuning (ReFT) objective. We empirically observe that FoldFlow-2 outperforms previous state-of-the-art protein structure-based generative models, improving over RFDiffusion in terms of unconditional generation across all metrics including designability, diversity, and novelty across all protein lengths, as well as exhibiting generalization on the task of equilibrium conformation sampling. Finally, we demonstrate that a fine-tuned FoldFlow-2 makes progress on challenging conditional design tasks such as designing scaffolds for the VHH nanobody., Comment: preprint

Motivated by a problem from incompressible fluid mechanics of Brenier (JAMS 1989), and its recent entropic relaxation by Arnaudo, Cruizero, L\'eonard & Zambrini (AIHP PS 2020), we study a problem of entropic minimization on the path space when the reference measure is a generic Feller semimartingale. We show that, under some regularity condition, our problem connects naturally with a, possibly non-local, version of the Hamilton-Jacobi-Bellman equation. Additionally, we study existence of minimizers when the reference measure in a Ornstein-Uhlenbeck process., Comment: 13 pages, comments welcome!

Differentially-private (DP) mechanisms can be embedded into the design of a machine learningalgorithm to protect the resulting model against privacy leakage, although this often comes with asignificant loss of accuracy. In this paper, we aim at improving this trade-off for rule lists modelsby establishing the smooth sensitivity of the Gini impurity and leveraging it to propose a DP greedyrule list algorithm. In particular, our theoretical analysis and experimental results demonstrate thatthe DP rule lists models integrating smooth sensitivity have higher accuracy that those using otherDP frameworks based on global sensitivity.

Machine learning techniques are increasingly used for high-stakes decision-making, such as college admissions, loan attribution or recidivism prediction. Thus, it is crucial to ensure that the models learnt can be audited or understood by human users, do not create or reproduce discrimination or bias, and do not leak sensitive information regarding their training data. Indeed, interpretability, fairness and privacy are key requirements for the development of responsible machine learning, and all three have been studied extensively during the last decade. However, they were mainly considered in isolation, while in practice they interplay with each other, either positively or negatively. In this Systematization of Knowledge (SoK) paper, we survey the literature on the interactions between these three desiderata. More precisely, for each pairwise interaction, we summarize the identified synergies and tensions. These findings highlight several fundamental theoretical and empirical conflicts, while also demonstrating that jointly considering these different requirements is challenging when one aims at preserving a high level of utility. To solve this issue, we also discuss possible conciliation mechanisms, showing that a careful design can enable to successfully handle these different concerns in practice.

We describe embeddings of $n$-dimensional Lorentzian manifolds, including Friedmann-Lema\^itre-Robertson-Walker spaces, in $\mathbb{R}^{n+2}$ such that the metrics of the submanifolds are inherited by a restriction from that of $\mathbb{R}^{n+2}$, and the action of the linear group SO$(2, n)$ of the ambient space reduces to conformal transformations on the submanifolds., Comment: 6 pages, no figures, v2: close match to published version

Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. Specifically, we define diffusion spectral entropy (DSE) in neural representations of a dataset as well as diffusion spectral mutual information (DSMI) between different variables representing data. First, we show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data that outperform classic Shannon entropy, nonparametric estimation, and mutual information neural estimation (MINE). We then study the evolution of representations in classification networks with supervised learning, self-supervision, or overfitting. We observe that (1) DSE of neural representations increases during training; (2) DSMI with the class label increases during generalizable learning but stays stagnant during overfitting; (3) DSMI with the input signal shows differing trends: on MNIST it increases, while on CIFAR-10 and STL-10 it decreases. Finally, we show that DSE can be used to guide better network initialization and that DSMI can be used to predict downstream classification accuracy across 962 models on ImageNet. The official implementation is available at https://github.com/ChenLiu-1996/DiffusionSpectralEntropy.

Macroscopic oscillations in the brain are involved in various cognitive and physiological processes, yet their precise function is not not completely understood. Communication Through Coherence (CTC) theory proposes that these rhythmic electrical patterns might serve to regulate the information flow between neural populations. Thus, to communicate effectively, neural populations must synchronize their oscillatory activity, ensuring that input volleys from the presynaptic population reach the postsynaptic one at its maximum phase of excitability. We consider an Excitatory-Inhibitory (E-I) network whose macroscopic activity is described by an exact mean-field model. The E-I network receives periodic inputs from either one or two external sources, for which effective communication will not be achieved in the absence of control. We explore strategies based on optimal control theory for phase-amplitude dynamics to design a control that sets the target population in the optimal phase to synchronize its activity with a specific presynaptic input signal and establish communication. The control mechanism resembles the role of a higher cortical area in the context of selective attention. To design the control, we use the phase-amplitude reduction of a limit cycle and leverage recent developments in this field in order to find the most effective control strategy regarding a defined cost function. Furthermore, we present results that guarantee the local controllability of the system close to the limit cycle.

The computational design of novel protein structures has the potential to impact numerous scientific disciplines greatly. Toward this goal, we introduce FoldFlow, a series of novel generative models of increasing modeling power based on the flow-matching paradigm over $3\mathrm{D}$ rigid motions -- i.e. the group $\text{SE}(3)$ -- enabling accurate modeling of protein backbones. We first introduce FoldFlow-Base, a simulation-free approach to learning deterministic continuous-time dynamics and matching invariant target distributions on $\text{SE}(3)$. We next accelerate training by incorporating Riemannian optimal transport to create FoldFlow-OT, leading to the construction of both more simple and stable flows. Finally, we design FoldFlow-SFM, coupling both Riemannian OT and simulation-free training to learn stochastic continuous-time dynamics over $\text{SE}(3)$. Our family of FoldFlow, generative models offers several key advantages over previous approaches to the generative modeling of proteins: they are more stable and faster to train than diffusion-based approaches, and our models enjoy the ability to map any invariant source distribution to any invariant target distribution over $\text{SE}(3)$. Empirically, we validate FoldFlow, on protein backbone generation of up to $300$ amino acids leading to high-quality designable, diverse, and novel samples., Comment: ICLR 2024 Spotlight

Interpretability is often pointed out as a key requirement for trustworthy machine learning. However, learning and releasing models that are inherently interpretable leaks information regarding the underlying training data. As such disclosure may directly conflict with privacy, a precise quantification of the privacy impact of such breach is a fundamental problem. For instance, previous work have shown that the structure of a decision tree can be leveraged to build a probabilistic reconstruction of its training dataset, with the uncertainty of the reconstruction being a relevant metric for the information leak. In this paper, we propose of a novel framework generalizing these probabilistic reconstructions in the sense that it can handle other forms of interpretable models and more generic types of knowledge. In addition, we demonstrate that under realistic assumptions regarding the interpretable models' structure, the uncertainty of the reconstruction can be computed efficiently. Finally, we illustrate the applicability of our approach on both decision trees and rule lists, by comparing the theoretical information leak associated to either exact or heuristic learning algorithms. Our results suggest that optimal interpretable models are often more compact and leak less information regarding their training data than greedily-built ones, for a given accuracy level.

We present simulation-free score and flow matching ([SF]$^2$M), a simulation-free objective for inferring stochastic dynamics given unpaired samples drawn from arbitrary source and target distributions. Our method generalizes both the score-matching loss used in the training of diffusion models and the recently proposed flow matching loss used in the training of continuous normalizing flows. [SF]$^2$M interprets continuous-time stochastic generative modeling as a Schr\"odinger bridge problem. It relies on static entropy-regularized optimal transport, or a minibatch approximation, to efficiently learn the SB without simulating the learned stochastic process. We find that [SF]$^2$M is more efficient and gives more accurate solutions to the SB problem than simulation-based methods from prior work. Finally, we apply [SF]$^2$M to the problem of learning cell dynamics from snapshot data. Notably, [SF]$^2$M is the first method to accurately model cell dynamics in high dimensions and can recover known gene regulatory networks from simulated data. Our code is available in the TorchCFM package at https://github.com/atong01/conditional-flow-matching., Comment: AISTATS 2024. Code: https://github.com/atong01/conditional-flow-matching

Abstract Meaning Representation (AMR) is a Semantic Parsing formalism that aims at providing a semantic graph abstraction representing a given text. Current approaches are based on autoregressive language models such as BART or T5, fine-tuned through Teacher Forcing to obtain a linearized version of the AMR graph from a sentence. In this paper, we present LeakDistill, a model and method that explores a modification to the Transformer architecture, using structural adapters to explicitly incorporate graph information into the learned representations and improve AMR parsing performance. Our experiments show how, by employing word-to-node alignment to embed graph structural information into the encoder at training time, we can obtain state-of-the-art AMR parsing through self-knowledge distillation, even without the use of additional data. We release the code at \url{http://www.github.com/sapienzanlp/LeakDistill}., Comment: ACL 2023. Please cite authors correctly using both lastnames ("Mart\'inez Lorenzo", "Huguet Cabot")

In this paper, we examine the current state-of-the-art in AMR parsing, which relies on ensemble strategies by merging multiple graph predictions. Our analysis reveals that the present models often violate AMR structural constraints. To address this issue, we develop a validation method, and show how ensemble models can exploit SMATCH metric weaknesses to obtain higher scores, but sometimes result in corrupted graphs. Additionally, we highlight the demanding need to compute the SMATCH score among all possible predictions. To overcome these challenges, we propose two novel ensemble strategies based on Transformer models, improving robustness to structural constraints, while also reducing the computational time. Our methods provide new insights for enhancing AMR parsers and metrics. Our code is available at \href{https://www.github.com/babelscape/AMRs-Assemble}{github.com/babelscape/AMRs-Assemble}., Comment: ACL 2023. Please cite authors correctly using both lastnames ("Mart\'inez Lorenzo", "Huguet Cabot")

Relation Extraction (RE) is a task that identifies relationships between entities in a text, enabling the acquisition of relational facts and bridging the gap between natural language and structured knowledge. However, current RE models often rely on small datasets with low coverage of relation types, particularly when working with languages other than English. In this paper, we address the above issue and provide two new resources that enable the training and evaluation of multilingual RE systems. First, we present SRED$^{\rm FM}$, an automatically annotated dataset covering 18 languages, 400 relation types, 13 entity types, totaling more than 40 million triplet instances. Second, we propose RED$^{\rm FM}$, a smaller, human-revised dataset for seven languages that allows for the evaluation of multilingual RE systems. To demonstrate the utility of these novel datasets, we experiment with the first end-to-end multilingual RE model, mREBEL, that extracts triplets, including entity types, in multiple languages. We release our resources and model checkpoints at https://www.github.com/babelscape/rebel, Comment: ACL 2023. Please cite authors correctly using both lastnames ("Huguet Cabot", "Ngonga Ngomo")

Cortical spreading depression and spreading depolarization (CSD) are waves of neuronal depolarization that spread across the cortex, leading to a temporary saturation of brain activity. They are associated to various brain disorders such as migraine and ischemia. We consider a reduced version of a biophysical model of a neuron-astrocyte network for the initiation and propagation of CSD waves (Huguet et al., Biophys. J. , 2016), consisting of reaction-diffusion equations. The reduced model considers only the dynamics of the neuronal and astrocytic membrane potentials and the extracellular potassium concentration, capturing the instigation process implicated in such waves. We present a computational and mathematical framework based on the parameterization method and singular perturbation theory to provide semi-analytical results on the existence of a wave solution and to compute it jointly with its velocity of propagation. The traveling wave solution can be seen as an heteroclinic connection of an associated system of ordinary differential equations with a slow-fast dynamics. The presence of distinct time scales within the system introduces numerical instabilities, which we successfully address through the identification of significant invariant manifolds and the implementation of the parameterization method. Our results provide a methodology that allows to identify efficiently and accurately the mechanisms responsible for the initiation of these waves and the wave propagation velocity., Comment: 36 pages, 13 figures

While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little attention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in the biomedical sciences. Thus, it becomes important to find ways to compare signals defined on such graphs. Here, we propose Graph Fourier MMD (GFMMD), a novel distance between distributions and signals on graphs. GFMMD is defined via an optimal witness function that is both smooth on the graph and maximizes difference in expectation between the pair of distributions on the graph. We find an analytical solution to this optimization problem as well as an embedding of distributions that results from this method. We also prove several properties of this method including scale invariance and applicability to disconnected graphs. We showcase it on graph benchmark datasets as well on single cell RNA-sequencing data analysis. In the latter, we use the GFMMD-based gene embeddings to find meaningful gene clusters. We also propose a novel type of score for gene selection called "gene localization score" which helps select genes for cellular state space characterization.

Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the underlying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifold model for the data. Neural FIM creates an extensible metric space from discrete point cloud data such that information from the metric can inform us of manifold characteristics such as volume and geodesics. We demonstrate Neural FIM's utility in selecting parameters for the PHATE visualization method as well as its ability to obtain information pertaining to local volume illuminating branching points and cluster centers embeddings of a toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs (immune cells)., Comment: 13 pages, 11 figures, 1 table

Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoretical links have been established. Here, we establish such a link via results in Riemannian geometry explicitly connecting heat diffusion to manifold distances. In this process, we also formulate a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. This novel perspective makes clearer the choices available in manifold learning and denoising. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances, and preserving cluster structure in toy datasets. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure, where our method enables interpolation of withheld timepoints of data. Finally, we show that parameters of our more general method can be configured to give results similar to PHATE (a state-of-the-art diffusion based manifold learning method) as well as SNE (an attraction/repulsion neighborhood based method that forms the basis of t-SNE)., Comment: 31 pages, 13 figures, 10 tables

Halophilic microorganisms have long been known to survive within the brine inclusions of salt crystals, as evidenced by the change in color for salt crystals containing pigmented halophiles. However, the molecular mechanisms allowing this survival has remained an open question for decades. While protocols for the surface sterilization of halite (NaCl) have enabled isolation of cells and DNA from within halite brine inclusions, "-omics" based approaches have faced two main technical challenges: (1) removal of all contaminating organic biomolecules (including proteins) from halite surfaces, and (2) performing selective biomolecule extractions directly from cells contained within halite brine inclusions with sufficient speed to avoid modifications in gene expression during extraction. In this study, we tested different methods to resolve these two technical challenges. Following this method development, we then applied the optimized methods to perform the first examination of the early acclimation of a model haloarchaeon (Halobacterium salinarum NRC-1) to halite brine inclusions. Examinations of the proteome of Halobacterium cells two months post-evaporation revealed a high degree of similarity with stationary phase liquid cultures, but with a sharp down-regulation of ribosomal proteins. While proteins for central metabolism were part of the shared proteome between liquid cultures and halite brine inclusions, proteins involved in cell mobility (archaellum, gas vesicles) were either absent or less abundant in halite samples. Proteins unique to cells within brine inclusions included transporters, suggesting modified interactions between cells and the surrounding brine inclusion microenvironment. The methods and hypotheses presented here enable future studies of the survival of halophiles in both culture model and natural halite systems.

Machine Learning models are increasingly used for decision making, in particular in high-stakes applications such as credit scoring, medicine or recidivism prediction. However, there are growing concerns about these models with respect to their lack of interpretability and the undesirable biases they can generate or reproduce. While the concepts of interpretability and fairness have been extensively studied by the scientific community in recent years, few works have tackled the general multi-class classification problem under fairness constraints, and none of them proposes to generate fair and interpretable models for multi-class classification. In this paper, we use Mixed-Integer Linear Programming (MILP) techniques to produce inherently interpretable scoring systems under sparsity and fairness constraints, for the general multi-class classification setup. Our work generalizes the SLIM (Supersparse Linear Integer Models) framework that was proposed by Rudin and Ustun to learn optimal scoring systems for binary classification. The use of MILP techniques allows for an easy integration of diverse operational constraints (such as, but not restricted to, fairness or sparsity), but also for the building of certifiably optimal models (or sub-optimal models with bounded optimality gap).

Within the de Sitter ambient space framework, the two different bases of the one-particle Hilbert space of the de Sitter group algebra are presented for the scalar case. Using field operator algebra and its Fock space construction in this formalism, we discuss the existence of asymptotic states in de Sitter QFT under an extension of the adiabatic hypothesis and prove the Fock space completeness theorem for the massive scalar field. We define the quantum state in the limit of future and past infinity on the Sitter hyperboloid in an observer-independent way. These results allow us to examine the existence of the $S$-matrix operator for de Sitter QFT in ambient space formalism, a question usually obscure in spacetime with a cosmological event horizon for a specific observer. Some similarities and differences between QFT in Minkowski and de Sitter spaces are discussed., Comment: 21 pages

We obtain new sharp weighted Poincar{\'e} inequalities on Riemannian manifolds for a general class of measures. When specialised to generalised Cauchy measures, this gives a unified and simple proof of the weighted Poincar{\'e} inequality for the whole range of parameters, with the optimal spectral gap, the error term and the extremal functions.