Search

We present a sensor-agnostic spectral transformer as the basis for spectral foundation models. To that end, we introduce a Universal Spectral Representation (USR) that leverages sensor meta-data, such as sensing kernel specifications and sensing wavelengths, to encode spectra obtained from any spectral instrument into a common representation, such that a single model can ingest data from any sensor. Furthermore, we develop a methodology for pre-training such models in a self-supervised manner using a novel random sensor-augmentation and reconstruction pipeline to learn spectral features independent of the sensing paradigm. We demonstrate that our architecture can learn sensor independent spectral features that generalize effectively to sensors not seen during training. This work sets the stage for training foundation models that can both leverage and be effective for the growing diversity of spectral data.

This paper describes the 2nd edition of the ICML Topological Deep Learning Challenge that was hosted within the ICML 2024 ELLIS Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM). The challenge focused on the problem of representing data in different discrete topological domains in order to bridge the gap between Topological Deep Learning (TDL) and other types of structured datasets (e.g. point clouds, graphs). Specifically, participants were asked to design and implement topological liftings, i.e. mappings between different data structures and topological domains --like hypergraphs, or simplicial/cell/combinatorial complexes. The challenge received 52 submissions satisfying all the requirements. This paper introduces the main scope of the challenge, and summarizes the main results and findings., Comment: Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM) at ICML 2024

We present a framework for inferring an atmospheric transmission profile from a spectral scene. This framework leverages a lightweight, physics-based simulator that is automatically tuned - by virtue of autodifferentiation and differentiable programming - to construct a surrogate atmospheric profile to model the observed data. We demonstrate utility of the methodology by (i) performing atmospheric correction, (ii) recasting spectral data between various modalities (e.g. radiance and reflectance at the surface and at the sensor), and (iii) inferring atmospheric transmission profiles, such as absorbing bands and their relative magnitudes., Comment: Manuscript accepted for presentation and publication at the 2024 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)

This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning. The competition asked participants to provide open-source implementations of topological neural networks from the literature by contributing to the python packages TopoNetX (data processing) and TopoModelX (deep learning). The challenge attracted twenty-eight qualifying submissions in its two-month duration. This paper describes the design of the challenge and summarizes its main findings.

Hyperspectral measurements from long range sensors can give a detailed picture of the items, materials, and chemicals in a scene but analysis can be difficult, slow, and expensive due to high spatial and spectral resolutions of state-of-the-art sensors. As such, sparsity is important to enable the future of spectral compression and analytics. It has been observed that environmental and atmospheric effects, including scattering, can produce nonlinear effects posing challenges for existing source separation and compression methods. We present a novel transformation into Hilbert spaces for pruning and constructing sparse representations via non-negative least squares minimization. Then we introduce max likelihood compression vectors to decrease information loss. Our approach is benchmarked against standard pruning and least squares as well as deep learning methods. Our methods are evaluated in terms of overall spectral reconstruction error and compression rate using real and synthetic data. We find that pruning least squares methods converge quickly unlike matching pursuit methods. We find that Hilbert space pruning can reduce error by as much as 40% of the error of standard pruning and also outperform neural network autoencoders.

Past work exploring adversarial vulnerability have focused on situations where an adversary can perturb all dimensions of model input. On the other hand, a range of recent works consider the case where either (i) an adversary can perturb a limited number of input parameters or (ii) a subset of modalities in a multimodal problem. In both of these cases, adversarial examples are effectively constrained to a subspace $V$ in the ambient input space $\mathcal{X}$. Motivated by this, in this work we investigate how adversarial vulnerability depends on $\dim(V)$. In particular, we show that the adversarial success of standard PGD attacks with $\ell^p$ norm constraints behaves like a monotonically increasing function of $\epsilon (\frac{\dim(V)}{\dim \mathcal{X}})^{\frac{1}{q}}$ where $\epsilon$ is the perturbation budget and $\frac{1}{p} + \frac{1}{q} =1$, provided $p > 1$ (the case $p=1$ presents additional subtleties which we analyze in some detail). This functional form can be easily derived from a simple toy linear model, and as such our results land further credence to arguments that adversarial examples are endemic to locally linear models on high dimensional spaces., Comment: Comments welcome! V2: minor edits for clarity

Successful deployment of geological carbon storage (GCS) requires an extensive use of reservoir simulators for screening, ranking and optimization of storage sites. However, the time scales of GCS are such that no sufficient long-term data is available yet to validate the simulators against. As a consequence, there is currently no solid basis for assessing the quality with which the dynamics of large-scale GCS operations can be forecasted. To meet this knowledge gap, we have conducted a major GCS validation benchmark study. To achieve reasonable time scales, a laboratory-size geological storage formation was constructed (the "FluidFlower"), forming the basis for both the experimental and computational work. A validation experiment consisting of repeated GCS operations was conducted in the FluidFlower, providing what we define as the true physical dynamics for this system. Nine different research groups from around the world provided forecasts, both individually and collaboratively, based on a detailed physical and petrophysical characterization of the FluidFlower sands. The major contribution of this paper is a report and discussion of the results of the validation benchmark study, complemented by a description of the benchmarking process and the participating computational models. The forecasts from the participating groups are compared to each other and to the experimental data by means of various indicative qualitative and quantitative measures. By this, we provide a detailed assessment of the capabilities of reservoir simulators and their users to capture both the injection and post-injection dynamics of the GCS operations.

It is often said that a deep learning model is "invariant" to some specific type of transformation. However, what is meant by this statement strongly depends on the context in which it is made. In this paper we explore the nature of invariance and equivariance of deep learning models with the goal of better understanding the ways in which they actually capture these concepts on a formal level. We introduce a family of invariance and equivariance metrics that allows us to quantify these properties in a way that disentangles them from other metrics such as loss or accuracy. We use our metrics to better understand the two most popular methods used to build invariance into networks: data augmentation and equivariant layers. We draw a range of conclusions about invariance and equivariance in deep learning models, ranging from whether initializing a model with pretrained weights has an effect on a trained model's invariance, to the extent to which invariance learned via training can generalize to out-of-distribution data., Comment: To appear at NeurIPS 2022

We present a method for the in-situ determination of Young's modulus of a nanomechanical string resonator subjected to tensile stress. It relies on measuring a large number of harmonic eigenmodes and allows to access Young's modulus even for the case of a stress-dominated frequency response. We use the proposed framework to obtain the Young's modulus of four different wafer materials, comprising the three different material platforms amorphous silicon nitride, crystalline silicon carbide and crystalline indium gallium phosphide. The resulting values are compared with theoretical and literature values where available, revealing the need to measure Young's modulus on the sample material under investigation for precise device characterization.

In many classification problems, we want a classifier that is robust to a range of non-semantic transformations. For example, a human can identify a dog in a picture regardless of the orientation and pose in which it appears. There is substantial evidence that this kind of invariance can significantly improve the accuracy and generalization of machine learning models. A common technique to teach a model geometric invariances is to augment training data with transformed inputs. However, which invariances are desired for a given classification task is not always known. Determining an effective data augmentation policy can require domain expertise or extensive data pre-processing. Recent efforts like AutoAugment optimize over a parameterized search space of data augmentation policies to automate the augmentation process. While AutoAugment and similar methods achieve state-of-the-art classification accuracy on several common datasets, they are limited to learning one data augmentation policy. Often times different classes or features call for different geometric invariances. We introduce Dynamic Network Augmentation (DNA), which learns input-conditional augmentation policies. Augmentation parameters in our model are outputs of a neural network and are implicitly learned as the network weights are updated. Our model allows for dynamic augmentation policies and performs well on data with geometric transformations conditional on input features.

We investigate how effective an attacker can be when it only learns from its victim's actions, without access to the victim's reward. In this work, we are motivated by the scenario where the attacker wants to behave strategically when the victim's motivations are unknown. We argue that one heuristic approach an attacker can use is to maximize the entropy of the victim's policy. The policy is generally not obfuscated, which implies it may be extracted simply by passively observing the victim. We provide such a strategy in the form of a reward-free exploration algorithm that maximizes the attacker's entropy during the exploration phase, and then maximizes the victim's empirical entropy during the planning phase. In our experiments, the victim agents are subverted through policy entropy maximization, implying an attacker might not need access to the victim's reward to succeed. Hence, reward-free attacks, which are based only on observing behavior, show the feasibility of an attacker to act strategically without knowledge of the victim's motives even if the victim's reward information is protected., Comment: Presented at the NeurIPS 2021 Workshop on Learning in Presence of Strategic Behavior

In recent years, nanomechanics has evolved into a mature field, with wide-ranging impact from sensing applications to fundamental physics, and it has now reached a stage which enables the fabrication and study of ever more elaborate devices. This has led to the emergence of arrays of coupled nanomechanical resonators as a promising field of research, serving as model systems to study collective dynamical phenomena such as synchronization or topological transport. From a general point of view, the arrays investigated so far represent scalar fields on a lattice. Moving to a scenario where these could be extended to vector fields would unlock a whole host of conceptually interesting additional phenomena, including the physics of polarization patterns in wave fields and their associated topology. Here we introduce a new platform, a two-dimensional array of coupled nanomechanical pillar resonators, whose orthogonal vibration directions encode a mechanical polarization degree of freedom. We demonstrate direct optical imaging of the collective dynamics, enabling us to analyze the emerging polarization patterns and follow their evolution with drive frequency., Comment: 7 pages main text, 5 figures, 11 pages appendices

In multiscale modelling, multiple models are used simultaneously to describe scale-dependent phenomena in a system of interest. Here we introduce a machine learning (ML)-based multiscale modelling framework for modelling hierarchical multiscale problems. In these problems, closure relations are required for the macroscopic problem in the form of constitutive relations. However, forming explicit closures for nonlinear and hysteretic processes remains challenging. Instead, we provide a framework for learning constitutive mappings given microscale data generated according to micro and macro transitions governed by two-scale homogenisation rules. The resulting data-driven model is then coupled to a macroscale simulator leading to a hybrid ML-physics-based modelling approach. Accordingly, we apply the multiscale framework within the context of transient phenomena in dual-porosity geomaterials. In these materials, the inter-porosity flow is a complex time-dependent function making its adoption within flow simulators challenging. We explore nonlinear feedforward autoregressive ML strategies for the constitutive modelling of this sequential problem. We demonstrate how to inject the resulting surrogate constitutive model into a simulator. We then compare the resulting hybrid approach to traditional dual-porosity and microscale models on a variety of tests. We show the hybrid approach to give high-quality results with respect to explicit microscale simulations without the computational burden of the latter. Lastly, the steps provided by the multiscale framework herein are sufficiently general to be applied to a variety of multiscale settings, using different data generation and learning techniques accordingly.

Building invariance to non-meaningful transformations is essential to building efficient and generalizable machine learning models. In practice, the most common way to learn invariance is through data augmentation. There has been recent interest in the development of methods that learn distributions on augmentation transformations from the training data itself. While such approaches are beneficial since they are responsive to the data, they ignore the fact that in many situations the range of transformations to which a model needs to be invariant changes depending on the particular class input belongs to. For example, if a model needs to be able to predict whether an image contains a starfish or a dog, we may want to apply random rotations to starfish images during training (since these do not have a preferred orientation), but we would not want to do this to images of dogs. In this work we introduce a method by which we can learn class conditional distributions on augmentation transformations. We give a number of examples where our methods learn different non-meaningful transformations depending on class and further show how our method can be used as a tool to probe the symmetries intrinsic to a potentially complex dataset., Comment: 10 pages, 6 figures, submitted to NeurIPS 2021

We present stress-sensitive permeability relationships for two-dimensional fracture networks in the Opalinus Clay from the Mont Terri underground rock laboratory. These relationships may be used as a proxy for fracture network permeability in numerical models that resolve large spatial scales and are used in a variety of GeoEnergy applications involving flow in shaly rocks. To obtain these relationships we present a numerical procedure that uses experimentally determined stress-permeability relationships to numerically compute the effective permeability of the network. The material discontinuities stemming from the fractures are treated by a simple contact-interaction algorithm that accounts for normal interaction between fracture walls, allowing us to calculate the permeability of a fracture network under different stress conditions. We apply the procedure to four fracture networks digitized from two galleries of the Mont Terri rock laboratory. These fracture networks are mapped from the damage zone of the Main Fault that intersects the Opalinus Clay. The networks show a maximum variation of four orders of magnitude when stress ranges from 1MPa to 20 MPa. Our numerical procedure not only establishes representative stress-permeability relationships for a fractured rock mass under stress, but also provides a proxy for fracture network permeability for simulation in fractured formations., Comment: 24 pages, 11 figures

Advances in natural language processing have resulted in increased capabilities with respect to multiple tasks. One of the possible causes of the observed performance gains is the introduction of increasingly sophisticated text representations. While many of the new word embedding techniques can be shown to capture particular notions of sentiment or associative structures, we explore the ability of two different word embeddings to uncover or capture the notion of logical shape in text. To this end we present a novel framework that we call Topological Word Embeddings which leverages mathematical techniques in dynamical system analysis and data driven shape extraction (i.e. topological data analysis). In this preliminary work we show that using a topological delay embedding we are able to capture and extract a different, shape-based notion of logic aimed at answering the question "Can we find a circle in a circular argument?"

Dual-continuum (DC) models can be tractable alternatives to explicit approaches for the numerical modelling of multiscale materials with multiphysics behaviours. This work concerns the conceptual and numerical modelling of poroelastically coupled dual-scale materials such as naturally fractured rock. Apart from a few exceptions, previous poroelastic DC models have assumed isotropy of the constituents and the dual-material. Additionally, it is common to assume that only one continuum has intrinsic stiffness properties. Finally, little has been done into validating whether the DC paradigm can capture the global poroelastic behaviours of explicit numerical representations at the DC modelling scale. We address the aforementioned knowledge gaps in two steps. First, we utilise a homogenisation approach based on Levin's theorem to develop a previously derived anisotropic poroelastic constitutive model. Our development incorporates anisotropic intrinsic stiffness properties of both continua. This addition is in analogy to anisotropic fractured rock masses with stiff fractures. Second, we perform numerical modelling to test the dual-continuum model against fine-scale explicit equivalents. In doing, we present our hybrid numerical framework, as well as the conditions required for interpretation of the numerical results. The tests themselves progress from materials with isotropic to anisotropic mechanical and flow properties. The fine-scale simulations show anisotropy can have noticeable effects on deformation and flow behaviour. However, our numerical experiments show the DC approach can capture the global poroelastic behaviours of both isotropic and anisotropic fine-scale representations.

A dual-continuum model can offer a practical approach to understanding first-order behaviours of poromechanically coupled multiscale systems. To close the governing equations, constitutive equations with models to calculate effective constitutive coefficients are required. Several coefficient models have been proposed within the literature. However, a holistic overview of the different modelling concepts is still missing. To address this we first compare and contrast the dominant models existing within the literature. In terms of the constitutive relations themselves, early relations were indirectly postulated that implicitly neglected the effect of the mechanical interaction arising between continuum pressures. Further, recent users of complete constitutive systems that include inter-continuum pressure coupling have explicitly neglected these couplings as a means of providing direct relations between composite and constituent properties, and to simplify coefficient models. Within the framework of micromechanics, we show heuristically that these explicit decouplings are in fact coincident with bounds on the effective parameters themselves. Depending on the formulation, these bounds correspond to end-member states of isostress or isostrain. We show the impacts of using constitutive coefficient models, decoupling assumptions and parameter bounds on poromechanical behaviours using analytical solutions for a 2D model problem. Based on the findings herein, we offer recommendations for how and when to use different coefficient modelling concepts., Comment: Transport in Porous Media (2019)