Search

Tracing the modifications of molecules in surface chemical reactions benefits from the possibility to image their orbitals. While delocalized frontier orbitals with {\pi}-character are imaged routinely with photoemission orbital tomography, they are not always sensitive to local chemical modifications, particularly the making and breaking of bonds at the molecular periphery. For such bonds, {\sigma}-orbitals would be far more revealing. Here, we show that these orbitals can indeed be imaged in a remarkably broad energy range, and that the plane wave approximation, an important ingredient of photoemission orbital tomography, is also well fulfilled for these orbitals. This makes photoemission orbital tomography a unique tool for the detailed analysis of surface chemical reactions. We demonstrate this by identifying the reaction product of a dehalogenation and cyclodehydrogenation reaction.

Supercontinuum sources are all-fiber pulsed laser-driven systems that provide high power spectral densities within ultra-broadband spectral ranges. The tailored process of generating broadband, bright, and spectrally flat supercontinua -- through a complex interplay of linear and non-linear processes -- has been recently pushed further towards longer wavelengths and has evolved enough to enter the field of mid-infrared (mid-IR) spectroscopy. In this work, we review the current state and perspectives of this technology that offers laser-like emission properties and instantaneous broadband spectral coverage comparable to thermal emitters. We aim to go beyond a literature review. Thus, we first discuss the basic principles of supercontinuum sources and then provide an experimental part focusing on the quantification and analysis of intrinsic emission properties such as typical power spectral densities, brightness levels, spectral stability, and beam quality (to the best of the authors' knowledge, the M2 factor for a mid-IR supercontinuum source is characterized for the first time). On this basis, we identify key competitive advantages of these alternative emitters for mid-IR spectroscopy over state-of-the-art technologies such as thermal sources or quantum cascade lasers. The specific features of supercontinuum radiation open up prospects of improving well-established techniques in mid-IR spectroscopy and trigger developments of novel analytical methods and instrumentation. The review concludes with a structured summary of recent advances and applications in various routine mid-IR spectroscopy scenarios that have benefited from the use of supercontinuum sources., Comment: 34 pages, 13 figures

Including covariant information, such as position, force, velocity or spin is important in many tasks in computational physics and chemistry. We introduce Steerable E(3) Equivariant Graph Neural Networks (SEGNNs) that generalise equivariant graph networks, such that node and edge attributes are not restricted to invariant scalars, but can contain covariant information, such as vectors or tensors. This model, composed of steerable MLPs, is able to incorporate geometric and physical information in both the message and update functions. Through the definition of steerable node attributes, the MLPs provide a new class of activation functions for general use with steerable feature fields. We discuss ours and related work through the lens of equivariant non-linear convolutions, which further allows us to pin-point the successful components of SEGNNs: non-linear message aggregation improves upon classic linear (steerable) point convolutions; steerable messages improve upon recent equivariant graph networks that send invariant messages. We demonstrate the effectiveness of our method on several tasks in computational physics and chemistry and provide extensive ablation studies., Comment: Published at ICLR 2022 (Spotlight paper), Github: https://github.com/RobDHess/Steerable-E3-GNN

Pairing is the fundamental requirement for fermionic superfluidity and superconductivity. To understand the mechanism behind pair formation is an ongoing challenge in the study of many strongly correlated fermionic systems. Cooper pairs are the key ingredient to BCS theory as the microscopic explanation of conventional superconductivity. They form between particles of opposite spin and momentum at the Fermi surface of the system. Here, we directly observe Cooper pairs in a mesoscopic two-dimensional Fermi gas. We apply an imaging scheme that enables us to extract the full in-situ momentum distribution of a strongly interacting Fermi gas with single particle and spin resolution. Our ultracold gas allows us to freely tune between a completely non-interacting, unpaired system and weak attractions, where we find Cooper pair correlations at the Fermi surface. When increasing the attractive interactions even further, the pairs gradually turn into deeply bound molecules breaking up the Fermi surface. Our mesoscopic system is closely related to the physics of nuclei, superconducting grains or quantum dots. With the precise control over interactions, particle number and potential landscape in our experiment, the observables we establish in this work provide a new approach to longstanding questions concerning not only such mesoscopic systems but also their connection to the macroscopic world., Comment: 19 pages, 12 figures

The abundance of data has given machine learning considerable momentum in natural sciences and engineering, though modeling of physical processes is often difficult. A particularly tough problem is the efficient representation of geometric boundaries. Triangularized geometric boundaries are well understood and ubiquitous in engineering applications. However, it is notoriously difficult to integrate them into machine learning approaches due to their heterogeneity with respect to size and orientation. In this work, we introduce an effective theory to model particle-boundary interactions, which leads to our new Boundary Graph Neural Networks (BGNNs) that dynamically modify graph structures to obey boundary conditions. The new BGNNs are tested on complex 3D granular flow processes of hoppers, rotating drums and mixers, which are all standard components of modern industrial machinery but still have complicated geometry. BGNNs are evaluated in terms of computational efficiency as well as prediction accuracy of particle flows and mixing entropies. BGNNs are able to accurately reproduce 3D granular flows within simulation uncertainties over hundreds of thousands of simulation timesteps. Most notably, in our experiments, particles stay within the geometric objects without using handcrafted conditions or restrictions., Comment: accepted for presentation at the Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23)

Recently, the application of machine learning models has gained momentum in natural sciences and engineering, which is a natural fit due to the abundance of data in these fields. However, the modeling of physical processes from simulation data without first principle solutions remains difficult. Here, we present a Graph Neural Networks approach towards accurate modeling of complex 3D granular flow simulation processes created by the discrete element method LIGGGHTS and concentrate on simulations of physical systems found in real world applications like rotating drums and hoppers. We discuss how to implement Graph Neural Networks that deal with 3D objects, boundary conditions, particle - particle, and particle - boundary interactions such that an accurate modeling of relevant physical quantities is made possible. Finally, we compare the machine learning based trajectories to LIGGGHTS trajectories in terms of particle flows and mixing entropies.

In our paper we analyze the attack surface of German hospitals and healthcare providers in 2020 during the COVID-19 Pandemic. The analysis looked at the publicly visible attack surface utilizing a Distributed Cyber Recon System, utilizing distributed Internet scanning, Big Data methods and scan data of 1,483 GB from more than 89 different global Internet scans. From the 1,555 identified German clinical entities, security posture analysis was conducted by looking at more than 13,000 service banners for version identification and subsequent CVE-based vulnerability identification. Primary analysis shows that 32 percent of the analyzed services were determined as vulnerable to various degrees and 36 percent of all hospitals showed numerous vulnerabilities. Further resulting vulnerability statistics were mapped against size of organization and hospital bed count., Comment: Preprint for NATO CCDCOE Annual International Conference on Cyber Conflict (CyCon) [Paper submitted to Peer Review Process]

We prove under commonly used assumptions the convergence of actor-critic reinforcement learning algorithms, which simultaneously learn a policy function, the actor, and a value function, the critic. Both functions can be deep neural networks of arbitrary complexity. Our framework allows showing convergence of the well known Proximal Policy Optimization (PPO) and of the recently introduced RUDDER. For the convergence proof we employ recently introduced techniques from the two time-scale stochastic approximation theory. Our results are valid for actor-critic methods that use episodic samples and that have a policy that becomes more greedy during learning. Previous convergence proofs assume linear function approximation, cannot treat episodic examples, or do not consider that policies become greedy. The latter is relevant since optimal policies are typically deterministic., Comment: 20 pages

In order to quickly adapt to new data, few-shot learning aims at learning from few examples, often by using already acquired knowledge. The new data often differs from the previously seen data due to a domain shift, that is, a change of the input-target distribution. While several methods perform well on small domain shifts like new target classes with similar inputs, larger domain shifts are still challenging. Large domain shifts may result in high-level concepts that are not shared between the original and the new domain, whereas low-level concepts like edges in images might still be shared and useful. For cross-domain few-shot learning, we suggest representation fusion to unify different abstraction levels of a deep neural network into one representation. We propose Cross-domain Hebbian Ensemble Few-shot learning (CHEF), which achieves representation fusion by an ensemble of Hebbian learners acting on different layers of a deep neural network. Ablation studies show that representation fusion is a decisive factor to boost cross-domain few-shot learning. On the few-shot datasets miniImagenet and tieredImagenet with small domain shifts, CHEF is competitive with state-of-the-art methods. On cross-domain few-shot benchmark challenges with larger domain shifts, CHEF establishes novel state-of-the-art results in all categories. We further apply CHEF on a real-world cross-domain application in drug discovery. We consider a domain shift from bioactive molecules to environmental chemicals and drugs with twelve associated toxicity prediction tasks. On these tasks, that are highly relevant for computational drug discovery, CHEF significantly outperforms all its competitors. Github: https://github.com/ml-jku/chef

Frontier orbitals, i.e., the highest occupied and lowest unoccupied orbitals of a molecule, generally determine molecular properties, such as chemical bonding and reactivities. Consequently, there has been a lot of interest in measuring them, despite the fact that, strictly speaking, they are not quantum-mechanical observables. Yet, with photoemission tomography a powerful technique has recently been introduced by which the electron distribution in orbitals of molecules adsorbed at surfaces can be imaged in momentum space. This has even been used for the identification of reaction intermediates in surface reactions. However, so far it has been impossible to follow an orbital's momentum-space dynamics in time, for example through an excitation process or a chemical reaction. Here, we report a key step in this direction: we combine time-resolved photoemission employing high laser harmonics and a recently developed momentum microscope to establish a tomographic, femtosecond pump-probe experiment of unoccupied molecular orbitals. Specifically, we measure the full momentum-space distribution of transiently excited electrons. Because in molecules this momentum-space distribution is closely linked to orbital shapes, our experiment offers the extraordinary possibility to observe ultrafast electron motion in time and space. This enables us to connect their excited states dynamics to specific real-space excitation pathways.

Reinforcement learning algorithms require many samples when solving complex hierarchical tasks with sparse and delayed rewards. For such complex tasks, the recently proposed RUDDER uses reward redistribution to leverage steps in the Q-function that are associated with accomplishing sub-tasks. However, often only few episodes with high rewards are available as demonstrations since current exploration strategies cannot discover them in reasonable time. In this work, we introduce Align-RUDDER, which utilizes a profile model for reward redistribution that is obtained from multiple sequence alignment of demonstrations. Consequently, Align-RUDDER employs reward redistribution effectively and, thereby, drastically improves learning on few demonstrations. Align-RUDDER outperforms competitors on complex artificial tasks with delayed rewards and few demonstrations. On the Minecraft ObtainDiamond task, Align-RUDDER is able to mine a diamond, though not frequently. Code is available at https://github.com/ml-jku/align-rudder. YouTube: https://youtu.be/HO-_8ZUl-UY, Comment: Github: https://github.com/ml-jku/align-rudder, YouTube: https://youtu.be/HO-_8ZUl-UY

For organic molecules adsorbed as well-oriented ultra-thin films on metallic surfaces, angle-resolved photoemission spectroscopy has evolved into a technique called photoemission tomography (PT). By approximating the final state of the photoemitted electron as a free electron, PT uses the angular dependence of the photocurrent, a so-called momentum map or k-map, and interprets it as the Fourier transform of the initial state's molecular orbital, thereby gains insights into the geometric and electronic structure of organic/metal interfaces. In this contribution, we present kMap.py which is a Python program that enables the user, via a PyQt-based graphical user interface, to simulate photoemission momentum maps of molecular orbitals and to perform a one-to-one comparison between simulation and experiment. Based on the plane wave approximation for the final state, simulated momentum maps are computed numerically from a fast Fourier transform of real space molecular orbital distributions, which are used as program input and taken from density functional calculations. The program allows the user to vary a number of simulation parameters such as the final state kinetic energy, the molecular orientation or the polarization state of the incident light field. Moreover, also experimental photoemission data can be loaded into the program enabling a direct visual comparison as well as an automatic optimization procedure to determine structural parameters of the molecules or weights of molecular orbitals contributions. With an increasing number of experimental groups employing photoemission tomography to study adsorbate layers, we expect kMap.py to serve as an ideal analysis software to further extend the applicability of PT.

A central mechanism in machine learning is to identify, store, and recognize patterns. How to learn, access, and retrieve such patterns is crucial in Hopfield networks and the more recent transformer architectures. We show that the attention mechanism of transformer architectures is actually the update rule of modern Hopfield networks that can store exponentially many patterns. We exploit this high storage capacity of modern Hopfield networks to solve a challenging multiple instance learning (MIL) problem in computational biology: immune repertoire classification. Accurate and interpretable machine learning methods solving this problem could pave the way towards new vaccines and therapies, which is currently a very relevant research topic intensified by the COVID-19 crisis. Immune repertoire classification based on the vast number of immunosequences of an individual is a MIL problem with an unprecedentedly massive number of instances, two orders of magnitude larger than currently considered problems, and with an extremely low witness rate. In this work, we present our novel method DeepRC that integrates transformer-like attention, or equivalently modern Hopfield networks, into deep learning architectures for massive MIL such as immune repertoire classification. We demonstrate that DeepRC outperforms all other methods with respect to predictive performance on large-scale experiments, including simulated and real-world virus infection data, and enables the extraction of sequence motifs that are connected to a given disease class. Source code and datasets: https://github.com/ml-jku/DeepRC, Comment: 10 pages (+appendix); Source code and datasets: https://github.com/ml-jku/DeepRC

We introduce a modern Hopfield network with continuous states and a corresponding update rule. The new Hopfield network can store exponentially (with the dimension of the associative space) many patterns, retrieves the pattern with one update, and has exponentially small retrieval errors. It has three types of energy minima (fixed points of the update): (1) global fixed point averaging over all patterns, (2) metastable states averaging over a subset of patterns, and (3) fixed points which store a single pattern. The new update rule is equivalent to the attention mechanism used in transformers. This equivalence enables a characterization of the heads of transformer models. These heads perform in the first layers preferably global averaging and in higher layers partial averaging via metastable states. The new modern Hopfield network can be integrated into deep learning architectures as layers to allow the storage of and access to raw input data, intermediate results, or learned prototypes. These Hopfield layers enable new ways of deep learning, beyond fully-connected, convolutional, or recurrent networks, and provide pooling, memory, association, and attention mechanisms. We demonstrate the broad applicability of the Hopfield layers across various domains. Hopfield layers improved state-of-the-art on three out of four considered multiple instance learning problems as well as on immune repertoire classification with several hundreds of thousands of instances. On the UCI benchmark collections of small classification tasks, where deep learning methods typically struggle, Hopfield layers yielded a new state-of-the-art when compared to different machine learning methods. Finally, Hopfield layers achieved state-of-the-art on two drug design datasets. The implementation is available at: https://github.com/ml-jku/hopfield-layers, Comment: 10 pages (+ appendix); 12 figures; Blog: https://ml-jku.github.io/hopfield-layers/; GitHub: https://github.com/ml-jku/hopfield-layers

Optical coherence tomography (OCT) is a high-resolution three-dimensional imaging technique that enables non-destructive measurements of surface and subsurface microstructures. Recent developments of OCT operating in the mid-infrared (MIR) range (around 4 {\mu}m) lifted fundamental scattering limitations and initiated applied material research in formerly inaccessible fields. The MIR spectral region, however, is also of great interest for spectroscopy and hyperspectral imaging, which allow highly selective and sensitive chemical studies of materials. In this contribution, we introduce an OCT system (dual-band, central wavelengths of 2 {\mu}m m and 4 {\mu}m) combined with MIR spectroscopy that is implemented as a raster scanning chemical imaging modality. The fully-integrated and cost-effective optical instrument is based on a single supercontinuum laser source (emission spectrum spanning from 1.1 {\mu}m to 4.4 {\mu}m). Capabilities of the in-situ correlative measurements are experimentally demonstrated by obtaining complex multidimensional material data, comprising morphological and chemical information, from a multi-layered composite ceramic-polymer specimen., Comment: 12 pages, 9 figures

Fourier transform infrared (FTIR) spectrometers have been the dominant technology in the field of mid-infrared (MIR)spectroscopy for decades. Supercontinuum laser sources operating in the MIR spectral region now offer the potential to enrich the field of FTIR spectroscopy due to their distinctive properties, such as high-brightness, broadband spectral coverage and enhanced stability. In our contribution, we introduce this advanced light source as a replacement for conventional thermal emitters. Furthermore, an approach to efficient coupling of pulsed MIR supercontinuum sources to FTIR spectrometers is proposed and considered in detail. The experimental part is devoted to pulse-to-pulse energy fluctuations of the applied supercontinuum laser, performance of the system, as well as the noise and long-term stability. Comparative measurements performed with a conventional FTIR instrument equipped with a thermal emitter illustrate that similar noise levels can be achieved with the supercontinuum-based system. The analytical performance of the supercontinuum-based FTIR spectrometer was tested for a concentration series of aqueous formaldehyde solutions in a liquid flow cell (500 $\mu$m path length) and compared with the conventional FTIR (130 $\mu$m path length). The results show a four-times-enhanced detection limit due to the extended path length enabled by the high brightness of the laser. In conclusion, FTIR spectrometers equipped with novel broadband MIR supercontinuum lasers could outperform traditional systems providing superior performance, e.g., interaction path lengths formerly unattainable, while maintaining low noise levels known from highly stable thermal emitters.