Search

Controlled ordinary differential equations driven by continuous bounded variation curves can be considered a continuous time analogue of recurrent neural networks for the construction of expressive features of the input curves. We ask up to which extent well known signature features of such curves can be reconstructed from controlled ordinary differential equations with (untrained) random vector fields. The answer turns out to be algebraically involved, but essentially the number of signature features, which can be reconstructed from the non-linear flow of the controlled ordinary differential equation, is exponential in its hidden dimension, when the vector fields are chosen to be neural with depth two. Moreover, we characterize a general linear independence condition on arbitrary vector fields, under which the signature features up to some fixed order can always be reconstructed. Algebraically speaking this complements in a quantitative manner several well known results from the theory of Lie algebras of vector fields and puts them in a context of machine learning., Comment: 37 pages, 7 figures

Solar wind is frequently categorized based on its respective solar source region. Two well-established categorizations of the coronal hole wind, the scheme based on the charge-state composition, and the scheme based on proton plasma, identify a very different fraction of solar wind in the data from the Advanced Composition Explorer (ACE) as coronal hole wind during the solar activity minimum at the end of solar cycle 24. We investigate possible explanations for the different identifications of the coronal wind in 2009 in the scheme based on the charge-state composition (almost only coronal hole wind) and in the scheme based on the proton plasma (almost no coronal hole wind at the same time). We compared the properties of the respective coronal hole wind types and their changes with solar activity cycle in 2001- 2010. As a comparison reference, we included the coronal hole wind as identified by an unsupervised machine-learning approach, k-means, in our analysis. We find that the scheme based on charge-state composition likely misidentifies some slow solar wind as coronal hole wind during the solar activity minimum. The k-means classification we considered includes two types of coronal hole wind, the first of which is dominant during the solar activity maximum, whereas the second is dominant during the solar activity minimum. A low fraction of coronal hole wind from low-latitude coronal holes observed by ACE in 2009 is plausible because during this time period, a very small number of low-latitude coronal holes was observed. The results imply that the origin-oriented solar wind classification needs to be revisited, and they also suggest that an explicit inclusion of the phase of the solar activity cycle can be expected to improve the classification of the solar wind.

The iterative bleaching extends multiplexity (IBEX) Knowledge-Base is a central portal for researchers adopting IBEX and related 2D and 3D immunofluorescence imaging methods. The design of the Knowledge-Base is modeled after efforts in the open-source software community and includes three facets: a development platform (GitHub), static website, and service for data archiving. The Knowledge-Base facilitates the practice of open science throughout the research life cycle by providing validation data for recommended and non-recommended reagents, e.g., primary and secondary antibodies. In addition to reporting negative data, the Knowledge-Base empowers method adoption and evolution by providing a venue for sharing protocols, videos, datasets, software, and publications. A dedicated discussion forum fosters a sense of community among researchers while addressing questions not covered in published manuscripts. Together, scientists from around the world are advancing scientific discovery at a faster pace, reducing wasted time and effort, and instilling greater confidence in the resulting data.

Time-resolved X-ray absorption spectroscopy (tr-XAS) has been shown to be a versatile measurement technique for investigating non-equilibrium dynamics. Novel X-ray free electron laser (XFEL) facilities like the European XFEL offer increased repetition rates for stroboscopic XAS experiments through a burst operation mode, which enables measurements with up to 4.5 MHz. These higher repetition rates lead to higher data acquisition rates but can also introduce long-lived excitations that persist and thus build up during each burst. Here, we report on such long-lived effects in Ni and NiO thin film samples that were measured at the European XFEL. We disentangle the long-lived excitations from the initial pump-induced change and perform a detailed modelling-based analysis of how they modify transient X-ray spectra. As a result, we link the long-lived effects in Ni to a local temperature increase, as well as the effects in NiO to excited charge carrier trapping through polaron formation. In addition, we present possible correction methods, as well as discuss ways in which the effects of these long-lived excitations could be minimized for future time-resolved X-ray absorption spectroscopy measurements.

When clustering molecular dynamics (MD) trajectories into a few metastable conformational states, the Markov state models (MSMs) assumption of timescale separation between fast intrastate fluctuations and rarely occurring interstate transitions is often not valid. Hence, the naive estimation of the macrostate transition matrix via simply counting transitions between the states leads to significantly too short implied timescales and thus to too fast population decays. In this work, we discuss advanced approaches to estimate the transition matrix. Assuming that Markovianity is at least given at the microstate level, we consider the Laplace-transform based method by Hummer and Szabo, as well as a direct microstate-to-macrostate projection, which by design yields correct macrostate population dynamics. Alternatively, we study the recently proposed quasi-MSM ansatz of Huang and coworkers to solve a generalized master equations, as well as a hybrid method that employs MD at short times and MSM at long times. Adopting a one-dimensional toy model and an all-atom folding trajectory of HP35, we discuss the virtues and shortcomings of the various approaches.

Large language models (LLMs) have demonstrated remarkable performance across various downstream tasks. However, the high computational and memory requirements of LLMs are a major bottleneck. To address this, parameter-efficient fine-tuning (PEFT) methods such as low-rank adaptation (LoRA) have been proposed to reduce computational costs while ensuring minimal loss in performance. Additionally, knowledge distillation (KD) has been a popular choice for obtaining compact student models from teacher models. In this work, we present KD-LoRA, a novel fine-tuning method that combines LoRA with KD. Our results demonstrate that KD-LoRA achieves performance comparable to full fine-tuning (FFT) and LoRA while significantly reducing resource requirements. Specifically, KD-LoRA retains 98% of LoRA's performance on the GLUE benchmark, while being 40% more compact. Additionally, KD-LoRA reduces GPU memory usage by 30% compared to LoRA, while decreasing inference time by 30% compared to both FFT and LoRA. We evaluate KD-LoRA across three encoder-only models: BERT, RoBERTa, and DeBERTaV3. Code is available at https://github.com/rambodazimi/KD-LoRA., Comment: Accepted at 4th NeurIPS Efficient Natural Language and Speech Processing Workshop (ENLSP-IV 2024)

The precision of contouring target structures and organs-at-risk (OAR) in radiotherapy planning is crucial for ensuring treatment efficacy and patient safety. Recent advancements in deep learning (DL) have significantly improved OAR contouring performance, yet the reliability of these models, especially in the presence of out-of-distribution (OOD) scenarios, remains a concern in clinical settings. This application study explores the integration of epistemic uncertainty estimation within the OAR contouring workflow to enable OOD detection in clinically relevant scenarios, using specifically compiled data. Furthermore, we introduce an advanced statistical method for OOD detection to enhance the methodological framework of uncertainty estimation. Our empirical evaluation demonstrates that epistemic uncertainty estimation is effective in identifying instances where model predictions are unreliable and may require an expert review. Notably, our approach achieves an AUC-ROC of 0.95 for OOD detection, with a specificity of 0.95 and a sensitivity of 0.92 for implant cases, underscoring its efficacy. This study addresses significant gaps in the current research landscape, such as the lack of ground truth for uncertainty estimation and limited empirical evaluations. Additionally, it provides a clinically relevant application of epistemic uncertainty estimation in an FDA-approved and widely used clinical solution for OAR segmentation from Varian, a Siemens Healthineers company, highlighting its practical benefits., Comment: Keywords: Epistemic Uncertainty - Out-of-Distribution Detection - CT Segmentation - OAR contouring - Radiotherapy

Celcomen leverages a mathematical causality framework to disentangle intra- and inter- cellular gene regulation programs in spatial transcriptomics and single-cell data through a generative graph neural network. It can learn gene-gene interactions, as well as generate post-perturbation counterfactual spatial transcriptomics, thereby offering access to experimentally inaccessible samples. We validated its disentanglement, identifiability, and counterfactual prediction capabilities through simulations and in clinically relevant human glioblastoma, human fetal spleen, and mouse lung cancer samples. Celcomen provides the means to model disease and therapy induced changes allowing for new insights into single-cell spatially resolved tissue responses relevant to human health.

Iterative Bleaching Extends multipleXity (IBEX) is a versatile method for highly multiplexed imaging of diverse tissues. Based on open science principles, we created the IBEX Knowledge-Base, a resource for reagents, protocols and more, to empower innovation., Comment: 8 pages, 1 figure, 9 references

The Path-dependent Neural Jump ODE (PD-NJ-ODE) is a model for online prediction of generic (possibly non-Markovian) stochastic processes with irregular (in time) and potentially incomplete (with respect to coordinates) observations. It is a model for which convergence to the $L^2$-optimal predictor, which is given by the conditional expectation, is established theoretically. Thereby, the training of the model is solely based on a dataset of realizations of the underlying stochastic process, without the need of knowledge of the law of the process. In the case where the underlying process is deterministic, the conditional expectation coincides with the process itself. Therefore, this framework can equivalently be used to learn the dynamics of ODE or PDE systems solely from realizations of the dynamical system with different initial conditions. We showcase the potential of our method by applying it to the chaotic system of a double pendulum. When training the standard PD-NJ-ODE method, we see that the prediction starts to diverge from the true path after about half of the evaluation time. In this work we enhance the model with two novel ideas, which independently of each other improve the performance of our modelling setup. The resulting dynamics match the true dynamics of the chaotic system very closely. The same enhancements can be used to provably enable the PD-NJ-ODE to learn long-term predictions for general stochastic datasets, where the standard model fails. This is verified in several experiments.

Lenses with dynamic focal length, also called zoom functionality, enable a variety of applications related to imaging and sensing. The traditional approach of stacking refractive lenses to achieve this functionality results in an expensive, heavy optical system. Especially for applications in the mid-infrared, light weight and compact form factor are required. In this work, we use a meta-optic triplet to demonstrate zoom imaging at mid-wave infrared wavelengths. By varying the axial distances between the optics, the meta-optic triplet achieves high quality imaging over a zoom range of 5x, with 50$^\circ$ full field of view in the widest configuration and an aperture of 8 mm. This triplet system demonstrates the potential for meta-optics to reduce conventional components in complex and multi-functional imaging systems to dramatically thinner and lighter components.

Robust utility optimization enables an investor to deal with market uncertainty in a structured way, with the goal of maximizing the worst-case outcome. In this work, we propose a generative adversarial network (GAN) approach to (approximately) solve robust utility optimization problems in general and realistic settings. In particular, we model both the investor and the market by neural networks (NN) and train them in a mini-max zero-sum game. This approach is applicable for any continuous utility function and in realistic market settings with trading costs, where only observable information of the market can be used. A large empirical study shows the versatile usability of our method. Whenever an optimal reference strategy is available, our method performs on par with it and in the (many) settings without known optimal strategy, our method outperforms all other reference strategies. Moreover, we can conclude from our study that the trained path-dependent strategies do not outperform Markovian ones. Lastly, we uncover that our generative approach for learning optimal, (non-) robust investments under trading costs generates universally applicable alternatives to well known asymptotic strategies of idealized settings.

Resonant Inelastic X-ray Scattering (RIXS) is an ideal X-ray spectroscopy method to push the combination of energy and time resolutions to the Fourier transform ultimate limit, because it is unaffected by the core-hole lifetime energy broadening. And in pump-probe experiments the interaction time is made very short by the same core-hole lifetime. RIXS is very photon hungry so it takes great advantage from high repetition rate pulsed X-ray sources like the European XFEL. The hRIXS instrument is designed for RIXS experiments in the soft X-ray range with energy resolution approaching the Fourier and the Heisenberg limits. It is based on a spherical grating with variable line spacing (VLS) and a position-sensitive 2D detector. Initially, two gratings are installed to adequately cover the whole photon energy range. With optimized spot size on the sample and small pixel detector the energy resolution can be better than 40 meV at any photon energy below 1000 eV. At the SCS instrument of the European XFEL the spectrometer can be easily positioned thanks to air-pads on a high-quality floor, allowing the scattering angle to be continuously adjusted over the 65-145 deg range. It can be coupled to two different sample interaction chamber, one for liquid jets and one for solids, each equipped at the state-of-the-art and compatible for optical laser pumping in collinear geometry. The measured performances, in terms of energy resolution and count rate on the detector, closely match design expectations. hRIXS is open to public users since the summer of 2022., Comment: 43 pages, 12 figures, Supplemental Material

Materials to be deployed in space applications have to undergo a variety of different test scenarios, simulating actual space conditions. Among these materials solar photovoltaic cells, optics, meta-materials and more will be directly exposed to space radiation and must be tested accordingly. From the design phase of such target materials to the final production, it is important to obtain information about their behavior and performance in defined irradiation scenarios and qualify them following the directions of relevant ECSS/ESA standards. An excellent method to cover part of these tests in a laboratory scale is the high-flux, high-fluence electron irradiation with the help of industrial electron beam generators. The present collaboration has developed different irradiation test setups and procedures, adapted to the existing electron beam generators at IPF in Dresden, Germany covering an energy range between 100 keV to 1.5 MeV. The test setups, aiming for high electron flux, uniform and large irradiation field sizes, and their capabilities for use in irradiation qualification tests for space applications are described in this paper., Comment: To be published on RADECS 2023 Data Workshop.It contains 4 pages, 7 Figures, 2 Tables, 7 References

We present convincing empirical results on the application of Randomized Signature Methods for non-linear, non-parametric drift estimation for a multi-variate financial market. Even though drift estimation is notoriously ill defined due to small signal to noise ratio, one can still try to learn optimal non-linear maps from data to future returns for the purposes of portfolio optimization. Randomized Signatures, in contrast to classical signatures, allow for high dimensional market dimension and provide features on the same scale. We do not contribute to the theory of Randomized Signatures here, but rather present our empirical findings on portfolio selection in real world settings including real market data and transaction costs.

The photoinduced switching of Fe(II)-based spin-crossover complexes from singlet to quintet takes place at ultrafast time scales. This a priori spin-forbidden transition triggered numerous time-resolved experiments of solvated samples to elucidate the mechanism at play. The involved intermediate states remain uncertain. We apply ultrafast x-ray spectroscopy in molecular films as a method sensitive to spin, electronic, and nuclear degrees of freedom. Combining the progress in molecule synthesis and film growth with the opportunities at x-ray free-electron lasers, we analyze the transient evolution of the Fe L3 fine structure at room temperature. Our measurements and calculations indicate the involvement of an Fe triplet intermediate state. The high-spin state saturates at half of the available molecules, limited by molecule-molecule interaction within the film.