Search

Recurrent neural networks are a successful neural architecture for many time-dependent problems, including time series analysis, forecasting, and modeling of dynamical systems. Training such networks with backpropagation through time is a notoriously difficult problem because their loss gradients tend to explode or vanish. In this contribution, we introduce a computational approach to construct all weights and biases of a recurrent neural network without using gradient-based methods. The approach is based on a combination of random feature networks and Koopman operator theory for dynamical systems. The hidden parameters of a single recurrent block are sampled at random, while the outer weights are constructed using extended dynamic mode decomposition. This approach alleviates all problems with backpropagation commonly related to recurrent networks. The connection to Koopman operator theory also allows us to start using results in this area to analyze recurrent neural networks. In computational experiments on time series, forecasting for chaotic dynamical systems, and control problems, as well as on weather data, we observe that the training time and forecasting accuracy of the recurrent neural networks we construct are improved when compared to commonly used gradient-based methods.

Dynamical systems (DS) theory is fundamental for many areas of science and engineering. It can provide deep insights into the behavior of systems evolving in time, as typically described by differential or recursive equations. A common approach to facilitate mathematical tractability and interpretability of DS models involves decomposing nonlinear DS into multiple linear DS separated by switching manifolds, i.e. piecewise linear (PWL) systems. PWL models are popular in engineering and a frequent choice in mathematics for analyzing the topological properties of DS. However, hand-crafting such models is tedious and only possible for very low-dimensional scenarios, while inferring them from data usually gives rise to unnecessarily complex representations with very many linear subregions. Here we introduce Almost-Linear Recurrent Neural Networks (AL-RNNs) which automatically and robustly produce most parsimonious PWL representations of DS from time series data, using as few PWL nonlinearities as possible. AL-RNNs can be efficiently trained with any SOTA algorithm for dynamical systems reconstruction (DSR), and naturally give rise to a symbolic encoding of the underlying DS that provably preserves important topological properties. We show that for the Lorenz and R\"ossler systems, AL-RNNs discover, in a purely data-driven way, the known topologically minimal PWL representations of the corresponding chaotic attractors. We further illustrate on two challenging empirical datasets that interpretable symbolic encodings of the dynamics can be achieved, tremendously facilitating mathematical and computational analysis of the underlying systems., Comment: 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

The parton branching method is crucial for Monte Carlo generators, which are essential for high-energy physics predictions. We examine the impact of soft gluons on inclusive collinear and Transverse Momentum Dependent (TMD) parton densities. By applying the Parton-Branching (PB) method, we identify the non-perturbative Sudakov form factor with the integration range $z \to 1$, which is often neglected in collinear parton shower approaches. The significance of soft gluons is demonstrated through the transverse momentum spectrum of Drell-Yan lepton pairs, resulting in an intrinsic-$k_t$ distribution width that remains independent of $\sqrt(s)$, contrary to observations in collinear parton shower approaches. This behavior is attributed to the non-perturbative Sudakov form factor., Comment: 5 pages, 2 Figures. Based on a presentation given at the 31st International Workshop on Deep Inelastic Scattering (DIS2024), 8-12 April 2024, Grenoble, France

uPDFevolv2 is a software package designed for evolving collinear and Transverse Momentum Dependent (TMD) parton densities using the DGLAP evolution equation. A comprehensive description of both the theoretical framework and technical implementation is given, accompanied by a detailed guide on program usage, focusing on customizable parameters. This report is as a technical release note for uPDFevolv version 2.5.03., Comment: Technical release note for version 2.5.03. Missing references added

The security of integrated circuits (ICs) can be broken by sophisticated physical attacks relying on failure analysis methods. Optical probing is one of the most prominent examples of such attacks, which can be accomplished in a matter of days, even with limited knowledge of the IC under attack. Unfortunately, few countermeasures are proposed in the literature, and none has been fabricated and tested in practice. These countermeasures usually require changing the standard cell libraries and, thus, are incompatible with digital and programmable platforms, such as field programmable gate arrays (FPGAs). In this work, we shift our attention from preventing the attack to detecting and responding to it. We introduce LaserEscape, the first fully digital and FPGA-compatible countermeasure to detect and mitigate optical probing attacks. LaserEscape incorporates digital delay-based sensors to reliably detect the physical alteration on the fabric caused by laser beam irradiations in real time. Furthermore, as a response to the attack, LaserEscape deploys real-time hiding approaches using randomized hardware reconfigurability. It realizes 1) moving target defense (MTD) to physically move the sensitive circuity under attack out of the probing field of focus to protect secret keys and 2) polymorphism to logically obfuscate the functionality of the targeted circuit to counter function extraction and reverse engineering attempts. We demonstrate the effectiveness and resiliency of our approach by performing optical probing attacks on protected and unprotected designs on a 28-nm FPGA. Our results show that optical probing attacks can be reliably detected and mitigated without interrupting the chip's operation.

Existing anti-malware software and reverse engineering toolkits struggle with stealthy sub-OS rootkits due to limitations of run-time kernel-level monitoring. A malicious kernel-level driver can bypass OS-level anti-virus mechanisms easily. Although static analysis of such malware is possible, obfuscation and packing techniques complicate offline analysis. Moreover, current dynamic analyzers suffer from virtualization performance overhead and create detectable traces that allow modern malware to evade them. To address these issues, we present \textit{The Reversing Machine} (TRM), a new hypervisor-based memory introspection design for reverse engineering, reconstructing memory offsets, and fingerprinting evasive and obfuscated user-level and kernel-level malware. TRM proposes two novel techniques that enable efficient and transparent analysis of evasive malware: hooking a binary using suspended process creation for hypervisor-based memory introspection, and leveraging Mode-Based Execution Control (MBEC) to detect user/kernel mode transitions and memory access patterns. Unlike existing malware detection environments, TRM can extract full memory traces in user and kernel spaces and hook the entire target memory map to reconstruct arrays, structures within the operating system, and possible rootkits. We perform TRM-assisted reverse engineering of kernel-level structures and show that it can speed up manual reverse engineering by 75\% on average. We obfuscate known malware with the latest packing tools and successfully perform similarity detection. Furthermore, we demonstrate a real-world attack by deploying a modified rootkit onto a driver that bypasses state-of-the-art security auditing tools. We show that TRM can detect each threat and that, out of 24 state-of-the-art AV solutions, only TRM can detect the most advanced threats.

The internal motion of partons inside hadrons has been studied through its impact on very low transverse momentum spectra of Drell-Yan (DY) pairs created in hadron-hadron collisions. We study DY production at next-to-leading order using the Parton Branching (PB) method which describes the evolution of transverse momentum dependent parton distributions. The main focus is on studying the intrinsic transverse momentum distribution (intrinsic-$k_T$) as a function of the center-of-mass energy $\sqrt s$. While collinear parton shower Monte Carlo event generators require intrinsic transverse momentum distributions strongly dependent on $\sqrt s$, this is not the case for the PB method. We perform a detailed study of the impact of soft parton emissions. We show that by requiring a minimal transverse momentum, $q_0$, of a radiated parton, a dependence of the width of the intrinsic-$k_T$ distribution as a function of $\sqrt{s}$ is observed. This dependence becomes stronger with increasing $q_0$., Comment: Reference replaced

In science we are interested in finding the governing equations, the dynamical rules, underlying empirical phenomena. While traditionally scientific models are derived through cycles of human insight and experimentation, recently deep learning (DL) techniques have been advanced to reconstruct dynamical systems (DS) directly from time series data. State-of-the-art dynamical systems reconstruction (DSR) methods show promise in capturing invariant and long-term properties of observed DS, but their ability to generalize to unobserved domains remains an open challenge. Yet, this is a crucial property we would expect from any viable scientific theory. In this work, we provide a formal framework that addresses generalization in DSR. We explain why and how out-of-domain (OOD) generalization (OODG) in DSR profoundly differs from OODG considered elsewhere in machine learning. We introduce mathematical notions based on topological concepts and ergodic theory to formalize the idea of learnability of a DSR model. We formally prove that black-box DL techniques, without adequate structural priors, generally will not be able to learn a generalizing DSR model. We also show this empirically, considering major classes of DSR algorithms proposed so far, and illustrate where and why they fail to generalize across the whole phase space. Our study provides the first comprehensive mathematical treatment of OODG in DSR, and gives a deeper conceptual understanding of where the fundamental problems in OODG lie and how they could possibly be addressed in practice.

Physical side-channel attacks can compromise the security of integrated circuits. Most physical side-channel attacks (e.g., power or electromagnetic) exploit the dynamic behavior of a chip, typically manifesting as changes in current consumption or voltage fluctuations where algorithmic countermeasures, such as masking, can effectively mitigate them. However, as demonstrated recently, these mitigation techniques are not entirely effective against backscattered side-channel attacks such as impedance analysis. In the case of an impedance attack, an adversary exploits the data-dependent impedance variations of the chip power delivery network (PDN) to extract secret information. In this work, we introduce RandOhm, which exploits a moving target defense (MTD) strategy based on the partial reconfiguration (PR) feature of mainstream FPGAs and programmable SoCs to defend against impedance side-channel attacks. We demonstrate that the information leakage through the PDN impedance could be significantly reduced via runtime reconfiguration of the secret-sensitive parts of the circuitry. Hence, by constantly randomizing the placement and routing of the circuit, one can decorrelate the data-dependent computation from the impedance value. Moreover, in contrast to existing PR-based countermeasures, RandOhm deploys open-source bitstream manipulation tools on programmable SoCs to speed up the randomization and provide real-time protection. To validate our claims, we apply RandOhm to AES ciphers realized on 28-nm FPGAs. We analyze the resiliency of our approach by performing non-profiled and profiled impedance analysis attacks and investigate the overhead of our mitigation in terms of delay and performance.

The Parton Branching (PB) method describes the evolution of transverse momentum dependent (TMD) parton distributions, covering all kinematic regions from small to large transverse momenta kT. The small kT-region is very sensitive both to the contribution of the intrinsic motion of partons (intrinsic kT) and to the resummation of soft gluons taken into account by the PB TMD evolution equations. We study the role of soft-gluon emissions in TMD as well as integrated parton distributions. We perform a detailed investigation of the PB TMD methodology at next-to-leading order (NLO) in Drell-Yan (DY) production for low transverse momenta. We present the extraction of the nonperturbative "intrinsic-kT" distribution from recent measurements of DY transverse momentum distributions at the LHC across a wide range in DY masses, including a detailed treatment of statistical, correlated and uncorrelated uncertainties. We comment on the (in)dependence of intrinsic transverse momentum on DY mass and center-of-mass energy, and on the comparison with other approaches., Comment: Replaced with journal version, minor edits

In August 2023, more than 30 students joined the Special Remote DESY summer-school to work on projects of importance for LHC experiments. In a dedicated initiative, analyses that had not been incorporated into the RIVET package were implemented and verified. Here, a brief description of the accomplished work is given, and a comparison of the measurements with predictions obtained from matched standard parton shower Monte Carlo event generators as well as with those obtained from Parton-Branching TMDs with corresponding parton showers are presented.

The Parton-Branching Method (PB) facilitates the determination of Transverse Momentum Dependent (TMD) parton densities across a wide \kt\ range, spanning small to large transverse momentum scales. In the small $k_T$ region, both intrinsic parton motion and resummed ultra-soft gluons are significant contributors. Our analysis highlights their crucial role in shaping integrated and TMD parton densities. Using PB-derived TMD parton densities and a NLO calculation in MC@NLO style, we compute the transverse momentum spectrum of Drell-Yan pairs across a broad mass range. The spectrum's sensitivity to the intrinsic $k_T$ distribution allows us to fine-tune parametric parameters. Starting from the PB-NLO-HERAI+II-2018 set2 TMD parton distributions, we determine the intrinsic $k_T$ distribution width, resulting in a slightly wider profile than the default set. Importantly, this width remains independent of Drell-Yan pair mass and center-of-mass energy ($\sqrt{s}$), distinguishing our approach., Comment: Presented at "The European Physical Society Conference on High Energy Physics (EPS-HEP2023)", 21-25 August 2023, Hamburg, Germany, Submitted to PoS - Proceedings of Science. Updated paper to include a reference published in EPJC

In this study, we present our latest findings regarding azimuthal distributions in vector boson + jets and multi-jet production at the Large Hadron Collider (LHC). These findings result from matching next-to-leading order (NLO) perturbative matrix elements with transverse momentum dependent (TMD) parton branching. We conduct a comprehensive comparative analysis of azimuthal correlations between Z boson-jet and jet-jet systems in the back-to-back region. These distinct azimuthal correlation patterns can help identify potential factorization-breaking effects in this region. Such effects depend on the different color and spin structures of the final states and their interactions with the initial states., Comment: Presented at "The European Physical Society Conference on High Energy Physics (EPS-HEP2023)", 21-25 August 2023, Hamburg, Germany, Submitted to PoS - Proceedings of Science

Neutral current Drell-Yan (DY) lepton-pair production is considered in the framework of the Standard Model Effective Field Theory (SMEFT). Using the open-source fit platform xFitter, we investigate the impact of high-statistics measurements of the neutral current DY (NCDY) forward-backward asymmetry $A_{\rm{FB}}$ near the weak boson mass scale in the present and forthcoming stages of the Large Hadron Collider (LHC). Besides recovering earlier results on the $A_{\rm{FB}}$ sensitivity to parton distribution functions, we analyze the precision determination of $Z$-boson couplings to left-handed and right-handed $u$-quarks and $d$-quarks, and explore Beyond-Standard-Model contributions using the SMEFT framework. We perform a sensitivity study and comment on the role of the $A_{\rm{FB}}$ asymmetry for the electroweak SMEFT fit and precision $Z$-boson physics at the LHC and high-luminosity HL-LHC., Comment: Revised version

Recurrent neural networks (RNNs) are popular machine learning tools for modeling and forecasting sequential data and for inferring dynamical systems (DS) from observed time series. Concepts from DS theory (DST) have variously been used to further our understanding of both, how trained RNNs solve complex tasks, and the training process itself. Bifurcations are particularly important phenomena in DS, including RNNs, that refer to topological (qualitative) changes in a system's dynamical behavior as one or more of its parameters are varied. Knowing the bifurcation structure of an RNN will thus allow to deduce many of its computational and dynamical properties, like its sensitivity to parameter variations or its behavior during training. In particular, bifurcations may account for sudden loss jumps observed in RNN training that could severely impede the training process. Here we first mathematically prove for a particular class of ReLU-based RNNs that certain bifurcations are indeed associated with loss gradients tending toward infinity or zero. We then introduce a novel heuristic algorithm for detecting all fixed points and k-cycles in ReLU-based RNNs and their existence and stability regions, hence bifurcation manifolds in parameter space. In contrast to previous numerical algorithms for finding fixed points and common continuation methods, our algorithm provides exact results and returns fixed points and cycles up to high orders with surprisingly good scaling behavior. We exemplify the algorithm on the analysis of the training process of RNNs, and find that the recently introduced technique of generalized teacher forcing completely avoids certain types of bifurcations in training. Thus, besides facilitating the DST analysis of trained RNNs, our algorithm provides a powerful instrument for analyzing the training process itself.

The role of soft (non-perturbative) gluons in collinear parton densities and parton shower event generators is investigated with the Parton Branching method as a solution of the DGLAP evolution equations. It is found that soft gluons play a significant role. Within the Parton Branching frame, the Sudakov form factor can be split into a perturbative and non-perturbative part. The non-perturbative part can be calculated analytically under certain conditions. It is shown that the inclusion of soft (non-perturbative) gluons in the parton density evolution is essential for the proper cancellation of divergent terms. It is argued that the non-perturbative part of the Sudakov form factor has its correspondence in Transverse Momentum Dependent parton distributions. Within the Parton Branching approach, this non-perturbative Sudakov form factor is constrained by fits of inclusive, collinear parton densities. We show that the non-perturbative Sudakov form factor and soft gluon emissions are essential for inclusive distributions (collinear parton densities and Drell-Yan transverse momentum spectra). We also show by using Parton Branching TMD parton shower, that the effect of soft gluons plays essentially no role in final state hadron spectra and jets., Comment: Text significantly updated

Chaotic dynamical systems (DS) are ubiquitous in nature and society. Often we are interested in reconstructing such systems from observed time series for prediction or mechanistic insight, where by reconstruction we mean learning geometrical and invariant temporal properties of the system in question (like attractors). However, training reconstruction algorithms like recurrent neural networks (RNNs) on such systems by gradient-descent based techniques faces severe challenges. This is mainly due to exploding gradients caused by the exponential divergence of trajectories in chaotic systems. Moreover, for (scientific) interpretability we wish to have as low dimensional reconstructions as possible, preferably in a model which is mathematically tractable. Here we report that a surprisingly simple modification of teacher forcing leads to provably strictly all-time bounded gradients in training on chaotic systems, and, when paired with a simple architectural rearrangement of a tractable RNN design, piecewise-linear RNNs (PLRNNs), allows for faithful reconstruction in spaces of at most the dimensionality of the observed system. We show on several DS that with these amendments we can reconstruct DS better than current SOTA algorithms, in much lower dimensions. Performance differences were particularly compelling on real world data with which most other methods severely struggled. This work thus led to a simple yet powerful DS reconstruction algorithm which is highly interpretable at the same time., Comment: Published in the Proceedings of the 40th International Conference on Machine Learning (ICML 2023)

The threats of physical side-channel attacks and their countermeasures have been widely researched. Most physical side-channel attacks rely on the unavoidable influence of computation or storage on current consumption or voltage drop on a chip. Such data-dependent influence can be exploited by, for instance, power or electromagnetic analysis. In this work, we introduce a novel non-invasive physical side-channel attack, which exploits the data-dependent changes in the impedance of the chip. Our attack relies on the fact that the temporarily stored contents in registers alter the physical characteristics of the circuit, which results in changes in the die's impedance. To sense such impedance variations, we deploy a well-known RF/microwave method called scattering parameter analysis, in which we inject sine wave signals with high frequencies into the system's power distribution network (PDN) and measure the echo of the signals. We demonstrate that according to the content bits and physical location of a register, the reflected signal is modulated differently at various frequency points enabling the simultaneous and independent probing of individual registers. Such side-channel leakage challenges the $t$-probing security model assumption used in masking, which is a prominent side-channel countermeasure. To validate our claims, we mount non-profiled and profiled impedance analysis attacks on hardware implementations of unprotected and high-order masked AES. We show that in the case of the profiled attack, only a single trace is required to recover the secret key. Finally, we discuss how a specific class of hiding countermeasures might be effective against impedance leakage.