Search

The presence of linear paths in parameter space between two different network solutions in certain cases, i.e., linear mode connectivity (LMC), has garnered interest from both theoretical and practical fronts. There has been significant research that either practically designs algorithms catered for connecting networks by adjusting for the permutation symmetries as well as some others that more theoretically construct paths through which networks can be connected. Yet, the core reasons for the occurrence of LMC, when in fact it does occur, in the highly non-convex loss landscapes of neural networks are far from clear. In this work, we take a step towards understanding it by providing a model of how the loss landscape needs to behave topographically for LMC (or the lack thereof) to manifest. Concretely, we present a `mountainside and ridge' perspective that helps to neatly tie together different geometric features that can be spotted in the loss landscape along the training runs. We also complement this perspective by providing a theoretical analysis of the barrier height, for which we provide empirical support, and which additionally extends as a faithful predictor of layer-wise LMC. We close with a toy example that provides further intuition on how barriers arise in the first place, all in all, showcasing the larger aim of the work -- to provide a working model of the landscape and its topography for the occurrence of LMC., Comment: ICML 2024 HiLD workshop paper

The learning process of a reinforcement learning (RL) agent remains poorly understood beyond the mathematical formulation of its learning algorithm. To address this gap, we introduce attention-oriented metrics (ATOMs) to investigate the development of an RL agent's attention during training. We tested ATOMs on three variations of a Pong game, each designed to teach the agent distinct behaviours, complemented by a behavioural assessment. Our findings reveal that ATOMs successfully delineate the attention patterns of an agent trained on each game variation, and that these differences in attention patterns translate into differences in the agent's behaviour. Through continuous monitoring of ATOMs during training, we observed that the agent's attention developed in phases, and that these phases were consistent across games. Finally, we noted that the agent's attention to its paddle emerged relatively late in the training and coincided with a marked increase in its performance score. Overall, we believe that ATOMs could significantly enhance our understanding of RL agents' learning processes, which is essential for improving their reliability and efficiency.

The electron dynamics in the unoccupied states of the Weyl semimetal PtBi$_2$ is studied by time- and angle-resolved photoemission spectroscopy (TR-ARPES). The measurement's result is the photoemission intensity $I$ as a function of at least four parameters: the emission angle and kinetic energy of the photoelectrons, the time delay between pump and probe laser pulses, and the probe laser photon energy that needs to be varied to access the full three-dimensional Brillouin zone of the material. The TR-ARPES results are reported in an accompanying paper. Here we focus on the technique of using $k$-means, an unsupervised machine learning technique, in order to discover trends in the four-dimensional data sets. We study how to compare the electron dynamics across the entire data set and how to reveal subtle variations between different data sets collected in the vicinity of the bulk Weyl points.

Using time- and angle-resolved photoemission spectroscopy, we examine the unoccupied electronic structure and electron dynamics of the type-I Weyl semimetal PtBi$_2$. Using the ability to change the probe photon energy over a wide range, we identify the predicted Weyl points in the unoccupied three-dimensional band structure and we discuss the effect of $k_\perp$ broadening in the normally unoccupied states. We characterise the electron dynamics close to the Weyl point and in other parts of three-dimensional Brillouin zone using $k$-means, an unsupervised machine learning technique. This reveals distinct differences -- in particular, dynamics that are faster in the parts of the Brillouin zone that host most of the bulk Fermi surface than in parts close to the Weyl points.

We study experimentally the manifestation of non-Weyl graph behavior in open systems using microwave networks. For this a coupling variation to the network is necessary, which was out of reach till now. The coupling to the environment is changed by indirectly varying the boundary condition at the coupling vertex from Dirichlet to Neumann using a dangling bond with variable length attached the coupling vertex. A transformation of equal length spectra to equal reflection phase spectra of the dangling bond allows to create spectra with different fixed coupling strength. This allows to follow the resonances in the complex plane as a function of the coupling. While going from closed (Dirichlet) to fully open (Neumann) graph we see resonances escaping via a superradiant transition leading to non-Weyl behavior if the coupling to the outside is balanced. The open tetrahedral graph displays a rich parametric dynamic of the resonances in the complex plane presenting loops, regions of connected resonances and resonances approaching infinite imaginary parts., Comment: 6 pages, 3 figures

Language Modelling has been a central part of Natural Language Processing for a very long time and in the past few years LSTM-based language models have been the go-to method for commercial language modeling. Recently, it has been shown that when looking at language modelling from a matrix factorization point of view, the final Softmax layer limits the expressiveness of the model, by putting an upper bound on the rank of the resulting matrix. Additionally, a new family of neural networks based called NeuralODEs, has been introduced as a continuous alternative to Residual Networks. Moreover, it has been shown that there is a connection between these models and Normalizing Flows. In this work we propose a new family of language models based on NeuralODEs and the continuous analogue of Normalizing Flows and manage to improve on some of the baselines., Comment: Master's thesis

PLATO (PLAnetary Transits and Oscillations of stars) is ESA's M3 mission designed to detect and characterise extrasolar planets and perform asteroseismic monitoring of a large number of stars. PLATO will detect small planets (down to <2 R_(Earth)) around bright stars (<11 mag), including terrestrial planets in the habitable zone of solar-like stars. With the complement of radial velocity observations from the ground, planets will be characterised for their radius, mass, and age with high accuracy (5 %, 10 %, 10 % for an Earth-Sun combination respectively). PLATO will provide us with a large-scale catalogue of well-characterised small planets up to intermediate orbital periods, relevant for a meaningful comparison to planet formation theories and to better understand planet evolution. It will make possible comparative exoplanetology to place our Solar System planets in a broader context. In parallel, PLATO will study (host) stars using asteroseismology, allowing us to determine the stellar properties with high accuracy, substantially enhancing our knowledge of stellar structure and evolution. The payload instrument consists of 26 cameras with 12cm aperture each. For at least four years, the mission will perform high-precision photometric measurements. Here we review the science objectives, present PLATO's target samples and fields, provide an overview of expected core science performance as well as a description of the instrument and the mission profile at the beginning of the serial production of the flight cameras. PLATO is scheduled for a launch date end 2026. This overview therefore provides a summary of the mission to the community in preparation of the upcoming operational phases.

Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work defines memorisation as the causal effect of training with an instance on the model's ability to predict that instance. This definition relies on a counterfactual: the ability to observe what would have happened had the model not seen that instance. Existing methods struggle to provide computationally efficient and accurate estimates of this counterfactual. Further, they often estimate memorisation for a model architecture rather than for a specific model instance. This paper fills an important gap in the literature, proposing a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics. Using this method, we characterise a model's memorisation profile--its memorisation trends across training--by only observing its behaviour on a small set of instances throughout training. In experiments with the Pythia model suite, we find that memorisation (i) is stronger and more persistent in larger models, (ii) is determined by data order and learning rate, and (iii) has stable trends across model sizes, thus making memorisation in larger models predictable from smaller ones., Comment: Published at the ACL 2024 Conference (main)

If we prepare an isolated, interacting quantum system in an eigenstate and perturb a local observable at an initial time, its expectation value will relax towards a thermal expectation value, even though the time evolution of the system is deterministic. The eigenstate thermalization hypothesis (ETH) of Deutsch and Srednicki suggests that this is possible because each eigenstate of the full quantum system acts as a thermal bath to its subsystems, such that the reduced density matrices of the subsystems resemble thermal density matrices. Here, we use the observation that the eigenvalue distribution of interacting quantum systems is a Gaussian under very general circumstances, and Dyson Brownian motion random matrix theory, to derive the ETH and thereby elevate it from hypothesis to theory. Our analysis provides a derivation of statistical mechanics which neither requires the concepts of ergodicity or typicality, nor that of entropy. Thermodynamic equilibrium follows solely from the applicability of quantum mechanics to large systems and the absence of integrability., Comment: 27 pages, 19 figures

Quantum contextuality describes situations where the statistics observed in different measurement contexts cannot be explained by a measurement independent reality of the system. The most simple case is observed in a three-dimensional Hilbert space, with five different measurement contexts related to each other by shared measurement outcomes. The quantum formalism defines the relations between these contexts in terms of well-defined relations between operators, and these relations can be used to reconstruct an unknown quantum state from a finite set of measurement results. Here, I introduce a reconstruction method based on the relations between the five measurement contexts that can violate the bounds of non-contextual statistics. A complete description of an arbitrary quantum state requires only five of the eight elements of a Kirkwood-Dirac quasi probability, but only an overcomplete set of eleven elements provides an unbiased description of all five contexts. A set of five fundamental relations between the eleven elements reveals a deterministic structure that links the five contexts. As illustrated by a number of examples, these relations provide a consistent description of contextual realities for the measurement outcomes of all five contexts., Comment: 19 pages, including 2 figures and 3 tables. Version 2 has additional background and references

Outlier Features (OFs) are neurons whose activation magnitudes significantly exceed the average over a neural network's (NN) width. They are well known to emerge during standard transformer training and have the undesirable effect of hindering quantisation in afflicted models. Despite their practical importance, little is known behind why OFs emerge during training, nor how one can minimise them. Our work focuses on the above questions, first identifying several quantitative metrics, such as the kurtosis over neuron activation norms, to measure OFs. With these metrics, we study how architectural and optimisation choices influence OFs, and provide practical insights to minimise OFs during training. As highlights, we introduce a novel unnormalised transformer block, the Outlier Protected block, and present a previously unknown benefit of non-diagonal preconditioning optimisers, finding both approaches to significantly reduce OFs and improve quantisation without compromising convergence speed, at scales of up to 7B parameters. Notably, our combination of OP block and non-diagonal preconditioner (SOAP) achieves 14.87 int8 weight-and-activation perplexity (from 14.71 in standard precision), compared to 63.4 int8 perplexity (from 16.00) with a default OF-prone combination of Pre-Norm model and Adam, when quantising OPT-125m models post-training. Overall, our findings shed new light on our understanding of, our ability to prevent, and the complexity of this important aspect of NN training dynamics., Comment: NeurIPS 2024 camera ready

Jupiter's moon Io hosts extensive volcanism driven by tidal heating. The isotopic composition of Io's inventory of volatile elements, including sulfur and chlorine, reflects its outgassing and mass loss history and provides an avenue for exploring its evolution. We used millimeter observations of Io's atmosphere to measure sulfur isotopes in gaseous SO2 and SO, and chlorine isotopes in gaseous NaCl and KCl. We find $^{34}$S/$^{32}$S=0.0595$\pm$0.0038 ($\delta^{34}$S=+347$\pm$86 per mille), which is highly enriched compared to average Solar System values and indicates that Io has lost 94 to 99% of its available sulfur. Our measurement of $^{37}$Cl/$^{35}$Cl=0.403$\pm$0.028 ($\delta^{37}$Cl=+263$\pm$88 per mille) shows chlorine is similarly enriched. These measurements indicate that Io has been volcanically active for most or all of its history, with potentially higher outgassing and mass-loss rates at earlier times., Comment: This is the author's version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science on May 10, 2024, DOI: 10.1126/science.adj0625

Superlattices from twisted graphene mono- and bi-layer systems give rise to on-demand many-body states such as Mott insulators and unconventional superconductors. These phenomena are ascribed to a combination of flat bands and strong Coulomb interactions. However, a comprehensive understanding is lacking because the low-energy band structure strongly changes when the electron filling is varied. Here, we gain direct access to the filling-dependent low energy bands of twisted bilayer graphene (TBG) and twisted double bilayer graphene (TDBG) by applying micro-focused angle-resolved photoemission spectroscopy to in situ gated devices. Our findings for the two systems are in stark contrast: The doping dependent dispersion for TBG can be described in a simple model, combining a filling-dependent rigid band shift with a many-body related bandwidth change. In TDBG, on the other hand, we find a complex behaviour of the low-energy bands, combining non-monotonous bandwidth changes and tuneable gap openings. Our work establishes the extent of electric field tunability of the low energy electronic states in twisted graphene superlattices and can serve to underpin the theoretical understanding of the resulting phenomena., Comment: 26 pages, 4 main figures and 7 supplementary figures

A search for full energy depositions from bosonic keV-scale dark matter candidates of masses between 65 keV and 1021 keV has been performed with data collected during Phase II of the GERmanium Detector Array (GERDA) experiment. Our analysis includes direct dark matter absorption as well as dark Compton scattering. With a total exposure of 105.5 kg yr, no evidence for a signal above the background has been observed. The resulting exclusion limits deduced with either Bayesian or Frequentist statistics are the most stringent direct constraints in the major part of the 140-1021 keV mass range. As an example, at a mass of 150 keV the dimensionless coupling of dark photons and axion-like particles to electrons has been constrained to $\alpha$'/$\alpha$ < 8.7x10$^{-24}$ and g$_{ae}$ < 3.3x10$^{-12}$ at 90% credible interval (CI), respectively. Additionally, a search for peak-like signals from beyond the Standard Model decays of nucleons and electrons is performed. We find for the inclusive decay of a single neutron in $^{76}$Ge a lower lifetime limit of $\tau_n$ > 1.5x10$^{24}$ yr and for a proton $\tau_p$ > 1.3x10$^{24}$ yr at 90% CI. For the electron decay e$^-\rightarrow\nu_e\gamma$ a lower limit of $\tau_e$ > 5.4x10$^{25}$ yr at 90% CI has been determined., Comment: 20 pages, 12 figures, 7 tables

We propose a reproducible pipeline for extracting representative signals from 2D topographic scans of the tips of cut wires. The process fully addresses many potential problems in the quality of wire cuts, including edge effects, extreme values, trends, missing values, angles, and warping. The resulting signals can be further used in source determination, which plays an important role in forensic examinations. With commonly used measurements such as the cross-correlation function, the procedure controls the false positive rate and false negative rate to the desirable values as the manual extraction pipeline but outperforms it with robustness and objectiveness., Comment: 6 pages, 12 figures, awarded the Student Paper Award in the Statistical Computing and Statistical Graphics Sections of the American Statistical Association (ASA) in 2024

Recent studies have focused on the relationship between charge density wave (CDW) collective electronic ground states and nontrivial topological states. Using angle-resolved photoemission and density functional theory, we establish that YTe$_3$ is a CDW-induced Kramers nodal line (KNL) metal, a newly proposed topological state of matter. YTe$_3$ is a non-magnetic quasi-2D chalcogenide with a CDW wave vector ($q_{\rm cdw}$) of 0.2907c$^*$. Scanning tunneling microscopy and low energy electron diffraction revealed two orthogonal CDW domains, each with a unidirectional CDW and similar YTe$_3$. The effective band structure (EBS) computations, using DFT-calculated folded bands, show excellent agreement with ARPES because a realistic x-ray crystal structure and twin domains are considered in the calculations. The Fermi surface and ARPES intensity plots show weak shadow bands displaced by $q_{\rm cdw}$ from the main bands. These are linked to CDW modulation, as the EBS calculation confirms. Bilayer split main and shadow bands suggest the existence of crossings, according to theory and experiment. DFT bands, including spin-orbit coupling, indicate a nodal line along the $\Sigma$ line from multiple band crossings perpendicular to the KNL. Additionally, doubly degenerate bands are only found along the KNL at all energies, with some bands dispersing through the Fermi level.

Out-of-distribution (OOD) detection is critical when deploying machine learning models in the real world. Outlier exposure methods, which incorporate auxiliary outlier data in the training process, can drastically improve OOD detection performance compared to approaches without advanced training strategies. We introduce Hopfield Boosting, a boosting approach, which leverages modern Hopfield energy (MHE) to sharpen the decision boundary between the in-distribution and OOD data. Hopfield Boosting encourages the model to concentrate on hard-to-distinguish auxiliary outlier examples that lie close to the decision boundary between in-distribution and auxiliary outlier data. Our method achieves a new state-of-the-art in OOD detection with outlier exposure, improving the FPR95 metric from 2.28 to 0.92 on CIFAR-10 and from 11.76 to 7.94 on CIFAR-100.

Since the advent of Deepfakes in digital media, the development of robust and reliable detection mechanism is urgently called for. In this study, we explore a novel approach to Deepfake detection by utilizing electroencephalography (EEG) measured from the neural processing of a human participant who viewed and categorized Deepfake stimuli from the FaceForensics++ datset. These measurements serve as input features to a binary support vector classifier, trained to discriminate between real and manipulated facial images. We examine whether EEG data can inform Deepfake detection and also if it can provide a generalized representation capable of identifying Deepfakes beyond the training domain. Our preliminary results indicate that human neural processing signals can be successfully integrated into Deepfake detection frameworks and hint at the potential for a generalized neural representation of artifacts in computer generated faces. Moreover, our study provides next steps towards the understanding of how digital realism is embedded in the human cognitive system, possibly enabling the development of more realistic digital avatars in the future.

A great deal of interest is directed nowadays towards the development of innovative technologies in the field of quantum information and quantum computing, with emphasis on obtaining reliable qubits as building blocks. The realization of highly stable, controllable and accessible hole spin qubits is strongly dependent on the quality of the materials hosting them. Ultra-clean germanium/silicon-germanium heterostructures have been predicted and proven to be promising candidates and due to their large scalability potential, they are opening the door towards the development of realistic and reliable solid state, all-electric, silicon-based quantum computers. In order to obtain ultra-clean germanium/silicon-germanium heterostructures in a reverse grading approach, the understanding and control over the growth of Ge virtual substrates and thin films is key. Here, we present a detailed study on the growth kinetics, morphology, and crystal quality of Ge thin films grown via chemical vapor deposition by investigating the effects of growth temperature, partial pressure of the precursor gas and the use of Ar or H2 atmosphere. The presence of carrier gases catalyzes the deposition rate and induces a smoothening on the surfaces of films grown at low temperatures. We investigated the surface roughness and threading dislocation density as a function of deposition temperature, partial pressure and gas mixture. Ge thin films deposited by diluting GeH4 in Ar or H2 were employed as virtual substrates for the growth of full Ge/SiGe QW heterostructures. Their defect density was analyzed and their electric transport properties were characterized via Hall measurements. Similar results were obtained for both carrier gases used.

Oxidation of two-dimensional (2D) layered materials has proven advantageous in creating oxide/2D material heterostructures, opening the door for a new paradigm of low-power electronic devices. Gallium (II) sulfide ($\beta$-GaS), a hexagonal phase group III monochalcogenide, is a wide bandgap semiconductor with a bandgap exceeding 3 eV in single and few layer form. Its oxide, gallium oxide (Ga$_2$O$_3$), combines large bandgap (4.4-5.3 eV) with high dielectric constant (~10). Despite the technological potential of both materials, controlled oxidation of atomically-thin $\beta$-GaS remains under-explored. This study focuses into the controlled oxidation of $\beta$-GaS using oxygen plasma treatment, achieving ultrathin native oxide (GaS$_x$O$_y$, ~4 nm) and GaS$_x$O$_y$/GaS heterostructures where the GaS layer beneath remains intact. By integrating such structures between metal electrodes and applying electric stresses as voltage ramps or pulses, we investigate their use for resistive random-access memory (ReRAM). The ultrathin nature of the produced oxide enables low operation power with energy use as low as 0.22 nJ per operation while maintaining endurance and retention of 350 cycles and 10$^4$ s, respectively. These results show the significant potential of the oxidation-based GaS$_x$O$_y$/GaS heterostructure for electronic applications and, in particular, low-power memory devices.

Pauli blocking in Fermi liquids imposes strong phase-space constraints on quasiparticle lifetimes, leading to a well-known quadratic-in-temperature decay rate of quasiparticle modes at low temperatures. In two-dimensional systems, however, even longer-lived modes are predicted (dubbed "odd-parity" modes). Here, we present an efficient method to evaluate the full spectrum of relaxational eigenmodes of a Fermi liquid within kinetic theory. We employ this method to study the experimentally relevant case of a Fermi liquid with screened Coulomb interactions and map out the decay rates of quasiparticle modes beyond the asymptotic low temperature limit up to the Fermi temperature, thus covering the temperature range of typical experiments. We confirm the existence of anomalously long-lived odd-parity modes and provide a comprehensive classification and detailed analysis of the relaxation spectrum. In particular, we find that (i) the odd-parity effect in the decay rates extends to temperatures as large as $T=0.1T_F$; (ii) there is only a small number of long-lived odd-parity modes, with an infinite number of remaining modes that show standard Fermi-liquid scaling; (iii) the ratio between the odd- and even-parity lifetimes is tunable with the Coulomb interaction strength, not just temperature, which reflects a difference in the microscopic relaxation mechanism of the modes. Our findings provide a comprehensive description of the nonequilibrium relaxation behavior of two-dimensional electron gases and bridge a significant gap in our understanding of these systems., Comment: 16 pages, 10 figures

Intercalation of epitaxial graphene on SiC(0001) with Sn results in a well-ordered 2D metallic Sn phase with a $(1\times1)$ structure at the interface between SiC substrate and quasi-freestanding graphene. The 2D\,Sn phase exhibits exotic electronic properties with Dirac-like and flat bands coexisting close to the Fermi level that exhibit both Zeeman- and Rashba-type spin splittings. Possible inter-layer interactions between the 2D\,Sn layer and graphene that may result in emerging electronic properties remain unexplored. We use time- and angle-resolved photoemission spectroscopy to reveal a surprisingly short-lived non-equilibrium carrier distribution inside the Dirac cone of graphene. Further, we find that the graphene $\pi$-band exhibits a transient down-shift that we attribute to charging of the graphene layer with holes. We interpret our results with support from density functional theory calculations of the graphene - 2D\,Sn heterostructure that reveal a substantial hybridization between graphene $\pi$-band and Sn $p_z$-states that opens up a $\sim230$\,meV band gap inside the Dirac cone and delocalizes the charge carriers over both the graphene and 2D\,Sn layers. Our results have important implications for the design of future ultrafast optoelectronic devices that may find applications in the fields of light harvesting and detection, as supercapacitors, or in novel quantum computing technologies., Comment: 19 pages, 5 figures

Understanding and controlling the electronic properties of two-dimensional materials is crucial for their potential applications in nano- and optoelectronics. Monolayer transition metal dichalcogenides such as WS$_2$ have garnered significant interest due to their strong light-matter interaction and extreme sensitivity of the band structure to the presence of photogenerated electron-hole pairs. In this study, we investigate the transient electronic structure of monolayer WS$_2$ on a graphene substrate after resonant excitation of the A-exciton using time- and angle-resolved photoemission spectroscopy. We observe a pronounced band structure renormalization including a substantial reduction of the transient band gap that is in good quantitative agreement with our {\it ab initio} theory that reveals the importance of both intrinsic WS$_2$ and extrinsic substrate contributions to the transient band structure of monolayer WS$_2$. Our findings not only deepen the fundamental understanding of band structure dynamics in two-dimensional materials but also offer valuable insights for the development of novel electronic and optoelectronic devices based on monolayer TMDs and their heterostructures with graphene., Comment: 17 pages, 4 figures

The presence of an absorber in one of the paths of an interferometer changes the output statistics of that interferometer in a fundamental manner. Since the individual quantum particles detected at any of the outputs of the interferometer have not been absorbed, any non-trivial effect of the absorber on the distribution of these particles over these paths is a counterfactual effect. Here, we quantify counterfactual effects by evaluating the information about the presence or absence of the absorber obtained from the output statistics, distinguishing between classical and quantum counterfactual effects. We identify the counterfactual gain which quantifies the advantage of quantum counterfactual protocols over classical counterfactual protocols, and show that this counterfactual gain can be separated into two terms: a semi-classical term related to the amplitude blocked by the absorber, and a Kirkwood-Dirac quasiprobability assigning a joint probability to the blocked path and the output port. A negative Kirkwood-Dirac term between a path and an output port indicates that inserting the absorber into that path will have a focussing effect, increasing the probability of particles arriving at that output port, resulting in a significant enhancement of the counterfactual gain. We show that the magnitude of quantum counterfactual effects cannot be explained by a simple removal of the absorbed particles, but originates instead from a well-defined back-action effect caused by the presence of the absorber in one path, on particles in other paths., Comment: 13 pages, 3 figures. Accepted for publication in Quantum Science and Technology; matches accepted version

In reaction-diffusion models of annihilation reactions in low dimensions, single-particle dynamics provides a bottleneck for reactions, leading to an anomalously slow approach to the empty state. Here, we construct a reaction model with a reciprocal bottleneck on particle dynamics where single-particle motion conserves the center of mass. We show that such a constrained reaction-diffusion dynamics does not approach an empty state but freezes at late times in a state with fragmented particle clusters, and that the late-time dynamics and asymptotic density are universal. Our setup thus constitutes a minimal model for the fragmentation of a one-dimensional lattice into independent particle clusters, and we provide exact results for the final density in the large-reaction rate limit. We suggest that the universal reaction dynamics could be observable in experiments with cold atoms or in the Auger recombination of exciton gases., Comment: 4 pages, 4 figures. Updated to agree with journal version, minor changes

Current diffusion models create photorealistic images given a text prompt as input but struggle to correctly bind attributes mentioned in the text to the right objects in the image. This is evidenced by our novel image-graph alignment model called EPViT (Edge Prediction Vision Transformer) for the evaluation of image-text alignment. To alleviate the above problem, we propose focused cross-attention (FCA) that controls the visual attention maps by syntactic constraints found in the input sentence. Additionally, the syntax structure of the prompt helps to disentangle the multimodal CLIP embeddings that are commonly used in T2I generation. The resulting DisCLIP embeddings and FCA are easily integrated in state-of-the-art diffusion models without additional training of these models. We show substantial improvements in T2I generation and especially its attribute-object binding on several datasets.\footnote{Code and data will be made available upon acceptance.

In this publication we introduce MAM-STM, a software to autonomously manipulate arbitrary moieties towards specific positions on a metal surface utilizing the tip of a scanning tunneling microscope (STM). Finding the optimal manipulation parameters for a specific moiety is challenging and time consuming, even for human experts. MAM-STM combines autonomous data acquisition with a sophisticated Q-learning implementation to determine the optimal bias voltage, the z-approach distance, and the tip position relative to the moiety. This then allows to arrange single molecules and atoms at will. In this work, we provide a tutorial based on a simulated response to offer a comprehensive explanation on how to use and customize MAM-STM. Additionally, we assess the performance of the machine learning algorithm by benchmarking it within a simulated stochastic environment.

Building on the notion of normed category as suggested by Lawvere, we introduce notions of Cauchy convergence and cocompleteness which differ from proposals in previous works. Key to our approach is to treat them consequentially as categories enriched in the monoidal-closed category of normed sets. Our notions largely lead to the anticipated outcomes when considering individual metric spaces as small normed categories, but they can be challenging when considering some large categories, like those of semi-normed or normed vector spaces and all linear maps, or of generalized metric spaces and all mappings. These are the key example categories discussed in detail in this paper. Working with a general commutative quantale V as a value recipient for norms, rather than only with Lawvere's quantale of the extended real half-line, we observe that the categorically atypical structure gap between objects and morphisms in the example categories is already present in the underlying normed category of the enriching category of V-normed sets. To show that this normed category and, in fact, all presheaf categories over it, are Cauchy cocomplete, we assume the quantale V to satisfy a couple of light alternative extra properties. Of utmost importance to the general theory is the fact that our notion of normed colimit is subsumed by the notion of weighted colimit of enriched category theory. With this theory we are able to prove that all V-normed categories have correct-size Cauchy cocompletions. We also prove a Banach Fixed Point Theorem for contractive endofunctors of Cauchy cocomplete normed categories., Comment: 50 pages

Multilinguality is crucial for extending recent advancements in language modelling to diverse linguistic communities. To maintain high performance while representing multiple languages, multilingual models ideally align representations, allowing what is learned in one language to generalise to others. Prior research has emphasised the importance of parallel data and shared vocabulary elements as key factors for such alignment. In this study, we investigate an unintuitive novel driver of cross-lingual generalisation: language imbalance. In controlled experiments on perfectly equivalent cloned languages, we observe that the existence of a predominant language during training boosts the performance of less frequent languages and leads to stronger alignment of model representations across languages. Furthermore, we find that this trend is amplified with scale: with large enough models or long enough training, we observe that bilingual training data with a 90/10 language split yields better performance on both languages than a balanced 50/50 split. Building on these insights, we design training schemes that can improve performance in all cloned languages, even without altering the training data. As we extend our analysis to real languages, we find that infrequent languages still benefit from frequent ones, yet whether language imbalance causes cross-lingual generalisation there is not conclusive.

We study poly-crystalline spherical drops of an aqueous suspension of highly charged colloidal spheres exposed to a colloid-free aqueous environment. Crystal contours were obtained from standard optical imaging. The crystal spheres first expand to nearly four times their initial volume before slowly shrinking due to dilution-induced melting. Exploiting coherent multiple-scattering by (110) Bragg reflecting crystals, time-dependent density profiles were recorded deep within the drop interior. These show a continuously flattening radial density gradient and a decreasing central density. Expansion curves and density profiles are qualitatively consistent with theoretical expectations based on dynamical density functional theory for the expansion of a spherical crystallite made of charged Brownian spheres. We anticipate that our study opens novel experimental access to densi-ty determination in turbid crystals., Comment: Thoroughly revised version. now 23 pages with 9 figures and 57 references; supplementary materials 15 pages with 11 figures and 22 references. Accepted for publication in Soft Matter in August 2024

We give a brief introduction to computational algebraic number theory in OSCAR. Our main focus is on number fields, rings of integers and their invariants. After recalling some classical results and their constructive counterparts, we showcase the functionality in two examples related to the investigation of the Cohen-Lenstra heuristic for quadratic fields and the Galois module structure of rings of integers., Comment: Submitted as chapter for the upcoming book on the computer algebra system OSCAR

Tokenisation is a core part of language models (LMs). It involves splitting a character sequence into subwords which are assigned arbitrary indices before being served to the LM. While typically lossless, however, this process may lead to less sample efficient LM training: as it removes character-level information, it could make it harder for LMs to generalise across similar subwords, such as now and Now. We refer to such subwords as near duplicates. In this paper, we study the impact of near duplicate subwords on LM training efficiency. First, we design an experiment that gives us an upper bound to how much we should expect a model to improve if we could perfectly generalise across near duplicates. We do this by duplicating each subword in our LM's vocabulary, creating perfectly equivalent classes of subwords. Experimentally, we find that LMs need roughly 17% more data when trained in a fully duplicated setting. Second, we investigate the impact of naturally occurring near duplicates on LMs. Here, we see that merging them considerably hurts LM performance. Therefore, although subword duplication negatively impacts LM training efficiency, naturally occurring near duplicates may not be as similar as anticipated, limiting the potential for performance improvements.

The current dropout rate in physics studies in Germany is about 60\%, with the majority of dropouts occurring in the first year. Consequently, the physics study entry phase poses a significant challenge for many students. Students' stress perceptions can provide more profound insights into the processes and challenges during that period. In a panel study featuring 67 measuring points involving up to 128 participants at each point, we investigated students' stress perceptions with the Perceived Stress Questionnaire (PSQ), identified underlying sources of stress, and assessed self-estimated workloads across two different cohorts. This examination occurred almost every week during the first semester, and for one cohort also in the second semester, yielding a total of 3,241 PSQ data points and 5,823 stressors. The PSQ data indicate a consistent stress trajectory across all three groups studied that is characterized by significant dynamics between measuring points, spanning from $M=20.1, SD=15.9$ to $M=63.6, SD=13.4$ on a scale from 0 to 100. Stress levels rise in the first weeks of the lecture, followed by stable, elevated stress levels until the exams and a relaxation phase afterward during the lecture-free time and Christmas vacation. In the first half of the lecture period, students primarily indicated the weekly exercise sheets, the physics lab course, and math courses as stressors; later on, preparation for exams and the exams themselves emerged as the most important stressors. Together with the students' self-estimated workloads that correlate with the PSQ scores, we can create a coherent picture of stress perceptions among first-year physics students, which builds the basis for supportive measures and interventions., Comment: 24 pages, 15 figures, 7 tables, revision after second round of peer review for Phys. Rev. Phys. Educ. Res., the manuscript has been streamlined, further minor improvements

It is shown that the duals of several categories of topological flavour, like the categories of ordered sets, generalised metric spaces, probabilistic metric spaces, topological spaces, approach spaces, are quasivarieties, presenting a common proof for all such results.

General policies represent reactive strategies for solving large families of planning problems like the infinite collection of solvable instances from a given domain. Methods for learning such policies from a collection of small training instances have been developed successfully for classical domains. In this work, we extend the formulations and the resulting combinatorial methods for learning general policies over fully observable, non-deterministic (FOND) domains. We also evaluate the resulting approach experimentally over a number of benchmark domains in FOND planning, present the general policies that result in some of these domains, and prove their correctness. The method for learning general policies for FOND planning can actually be seen as an alternative FOND planning method that searches for solutions, not in the given state space but in an abstract space defined by features that must be learned as well., Comment: presented at IJCAI'24

Here we examine whether the personality dimension of openness to experience can be predicted from the individual google search history. By web scraping, individual text corpora (ICs) were generated from 214 participants with a mean number of 5 million word tokens. We trained word2vec models and used the similarities of each IC to label words, which were derived from a lexical approach of personality. These IC-label-word similarities were utilized as predictive features in neural models. For training and validation, we relied on 179 participants and held out a test sample of 35 participants. A grid search with varying number of predictive features, hidden units and boost factor was performed. As model selection criterion, we used R2 in the validation samples penalized by the absolute R2 difference between training and validation. The selected neural model explained 35% of the openness variance in the test sample, while an ensemble model with the same architecture often provided slightly more stable predictions for intellectual interests, knowledge in humanities and level of education. Finally, a learning curve analysis suggested that around 500 training participants are required for generalizable predictions. We discuss ICs as a complement or replacement of survey-based psychodiagnostics., Comment: Proceedings of the 8th workshop on Cognitive Aspects of the Lexicon (CogALex-VIII), LREC/Coling 2024

HESS J1813$-$178 is a very-high-energy $\gamma$-ray source spatially coincident with the young and energetic pulsar PSR J1813$-$1749 and thought to be associated with its pulsar wind nebula (PWN). Recently, evidence for extended high-energy emission in the vicinity of the pulsar has been revealed in the Fermi Large Area Telescope (LAT) data. This motivates revisiting the HESS J1813$-$178 region, taking advantage of improved analysis methods and an extended data set. Using data taken by the High Energy Stereoscopic System (H.E.S.S.) experiment and the Fermi-LAT, we aim to describe the $\gamma$-ray emission in the region with a consistent model, to provide insights into its origin. We performed a likelihood-based analysis on 32 hours of H.E.S.S. data and 12 years of Fermi-LAT data and fit a spectro-morphological model to the combined datasets. These results allowed us to develop a physical model for the origin of the observed $\gamma$-ray emission in the region. In addition to the compact very-high-energy $\gamma$-ray emission centered on the pulsar, we find a significant yet previously undetected component along the Galactic plane. With Fermi-LAT data, we confirm extended high-energy emission consistent with the position and elongation of the extended emission observed with H.E.S.S. These results establish a consistent description of the emission in the region from GeV energies to several tens of TeV. This study suggests that HESS J1813$-$178 is associated with a $\gamma$-ray PWN powered by PSR J1813$-$1749. A possible origin of the extended emission component is inverse Compton emission from electrons and positrons that have escaped the confines of the pulsar and form a halo around the PWN., Comment: 13+5 pages, 13+11 figures. Accepted for publication in A&A. Corresponding authors: T.Wach, A.Mitchell, V.Joshi, P.Chamb\'ery

The Crab Nebula is a unique laboratory for studying the acceleration of electrons and positrons through their non-thermal radiation. Observations of very-high-energy $\gamma$ rays from the Crab Nebula have provided important constraints for modelling its broadband emission. We present the first fully self-consistent analysis of the Crab Nebula's $\gamma$-ray emission between 1 GeV and $\sim$100 TeV, that is, over five orders of magnitude in energy. Using the open-source software package Gammapy, we combined 11.4 yr of data from the Fermi Large Area Telescope and 80 h of High Energy Stereoscopic System (H.E.S.S.) data at the event level and provide a measurement of the spatial extension of the nebula and its energy spectrum. We find evidence for a shrinking of the nebula with increasing $\gamma$-ray energy. Furthermore, we fitted several phenomenological models to the measured data, finding that none of them can fully describe the spatial extension and the spectral energy distribution at the same time. Especially the extension measured at TeV energies appears too large when compared to the X-ray emission. Our measurements probe the structure of the magnetic field between the pulsar wind termination shock and the dust torus, and we conclude that the magnetic field strength decreases with increasing distance from the pulsar. We complement our study with a careful assessment of systematic uncertainties., Comment: 18+6 pages, 15+2 figures. Accepted for publication in A&A. Corresponding authors: M. Meyer, L. Mohrmann, T. Unbehaun. v2: after A&A language editing

Materials and methods: First, a dual-time approach was assessed, for which the CNN was provided sequences of the MRI that initially depicted new MM (diagnosis MRI) as well as of a prediagnosis MRI: inclusion of only contrast-enhanced T1-weighted images (CNNdual_ce) was compared with inclusion of also the native T1-weighted images, T2-weighted images, and FLAIR sequences of both time points (CNNdual_all).Second, results were compared with the corresponding single time approaches, in which the CNN was provided exclusively the respective sequences of the diagnosis MRI.Casewise diagnostic performance parameters were calculated from 5-fold cross-validation. Results: In total, 94 cases with 494 MMs were included. Overall, the highest diagnostic performance was achieved by inclusion of only the contrast-enhanced T1-weighted images of the diagnosis and of a prediagnosis MRI (CNNdual_ce, sensitivity = 73%, PPV = 25%, F1-score = 36%). Using exclusively contrast-enhanced T1-weighted images as input resulted in significantly less false-positives (FPs) compared with inclusion of further sequences beyond contrast-enhanced T1-weighted images (FPs = 5/7 for CNNdual_ce/CNNdual_all, P < 1e-5). Comparison of contrast-enhanced dual and mono time approaches revealed that exclusion of prediagnosis MRI significantly increased FPs (FPs = 5/10 for CNNdual_ce/CNNce, P < 1e-9).Approaches with only native sequences were clearly inferior to CNNs that were provided contrast-enhanced sequences. Conclusions: Automated MM detection on contrast-enhanced T1-weighted images performed with high sensitivity. Frequent FPs due to artifacts and vessels were significantly reduced by additional inclusion of prediagnosis MRI, but not by inclusion of further sequences beyond contrast-enhanced T1-weighted images. Future studies might investigate different change detection architectures for computer-aided detection.

We propose a fresh take on understanding the mechanisms of neural networks by analyzing the rich directional structure of optimization trajectories, represented by their pointwise parameters. Towards this end, we introduce some natural notions of the complexity of optimization trajectories, both qualitative and quantitative, which hallmark the directional nature of optimization in neural networks: when is there redundancy, and when exploration. We use them to reveal the inherent nuance and interplay involved between various optimization choices, such as momentum and weight decay. Further, the trajectory perspective helps us see the effect of scale on regularizing the directional nature of trajectories, and as a by-product, we also observe an intriguing heterogeneity of Q,K,V dynamics in the middle attention layers in LLMs and which is homogenized by scale. Importantly, we put the significant directional redundancy observed to the test by demonstrating that training only scalar batchnorm parameters some while into training matches the performance of training the entire network, which thus exhibits the potential of hybrid optimization schemes that are geared towards efficiency., Comment: Preprint, 57 pages

Low-cost, vision-centric 3D perception systems for autonomous driving have made significant progress in recent years, narrowing the gap to expensive LiDAR-based methods. The primary challenge in becoming a fully reliable alternative lies in robust depth prediction capabilities, as camera-based systems struggle with long detection ranges and adverse lighting and weather conditions. In this work, we introduce HyDRa, a novel camera-radar fusion architecture for diverse 3D perception tasks. Building upon the principles of dense BEV (Bird's Eye View)-based architectures, HyDRa introduces a hybrid fusion approach to combine the strengths of complementary camera and radar features in two distinct representation spaces. Our Height Association Transformer module leverages radar features already in the perspective view to produce more robust and accurate depth predictions. In the BEV, we refine the initial sparse representation by a Radar-weighted Depth Consistency. HyDRa achieves a new state-of-the-art for camera-radar fusion of 64.2 NDS (+1.8) and 58.4 AMOTA (+1.5) on the public nuScenes dataset. Moreover, our new semantically rich and spatially accurate BEV features can be directly converted into a powerful occupancy representation, beating all previous camera-based methods on the Occ3D benchmark by an impressive 3.7 mIoU. Code and models are available at https://github.com/phi-wol/hydra., Comment: 10 pages, 4 figures Added eval on VoD

Aggregated flexibilities or PQ-capabilities (active and reactive power capabilities) are termed in literature as Feasible Operating Regions (FORs). The FORs from underlying active distribution grids can effectively contribute to the operational management at the HV grid level. The HV buses are allocated aggregated FORs from the underlying MV grids, which are inherently nonlinear and non-convex. Therefore, two approaches are proposed in the paper to apply the FOR constraints in the HV grid operational management. First, a mixed integer linear programming (MILP) based optimization approach for alleviating the HV grid constraint violations is proposed, which addresses the non-convexity of the FOR using piecewise segmentation. Furthermore, the MILP method is enhanced to consider the influence of the HV bus voltage on the underlying MV grid flexibilities resulting in a three dimensional PQ(V)-FOR. Second, a convexification approach is proposed, which uses a convex approximation of the non-convex 3D PQ(V)-FOR shape for implementation in a linear optimization method. Results reveal a robust utilization of the distribution flexibilities to maintain grid security and reliability at the HV grid level. Comparisons present increased computation times for the MILP method which are significantly improved using the convexification based approach.

Monochromatic gamma-ray signals constitute a potential smoking gun signature for annihilating or decaying dark matter particles that could relatively easily be distinguished from astrophysical or instrumental backgrounds. We provide an updated assessment of the sensitivity of the Cherenkov Telescope Array (CTA) to such signals, based on observations of the Galactic centre region as well as of selected dwarf spheroidal galaxies. We find that current limits and detection prospects for dark matter masses above 300 GeV will be significantly improved, by up to an order of magnitude in the multi-TeV range. This demonstrates that CTA will set a new standard for gamma-ray astronomy also in this respect, as the world's largest and most sensitive high-energy gamma-ray observatory, in particular due to its exquisite energy resolution at TeV energies and the adopted observational strategy focussing on regions with large dark matter densities. Throughout our analysis, we use up-to-date instrument response functions, and we thoroughly model the effect of instrumental systematic uncertainties in our statistical treatment. We further present results for other potential signatures with sharp spectral features, e.g.~box-shaped spectra, that would likewise very clearly point to a particle dark matter origin., Comment: 44 pages JCAP style (excluding author list and references), 19 figures; minor changes to match published version

Mass loss at the asymptotic giant branch (AGB) plays an important role not only in the final fates of stars, but also in the chemical evolution of galaxies. Nevertheless, the metallicity effects on AGB mass loss are not yet fully understood. We present spatially resolved observations of an AGB star, V3, in the metal-poor globular cluster 47 Tuc (NGC 104). The AGB star 47 Tuc V3 was observed using the GRAVITY instrument at ESO's Very Large Telescope Interferometer (VLTI) at 2-2.45 micron with a projected baseline length of up to 96 m. The object 47 Tuc V3 has been spatially resolved and stands as the first to attempt to spatially resolve an individual star in a globular cluster. The uniform-disk fit to the observed data results in an angular diameter of ~0.7 mas. Our modeling of the spectral energy distribution and near-infrared interferometric GRAVITY data suggests that the observed data can be explained by an optically thin dust shell with a 0.55 micron optical depth of 0.05-0.25, consisting of metallic iron grains, likely together with effects of the extended atmosphere of the central star. The dust temperature at the inner shell boundary is 500-800 K (corresponding to 23-90 stellar radii), significantly lower than observed in nearby oxygen-rich AGB stars. Radiation pressure on small (< 0.05 micron) iron grains is not sufficient to drive stellar winds. Therefore, iron grains may grow to larger sizes, even in the metal-poor environment. Alternatively, it is possible that the observed iron grain formation is a result of the mass outflow initiated by some other mechanism(s). The sensitivity and angular resolution of VLTI provides a new window onto spatially resolving individual stars in metal-poor globular clusters. This allows us to improve subsequent studies of the metallicity dependence of dust formation and mass loss., Comment: 7 pages, 3 figures, accepted for publication in Astronomy and Astrophysics

Hundreds of millions of people now interact with language models, with uses ranging from serving as a writing aid to informing hiring decisions. Yet these language models are known to perpetuate systematic racial prejudices, making their judgments biased in problematic ways about groups like African Americans. While prior research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice: we extend research showing that Americans hold raciolinguistic stereotypes about speakers of African American English and find that language models have the same prejudice, exhibiting covert stereotypes that are more negative than any human stereotypes about African Americans ever experimentally recorded, although closest to the ones from before the civil rights movement. By contrast, the language models' overt stereotypes about African Americans are much more positive. We demonstrate that dialect prejudice has the potential for harmful consequences by asking language models to make hypothetical decisions about people, based only on how they speak. Language models are more likely to suggest that speakers of African American English be assigned less prestigious jobs, be convicted of crimes, and be sentenced to death. Finally, we show that existing methods for alleviating racial bias in language models such as human feedback training do not mitigate the dialect prejudice, but can exacerbate the discrepancy between covert and overt stereotypes, by teaching language models to superficially conceal the racism that they maintain on a deeper level. Our findings have far-reaching implications for the fair and safe employment of language technology.