Search

Transformer models are increasingly prevalent in various applications, yet our understanding of their internal workings remains limited. This paper investigates the modularity and task specialization of neurons within transformer architectures, focusing on both vision (ViT) and language (Mistral 7B) models. Using a combination of selective pruning and MoEfication clustering techniques, we analyze the overlap and specialization of neurons across different tasks and data subsets. Our findings reveal evidence of task-specific neuron clusters, with varying degrees of overlap between related tasks. We observe that neuron importance patterns persist to some extent even in randomly initialized models, suggesting an inherent structure that training refines. Additionally, we find that neuron clusters identified through MoEfication correspond more strongly to task-specific neurons in earlier and later layers of the models. This work contributes to a more nuanced understanding of transformer internals and offers insights into potential avenues for improving model interpretability and efficiency., Comment: 11 pages, 6 figures

When fits of the same physical model to two different datasets disagree, we call this tension. Several apparent tensions in cosmology have occupied researchers in recent years, and a number of different metrics have been proposed to quantify tension. Many of these metrics suffer from limiting assumptions, and correctly calibrating these is essential if we want to successfully determine whether discrepancies are significant. A commonly used metric of tension is the evidence ratio R. The statistic has been widely adopted by the community as a Bayesian way of quantifying tensions, however, it has a non-trivial dependence on the prior that is not always accounted for properly. We show that this can be calibrated out effectively with Neural Ratio Estimation. We demonstrate our proposed calibration technique with an analytic example, a toy example inspired by 21-cm cosmology, and with observations of the Baryon Acoustic Oscillations from the Dark Energy Spectroscopic Instrument~(DESI) and the Sloan Digital Sky Survey~(SDSS). We find no significant tension between DESI and SDSS.

Whilst holding great promise for low noise, ease of operation and networking, useful photonic quantum computing has been precluded by the need for beyond-state-of-the-art components, manufactured by the millions. Here we introduce a manufacturable platform for quantum computing with photons. We benchmark a set of monolithically-integrated silicon photonics-based modules to generate, manipulate, network, and detect photonic qubits, demonstrating dual-rail photonic qubits with $99.98\% \pm 0.01\%$ state preparation and measurement fidelity, Hong-Ou-Mandel quantum interference between independent photon sources with $99.50\%\pm0.25\%$ visibility, two-qubit fusion with $99.22\%\pm0.12\%$ fidelity, and a chip-to-chip qubit interconnect with $99.72\%\pm0.04\%$ fidelity, not accounting for loss. In addition, we preview a selection of next generation technologies, demonstrating low-loss silicon nitride waveguides and components, fabrication-tolerant photon sources, high-efficiency photon-number-resolving detectors, low-loss chip-to-fiber coupling, and barium titanate electro-optic phase shifters., Comment: 8 pages, 5 figures

The Cosmic Dawn was marked by the formation of the first stars, and preceded the Epoch of Reionization (EoR), when the Universe underwent a fundamental transformation caused by the radiation from these first stars and galaxies. Interferometric 21-cm experiments aim to probe redshifted neutral hydrogen signals from these periods, constraining the conditions of the early Universe. The SKA-Low instrument of the Square Kilometre Array (SKA) is envisaged to be the largest and most sensitive radio telescope at metre and centimetre wavelengths. The latest Aperture Array Verification Systems feature 7-m coaxial transmission lines connecting the low noise amplifiers to optical transmitters at the front of the analogue-receiving chain. An impedance mismatch between these components results in a partially reflected electromagnetic signal, introducing chromatic aberrations into the instrument bandpass. This causes power from the foreground signals to appear at higher delays, potentially contaminating the 'EoR window', a region in which the 21-cm signal should be detectable. We present an end-to-end simulation pipeline for SKA-Low using a composite sky model combining radio foregrounds from the Galactic and Extragalactic All-Sky MWA (GLEAM) Survey, Haslam $408\,$MHz, and a 1.5-cGpc 21-cm brightness temperature cube generated with the 21cmSPACE simulator. We derive a model for the scattering parameters of a coaxial transmission line in terms of its specifications and bulk material properties. Assuming identical cables of length $\leq 15.0\,$m with impedance mismatch $\leq 10\, \Omega$, the reflection is confined below the EoR window. However, we demonstrate that even a 0.1 per cent length tolerance introduces contamination with an absolute fractional difference of $\sim 10$ per cent across all accessible k-modes., Comment: 17 pages, 12 figures

The hydrogen 21-cm signal is predicted to be the richest probe of the young Universe including eras known as the cosmic Dark Ages, the Cosmic Dawn when the first star and black hole formed, and the Epoch of Reionization. This signal holds the key to deciphering processes that take place at the early stages of cosmic history. In this opinion piece, we discuss the potential scientific merit of lunar observations of the 21-cm signal and their advantages over more affordable terrestrial efforts. The moon is a prime location for radio cosmology which will enable precision observations of the low-frequency radio sky. The uniqueness of such observations is that they will provide an unparalleled opportunity to test cosmology and the nature of dark matter using the Dark Ages 21-cm signal. No less enticing is the opportunity to obtain a much clearer picture of Cosmic Dawn than what is achievable from the ground, which will allow us to probe properties of the first stars and black holes., Comment: 15 pages, 3 figures, accepted for publication in Philosophical Transactions A

Nonreciprocity in space-time modulated photonic crystals has been investigated in the context of nonreciprocal propagation and polarization. Here, we investigate a reconfigurable nonreciprocal wave growth in space-time modulated crystals. Imposing an adaptable progressive phase shift between successive time-modulated cells results in blue and red shifts of the forward and backward momentum band gaps around the typical 0.5 growth normalized frequency. We applied this spatiotemporal scheme to engineering the dispersion relation of a loaded transmission line$-$a 1D periodic structure$-$in the microwave regime.

When an electromagnetic (EM) wave is propagating in a medium whose properties are varied abruptly in time, the wave experiences refractions and reflections known as "time-refractions" and "time-reflections", both manifesting spectral translation as a consequence of the abrupt change of the medium and the conservation of momentum. However, while the time-refracted wave continues to propagate with the same wave-vector, the time-reflected wave is propagating backward with a conjugate phase, despite the lack of any spatial interface. Importantly, while time-refraction is always significant, observing time-reflection poses a major challenge - because it requires a large change in the medium occurring within a single cycle. For that reason, time-reflection of EM waves was observed only recently. Here, we present the observation of microwave pulses at the highest frequency ever observed (0.59 GHz), and the experimental evidence of the phase-conjugation nature of time-reflected waves. Our experiments are carried out in a periodically-loaded microstrip line with optically-controlled picosecond-switchable photodiodes. Our system paves the way to the experimental realization of Photonic Time-Crystals at GHz frequencies.

We investigate and simulate signal amplification in a transmission line (TL) with time-modulated characteristic impedance Zo. Periodically varying $Z_o$ is achieved by loading TL with a sinusoidally time-modulated capacitor (TMC). For a detailed study, three models are considered: a lossless L-C TL lumped model loaded with shunt infinite quality factor (Q) TMC, a TL loaded with a shunt infinite Q TMC, and finite Q TMC. By solving the eigenvalue problem in all models, dispersion diagrams (DD) are plotted with a created momentum band gap (MBG) at a modulation frequency double the signal frequency. Within MBG, only imaginary frequencies are found and correlated to MBG width and signal growth level. Using Harmonics Balance (HB) and Transient Simulation (TS), signal amplification is confirmed, and the obtained results are consistent with the DD outcomes. In the second model, the effect of TL length on amplification is investigated and explained by studying the unit cell's Bloch impedance. The loss effect is considered by adding a series resistance (Rc) to the third model's TMC (finite Q). Decreasing amplification levels, confirmed by circuit modeling, due to the increase of Rc value is explained by studying real and imaginary DDs and the attenuation constant.

Normalizing flows are an established approach for modelling complex probability densities through invertible transformations from a base distribution. However, the accuracy with which the target distribution can be captured by the normalizing flow is strongly influenced by the topology of the base distribution. A mismatch between the topology of the target and the base can result in a poor performance, as is typically the case for multi-modal problems. A number of different works have attempted to modify the topology of the base distribution to better match the target, either through the use of Gaussian Mixture Models (Izmailov et al., 2020; Ardizzone et al., 2020; Hagemann & Neumayer, 2021) or learned accept/reject sampling (Stimper et al., 2022). We introduce piecewise normalizing flows which divide the target distribution into clusters, with topologies that better match the standard normal base distribution, and train a series of flows to model complex multi-modal targets. We demonstrate the performance of the piecewise flows using some standard benchmarks and compare the accuracy of the flows to the approach taken in Stimper et al. (2022) for modelling multi-modal distributions. We find that our approach consistently outperforms the approach in Stimper et al. (2022) with a higher emulation accuracy on the standard benchmarks., Comment: 11 pages, 5 figures

The inverse-Compton effect (IC) is a widely recognized cooling mechanism for both relativistic and thermal electrons in various astrophysical environments, including the intergalactic medium and X-ray emitting plasmas. Its effect on thermal electrons is however frequently overlooked in theoretical and numerical models of colliding-wind binaries (CWB). In this article, we provide a comprehensive investigation of the impact of IC cooling in CWBs, presenting general results for when the photon fields of the stars dominate the cooling of the thermal plasma and when shocks at the stagnation point are expected to be radiative. Our analysis shows that IC cooling is the primary cooling process for the shocked-wind layer over a significant portion of the relevant parameter space, particularly in eccentric systems with large wind-momentum ratios, e.g., those containing a Wolf-Rayet and O-type star. Using the binary system WR 140 as a case study, we demonstrate that IC cooling leads to a strongly radiative shocked wind near periastron, which may otherwise remain adiabatic if only collisional cooling was considered. Our results are further supported by 2D and 3D simulations of wind-wind collisions. Specifically, 3D magnetohydrodynamic simulations of WR 140 show a significant decrease in hard-X-ray emission around periastron, in agreement with observations but in contrast to equivalent simulations that omit IC cooling. A novel method is proposed for constraining mass-loss rates of both stars in eccentric binaries where the wind-collision zone switches from adiabatic to radiative approaching periastron. IC scattering is an important cooling process in the thermal plasma of CWBs., Comment: Accepted to MNRAS, 17 pages, 13 figures

TORCH is a novel time-of-flight detector, designed to provide charged particle identification of pions, kaons and protons in the momentum range 2-20 GeV/c over a 9.5 m flight path. A detector module, comprising a 10mm thick quartz plate, provides a source of Cherenkov photons which propagate via total internal reflection to one end of the plate. Here, the photons are focused onto an array of custom-designed Micro-Channel Plate Photo-Multiplier Tubes (MCP-PMTs) which measure their positions and arrival times. The target time resolution per photon is 70 ps which, for 30 detected photons per charged particle, results in a 10-15 ps time-of-flight resolution. A 1.25 m length TORCH prototype module employing two MCP-PMTs has been developed, and tested at the CERN PS using a charged hadron beam of 8 GeV/c momentum. The construction of the module, the properties of the MCP-PMTs and the readout electronics are described. Measurements of the collected photon yields and single-photon time resolutions have been performed as a function of particle entry points on the plate and compared to expectations. These studies show that the performance of the TORCH prototype approaches the design goals for the full-scale detector., Comment: 18 pages, 16 figures, to be published in Nucl. Instrum. Methods A. During review, clarity added to description throughout; updated figures 15 and 16; revised argument in section 6.3

Thin synchrotron-emitting filaments are increasingly seen in the intracluster medium (ICM). We present the first example of a direct interaction between a magnetic filament, a radio jet, and a dense ICM clump in the poor cluster Abell 194. This enables the first exploration of the dynamics and possible histories of magnetic fields and cosmic rays in such filaments. Our observations are from the MeerKAT Galaxy Cluster Legacy Survey and the LOFAR Two Metre Sky Survey. Prominent 220 kpc long filaments extend east of radio galaxy 3C40B, with very faint extensions to 300 kpc, and show signs of interaction with its northern jet. They curve around a bend in the jet and intersect the jet in Faraday depth space. The X-ray surface brightness drops across the filaments; this suggests that the relativistic particles and fields contribute significantly to the pressure balance and evacuate the thermal plasma in a $\sim$35 kpc cylinder. We explore whether the relativistic electrons could have streamed along the filaments from 3C40B, and present a plausible alternative whereby magnetized filaments are a) generated by shear motions in the large-scale, post-merger ICM flow, b) stretched by interactions with the jet and flows in the ICM, amplifying the embedded magnetic fields, and c) perfused by re-energized relativistic electrons through betatron-type acceleration or diffusion of turbulently accelerated ICM cosmic ray electrons. We use the Faraday depth measurements to reconstruct some of the 3D structures of the filaments and of 3C40A and B., Comment: Accepted for publication in ApJ. 32 pages, 28 figures. Each of the last four figures has a corresponding movie file (mpeg), available under the Ancillary Files associated with this preprint

This Chapter provides a brief tutorial on some aspects of plasma physics that are fundamental to understanding the dynamics and energetics of the intracluster medium (ICM). The tutorial is split into two parts: one that focuses on the thermal plasma component -- its stability, viscosity, conductivity, and ability to amplify magnetic fields to dynamical strengths via turbulence and other plasma processes; and one that focuses on the non-thermal population of charged particles known as cosmic rays -- their acceleration, re-acceleration, and transport throughout the cluster volume. Observational context is woven throughout the narrative, from constraints on the strength and geometry of intracluster magnetic fields and the effective viscosity of the ICM, to examples of radio halos, radio relics, and cluster shocks that can test theories of particle acceleration. The promise of future X-ray missions to probe intracluster turbulence and discover the impact of small-scale plasma physics, coupled with sensitive, high-resolution radio observations of synchrotron-emitting plasma that reveal the properties of intracluster magnetic fields and particle-acceleration mechanisms, are likely to establish galaxy clusters as the premier cosmic laboratories for deciphering the fundamental physics of hot, dilute plasmas., Comment: This Chapter will appear in the Section "Galaxy Clusters" (Section Editors: E. Pointecouteau, E. Rasia, A. Simionescu) of the "Handbook of X-ray and Gamma-ray Astrophysics" (Editors in chief: C. Bambi and A. Santangelo)

TORCH is a large-area, high-precision time-of-flight (ToF) detector designed to provide charged-particle identification in the 2-20 GeV$/c$ momentum range. Prompt Cherenkov photons emitted by charged hadrons as they traverse a 10mm quartz radiator are propagated to the periphery of the detector, where they are focused onto an array of micro-channel plate photomultiplier tubes (MCP-PMTs). The position and arrival times of the photons are used to infer the particles' time of entry in the radiator, to identify hadrons based on their ToF. The MCP-PMTs were developed with an industrial partner to satisfy the stringent requirements of the TORCH detector. The requirements include a finely segmented anode, excellent time resolution, and a long lifetime. Over an approximately 10m flight distance, the difference in ToF between a kaon and a pion with 10GeV$/c$ momentum is 35ps, leading to a 10-15ps per track timing resolution requirement. On average 30 photons per hadron are detected, which translates to a single-photon time resolution of 70ps. The TORCH research and development program aims to demonstrate the validity of the detector concept through laboratory and beam tests, results from which are presented. A timing resolution of 70-100ps was reached in beam tests, approaching the TORCH design goal. Laboratory timing tests consist of operating the MCP-PMTs coupled to the TORCH readout electronics. A time resolution of about 50ps was measured, meeting the TORCH target timing resolution., Comment: Vienna Instrumentation Conference 2022 proceedings. 13 pages,5 figures, 1 table