Search

Magnetized beams beam with significant canonical angular momentum are critical to electron cooling of hadron beams such as contemplated in next-generation hadron and electron-ion colliders. The transport of magnetized electron beams over long distances in a locally non-axisymmetric external field is challenging. An alternative is to transform the beam into an uncoupled "flat beam", transport the produced "flat" beam over a long distance, and reintroduce the cross-plane coupling to "re-magnetize" the beam. In this paper, we demonstrate via numerical simulation and laboratory experiments such a cascaded-transformation approach., Comment: Submitted to Proceedings of AAC 2024 Workshop

Existing preference optimization methods are mainly designed for directly learning from human feedback with the assumption that paired examples (preferred vs. dis-preferred) are available. In contrast, we propose a method that can leverage unpaired preferred or dis-preferred examples, and works even when only one type of feedback (positive or negative) is available. This flexibility allows us to apply it in scenarios with varying forms of feedback and models, including training generative language models based on human feedback as well as training policies for sequential decision-making problems, where learned (value) functions are available. Our approach builds upon the probabilistic framework introduced in (Dayan and Hinton, 1997), which proposes to use expectation-maximization (EM) to directly optimize the probability of preferred outcomes (as opposed to classic expected reward maximization). To obtain a practical algorithm, we identify and address a key limitation in current EM-based methods: when applied to preference optimization, they solely maximize the likelihood of preferred examples, while neglecting dis-preferred samples. We show how one can extend EM algorithms to explicitly incorporate dis-preferred outcomes, leading to a novel, theoretically grounded, preference optimization algorithm that offers an intuitive and versatile way to learn from both positive and negative feedback.

Hate speech is a harmful form of online expression, often manifesting as derogatory posts. It is a significant risk in digital environments. With the rise of Large Language Models (LLMs), there is concern about their potential to replicate hate speech patterns, given their training on vast amounts of unmoderated internet data. Understanding how LLMs respond to hate speech is crucial for their responsible deployment. However, the behaviour of LLMs towards hate speech has been limited compared. This paper investigates the reactions of seven state-of-the-art LLMs (LLaMA 2, Vicuna, LLaMA 3, Mistral, GPT-3.5, GPT-4, and Gemini Pro) to hate speech. Through qualitative analysis, we aim to reveal the spectrum of responses these models produce, highlighting their capacity to handle hate speech inputs. We also discuss strategies to mitigate hate speech generation by LLMs, particularly through fine-tuning and guideline guardrailing. Finally, we explore the models' responses to hate speech framed in politically correct language.

Reward models (RMs) play a pivotal role in aligning large language models (LLMs) with human preferences. However, traditional RM training, which relies on response pairs tied to specific prompts, struggles to disentangle prompt-driven preferences from prompt-independent artifacts, such as response length and format. In this work, we expose a fundamental limitation of current RM training methods, where RMs fail to effectively distinguish between contextual signals and irrelevant artifacts when determining preferences. To address this, we introduce a causal framework that learns preferences independent of these artifacts and propose a novel data augmentation technique designed to eliminate them. Extensive experiments show that our approach successfully filters out undesirable artifacts, yielding a more robust reward model (RRM). Our RRM improves the performance of a pairwise reward model trained on Gemma-2-9b-it, on RewardBench, increasing accuracy from 80.61% to 84.15%. Additionally, we train two DPO policies using both the RM and RRM, demonstrating that the RRM significantly enhances DPO-aligned policies, improving MT-Bench scores from 7.27 to 8.31 and length-controlled win-rates in AlpacaEval-2 from 33.46% to 52.49%.

Recent studies have shown that large language models' (LLMs) mathematical problem-solving capabilities can be enhanced by integrating external tools, such as code interpreters, and employing multi-turn Chain-of-Thought (CoT) reasoning. While current methods focus on synthetic data generation and Supervised Fine-Tuning (SFT), this paper studies the complementary direct preference learning approach to further improve model performance. However, existing direct preference learning algorithms are originally designed for the single-turn chat task, and do not fully address the complexities of multi-turn reasoning and external tool integration required for tool-integrated mathematical reasoning tasks. To fill in this gap, we introduce a multi-turn direct preference learning framework, tailored for this context, that leverages feedback from code interpreters and optimizes trajectory-level preferences. This framework includes multi-turn DPO and multi-turn KTO as specific implementations. The effectiveness of our framework is validated through training of various language models using an augmented prompt set from the GSM8K and MATH datasets. Our results demonstrate substantial improvements: a supervised fine-tuned Gemma-1.1-it-7B model's performance increased from 77.5% to 83.9% on GSM8K and from 46.1% to 51.2% on MATH. Similarly, a Gemma-2-it-9B model improved from 84.1% to 86.3% on GSM8K and from 51.0% to 54.5% on MATH., Comment: A multi-turn direct preference learning framework for tool-integrated reasoning tasks

The workshop focused on the application of ANAs to particle physics keeping in mind the ultimate goal of a collider at the energy frontier (10\,TeV, e$^+$/e$^-$, e$^-$/e$^-$, or $\gamma\gamma$). The development of ANAs is conducted at universities and national laboratories worldwide. The community is thematically broad and diverse, in particular since lasers suitable for ANA research (multi-hundred-terawatt peak power, a few tens of femtosecond-long pulses) and acceleration of electrons to hundreds of mega electron volts to multi giga electron volts became commercially available. The community spans several continents (Europe, America, Asia), including more than 62 laboratories in more than 20 countries. It is among the missions of the ICFA-ANA panel to feature the amazing progress made with ANAs, to provide international coordination and to foster international collaborations towards a future HEP collider. The scope of this edition of the workshop was to discuss the recent progress and necessary steps towards realizing a linear collider for particle physics based on novel-accelerator technologies (laser or beam driven in plasma or structures). Updates on the relevant aspects of the European Strategy for Particle Physics (ESPP) Roadmap Process as well as of the P5 (in the US) were presented, and ample time was dedicated to discussions. The major outcome of the workshop is the decision for ALEGRO to coordinate efforts in Europe, in the US, and in Asia towards a pre-CDR for an ANA-based, 10\,TeV CM collider. This goal of this coordination is to lead to a funding proposal to be submitted to both EU and EU/US funding agencies. This document presents a summary of the workshop, as seen by the co-chairs, as well as short 'one-pagers' written by the presenters at the workshop., Comment: 72 pages

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

The dominant framework for alignment of large language models (LLM), whether through reinforcement learning from human feedback or direct preference optimisation, is to learn from preference data. This involves building datasets where each element is a quadruplet composed of a prompt, two independent responses (completions of the prompt) and a human preference between the two independent responses, yielding a preferred and a dis-preferred response. Such data is typically scarce and expensive to collect. On the other hand, \emph{single-trajectory} datasets where each element is a triplet composed of a prompt, a response and a human feedback is naturally more abundant. The canonical element of such datasets is for instance an LLM's response to a user's prompt followed by a user's feedback such as a thumbs-up/down. Consequently, in this work, we propose DRO, or \emph{Direct Reward Optimisation}, as a framework and associated algorithms that do not require pairwise preferences. DRO uses a simple mean-squared objective that can be implemented in various ways. We validate our findings empirically, using T5 encoder-decoder language models, and show DRO's performance over selected baselines such as Kahneman-Tversky Optimization (KTO). Thus, we confirm that DRO is a simple and empirically compelling method for single-trajectory policy optimisation.

Probing the quantum geometry and topology in condensed matter systems has relied heavily on static electronic transport experiments in magnetic fields. Yet, contact-free optical measurements have rarely been explored. Magnetic dichroism (MCD), the nonreciprocal absorption of circular polarized light, was theoretically linked to the quantized anomalous Hall effect in magnetic insulators and can identify the bands and momenta responsible for the underlying Berry Curvature (BC). Detecting BC through MCD faces two challenges: First, the relevant inter-band transitions usually generate MCD in the infrared (IR) range, requiring large samples with high quality. Second, while most magnetic materials are metallic, the relation between MCD and BC in metals remains unclear. Here, we report the observation of MCD in the IR range along with the anomalous Hall effect in thin film MnBi2Te4. Both phenomena emerge with a field-driven phase transition from an antiferromagnet to a canted ferromagnet. By theoretically relating the MCD to the anomalous Hall effect via BC in a metal, we show that this transition accompanies an abrupt onset of BC, signaling a topological phase transition from a topological insulator to a doped Chern insulator. Our density functional theory calculation suggests the MCD signal mainly originates from an optical transition at the Brillouin zone edge, hinting at a potential new source of BC away from the commonly considered {\Gamma} point. Our findings demonstrate a novel experimental approach for detecting BC and identifying the responsible bands and momenta, generally applicable to magnetic materials.

Reinforcement Learning from Human Feedback (RLHF) has become the standard approach for aligning Large Language Models (LLMs) with human preferences, allowing LLMs to demonstrate remarkable abilities in various tasks. Existing methods work by emulating the preferences at the single decision (turn) level, limiting their capabilities in settings that require planning or multi-turn interactions to achieve a long-term goal. In this paper, we address this issue by developing novel methods for Reinforcement Learning (RL) from preference feedback between two full multi-turn conversations. In the tabular setting, we present a novel mirror-descent-based policy optimization algorithm for the general multi-turn preference-based RL problem, and prove its convergence to Nash equilibrium. To evaluate performance, we create a new environment, Education Dialogue, where a teacher agent guides a student in learning a random topic, and show that a deep RL variant of our algorithm outperforms RLHF baselines. Finally, we show that in an environment with explicit rewards, our algorithm recovers the same performance as a reward-based RL baseline, despite relying solely on a weaker preference signal.

Ensuring alignment of language models' outputs with human preferences is critical to guarantee a useful, safe, and pleasant user experience. Thus, human alignment has been extensively studied recently and several methods such as Reinforcement Learning from Human Feedback (RLHF), Direct Policy Optimisation (DPO) and Sequence Likelihood Calibration (SLiC) have emerged. In this paper, our contribution is two-fold. First, we show the equivalence between two recent alignment methods, namely Identity Policy Optimisation (IPO) and Nash Mirror Descent (Nash-MD). Second, we introduce a generalisation of IPO, named IPO-MD, that leverages the regularised sampling approach proposed by Nash-MD. This equivalence may seem surprising at first sight, since IPO is an offline method whereas Nash-MD is an online method using a preference model. However, this equivalence can be proven when we consider the online version of IPO, that is when both generations are sampled by the online policy and annotated by a trained preference model. Optimising the IPO loss with such a stream of data becomes then equivalent to finding the Nash equilibrium of the preference model through self-play. Building on this equivalence, we introduce the IPO-MD algorithm that generates data with a mixture policy (between the online and reference policy) similarly as the general Nash-MD algorithm. We compare online-IPO and IPO-MD to different online versions of existing losses on preference data such as DPO and SLiC on a summarisation task.

Beams with cross-plane coupling or extreme asymmetries between the two transverse phase spaces are often encountered in particle accelerators. Flat beams with large transverse-emittance ratios are critical for future linear colliders. Similarly, magnetized beams with significant cross-plane coupling are expected to enhance the performance of electron cooling in hadron beams. Preparing these beams requires precise control and characterization of the four-dimensional transverse phase space. In this study, we employ generative phase space reconstruction (GPSR) techniques to rapidly characterize magnetized and flat-beam phase-space distributions using a conventional quadrupole-scan method. The reconstruction technique is experimentally demonstrated on an electron beam produced at the Argonne Wakefield Accelerator and successfully benchmarked against conventional diagnostics techniques. Specifically, we show that predicted beam parameters from the reconstructed phase-space distributions (e.g. as magnetization and flat beam emittances) are in excellent agreement with those measured from the conventional diagnostic methods.

Offline preference optimization allows fine-tuning large models directly from offline data, and has proved effective in recent alignment practices. We propose generalized preference optimization (GPO), a family of offline losses parameterized by a general class of convex functions. GPO enables a unified view over preference optimization, encompassing existing algorithms such as DPO, IPO and SLiC as special cases, while naturally introducing new variants. The GPO framework also sheds light on how offline algorithms enforce regularization, through the design of the convex function that defines the loss. Our analysis and experiments reveal the connections and subtle differences between the offline regularization and the KL divergence regularization intended by the canonical RLHF formulation. In a controlled setting akin to Gao et al 2023, we also show that different GPO variants achieve similar trade-offs between regularization and performance, though the optimal values of hyper-parameter might differ as predicted by theory. In all, our results present new algorithmic toolkits and empirical insights to alignment practitioners., Comment: Accepted at ICML 2023 main conference

Direct alignment from preferences (DAP) methods, such as DPO, have recently emerged as efficient alternatives to reinforcement learning from human feedback (RLHF), that do not require a separate reward model. However, the preference datasets used in DAP methods are usually collected ahead of training and never updated, thus the feedback is purely offline. Moreover, responses in these datasets are often sampled from a language model distinct from the one being aligned, and since the model evolves over training, the alignment phase is inevitably off-policy. In this study, we posit that online feedback is key and improves DAP methods. Our method, online AI feedback (OAIF), uses an LLM as annotator: on each training iteration, we sample two responses from the current model and prompt the LLM annotator to choose which one is preferred, thus providing online feedback. Despite its simplicity, we demonstrate via human evaluation in several tasks that OAIF outperforms both offline DAP and RLHF methods. We further show that the feedback leveraged in OAIF is easily controllable, via instruction prompts to the LLM annotator., Comment: 18 pages, 9 figures, 4 tables

Hate speech represents a pervasive and detrimental form of online discourse, often manifested through an array of slurs, from hateful tweets to defamatory posts. As such speech proliferates, it connects people globally and poses significant social, psychological, and occasionally physical threats to targeted individuals and communities. Current computational linguistic approaches for tackling this phenomenon rely on labelled social media datasets for training. For unifying efforts, our study advances in the critical need for a comprehensive meta-collection, advocating for an extensive dataset to help counteract this problem effectively. We scrutinized over 60 datasets, selectively integrating those pertinent into MetaHate. This paper offers a detailed examination of existing collections, highlighting their strengths and limitations. Our findings contribute to a deeper understanding of the existing datasets, paving the way for training more robust and adaptable models. These enhanced models are essential for effectively combating the dynamic and complex nature of hate speech in the digital realm.

Inorganic semiconductors based on heavy pnictogen cations (Sb3+ and Bi3+) have gained significant attention as potential nontoxic and stable alternatives to lead-halide perovskites for solar cell applications. A limitation of these novel materials, which is being increasingly commonly found, is carrier localization, which substantially reduces mobilities and diffusion lengths. Herein, the layered p\v{r}\'ibramite CuSbSe2 is investigated and discovered to have delocalized free carriers, as shown through optical pump terahertz probe spectroscopy and temperature-dependent mobility measurements. Using a combination of theory and experiment, it is found that the underlying factors are: 1) weak coupling to acoustic phonons due to low deformation potentials, as lattice distortions are primarily accommodated through rigid inter-layer movement rather than straining inter-atomic bonds, and 2) weak coupling to optical phonons due to the ionic contributions to the dielectric constant being low compared to electronic contributions. This work provides important insights into how pnictogen-based semiconductors avoiding carrier localization could be identified., Comment: 47 pages, 4 figures

Here we report on Landau level spectroscopy of an epitaxially grown thin film of the topological insulator Sb2Te3, complemented by ellipsometry and magneto-transport measurements. The observed response suggests that Sb2Te3 is a direct-gap semiconductor with the fundamental band gap located at the \Gamma point, or along the trigonal axis, and its width reaches Eg = 190 meV at low temperatures. Our data also indicate the presence of other low-energy extrema with a higher multiplicity in both the conduction and valence bands. The conclusions based on our experimental data are confronted with and to a great extent corroborated by the electronic band structure calculated using the GW method., Comment: 11 pages, 8 figures, to be published in Phys. Rev. B

Reinforcement learning from human feedback (RLHF) has emerged as the main paradigm for aligning large language models (LLMs) with human preferences. Typically, RLHF involves the initial step of learning a reward model from human feedback, often expressed as preferences between pairs of text generations produced by a pre-trained LLM. Subsequently, the LLM's policy is fine-tuned by optimizing it to maximize the reward model through a reinforcement learning algorithm. However, an inherent limitation of current reward models is their inability to fully represent the richness of human preferences and their dependency on the sampling distribution. In this study, we introduce an alternative pipeline for the fine-tuning of LLMs using pairwise human feedback. Our approach entails the initial learning of a preference model, which is conditioned on two inputs given a prompt, followed by the pursuit of a policy that consistently generates responses preferred over those generated by any competing policy, thus defining the Nash equilibrium of this preference model. We term this approach Nash learning from human feedback (NLHF). In the context of a tabular policy representation, we present a novel algorithmic solution, Nash-MD, founded on the principles of mirror descent. This algorithm produces a sequence of policies, with the last iteration converging to the regularized Nash equilibrium. Additionally, we explore parametric representations of policies and introduce gradient descent algorithms for deep-learning architectures. To demonstrate the effectiveness of our approach, we present experimental results involving the fine-tuning of a LLM for a text summarization task. We believe NLHF offers a compelling avenue for preference learning and policy optimization with the potential of advancing the field of aligning LLMs with human preferences.

The prevalent deployment of learning from human preferences through reinforcement learning (RLHF) relies on two important approximations: the first assumes that pairwise preferences can be substituted with pointwise rewards. The second assumes that a reward model trained on these pointwise rewards can generalize from collected data to out-of-distribution data sampled by the policy. Recently, Direct Preference Optimisation (DPO) has been proposed as an approach that bypasses the second approximation and learn directly a policy from collected data without the reward modelling stage. However, this method still heavily relies on the first approximation. In this paper we try to gain a deeper theoretical understanding of these practical algorithms. In particular we derive a new general objective called $\Psi$PO for learning from human preferences that is expressed in terms of pairwise preferences and therefore bypasses both approximations. This new general objective allows us to perform an in-depth analysis of the behavior of RLHF and DPO (as special cases of $\Psi$PO) and to identify their potential pitfalls. We then consider another special case for $\Psi$PO by setting $\Psi$ simply to Identity, for which we can derive an efficient optimisation procedure, prove performance guarantees and demonstrate its empirical superiority to DPO on some illustrative examples.

Magneto-spectroscopy methods have been employed to study the zero-wavevector magnon excitations in MnPSe$_3$. Experiments carried out as a function of temperature and the applied magnetic field show that two low-energy magnon branches of MnPSe$_3$ in its antiferromagnetic phase are gapped. The observation of two low-energy magnon gaps (at 14 and 0.7 cm$^{-1}$) implies that MnPSe$_3$ is a biaxial antiferromagnet. A relatively strong out-of-plane anisotropy imposes the spin alignment to be in-plane whereas the spin directionality within the plane is governed by a factor of 2.5 $\times$ 10$^{-3}$ weaker in-plane anisotropy., Comment: 9 pages, 3 figures

The Snowmass report on rare processes and precision measurements recommended Mu2e-II and a next generation muon facility at Fermilab (Advanced Muon Facility) as priorities for the frontier. The Workshop on a future muon program at FNAL was held in March 2023 to discuss design studies for Mu2e-II, organizing efforts for the next generation muon facility, and identify synergies with other efforts (e.g., muon collider). Topics included high-power targetry, status of R&D for Mu2e-II, development of compressor rings, FFA and concepts for muon experiments (conversion, decays, muonium and other opportunities) at AMF. This document summarizes the workshop discussions with a focus on future R&D tasks needed to realize these concepts., Comment: 68 pages, 36 figures

Neutron dark decays have been suggested as a solution to the discrepancy between bottle and beam experiments, providing a dark matter candidate that can be searched for in halo nuclei. The free neutron in the final state following the decay of $^6$He into $^4$He $+$ $n$ + $\chi$ provides an exceptionally clean detection signature when combined with a high efficiency neutron detector. Using a high-intensity $^6$He$^+$ beam at GANIL, a search for a coincident neutron signal resulted in an upper limit on a dark decay branching ratio of Br$_\chi \leq 4.0\times10^{-10}$ (95\% C.L.). Using the dark neutron decay model proposed originally by Fornal and Grinstein, we translate this into an upper bound on a dark neutron branching ratio of $\mathcal{O}(10^{-5})$, improving over global constraints by one to several orders of magnitude depending on $m_\chi$.

Relativistic charged-particle beams which generate intense longitudinal fields in accelerating structures also inherently couple to transverse modes. The effects of this coupling may lead to beam break-up instability, and thus must be countered to preserve beam quality in applications such as linear colliders. Beams with highly asymmetric transverse sizes (flat-beams) have been shown to suppress the initial instability in slab-symmetric structures. However, as the coupling to transverse modes remains, this solution serves only to delay instability. In order to understand the hazards of transverse coupling in such a case, we describe here an experiment characterizing the transverse effects on a flat-beam, traversing near a planar dielectric lined structure. The measurements reveal the emergence of a previously unobserved skew-quadrupole-like interaction when the beam is canted transversely, which is not present when the flat-beam travels parallel to the dielectric surface. We deploy a multipole field fitting algorithm to reconstruct the projected transverse wakefields from the data. We generate the effective kick vector map using a simple two-particle theoretical model, with particle-in-cell simulations used to provide further insight for realistic particle distributions., Comment: Six pages, seven figures. Submitted to Physical Review

The receptivity of a laminar boundary layer flow to small amplitude wall roughness is investigated on an ONERA-D swept aerofoil by introducing a dedicated resolvent operator based on linearised small amplitude wall displacements. The singular value decomposition of this operator for a given spanwise wavenumber provides optimal wall roughness and flow responses that maximise an input-output gain. At the most receptive spanwise wavenumber, the optimal response is a cross-flow mode associated with an optimal roughness located close to the attachment-line and presenting a wavy shape with a wavevector nearly orthogonal to the external streamlines. The method therefore allows direct identification of the location and structure (chordwise and spanwise wavenumbers) of the most receptive roughness. For various given wall roughness shapes and locations (periodic or compact in the chordwise and/or spanwise directions), an approximation of the response based on the dominant optimal response is shown to accurately match the total response downstream of the roughness. The method therefore allows a straightforward computation of the response of the flow to any given small amplitude roughness.

Beam-driven wakefield accelerators are foreseen to enable compact accelerator-based light sources and play a critical role in future linear-collider concepts. This class of wakefield acceleration has been extensively studied over the last four decades with a focus on demonstrating its ability to support high-accelerating gradient and, most recently, enhanced transformer ratios. Yet, the associated detrimental transverse wakefields have not been examined in as many details due to the limited diagnostics available. In this paper, we introduce a beam-based single-shot transverse-wakefield measurement technique. The approach employs a witness ''hollow" electron beam to probe the wakefields generated by a drive bunch. We show how the transverse distortions of the hollow probe provide a direct measurement of the wakefield distribution within the area circumscribed by the probe. The ability to directly measure a full structure of the transverse wakefield could help to develop mitigation schemes and ultimately suppress the adverse beam-break-up instabilities. We discuss a practical implementation of the method and demonstrate its performance with the help of start-to-end simulations., Comment: 13 pages 15 figures

Cooling of beams circulating in storage rings is critical for many applications including particle colliders and synchrotron light sources. A method enabling unprecedented beam-cooling rates, optical stochastic cooling (OSC), was recently demonstrated in the IOTA electron storage ring at Fermilab. This paper describes the numerical implementation of the OSC process in the particle-tracking program ELEGANT and discusses the validation of the developed model with available experimental data. The model is also employed to highlight some features associated with different modes of operation of OSC. The developed simulation tool should be valuable in guiding future configurations of optical stochastic cooling and, more broadly, modeling self-field-based beam manipulations., Comment: 15 pages, 10 figures

A single-shot method to reconstruct the transverse self-wakefields acting on a beam, based only on screen images, is introduced. By employing numerical optimization with certain approximations, a relatively high-dimensional parameter space is efficiently explored to determine the multipole components of the transverse-wakefield topology up to desired order. The reconstruction technique complements simulations, which are able to directly describe the wakefield composition based on experimental conditions. The technique is applied to representative simulation results as a benchmark, and also to experimental data on wakefield observations driven in dielectric-lined structures., Comment: 10 pages, 8 figures

We introduce Robust Exploration via Clustering-based Online Density Estimation (RECODE), a non-parametric method for novelty-based exploration that estimates visitation counts for clusters of states based on their similarity in a chosen embedding space. By adapting classical clustering to the nonstationary setting of Deep RL, RECODE can efficiently track state visitation counts over thousands of episodes. We further propose a novel generalization of the inverse dynamics loss, which leverages masked transformer architectures for multi-step prediction; which in conjunction with RECODE achieves a new state-of-the-art in a suite of challenging 3D-exploration tasks in DM-Hard-8. RECODE also sets new state-of-the-art in hard exploration Atari games, and is the first agent to reach the end screen in "Pitfall!".

Competition between exchange interactions and magnetocrystalline anisotropy may bring new magnetic states that are of great current interest. An applied hydrostatic pressure can further be used to tune their balance. In this work we investigate the magnetization process of a biaxial antiferromagnet in an external magnetic field applied along the easy axis. We find that the single metamagnetic transition of the Ising type observed in this material under ambient pressure transforms under hydrostatic pressure into two transitions, a first-order spin flop transition followed by a second order transition towards a polarized ferromagnetic state near saturation. This reversible tuning into a new magnetic phase is obtained in layered bulk CrSBr at low temperature by varying the interlayer distance using high hydrostatic pressure, which efficiently acts on the interlayer magnetic exchange, and is probed by magneto-optical spectroscopy., Comment: 3 Figures, 6 pages. To appear in ACS NanoLetters

The low-lying structure of $^{15}$C has been investigated via the neutron-removal $^{16}$C$(d,t)$ reaction. Along with bound neutron sd-shell hole states, unbound p-shell hole states have been firmly confirmed. The excitation energies and the deduced spectroscopic factors of the cross-shell states are an important measure of the $[(p)^{-1}(sd)^{2}]$ neutron configurations in $^{15}$C. Our results show a very good agreement with shell-model calculations using the SFO-tls interaction for $^{15}$C. However, a modification of the $p$-$sd$ and $sd$-$sd$ monopole terms was applied in order to reproduce the $N=9$ isotone $^{17}$O. In addition, the excitation energies and spectroscopic factors have been compared to the first calculations of $^{15}$C with the $ab~ initio$ self-consistent Green's function method employing the NNLO$_{sat}$ interaction. The results show the sensitivity to the size of the $N=8$ shell gap and highlight the need of going beyond the current truncation scheme in the theory.

Self-predictive unsupervised learning methods such as BYOL or SimSiam have shown impressive results, and counter-intuitively, do not collapse to trivial representations. In this work, we aim at exploring the simplest possible mathematical arguments towards explaining the underlying mechanisms behind self-predictive unsupervised learning. We start with the observation that those methods crucially rely on the presence of a predictor network (and stop-gradient). With simple linear algebra, we show that when using a linear predictor, the optimal predictor is close to an orthogonal projection, and propose a general framework based on orthonormalization that enables to interpret and give intuition on why BYOL works. In addition, this framework demonstrates the crucial role of the exponential moving average and stop-gradient operator in BYOL as an efficient orthonormalization mechanism. We use these insights to propose four new \emph{closed-form predictor} variants of BYOL to support our analysis. Our closed-form predictors outperform standard linear trainable predictor BYOL at $100$ and $300$ epochs (top-$1$ linear accuracy on ImageNet).

In this paper we present the first high-resolution laser spectroscopy results obtained at the GISELE laser laboratory of the GANIL-SPIRAL2 facility, in preparation for the first experiments with the S$^3$-Low Energy Branch. Studies of neutron-deficient radioactive isotopes of erbium and tin represent the first physics cases to be studied at S$^3$. The measured isotope-shift and hyperfine structure data are presented for stable isotopes of these elements. The erbium isotopes were studied using the $4f^{12}6s^2$ $^3H_6 \rightarrow 4f^{12}(^3 H)6s6p$ $J = 5$ atomic transition (415 nm) and the tin isotopes were studied by the $5s^25p^2 (^3P_0) \rightarrow 5s^25p6s (^3P_1)$ atomic transition (286.4 nm), and are used as a benchmark of the laser setup. Additionally, the tin isotopes were studied by the $5s^25p6s (^3P_1) \rightarrow 5s^25p6p (^3P_2)$ atomic transition (811.6 nm), for which new isotope-shift data was obtained and the corresponding field-shift $F_{812}$ and mass-shift $M_{812}$ factors are presented.

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Despite its recent empirical success, such algorithms have an apparent defect: trivial representations (such as constants) minimize the prediction error, yet it is obviously undesirable to converge to such solutions. Our central insight is that careful designs of the optimization dynamics are critical to learning meaningful representations. We identify that a faster paced optimization of the predictor and semi-gradient updates on the representation, are crucial to preventing the representation collapse. Then in an idealized setup, we show self-predictive learning dynamics carries out spectral decomposition on the state transition matrix, effectively capturing information of the transition dynamics. Building on the theoretical insights, we propose bidirectional self-predictive learning, a novel self-predictive algorithm that learns two representations simultaneously. We examine the robustness of our theoretical insights with a number of small-scale experiments and showcase the promise of the novel representation learning algorithm with large-scale experiments.

Van der Waals assembly enables exquisite design of electronic states in two-dimensional (2D) materials, often by superimposing a long-wavelength periodic potential on a crystal lattice using moir\'e superlattices. Here we show that electronic states in three-dimensional (3D) crystals such as graphite can also be tuned by the superlattice potential arising at the interface with another crystal, namely, crystallographically aligned hexagonal boron nitride. Such alignment is found to result in a multitude of Lifshitz transitions and Brown-Zak oscillations for near-surface 2D states whereas, in high magnetic fields, fractal states of Hofstadter's butterfly extend deep into graphite's bulk. Our work shows a venue to control 3D spectra by using the approach of 2D twistronics.

Using Landau level spectroscopy, we determine the temperature dependence of the energy band gap in zirconium pentatelluride (ZrTe$_5$). We find that the band gap reaches $E_g=(5 \pm 1)$ meV at low temperatures and increases monotonously when the temperature is raised. This implies that ZrTe$_5$ is a weak topological insulator, with non-inverted ordering of electronic bands in the center of the Brillouin zone. Our magneto-transport experiments performed in parallel show that the resistivity anomaly in ZrTe$_5$ is not connected with the temperature dependence of the band gap., Comment: 6 pages, 2 figures

Microwave absorption experiments employing a phase-sensitive external resistive detection are performed for a topical van der Waals antiferromagnet CrSBr. The field dependence of two resonance modes is measured in an applied field parallel to the three principal crystallographic directions, revealing anisotropies and magnetic transitions in this material. To account for the observed results, we formulate a microscopic spin model with a bi-axial single-ion anisotropy and inter-plane exchange. Theoretical calculations give an excellent description of full magnon spectra enabling us to precisely determine microscopic interaction parameters for CrSBr., Comment: includes a supplementary information document

We report the development of a multileaf collimator (MLC) for charged particle beams, based on independently actuated tungsten strips which can selectively scatter unwanted particles. The MLC is used in conjunction with an emittance exchange beamline to rapidly generate highly variable longitudinal bunch profiles. The developed MLC consists of 40 independent leaves that are 2 mm wide and can move up to 10 mm, and operates in an ultra high vacuum environment, enabled by novel features such as magnetically coupled actuation. An experiment at the Argonne Wakefield Accelerator, which previously used inflexible, laser-cut masks for beam shaping before an emittance exchange beamline, was conducted to test functionality. The experiment demonstrated myriad transverse mask silhouettes, as measured on a scintillator downstream of the MLC and the corresponding longitudinal profiles after emittance exchange, as measured using a transverse deflecting cavity. Rapidly changing between mask shapes enables expeditious execution of various experiments without the downtime associated with traditional methods. The many degrees of freedom of the MLC can enable optimization of experimental figures of merit using feed-forward control and advanced machine learning methods., Comment: 7 pages, 7 figures

To achieve Ampere-class electron beam accelerators the pulse delivery rate need to be much higher than the typical photo injector repetition rate of the order of a few kilohertz. We propose here an injector which can, in principle, generate electron bunches at the same rate as the operating RF frequency. A conduction-cooled superconducting radio frequency (SRF) cavity operating in the CW mode and housing a field emission element at its region of high axial electric field can be a viable method of generating high repetition-rate electron bunches. In this paper, we report the development and experiments on a conduction-cooled Nb3Sn cavity with a niobium rod intended as a field emitter support. The initial experiments demonstrate 0.4 MV/m average accelerating gradient, which is equivalent of peak gradient of 3.2 MV/m. The measured RF cavity quality factor is 1.4 x 108 slightly above our goal. The achieved field gradient is limited by the relatively low input RF power and by the poor coupling between the external power supply and the RF cavity. With ideal coupling the field gradient can be as high as 0.6 MV/m still below our goal of about 1 MV/m.

Electron sources are essential to an array of electron accelerator supporting research in high-energy physics and beyond. This report summarizes the "Snowmass 2021 Electron Source Workshop" which reviewed the current state-of-the art research and identified some possible research directions.

We introduce DeepNash, an autonomous agent capable of learning to play the imperfect information game Stratego from scratch, up to a human expert level. Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. This popular game has an enormous game tree on the order of $10^{535}$ nodes, i.e., $10^{175}$ times larger than that of Go. It has the additional complexity of requiring decision-making under imperfect information, similar to Texas hold'em poker, which has a significantly smaller game tree (on the order of $10^{164}$ nodes). Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. Episodes are long, with often hundreds of moves before a player wins, and situations in Stratego can not easily be broken down into manageably-sized sub-problems as in poker. For these reasons, Stratego has been a grand challenge for the field of AI for decades, and existing AI methods barely reach an amateur level of play. DeepNash uses a game-theoretic, model-free deep reinforcement learning method, without search, that learns to master Stratego via self-play. The Regularised Nash Dynamics (R-NaD) algorithm, a key component of DeepNash, converges to an approximate Nash equilibrium, instead of 'cycling' around it, by directly modifying the underlying multi-agent learning dynamics. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform, competing with human expert players.