Search

We present BabyLlama-2, a 345 million parameter model distillation-pretrained from two teachers on a 10 million word corpus for the BabyLM competition. On BLiMP and SuperGLUE benchmarks, BabyLlama-2 outperforms baselines trained on both 10 and 100 million word datasets with the same data mix, as well as its teacher models. Through an extensive hyperparameter sweep, we demonstrate that the advantages of distillation cannot be attributed to suboptimal hyperparameter selection of the teachers. Our findings underscore the need for further investigation into distillation techniques, particularly in data-limited settings., Comment: 9 pages, 3 figures, 5 tables, submitted to the BabyLM Challenge (CoNLL 2024 Shared Task)

Axion-like particles (ALPs) and heavy neutral leptons (HNLs) are two well-motivated classes of particles beyond the Standard Model. It is intriguing to explore the new detection opportunities that may arise if both particle types coexist. Part of the authors already investigated this scenario in a previous publication, within a simplified model containing an ALP and a single HNL, identifying particularly promising processes that could be searched for at the LHC. In this paper, we first consider the same setup with a broader range of both production processes and final states, both at the High-Luminosity LHC and at a future muon collider. Subsequently, we expand it to the more realistic scenario with at least two HNLs, necessary to describe the active neutrino masses. Different phenomenological signals are expected and we examine the complexities that emerge in this setup. This study paves the way for dedicated analysis at (forthcoming) colliders, potentially pinpointing the dynamics of ALPs and HNLs., Comment: 43 pages, 12 figures

The increasing interest in Long-Lived Particles (LLPs) has led to numerous proposed experiments in order to search for them. However, the sensitivity estimates published by these experiments tend to rely on disparate assumptions. To ensure an accurate comparison of their potential to find LLPs, a unified estimation of their sensitivity is therefore required. In this contribution, we introduce \texttt{SensCalc}, a \texttt{Mathematica}-based code that uses a semi-analytic approach to calculate the event rate of GeV-scale LLPs, and we present several case studies., Comment: EPS-HEP 2023 proceedings

We present our submission to the BabyLM challenge, whose goal was to improve the sample efficiency of language models. We trained an ensemble consisting of a GPT-2 and small LLaMA models on the developmentally-plausible, 10M-word BabyLM dataset, then distilled it into a small, 58M-parameter LLaMA model, which exceeds in performance both of its teachers as well as a similar model trained without distillation. This suggests that distillation can not only retain the full performance of the teacher model when the latter is trained on a sufficiently small dataset; it can exceed it, and lead to significantly better performance than direct training., Comment: 11 pages, 4 figures, 4 tables, submitted to the BabyLM Challenge and accepted as archival full paper (CoNLL--CMCL 2023 Shared Task), checkpoint available at https://huggingface.co/timinar/baby-llama-58m, training code available at https://github.com/timinar/BabyLlama

Axion-like particles (ALPs) and heavy neutral leptons (HNLs) are both well-motivated extensions of the Standard Model. As ALPs couple to on-shell fermions proportionally to their masses, processes involving both types of particles may give rise, for TeV HNLs, to relevant phenomenology at colliders. In this work, we point out a particularly clean process, whose final state consists of four jets and two charged leptons, and we estimate its current and future sensitivity at the LHC. For on-shell HNLs, i) there is no dependence on the overall scale of the mixing between HNLs and the active neutrinos, making this process sensitive down to the type-I Seesaw line; ii) the signal strength of the process is sizable only as far as the HNL masses are below a few TeVs, contrary to what the proportionality to masses of the ALP couplings may suggest. Although ALPs and HNLs have been mainly studied independently in the literature, considering their interplay may lead to joint limits that are much stronger than those on the individual particles taken separately. This concise study paves the way for similar searches at colliders and other experiments., Comment: 7 pages, 5 figures

The existence of non-zero neutrino masses points to the likely existence of multiple SM neutral fermions. When such states are heavy enough that they cannot be produced in oscillations, they are referred to as Heavy Neutral Leptons (HNLs). In this white paper we discuss the present experimental status of HNLs including colliders, beta decay, accelerators, as well as astrophysical and cosmological impacts. We discuss the importance of continuing to search for HNLs, and its potential impact on our understanding on key fundamental questions, and additionally we outline the future prospects for next-generation future experiments or upcoming accelerator run scenarios., Comment: 82 pages, 34 figures. Contribution to Snowmass 2021