Search

The challenge of mapping AI architectures to GPU hardware is creating a critical bottleneck in AI progress. Despite substantial efforts, hand-written custom kernels fail to meet their theoretical performance thresholds, even on well-established operations like linear attention. The diverse hardware capabilities of GPUs might suggest that we need a wide variety of techniques to achieve high performance. However, our work explores whether a small number of key abstractions can drastically simplify the process. We present ThunderKittens (TK), a framework for writing performant AI kernels while remaining easy to use and maintain. Our abstractions map to the three levels of the GPU hierarchy: (1) at the warp-level, we provide 16x16 matrix tiles as basic data structures and PyTorch-like parallel compute operations over tiles, (2) at the thread-block level, we provide a template for overlapping asynchronous operations across parallel warps, and (3) at the grid-level, we provide support to help hide the block launch and tear-down, and memory costs. We show the value of TK by providing kernels that match or outperform prior kernels for a range of AI operations. We match CuBLAS and FlashAttention-3 on GEMM and attention inference performance and outperform the strongest baselines by $10-40\%$ on attention backwards, $8\times$ on state space models, and $14\times$ on linear attention.

The search for accreted satellites in the Galactic disk is a challenging task, to which Gaia plays a crucial role in synergy with ground-based spectroscopic surveys. In 2021, Re Fiorentin et al. discovered five substructures with disk kinematics including Icarus. To gain more insight into the origin of Icarus as a remnant of a dwarf galaxy rather than a signature of secular processes of disk formation, we complement astrometric Gaia DR3 data with spectroscopy from APOGEE DR17 and GALAH DR3, and explore the chemo-dynamical distributions within 3 kpc of the Sun. We select 622 stars in the accreted/unevolved regions of [Mg/Mn]-[Al/Fe] and [Mg/Fe]-[Fe/H], where we identify 81 and 376 stars with $-2<{\rm[Fe/H]}<-0.7$ belonging to Icarus and Gaia-Sausage-Enceladus (GSE), respectively. The revised properties of Icarus are: $\langle V+V_{\rm LSR}\rangle\simeq171~\rm{km~s}^{-1}$, $\sigma_{V}\simeq37\rm{km~s}^{-1}$, $\langle e\rangle\simeq0.36$, $\langle{\rm[Fe/H]}\rangle\simeq-1.35$, $\langle{\rm[Mg/Fe]}\rangle\simeq+0.27$, $\langle{\rm[Al/Fe]}\rangle\simeq-0.13$, and $\langle{\rm[Mn/Fe]}\rangle\simeq-0.39$. From the CMD of its members, Icarus appears older than 12 Gyr. Such age and dynamical properties are reminiscent of the metal-weak thick disk. However, detailed chemical analysis in the diagnostic spaces [Ni/Fe]-[(C+N)/O], [Y/Eu]-[Fe/H], [Eu/Mg]-[Fe/H], [Ba/Y]-[Fe/H], and [Ba/Mg]-[Mg/H] evidences that Icarus and GSE occupy the accreted region, well separated from the bulk of in situ disk stars. Updated comparisons with N-body simulations confirm that Icarus' stars are consistent with the debris of a dwarf galaxy with a stellar mass of $\sim~10^9~M_\odot$ accreted onto a primordial disk on an initial prograde low-inclination orbit., Comment: 18 pages, 8 figures, 3 Tables. Accepted for publication into The Astrophysical Journal

Among the great mysteries that physics has not yet solved are undoubtedly those of dark energy and dark matter. In this chapter we deal with the first of them. We will expound in detail the motivations that led to hypothesise the existence of dark energy, the importance of understanding its nature, its main possible theoretical explanations and a list of the many attempts at modelling it. We conclude with a description of the most recent and future missions on Earth and in space devised to shed light on this mystery., Comment: 35 pages

Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear attention -- avoiding the expensive pretraining costs. However, linearizing LLMs often significantly degrades model quality, still requires training over billions of tokens, and remains limited to smaller 1.3B to 7B LLMs. We thus propose Low-rank Linear Conversion via Attention Transfer (LoLCATs), a simple two-step method that improves LLM linearizing quality with orders of magnitudes less memory and compute. We base these steps on two findings. First, we can replace an LLM's softmax attentions with closely-approximating linear attentions, simply by training the linear attentions to match their softmax counterparts with an output MSE loss ("attention transfer"). Then, this enables adjusting for approximation errors and recovering LLM quality simply with low-rank adaptation (LoRA). LoLCATs significantly improves linearizing quality, training efficiency, and scalability. We significantly reduce the linearizing quality gap and produce state-of-the-art subquadratic LLMs from Llama 3 8B and Mistral 7B v0.1, leading to 20+ points of improvement on 5-shot MMLU. Furthermore, LoLCATs does so with only 0.2% of past methods' model parameters and 0.4% of their training tokens. Finally, we apply LoLCATs to create the first linearized 70B and 405B LLMs (50x larger than prior work). When compared with prior approaches under the same compute budgets, LoLCATs significantly improves linearizing quality, closing the gap between linearized and original Llama 3.1 70B and 405B LLMs by 77.8% and 78.1% on 5-shot MMLU., Comment: 47 pages, 20 figures, 18 tables, preprint

Large Language Models (LLMs) have the potential to automate reward engineering by leveraging their broad domain knowledge across various tasks. However, they often need many iterations of trial-and-error to generate effective reward functions. This process is costly because evaluating every sampled reward function requires completing the full policy optimization process for each function. In this paper, we introduce an LLM-driven reward generation framework that is able to produce state-of-the-art policies on the challenging Bi-DexHands benchmark with 20x fewer reward function samples than the prior state-of-the-art work. Our key insight is that we reduce the problem of generating task-specific rewards to the problem of coarsely estimating task progress. Our two-step solution leverages the task domain knowledge and the code synthesis abilities of LLMs to author progress functions that estimate task progress from a given state. Then, we use this notion of progress to discretize states, and generate count-based intrinsic rewards using the low-dimensional state space. We show that the combination of LLM-generated progress functions and count-based intrinsic rewards is essential for our performance gains, while alternatives such as generic hash-based counts or using progress directly as a reward function fall short., Comment: 26 pages, 5 figures

Vector Quantized Variational AutoEncoders (VQ-VAEs) are designed to compress a continuous input to a discrete latent space and reconstruct it with minimal distortion. They operate by maintaining a set of vectors -- often referred to as the codebook -- and quantizing each encoder output to the nearest vector in the codebook. However, as vector quantization is non-differentiable, the gradient to the encoder flows around the vector quantization layer rather than through it in a straight-through approximation. This approximation may be undesirable as all information from the vector quantization operation is lost. In this work, we propose a way to propagate gradients through the vector quantization layer of VQ-VAEs. We smoothly transform each encoder output into its corresponding codebook vector via a rotation and rescaling linear transformation that is treated as a constant during backpropagation. As a result, the relative magnitude and angle between encoder output and codebook vector becomes encoded into the gradient as it propagates through the vector quantization layer and back to the encoder. Across 11 different VQ-VAE training paradigms, we find this restructuring improves reconstruction metrics, codebook utilization, and quantization error. Our code is available at https://github.com/cfifty/rotation_trick.

Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while LLM-generated data is less labor-intensive, it may violate user privacy agreements or terms of service of LLM providers. Therefore, we seek a way of constructing instruction datasets with samples that are not generated by humans or LLMs but still improve LLM generative capabilities. In this work, we introduce Cookbook, a framework that programmatically generates training data consisting of simple patterns over random tokens, resulting in a scalable, cost-effective approach that avoids legal and privacy issues. First, Cookbook uses a template -- a data generating Python function -- to produce training data that encourages the model to learn an explicit pattern-based rule that corresponds to a desired task. We find that fine-tuning on Cookbook-generated data is able to improve performance on its corresponding task by up to 52.7 accuracy points. Second, since instruction datasets improve performance on multiple downstream tasks simultaneously, Cookbook algorithmically learns how to mix data from various templates to optimize performance on multiple tasks. On the standard multi-task GPT4ALL evaluation suite, Mistral-7B fine-tuned using a Cookbook-generated dataset attains the best accuracy on average compared to other 7B parameter instruction-tuned models and is the best performing model on 3 out of 8 tasks. Finally, we analyze when and why Cookbook improves performance and present a metric that allows us to verify that the improvement is largely explained by the model's generations adhering better to template rules., Comment: COLM 2024

Inference-time techniques are emerging as highly effective tools to enhance large language model (LLM) capabilities. However, best practices for developing systems that combine these techniques remain underdeveloped due to our limited understanding of the utility of individual inference-time techniques and the interactions between them. Additionally, efficiently and automatically searching the space of model choices, inference-time techniques, and their compositions is challenging due to the large design space. To address these challenges, we introduce Archon, a modular framework for selecting, combining, and stacking layers of inference-time techniques to construct optimized LLM systems for target benchmarks. Rather than relying on a single LLM called once, we leverage a diverse set of LLMs and inference-time techniques, creating LLM systems greater than the sum of their parts. Archon defines an extensible design space, encompassing techniques such as generation ensembling, repeated sampling, ranking, fusion, critiquing, verification, and unit testing. It transforms the problem of building LLM systems into a hyperparameter optimization objective. Given the available LLMs, inference-time techniques, and compute budget, Archon utilizes hyperparameter search techniques to discover optimized architectures for target benchmark(s). We evaluate Archon architectures across a range of instruction-following, reasoning, and coding benchmarks, including MT-Bench, Arena-Hard-Auto, AlpacaEval 2.0, MixEval, MixEval Hard, MATH, and CodeContests. Archon architectures outperform frontier models, such as GPT-4o and Claude 3.5 Sonnet, on these benchmarks, achieving an average accuracy increase of 15.1 percentage points by using all available LLMs. We make our code and datasets available publicly on Github: https://github.com/ScalingIntelligence/Archon.

The control of quantum systems has been proven to possess trap-free optimization landscapes under the satisfaction of proper assumptions. However, many details of the landscape geometry and their influence on search efficiency still need to be fully understood. This paper numerically explores the path-connectedness of globally optimal control solutions forming the top manifold of the landscape. We randomly sample a plurality of optimal controls in the top manifold to assess the existence of a continuous path at the top of the landscape that connects two arbitrary optimal solutions. It is shown that for different quantum control objectives including state-to-state transition probabilities, observable expectation values and unitary transformations, such a continuous path can be readily found, implying that these top manifolds are fundamentally path-connected. The significance of the latter conjecture lies in seeking locations in the top manifold where an ancillary objective can also be optimized while maintaining the full optimality of the original objective that defined the landscape., Comment: 34 pages, 10 figures

Two-dimensional crystals stack together through weak van der Waals (vdW) forces, offering unlimited possibilities to play with layer number, order and twist angle in vdW heterostructures (HSs). The realisation of high-performance optoelectronic devices, however, requires the achievement of specific band alignments, $k$-space matching between conduction band minima and valence band maxima, as well as efficient charge transfer between the constituent layers. Fine tuning mechanisms to design ideal HSs are lacking. Here, we show that layer-selective strain engineering can be exploited as an extra degree of freedom in vdW HSs to tailor their band alignment and optical properties. To that end, strain is selectively applied to MS$_2$ (M=Mo,W) monolayers in InSe/MS$_2$ HSs. This triggers a giant PL enhancement of the highly tuneable but weakly emitting InSe by one to three orders of magnitude. Resonant PL excitation measurements, supported by first-principle calculations, provide evidence of a strain-activated direct charge transfer from the MS$_2$ MLs toward InSe. This significant emission enhancement achieved for InSe widens its range of applications for optoelectronics.

Research conducted previously has focused on either attitudes toward or behaviors associated with autonomous driving. In this paper, we bridge these two dimensions by exploring how attitudes towards autonomous driving influence behavior in an autonomous car. We conducted a field experiment with twelve participants engaged in non-driving related tasks. Our findings indicate that attitudes towards autonomous driving do not affect participants' driving interventions in vehicle control and eye glance behavior. Therefore, studies on autonomous driving technology lacking field tests might be unreliable for assessing the potential behaviors, attitudes, and acceptance of autonomous vehicles., Comment: 12 pages

Electrochemical reduction is a promising way to make use of CO$_2$ as feedstock for generating renewable fuel and valuable chemicals. Several metals can be used in the electrocatalyst to generate CO and formic acid but hydrogen formation is an unwanted side reaction that can even be dominant. The lack of selectivity is in general a significant problem, but silver-based electrocatalysts have been shown to be highly selective for CO with over over 90% faradaic efficiency when the applied voltage is below -1 V vs. RHE. Hydrogen formation is then insignificant and little formate is formed even though it is thermodynamically favored. We present calculations of the activation free energy for the various elementary steps as a function of applied voltage at the three low index facets, Ag(111), Ag(100) and Ag(110), as well as experimental measurements on polycrystalline electrodes, to identify the reason for this high selectivity. The formation of formic acid is suppressed because of the low coverage of adsorbed hydrogen and kinetic hindrance to the formation of the HCOO* intermediate, while *COOH, a key intermediate in CO formation, is thermodynamically unstable until the applied voltage reaches -1 V vs. RHE, at which point the kinetics for its formation are more favorable than for hydrogen. The calculated results are consistent with experimental measurements carried out for acidic conditions and provide an atomic scale insight into the high CO selectivity of silver-based electrocatalysts.

We explore a two-dimensional Hubbard model adapted to host altermagnetic states. Utilizing Hartree-Fock (HF) and dynamical mean field theory (DMFT), we uncover that the magnetic solutions of this model feature Dirac points in their spectrum. HF predicts a gap opening at a critical interaction strength, a result corroborated by DMFT calculations at zero temperature. However, at finite temperature and high interaction strengths, Dirac points re-emerge at high energies in the spectral function, even if they are absent in the non-interacting and HF-predicted band structures. Analytical arguments reveal that this phenomenon arises near Mott insulating solutions from the frequency dependence of the self-energy, a behavior not captured by static mean-field theory. We also propose perturbations to the model that can induce a topological gap in the spectrum, leading to a transition from topologically trivial to non-trivial states by varying doping, interaction strength, and temperature.

As autonomous vehicle technology advances, the precise assessment of safety in complex traffic scenarios becomes crucial, especially in mixed-vehicle environments where human perception of safety must be taken into account. This paper presents a framework designed for assessing traffic safety in multi-vehicle situations, facilitating the simultaneous utilization of diverse objective safety metrics. Additionally, it allows the integration of subjective perception of safety by adjusting model parameters. The framework was applied to evaluate various model configurations in car-following scenarios on a highway, utilizing naturalistic driving datasets. The evaluation of the model showed an outstanding performance, particularly when integrating multiple objective safety measures. Furthermore, the performance was significantly enhanced when considering all surrounding vehicles.

A new Newtonian limit of general relativity is established for stationary axisymmetric gravitationally bound differentially rotating matter distributions with internal pressure. The self-consistent coupling of quasilocal gravitational energy and angular momentum leads to a modified Poisson equation. The coupled equations of motion of the effective fluid elements are also modified, with quasilocal angular momentum and frame-dragging leading to novel dynamics. The solutions of the full system reproduce the phenonomenology of collisionless dark matter for disc galaxies, offering an explanation for their observed rotation curves. Halos of abundant cold dark matter particles are not required., Comment: 6 pages, 2 figures

Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit the amount of compute to only one attempt per problem. Here, we explore inference compute as another axis for scaling by increasing the number of generated samples. Across multiple tasks and models, we observe that coverage - the fraction of problems solved by any attempt - scales with the number of samples over four orders of magnitude. In domains like coding and formal proofs, where all answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-V2-Coder-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-attempt state-of-the-art of 43% which uses more capable frontier models. Moreover, using current API pricing, amplifying the cheaper DeepSeek model with five samples is more cost-effective and solves more issues than paying a premium for one sample from GPT-4o or Claude 3.5 Sonnet. Interestingly, the relationship between coverage and the number of samples is often log-linear and can be modelled with an exponentiated power law, suggesting the existence of inference-time scaling laws. Finally, we find that identifying correct samples out of many generations remains an important direction for future research in domains without automatic verifiers. When solving math word problems from GSM8K and MATH, coverage with Llama-3 models grows to over 95% with 10,000 samples. However, common methods to pick correct solutions from a sample collection, such as majority voting or reward models, plateau beyond several hundred samples and fail to fully scale with the sample budget.

The Southern Photometric Local Universe Survey (S-PLUS) is a project to map $\sim9300$ sq deg of the sky using twelve bands (seven narrow and five broadbands). Observations are performed with the T80-South telescope, a robotic telescope located at the Cerro Tololo Observatory in Chile. The survey footprint consists of several large contiguous areas, including fields at high and low galactic latitudes, and towards the Magellanic Clouds. S-PLUS uses fixed exposure times to reach point source depths of about $21$ mag in the $griz$ and $20$ mag in the $u$ and the narrow filters. This paper describes the S-PLUS Data Release 4 (DR4), which includes calibrated images and derived catalogues for over 3000 sq deg, covering the aforementioned area. The catalogues provide multi-band photometry performed with the tools \texttt{DoPHOT} and \texttt{SExtractor} -- point spread function (\PSF) and aperture photometry, respectively. In addition to the characterization, we also present the scientific potential of the data. We use statistical tools to present and compare the photometry obtained through different methods. Overall we find good agreement between the different methods, with a slight systematic offset of 0.05\,mag between our \PSF and aperture photometry. We show that the astrometry accuracy is equivalent to that obtained in previous S-PLUS data releases, even in very crowded fields where photometric extraction is challenging. The depths of main survey (MS) photometry for a minimum signal-to-noise ratio $S/N = 3$ reach from $\sim19.5$ for the bluer bands to $\sim21.5$ mag on the red. The range of magnitudes over which accurate \PSF photometry is obtained is shallower, reaching $\sim19$ to $\sim20.5$ mag depending on the filter. Based on these photometric data, we provide star-galaxy-quasar classification and photometric redshift for millions of objects., Comment: 26 pages, 17 figures, 14 tables, accepted for A&A

Federated Learning (FL) is a distributed machine learning paradigm facilitating participants to collaboratively train a model without revealing their local data. However, when FL is deployed into the wild, some intelligent clients can deliberately deviate from the standard training process to make the global model inclined toward their local model, thereby prioritizing their local data distribution. We refer to this novel category of misbehaving clients as selfish. In this paper, we propose a Robust aggregation strategy for FL server to mitigate the effect of Selfishness (in short RFL-Self). RFL-Self incorporates an innovative method to recover (or estimate) the true updates of selfish clients from the received ones, leveraging robust statistics (median of norms) of the updates at every round. By including the recovered updates in aggregation, our strategy offers strong robustness against selfishness. Our experimental results, obtained on MNIST and CIFAR-10 datasets, demonstrate that just 2% of clients behaving selfishly can decrease the accuracy by up to 36%, and RFL-Self can mitigate that effect without degrading the global model performance., Comment: 10 pages, 16 figures. European Conference on Artificial Intelligence (ECAI) 2024

Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use a constant amount of memory during inference. However, due to the limited memory, recurrent LMs cannot recall and use all the information in long contexts leading to brittle in-context learning (ICL) quality. A key challenge for efficient LMs is selecting what information to store versus discard. In this work, we observe the order in which information is shown to the LM impacts the selection difficulty. To formalize this, we show that the hardness of information recall reduces to the hardness of a problem called set disjointness (SD), a quintessential problem in communication complexity that requires a streaming algorithm (e.g., recurrent model) to decide whether inputted sets are disjoint. We empirically and theoretically show that the recurrent memory required to solve SD changes with set order, i.e., whether the smaller set appears first in-context. Our analysis suggests, to mitigate the reliance on data order, we can put information in the right order in-context or process prompts non-causally. Towards that end, we propose: (1) JRT-Prompt, where context gets repeated multiple times in the prompt, effectively showing the model all data orders. This gives $11.0 \pm 1.3$ points of improvement, averaged across $16$ recurrent LMs and the $6$ ICL tasks, with $11.9\times$ higher throughput than FlashAttention-2 for generation prefill (length $32$k, batch size $16$, NVidia H100). We then propose (2) JRT-RNN, which uses non-causal prefix-linear-attention to process prompts and provides $99\%$ of Transformer quality at $360$M params., $30$B tokens and $96\%$ at $1.3$B params., $50$B tokens on average across the tasks, with $19.2\times$ higher throughput for prefill than FA2.

The CUbesat Solar Polarimeter (CUSP) project is a future CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band, by means of a Compton scattering polarimeter. CUSP will allow us to study the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. The project is in the framework of the Italian Space Agency Alcor Program, which aims to develop new CubeSat missions. CUSP is approved for a Phase B study that will last for 12 months, starting in mid-2024. We report on the current status of the CUSP mission project as the outcome of the Phase A., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024. arXiv admin note: substantial text overlap with arXiv:2208.06211

The CUbesat Solar Polarimeter (CUSP) project is a CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter. CUSP will allow the study of the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. CUSP is a project in the framework of the Alcor Program of the Italian Space Agency aimed at developing new CubeSat missions. It is approved for a Phase B study. In this work, we report on the characterization of the Avalanche Photodiodes (APDs) that will be used as readout sensors of the absorption stage of the Compton polarimeter. We assessed the APDs gain and energy resolution as a function of temperature by irradiating the sensor with a \textsuperscript{55}Fe radioactive source. Moreover, the APDs were also characterized as being coupled to a GAGG scintillator., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

The CUbesat Solar Polarimeter (CUSP) project aims to develop a constellation of two CubeSats orbiting the Earth to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter on board of each satellite. CUSP will allow to study the magnetic reconnection and particle acceleration in the flaring magnetic structures. CUSP is a project approved for a Phase B study by the Italian Space Agency in the framework of the Alcor program aimed to develop CubeSat technologies and missions. In this paper we describe the a method for a multi-physical simulation analysis while analyzing some possible design optimization of the payload design solutions adopted. In particular, we report the mechanical design for each structural component, the results of static and dynamic finite element analysis, the preliminary thermo-mechanical analysis for two specific thermal cases (hot and cold orbit) and a topological optimization of the interface between the platform and the payload., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

The CUbesat Solar Polarimeter (CUSP) project is a CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter. CUSP will allow to study the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. CUSP is a project in the framework of the Alcor Program of the Italian Space Agency aimed to develop new CubeSat missions. It is approved for a Phase B study. In this work, we report on the accurate simulation of the detector's response to evaluate the scientific performance. A GEANT4 Monte Carlo simulation is used to assess the physical interactions of the source photons with the detector and the passive materials. Using this approach, we implemented a detailed CUSP Mass Model. In this work, we report on the evaluation of the detector's effective area as a function of the beam energy., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

We describe the planned near-term and potential longer-term upgrades of the Belle II detector at the SuperKEKB electron-positron collider operating at the KEK laboratory in Tsukuba, Japan. These upgrades will allow increasingly sensitive searches for possible new physics beyond the Standard Model in flavor, tau, electroweak and dark sector physics that are both complementary to and competitive with the LHC and other experiments., Comment: Editor: F. Forti 170 pages

We introduce MindSpore Quantum, a pioneering hybrid quantum-classical framework with a primary focus on the design and implementation of noisy intermediate-scale quantum (NISQ) algorithms. Leveraging the robust support of MindSpore, an advanced open-source deep learning training/inference framework, MindSpore Quantum exhibits exceptional efficiency in the design and training of variational quantum algorithms on both CPU and GPU platforms, delivering remarkable performance. Furthermore, this framework places a strong emphasis on enhancing the operational efficiency of quantum algorithms when executed on real quantum hardware. This encompasses the development of algorithms for quantum circuit compilation and qubit mapping, crucial components for achieving optimal performance on quantum processors. In addition to the core framework, we introduce QuPack, a meticulously crafted quantum computing acceleration engine. QuPack significantly accelerates the simulation speed of MindSpore Quantum, particularly in variational quantum eigensolver (VQE), quantum approximate optimization algorithm (QAOA), and tensor network simulations, providing astonishing speed. This combination of cutting-edge technologies empowers researchers and practitioners to explore the frontiers of quantum computing with unprecedented efficiency and performance.

Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, measuring, improving, and automating enterprise workflows. However, research has focused almost exclusively on one task - full end-to-end automation using agents based on multimodal foundation models (FMs) like GPT-4. This focus on automation ignores the reality of how most BPM tools are applied today - simply documenting the relevant workflow takes 60% of the time of the typical process optimization project. To address this gap we present WONDERBREAD, the first benchmark for evaluating multimodal FMs on BPM tasks beyond automation. Our contributions are: (1) a dataset containing 2928 documented workflow demonstrations; (2) 6 novel BPM tasks sourced from real-world applications ranging from workflow documentation to knowledge transfer to process improvement; and (3) an automated evaluation harness. Our benchmark shows that while state-of-the-art FMs can automatically generate documentation (e.g. recalling 88% of the steps taken in a video demonstration of a workflow), they struggle to re-apply that knowledge towards finer-grained validation of workflow completion (F1 < 0.3). We hope WONDERBREAD encourages the development of more "human-centered" AI tooling for enterprise applications and furthers the exploration of multimodal FMs for the broader universe of BPM tasks. We publish our dataset and experiments here: https://github.com/HazyResearch/wonderbread

Several neutrino detectors, KamLAND, Daya Bay, Double Chooz, RENO, and the forthcoming large-scale JUNO, rely on liquid scintillator to detect reactor antineutrino interactions. In this context, inverse beta decay represents the golden channel for antineutrino detection, providing a pair of correlated events, thus a strong experimental signature to distinguish the signal from a variety of backgrounds. However, given the low cross-section of antineutrino interactions, the development of a powerful event selection algorithm becomes imperative to achieve effective discrimination between signal and backgrounds. In this study, we introduce a machine learning (ML) model to achieve this goal: a fully connected neural network as a powerful signal-background discriminator for a large liquid scintillator detector. We demonstrate, using the JUNO detector as an example, that, despite the already high efficiency of a cut-based approach, the presented ML model can further improve the overall event selection efficiency. Moreover, it allows for the retention of signal events at the detector edges that would otherwise be rejected because of the overwhelming amount of background events in that region. We also present the first interpretable analysis of the ML approach for event selection in reactor neutrino experiments. This method provides insights into the decision-making process of the model and offers valuable information for improving and updating traditional event selection approaches.

The optical and radiochemical purification of the scintillating liquid, which will fill the central detector of the JUNO experiment, plays a crucial role in achieving its scientific goals. Given its gigantic mass and dimensions and an unprecedented target value of about 3% @ 1 MeV in energy resolution, JUNO has set severe requirements on the parameters of its scintillator, such as attenuation length (Lat>20 m at 430 nm), transparency, light yield, and content of radioactive contaminants (238U,232Th<10-15 g/g). To accomplish these needs, the scintillator will be processed using several purification methods, including distillation in partial vacuum and gas stripping, which are performed in two large scale plants installed at the JUNO site. In this paper, layout, operating principles, and technical aspects which have driven the design and construction of the distil- lation and gas stripping plants are reviewed. The distillation is effective in enhancing the optical properties and removing heavy radio-impurities (238U,232Th, 40K), while the stripping process exploits pure water steam and high-purity nitrogen to extract gaseous contaminants (222Rn, 39Ar, 85Kr, O2) from the scintillator. The plant operating parameters have been tuned during the recent com- missioning phase at the JUNO site and several QA/QC measurements and tests have been performed to evaluate the performances of the plants. Some preliminary results on the efficiency of these purification processes will be shown., Comment: 11 pages, 7 figures

This report presents a short summary of the activities of the "Standard Model" working group for the "Physics at TeV Colliders" workshop (Les Houches, France, 12-30 June, 2023)., Comment: Proceedings of the Standard Model Working Group of the 2023 Les Houches Workshop, Physics at TeV Colliders, Les Houches 12-30 June 2023. 48 pages

The acquisition and analysis of high-quality sensor data constitute an essential requirement in shaping the development of fully autonomous driving systems. This process is indispensable for enhancing road safety and ensuring the effectiveness of the technological advancements in the automotive industry. This study introduces the Interaction of Autonomous and Manually-Controlled Vehicles (IAMCV) dataset, a novel and extensive dataset focused on inter-vehicle interactions. The dataset, enriched with a sophisticated array of sensors such as Light Detection and Ranging, cameras, Inertial Measurement Unit/Global Positioning System, and vehicle bus data acquisition, provides a comprehensive representation of real-world driving scenarios that include roundabouts, intersections, country roads, and highways, recorded across diverse locations in Germany. Furthermore, the study shows the versatility of the IAMCV dataset through several proof-of-concept use cases. Firstly, an unsupervised trajectory clustering algorithm illustrates the dataset's capability in categorizing vehicle movements without the need for labeled training data. Secondly, we compare an online camera calibration method with the Robot Operating System-based standard, using images captured in the dataset. Finally, a preliminary test employing the YOLOv8 object-detection model is conducted, augmented by reflections on the transferability of object detection across various LIDAR resolutions. These use cases underscore the practical utility of the collected dataset, emphasizing its potential to advance research and innovation in the area of intelligent vehicles.

Though early childhood educators have a long history of emphasizing learning through play to foster the development of the whole child, including social-emotional competencies, the proliferation of formal social-emotional learning (SEL) curricula first seen in kindergarten to 12th grade has expanded to prekindergarten. This report presents initial implementation findings of the Positive Emotional Development and Learning Skills (PEDALS) program, an SEL program in prekindergarten classrooms in Western and Central New York and Southeast Michigan. PEDALS is a two-year program that combines the Second Step Early Learning SEL curriculum with comprehensive supports for teachers and site directors. Findings indicate that educators perceived the Second Step curriculum, PEDALS training, and PEDALS coaching support very favorably; strongly believed in supporting children's social-emotional development; and had a strong sense of self-efficacy for doing so. Observations indicated that educators were highly skilled in implementing Second Step lessons; however, teachers used strategies less consistently when children were having a difficult time or experiencing interpersonal conflict. Educators identified PEDALS resources for developing children's social-emotional competencies as the factors that provided the most implementation support; meanwhile, they identified some contextual factors, such as children's challenging behavior and lack of planning time, as the factors that most hindered implementation. Preliminary findings suggest that many preconditions for effective SEL instruction were met. This implementation study is a part of a larger evaluation of PEDALS that also will estimate the effect of the program on children's SEL and executive functioning skills. A second report will be published at the study's conclusion. [Additional funding from the Health Foundation for Western & Central New York.]

In the field of rare event physics, it is common to have huge masses of organic liquid scintillator as detection medium. In particular, they are widely used to study neutrino properties or astrophysical neutrinos. Thanks to its safety properties (such as low toxicity and high flash point) and easy scalability, linear alkyl benzene is the most common solvent used to produce liquid scintillators for large mass experiments. The knowledge of the refractive index is a pivotal point to understand the detector response, as this quantity (and its wavelength dependence) affects the Cherenkov radiation and photon propagation in the medium. In this paper, we report the measurement of the refractive index of the JUNO liquid scintillator between 260-1064 nm performed with two different methods (an ellipsometer and a refractometer), with a sub percent level precision. In addition, we used an interferometer to measure the group velocity in the JUNO liquid scintillator and verify the expected value derived from the refractive index measurements., Comment: 6 pages, 9 figures

We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 \nu$ or $nn \rightarrow 2 \nu$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\bar{\nu}_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $\tau/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $\tau/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$., Comment: 28 pages, 7 figures, 4 tables

Pulse-based quantum machine learning (QML) models possess full expressivity when they are ensemble controllable. However, it has also been shown that barren plateaus emerge in such models, rendering training intractable for systems with large dimension. In this paper, we show that the trade-off is closely related to the controllability of the underlying pulse-based models. We first apply the Fliess-series expansion to pulse-based QML models to investigate the effect of control system structure on model expressivity, which leads to a universal criterion for assessing the expressivity of generic QML models. Guided by this criterion, we then demonstrate how designing pulse-based models on low-dimensional manifolds can balance expressivity and trainability. Finally, numerical experiments are carried out to verify the proposed criterion and our analysis, which futher demonstrate that increasing dimensionality enhances expressivity but avoids barren plateaus if the model is designed with limited controllability on a submanifold. Our approach provides a promising path for designing pulse-based QML models that are both highly expressive and trainable.

In research evaluation, the fair representation of panels is usually defined in terms of observable characteristics of scholars such as gender or affiliations. An empirical strategy is proposed for exploring hidden connections between panellists such that, despite the respect of formal requirements, the panel could be considered alike as unfair with respect to the representation of diversity of research approaches and methodologies. The case study regards the three panels selected to evaluate research in economics, statistics and business during the Italian research assessment exercises. The first two panels were appointed directly by the governmental agency responsible for the evaluation, while the third was randomly selected. Hence the third panel can be considered as a control for evaluating about the fairness of the others. The fair representation is explored by comparing the networks of panellists based on their co-authorship relations, the networks based on journals in which they published and the networks based on their affiliated institutions (universities, research centres and newspapers). The results show that the members of the first two panels had connections much higher than the members of the control group. Hence the composition of the first two panels should be considered as unfair, as the results of the research assessments., Comment: 40 pages, 6 figures

We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size. We achieve this using properties of the proposed frequency domain transfer function parametrization, which enables direct computation of its corresponding convolutional kernel's spectrum via a single Fast Fourier Transform. Our experimental results across multiple sequence lengths and state sizes illustrates, on average, a 35% training speed improvement over S4 layers -- parametrized in time-domain -- on the Long Range Arena benchmark, while delivering state-of-the-art downstream performances over other attention-free approaches. Moreover, we report improved perplexity in language modeling over a long convolutional Hyena baseline, by simply introducing our transfer function parametrization. Our code is available at https://github.com/ruke1ire/RTF., Comment: Resubmission 02/06/2024: Fixed minor typo of recurrent form RTF

Automating enterprise workflows could unlock $4 trillion/year in productivity gains. Despite being of interest to the data management community for decades, the ultimate vision of end-to-end workflow automation has remained elusive. Current solutions rely on process mining and robotic process automation (RPA), in which a bot is hard-coded to follow a set of predefined rules for completing a workflow. Through case studies of a hospital and large B2B enterprise, we find that the adoption of RPA has been inhibited by high set-up costs (12-18 months), unreliable execution (60% initial accuracy), and burdensome maintenance (requiring multiple FTEs). Multimodal foundation models (FMs) such as GPT-4 offer a promising new approach for end-to-end workflow automation given their generalized reasoning and planning abilities. To study these capabilities we propose ECLAIR, a system to automate enterprise workflows with minimal human supervision. We conduct initial experiments showing that multimodal FMs can address the limitations of traditional RPA with (1) near-human-level understanding of workflows (93% accuracy on a workflow understanding task) and (2) instant set-up with minimal technical barrier (based solely on a natural language description of a workflow, ECLAIR achieves end-to-end completion rates of 40%). We identify human-AI collaboration, validation, and self-improvement as open challenges, and suggest ways they can be solved with data management techniques. Code is available at: https://github.com/HazyResearch/eclair-agents

Aims. We explore the dynamical friction on a test mass in gravitational systems in the Quasi linear formulation of Modified Newtonian Dynamics (QuMOND). Methods. Exploiting the quasi linearity of QuMOND we derive a simple expression for the dynamical friction in akin to its Newtonian counterpart in the standard Chandrasekhar derivation. Moreover, adopting a mean field approach based on the Liouville equation we obtain a more rigorous (though in integral form) dynamical friction formula that can be evaluated numerically for a given choice of the QuMOND interpolation function. Results. Consistently with previous work, we observe that dynamical friction is stronger in MOND with respect to a baryon only Newtonian system with the same mass distribution. This amounts to a correction of the Coulomb logarithmic factor via extra terms proportional to the MOND radius of the system. Moreover, with the aid of simple numerical experiments we confirm our theoretical predictions and those of previous work on MOND., Comment: 12 pages, 3 Figures. Version accepted for publication in A&A

We propose a method to trap polar molecules with the electrical force induced by the surface acoustic wave (SAW) on piezoelectric materials. In this approach, the electrical force is perpendicular to the moving direction of the polar molecules, and is used to control the positions of trapped polar molecules in the direction orthogonal to the acoustic transmission. By virtue of an external electrical force, the SAW-induced electrical field can trap the polar molecules into single-layer or multi-layer lattices. The arrangement of molecules can affect the binding energy and localization of the molecule array. Then the one- or two-dimensional trapped polar molecule arrays can be used to construct the Bose-Hubbard (BH) model, whose energy and dynamics are affected by the localizations of the trapped molecules. We find that the phase transitions between the superfluid and Mott insulator based on trapped polar molecule BH model can be modulated by the SAW induced electrical potential., Comment: 18 pages, 10 figures

Gravitational waves from black-hole merging events have revealed a population of extra-galactic BHs residing in short-period binaries with masses that are higher than expected based on most stellar evolution models - and also higher than known stellar-origin black holes in our Galaxy. It has been proposed that those high-mass BHs are the remnants of massive metal-poor stars. Gaia astrometry is expected to uncover many Galactic wide-binary systems containing dormant BHs, which may not have been detected before. The study of this population will provide new information on the BH-mass distribution in binaries and shed light on their formation mechanisms and progenitors. As part of the validation efforts in preparation for the fourth Gaia data release (DR4), we analysed the preliminary astrometric binary solutions, obtained by the Gaia Non-Single Star pipeline, to verify their significance and to minimise false-detection rates in high-mass-function orbital solutions. The astrometric binary solution of one source, Gaia BH3, implies the presence of a 32.70 \pm 0.82 M\odot BH in a binary system with a period of 11.6 yr. Gaia radial velocities independently validate the astrometric orbit. Broad-band photometric and spectroscopic data show that the visible component is an old, very metal-poor giant of the Galactic halo, at a distance of 590 pc. The BH in the Gaia BH3 system is more massive than any other Galactic stellar-origin BH known thus far. The low metallicity of the star companion supports the scenario that metal-poor massive stars are progenitors of the high-mass BHs detected by gravitational-wave telescopes. The Galactic orbit of the system and its metallicity indicate that it might belong to the Sequoia halo substructure. Alternatively, and more plausibly, it could belong to the ED-2 stream, which likely originated from a globular cluster that had been disrupted by the Milky Way., Comment: 23 pages, accepted fro publication in A&A Letters. New version with small fixes

The control of flying qubits carried by itinerant photons is ubiquitous in quantum communication networks. In addition to their logical states, the shape of flying qubits must also be tailored to match the remote receiver. In this paper, we introduce the quantum optimal control theory to the design of flying-qubit shaping protocols. A gradient-based algorithm is proposed for the generation of arbitrary-shape flying qubits with general non-ideal emitters. Simulations show that, as a joint control with the traditionally used tunable coupler, coherent driving fields can be applied to the shaping when the coupling strength is fixed or limited. The optimized control protocols can effectively suppress unwanted level leakage and multi-photon radiation. The method provides a systematic approach to high-fidelity control of flying qubits using realistic quantum devices., Comment: 9 pages,6 figures

This work presents a procedure to solve the Euler equations by explicitly updating, in a conservative manner, a generic thermodynamic variable such as temperature, pressure or entropy instead of the total energy. The presented procedure is valid for any equation of state and spatial discretization. When using complex equations of state such as Span-Wagner, choosing the temperature as the generic thermodynamic variable yields great reductions in the computational costs associated to thermodynamic evaluations. Results computed with a state of the art thermodynamic model are presented, and computational times are analyzed. Particular attention is dedicated to the conservation of total energy, the propagation speed of shock waves and jump conditions. The procedure is thoroughly tested using the Span-Wagner equation of state through the CoolProp thermodynamic library and the Van der Waals equation of state, both in the ideal and non-ideal compressible fluid-dynamics regimes, by comparing it to the standard total energy update and analytical solutions where available., Comment: Final version. Accepted version of: Computers & Fluids (2024) 279:106340 Published version available at 10.1016/j.compfluid.2024.106340

The development of deep learning architectures is a resource-demanding process, due to a vast design space, long prototyping times, and high compute costs associated with at-scale model training and evaluation. We set out to simplify this process by grounding it in an end-to-end mechanistic architecture design (MAD) pipeline, encompassing small-scale capability unit tests predictive of scaling laws. Through a suite of synthetic token manipulation tasks such as compression and recall, designed to probe capabilities, we identify and test new hybrid architectures constructed from a variety of computational primitives. We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis, training over 500 language models between 70M to 7B parameters. Surprisingly, we find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures via isolated proxy tasks. The new architectures found via MAD, based on simple ideas such as hybridization and sparsity, outperform state-of-the-art Transformer, convolutional, and recurrent architectures (Transformer++, Hyena, Mamba) in scaling, both at compute-optimal budgets and in overtrained regimes. Overall, these results provide evidence that performance on curated synthetic tasks can be predictive of scaling laws, and that an optimal architecture should leverage specialized layers via a hybrid topology.