Search

In the last decades multifaceted manifestations of the breakdown of the self-consistent perturbation theory have been identified for the many-electron problem. Yet, the investigations have been so far mostly limited to paramagnetic states, where symmetry breaking is not allowed. Here, we extend the analysis to the spontaneously symmetry-broken antiferromagnetic (AF) phase of the repulsive Hubbard model. To this aim, we calculated two-particle quantities using dynamical mean-field theory for the AF-ordered Hubbard model and studied the possible occurrence of divergences of the irreducible vertex functions in the charge and spin sectors. Our calculations pinpoint the divergences in the AF phase diagram, showing that while the onset of AF order mitigates the breakdown of the perturbation expansion, it does not fully prevent it. Moreover, we have been able to link the changes in the dynamical structure of the corresponding generalized susceptibilities to the physical crossover from a weak-coupling (Slater) to a strong-coupling (Heisenberg) antiferromagnet, which takes place as the interaction strength is gradually increased. Finally, we discuss possible physical consequences of the irreducible vertex divergences in triggering phase-separation instabilities within the AF phase and elaborate on the implications of our findings for two-dimensional systems, where the onset of a long-range AF order is prevented by the Mermin-Wagner theorem., Comment: 21 pages, 10 figures

Large language models are increasingly becoming a cornerstone technology in artificial intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset composition and filtering remain largely elusive. Many of the top-performing models lack transparency in their dataset curation and model development processes, posing an obstacle to the development of fully open language models. In this paper, we identify three core data-related challenges that must be addressed to advance open-source language models. These include (1) transparency in model development, including the data curation process, (2) access to large quantities of high-quality data, and (3) availability of artifacts and metadata for dataset curation and analysis. To address these challenges, we release RedPajama-V1, an open reproduction of the LLaMA training dataset. In addition, we release RedPajama-V2, a massive web-only dataset consisting of raw, unfiltered text data together with quality signals and metadata. Together, the RedPajama datasets comprise over 100 trillion tokens spanning multiple domains and with their quality signals facilitate the filtering of data, aiming to inspire the development of numerous new datasets. To date, these datasets have already been used in the training of strong language models used in production, such as Snowflake Arctic, Salesforce's XGen and AI2's OLMo. To provide insight into the quality of RedPajama, we present a series of analyses and ablation studies with decoder-only language models with up to 1.6B parameters. Our findings demonstrate how quality signals for web data can be effectively leveraged to curate high-quality subsets of the dataset, underscoring the potential of RedPajama to advance the development of transparent and high-performing language models at scale., Comment: 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks

Multimedia streaming protocols are becoming increasingly popular in Crowdsensing due to their ability to deliver high-quality video content over the internet in real-time. Streaming multimedia content, as in the context of live video streaming, requires high bandwidth and large storage capacity to ensure a sufficient throughput. Crowdsensing can distribute information about shared video contents among multiple users in network, reducing storage capacity and computational and bandwidth requirements. However, Crowdsensing introduces several security constraints that must be taken into account to ensure the confidentiality, integrity, and availability of the data. In the specific case of video streaming, commonly named as visual crowdsensing (VCS) within this context, data is transmitted over wireless networks, making it vulnerable to security breaches and susceptible to eavesdropping and interception by attackers. Multimedias often contains sensitive user data and may be subject to various privacy laws, including data protection laws and laws related to photography and video recording, based on local GDPR (General Data Protection Regulation). For this reason the realization of a secure protocol optimized for a distributed data streaming in real-time becomes increasingly important in crowdsensing and smart-enviroment context. In this article, we will discuss the use of a symmetric AES-CTR encryption based protocol for securing data streaming over a crowd-sensed network.

Language model performance depends on identifying the optimal mixture of data groups to train on (e.g., law, code, math). Prior work has proposed a diverse set of methods to efficiently learn mixture proportions, ranging from fitting regression models over training runs to dynamically updating proportions throughout training. Surprisingly, we find that no existing method consistently outperforms a simple stratified sampling baseline in terms of average test perplexity per group. In this paper, we study the cause of this inconsistency by unifying existing methods into a standard optimization framework. We show that all methods set proportions to minimize total loss, subject to a method-specific mixing law -- an assumption on how loss is a function of mixture proportions. We find that existing parameterizations of mixing laws can express the true loss-proportion relationship empirically, but the methods themselves often set the mixing law parameters inaccurately, resulting in poor and inconsistent performance. Finally, we leverage the insights from our framework to derive a new online method named Aioli, which directly estimates the mixing law parameters throughout training and uses them to dynamically adjust proportions. Empirically, Aioli outperforms stratified sampling on 6 out of 6 datasets by an average of 0.28 test perplexity points, whereas existing methods fail to consistently beat stratified sampling, doing up to 6.9 points worse. Moreover, in a practical setting where proportions are learned on shorter runs due to computational constraints, Aioli can dynamically adjust these proportions over the full training run, consistently improving performance over existing methods by up to 12.01 test perplexity points.

Low precision training and inference affect both the quality and cost of language models, but current scaling laws do not account for this. In this work, we devise "precision-aware" scaling laws for both training and inference. We propose that training in lower precision reduces the model's "effective parameter count," allowing us to predict the additional loss incurred from training in low precision and post-train quantization. For inference, we find that the degradation introduced by post-training quantization increases as models are trained on more data, eventually making additional pretraining data actively harmful. For training, our scaling laws allow us to predict the loss of a model with different parts in different precisions, and suggest that training larger models in lower precision may be compute optimal. We unify the scaling laws for post and pretraining quantization to arrive at a single functional form that predicts degradation from training and inference in varied precisions. We fit on over 465 pretraining runs and validate our predictions on model sizes up to 1.7B parameters trained on up to 26B tokens.

The challenge of mapping AI architectures to GPU hardware is creating a critical bottleneck in AI progress. Despite substantial efforts, hand-written custom kernels fail to meet their theoretical performance thresholds, even on well-established operations like linear attention. The diverse hardware capabilities of GPUs might suggest that we need a wide variety of techniques to achieve high performance. However, our work explores whether a small number of key abstractions can drastically simplify the process. We present ThunderKittens (TK), a framework for writing performant AI kernels while remaining easy to use and maintain. Our abstractions map to the three levels of the GPU hierarchy: (1) at the warp-level, we provide 16x16 matrix tiles as basic data structures and PyTorch-like parallel compute operations over tiles, (2) at the thread-block level, we provide a template for overlapping asynchronous operations across parallel warps, and (3) at the grid-level, we provide support to help hide the block launch and tear-down, and memory costs. We show the value of TK by providing kernels that match or outperform prior kernels for a range of AI operations. We match CuBLAS and FlashAttention-3 on GEMM and attention inference performance and outperform the strongest baselines by $10-40\%$ on attention backwards, $8\times$ on state space models, and $14\times$ on linear attention.

The search for accreted satellites in the Galactic disk is a challenging task, to which Gaia plays a crucial role in synergy with ground-based spectroscopic surveys. In 2021, Re Fiorentin et al. discovered five substructures with disk kinematics including Icarus. To gain more insight into the origin of Icarus as a remnant of a dwarf galaxy rather than a signature of secular processes of disk formation, we complement astrometric Gaia DR3 data with spectroscopy from APOGEE DR17 and GALAH DR3, and explore the chemo-dynamical distributions within 3 kpc of the Sun. We select 622 stars in the accreted/unevolved regions of [Mg/Mn]-[Al/Fe] and [Mg/Fe]-[Fe/H], where we identify 81 and 376 stars with $-2<{\rm[Fe/H]}<-0.7$ belonging to Icarus and Gaia-Sausage-Enceladus (GSE), respectively. The revised properties of Icarus are: $\langle V+V_{\rm LSR}\rangle\simeq171~\rm{km~s}^{-1}$, $\sigma_{V}\simeq37\rm{km~s}^{-1}$, $\langle e\rangle\simeq0.36$, $\langle{\rm[Fe/H]}\rangle\simeq-1.35$, $\langle{\rm[Mg/Fe]}\rangle\simeq+0.27$, $\langle{\rm[Al/Fe]}\rangle\simeq-0.13$, and $\langle{\rm[Mn/Fe]}\rangle\simeq-0.39$. From the CMD of its members, Icarus appears older than 12 Gyr. Such age and dynamical properties are reminiscent of the metal-weak thick disk. However, detailed chemical analysis in the diagnostic spaces [Ni/Fe]-[(C+N)/O], [Y/Eu]-[Fe/H], [Eu/Mg]-[Fe/H], [Ba/Y]-[Fe/H], and [Ba/Mg]-[Mg/H] evidences that Icarus and GSE occupy the accreted region, well separated from the bulk of in situ disk stars. Updated comparisons with N-body simulations confirm that Icarus' stars are consistent with the debris of a dwarf galaxy with a stellar mass of $\sim~10^9~M_\odot$ accreted onto a primordial disk on an initial prograde low-inclination orbit., Comment: 18 pages, 8 figures, 3 Tables. Accepted for publication into The Astrophysical Journal

Among the great mysteries that physics has not yet solved are undoubtedly those of dark energy and dark matter. In this chapter we deal with the first of them. We will expound in detail the motivations that led to hypothesise the existence of dark energy, the importance of understanding its nature, its main possible theoretical explanations and a list of the many attempts at modelling it. We conclude with a description of the most recent and future missions on Earth and in space devised to shed light on this mystery., Comment: 35 pages

Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear attention -- avoiding the expensive pretraining costs. However, linearizing LLMs often significantly degrades model quality, still requires training over billions of tokens, and remains limited to smaller 1.3B to 7B LLMs. We thus propose Low-rank Linear Conversion via Attention Transfer (LoLCATs), a simple two-step method that improves LLM linearizing quality with orders of magnitudes less memory and compute. We base these steps on two findings. First, we can replace an LLM's softmax attentions with closely-approximating linear attentions, simply by training the linear attentions to match their softmax counterparts with an output MSE loss ("attention transfer"). Then, this enables adjusting for approximation errors and recovering LLM quality simply with low-rank adaptation (LoRA). LoLCATs significantly improves linearizing quality, training efficiency, and scalability. We significantly reduce the linearizing quality gap and produce state-of-the-art subquadratic LLMs from Llama 3 8B and Mistral 7B v0.1, leading to 20+ points of improvement on 5-shot MMLU. Furthermore, LoLCATs does so with only 0.2% of past methods' model parameters and 0.4% of their training tokens. Finally, we apply LoLCATs to create the first linearized 70B and 405B LLMs (50x larger than prior work). When compared with prior approaches under the same compute budgets, LoLCATs significantly improves linearizing quality, closing the gap between linearized and original Llama 3.1 70B and 405B LLMs by 77.8% and 78.1% on 5-shot MMLU., Comment: 47 pages, 20 figures, 18 tables, preprint

Large Language Models (LLMs) have the potential to automate reward engineering by leveraging their broad domain knowledge across various tasks. However, they often need many iterations of trial-and-error to generate effective reward functions. This process is costly because evaluating every sampled reward function requires completing the full policy optimization process for each function. In this paper, we introduce an LLM-driven reward generation framework that is able to produce state-of-the-art policies on the challenging Bi-DexHands benchmark with 20x fewer reward function samples than the prior state-of-the-art work. Our key insight is that we reduce the problem of generating task-specific rewards to the problem of coarsely estimating task progress. Our two-step solution leverages the task domain knowledge and the code synthesis abilities of LLMs to author progress functions that estimate task progress from a given state. Then, we use this notion of progress to discretize states, and generate count-based intrinsic rewards using the low-dimensional state space. We show that the combination of LLM-generated progress functions and count-based intrinsic rewards is essential for our performance gains, while alternatives such as generic hash-based counts or using progress directly as a reward function fall short., Comment: 26 pages, 5 figures

Vector Quantized Variational AutoEncoders (VQ-VAEs) are designed to compress a continuous input to a discrete latent space and reconstruct it with minimal distortion. They operate by maintaining a set of vectors -- often referred to as the codebook -- and quantizing each encoder output to the nearest vector in the codebook. However, as vector quantization is non-differentiable, the gradient to the encoder flows around the vector quantization layer rather than through it in a straight-through approximation. This approximation may be undesirable as all information from the vector quantization operation is lost. In this work, we propose a way to propagate gradients through the vector quantization layer of VQ-VAEs. We smoothly transform each encoder output into its corresponding codebook vector via a rotation and rescaling linear transformation that is treated as a constant during backpropagation. As a result, the relative magnitude and angle between encoder output and codebook vector becomes encoded into the gradient as it propagates through the vector quantization layer and back to the encoder. Across 11 different VQ-VAE training paradigms, we find this restructuring improves reconstruction metrics, codebook utilization, and quantization error. Our code is available at https://github.com/cfifty/rotation_trick.

Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while LLM-generated data is less labor-intensive, it may violate user privacy agreements or terms of service of LLM providers. Therefore, we seek a way of constructing instruction datasets with samples that are not generated by humans or LLMs but still improve LLM generative capabilities. In this work, we introduce Cookbook, a framework that programmatically generates training data consisting of simple patterns over random tokens, resulting in a scalable, cost-effective approach that avoids legal and privacy issues. First, Cookbook uses a template -- a data generating Python function -- to produce training data that encourages the model to learn an explicit pattern-based rule that corresponds to a desired task. We find that fine-tuning on Cookbook-generated data is able to improve performance on its corresponding task by up to 52.7 accuracy points. Second, since instruction datasets improve performance on multiple downstream tasks simultaneously, Cookbook algorithmically learns how to mix data from various templates to optimize performance on multiple tasks. On the standard multi-task GPT4ALL evaluation suite, Mistral-7B fine-tuned using a Cookbook-generated dataset attains the best accuracy on average compared to other 7B parameter instruction-tuned models and is the best performing model on 3 out of 8 tasks. Finally, we analyze when and why Cookbook improves performance and present a metric that allows us to verify that the improvement is largely explained by the model's generations adhering better to template rules., Comment: COLM 2024

Inference-time techniques are emerging as highly effective tools to enhance large language model (LLM) capabilities. However, best practices for developing systems that combine these techniques remain underdeveloped due to our limited understanding of the utility of individual inference-time techniques and the interactions between them. Additionally, efficiently and automatically searching the space of model choices, inference-time techniques, and their compositions is challenging due to the large design space. To address these challenges, we introduce Archon, a modular framework for selecting, combining, and stacking layers of inference-time techniques to construct optimized LLM systems for target benchmarks. Rather than relying on a single LLM called once, we leverage a diverse set of LLMs and inference-time techniques, creating LLM systems greater than the sum of their parts. Archon defines an extensible design space, encompassing techniques such as generation ensembling, repeated sampling, ranking, fusion, critiquing, verification, and unit testing. It transforms the problem of building LLM systems into a hyperparameter optimization objective. Given the available LLMs, inference-time techniques, and compute budget, Archon utilizes hyperparameter search techniques to discover optimized architectures for target benchmark(s). We evaluate Archon architectures across a range of instruction-following, reasoning, and coding benchmarks, including MT-Bench, Arena-Hard-Auto, AlpacaEval 2.0, MixEval, MixEval Hard, MATH, and CodeContests. Archon architectures outperform frontier models, such as GPT-4o and Claude 3.5 Sonnet, on these benchmarks, achieving an average accuracy increase of 15.1 percentage points by using all available LLMs. We make our code and datasets available publicly on Github: https://github.com/ScalingIntelligence/Archon.

The control of quantum systems has been proven to possess trap-free optimization landscapes under the satisfaction of proper assumptions. However, many details of the landscape geometry and their influence on search efficiency still need to be fully understood. This paper numerically explores the path-connectedness of globally optimal control solutions forming the top manifold of the landscape. We randomly sample a plurality of optimal controls in the top manifold to assess the existence of a continuous path at the top of the landscape that connects two arbitrary optimal solutions. It is shown that for different quantum control objectives including state-to-state transition probabilities, observable expectation values and unitary transformations, such a continuous path can be readily found, implying that these top manifolds are fundamentally path-connected. The significance of the latter conjecture lies in seeking locations in the top manifold where an ancillary objective can also be optimized while maintaining the full optimality of the original objective that defined the landscape., Comment: 34 pages, 10 figures

Two-dimensional crystals stack together through weak van der Waals (vdW) forces, offering unlimited possibilities to play with layer number, order and twist angle in vdW heterostructures (HSs). The realisation of high-performance optoelectronic devices, however, requires the achievement of specific band alignments, $k$-space matching between conduction band minima and valence band maxima, as well as efficient charge transfer between the constituent layers. Fine tuning mechanisms to design ideal HSs are lacking. Here, we show that layer-selective strain engineering can be exploited as an extra degree of freedom in vdW HSs to tailor their band alignment and optical properties. To that end, strain is selectively applied to MS$_2$ (M=Mo,W) monolayers in InSe/MS$_2$ HSs. This triggers a giant PL enhancement of the highly tuneable but weakly emitting InSe by one to three orders of magnitude. Resonant PL excitation measurements, supported by first-principle calculations, provide evidence of a strain-activated direct charge transfer from the MS$_2$ MLs toward InSe. This significant emission enhancement achieved for InSe widens its range of applications for optoelectronics.

Research conducted previously has focused on either attitudes toward or behaviors associated with autonomous driving. In this paper, we bridge these two dimensions by exploring how attitudes towards autonomous driving influence behavior in an autonomous car. We conducted a field experiment with twelve participants engaged in non-driving related tasks. Our findings indicate that attitudes towards autonomous driving do not affect participants' driving interventions in vehicle control and eye glance behavior. Therefore, studies on autonomous driving technology lacking field tests might be unreliable for assessing the potential behaviors, attitudes, and acceptance of autonomous vehicles., Comment: 12 pages

Electrochemical reduction is a promising way to make use of CO$_2$ as feedstock for generating renewable fuel and valuable chemicals. Several metals can be used in the electrocatalyst to generate CO and formic acid but hydrogen formation is an unwanted side reaction that can even be dominant. The lack of selectivity is in general a significant problem, but silver-based electrocatalysts have been shown to be highly selective for CO with over over 90% faradaic efficiency when the applied voltage is below -1 V vs. RHE. Hydrogen formation is then insignificant and little formate is formed even though it is thermodynamically favored. We present calculations of the activation free energy for the various elementary steps as a function of applied voltage at the three low index facets, Ag(111), Ag(100) and Ag(110), as well as experimental measurements on polycrystalline electrodes, to identify the reason for this high selectivity. The formation of formic acid is suppressed because of the low coverage of adsorbed hydrogen and kinetic hindrance to the formation of the HCOO* intermediate, while *COOH, a key intermediate in CO formation, is thermodynamically unstable until the applied voltage reaches -1 V vs. RHE, at which point the kinetics for its formation are more favorable than for hydrogen. The calculated results are consistent with experimental measurements carried out for acidic conditions and provide an atomic scale insight into the high CO selectivity of silver-based electrocatalysts.

We explore a two-dimensional Hubbard model adapted to host altermagnetic states. Utilizing Hartree-Fock (HF) and dynamical mean field theory (DMFT), we uncover that the magnetic solutions of this model feature Dirac points in their spectrum. HF predicts a gap opening at a critical interaction strength, a result corroborated by DMFT calculations at zero temperature. However, at finite temperature and high interaction strengths, Dirac points re-emerge at high energies in the spectral function, even if they are absent in the non-interacting and HF-predicted band structures. Analytical arguments reveal that this phenomenon arises near Mott insulating solutions from the frequency dependence of the self-energy, a behavior not captured by static mean-field theory. We also propose perturbations to the model that can induce a topological gap in the spectrum, leading to a transition from topologically trivial to non-trivial states by varying doping, interaction strength, and temperature.

As autonomous vehicle technology advances, the precise assessment of safety in complex traffic scenarios becomes crucial, especially in mixed-vehicle environments where human perception of safety must be taken into account. This paper presents a framework designed for assessing traffic safety in multi-vehicle situations, facilitating the simultaneous utilization of diverse objective safety metrics. Additionally, it allows the integration of subjective perception of safety by adjusting model parameters. The framework was applied to evaluate various model configurations in car-following scenarios on a highway, utilizing naturalistic driving datasets. The evaluation of the model showed an outstanding performance, particularly when integrating multiple objective safety measures. Furthermore, the performance was significantly enhanced when considering all surrounding vehicles.

A new Newtonian limit of general relativity is established for stationary axisymmetric gravitationally bound differentially rotating matter distributions with internal pressure. The self-consistent coupling of quasilocal gravitational energy and angular momentum leads to a modified Poisson equation. The coupled equations of motion of the effective fluid elements are also modified, with quasilocal angular momentum and frame-dragging leading to novel dynamics. The solutions of the full system reproduce the phenonomenology of collisionless dark matter for disc galaxies, offering an explanation for their observed rotation curves. Halos of abundant cold dark matter particles are not required., Comment: 6 pages, 2 figures

Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit the amount of compute to only one attempt per problem. Here, we explore inference compute as another axis for scaling by increasing the number of generated samples. Across multiple tasks and models, we observe that coverage - the fraction of problems solved by any attempt - scales with the number of samples over four orders of magnitude. In domains like coding and formal proofs, where all answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-V2-Coder-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-attempt state-of-the-art of 43% which uses more capable frontier models. Moreover, using current API pricing, amplifying the cheaper DeepSeek model with five samples is more cost-effective and solves more issues than paying a premium for one sample from GPT-4o or Claude 3.5 Sonnet. Interestingly, the relationship between coverage and the number of samples is often log-linear and can be modelled with an exponentiated power law, suggesting the existence of inference-time scaling laws. Finally, we find that identifying correct samples out of many generations remains an important direction for future research in domains without automatic verifiers. When solving math word problems from GSM8K and MATH, coverage with Llama-3 models grows to over 95% with 10,000 samples. However, common methods to pick correct solutions from a sample collection, such as majority voting or reward models, plateau beyond several hundred samples and fail to fully scale with the sample budget.

The Southern Photometric Local Universe Survey (S-PLUS) is a project to map $\sim9300$ sq deg of the sky using twelve bands (seven narrow and five broadbands). Observations are performed with the T80-South telescope, a robotic telescope located at the Cerro Tololo Observatory in Chile. The survey footprint consists of several large contiguous areas, including fields at high and low galactic latitudes, and towards the Magellanic Clouds. S-PLUS uses fixed exposure times to reach point source depths of about $21$ mag in the $griz$ and $20$ mag in the $u$ and the narrow filters. This paper describes the S-PLUS Data Release 4 (DR4), which includes calibrated images and derived catalogues for over 3000 sq deg, covering the aforementioned area. The catalogues provide multi-band photometry performed with the tools \texttt{DoPHOT} and \texttt{SExtractor} -- point spread function (\PSF) and aperture photometry, respectively. In addition to the characterization, we also present the scientific potential of the data. We use statistical tools to present and compare the photometry obtained through different methods. Overall we find good agreement between the different methods, with a slight systematic offset of 0.05\,mag between our \PSF and aperture photometry. We show that the astrometry accuracy is equivalent to that obtained in previous S-PLUS data releases, even in very crowded fields where photometric extraction is challenging. The depths of main survey (MS) photometry for a minimum signal-to-noise ratio $S/N = 3$ reach from $\sim19.5$ for the bluer bands to $\sim21.5$ mag on the red. The range of magnitudes over which accurate \PSF photometry is obtained is shallower, reaching $\sim19$ to $\sim20.5$ mag depending on the filter. Based on these photometric data, we provide star-galaxy-quasar classification and photometric redshift for millions of objects., Comment: 26 pages, 17 figures, 14 tables, accepted for A&A

Federated Learning (FL) is a distributed machine learning paradigm facilitating participants to collaboratively train a model without revealing their local data. However, when FL is deployed into the wild, some intelligent clients can deliberately deviate from the standard training process to make the global model inclined toward their local model, thereby prioritizing their local data distribution. We refer to this novel category of misbehaving clients as selfish. In this paper, we propose a Robust aggregation strategy for FL server to mitigate the effect of Selfishness (in short RFL-Self). RFL-Self incorporates an innovative method to recover (or estimate) the true updates of selfish clients from the received ones, leveraging robust statistics (median of norms) of the updates at every round. By including the recovered updates in aggregation, our strategy offers strong robustness against selfishness. Our experimental results, obtained on MNIST and CIFAR-10 datasets, demonstrate that just 2% of clients behaving selfishly can decrease the accuracy by up to 36%, and RFL-Self can mitigate that effect without degrading the global model performance., Comment: 10 pages, 16 figures. European Conference on Artificial Intelligence (ECAI) 2024

Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use a constant amount of memory during inference. However, due to the limited memory, recurrent LMs cannot recall and use all the information in long contexts leading to brittle in-context learning (ICL) quality. A key challenge for efficient LMs is selecting what information to store versus discard. In this work, we observe the order in which information is shown to the LM impacts the selection difficulty. To formalize this, we show that the hardness of information recall reduces to the hardness of a problem called set disjointness (SD), a quintessential problem in communication complexity that requires a streaming algorithm (e.g., recurrent model) to decide whether inputted sets are disjoint. We empirically and theoretically show that the recurrent memory required to solve SD changes with set order, i.e., whether the smaller set appears first in-context. Our analysis suggests, to mitigate the reliance on data order, we can put information in the right order in-context or process prompts non-causally. Towards that end, we propose: (1) JRT-Prompt, where context gets repeated multiple times in the prompt, effectively showing the model all data orders. This gives $11.0 \pm 1.3$ points of improvement, averaged across $16$ recurrent LMs and the $6$ ICL tasks, with $11.9\times$ higher throughput than FlashAttention-2 for generation prefill (length $32$k, batch size $16$, NVidia H100). We then propose (2) JRT-RNN, which uses non-causal prefix-linear-attention to process prompts and provides $99\%$ of Transformer quality at $360$M params., $30$B tokens and $96\%$ at $1.3$B params., $50$B tokens on average across the tasks, with $19.2\times$ higher throughput for prefill than FA2.

The CUbesat Solar Polarimeter (CUSP) project is a future CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band, by means of a Compton scattering polarimeter. CUSP will allow us to study the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. The project is in the framework of the Italian Space Agency Alcor Program, which aims to develop new CubeSat missions. CUSP is approved for a Phase B study that will last for 12 months, starting in mid-2024. We report on the current status of the CUSP mission project as the outcome of the Phase A., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024. arXiv admin note: substantial text overlap with arXiv:2208.06211

The CUbesat Solar Polarimeter (CUSP) project is a CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter. CUSP will allow the study of the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. CUSP is a project in the framework of the Alcor Program of the Italian Space Agency aimed at developing new CubeSat missions. It is approved for a Phase B study. In this work, we report on the characterization of the Avalanche Photodiodes (APDs) that will be used as readout sensors of the absorption stage of the Compton polarimeter. We assessed the APDs gain and energy resolution as a function of temperature by irradiating the sensor with a \textsuperscript{55}Fe radioactive source. Moreover, the APDs were also characterized as being coupled to a GAGG scintillator., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

The CUbesat Solar Polarimeter (CUSP) project aims to develop a constellation of two CubeSats orbiting the Earth to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter on board of each satellite. CUSP will allow to study the magnetic reconnection and particle acceleration in the flaring magnetic structures. CUSP is a project approved for a Phase B study by the Italian Space Agency in the framework of the Alcor program aimed to develop CubeSat technologies and missions. In this paper we describe the a method for a multi-physical simulation analysis while analyzing some possible design optimization of the payload design solutions adopted. In particular, we report the mechanical design for each structural component, the results of static and dynamic finite element analysis, the preliminary thermo-mechanical analysis for two specific thermal cases (hot and cold orbit) and a topological optimization of the interface between the platform and the payload., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

The CUbesat Solar Polarimeter (CUSP) project is a CubeSat mission orbiting the Earth aimed to measure the linear polarization of solar flares in the hard X-ray band by means of a Compton scattering polarimeter. CUSP will allow to study the magnetic reconnection and particle acceleration in the flaring magnetic structures of our star. CUSP is a project in the framework of the Alcor Program of the Italian Space Agency aimed to develop new CubeSat missions. It is approved for a Phase B study. In this work, we report on the accurate simulation of the detector's response to evaluate the scientific performance. A GEANT4 Monte Carlo simulation is used to assess the physical interactions of the source photons with the detector and the passive materials. Using this approach, we implemented a detailed CUSP Mass Model. In this work, we report on the evaluation of the detector's effective area as a function of the beam energy., Comment: Proceeding of SPIE Conference "Astronomical Telescopes+ Instrumentation", Yokohama (Japan), 16-21 June 2024

We describe the planned near-term and potential longer-term upgrades of the Belle II detector at the SuperKEKB electron-positron collider operating at the KEK laboratory in Tsukuba, Japan. These upgrades will allow increasingly sensitive searches for possible new physics beyond the Standard Model in flavor, tau, electroweak and dark sector physics that are both complementary to and competitive with the LHC and other experiments., Comment: Editor: F. Forti 170 pages

We introduce MindSpore Quantum, a pioneering hybrid quantum-classical framework with a primary focus on the design and implementation of noisy intermediate-scale quantum (NISQ) algorithms. Leveraging the robust support of MindSpore, an advanced open-source deep learning training/inference framework, MindSpore Quantum exhibits exceptional efficiency in the design and training of variational quantum algorithms on both CPU and GPU platforms, delivering remarkable performance. Furthermore, this framework places a strong emphasis on enhancing the operational efficiency of quantum algorithms when executed on real quantum hardware. This encompasses the development of algorithms for quantum circuit compilation and qubit mapping, crucial components for achieving optimal performance on quantum processors. In addition to the core framework, we introduce QuPack, a meticulously crafted quantum computing acceleration engine. QuPack significantly accelerates the simulation speed of MindSpore Quantum, particularly in variational quantum eigensolver (VQE), quantum approximate optimization algorithm (QAOA), and tensor network simulations, providing astonishing speed. This combination of cutting-edge technologies empowers researchers and practitioners to explore the frontiers of quantum computing with unprecedented efficiency and performance.

Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, measuring, improving, and automating enterprise workflows. However, research has focused almost exclusively on one task - full end-to-end automation using agents based on multimodal foundation models (FMs) like GPT-4. This focus on automation ignores the reality of how most BPM tools are applied today - simply documenting the relevant workflow takes 60% of the time of the typical process optimization project. To address this gap we present WONDERBREAD, the first benchmark for evaluating multimodal FMs on BPM tasks beyond automation. Our contributions are: (1) a dataset containing 2928 documented workflow demonstrations; (2) 6 novel BPM tasks sourced from real-world applications ranging from workflow documentation to knowledge transfer to process improvement; and (3) an automated evaluation harness. Our benchmark shows that while state-of-the-art FMs can automatically generate documentation (e.g. recalling 88% of the steps taken in a video demonstration of a workflow), they struggle to re-apply that knowledge towards finer-grained validation of workflow completion (F1 < 0.3). We hope WONDERBREAD encourages the development of more "human-centered" AI tooling for enterprise applications and furthers the exploration of multimodal FMs for the broader universe of BPM tasks. We publish our dataset and experiments here: https://github.com/HazyResearch/wonderbread

Several neutrino detectors, KamLAND, Daya Bay, Double Chooz, RENO, and the forthcoming large-scale JUNO, rely on liquid scintillator to detect reactor antineutrino interactions. In this context, inverse beta decay represents the golden channel for antineutrino detection, providing a pair of correlated events, thus a strong experimental signature to distinguish the signal from a variety of backgrounds. However, given the low cross-section of antineutrino interactions, the development of a powerful event selection algorithm becomes imperative to achieve effective discrimination between signal and backgrounds. In this study, we introduce a machine learning (ML) model to achieve this goal: a fully connected neural network as a powerful signal-background discriminator for a large liquid scintillator detector. We demonstrate, using the JUNO detector as an example, that, despite the already high efficiency of a cut-based approach, the presented ML model can further improve the overall event selection efficiency. Moreover, it allows for the retention of signal events at the detector edges that would otherwise be rejected because of the overwhelming amount of background events in that region. We also present the first interpretable analysis of the ML approach for event selection in reactor neutrino experiments. This method provides insights into the decision-making process of the model and offers valuable information for improving and updating traditional event selection approaches.

The optical and radiochemical purification of the scintillating liquid, which will fill the central detector of the JUNO experiment, plays a crucial role in achieving its scientific goals. Given its gigantic mass and dimensions and an unprecedented target value of about 3% @ 1 MeV in energy resolution, JUNO has set severe requirements on the parameters of its scintillator, such as attenuation length (Lat>20 m at 430 nm), transparency, light yield, and content of radioactive contaminants (238U,232Th<10-15 g/g). To accomplish these needs, the scintillator will be processed using several purification methods, including distillation in partial vacuum and gas stripping, which are performed in two large scale plants installed at the JUNO site. In this paper, layout, operating principles, and technical aspects which have driven the design and construction of the distil- lation and gas stripping plants are reviewed. The distillation is effective in enhancing the optical properties and removing heavy radio-impurities (238U,232Th, 40K), while the stripping process exploits pure water steam and high-purity nitrogen to extract gaseous contaminants (222Rn, 39Ar, 85Kr, O2) from the scintillator. The plant operating parameters have been tuned during the recent com- missioning phase at the JUNO site and several QA/QC measurements and tests have been performed to evaluate the performances of the plants. Some preliminary results on the efficiency of these purification processes will be shown., Comment: 11 pages, 7 figures

This report presents a short summary of the activities of the "Standard Model" working group for the "Physics at TeV Colliders" workshop (Les Houches, France, 12-30 June, 2023)., Comment: Proceedings of the Standard Model Working Group of the 2023 Les Houches Workshop, Physics at TeV Colliders, Les Houches 12-30 June 2023. 48 pages

The acquisition and analysis of high-quality sensor data constitute an essential requirement in shaping the development of fully autonomous driving systems. This process is indispensable for enhancing road safety and ensuring the effectiveness of the technological advancements in the automotive industry. This study introduces the Interaction of Autonomous and Manually-Controlled Vehicles (IAMCV) dataset, a novel and extensive dataset focused on inter-vehicle interactions. The dataset, enriched with a sophisticated array of sensors such as Light Detection and Ranging, cameras, Inertial Measurement Unit/Global Positioning System, and vehicle bus data acquisition, provides a comprehensive representation of real-world driving scenarios that include roundabouts, intersections, country roads, and highways, recorded across diverse locations in Germany. Furthermore, the study shows the versatility of the IAMCV dataset through several proof-of-concept use cases. Firstly, an unsupervised trajectory clustering algorithm illustrates the dataset's capability in categorizing vehicle movements without the need for labeled training data. Secondly, we compare an online camera calibration method with the Robot Operating System-based standard, using images captured in the dataset. Finally, a preliminary test employing the YOLOv8 object-detection model is conducted, augmented by reflections on the transferability of object detection across various LIDAR resolutions. These use cases underscore the practical utility of the collected dataset, emphasizing its potential to advance research and innovation in the area of intelligent vehicles.

Though early childhood educators have a long history of emphasizing learning through play to foster the development of the whole child, including social-emotional competencies, the proliferation of formal social-emotional learning (SEL) curricula first seen in kindergarten to 12th grade has expanded to prekindergarten. This report presents initial implementation findings of the Positive Emotional Development and Learning Skills (PEDALS) program, an SEL program in prekindergarten classrooms in Western and Central New York and Southeast Michigan. PEDALS is a two-year program that combines the Second Step Early Learning SEL curriculum with comprehensive supports for teachers and site directors. Findings indicate that educators perceived the Second Step curriculum, PEDALS training, and PEDALS coaching support very favorably; strongly believed in supporting children's social-emotional development; and had a strong sense of self-efficacy for doing so. Observations indicated that educators were highly skilled in implementing Second Step lessons; however, teachers used strategies less consistently when children were having a difficult time or experiencing interpersonal conflict. Educators identified PEDALS resources for developing children's social-emotional competencies as the factors that provided the most implementation support; meanwhile, they identified some contextual factors, such as children's challenging behavior and lack of planning time, as the factors that most hindered implementation. Preliminary findings suggest that many preconditions for effective SEL instruction were met. This implementation study is a part of a larger evaluation of PEDALS that also will estimate the effect of the program on children's SEL and executive functioning skills. A second report will be published at the study's conclusion. [Additional funding from the Health Foundation for Western & Central New York.]

In the field of rare event physics, it is common to have huge masses of organic liquid scintillator as detection medium. In particular, they are widely used to study neutrino properties or astrophysical neutrinos. Thanks to its safety properties (such as low toxicity and high flash point) and easy scalability, linear alkyl benzene is the most common solvent used to produce liquid scintillators for large mass experiments. The knowledge of the refractive index is a pivotal point to understand the detector response, as this quantity (and its wavelength dependence) affects the Cherenkov radiation and photon propagation in the medium. In this paper, we report the measurement of the refractive index of the JUNO liquid scintillator between 260-1064 nm performed with two different methods (an ellipsometer and a refractometer), with a sub percent level precision. In addition, we used an interferometer to measure the group velocity in the JUNO liquid scintillator and verify the expected value derived from the refractive index measurements., Comment: 6 pages, 9 figures

We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 \nu$ or $nn \rightarrow 2 \nu$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\bar{\nu}_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $\tau/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $\tau/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$., Comment: 28 pages, 7 figures, 4 tables

Pulse-based quantum machine learning (QML) models possess full expressivity when they are ensemble controllable. However, it has also been shown that barren plateaus emerge in such models, rendering training intractable for systems with large dimension. In this paper, we show that the trade-off is closely related to the controllability of the underlying pulse-based models. We first apply the Fliess-series expansion to pulse-based QML models to investigate the effect of control system structure on model expressivity, which leads to a universal criterion for assessing the expressivity of generic QML models. Guided by this criterion, we then demonstrate how designing pulse-based models on low-dimensional manifolds can balance expressivity and trainability. Finally, numerical experiments are carried out to verify the proposed criterion and our analysis, which futher demonstrate that increasing dimensionality enhances expressivity but avoids barren plateaus if the model is designed with limited controllability on a submanifold. Our approach provides a promising path for designing pulse-based QML models that are both highly expressive and trainable.