Search

We consider the universal discrete filtering problem, where an input sequence generated by an unknown source passes through a discrete memoryless channel, and the goal is to estimate its components based on the output sequence with limited lookahead or delay. We propose and establish the universality of a family of schemes for this setting. These schemes are induced by universal Sequential Probability Assignments (SPAs), and inherit their computational properties. We show that the schemes induced by LZ78 are practically implementable and well-suited for scenarios with limited computational resources and latency constraints. In passing, we use some of the intermediate results to obtain upper and lower bounds that appear to be new, in the purely Bayesian setting, on the optimal filtering performance in terms, respectively, of the mutual information between the noise-free and noisy sequence, and the entropy of the noise-free sequence causally conditioned on the noisy one.

We investigate the high-energy fixed-angle scattering of pions and $\rho$-mesons in a bottom-up holographic QCD model. To this end, we generalise the approach of Polchinski and Strassler arXiv:hep-th/0109174 to write an ansatz for meson scattering amplitudes based on superstring scattering amplitudes in asymptotically AdS space. We demonstrate that our generalisation of the Polchinski--Strassler proposal is necessary to describe $\rho$-meson scattering consistently with the Nambu--Goldstone boson equivalence theorem. Our results for pion and $\rho$-meson scattering amplitudes are in agreement with the constituent counting rule found in QCD. Moreover, our proposal for 2-to-2 scattering amplitudes provides a method for computing scattering angle dependence., Comment: 28 pages, 3 figures

We propose and study a family of universal sequential probability assignments on individual sequences, based on the incremental parsing procedure of the Lempel-Ziv (LZ78) compression algorithm. We show that the normalized log loss under any of these models converges to the normalized LZ78 codelength, uniformly over all individual sequences. To establish the universality of these models, we consolidate a set of results from the literature relating finite-state compressibility to optimal log-loss under Markovian and finite-state models. We also consider some theoretical and computational properties of these models when viewed as probabilistic sources. Finally, we present experimental results showcasing the potential benefit of using this family -- as models and as sources -- for compression, generation, and classification., Comment: 31 pages, 5 figures, submitted to IEEE Transactions on Information Theory

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

In this paper, the authors introduce a lightweight dataset to interpret IoT (Internet of Things) activity in preparation to create decoys by replicating known data traffic patterns. The dataset comprises different scenarios in a real network setting. This paper also surveys information related to other IoT datasets along with the characteristics that make our data valuable. Many of the datasets available are synthesized (simulated) or often address industrial applications, while the IoT dataset we present is based on likely smart home scenarios. Further, there are only a limited number of IoT datasets that contain both normal operation and attack scenarios. A discussion of the network configuration and the steps taken to prepare this dataset are presented as we prepare to create replicative patterns for decoy purposes. The dataset, which we refer to as IoT Flex Data, consists of four categories, namely, IoT benign idle, IoT benign active, IoT setup, and malicious (attack) traffic associating the IoT devices with the scenarios under consideration., Comment: 13 pages, 4 figures, 4 tables

Stellar abundances of cerium are of high current interest based both on observations and theoretical models, especially with regard to the neutron--magic $^{140}$Ce isotope. A large discrepancy of $s-$process stellar models relative to cerium abundance observed in globular clusters was highlighted, pointing to possible uncertainties in experimental nuclear reaction rates. In this work, the stellar neutron capture cross section of the stable cerium isotopes $^{136}$Ce, $^{138}$Ce, $^{140}$Ce, and $^{142}$Ce, were re-measured. A $^{nat}$Ce sample was irradiated with quasi-Maxwellian neutrons at $kT = 34.2$ keV using the $^{7}$Li($p,n$) reaction. The neutron field with an intensity of $3-5 \times 10^{10}$ n/s was produced by irradiating the liquid-lithium target (LiLiT) with a mA proton beam at an energy (1.92 MeV) just above the threshold at Soreq Applied Research Accelerator Facility (SARAF). The activities of the $^{nat}$Ce neutron capture products were measured using a shielded High Purity Germanium detector. Cross sections were extracted relative to that of the $^{197}$Au(n,$\gamma$) reaction and the Maxwellian-averaged cross section (MACS) of the Ce isotopes were derived. The MACS values extracted from this experiment are generally consistent with previous measurements and show for $^{140}$Ce a value $\approx 15$\% smaller than most recent experimental values., Comment: 9 pages, 12 figures. This work is supported in part by the Pazy Foundation (Israel) and the German Israeli Foundation (GIF) under Grant No. I-1500-303.7/2019. M.P. and M.F. acknowledge support by the European Union (ChETEC-INFRA)

Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights -- causing destructive interference between tasks. The resulting effects, such as catastrophic forgetting of earlier tasks, make it challenging to obtain good performance on multiple tasks at the same time. To mitigate this, we propose Lottery Ticket Adaptation (LoTA), a sparse adaptation method that identifies and optimizes only a sparse subnetwork of the model. We evaluate LoTA on a wide range of challenging tasks such as instruction following, reasoning, math, and summarization. LoTA obtains better performance than full fine-tuning and low-rank adaptation (LoRA), and maintains good performance even after training on other tasks -- thus, avoiding catastrophic forgetting. By extracting and fine-tuning over lottery tickets (or sparse task vectors), LoTA also enables model merging over highly dissimilar tasks. Our code is made publicly available at https://github.com/kiddyboots216/lottery-ticket-adaptation.

We establish a new tool for studying strongly coupled matter: an effective theory of black holes in gravity, which maps to a hydrodynamic description of field theories via the gauge-gravity duality. Our approach is inspired by previously known effective theories found in the limit of high number of dimensions. We argue that the new approach can accurately describe phase transitions in a wide class of theories, such as the Yang-Mills and other nearly critical field theories. As an application, we analyze the interface between confining and deconfining phases in holographic Yang-Mills theory., Comment: 10 pages, 2 figures

Background: Social support (SS) and social isolation (SI) are social determinants of health (SDOH) associated with psychiatric outcomes. In electronic health records (EHRs), individual-level SS/SI is typically documented as narrative clinical notes rather than structured coded data. Natural language processing (NLP) algorithms can automate the otherwise labor-intensive process of data extraction. Data and Methods: Psychiatric encounter notes from Mount Sinai Health System (MSHS, n=300) and Weill Cornell Medicine (WCM, n=225) were annotated and established a gold standard corpus. A rule-based system (RBS) involving lexicons and a large language model (LLM) using FLAN-T5-XL were developed to identify mentions of SS and SI and their subcategories (e.g., social network, instrumental support, and loneliness). Results: For extracting SS/SI, the RBS obtained higher macro-averaged f-scores than the LLM at both MSHS (0.89 vs. 0.65) and WCM (0.85 vs. 0.82). For extracting subcategories, the RBS also outperformed the LLM at both MSHS (0.90 vs. 0.62) and WCM (0.82 vs. 0.81). Discussion and Conclusion: Unexpectedly, the RBS outperformed the LLMs across all metrics. Intensive review demonstrates that this finding is due to the divergent approach taken by the RBS and LLM. The RBS were designed and refined to follow the same specific rules as the gold standard annotations. Conversely, the LLM were more inclusive with categorization and conformed to common English-language understanding. Both approaches offer advantages and are made available open-source for future testing., Comment: 2 figures, 3 tables

We propose a novel indicator for chaotic quantum scattering processes, the scattering form factor (ScFF). It is based on mapping the locations of peaks in the scattering amplitude to random matrix eigenvalues, and computing the analog of the spectral form factor (SFF). We compute the spectral and scattering form factors of several non-chaotic systems. We determine the ScFF associated with the phase shifts of the leaky torus, closely related to the distribution of the zeros of Riemann zeta function. We compute the ScFF for the decay amplitude of a highly excited string states into two tachyons. We show that it displays the universal features expected from random matrix theory - a decline, a ramp and a plateau - and is in general agreement with the Gaussian unitary ensemble. It also shows some new features, owning to the special structure of the string amplitude, including a "bump" before the ramp associated with gaps in the average eigenvalue density. The "bump" is removed for highly excited string states with an appropriate state dependent unfolding. We also discuss the SFF for the Gaussian $\beta$- ensemble, writing an interpolation between the known results of the Gaussian orthogonal, unitary, and symplectic ensembles., Comment: 47 pages, 17 figures

A few years ago it was shown that the superconformal index of the $\mathcal{N}=4$ supersymmetric $SU(N)$ Yang-Mills theory in the large $N$ limit matches with the entropy of $1/16$-supersymmetric black holes in type IIB string theory on $AdS_5\times S^5$. In some cases, an even more detailed match between the two sides is possible. When the two angular momentum chemical potentials in the index are equal, the superconformal index can be written as a discrete sum of Bethe ansatz solutions, and it was shown that specific terms in this sum are in a one-to-one correspondence to stable black hole solutions, and that the matching can be extended to non-perturbative contributions from wrapped D3-branes. A Bethe ansatz approach to computing the superconformal index exists also when the ratio of the angular momentum chemical potentials is any rational number, but in those cases it involves a sum over a very large number of terms (growing exponentially with $N$). Benini et al showed that a specific one of these terms matches with the black hole, but the role of the other terms is not clear. In this paper we analyze some of the additional contributions to the index in the Bethe ansatz approach, and we find that their matching to the gravity side is much more complicated than in the case of equal chemical potentials. In particular, we find some contributions that are larger than the one which was found to match the black holes, so that they must cancel with other large contributions. We give some evidence that cancellations of this type are possible, but we leave a full understanding of how they work to the future., Comment: v2 - references added

Simulating quantum circuits on classical hardware is a powerful and necessary tool for developing and testing quantum algorithms and hardware as well as evaluating claims of quantum supremacy in the Noisy Intermediate-Scale Quantum (NISQ) regime. Schr\"odinger-style simulations are limited by the exponential growth of the number of state amplitudes which need to be stored. In this work, we apply scalar and vector quantization to Schr\"odinger-style quantum circuit simulations as lossy compression schemes to reduce the number of bits needed to simulate quantum circuits. Using quantization, we can maintain simulation fidelities $>0.99$ when simulating the Quantum Fourier Transform, while using only 7 significand bits in a floating-point number to characterize the real and imaginary components of each amplitude. Furthermore, using vector quantization, we propose a method to bound the number of bits/amplitude needed to store state vectors in a simulation of a circuit that achieves a desired fidelity, and show that for a 6 qubit simulation of the Quantum Fourier Transform, 15 bits/amplitude is sufficient to maintain fidelity $>0.9$ at $10^4$ depth.

A comparison of the distances to the Huanan Seafood Market of early Covid cases with known links to the market versus cases without known links shows results apparently incompatible with a location model lacking proximity ascertainment bias. The sign of the difference instead agrees with a model in which such ascertainment bias is large. In the presence of such bias inferences based on the clustering of case locations become unreliable., Comment: short, v2 updates a reference and notes that linked and unlinked cases are not displaced in the same direction from the market. v3 more refs. v4-small correction to an angle, remembered that latitude and longitude are not equivalent except at equator. v5 includes response to Debarre/Worobey Reply