Search

How effectively can LLM-based AI assistants utilize their memory (context) to perform various tasks? Traditional data benchmarks, which are often manually crafted, suffer from several limitations: they are static, susceptible to overfitting, difficult to interpret, and lack actionable insights--failing to pinpoint the specific capabilities a model lacks when it does not pass a test. In this paper, we present a framework for automatically generating a comprehensive set of tests to evaluate models' abilities to use their memory effectively. Our framework extends the range of capability tests beyond the commonly explored (passkey, key-value, needle in the haystack) search, a dominant focus in the literature. Specifically, we evaluate models on atomic tasks such as searching, recalling, editing, matching, comparing information in context memory, and performing basic operations when inputs are structured into distinct blocks, simulating real-world data. Additionally, we design composite tests to investigate the models' ability to maintain state while operating on memory. Our benchmark enables an interpretable, detailed assessment of memory capabilities of LLMs.

Future LLM agents are likely to communicate on behalf of users with other entity-representing agents on tasks that entail long-horizon plans with interdependent goals. Current work does not focus on such agentic networks, nor does it address their challenges. Thus, we first identify the required properties of agents' communication, which should be proactive and adaptable. It needs to satisfy 1) privacy: agents should not share more than what is needed for the task, and 2) security: the communication must preserve integrity and maintain utility against selfish entities. We design a use case (travel planning) as a testbed that exemplifies these requirements, and we show examples of how this can go wrong. Next, we propose a practical design, inspired by established network security principles, for constrained LLM agentic networks that balance adaptability, security, and privacy. Our framework automatically constructs and updates task-specific rules from prior simulations to build firewalls. We offer layers of defense to 1) convert free-form input to a task-specific protocol, 2) dynamically abstract users' data to a task-specific degree of permissiveness, and 3) self-correct the agents' trajectory.

Binary-level pointer analysis can be of use in symbolic execution, testing, verification, and decompilation of software binaries. In various such contexts, it is crucial that the result is trustworthy, i.e., it can be formally established that the pointer designations are overapproximative. This paper presents an approach to formally proven correct binary-level pointer analysis. A salient property of our approach is that it first generically considers what proof obligations a generic abstract domain for pointer analysis must satisfy. This allows easy instantiation of different domains, varying in precision, while preserving the correctness of the analysis. In the trade-off between scalability and precision, such customization allows "meaningful" precision (sufficiently precise to ensure basic sanity properties, such as that relevant parts of the stack frame are not overwritten during function execution) while also allowing coarse analysis when pointer computations have become too obfuscated during compilation for sound and accurate bounds analysis. We experiment with three different abstract domains with high, medium, and low precision. Evaluation shows that our approach is able to derive designations for memory writes soundly in COTS binaries, in a context-sensitive interprocedural fashion.

We study the thermodynamic features of static, spherically-symmetric Schwarzschild black holes adopting different types of Barrow entropy. Specifically, in addition to the standard Barrow entropy, we consider a logarithmic-corrected type of this entropy by taking into account some loop quantum gravity effects. Moreover, we investigate the black hole thermodynamics from the viewpoint of Barrow entropy in presence of non-extensivity effects coming from the Tsallis statistics. Finally, we compare the results obtained for different Barrow-based entropies., Comment: 19 pages, 2 figures, 1 table, Accepted for publication in EPJC

We explore various aspects of semi-classical spin hydrodynamics, where hydrodynamic currents are derived from an expansion in the reduced Planck constant $\hbar$, incorporating both flat and curved spacetimes. After establishing covariant definitions for angular momentum currents, we demonstrate that the conservation of the energy-momentum tensor requires modifications involving the Riemann curvature and the spin tensors. We also revise pseudo-gauge transformations to ensure their applicability in curved spacetimes. Key assumptions for semi-classical spin hydrodynamics are introduced, enabling studies without explicitly invoking quantum kinetic theory. We derive and analyze the linearized semi-classical spin hydrodynamic equations, proving that spin and fluid modes decouple in the linear regime. As a concrete example, we study the ideal-spin approximation in a dissipative fluid with shear viscosity. This analysis confirms our general result: the damping of spin waves is governed solely by spin relaxation time coefficients, independent of linear fluid perturbations. We also examine the Gibbs stability criterion and reveal its limitations at first order in $\hbar$, signaling the inherent anisotropy of the equilibrium state, which remains unaddressed in current semi-classical spin hydrodynamics formulations. Finally, within a conformal Bjorken flow background and using the slow-roll approximation attractor for the fluid sector, we show that the relaxation of the spin potential is governed by spin relaxation time coefficients, mirroring the damping behavior of spin waves in the linear regime., Comment: 31 pages, 2 figures, comments are welcome

The Bekenstein bound, inspired by the physics of black holes, is introduced to constrain the entropy growth of a physical system down to the quantum level in the context of a generalized second law of thermodynamics. We first show that the standard Bekenstein bound is violated when the entropy of a Schwarzschild black hole is described in non-Gaussian statistics Barrow, Tsallis, and Kaniadakis due to the presence of the related indices $\Delta$, $q$ and $\kappa$, respectively. Then, by adding the GUP effects into the Bekenstein bound, we find that the generalized bound is satisfied in the context of the mentioned entropies through a possible connection between the entropies indices and the GUP parameter $\beta$., Comment: 9 pages, 3 figures, Accepted to publish in PLB

Thermoelectric transport has been widely used to study Abrikosov vortex dynamics in unconventional superconductors. However, only a few thermoelectric studies have been conducted near the dimensional crossover that occurs when the vortex-vortex interaction length scale becomes comparable to the sample size. Here we report the effects of finite size on the dissipation mechanisms of the Nernst effect in the optimally doped $\text{Bi}_{2}\text{Sr}_{2}\text{CaCu}_{2}\text{O}_{8+x}$ high-temperature superconductor, down to the atomic length limit. To access this regime, we develop a new generation of thermoelectric chips based on silicon nitride microprinted circuit boards. These chips ensure optimized signals while preventing sample deterioration. Our results demonstrate that lateral confinement at the nanoscale can effectively reduce vortex dissipation. Investigating vortex dissipation at the micro- and nano-scale is essential for creating stable, miniaturized superconducting circuits.

Prior Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs), adapted from classification model attacks, fail due to ignoring the generative process of LLMs across token sequences. In this paper, we present a novel attack that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior loss-based approaches, revealing context-dependent memorization patterns in pre-trained LLMs.

We study slow-roll inflation as a common feature in Metric-affine Theories of Gravity. In particular, we take into account extended metric, teleparallel and symmetric-teleparallel theories of gravity, based on different geometric invariants, discussing analogies and differences. The analysis for each model is performed in two approaches. First, we focus on the {\it potential-slow-roll approach} by studying the reconstructed potentials for different forms of the extended models related to the considered gravitational theory in the Einstein frame. Secondly, we investigate the {\it Hubble-slow-roll approach} for some conventional inflationary potentials related to the specific extended model in the Jordan frame. We compare all results with cosmic microwave background anisotropy observations coming from Planck 2018 and BICEP2/Keck array satellites in order to find the observational constraints on the parameters space of the models as well as their prediction from the spectral parameters. Eventually, we attempt to present a qualitative comparison between three classes of considered modified gravities using the obtained inflationary results. The aim is to select, in principle, cosmological signatures capable of discriminating among concurrent models in view to point out the representation of gravity, according with geometric invariants, which better addresses the early universe dynamics., Comment: 56 pages, 23 figures, 2 tables

Machine learning models can leak private information about their training data, but the standard methods to measure this risk, based on membership inference attacks (MIAs), have a major limitation. They only check if a given data point \textit{exactly} matches a training point, neglecting the potential of similar or partially overlapping data revealing the same private information. To address this issue, we introduce the class of range membership inference attacks (RaMIAs), testing if the model was trained on any data in a specified range (defined based on the semantics of privacy). We formulate the RaMIAs game and design a principled statistical test for its complex hypotheses. We show that RaMIAs can capture privacy loss more accurately and comprehensively than MIAs on various types of data, such as tabular, image, and language. RaMIA paves the way for a more comprehensive and meaningful privacy auditing of machine learning algorithms.

The differential rotation plays a crucial role in the dynamics of the Sun. We study the solar rotation and its correlation with solar activity by applying a modified machine learning algorithm to identify and track coronal bright points (CBPs) from the Solar Dynamics Observatory/Atmospheric Imaging Assembly observations at 193 \AA\ during cycle 24. For more than 321,440 CBPs, the sidereal and meridional velocities are computed. We find the occurring height of CBPs about 5627 km above the photosphere. We obtain a rotational map for the corona by tracking CBPs at the formation height of Fe\,{\sc xii} (193 \AA) emissions. The equator rotation (14.$^{\circ}$40 to 14.$^{\circ}$54 day$^{-1}$) and latitudinal gradient of rotation ($ - $3.$^{\circ}$0 to $ - $2.$^{\circ}$64 day$^{-1}$) show very slightly positive and negative trends with solar activity (sunspots and flares), respectively. For cycle 24, our investigations show that the northern hemisphere has more differential rotation than the southern hemisphere, confirmed by the asymmetry of the midlatitude rotation parameter. The asymmetry (ranked) of the latitudinal gradient of the rotation parameter is concordant with the sunspot numbers for 7 yr within the 9 yr of the cycle; however, for only 3 yr, it is concordant with the flare index. The minimum horizontal Reynolds stress changes from about $ - $2500 m$^{2}$ s$^{-2}$ (corresponding to high activity) in 2012 and 2014 to $ - $100 m$^{2}$ s$^{-2}$ (corresponding to low activity) in 2019 over 5$^{\circ}$ to 35$^{\circ}$ latitudes within cycle 24. We conclude that the negative horizontal Reynolds stress (momentum transfer toward the Sun's equator) is a helpful indication of solar activity.

Watermarking is a key technique for detecting AI-generated text. In this work, we study its vulnerabilities and introduce the Smoothing Attack, a novel watermark removal method. By leveraging the relationship between the model's confidence and watermark detectability, our attack selectively smoothes the watermarked content, erasing watermark traces while preserving text quality. We validate our attack on open-source models ranging from $1.3$B to $30$B parameters on $10$ different watermarks, demonstrating its effectiveness. Our findings expose critical weaknesses in existing watermarking schemes and highlight the need for stronger defenses.

The principle of data minimization aims to reduce the amount of data collected, processed or retained to minimize the potential for misuse, unauthorized access, or data breaches. Rooted in privacy-by-design principles, data minimization has been endorsed by various global data protection regulations. However, its practical implementation remains a challenge due to the lack of a rigorous formulation. This paper addresses this gap and introduces an optimization framework for data minimization based on its legal definitions. It then adapts several optimization algorithms to perform data minimization and conducts a comprehensive evaluation in terms of their compliance with minimization objectives as well as their impact on user privacy. Our analysis underscores the mismatch between the privacy expectations of data minimization and the actual privacy benefits, emphasizing the need for approaches that account for multiple facets of real-world privacy risks.

We consider inflation with a constant rate of rolling in which a complex scalar field plays the role of inflaton during the inflationary epoch. We implement the inflationary analysis for an accredited angular speed $\dot{\theta}$ which satisfies our dynamical equations. Scalar and tensorial perturbations generated in the framework of constant roll inflation with a complex field are studied. In this respect, we find analytically solutions to the gauge invariant fluctuations, with which an expression for the scalar power spectrum together with its scalar index spectral in this scenario were found. By comparing the obtained results with the observations coming from the cosmic microwave background anisotropies, the constraints on the parameters space of the model and also its predictions are analyzed and discussed., Comment: 14 pages, 3 figures, to appear in Annals of Physics

Machine Learning (ML) algorithms are generally designed for scenarios in which all data is stored in one data center, where the training is performed. However, in many applications, e.g., in the healthcare domain, the training data is distributed among several entities, e.g., different hospitals or patients' mobile devices/sensors. At the same time, transferring the data to a central location for learning is certainly not an option, due to privacy concerns and legal issues, and in certain cases, because of the communication and computation overheads. Federated Learning (FL) is the state-of-the-art collaborative ML approach for training an ML model across multiple parties holding local data samples, without sharing them. However, enabling learning from distributed data over such edge Internet of Things (IoT) systems (e.g., mobile-health and wearable technologies, involving sensitive personal/medical data) in a privacy-preserving fashion presents a major challenge mainly due to their stringent resource constraints, i.e., limited computing capacity, communication bandwidth, memory storage, and battery lifetime. In this paper, we propose a privacy-preserving edge FL framework for resource-constrained mobile-health and wearable technologies over the IoT infrastructure. We evaluate our proposed framework extensively and provide the implementation of our technique on Amazon's AWS cloud platform based on the seizure detection application in epilepsy monitoring using wearable technologies.

We show that, in ideal-spin hydrodynamics, the components of the spin tensor follow damped wave equations. The damping rate is related to nonlocal collisions of the particles in the fluid, which enter at first order in $\hbar$ in a semi-classical expansion. This rate provides an estimate for the timescale of spin equilibration and is computed by considering a system of spin-1/2 fermions interacting via a quartic self-interaction as well as via (screened) one-gluon exchange. It is found that the relaxation times of the components of the spin tensor can become very large compared to the usual dissipative timescales of the system. Our results suggest that the spin degrees of freedom in a heavy-ion collision may not be in equilibrium by the time of freeze-out, and thus should be treated dynamically., Comment: 13 pages, 1 figure

To implement important quality attributes of software such as architectural security tactics, developers incorporate API of software frameworks, as building blocks, to avoid re-inventing the wheel and improve their productivity. However, this is a challenging and error-prone task, especially for novice programmers. Despite the advances in the field of API-based program synthesis, the state-of-the-art suffers from a twofold shortcoming when it comes to architectural tactic implementation tasks. First, the specification of the desired tactic must be explicitly expressed, which is out of the knowledge of such programmers. Second, these approaches synthesize a block of code and leave the task of breaking it down into smaller pieces, adding each piece to the proper location in the code, and establishing correct dependencies between each piece and its surrounding environment as well as the other pieces, to the programmer. To mitigate these challenges, we introduce IPSynth, a novel inter-procedural program synthesis approach that automatically learns the specification of the tactic, synthesizes the tactic as inter-related code snippets, and adds them to an existing code base. We extend our first-place award-winning extended abstract recognized at the 36th IEEE/ACM International Conference on Automated Software Engineering (ASE'21) research competition track. In this paper, we provide the details of the approach, present the results of the experimental evaluation of IPSynth, and analyses and insights for a more comprehensive exploration of the research topic. Moreover, we compare the results of our approach to one of the most powerful code generator tools, ChatGPT. Our results show that our approach can accurately locate corresponding spots in the program, synthesize needed code snippets, add them to the program, and outperform ChatGPT in inter-procedural tactic synthesis tasks.

We expand the classical phase-space distribution function to incorporate couplings between spin and electromagnetic fields. This extension has led to the derivation of modified constitutive relations for the charge current, energy-momentum tensor, and spin tensor. Due to these couplings, the new tensors are revised from their perfect fluid analogues, creating an interplay between the background and spin fluid equations of motion. The corrections introduced in our framework have the potential to shed light on the experimentally observed discrepancies in the spin polarization measurements of Lambda hyperons., Comment: Contribution to SPIN 2023 conference

In this work, we explore the prospect of generalizing the constant-roll condition in canonical inflationary model to non-canonical models. To find a natural generalization, we focus on three manifestations of this condition and construct constant-roll models corresponding to each manifestation. These models are not equivalent but reduce to the familiar constant-roll model in canonical limit. To showcase the applicability of our generalized mechanism, we examine a specific class of non-canonical models, which can be viewed as extensions of k/G inflation. In these models sound speed is constant. We conduct a comparative study, and with an analytical examination of the model, specify instances when our constant-roll conditions yield dissimilar outcomes and when they exhibit analogies. We also apply our findings to scrutinize another kinetically driven inflationary model with varying sound speed. We demonstrate that each of our constant-roll conditions leads to a unique set of solutions. Afterward, we construct a four-stage constant-roll kinetically driven inflation that complies with CMB constraints, it sustains for a sufficiently long period of time, and finally gracefully exits. In this model the spectrum of curvature perturbations is enhanced in a brief phase of non-slow-roll inflationary evolution. Employing numerical methods, we analyse this scenario to elucidate how altering the constant-roll condition impacts the power spectrum and the model's dynamics., Comment: 40 pages, 12 figures; v3, JCAP version

Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning., Comment: ICML 2024