Search

With the growing complexity and capability of large language models, a need to understand model reasoning has emerged, often motivated by an underlying goal of controlling and aligning models. While numerous interpretability and steering methods have been proposed as solutions, they are typically designed either for understanding or for control, seldom addressing both, with the connection between interpretation and control more broadly remaining tenuous. Additionally, the lack of standardized applications, motivations, and evaluation metrics makes it difficult to assess these methods' practical utility and efficacy. To address this, we propose intervention as a fundamental goal of interpretability and introduce success criteria to evaluate how well methods are able to control model behavior through interventions. We unify and extend four popular interpretability methods--sparse autoencoders, logit lens, tuned lens, and probing--into an abstract encoder-decoder framework. This framework maps intermediate latent representations to human-interpretable feature spaces, enabling interventions on these interpretable features, which can then be mapped back to latent representations to control model outputs. We introduce two new evaluation metrics: intervention success rate and the coherence-intervention tradeoff, designed to measure the accuracy of explanations and their utility in controlling model behavior. Our findings reveal that (1) although current methods allow for intervention, they are inconsistent across models and features, (2) lens-based methods outperform others in achieving simple, concrete interventions, and (3) interventions often compromise model performance and coherence, underperforming simpler alternatives, such as prompting, for steering model behavior and highlighting a critical shortcoming of current interpretability approaches in real-world applications requiring control.

Nodal line semimetals, a class of topological quantum materials, exhibit a variety of novel phenomena due to their properties, such as bands touching on a one-dimensional line or a ring in the Brillouin zone and drumhead-like surface states. In addition, these semimetals are protected by the combined space-inversion and time-reversal ($\mathcal{PT}$) symmetry. In this study, we investigate the longitudinal DC conductivity of the Dirac nodal line semimetals for the broken $\mathcal{PT}$-symmetric system by the mass term. Here, using the quantum kinetic technique, we find the intrinsic (field-driven) and extrinsic (scattering-driven) contributions to the total DC conductivity due to interband effects. Interestingly, the resulting intrinsic conductivity is the Fermi sea contribution, while the extrinsic stems from the Fermi surface contribution. We show that at low chemical potential, the extrinsic part contributes more and dominates over the traditional Drude intraband term, while at the high chemical potential, the intrinsic conductivity contributes. Furthermore, the total DC response due to interband effects saturates at high chemical potential and its strength decreases with increasing mass value. Our findings suggest that the extrinsic contributions are rich enough to understand the overall feature of the response for the three-dimensional system., Comment: Accepted for publication in the Physical Review B

The abundance of massive primordial black holes has historically been constrained by dynamical probes. Since these objects can participate in hard few-body scattering processes, they can readily transfer energy to stellar systems, and, in particular, can disrupt wide binaries. However, disruption is not the only possible outcome of such few-body processes. Primordial black holes could also participate in exchange processes, in which one component of a binary system is ejected and replaced by the black hole itself. In this case, the remaining object in the binary would dynamically appear to have an invisible companion. We study the rate of exchange processes for primordial black holes as a component of dark matter and evaluate possible mechanisms for detecting such binaries. We find that many such binaries plausibly exist in the Solar neighborhood, and show that this process can account for observed binary systems whose properties run counter to the predictions of isolated binary evolution., Comment: 15 pages, 4 figures

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

The SSPACE Astrobiology Payload (SAP) series, starting with the SAP-1 project is designed to conduct in-situ microbiology experiments in low earth orbit. This payload series aims to understand the behaviour of microbial organisms in space, particularly those critical for human health, and the corresponding effects due to microgravity and solar/galactic radiation. SAP-1 focuses on studying Bacillus clausii and Bacillus coagulans, bacteria beneficial to humans. It aims to provide a space laboratory for astrobiology experiments under microgravity conditions. The hardware developed for these experiments is indigenous and tailored to meet the unique requirements of autonomous microbiology experiments by controlling pressure, temperature, and nutrition flow to bacteria. A rotating platform, which forms the core design, is innovatively utilised to regulate the flow and mixing of nutrients with dormant bacteria. The technology demonstration models developed at SSPACE have yielded promising results, with ongoing efforts to refine, adapt for space conditions, and prepare for integration with nanosatellites or space modules. The anticipated payload will be compact, approximately 1U in size (10cm x 10cm x 10cm), consume less than 5W power, and offer flexibility for various microbiological studies., Comment: 20 Pages, Submitted to Advances in Space Research

Do different generative image models secretly learn similar underlying representations? We investigate this by measuring the latent space similarity of four different models: VAEs, GANs, Normalizing Flows (NFs), and Diffusion Models (DMs). Our methodology involves training linear maps between frozen latent spaces to "stitch" arbitrary pairs of encoders and decoders and measuring output-based and probe-based metrics on the resulting "stitched'' models. Our main findings are that linear maps between latent spaces of performant models preserve most visual information even when latent sizes differ; for CelebA models, gender is the most similarly represented probe-able attribute. Finally we show on an NF that latent space representations converge early in training.

As Artificial Intelligence (AI) tools are increasingly employed in diverse real-world applications, there has been significant interest in regulating these tools. To this end, several regulatory frameworks have been introduced by different countries worldwide. For example, the European Union recently passed the AI Act, the White House issued an Executive Order on safe, secure, and trustworthy AI, and the White House Office of Science and Technology Policy issued the Blueprint for an AI Bill of Rights (AI BoR). Many of these frameworks emphasize the need for auditing and improving the trustworthiness of AI tools, underscoring the importance of safety, privacy, explainability, fairness, and human fallback options. Although these regulatory frameworks highlight the necessity of enforcement, practitioners often lack detailed guidance on implementing them. Furthermore, the extensive research on operationalizing each of these aspects is frequently buried in technical papers that are difficult for practitioners to parse. In this write-up, we address this shortcoming by providing an accessible overview of existing literature related to operationalizing regulatory principles. We provide easy-to-understand summaries of state-of-the-art literature and highlight various gaps that exist between regulatory guidelines and existing AI research, including the trade-offs that emerge during operationalization. We hope that this work not only serves as a starting point for practitioners interested in learning more about operationalizing the regulatory guidelines outlined in the Blueprint for an AI BoR but also provides researchers with a list of critical open problems and gaps between regulations and state-of-the-art AI research. Finally, we note that this is a working paper and we invite feedback in line with the purpose of this document as described in the introduction., Comment: 15 pages

This work expands on our recently introduced low Mach enthalpy method [1] for simulating the melting and solidification of a phase change material (PCM) alongside (or without) an ambient gas phase. The method captures PCM's volume change (shrinkage or expansion) by accounting for density change-induced flows. We present several improvements to the original work. First, we introduce consistent time integration schemes for the mass, momentum, and enthalpy equations, which enhance the method stability. Demonstrating the effectiveness of this scheme, we show that a system free of external forces and heat sources can conserve its initial mass, momentum, enthalpy, and phase composition. This allows the system to transition from a non-isothermal, non-equilibrium, phase-changing state to an isothermal, equilibrium state without exhibiting unrealistic behavior. Furthermore, we show that the low Mach enthalpy method accurately simulates thermocapillary flows without introducing spurious phase changes. We propose an analytical model to validate advanced CFD codes for simulating metal manufacturing processes like welding and 3D printing. These processes involve a heat source melting metal or alloy in an inert gas environment. Traditionally, validation relied on manipulating material properties to match complex experiments. Our model uses the Stefan problem with a density jump to provide a straightforward method for validating multiphysics simulations involving heat sources and phase changes in three-phase flows.Lastly, we demonstrate the practical utility of the method in modeling porosity defects (gas bubble trapping) during metal solidification. A field extension technique is used to accurately apply surface tension forces in a three-phase flow situation. This is where part of the bubble surface is trapped within the (moving) solidification front.

Perovskite structure materials based on the Ba1-xGdxTiO3 system, where x = 0.001, 0.002, 0.003, 0.004 and 0.005, were prepared via the Pechini chemical synthesis route. The dielectric properties have been analyzed over a wide temperature and frequency range, revealing a significant contribution of the conduction mechanisms in the dielectric response of the studied ceramics. In fact, by using the Davidson-Cole formalism, the observed electrical behavior was found to be associated with relaxation processes related to intrinsic defects mobility promoted by a thermally-activated polaronic mechanism. The obtained values of the activation energy for the relaxation processes, estimated from the Arrhenius law for the mean relaxation time, revealed a decrease from 0.29 up to 0.21 eV as the Gd-doping concentration increases, which suggests the conduction process to be associated with the polaronic effects due to the coexistence of Ti4+ and Ti3+ ions in the structure. Analysis from the conductivity formalism, by using the Jonscher universal power-law, confirmed the polaron-type conduction mechanism for the dielectric dispersion, as suggested by the dielectric analysis, being the nature of the hopping mechanism governed by small polaron hopping (SPH) charge transport in the studied Ba1-xGdxTiO3 ceramics., Comment: 26 pages, 9 figures, 1 table

Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4,409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question categories, reflecting varied entity popularity from popular to long-tail, and temporal dynamisms ranging from years to seconds. Our evaluation of this benchmark highlights the gap to fully trustworthy QA. Whereas most advanced LLMs achieve <=34% accuracy on CRAG, adding RAG in a straightforward manner improves the accuracy only to 44%. State-of-the-art industry RAG solutions only answer 63% of questions without any hallucination. CRAG also reveals much lower accuracy in answering questions regarding facts with higher dynamism, lower popularity, or higher complexity, suggesting future research directions. The CRAG benchmark laid the groundwork for a KDD Cup 2024 challenge and attracted thousands of participants and submissions. We commit to maintaining CRAG to serve research communities in advancing RAG solutions and general QA solutions. CRAG is available at https://github.com/facebookresearch/CRAG/., Comment: NeurIPS 2024 Datasets and Benchmarks Track

The exploration of dark sector interactions via gravitational waves (GWs) from binary inspirals has been a subject of recent interest. We study dark forces using extreme mass ratio inspirals (EMRIs), pointing out two issues of interest. Firstly, the innermost stable circular orbit (ISCO) of the EMRI, which sets the characteristic length scale of the system and hence the dark force range to which it exhibits enhanced sensitivity, probes force mediator masses that complement those studied with supermassive black hole (SMBH) or neutron star binaries. The LISA mission (the proposed $\mu$Ares detector) will probe mediators with masses $m_V \sim 10^{-16}~{\rm eV}$ ($m_V \sim 10^{-18}~{\rm eV}$), corresponding to ISCOs of $10^6 M_\odot$ ($10^8 M_\odot$) central SMBHs. Secondly, while the sensitivity to dark couplings is typically limited by the uncertainty in the binary component masses, independent mass measurements of the central SMBH through reverberation mapping campaigns or the motion of dynamical tracers enable one to break this degeneracy. Our results, therefore, highlight the necessity for coordinated studies, loosely referred to as "multimessenger", between future $\mu{\rm Hz}-{\rm mHz}$ GW observatories and ongoing and forthcoming SMBH mass measurement campaigns, including OzDES-RM, SDSS-RM, and SDSS-V Black Hole Mapper., Comment: 16+6 pages, 6 figures

For decades, the computational multiphase flow community has grappled with mass loss in the level set method. Numerous solutions have been proposed, from fixing the reinitialization step to combining the level set method with other conservative schemes. However, our work reveals a more fundamental culprit: the smooth Heaviside and delta functions inherent to the standard formulation. Even if reinitialization is done exactly, i.e., the zero contour interface remains stationary, the use of smooth functions lead to violation of mass conservation. We propose a novel approach using variational analysis to incorporate a mass conservation constraint. This introduces a Lagrange multiplier that enforces overall mass balance. Notably, as the delta function sharpens, i.e., approaches the Dirac delta limit, the Lagrange multiplier approaches zero. However, the exact Lagrange multiplier method disrupts the signed distance property of the level set function. This motivates us to develop an approximate version of the Lagrange multiplier that preserves both overall mass and signed distance property of the level set function. Our framework even recovers existing mass-conserving level set methods, revealing some inconsistencies in prior analyses. We extend this approach to three-phase flows for fluid-structure interaction (FSI) simulations. We present variational equations in both immersed and non-immersed forms, demonstrating the convergence of the former formulation to the latter when the body delta function sharpens. Rigorous test problems confirm that the FSI dynamics produced by our simple, easy-to-implement immersed formulation with the approximate Lagrange multiplier method are accurate and match state-of-the-art solvers.

In this paper, we discuss the development of an annotation schema to build datasets for evaluating the offline harm potential of social media texts. We define "harm potential" as the potential for an online public post to cause real-world physical harm (i.e., violence). Understanding that real-world violence is often spurred by a web of triggers, often combining several online tactics and pre-existing intersectional fissures in the social milieu, to result in targeted physical violence, we do not focus on any single divisive aspect (i.e., caste, gender, religion, or other identities of the victim and perpetrators) nor do we focus on just hate speech or mis/dis-information. Rather, our understanding of the intersectional causes of such triggers focuses our attempt at measuring the harm potential of online content, irrespective of whether it is hateful or not. In this paper, we discuss the development of a framework/annotation schema that allows annotating the data with different aspects of the text including its socio-political grounding and intent of the speaker (as expressed through mood and modality) that together contribute to it being a trigger for offline harm. We also give a comparative analysis and mapping of our framework with some of the existing frameworks., Comment: Accepted for: LREC COLING 2024

Today's virtual reality (VR) systems provide immersive interactions that seamlessly connect users with online services and one another. However, these immersive interfaces also introduce new vulnerabilities, making it easier for users to fall prey to new attacks. In this work, we introduce the immersive hijacking attack, where a remote attacker takes control of a user's interaction with their VR system, by trapping them inside a malicious app that masquerades as the full VR interface. Once trapped, all of the user's interactions with apps, services and other users can be recorded and modified without their knowledge. This not only allows traditional privacy attacks but also introduces new interaction attacks, where two VR users encounter vastly different immersive experiences during their interaction. We present our implementation of the immersive hijacking attack on Meta Quest headsets and conduct IRB-approved user studies that validate its efficacy and stealthiness. Finally, we examine effectiveness and tradeoffs of various potential defenses, and propose a multifaceted defense pipeline., Comment: 13 pages

Understanding the environmental impact of professional activities is becoming paramount in current times, especially within sectors that historically have had significant resource utilisation, such as High Energy Physics (HEP) and related fields. The young High Energy Physicists (yHEP) association launched the Know your footprint (Kyf) campaign to evaluate the CO$_\text{2}$-equivalent emissions generated by HEP-related research. This study delves into the carbon footprints associated with four distinct categories: Experiments, tied to extensive collaborations with substantial infrastructure; Institutional, representing the resource consumption of research institutes and universities; Computing, focusing on simulations and data analysis; and Travel, covering professional trips such as to conferences, meetings, and workshops. The findings in this assessment are integrated into a tool for self-evaluation, the Know-your-footprint (Kyf) calculator, which allows colleagues to assess their personal and professional footprint, and optionally share their data with the yHEP association. The aim of the Kyf campaign is to heighten awareness, foster sustainability, and inspire the community to adopt more environmentally-responsible research methodologies., Comment: 16 pages, 5 figures

Numerical simulation of moving immersed solid bodies in fluids is now practiced routinely following pioneering work of Peskin and co-workers on immersed boundary method (IBM), Glowinski and co-workers on fictitious domain method (FDM), and others on related methods. A variety of variants of IBM and FDM approaches have been published, most of which rely on using a background mesh for the fluid equations and tracking the solid body using Lagrangian points. The key idea that is common to these methods is to assume that the entire fluid-solid domain is a fluid and then to constrain the fluid within the solid domain to move in accordance with the solid governing equations. The immersed solid body can be rigid or deforming. Thus, in all these methods the fluid domain is extended into the solid domain. In this review, we provide a mathemarical perspective of various immersed methods by recasting the governing equations in an extended domain form for the fluid. The solid equations are used to impose appropriate constraints on the fluid that is extended into the solid domain. This leads to extended domain constrained fluid-solid governing equations that provide a unified framework for various immersed body techniques. The unified constrained governing equations in the strong form are independent of the temporal or spatial discretization schemes. We show that particular choices of time stepping and spatial discretization lead to different techniques reported in literature ranging from freely moving rigid to elastic self-propelling bodies. These techniques have wide ranging applications including aquatic locomotion, underwater vehicles, car aerodynamics, and organ physiology (e.g. cardiac flow, esophageal transport, respiratory flows), wave energy convertors, among others. We conclude with comments on outstanding challenges and future directions.

CLIP embeddings have demonstrated remarkable performance across a wide range of multimodal applications. However, these high-dimensional, dense vector representations are not easily interpretable, limiting our understanding of the rich structure of CLIP and its use in downstream applications that require transparency. In this work, we show that the semantic structure of CLIP's latent space can be leveraged to provide interpretability, allowing for the decomposition of representations into semantic concepts. We formulate this problem as one of sparse recovery and propose a novel method, Sparse Linear Concept Embeddings, for transforming CLIP representations into sparse linear combinations of human-interpretable concepts. Distinct from previous work, SpLiCE is task-agnostic and can be used, without training, to explain and even replace traditional dense CLIP representations, maintaining high downstream performance while significantly improving their interpretability. We also demonstrate significant use cases of SpLiCE representations including detecting spurious correlations and model editing., Comment: 25 pages, 15 figures, NeurIPS 2024. Code is provided at https://github.com/AI4LIFE-GROUP/SpLiCE

This work investigates how hierarchically structured data can help neural networks learn conceptual representations of cathedrals. The underlying WikiScenes dataset provides a spatially organized hierarchical structure of cathedral components. We propose a novel hierarchical contrastive training approach that leverages a triplet margin loss to represent the data's spatial hierarchy in the encoder's latent space. As such, the proposed approach investigates if the dataset structure provides valuable information for self-supervised learning. We apply t-SNE to visualize the resultant latent space and evaluate the proposed approach by comparing it with other dataset-specific contrastive learning methods using a common downstream classification task. The proposed method outperforms the comparable weakly-supervised and baseline methods. Our findings suggest that dataset structure is a valuable modality for weakly-supervised learning.

Student attention is an indispensable input for uncovering their goals, intentions, and interests, which prove to be invaluable for a multitude of research areas, ranging from psychology to interactive systems. However, most existing methods to classify attention fail to model its complex nature. To bridge this gap, we propose AttentioNet, a novel Convolutional Neural Network-based approach that utilizes Electroencephalography (EEG) data to classify attention into five states: Selective, Sustained, Divided, Alternating, and relaxed state. We collected a dataset of 20 subjects through standard neuropsychological tasks to elicit different attentional states. The average across-student accuracy of our proposed model at this configuration is 92.3% (SD=3.04), which is well-suited for end-user applications. Our transfer learning-based approach for personalizing the model to individual subjects effectively addresses the issue of individual variability in EEG signals, resulting in improved performance and adaptability of the model for real-world applications. This represents a significant advancement in the field of EEG-based classification. Experimental results demonstrate that AttentioNet outperforms a popular EEGnet baseline (p-value < 0.05) in both subject-independent and subject-dependent settings, confirming the effectiveness of our proposed approach despite the limitations of our dataset. These results highlight the promising potential of AttentioNet for attention classification using EEG data., Comment: 8 pages, 4 figures, Accepted in AFFECTIVE COMPUTING + INTELLIGENT INTERACTION Conference 2023