Search

Video understanding has witnessed significant progress with recent video foundation models demonstrating strong performance owing to self-supervised pre-training objectives; Masked Autoencoders (MAE) being the design of choice. Nevertheless, the majority of prior works that leverage MAE pre-training have focused on relatively short video representations (16 / 32 frames in length) largely due to hardware memory and compute limitations that scale poorly with video length due to the dense memory-intensive self-attention decoding. One natural strategy to address these challenges is to subsample tokens to reconstruct during decoding (or decoder masking). In this work, we propose an effective strategy for prioritizing tokens which allows training on longer video sequences (128 frames) and gets better performance than, more typical, random and uniform masking strategies. The core of our approach is an adaptive decoder masking strategy that prioritizes the most important tokens and uses quantized tokens as reconstruction objectives. Our adaptive strategy leverages a powerful MAGVIT-based tokenizer that jointly learns the tokens and their priority. We validate our design choices through exhaustive ablations and observe improved performance of the resulting long-video (128 frames) encoders over short-video (32 frames) counterparts. With our long-video masked autoencoder (LVMAE) strategy, we surpass state-of-the-art on Diving48 by 3.9 points and EPIC-Kitchens-100 verb classification by 2.5 points while relying on a simple core architecture and video-only pre-training (unlike some of the prior works that require millions of labeled video-text pairs or specialized encoders)., Comment: 10.5 pages of main paper, 25 pages total, 4 figures and 10 tables. To appear in NeurIPS'24

We introduce the notions of weighted extremal K{\"a}hler twins together with the related notion of extremal Sasaki twins. In the K\"ahler setting this leads to a generalization of the twinning phenomenon appearing among LeBrun's strongly Hermitian solutions to the Einstein-Maxwell equations on the first Hirzebruch surface \cite{Leb16} to weighted extremal metrics on Hirzebruch surfaces in general. We discover that many twins appear and that this can be viewed in the Sasaki setting as a case where we have more than one extremal ray in the Sasaki cone even when we do not allow changes within the isotopy class. We also study extremal Sasaki twins directly in the Sasaki setting with a main focus on the toric Sasaki case., Comment: 35 pages, 3 figures

This work is a sequel to our two 2023 publications [PLB 837 137669, NPA 1039 122725] where fitting 14 1$s_\Lambda$ and 1$p_\Lambda$ single-particle binding energies in hypernuclei across the periodic table led to a well-defined $\Lambda$-nucleus optical potential. The potential consists of a Pauli modified linear-density ($\Lambda N$) and a quadratic-density ($\Lambda NN$) terms. The present work reports on extending the above analysis to 21 $\Lambda$ single-particle data points input by including 1$d_\Lambda$ and 1$f_\Lambda$ states in medium-weight and heavy hypernuclei. The upgraded results for the $\Lambda N$ and $\Lambda NN$ potential depths at nuclear-matter density $\rho_0=0.17$~fm$^{-3}$, $D^{(2)}_\Lambda=-37.5\mp 0.7$~MeV and $D^{(3)}_\Lambda=+9.8\pm 1.2$~MeV together with the total depth $D_\Lambda=-27.7\pm 0.5$~MeV, agree within errors with the earlier results. The $\Lambda$ hypernuclear overbinding associated with the $\Lambda N$-induced potential depth $D^{(2)}_\Lambda$ agrees quantitatively with a recent combined analysis of low-energy $\Lambda p$ scattering data and correlation functions [PLB 850 (2024) 138550]. These results, particularly the size of the repulsive $D^{(3)}_\Lambda$, provide an essential input towards resolving the 'hyperon puzzle' in the core of neutron stars. We also show that a key property of our $\Lambda NN$-induced potential term, i.e. a need to suppress the quadratic-density $\Lambda NN$ term involving an excess neutron and a $N=Z$ core nucleon, can be tested in the forthcoming JLab E12-15-008 experiment., Comment: 10 pages, 3 figures, 1 table. Submitted to Proceedings of International Conference on Exotic Atoms and Related Topics and Conference on Low Energy Antiprotons (EXA-LEAP2024)

Large language models can absorb a massive amount of knowledge through pretraining, but pretraining is inefficient for acquiring long-tailed or specialized facts. Therefore, fine-tuning on specialized or new knowledge that reflects changes in the world has become popular, though it risks disrupting the model's original capabilities. We study this fragility in the context of continual memorization, where the model is trained on a small set of long-tail factoids (factual associations) and must retain these factoids after multiple stages of subsequent training on other datasets. Through extensive experiments, we show that LLMs suffer from forgetting across a wide range of subsequent tasks, and simple replay techniques do not fully prevent forgetting, especially when the factoid datasets are trained in the later stages. We posit that there are two ways to alleviate forgetting: 1) protect the memorization process as the model learns the factoids, or 2) reduce interference from training in later stages. With this insight, we develop an effective mitigation strategy: REMIX (Random and Generic Data Mixing). REMIX prevents forgetting by mixing generic data sampled from pretraining corpora or even randomly generated word sequences during each stage, despite being unrelated to the memorized factoids in the first stage. REMIX can recover performance from severe forgetting, often outperforming replay-based methods that have access to the factoids from the first stage. We then analyze how REMIX alters the learning process and find that successful forgetting prevention is associated with a pattern: the model stores factoids in earlier layers than usual and diversifies the set of layers that store these factoids. The efficacy of REMIX invites further investigation into the underlying dynamics of memorization and forgetting, opening exciting possibilities for future research.

Large language models (LLMs) generate diverse, situated, persuasive texts from a plurality of potential perspectives, influenced heavily by their prompts and training data. As part of LLM adoption, we seek to characterize - and ideally, manage - the socio-cultural values that they express, for reasons of safety, accuracy, inclusion, and cultural fidelity. We present a validated approach to automatically (1) extracting heterogeneous latent value propositions from texts, (2) assessing resonance and conflict of values with texts, and (3) combining these operations to characterize the pluralistic value alignment of human-sourced and LLM-sourced textual data.

Audio-based disease prediction is emerging as a promising supplement to traditional medical diagnosis methods, facilitating early, convenient, and non-invasive disease detection and prevention. Multimodal fusion, which integrates features from various domains within or across bio-acoustic modalities, has proven effective in enhancing diagnostic performance. However, most existing methods in the field employ unilateral fusion strategies that focus solely on either intra-modal or inter-modal fusion. This approach limits the full exploitation of the complementary nature of diverse acoustic feature domains and bio-acoustic modalities. Additionally, the inadequate and isolated exploration of latent dependencies within modality-specific and modality-shared spaces curtails their capacity to manage the inherent heterogeneity in multimodal data. To fill these gaps, we propose AuD-Former, a hierarchical transformer network designed for general multimodal audio-based disease prediction. Specifically, we seamlessly integrate intra-modal and inter-modal fusion in a hierarchical manner and proficiently encode the necessary intra-modal and inter-modal complementary correlations, respectively. Comprehensive experiments demonstrate that AuD-Former achieves state-of-the-art performance in predicting three diseases: COVID-19, Parkinson's disease, and pathological dysarthria, showcasing its promising potential in a broad context of audio-based disease prediction tasks. Additionally, extensive ablation studies and qualitative analyses highlight the significant benefits of each main component within our model.

We introduce a framework that allows for the exact analytic treatment of quantum dynamics subject to coherent noise. The noise is modeled via unitary evolution under a Hamiltonian drawn from a random-matrix ensemble for arbitrary Hilbert-space dimension $N$. While the methods we develop apply to generic such ensembles with a notion of rotation invariance, we focus largely on the Gaussian unitary ensemble (GUE). Averaging over the ensemble of ''noisy'' Hamiltonians produces an effective quantum channel, the properties of which are analytically calculable and determined by the spectral form factors of the relevant ensemble. We compute spectral form factors of the GUE exactly for any finite $N$, along with the corresponding GUE quantum channel, and its variance. Key advantages of our approach include the ability to access exact analytic results for any $N$ and the ability to tune to the noise-free limit (in contrast, e.g., to the Haar ensemble), and analytic access to moments beyond the variance. We also highlight some unusual features of the GUE channel, including the nonmonotonicity of the coefficients of various operators as a function of noise strength and the failure to saturate the Haar-random limit, even with infinite noise strength., Comment: 18 pages, 5 figures

Light cone selection effects on cosmic observables must be precisely accounted for in the next generation of surveys, including the Dark Energy Spectroscopic Instrument (DESI) survey. This will allow us to correctly model the data and extract subtle shifts from general-relativistic effects. We examine the effects of peculiar velocities on color selection in spectroscopic galaxy surveys, with a focus on their implications for the galaxy clustering dipole $P_1(k)$. Using DESI Emission Line Galaxy (ELG) targets, we show that peculiar velocities can shift spectral emission features into or out of filter bands, modifying galaxy colors and thereby changing galaxy selection. This phenomenon mimics the effect of evolution bias, and we refer to it as the Doppler bias, $b_D$. The Doppler bias is of comparable size to the evolution bias at $0.8 < z < 1$, where it is largest. This enhances the ELG-LRG (Luminous Red Galaxy) cross-correlation dipole by 15-30%. This could be detectable at the $\sim$4$\sigma$ level for the full DESI survey. Additionally, we found that our $b_D$ estimate is impacted by the incompleteness of the parent ELG sample. Therefore, this work highlights the essential need for careful consideration of spectral-dependent biases caused by peculiar velocities during the selection phase of galaxy surveys, to enable unbiased analyses., Comment: 31 pages, 10 figures

In "Embers of Autoregression" (McCoy et al., 2023), we showed that several large language models (LLMs) have some important limitations that are attributable to their origins in next-word prediction. Here we investigate whether these issues persist with o1, a new system from OpenAI that differs from previous LLMs in that it is optimized for reasoning. We find that o1 substantially outperforms previous LLMs in many cases, with particularly large improvements on rare variants of common tasks (e.g., forming acronyms from the second letter of each word in a list, rather than the first letter). Despite these quantitative improvements, however, o1 still displays the same qualitative trends that we observed in previous systems. Specifically, o1 -- like previous LLMs -- is sensitive to the probability of examples and tasks, performing better and requiring fewer "thinking tokens" in high-probability settings than in low-probability ones. These results show that optimizing a language model for reasoning can mitigate but might not fully overcome the language model's probability sensitivity., Comment: 6 pages; updated to fix typo in Fig 4 caption

Two-dimensional (2D) 1T-VSe$_2$ has prompted significant interest due to the discrepancies regarding alleged ferromagnetism (FM) at room temperature, charge density wave (CDW) states and the interplay between the two. We employed a combined Diffusion Monte Carlo (DMC) and density functional theory (DFT) approach to accurately investigate the magnetic properties and response of strain of monolayer 1T-VSe$_2$. Our calculations show the delicate competition between various phases, revealing critical insights into the relationship between their energetic and structural properties. We went on to perform Classical Monte Carlo simulations informed by our DMC and DFT results, and found the magnetic transition temperature ($T_c$) of the undistorted (non-CDW) FM phase to be 228 K and the distorted (CDW) phase to be 68 K. Additionally, we studied the response of biaxial strain on the energetic stability and magnetic properties of various phases of 2D 1T-VSe$_2$ and found that small amounts of strain can enhance the $T_c$, suggesting a promising route for engineering and enhancing magnetic behavior. Finally, we synthesized 1T-VSe$_2$ and performed Raman spectroscopy measurements, which were in close agreement with our calculated results. Our work emphasizes the role of highly accurate DMC methods in advancing the understanding of monolayer 1T-VSe$_2$ and provides a robust framework for future studies of 2D magnetic materials.

Understanding and modeling collective intelligence is essential for addressing complex social systems. Directed graphs called fuzzy cognitive maps (FCMs) offer a powerful tool for encoding causal mental models, but extracting high-integrity FCMs from text is challenging. This study presents an approach using large language models (LLMs) to automate FCM extraction. We introduce novel graph-based similarity measures and evaluate them by correlating their outputs with human judgments through the Elo rating system. Results show positive correlations with human evaluations, but even the best-performing measure exhibits limitations in capturing FCM nuances. Fine-tuning LLMs improves performance, but existing measures still fall short. This study highlights the need for soft similarity measures tailored to FCM extraction, advancing collective intelligence modeling with NLP., Comment: Camera-ready version accepted for publication in the EMNLP 2024 Workshop NLP4Science

Stochastic resetting, a method for accelerating target search in random processes, often incurs temporal and energetic costs. For a diffusing particle, a lower bound exists for the energetic cost of reaching the target, which is attained at low resetting rates and equals the direct linear transportation cost against fluid drag. Here, we study ``smart resetting," a strategy that aims to beat this lower bound. By strategically resetting the particle only when this benefits its progress toward the target, smart resetting leverages information to minimize energy consumption. We analytically calculate the energetic cost per mean first passage time and show that smart resetting consistently reduces the energetic cost compared to regular resetting. Surprisingly, smart resting achieves the minimum energy cost previously established for regular resetting, irrespective of the resetting rate. Yet, it fails to reduce this cost further. We extend our findings in two ways: first, by examining nonlinear energetic cost functions, and second, by considering smart resetting of drift-diffusion processes.

We propose a new method for generating realistic datasets with distribution shifts using any decoder-based generative model. Our approach systematically creates datasets with varying intensities of distribution shifts, facilitating a comprehensive analysis of model performance degradation. We then use these generated datasets to evaluate the performance of various commonly used networks and observe a consistent decline in performance with increasing shift intensity, even when the effect is almost perceptually unnoticeable to the human eye. We see this degradation even when using data augmentations. We also find that enlarging the training dataset beyond a certain point has no effect on the robustness and that stronger inductive biases increase robustness., Comment: ECCV2024, "Synthetic Data for Computer Vision" workshop

We introduce a benchmark to directly evaluate the alignment between human observers and vision models on a 3D shape inference task. We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape: given a set of images, participants identify which contain the same/different objects, despite considerable viewpoint variation. We draw from a diverse range of images that include common objects (e.g., chairs) as well as abstract shapes (i.e., procedurally generated `nonsense' objects). After constructing over 2000 unique image sets, we administer these tasks to human participants, collecting 35K trials of behavioral data from over 500 participants. This includes explicit choice behaviors as well as intermediate measures, such as reaction time and gaze data. We then evaluate the performance of common vision models (e.g., DINOv2, MAE, CLIP). We find that humans outperform all models by a wide margin. Using a multi-scale evaluation approach, we identify underlying similarities and differences between models and humans: while human-model performance is correlated, humans allocate more time/processing on challenging trials. All images, data, and code can be accessed via our project page., Comment: Project page: https://tzler.github.io/MOCHI/ Code: https://github.com/tzler/mochi_code Huggingface dataset: https://huggingface.co/datasets/tzler/MOCHI

We study projection determinantal point processes and their connection to the squared Grassmannian. We prove that the log-likelihood function of this statistical model has $(n - 1)!/2$ critical points, all of which are real and positive, thereby settling a conjecture of Devriendt, Friedman, Reinke, and Sturmfels.

The purpose of this study was to evaluate the implementation and impact of the Pack program. The Pack was developed by the New York Hall of Science (NYSCI) and includes a digital game and set of curricular and professional development resources that aim to support computational thinking teaching and learning in middle school science and computer science classrooms. The evaluation of the Pack was funded with an Education Innovation and Research grant from the U.S. Department of Education and was carried out by the American Institutes for Research (AIR) during the 2022-23 school year in New York City middle schools. AIR examined the Pack intervention's classroom implementation by teachers and their Grade 6-8 students as well as outcomes related to students' computational thinking skills and their attitudes toward problem solving. Overall, the study yielded promising results about the feasibility and utility of implementing the Pack in middle school science and computer science classrooms. Based on teacher survey and interview responses, teachers reported that NYSCI's professional development was helpful to their teaching and reported that the Pack digital game and lessons were valuable for computational thinking teaching and learning. Using teacher survey and extant data, AIR found that teachers were able to implement the Pack program components with fidelity. To measure the relationship between students' use of the Pack and outcomes related to students' attitudes toward problem solving and computational thinking skills, AIR administered a pre- and posttest survey of student problem solving attitudes and a computational thinking knowledge assessment. AIR found that there were no significant associations between students' use of the Pack and their beliefs about their ability to apply their problem-solving skills to new situations or their beliefs about problem-solving ability as a fixed trait. Similarly, there were no significant associations between teachers' or students' reported use of the Pack and students' scores on a computational thinking assessment. There were also no significant associations between students' reported participation in computational thinking while using the Pack and their scores on a computational thinking assessment.

The emergence of cognition requires a framework that bridges evolutionary principles with neurocomputational mechanisms. This paper introduces the novel "thoughtseed" framework, proposing that cognition arises from the dynamic interaction of self-organizing units of embodied knowledge called "thoughtseeds" within the Global Workspace of consciousness. Leveraging foundational concepts from evolutionary theory, neuronal packets, and free energy principle, we propose a hierarchical model of cognitive states, comprising Neuronal Packet Domains (NPDs), Knowledge Domains (KDs), the thoughtseed network, and meta-cognition. This hierarchical interplay, mediated by nested Markov blankets and reciprocal message passing, facilitates the emergence of thoughtseeds as coherent patterns of activity that guide perception, action, and learning. Thoughtseeds, posited as fundamental units of thought, compete for dominance within the Global Workspace, with the dominant thoughtseed shaping conscious experience and guiding behavior. We present a mathematical framework grounded in active inference and dynamical systems theory to model thoughtseed dynamics and their contribution to the unitary nature of consciousness. The thoughtseed framework offers a promising step towards a novel, biologically-grounded model for understanding the organizing principles and emergence of embodied cognition, offering a unified account of cognitive phenomena, with potential applications in understanding consciousness, attention, and decision-making., Comment: 2nd version. Reduced in size from 88 pages to 48 pages, to focus on the core framework. Items removed: Detailed discussions on Evolutionary Biology, contemplative traditions and computational theories of Mind. Replaced Inner Screen Model with Global Workspace Theory. Mathematical equations have been streamlined and is better organized. preprint arXiv:2408.15982

Relative free energy calculations are now widely used in academia and industry, but the accuracy is often limited by poor sampling of the complexes conformational ensemble. To address this, we have developed a novel method termed Multi-Topology Replica Exchange of Expanded Ensembles (MT-REXEE). This method enables parallel expanded ensemble calculations, facilitating iterative relative free energy computations while allowing conformational exchange between parallel transformations. These iterative transformations are adaptable to any set of systems with a common backbone or central substructure. We demonstrate that the MT-REXEE method maintains thermodynamic cycle closure to the same extent as standard expanded ensemble for both solvation free energy and relative binding free energy. The transformations tested involve simple systems that incorporate diverse heavy atoms and multi-site perturbations of a small molecule core resembling multi-site $\lambda$ dynamics, without necessitating modifications to the MD code, which in our initial implementation is GROMACS. We outline a systematic approach for topology set-up and provide instructions on how to perform inter-replicate coordinate modifications. This work shows that MT-REEXE can be used to perform accurate and reproducible free energy estimates and prompts expansion to more complex test systems and other molecular dynamics simulation infrastructures.

This research presents a comprehensive framework for analyzing liquidity in financial markets, particularly in the context of high-frequency trading. By leveraging advanced machine learning classification techniques, including Logistic Regression, Support Vector Machine, and Random Forest, the study aims to predict minute-level price movements using an extensive set of liquidity metrics derived from the Trade and Quote (TAQ) data. The findings reveal that employing a broad spectrum of liquidity measures yields higher predictive accuracy compared to models utilizing a reduced subset of features. Key liquidity metrics, such as Liquidity Ratio, Flow Ratio, and Turnover, consistently emerged as significant predictors across all models, with the Random Forest algorithm demonstrating superior accuracy. This study not only underscores the critical role of liquidity in market stability and transaction costs but also highlights the complexities involved in short-interval market predictions. The research suggests that a comprehensive set of liquidity measures is essential for accurate prediction, and proposes future work to validate these findings across different stock datasets to assess their generalizability.

The Active Inference framework models perception and action as a unified process, where agents use probabilistic models to predict and actively minimize sensory discrepancies. In complement and contrast, traditional population-based metaheuristics rely on reactive environmental interactions without anticipatory adaptation. This paper proposes the integration of Active Inference into these metaheuristics to enhance performance through anticipatory environmental adaptation. We demonstrate this approach specifically with Ant Colony Optimization (ACO) on the Travelling Salesman Problem (TSP). Experimental results indicate that Active Inference can yield some improved solutions with only a marginal increase in computational cost, with interesting patterns of performance that relate to number and topology of nodes in the graph. Further work will characterize where and when different types of Active Inference augmentation of population metaheuristics may be efficacious.

We report a twisted triple bilayer graphene platform consisting of three units of Bernal bilayer graphene (BLG) consecutively twisted at 1.49{\deg} and 1.68{\deg}. We observe inter-moir\'e Hofstadter butterflies from two co-existing moir\'e superlattices and a Hofstadter butterfly from reconstructed moir\'e-of-moir\'e lattice, and show that their Brown-Zak (BZ) oscillations quantitatively agree with each other, both evidencing strong atomic reconstruction with a lattice constant of 18.1 nm. We further demonstrate such atomic reconstruction strongly enhances the Berry curvature of each moir\'e and moir\'e-of-moir\'e band-insulator state, characterized by measured strong non-local valley Hall effect (VHE) that sensitively depends on the inter-moir\'e competition strength, tunable by manipulating the out-of-the-plane carrier distribution which controls the magnitude of the valley currents. Our study sheds new light on the microscopic mechanism of atomic and electronic reconstruction in twisted-multilayer systems, by investigating novel emergent quantum phenomena of reconstructed quasi-crystalline moir\'e-of-moir\'e superlattice, including a new type of moir\'e-of-moir\'e band-insulator states and atomically enhanced moir\'e Berry curvature. We show that the reconstructed electronic band can be versatilely tuned by electrostatics, providing an approach towards engineering the band structure and its topology for a novel quantum material platform with designer electrical and optical properties.

An I-surface $X$ is a surface of general type with $K_X^2 =1$ and $p_g(X) =2$. This paper studies the asymptotic behavior of the period map for I-surfaces acquiring simple elliptic singularities. First we describe the relationship between the deformation theory of such surfaces and their $d$-semistable models. Next we analyze the mixed Hodge structures on the $d$-semistable models, the corresponding limiting mixed Hodge structures, and the monodromy. There are $6$ possible boundary strata for which the relevant limiting mixed Hodge structures satisfy: $\dim W_1 = 4$, and hence $W_2/W_1$ is of pure type $(1,1)$. We show that, in each case, the nilpotent orbit of limiting mixed Hodge structures determines the boundary stratum and prove a global Torelli theorem for one such stratum., Comment: 40 pages. v.2: typos corrected and other minor revisions

With the growing prevalence of large language models, it is increasingly common to annotate datasets for machine learning using pools of crowd raters. However, these raters often work in isolation as individual crowdworkers. In this work, we regard annotation not merely as inexpensive, scalable labor, but rather as a nuanced interpretative effort to discern the meaning of what is being said in a text. We describe a novel, collaborative, and iterative annotator-in-the-loop methodology for annotation, resulting in a 'Bridging Benchmark Dataset' of comments relevant to bridging divides, annotated from 11,973 textual posts in the Civil Comments dataset. The methodology differs from popular anonymous crowd-rating annotation processes due to its use of an in-depth, iterative engagement with seven US-based raters to (1) collaboratively refine the definitions of the to-be-annotated concepts and then (2) iteratively annotate complex social concepts, with check-in meetings and discussions. This approach addresses some shortcomings of current anonymous crowd-based annotation work, and we present empirical evidence of the performance of our annotation process in the form of inter-rater reliability. Our findings indicate that collaborative engagement with annotators can enhance annotation methods, as opposed to relying solely on isolated work conducted remotely. We provide an overview of the input texts, attributes, and annotation process, along with the empirical results and the resulting benchmark dataset, categorized according to the following attributes: Alienation, Compassion, Reasoning, Curiosity, Moral Outrage, and Respect., Comment: Accepted Paper at AAAI/ACM Artificial Intelligence, Ethics, and Society (AIES)

Clock transitions (CTs) in spin systems, which occur at avoided level crossings, enhance quantum coherence lifetimes T$_2$ because the transition becomes immune to the decohering effects of magnetic field fluctuations to first order. We present the first electron-spin resonance (ESR) characterization of CTs in certain defect-rich silica glasses, noting coherence times up to 16 $\mu$s at the CTs. We find CT behavior at zero magnetic field in borosilicate and aluminosilicate glasses, but not in a variety of silica glasses lacking boron or aluminum. Annealing reduces or eliminates the zero-field signal. Since boron and aluminum have the same valence and are acceptors when substituted for silicon, we suggest the observed CT behavior could be generated by a spin-1 boron vacancy center within the borosilicate glass, and similarly, an aluminum-vacancy center in the aluminosilicate glass., Comment: The Main Manuscript features 9 pages, 5 figures, and 1 table. The Supplement features an additional 6 pages, 4 figures, and two tables

CXL (Compute Express Link) is an emerging open industry-standard interconnect between processing and memory devices that is expected to revolutionize the way systems are designed in the near future. It enables cache-coherent shared memory pools in a disaggregated fashion at unprecedented scales, allowing algorithms to interact with a variety of storage devices using simple loads and stores in a cacheline granularity. Alongside with unleashing unique opportunities for a wide range of applications, CXL introduces new challenges of data management and crash consistency. Alas, CXL lacks an adequate programming model, which makes reasoning about the correctness and expected behaviors of algorithms and systems on top of it nearly impossible. In this work, we present CXL0, the first programming model for concurrent programs running on top of CXL. We propose a high-level abstraction for CXL memory accesses and formally define operational semantics on top of that abstraction. We provide a set of general transformations that adapt concurrent algorithms to the new disruptive technology. Using these transformations, every linearizable algorithm can be easily transformed into its provably correct version in the face of a full-system or sub-system crash. We believe that this work will serve as the stepping stone for systems design and modelling on top of CXL, and support the development of future models as software and hardware evolve.

In recent years, Large Language Models (LLM) have demonstrated impressive capabilities in the field of natural language processing (NLP). This paper explores the application of LLMs in negotiation transcript analysis by the Vanderbilt AI Negotiation Lab. Starting in September 2022, we applied multiple strategies using LLMs from zero shot learning to fine tuning models to in-context learning). The final strategy we developed is explained, along with how to access and use the model. This study provides a sense of both the opportunities and roadblocks for the implementation of LLMs in real life applications and offers a model for how LLMs can be applied to coding in other fields.

This work presents a novel approach to monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, symmetries, occlusions, and lack of annotated real-world data. The method leverages synthetic data generation and domain adaptation techniques to overcome these obstacles. The proposed approach consists of three main components: (1) synthetic data generation using 3D modeling of surgical tools with articulation rigging and physically-based rendering; (2) a tailored pose estimation framework combining object detection with pose estimation and a hybrid geometric fusion strategy; and (3) a training strategy that utilizes both synthetic and real unannotated data, employing domain adaptation on real video data using automatically generated pseudo-labels. Evaluations conducted on videos of open surgery demonstrate the good performance and real-world applicability of the proposed method, highlighting its potential for integration into medical augmented reality and robotic systems. The approach eliminates the need for extensive manual annotation of real surgical data.

Transformer-based chatbots can conduct fluent, natural-sounding conversations, but we have limited understanding of the mechanisms underlying their behavior. Prior work has taken a bottom-up approach to understanding Transformers by constructing Transformers for various synthetic and formal language tasks, such as regular expressions and Dyck languages. However, it is not obvious how to extend this approach to understand more naturalistic conversational agents. In this work, we take a step in this direction by constructing a Transformer that implements the ELIZA program, a classic, rule-based chatbot. ELIZA illustrates some of the distinctive challenges of the conversational setting, including both local pattern matching and long-term dialog state tracking. We build on constructions from prior work -- in particular, for simulating finite-state automata -- showing how simpler constructions can be composed and extended to give rise to more sophisticated behavior. Next, we train Transformers on a dataset of synthetically generated ELIZA conversations and investigate the mechanisms the models learn. Our analysis illustrates the kinds of mechanisms these models tend to prefer -- for example, models favor an induction head mechanism over a more precise, position based copying mechanism; and using intermediate generations to simulate recurrent data structures, like ELIZA's memory mechanisms. Overall, by drawing an explicit connection between neural chatbots and interpretable, symbolic mechanisms, our results offer a new setting for mechanistic analysis of conversational agents., Comment: Code and data are available at https://github.com/princeton-nlp/ELIZA-Transformer

It is a fundamental behavior that different individuals see the world in a largely similar manner. This is an essential basis for humans' ability to cooperate and communicate. However, what are the neuronal properties that underlie these inter-subject commonalities of our visual world? Finding out what aspects of neuronal coding remain invariant across individuals' brains will shed light not only on this fundamental question but will also point to the neuronal coding scheme as the basis of visual perception. Here, we address this question by obtaining intracranial recordings from three cohorts of patients taking part in a different visual recognition task (overall 19 patients and 244 high-order visual contacts included in the analyses) and examining the neuronal coding scheme most consistent across individuals' visual cortex. Our results highlight relational coding - expressed by the set of similarity distances between profiles of pattern activations - as the most consistent representation across individuals. Alternative coding schemes, such as population vector coding or linear coding, failed to achieve similar inter-subject consistency. Our results thus support relational coding as the central neuronal code underlying individuals' shared perceptual content in the human brain.

Stellar abundances of cerium are of high current interest based both on observations and theoretical models, especially with regard to the neutron--magic $^{140}$Ce isotope. A large discrepancy of $s-$process stellar models relative to cerium abundance observed in globular clusters was highlighted, pointing to possible uncertainties in experimental nuclear reaction rates. In this work, the stellar neutron capture cross section of the stable cerium isotopes $^{136}$Ce, $^{138}$Ce, $^{140}$Ce, and $^{142}$Ce, were re-measured. A $^{nat}$Ce sample was irradiated with quasi-Maxwellian neutrons at $kT = 34.2$ keV using the $^{7}$Li($p,n$) reaction. The neutron field with an intensity of $3-5 \times 10^{10}$ n/s was produced by irradiating the liquid-lithium target (LiLiT) with a mA proton beam at an energy (1.92 MeV) just above the threshold at Soreq Applied Research Accelerator Facility (SARAF). The activities of the $^{nat}$Ce neutron capture products were measured using a shielded High Purity Germanium detector. Cross sections were extracted relative to that of the $^{197}$Au(n,$\gamma$) reaction and the Maxwellian-averaged cross section (MACS) of the Ce isotopes were derived. The MACS values extracted from this experiment are generally consistent with previous measurements and show for $^{140}$Ce a value $\approx 15$\% smaller than most recent experimental values., Comment: 9 pages, 12 figures. This work is supported in part by the Pazy Foundation (Israel) and the German Israeli Foundation (GIF) under Grant No. I-1500-303.7/2019. M.P. and M.F. acknowledge support by the European Union (ChETEC-INFRA)

The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Recent work has automated the task of discovering circuits. Yet, these methods have practical limitations, as they rely either on inefficient search algorithms or inaccurate approximations. In this paper, we frame automated circuit discovery as an optimization problem and propose *Edge Pruning* as an effective and scalable solution. Edge Pruning leverages gradient-based pruning techniques, but instead of removing neurons or components, it prunes the \emph{edges} between components. Our method finds circuits in GPT-2 that use less than half the number of edges compared to circuits found by previous methods while being equally faithful to the full model predictions on standard circuit-finding tasks. Edge Pruning is efficient even with as many as 100K examples, outperforming previous methods in speed and producing substantially better circuits. It also perfectly recovers the ground-truth circuits in two models compiled with Tracr. Thanks to its efficiency, we scale Edge Pruning to CodeLlama-13B, a model over 100x the scale that prior methods operate on. We use this setting for a case study comparing the mechanisms behind instruction prompting and in-context learning. We find two circuits with more than 99.96% sparsity that match the performance of the full model and reveal that the mechanisms in the two settings overlap substantially. Our case study shows that Edge Pruning is a practical and scalable tool for interpretability and sheds light on behaviors that only emerge in large models., Comment: We release our code and data publicly at https://github.com/princeton-nlp/Edge-Pruning

In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. We provide both data-dependent and oracle upper bounds that apply regardless of model and algorithm choice, as well as bounds for Gibbs posteriors. We derive an algorithm inspired by our bounds and demonstrate empirically that our approach yields improved forward and backward transfer.

We study the space of non-degenerate traces on quantized Kleinian singularities of type A by studying their complement, the degenerate traces. In particular, we find the dimension of the space of twisted traces as a function of the corresponding automorphism and the quantization parameters, encoded in a polynomial $P$., Comment: 11 pages, preliminary version

A typical process of learning to play a piece on a piano consists of a progression through a series of practice units that focus on individual dimensions of the skill, the so-called practice modes. Practice modes in learning to play music comprise a particularly large set of possibilities, such as hand coordination, posture, articulation, ability to read a music score, correct timing or pitch, etc. Self-guided practice is known to be suboptimal, and a model that schedules optimal practice to maximize a learner's progress still does not exist. Because we each learn differently and there are many choices for possible piano practice tasks and methods, the set of practice modes should be dynamically adapted to the human learner, a process typically guided by a teacher. However, having a human teacher guide individual practice is not always feasible since it is time-consuming, expensive, and often unavailable. In this work, we present a modeling framework to guide the human learner through the learning process by choosing the practice modes generated by a policy model. To this end, we present a computational architecture building on a Gaussian process that incorporates 1) the learner state, 2) a policy that selects a suitable practice mode, 3) performance evaluation, and 4) expert knowledge. The proposed policy model is trained to approximate the expert-learner interaction during a practice session. In our future work, we will test different Bayesian optimization techniques, e.g., different acquisition functions, and evaluate their effect on the learning progress.

Our interest was to evaluate changes in fixation duration as a function of time-on-task (TOT) during a random saccade task. We employed a large, publicly available dataset. The frequency histogram of fixation durations was multimodal and modelled as a Gaussian mixture. We found five fixation types. The ``ideal'' response would be a single accurate saccade after each target movement, with a typical saccade latency of 200-250 msec, followed by a long fixation (> 800 msec) until the next target jump. We found fixations like this, but they comprised only 10% of all fixations and were the first fixation after target movement only 23.4% of the time. More frequently (57.4% of the time), the first fixation after target movement was short (117.7 msec mean) and was commonly followed by a corrective saccade. Across the entire 100 sec of the task, median total fixation duration decreased. This decrease was approximated with a power law fit with R^2=0.94. A detailed examination of the frequency of each of our five fixation types over time on task (TOT) revealed that the three shortest duration fixation types became more and more frequent with TOT whereas the two longest fixations became less and less frequent. In all cases, the changes over TOT followed power law relationships, with R^2 values between 0.73 and 0.93. We concluded that, over the 100 second duration of our task, long fixations are common in the first 15 to 22 seconds but become less common after that. Short fixations are relatively uncommon in the first 15 to 22 seconds but become more and more common as the task progressed. Apparently. the ability to produce an ideal response, although somewhat likely in the first 22 seconds, rapidly declines. This might be related to a noted decline in saccade accuracy over time., Comment: 23 pages, 19 figures

We have developed an in-situ index of refraction profile n(z) for cold polar ice, using the transit times of radio signals broadcast from an englacial transmitter to 2-5 km distant radio-frequency receivers, deployed at depths up to 200 m. For propagation through a non-uniform medium, Maxwell's equations generally admit two ray propagation solutions from a given transmitter, corresponding to a direct path (D) and a refracted or reflected path (R); the measured D vs. R timing differences (dt(D,R)) are determined by the refractive index profile. We constrain n(z) near South Pole, where the Askaryan Radio Array (ARA) neutrino observatory is located, by simulating D and R ray paths via ray tracing and comparing simulations to measured dt(D,R) values. Using previous ice density data as a proxy for n(z), we demonstrate that our data strongly favors a glaciologically-motivated three-phase densification model rather than a single exponential scale height model. Effective volume simulations for a detector of ARA station antenna depths yield a 14\% increase in neutrino sensitivity over a range of $10^{17} - 10^{21}$ eV using the three-phase model compared to a single exponential.

We present a method for classifying the expertise of a pathologist based on how they allocated their attention during a cancer reading. We engage this decoding task by developing a novel method for predicting the attention of pathologists as they read whole-slide Images (WSIs) of prostate and make cancer grade classifications. Our ground truth measure of a pathologists' attention is the x, y and z (magnification) movement of their viewport as they navigated through WSIs during readings, and to date we have the attention behavior of 43 pathologists reading 123 WSIs. These data revealed that specialists have higher agreement in both their attention and cancer grades compared to general pathologists and residents, suggesting that sufficient information may exist in their attention behavior to classify their expertise level. To attempt this, we trained a transformer-based model to predict the visual attention heatmaps of resident, general, and specialist (GU) pathologists during Gleason grading. Based solely on a pathologist's attention during a reading, our model was able to predict their level of expertise with 75.3%, 56.1%, and 77.2% accuracy, respectively, better than chance and baseline models. Our model therefore enables a pathologist's expertise level to be easily and objectively evaluated, important for pathology training and competency assessment. Tools developed from our model could also be used to help pathology trainees learn how to read WSIs like an expert.

Our national survey of the early intervention (EI) and early childhood special education (ECSE) workforce highlights key strengths and concerns. Workers find their roles rewarding, appreciate making a difference for children and families, and are well-qualified, with most holding at least a bachelor's degree. However, the workforce lacks diversity, which may not reflect the children served, and nearly all workers are female, while more boys receive EI and ECSE services. High-stress levels, large caseloads, inadequate support, and poor compensation are significant issues. Addressing these challenges through better training, support, and innovative solutions is essential to retain and attract qualified professionals, ensuring children and families receive the necessary services without delays. [Funding for this report was provided by Early Childhood Personnel Center at the University of Connecticut.]