Search

We introduce a framework that allows for the exact analytic treatment of quantum dynamics subject to coherent noise. The noise is modeled via unitary evolution under a Hamiltonian drawn from a random-matrix ensemble for arbitrary Hilbert-space dimension $N$. While the methods we develop apply to generic such ensembles with a notion of rotation invariance, we focus largely on the Gaussian unitary ensemble (GUE). Averaging over the ensemble of ''noisy'' Hamiltonians produces an effective quantum channel, the properties of which are analytically calculable and determined by the spectral form factors of the relevant ensemble. We compute spectral form factors of the GUE exactly for any finite $N$, along with the corresponding GUE quantum channel, and its variance. Key advantages of our approach include the ability to access exact analytic results for any $N$ and the ability to tune to the noise-free limit (in contrast, e.g., to the Haar ensemble), and analytic access to moments beyond the variance. We also highlight some unusual features of the GUE channel, including the nonmonotonicity of the coefficients of various operators as a function of noise strength and the failure to saturate the Haar-random limit, even with infinite noise strength., Comment: 18 pages, 5 figures

Light cone selection effects on cosmic observables must be precisely accounted for in the next generation of surveys, including the Dark Energy Spectroscopic Instrument (DESI) survey. This will allow us to correctly model the data and extract subtle shifts from general-relativistic effects. We examine the effects of peculiar velocities on color selection in spectroscopic galaxy surveys, with a focus on their implications for the galaxy clustering dipole $P_1(k)$. Using DESI Emission Line Galaxy (ELG) targets, we show that peculiar velocities can shift spectral emission features into or out of filter bands, modifying galaxy colors and thereby changing galaxy selection. This phenomenon mimics the effect of evolution bias, and we refer to it as the Doppler bias, $b_D$. The Doppler bias is of comparable size to the evolution bias at $0.8 < z < 1$, where it is largest. This enhances the ELG-LRG (Luminous Red Galaxy) cross-correlation dipole by 15-30%. This could be detectable at the $\sim$4$\sigma$ level for the full DESI survey. Additionally, we found that our $b_D$ estimate is impacted by the incompleteness of the parent ELG sample. Therefore, this work highlights the essential need for careful consideration of spectral-dependent biases caused by peculiar velocities during the selection phase of galaxy surveys, to enable unbiased analyses., Comment: 31 pages, 10 figures

In "Embers of Autoregression" (McCoy et al., 2023), we showed that several large language models (LLMs) have some important limitations that are attributable to their origins in next-word prediction. Here we investigate whether these issues persist with o1, a new system from OpenAI that differs from previous LLMs in that it is optimized for reasoning. We find that o1 substantially outperforms previous LLMs in many cases, with particularly large improvements on rare variants of common tasks (e.g., forming acronyms from the second letter of each word in a list, rather than the first letter). Despite these quantitative improvements, however, o1 still displays the same qualitative trends that we observed in previous systems. Specifically, o1 -- like previous LLMs -- is sensitive to the probability of examples and tasks, performing better and requiring fewer "thinking tokens" in high-probability settings than in low-probability ones. These results show that optimizing a language model for reasoning can mitigate but might not fully overcome the language model's probability sensitivity., Comment: 6 pages; updated to fix typo in Fig 4 caption

Two-dimensional (2D) 1T-VSe$_2$ has prompted significant interest due to the discrepancies regarding alleged ferromagnetism (FM) at room temperature, charge density wave (CDW) states and the interplay between the two. We employed a combined Diffusion Monte Carlo (DMC) and density functional theory (DFT) approach to accurately investigate the magnetic properties and response of strain of monolayer 1T-VSe$_2$. Our calculations show the delicate competition between various phases, revealing critical insights into the relationship between their energetic and structural properties. We went on to perform Classical Monte Carlo simulations informed by our DMC and DFT results, and found the magnetic transition temperature ($T_c$) of the undistorted (non-CDW) FM phase to be 228 K and the distorted (CDW) phase to be 68 K. Additionally, we studied the response of biaxial strain on the energetic stability and magnetic properties of various phases of 2D 1T-VSe$_2$ and found that small amounts of strain can enhance the $T_c$, suggesting a promising route for engineering and enhancing magnetic behavior. Finally, we synthesized 1T-VSe$_2$ and performed Raman spectroscopy measurements, which were in close agreement with our calculated results. Our work emphasizes the role of highly accurate DMC methods in advancing the understanding of monolayer 1T-VSe$_2$ and provides a robust framework for future studies of 2D magnetic materials.

Understanding and modeling collective intelligence is essential for addressing complex social systems. Directed graphs called fuzzy cognitive maps (FCMs) offer a powerful tool for encoding causal mental models, but extracting high-integrity FCMs from text is challenging. This study presents an approach using large language models (LLMs) to automate FCM extraction. We introduce novel graph-based similarity measures and evaluate them by correlating their outputs with human judgments through the Elo rating system. Results show positive correlations with human evaluations, but even the best-performing measure exhibits limitations in capturing FCM nuances. Fine-tuning LLMs improves performance, but existing measures still fall short. This study highlights the need for soft similarity measures tailored to FCM extraction, advancing collective intelligence modeling with NLP., Comment: Camera-ready version accepted for publication in the EMNLP 2024 Workshop NLP4Science

Stochastic resetting, a method for accelerating target search in random processes, often incurs temporal and energetic costs. For a diffusing particle, a lower bound exists for the energetic cost of reaching the target, which is attained at low resetting rates and equals the direct linear transportation cost against fluid drag. Here, we study ``smart resetting," a strategy that aims to beat this lower bound. By strategically resetting the particle only when this benefits its progress toward the target, smart resetting leverages information to minimize energy consumption. We analytically calculate the energetic cost per mean first passage time and show that smart resetting consistently reduces the energetic cost compared to regular resetting. Surprisingly, smart resting achieves the minimum energy cost previously established for regular resetting, irrespective of the resetting rate. Yet, it fails to reduce this cost further. We extend our findings in two ways: first, by examining nonlinear energetic cost functions, and second, by considering smart resetting of drift-diffusion processes.

We propose a new method for generating realistic datasets with distribution shifts using any decoder-based generative model. Our approach systematically creates datasets with varying intensities of distribution shifts, facilitating a comprehensive analysis of model performance degradation. We then use these generated datasets to evaluate the performance of various commonly used networks and observe a consistent decline in performance with increasing shift intensity, even when the effect is almost perceptually unnoticeable to the human eye. We see this degradation even when using data augmentations. We also find that enlarging the training dataset beyond a certain point has no effect on the robustness and that stronger inductive biases increase robustness., Comment: ECCV2024, "Synthetic Data for Computer Vision" workshop

We introduce a benchmark to directly evaluate the alignment between human observers and vision models on a 3D shape inference task. We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape: given a set of images, participants identify which contain the same/different objects, despite considerable viewpoint variation. We draw from a diverse range of images that include common objects (e.g., chairs) as well as abstract shapes (i.e., procedurally generated `nonsense' objects). After constructing over 2000 unique image sets, we administer these tasks to human participants, collecting 35K trials of behavioral data from over 500 participants. This includes explicit choice behaviors as well as intermediate measures, such as reaction time and gaze data. We then evaluate the performance of common vision models (e.g., DINOv2, MAE, CLIP). We find that humans outperform all models by a wide margin. Using a multi-scale evaluation approach, we identify underlying similarities and differences between models and humans: while human-model performance is correlated, humans allocate more time/processing on challenging trials. All images, data, and code can be accessed via our project page., Comment: Project page: https://tzler.github.io/MOCHI/ Code: https://github.com/tzler/mochi_code Huggingface dataset: https://huggingface.co/datasets/tzler/MOCHI

We study projection determinantal point processes and their connection to the squared Grassmannian. We prove that the log-likelihood function of this statistical model has $(n - 1)!/2$ critical points, all of which are real and positive, thereby settling a conjecture of Devriendt, Friedman, Reinke, and Sturmfels.

The emergence of cognition requires a framework that bridges evolutionary principles with neurocomputational mechanisms. This paper introduces the "thoughtseed" framework, proposing that cognition arises from the dynamic interaction of self-organizing units of embodied knowledge called "thoughtseeds." We leverage evolutionary theory, "neuronal packets," and the "Inner Screen" hypothesis within Free Energy Principle, and propose a four-level hierarchical model of the cognitive agent's internal states: Neuronal Packet Domains (NPDs), Knowledge Domains (KDs), thoughtseeds network, and meta-cognition. The dynamic interplay within this hierarchy, mediated by nested Markov blankets and reciprocal message passing, facilitates the emergence of thoughtseeds as coherent patterns of activity that guide perception, action, and learning. The framework further explores the role of the organism's Umwelt and the principles of active inference, especially the generative model at each nested level, in shaping the selection and activation of thoughtseeds, leading to adaptive behavior through surprise minimization. The "Inner Screen" is posited as the locus of conscious experience, where the content of the dominant thoughtseed is projected, maintaining a unitary conscious experience. Active thoughtseeds are proposed as the fundamental units of thought that contribute to the "content of consciousness." We present a mathematical framework grounded in active inference and dynamical systems theory. The thoughtseed framework represents an initial but promising step towards a novel, biologically-grounded model for understanding the organizing principles and emergence of embodied cognition, offering a unified account of cognitive phenomena, from basic physiological regulation to higher-order thought processes, and potentially bridge neuroscience and contemplative traditions., Comment: I will be referencing the pre-print for my poster/Spotlight Presentation at IWAI,https://iwaiworkshop.github.io/ on September -10, 2024 at Oxford, England. It has embedded 5 Figures and 2 Tables. it has mathematical equations 35+ mathematical equations, with explanations in Supllementary Section

Relative free energy calculations are now widely used in academia and industry, but the accuracy is often limited by poor sampling of the complexes conformational ensemble. To address this, we have developed a novel method termed Multi-Topology Replica Exchange of Expanded Ensembles (MT-REXEE). This method enables parallel expanded ensemble calculations, facilitating iterative relative free energy computations while allowing conformational exchange between parallel transformations. These iterative transformations are adaptable to any set of systems with a common backbone or central substructure. We demonstrate that the MT-REXEE method maintains thermodynamic cycle closure to the same extent as standard expanded ensemble for both solvation free energy and relative binding free energy. The transformations tested involve simple systems that incorporate diverse heavy atoms and multi-site perturbations of a small molecule core resembling multi-site $\lambda$ dynamics, without necessitating modifications to the MD code, which in our initial implementation is GROMACS. We outline a systematic approach for topology set-up and provide instructions on how to perform inter-replicate coordinate modifications. This work shows that MT-REEXE can be used to perform accurate and reproducible free energy estimates and prompts expansion to more complex test systems and other molecular dynamics simulation infrastructures.

This research presents a comprehensive framework for analyzing liquidity in financial markets, particularly in the context of high-frequency trading. By leveraging advanced machine learning classification techniques, including Logistic Regression, Support Vector Machine, and Random Forest, the study aims to predict minute-level price movements using an extensive set of liquidity metrics derived from the Trade and Quote (TAQ) data. The findings reveal that employing a broad spectrum of liquidity measures yields higher predictive accuracy compared to models utilizing a reduced subset of features. Key liquidity metrics, such as Liquidity Ratio, Flow Ratio, and Turnover, consistently emerged as significant predictors across all models, with the Random Forest algorithm demonstrating superior accuracy. This study not only underscores the critical role of liquidity in market stability and transaction costs but also highlights the complexities involved in short-interval market predictions. The research suggests that a comprehensive set of liquidity measures is essential for accurate prediction, and proposes future work to validate these findings across different stock datasets to assess their generalizability.

The Active Inference framework models perception and action as a unified process, where agents use probabilistic models to predict and actively minimize sensory discrepancies. In complement and contrast, traditional population-based metaheuristics rely on reactive environmental interactions without anticipatory adaptation. This paper proposes the integration of Active Inference into these metaheuristics to enhance performance through anticipatory environmental adaptation. We demonstrate this approach specifically with Ant Colony Optimization (ACO) on the Travelling Salesman Problem (TSP). Experimental results indicate that Active Inference can yield some improved solutions with only a marginal increase in computational cost, with interesting patterns of performance that relate to number and topology of nodes in the graph. Further work will characterize where and when different types of Active Inference augmentation of population metaheuristics may be efficacious.

We report a twisted triple bilayer graphene platform consisting of three units of Bernal bilayer graphene (BLG) consecutively twisted at 1.49{\deg} and 1.68{\deg}. We observe inter-moir\'e Hofstadter butterflies from two co-existing moir\'e superlattices and a Hofstadter butterfly from reconstructed moir\'e-of-moir\'e lattice, and show that their Brown-Zak (BZ) oscillations quantitatively agree with each other, both evidencing strong atomic reconstruction with a lattice constant of 18.1 nm. We further demonstrate such atomic reconstruction strongly enhances the Berry curvature of each moir\'e and moir\'e-of-moir\'e band-insulator state, characterized by measured strong non-local valley Hall effect (VHE) that sensitively depends on the inter-moir\'e competition strength, tunable by manipulating the out-of-the-plane carrier distribution which controls the magnitude of the valley currents. Our study sheds new light on the microscopic mechanism of atomic and electronic reconstruction in twisted-multilayer systems, by investigating novel emergent quantum phenomena of reconstructed quasi-crystalline moir\'e-of-moir\'e superlattice, including a new type of moir\'e-of-moir\'e band-insulator states and atomically enhanced moir\'e Berry curvature. We show that the reconstructed electronic band can be versatilely tuned by electrostatics, providing an approach towards engineering the band structure and its topology for a novel quantum material platform with designer electrical and optical properties.

An I-surface $X$ is a surface of general type with $K_X^2 =1$ and $p_g(X) =2$. This paper studies the asymptotic behavior of the period map for I-surfaces acquiring simple elliptic singularities. First we describe the relationship between the deformation theory of such surfaces and their $d$-semistable models. Next we analyze the mixed Hodge structures on the $d$-semistable models, the corresponding limiting mixed Hodge structures, and the monodromy. There are $6$ possible boundary strata for which the relevant limiting mixed Hodge structures satisfy: $\dim W_1 = 4$, and hence $W_2/W_1$ is of pure type $(1,1)$. We show that, in each case, the nilpotent orbit of limiting mixed Hodge structures determines the boundary stratum and prove a global Torelli theorem for one such stratum., Comment: 40 pages. v.2: typos corrected and other minor revisions

With the growing prevalence of large language models, it is increasingly common to annotate datasets for machine learning using pools of crowd raters. However, these raters often work in isolation as individual crowdworkers. In this work, we regard annotation not merely as inexpensive, scalable labor, but rather as a nuanced interpretative effort to discern the meaning of what is being said in a text. We describe a novel, collaborative, and iterative annotator-in-the-loop methodology for annotation, resulting in a 'Bridging Benchmark Dataset' of comments relevant to bridging divides, annotated from 11,973 textual posts in the Civil Comments dataset. The methodology differs from popular anonymous crowd-rating annotation processes due to its use of an in-depth, iterative engagement with seven US-based raters to (1) collaboratively refine the definitions of the to-be-annotated concepts and then (2) iteratively annotate complex social concepts, with check-in meetings and discussions. This approach addresses some shortcomings of current anonymous crowd-based annotation work, and we present empirical evidence of the performance of our annotation process in the form of inter-rater reliability. Our findings indicate that collaborative engagement with annotators can enhance annotation methods, as opposed to relying solely on isolated work conducted remotely. We provide an overview of the input texts, attributes, and annotation process, along with the empirical results and the resulting benchmark dataset, categorized according to the following attributes: Alienation, Compassion, Reasoning, Curiosity, Moral Outrage, and Respect., Comment: Accepted Paper at AAAI/ACM Artificial Intelligence, Ethics, and Society (AIES)

The purpose of this study was to evaluate the implementation and impact of the Pack program. The Pack was developed by the New York Hall of Science (NYSCI) and includes a digital game and set of curricular and professional development resources that aim to support computational thinking teaching and learning in middle school science and computer science classrooms. The evaluation of the Pack was funded with an Education Innovation and Research grant from the U.S. Department of Education and was carried out by the American Institutes for Research (AIR) during the 2022-23 school year in New York City middle schools. AIR examined the Pack intervention's classroom implementation by teachers and their Grade 6-8 students as well as outcomes related to students' computational thinking skills and their attitudes toward problem solving. Overall, the study yielded promising results about the feasibility and utility of implementing the Pack in middle school science and computer science classrooms. Based on teacher survey and interview responses, teachers reported that NYSCI's professional development was helpful to their teaching and reported that the Pack digital game and lessons were valuable for computational thinking teaching and learning. Using teacher survey and extant data, AIR found that teachers were able to implement the Pack program components with fidelity. To measure the relationship between students' use of the Pack and outcomes related to students' attitudes toward problem solving and computational thinking skills, AIR administered a pre- and posttest survey of student problem solving attitudes and a computational thinking knowledge assessment. AIR found that there were no significant associations between students' use of the Pack and their beliefs about their ability to apply their problem-solving skills to new situations or their beliefs about problem-solving ability as a fixed trait. Similarly, there were no significant associations between teachers' or students' reported use of the Pack and students' scores on a computational thinking assessment. There were also no significant associations between students' reported participation in computational thinking while using the Pack and their scores on a computational thinking assessment.

Clock transitions (CTs) in spin systems, which occur at avoided level crossings, enhance quantum coherence lifetimes T$_2$ because the transition becomes immune to the decohering effects of magnetic field fluctuations to first order. We present the first electron-spin resonance (ESR) characterization of CTs in certain defect-rich silica glasses, noting coherence times up to 16 $\mu$s at the CTs. We find CT behavior at zero magnetic field in borosilicate and aluminosilicate glasses, but not in a variety of silica glasses lacking boron or aluminum. Annealing reduces or eliminates the zero-field signal. Since boron and aluminum have the same valence and are acceptors when substituted for silicon, we suggest the observed CT behavior could be generated by a spin-1 boron vacancy center within the borosilicate glass, and similarly, an aluminum-vacancy center in the aluminosilicate glass., Comment: The Main Manuscript features 9 pages, 5 figures, and 1 table. The Supplement features an additional 6 pages, 4 figures, and two tables

CXL (Compute Express Link) is an emerging open industry-standard interconnect between processing and memory devices that is expected to revolutionize the way systems are designed in the near future. It enables cache-coherent shared memory pools in a disaggregated fashion at unprecedented scales, allowing algorithms to interact with a variety of storage devices using simple loads and stores in a cacheline granularity. Alongside with unleashing unique opportunities for a wide range of applications, CXL introduces new challenges of data management and crash consistency. Alas, CXL lacks an adequate programming model, which makes reasoning about the correctness and expected behaviors of algorithms and systems on top of it nearly impossible. In this work, we present CXL0, the first programming model for concurrent programs running on top of CXL. We propose a high-level abstraction for CXL memory accesses and formally define operational semantics on top of that abstraction. We provide a set of general transformations that adapt concurrent algorithms to the new disruptive technology. Using these transformations, every linearizable algorithm can be easily transformed into its provably correct version in the face of a full-system or sub-system crash. We believe that this work will serve as the stepping stone for systems design and modelling on top of CXL, and support the development of future models as software and hardware evolve.

In recent years, Large Language Models (LLM) have demonstrated impressive capabilities in the field of natural language processing (NLP). This paper explores the application of LLMs in negotiation transcript analysis by the Vanderbilt AI Negotiation Lab. Starting in September 2022, we applied multiple strategies using LLMs from zero shot learning to fine tuning models to in-context learning). The final strategy we developed is explained, along with how to access and use the model. This study provides a sense of both the opportunities and roadblocks for the implementation of LLMs in real life applications and offers a model for how LLMs can be applied to coding in other fields.

This work presents a novel approach to monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, symmetries, occlusions, and lack of annotated real-world data. The method leverages synthetic data generation and domain adaptation techniques to overcome these obstacles. The proposed approach consists of three main components: (1) synthetic data generation using 3D modeling of surgical tools with articulation rigging and physically-based rendering; (2) a tailored pose estimation framework combining object detection with pose estimation and a hybrid geometric fusion strategy; and (3) a training strategy that utilizes both synthetic and real unannotated data, employing domain adaptation on real video data using automatically generated pseudo-labels. Evaluations conducted on videos of open surgery demonstrate the good performance and real-world applicability of the proposed method, highlighting its potential for integration into medical augmented reality and robotic systems. The approach eliminates the need for extensive manual annotation of real surgical data.

Transformer-based chatbots can conduct fluent, natural-sounding conversations, but we have limited understanding of the mechanisms underlying their behavior. Prior work has taken a bottom-up approach to understanding Transformers by constructing Transformers for various synthetic and formal language tasks, such as regular expressions and Dyck languages. However, it is not obvious how to extend this approach to understand more naturalistic conversational agents. In this work, we take a step in this direction by constructing a Transformer that implements the ELIZA program, a classic, rule-based chatbot. ELIZA illustrates some of the distinctive challenges of the conversational setting, including both local pattern matching and long-term dialog state tracking. We build on constructions from prior work -- in particular, for simulating finite-state automata -- showing how simpler constructions can be composed and extended to give rise to more sophisticated behavior. Next, we train Transformers on a dataset of synthetically generated ELIZA conversations and investigate the mechanisms the models learn. Our analysis illustrates the kinds of mechanisms these models tend to prefer -- for example, models favor an induction head mechanism over a more precise, position based copying mechanism; and using intermediate generations to simulate recurrent data structures, like ELIZA's memory mechanisms. Overall, by drawing an explicit connection between neural chatbots and interpretable, symbolic mechanisms, our results offer a new setting for mechanistic analysis of conversational agents., Comment: Code and data are available at https://github.com/princeton-nlp/ELIZA-Transformer

It is a fundamental behavior that different individuals see the world in a largely similar manner. This is an essential basis for humans' ability to cooperate and communicate. However, what are the neuronal properties that underlie these inter-subject commonalities of our visual world? Finding out what aspects of neuronal coding remain invariant across individuals' brains will shed light not only on this fundamental question but will also point to the neuronal coding scheme as the basis of visual perception. Here, we address this question by obtaining intracranial recordings from three cohorts of patients taking part in a different visual recognition task (overall 19 patients and 244 high-order visual contacts included in the analyses) and examining the neuronal coding scheme most consistent across individuals' visual cortex. Our results highlight relational coding - expressed by the set of similarity distances between profiles of pattern activations - as the most consistent representation across individuals. Alternative coding schemes, such as population vector coding or linear coding, failed to achieve similar inter-subject consistency. Our results thus support relational coding as the central neuronal code underlying individuals' shared perceptual content in the human brain.

Stellar abundances of cerium are of high current interest based both on observations and theoretical models, especially with regard to the neutron--magic $^{140}$Ce isotope. A large discrepancy of $s-$process stellar models relative to cerium abundance observed in globular clusters was highlighted, pointing to possible uncertainties in experimental nuclear reaction rates. In this work, the stellar neutron capture cross section of the stable cerium isotopes $^{136}$Ce, $^{138}$Ce, $^{140}$Ce, and $^{142}$Ce, were re-measured. A $^{nat}$Ce sample was irradiated with quasi-Maxwellian neutrons at $kT = 34.2$ keV using the $^{7}$Li($p,n$) reaction. The neutron field with an intensity of $3-5 \times 10^{10}$ n/s was produced by irradiating the liquid-lithium target (LiLiT) with a mA proton beam at an energy (1.92 MeV) just above the threshold at Soreq Applied Research Accelerator Facility (SARAF). The activities of the $^{nat}$Ce neutron capture products were measured using a shielded High Purity Germanium detector. Cross sections were extracted relative to that of the $^{197}$Au(n,$\gamma$) reaction and the Maxwellian-averaged cross section (MACS) of the Ce isotopes were derived. The MACS values extracted from this experiment are generally consistent with previous measurements and show for $^{140}$Ce a value $\approx 15$\% smaller than most recent experimental values., Comment: 9 pages, 12 figures. This work is supported in part by the Pazy Foundation (Israel) and the German Israeli Foundation (GIF) under Grant No. I-1500-303.7/2019. M.P. and M.F. acknowledge support by the European Union (ChETEC-INFRA)

The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Recent work has automated the task of discovering circuits. Yet, these methods have practical limitations, as they rely either on inefficient search algorithms or inaccurate approximations. In this paper, we frame automated circuit discovery as an optimization problem and propose *Edge Pruning* as an effective and scalable solution. Edge Pruning leverages gradient-based pruning techniques, but instead of removing neurons or components, it prunes the \emph{edges} between components. Our method finds circuits in GPT-2 that use less than half the number of edges compared to circuits found by previous methods while being equally faithful to the full model predictions on standard circuit-finding tasks. Edge Pruning is efficient even with as many as 100K examples, outperforming previous methods in speed and producing substantially better circuits. It also perfectly recovers the ground-truth circuits in two models compiled with Tracr. Thanks to its efficiency, we scale Edge Pruning to CodeLlama-13B, a model over 100x the scale that prior methods operate on. We use this setting for a case study comparing the mechanisms behind instruction prompting and in-context learning. We find two circuits with more than 99.96% sparsity that match the performance of the full model and reveal that the mechanisms in the two settings overlap substantially. Our case study shows that Edge Pruning is a practical and scalable tool for interpretability and sheds light on behaviors that only emerge in large models., Comment: We release our code and data publicly at https://github.com/princeton-nlp/Edge-Pruning

In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. We provide both data-dependent and oracle upper bounds that apply regardless of model and algorithm choice, as well as bounds for Gibbs posteriors. We derive an algorithm inspired by our bounds and demonstrate empirically that our approach yields improved forward and backward transfer.

We study the space of non-degenerate traces on quantized Kleinian singularities of type A by studying their complement, the degenerate traces. In particular, we find the dimension of the space of twisted traces as a function of the corresponding automorphism and the quantization parameters, encoded in a polynomial $P$., Comment: 11 pages, preliminary version

A typical process of learning to play a piece on a piano consists of a progression through a series of practice units that focus on individual dimensions of the skill, the so-called practice modes. Practice modes in learning to play music comprise a particularly large set of possibilities, such as hand coordination, posture, articulation, ability to read a music score, correct timing or pitch, etc. Self-guided practice is known to be suboptimal, and a model that schedules optimal practice to maximize a learner's progress still does not exist. Because we each learn differently and there are many choices for possible piano practice tasks and methods, the set of practice modes should be dynamically adapted to the human learner, a process typically guided by a teacher. However, having a human teacher guide individual practice is not always feasible since it is time-consuming, expensive, and often unavailable. In this work, we present a modeling framework to guide the human learner through the learning process by choosing the practice modes generated by a policy model. To this end, we present a computational architecture building on a Gaussian process that incorporates 1) the learner state, 2) a policy that selects a suitable practice mode, 3) performance evaluation, and 4) expert knowledge. The proposed policy model is trained to approximate the expert-learner interaction during a practice session. In our future work, we will test different Bayesian optimization techniques, e.g., different acquisition functions, and evaluate their effect on the learning progress.

Our interest was to evaluate changes in fixation duration as a function of time-on-task (TOT) during a random saccade task. We employed a large, publicly available dataset. The frequency histogram of fixation durations was multimodal and modelled as a Gaussian mixture. We found five fixation types. The ``ideal'' response would be a single accurate saccade after each target movement, with a typical saccade latency of 200-250 msec, followed by a long fixation (> 800 msec) until the next target jump. We found fixations like this, but they comprised only 10% of all fixations and were the first fixation after target movement only 23.4% of the time. More frequently (57.4% of the time), the first fixation after target movement was short (117.7 msec mean) and was commonly followed by a corrective saccade. Across the entire 100 sec of the task, median total fixation duration decreased. This decrease was approximated with a power law fit with R^2=0.94. A detailed examination of the frequency of each of our five fixation types over time on task (TOT) revealed that the three shortest duration fixation types became more and more frequent with TOT whereas the two longest fixations became less and less frequent. In all cases, the changes over TOT followed power law relationships, with R^2 values between 0.73 and 0.93. We concluded that, over the 100 second duration of our task, long fixations are common in the first 15 to 22 seconds but become less common after that. Short fixations are relatively uncommon in the first 15 to 22 seconds but become more and more common as the task progressed. Apparently. the ability to produce an ideal response, although somewhat likely in the first 22 seconds, rapidly declines. This might be related to a noted decline in saccade accuracy over time., Comment: 23 pages, 19 figures

We develop an in-situ index of refraction profile using the transit time of radio signals broadcast from an englacial transmitter to 2-5 km distant radio-frequency receivers, deployed at depths up to 200 m. Maxwell's equations generally admit two ray propagation solutions from a given transmitter, corresponding to a direct path (D) and a refracted path (R); the measured D vs. R (dt(D,R)) timing differences provide constraints on the index of refraction profile near South Pole, where the Askaryan Radio Array (ARA) neutrino observatory is located. We constrain the refractive index profile by simulating D and R ray paths via ray tracing and comparing those to measured dt(D,R) signals. Using previous ice density data as a proxy for n(z), we demonstrate that our data strongly favors a glaciologically-motivated three-phase densification model rather than a single exponential scale height model. Simulations show that the single exponential model overestimates ARA neutrino sensitivity compared to the three-phase model.

Domain wall (DW) devices have garnered recent interest for diverse applications including memory, logic, and neuromorphic primitives; fast, accurate device models are therefore imperative for large-scale system design and verification. Extant DW motion models are sub-optimal for large-scale system design either over-consuming compute resources with physics-heavy equations or oversimplifying the physics, drastically reducing model accuracy. We propose a DW model inspired by the phenomenological similarities between motions of a DW and a classical object being acted on by forces like air resistance or static friction. Our proposed phenomenological model predicts DW motion within 1.2% on average compared with micromagnetic simulations that are 400 times slower. Additionally our model is seven times faster than extant collective coordinate models and 14 times more accurate than extant hyper-reduced models making it an essential tool for large-scale DW circuit design and simulation. The model is publicly posted along with scripts that automatically extract model parameters from user-provided simulation or experimental data to extend the model to alternative micromagnetic parameters.

Peer review is a popular feedback mechanism in higher education that actively engages students and provides researchers with a means to assess student engagement. However, there is little empirical support for the durability of peer review, particularly when using data predictive modeling to analyze student comments. This study uses Na\"ive Bayes modeling to analyze peer review data obtained from an undergraduate visual literacy course over five years. We expand on the research of Friedman and Rosen and Beasley et al. by focusing on the Na\"ive Bayes model of students' remarks. Our findings highlight the utility of Na\"ive Bayes modeling, particularly in the analysis of student comments based on parts of speech, where nouns emerged as the prominent category. Additionally, when examining students' comments using the visual peer review rubric, the lie factor emerged as the predominant factor. Comparing Na\"ive Bayes model to Beasley's approach, we found both help instructors map directions taken in the class, but the Na\"ive Bayes model provides a more specific outline for forecasting with a more detailed framework for identifying core topics within the course, enhancing the forecasting of educational directions. Through the application of the Holdout Method and $\mathrm{k}$-fold cross-validation with continuity correction, we have validated the model's predictive accuracy, underscoring its effectiveness in offering deep insights into peer review mechanisms. Our study findings suggest that using predictive modeling to assess student comments can provide a new way to better serve the students' classroom comments on their visual peer work. This can benefit courses by inspiring changes to course content, reinforcement of course content, modification of projects, or modifications to the rubric itself., Comment: 8 pages, 6 figures, IEEE EDUCON 2024 conference

This paper explores the possibilities of the current generation of Large Language Models for incorporating Machine Learning Operations (MLOps) functionalities into ML training code bases. We evaluate the performance of OpenAI (gpt-3.5-turbo) and WizardCoder (open-source, 15B parameters) models on the automated accomplishment of various MLOps functionalities in different settings. We perform a benchmarking study that assesses the ability of these models to: (1) adapt existing code samples (Inlining) with component-specific MLOps functionality such as MLflow and Weights & Biases for experiment tracking, Optuna for hyperparameter optimization etc., and (2) perform the task of Translation from one component of an MLOps functionality to another, e.g., translating existing GitPython library based version control code to Data Version Control library based. We also propose three different approaches that involve teaching LLMs to comprehend the API documentation of the components as a reference while accomplishing the Translation tasks. In our evaluations, the gpt-3.5-turbo model significantly outperforms WizardCoder by achieving impressive Pass@3 accuracy in model optimization (55% compared to 0% by WizardCoder), experiment tracking (100%, compared to 62.5% by WizardCoder), model registration (92% compared to 42% by WizardCoder) and hyperparameter optimization (83% compared to 58% by WizardCoder) on average, in their best possible settings, showcasing its superior code adaptability performance in complex MLOps tasks., Comment: The work was completed during 2Q, 3Q of Year 2023, when WizardCoder was the top performing Open source LLM for coding. Newer and better models have emerged since then. The processes and methodologies utilized for this benchmarking can still be utilized for evaluating the current SoTA models

Advances in underwater imaging enable the collection of extensive seafloor image datasets that are necessary for monitoring important benthic ecosystems. The ability to collect seafloor imagery has outpaced our capacity to analyze it, hindering expedient mobilization of this crucial environmental information. Recent machine learning approaches provide opportunities to increase the efficiency with which seafloor image datasets are analyzed, yet large and consistent datasets necessary to support development of such approaches are scarce. Here we present BenthicNet: a global compilation of seafloor imagery designed to support the training and evaluation of large-scale image recognition models. An initial set of over 11.4 million images was collected and curated to represent a diversity of seafloor environments using a representative subset of 1.3 million images. These are accompanied by 2.6 million annotations translated to the CATAMI scheme, which span 190,000 of the images. A large deep learning model was trained on this compilation and preliminary results suggest it has utility for automating large and small-scale image analysis tasks. The compilation and model are made openly available for use by the scientific community at https://doi.org/10.20383/103.0614.

In recent years, the challenge of extracting information from business documents has emerged as a critical task, finding applications across numerous domains. This effort has attracted substantial interest from both industry and academy, highlighting its significance in the current technological landscape. Most datasets in this area are primarily focused on Key Information Extraction (KIE), where the extraction process revolves around extracting information using a specific, predefined set of keys. Unlike most existing datasets and benchmarks, our focus is on discovering key-value pairs (KVPs) without relying on predefined keys, navigating through an array of diverse templates and complex layouts. This task presents unique challenges, primarily due to the absence of comprehensive datasets and benchmarks tailored for non-predetermined KVP extraction. To address this gap, we introduce KVP10k , a new dataset and benchmark specifically designed for KVP extraction. The dataset contains 10707 richly annotated images. In our benchmark, we also introduce a new challenging task that combines elements of KIE as well as KVP in a single task. KVP10k sets itself apart with its extensive diversity in data and richly detailed annotations, paving the way for advancements in the field of information extraction from complex business documents., Comment: accepted ICDAR2024

We present a method for classifying the expertise of a pathologist based on how they allocated their attention during a cancer reading. We engage this decoding task by developing a novel method for predicting the attention of pathologists as they read whole-slide Images (WSIs) of prostate and make cancer grade classifications. Our ground truth measure of a pathologists' attention is the x, y and z (magnification) movement of their viewport as they navigated through WSIs during readings, and to date we have the attention behavior of 43 pathologists reading 123 WSIs. These data revealed that specialists have higher agreement in both their attention and cancer grades compared to general pathologists and residents, suggesting that sufficient information may exist in their attention behavior to classify their expertise level. To attempt this, we trained a transformer-based model to predict the visual attention heatmaps of resident, general, and specialist (GU) pathologists during Gleason grading. Based solely on a pathologist's attention during a reading, our model was able to predict their level of expertise with 75.3%, 56.1%, and 77.2% accuracy, respectively, better than chance and baseline models. Our model therefore enables a pathologist's expertise level to be easily and objectively evaluated, important for pathology training and competency assessment. Tools developed from our model could also be used to help pathology trainees learn how to read WSIs like an expert.

Our national survey of the early intervention (EI) and early childhood special education (ECSE) workforce highlights key strengths and concerns. Workers find their roles rewarding, appreciate making a difference for children and families, and are well-qualified, with most holding at least a bachelor's degree. However, the workforce lacks diversity, which may not reflect the children served, and nearly all workers are female, while more boys receive EI and ECSE services. High-stress levels, large caseloads, inadequate support, and poor compensation are significant issues. Addressing these challenges through better training, support, and innovative solutions is essential to retain and attract qualified professionals, ensuring children and families receive the necessary services without delays. [Funding for this report was provided by Early Childhood Personnel Center at the University of Connecticut.]