Search

Transformers have demonstrated remarkable in-context learning capabilities across various domains, including statistical learning tasks. While previous work has shown that transformers can implement common learning algorithms, the adversarial robustness of these learned algorithms remains unexplored. This work investigates the vulnerability of in-context learning in transformers to \textit{hijacking attacks} focusing on the setting of linear regression tasks. Hijacking attacks are prompt-manipulation attacks in which the adversary's goal is to manipulate the prompt to force the transformer to generate a specific output. We first prove that single-layer linear transformers, known to implement gradient descent in-context, are non-robust and can be manipulated to output arbitrary predictions by perturbing a single example in the in-context training set. While our experiments show these attacks succeed on linear transformers, we find they do not transfer to more complex transformers with GPT-2 architectures. Nonetheless, we show that these transformers can be hijacked using gradient-based adversarial attacks. We then demonstrate that adversarial training enhances transformers' robustness against hijacking attacks, even when just applied during finetuning. Additionally, we find that in some settings, adversarial training against a weaker attack model can lead to robustness to a stronger attack model. Lastly, we investigate the transferability of hijacking attacks across transformers of varying scales and initialization seeds, as well as between transformers and ordinary least squares (OLS). We find that while attacks transfer effectively between small-scale transformers, they show poor transferability in other scenarios (small-to-large scale, large-to-large scale, and between transformers and OLS).

We classify actions of generalized Taft algebras on preprojective algebras of extended Dynkin quivers of type $A$. This may be viewed as an extension of the problem of classifying actions on the polynomial ring in two variables. In cases where the grouplike element acts via rotation on the underlying quiver, we compute invariants of the Taft action and, in certain cases, show that the invariant ring is isomorphic to the center of the preprojective algebra.

The effect of regularizers such as weight decay when training deep neural networks is not well understood. We study the influence of weight decay as well as $L2$-regularization when training neural network models in which parameter matrices interact multiplicatively. This combination is of particular interest as this parametrization is common in attention layers, the workhorse of transformers. Here, key-query, as well as value-projection parameter matrices, are multiplied directly with each other: $W_K^TW_Q$ and $PW_V$. We extend previous results and show on one hand that any local minimum of a $L2$-regularized loss of the form $L(AB^\top) + \lambda (\|A\|^2 + \|B\|^2)$ coincides with a minimum of the nuclear norm-regularized loss $L(AB^\top) + \lambda\|AB^\top\|_*$, and on the other hand that the 2 losses become identical exponentially quickly during training. We thus complement existing works linking $L2$-regularization with low-rank regularization, and in particular, explain why such regularization on the matrix product affects early stages of training. Based on these theoretical insights, we verify empirically that the key-query and value-projection matrix products $W_K^TW_Q, PW_V$ within attention layers, when optimized with weight decay, as usually done in vision tasks and language modelling, indeed induce a significant reduction in the rank of $W_K^TW_Q$ and $PW_V$, even in fully online training. We find that, in accordance with existing work, inducing low rank in attention matrix products can damage language model performance, and observe advantages when decoupling weight decay in attention layers from the rest of the parameters.

Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. How can we achieve cooperation among self-interested, independent learning agents? Promising recent work has shown that in certain tasks cooperation can be established between learning-aware agents who model the learning dynamics of each other. Here, we present the first unbiased, higher-derivative-free policy gradient algorithm for learning-aware reinforcement learning, which takes into account that other agents are themselves learning through trial and error based on multiple noisy trials. We then leverage efficient sequence models to condition behavior on long observation histories that contain traces of the learning dynamics of other agents. Training long-context policies with our algorithm leads to cooperative behavior and high returns on standard social dilemmas, including a challenging environment where temporally-extended action coordination is required. Finally, we derive from the iterated prisoner's dilemma a novel explanation for how and when cooperation arises among self-interested learning-aware agents.

We examine the language capabilities of language models (LMs) from the critical perspective of human language acquisition. Building on classical language development theories, we propose a three-stage framework to assess the abilities of LMs, ranging from preliminary word understanding to complex grammar and complex logical reasoning. Using this framework, we evaluate the generative capacities of LMs using methods from linguistic research. Results indicate that although recent LMs outperform earlier models in overall performance, their developmental trajectory does not strictly follow the path of human language acquisition. Notably, in generation tasks, LMs are more similar to human performance in areas where information is easier to extract from the corpus, such as average word length, clauses, and auxiliary verbs. Newer LMs did not exhibit significant progress in terms of specific dimensions, such as clauses and auxiliary verbs, where the variation across corpora is relatively limited. Register theory offers a plausible explanation for these observations, suggesting that the linguistic features of the training data have a substantial impact on the models' abilities.

PSR J0737$-$3039A/B is unique among double neutron star systems. Its near-perfect edge-on orbit causes the fast spinning pulsar A to be eclipsed by the magnetic field of the slow spinning pulsar B. Using high-sensitivity MeerKAT radio observations combined with updated constraints on the system geometry, we studied the impact of these eclipses on the incident polarization properties of pulsar A. Averaging light curves together after correcting for the rotation of pulsar B revealed enormous amounts of circular polarization and rapid changes in the linear polarization position angle, which occur at phases where emission from pulsar A is partially transmitted through the magnetosphere of pulsar B. These behaviours confirm that the eclipse mechanism is the result of synchrotron absorption in a relativistic pair-plasma confined to the closed-field region of pulsar B's truncated dipolar magnetic field. We demonstrate that changes in circular polarization handedness throughout the eclipses are directly tied to the average line of sight magnetic field direction of pulsar B, from which we unambiguously determine the complete magnetic and viewing geometry of the pulsar., Comment: 8 pages, 6 figures. Accepted for publication in MNRAS

Spatial awareness is key to enable embodied multimodal AI systems. Yet, without vast amounts of spatial supervision, current Visual Language Models (VLMs) struggle at this task. In this paper, we introduce LynX, a framework that equips pretrained VLMs with visual grounding ability without forgetting their existing image and language understanding skills. To this end, we propose a Dual Mixture of Experts module that modifies only the decoder layer of the language model, using one frozen Mixture of Experts (MoE) pre-trained on image and language understanding and another learnable MoE for new grounding capabilities. This allows the VLM to retain previously learned knowledge and skills, while acquiring what is missing. To train the model effectively, we generate a high-quality synthetic dataset we call SCouT, which mimics human reasoning in visual grounding. This dataset provides rich supervision signals, describing a step-by-step multimodal reasoning process, thereby simplifying the task of visual grounding. We evaluate LynX on several object detection and visual grounding datasets, demonstrating strong performance in object detection, zero-shot localization and grounded reasoning while maintaining its original image and language understanding capabilities on seven standard benchmark datasets.

Effective spoken dialog systems should facilitate natural interactions with quick and rhythmic timing, mirroring human communication patterns. To reduce response times, previous efforts have focused on minimizing the latency in automatic speech recognition (ASR) to optimize system efficiency. However, this approach requires waiting for ASR to complete processing until a speaker has finished speaking, which limits the time available for natural language processing (NLP) to formulate accurate responses. As humans, we continuously anticipate and prepare responses even while the other party is still speaking. This allows us to respond appropriately without missing the optimal time to speak. In this work, as a pioneering study toward a conversational system that simulates such human anticipatory behavior, we aim to realize a function that can predict the forthcoming words and estimate the time remaining until the end of an utterance (EOU), using the middle portion of an utterance. To achieve this, we propose a training strategy for an encoder-decoder-based ASR system, which involves masking future segments of an utterance and prompting the decoder to predict the words in the masked audio. Additionally, we develop a cross-attention-based algorithm that incorporates both acoustic and linguistic information to accurately detect the EOU. The experimental results demonstrate the proposed model's ability to predict upcoming words and estimate future EOU events up to 300ms prior to the actual EOU. Moreover, the proposed training strategy exhibits general improvements in ASR performance., Comment: Submitted to ICASSP2025

This paper studies natural disasters and the psychological costs of climate change. It presents what we believe to be the first evidence that higher temperature variability and not a higher level of temperature is what predicts natural disasters. This conclusion holds whether or not we control for the (incorrectly signed) impact of temperature. The analysis draws upon long-differences regression equations using GDIS data from 1960-2018 for 176 countries and the contiguous states of the USA. Results are checked on FEMA data. Wellbeing impact losses are calculated. To our knowledge, the paper's results are unknown to natural and social scientists., Comment: 54 pages

The SoccerNet 2024 challenges represent the fourth annual video understanding challenges organized by the SoccerNet team. These challenges aim to advance research across multiple themes in football, including broadcast video understanding, field understanding, and player understanding. This year, the challenges encompass four vision-based tasks. (1) Ball Action Spotting, focusing on precisely localizing when and which soccer actions related to the ball occur, (2) Dense Video Captioning, focusing on describing the broadcast with natural language and anchored timestamps, (3) Multi-View Foul Recognition, a novel task focusing on analyzing multiple viewpoints of a potential foul incident to classify whether a foul occurred and assess its severity, (4) Game State Reconstruction, another novel task focusing on reconstructing the game state from broadcast videos onto a 2D top-view map of the field. Detailed information about the tasks, challenges, and leaderboards can be found at https://www.soccer-net.org, with baselines and development kits available at https://github.com/SoccerNet., Comment: 7 pages, 1 figure

Gentle algebras are a class of special biserial algebra whose representation theory has been thoroughly described. In this paper, we consider the infinite dimensional generalizations of gentle algebras, referred to as locally gentle algebras. We give combinatorial descriptions of the center, spectrum, and homological dimensions of a locally gentle algebra, including an explicit injective resolution. We classify when these algebras are Artin-Schelter Gorenstein, Artin-Schelter regular, and Cohen-Macaulay, and provide an analogue of Stanley's theorem for locally gentle algebras.

This project, titled "Machine Translation with Large Language Models: Decoder-only vs. Encoder-Decoder," aims to develop a multilingual machine translation (MT) model. Focused on Indian regional languages, especially Telugu, Tamil, and Malayalam, the model seeks to enable accurate and contextually appropriate translations across diverse language pairs. By comparing Decoder-only and Encoder-Decoder architectures, the project aims to optimize translation quality and efficiency, advancing cross-linguistic communication tools.The primary objective is to develop a model capable of delivering high-quality translations that are accurate and contextually appropriate. By leveraging large language models, specifically comparing the effectiveness of Decoder-only and Encoder-Decoder architectures, the project seeks to optimize translation performance and efficiency across multilingual contexts. Through rigorous experimentation and analysis, this project aims to advance the field of machine translation, contributing valuable insights into the effectiveness of different model architectures and paving the way for enhanced cross-linguistic communication tools.

In recent years, the widespread informatization and rapid data explosion have increased the demand for high-performance heterogeneous systems that integrate multiple computing cores such as CPUs, Graphics Processing Units (GPUs), Application Specific Integrated Circuits (ASICs), and Field Programmable Gate Arrays (FPGAs). The combination of CPU and GPU is particularly popular due to its versatility. However, these heterogeneous systems face significant security and privacy risks. Advances in privacy-preserving techniques, especially hardware-based Trusted Execution Environments (TEEs), offer effective protection for GPU applications. Nonetheless, the potential security risks involved in extending TEEs to GPUs in heterogeneous systems remain uncertain and need further investigation. To investigate these risks in depth, we study the existing popular GPU TEE designs and summarize and compare their key implications. Additionally, we review existing powerful attacks on GPUs and traditional TEEs deployed on CPUs, along with the efforts to mitigate these threats. We identify potential attack surfaces introduced by GPU TEEs and provide insights into key considerations for designing secure GPU TEEs. This survey is timely as new TEEs for heterogeneous systems, particularly GPUs, are being developed, highlighting the need to understand potential security threats and build both efficient and secure systems., Comment: 35 pages, 7 figures

Randomization is a powerful tool that endows algorithms with remarkable properties. For instance, randomized algorithms excel in adversarial settings, often surpassing the worst-case performance of deterministic algorithms with large margins. Furthermore, their success probability can be amplified by simple strategies such as repetition and majority voting. In this paper, we enhance deep neural networks, in particular transformer models, with randomization. We demonstrate for the first time that randomized algorithms can be instilled in transformers through learning, in a purely data- and objective-driven manner. First, we analyze known adversarial objectives for which randomized algorithms offer a distinct advantage over deterministic ones. We then show that common optimization techniques, such as gradient descent or evolutionary strategies, can effectively learn transformer parameters that make use of the randomness provided to the model. To illustrate the broad applicability of randomization in empowering neural networks, we study three conceptual tasks: associative recall, graph coloring, and agents that explore grid worlds. In addition to demonstrating increased robustness against oblivious adversaries through learned randomization, our experiments reveal remarkable performance improvements due to the inherently random nature of the neural networks' computation and predictions.

We present here the current state of a collection of promising ultraviolet technologies in preparation for the Habitable Worlds Observatory. Working with experts representing a significant number of groups working in the ultraviolet, we summarize some of the leading science drivers, present an argument for a 100 nm blue wavelength cutoff, and gather current state of the art of UV technologies. We present the state of the art of contamination control, a crucial piece of the UV instrument plan. We explore next steps with individual technologies, as well as present paths forward with systems level testing and development.

Microscopic traffic simulations are used to evaluate the impact of infrastructure modifications and evolving vehicle technologies, such as connected and automated driving. Simulated vehicles are controlled via car-following, lane-changing and junction models, which are designed to imitate human driving behavior. However, physics-based car-following models (CFMs) cannot fully replicate measured vehicle trajectories. Therefore, we present model extensions for the Intelligent Driver Model (IDM), of which some are already included in the Extended Intelligent Driver Model (EIDM), to improve calibration and validation results. They consist of equations based on vehicle dynamics and drive off procedures. In addition, parameter selection plays a decisive role. Thus, we introduce a framework to calibrate CFMs using drone data captured at a signalized intersection in Stuttgart, Germany. We compare the calibration error of the Krauss Model with the IDM and EIDM. In this setup, the EIDM achieves a 17.78 % lower mean error than the IDM, based on the distance difference between real world and simulated vehicles. Adding vehicle dynamics equations to the EIDM further improves the results by an additional 18.97 %. The calibrated vehicle-driver combinations are then investigated by simulating the traffic in three different scenarios: at the original intersection, in a closed loop and in a stop-and-go wave. The data shows that the improved calibration process of individual vehicles, openly available at https://www.github.com/stepeos/pycarmodel_calibration, also provides more accurate macroscopic results., Comment: Accepted to the 27th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2024)

Aspera is a NASA Astrophysics Pioneers SmallSat mission designed to study diffuse OVI emission from the warm-hot phase gas in the halos of nearby galaxies. Its payload consists of two identical Rowland Circle-type long-slit spectrographs, sharing a single MicroChannel plate detector. Each spectrograph channel consists of an off-axis parabola primary mirror and a toroidal diffraction grating optimized for the 1013-1057 Angstroms bandpass. Despite the simple configuration, the optical alignment/integration process for Aspera is challenging due to tight optical alignment tolerances, driven by the compact form factor, and the contamination sensitivity of the Far-Ultraviolet optics and detectors. In this paper, we discuss implementing a novel multi-phase approach to meet these requirements using state-of-the-art optical metrology tools. For coarsely positioning the optics we use a blue-laser 3D scanner while the fine alignment is done with a Zygo interferometer and a custom computer-generated hologram. The detector focus requires iterative in-vacuum alignment using a Vacuum UV collimator. The alignment is done in a controlled cleanroom facility at the University of Arizona., Comment: Manuscript submitted for Proceedings of Astronomical Telescopes + Instrumentation 2024, Paper no. 13093-9

Many tasks can be composed from a few independent components. This gives rise to a combinatorial explosion of possible tasks, only some of which might be encountered during training. Under what circumstances can transformers compositionally generalize from a subset of tasks to all possible combinations of tasks that share similar components? Here we study a modular multitask setting that allows us to precisely control compositional structure in the data generation process. We present evidence that transformers learning in-context struggle to generalize compositionally on this task despite being in principle expressive enough to do so. Compositional generalization becomes possible only when introducing a bottleneck that enforces an explicit separation between task inference and task execution., Comment: ICML 2024 workshop on Next Generation of Sequence Modeling Architectures

We use data from the MeerTime project on the MeerKAT telescope to ask whether the radio emission properties of millisecond pulsars (MSPs) and slowly rotating, younger pulsars (SPs) are similar or different. We show that the flux density spectra of both populations are similarly steep, and the widths of MSP pulsar profiles obey the same dependence on the rotational period as slow pulsars. We also show that the polarization of MSPs has similar properties to slow pulsars. The commonly used pseudo-luminosity of pulsars, defined as the product of the flux density and the distance squared, is not appropriate for drawing conclusions about the relative intrinsic radio luminosity of SPs and MSPs. We show that it is possible to scale the pseudo-luminosity to account for the pulse duty cycle and the solid angle of the radio beam, in such a way that MSPs and SPs do not show clear differences in intrinsic luminosity. The data, therefore, support common emission physics between the two populations in spite of orders of magnitude difference in their period derivatives and inferred, surface, dipole magnetic field strengths., Comment: 9 pages, accepted for publication in MNRAS

This draft article outlines a prediction challenge where the target is to forecast the number of fatalities in armed conflicts, in the form of the UCDP `best' estimates, aggregated to the VIEWS units of analysis. It presents the format of the contributions, the evaluation metric, and the procedures, and a brief summary of the contributions. The article serves a function analogous to a pre-analysis plan: a statement of the forecasting models made publicly available before the true future prediction window commences. More information on the challenge, and all data referred to in this document, can be found at https://viewsforecasting.org/research/prediction-challenge-2023., Comment: Forecasting competition, conflict forecasting, forecasting with uncertainty

Several sources of repeating coherent bursts of radio emission with periods of many minutes have now been reported in the literature. These "ultra-long period" (ULP) sources have no clear multi-wavelength counterparts and challenge canonical pulsar emission models, leading to debate regarding their nature. In this work we report the discovery of a bright, highly-polarised burst of radio emission at low Galactic latitude as part of a wide-field survey for transient and variable radio sources. ASKAP\,J175534.9$-$252749.1 does not appear to repeat, with only a single intense two-minute $\sim$200-mJy burst detected from 60~hours of observations. The burst morphology and polarisation properties are comparable to those of classical pulsars but the duration is more than one hundred times longer, analogous to ULPs. Combined with the existing ULP population, this suggests that these sources have a strong Galactic latitude dependence and hints at an unexplored population of transient and variable radio sources in the thin disk of the Milky Way. The resemblance of this burst with both ULPs and pulsars calls for a unified coherent emission model for objects with spin periods from milliseconds to tens of minutes. However, whether or not these are all neutron stars or have the same underlying power source remains open for debate.

This work does not introduce a new method. Instead, we present an interesting finding that questions the necessity of the inductive bias -- locality in modern computer vision architectures. Concretely, we find that vanilla Transformers can operate by directly treating each individual pixel as a token and achieve highly performant results. This is substantially different from the popular design in Vision Transformer, which maintains the inductive bias from ConvNets towards local neighborhoods (e.g. by treating each 16x16 patch as a token). We mainly showcase the effectiveness of pixels-as-tokens across three well-studied tasks in computer vision: supervised learning for object classification, self-supervised learning via masked autoencoding, and image generation with diffusion models. Although directly operating on individual pixels is less computationally practical, we believe the community must be aware of this surprising piece of knowledge when devising the next generation of neural architectures for computer vision., Comment: Technical report, 23 pages

Existing perception methods for autonomous driving fall short of recognizing unknown entities not covered in the training data. Open-vocabulary methods offer promising capabilities in detecting any object but are limited by user-specified queries representing target classes. We propose AutoVoc3D, a framework for automatic object class recognition and open-ended segmentation. Evaluation on nuScenes showcases AutoVoc3D's ability to generate precise semantic classes and accurate point-wise segmentation. Moreover, we introduce Text-Point Semantic Similarity, a new metric to assess the semantic similarity between text and point cloud without eliminating novel classes., Comment: Accepted by CVPR 2024 OpenSun3D Workshop

A new breed of gated-linear recurrent neural networks has reached state-of-the-art performance on a range of sequence modeling problems. Such models naturally handle long sequences efficiently, as the cost of processing a new input is independent of sequence length. Here, we explore another advantage of these stateful sequence models, inspired by the success of model merging through parameter interpolation. Building on parallels between fine-tuning and in-context learning, we investigate whether we can treat internal states as task vectors that can be stored, retrieved, and then linearly combined, exploiting the linearity of recurrence. We study this form of fast model merging on Mamba-2.8b, a pretrained recurrent model, and present preliminary evidence that simple linear state interpolation methods suffice to improve next-token perplexity as well as downstream in-context learning task performance.

We present an experimental investigation of coherent crosstalk cancellation methods for light delivered to a linear ion chain cryogenic quantum register. The ions are individually addressed using focused laser beams oriented perpendicular to the crystal axis, which are created by imaging each output of a multi-core photonic-crystal fibre waveguide array onto a single ion. The measured nearest-neighbor native crosstalk intensity of this device for ions spaced by 5 $\mu$m is found to be $\sim 10^{-2}$. We show that we can suppress this intensity crosstalk from waveguide channel coupling and optical diffraction effects by a factor $>10^3$ using cancellation light supplied to neighboring channels which destructively interferes with the crosstalk. We measure a rotation error per gate on the order of $\epsilon_{x} \sim 10^{-5}$ on spectator qubits, demonstrating a suppression of crosstalk error by a factor of $> 10^2$. We compare the performance to composite pulse methods for crosstalk cancellation, and describe the appropriate calibration methods and procedures to mitigate phase drifts between these different optical paths, including accounting for problems arising due to pulsing of optical modulators.

3D Gaussian Splatting has emerged as a powerful representation of geometry and appearance for RGB-only dense Simultaneous Localization and Mapping (SLAM), as it provides a compact dense map representation while enabling efficient and high-quality map rendering. However, existing methods show significantly worse reconstruction quality than competing methods using other 3D representations, e.g. neural points clouds, since they either do not employ global map and pose optimization or make use of monocular depth. In response, we propose the first RGB-only SLAM system with a dense 3D Gaussian map representation that utilizes all benefits of globally optimized tracking by adapting dynamically to keyframe pose and depth updates by actively deforming the 3D Gaussian map. Moreover, we find that refining the depth updates in inaccurate areas with a monocular depth estimator further improves the accuracy of the 3D reconstruction. Our experiments on the Replica, TUM-RGBD, and ScanNet datasets indicate the effectiveness of globally optimized 3D Gaussians, as the approach achieves superior or on par performance with existing RGB-only SLAM methods methods in tracking, mapping and rendering accuracy while yielding small map sizes and fast runtimes. The source code is available at https://github.com/eriksandstroem/Splat-SLAM., Comment: 21 pages

Neural Architecture Search (NAS) methods have shown to output networks that largely outperform human-designed networks. However, conventional NAS methods have mostly tackled the single dataset scenario, incuring in a large computational cost as the procedure has to be run from scratch for every new dataset. In this work, we focus on predictor-based algorithms and propose a simple and efficient way of improving their prediction performance when dealing with data distribution shifts. We exploit the Kronecker-product on the randomly wired search-space and create a small NAS benchmark composed of networks trained over four different datasets. To improve the generalization abilities, we propose GRASP-GCN, a ranking Graph Convolutional Network that takes as additional input the shape of the layers of the neural networks. GRASP-GCN is trained with the not-at-convergence accuracies, and improves the state-of-the-art of 3.3 % for Cifar-10 and increasing moreover the generalization abilities under data distribution shift.

The S-shaped swing of the linear polarization position angle (PPA) observed in many pulsars can be interpreted by the rotating vector model (RVM). However, efforts to fit the RVM for a large sample of pulsars observed with the MeerKAT telescope as a part of the Thousand-Pulsar-Array (TPA) programme, only succeeded for about half the cases. High time-resolution studies suggest that the failed cases arise due to the presence of orthogonal polarization modes, or highly disordered distribution of PPA points. One such example is PSR~J1645-0317. Recently it has been shown that the RVM can be recovered in this pulsar by using only time samples which are greater than 80% linearly polarized. In this work we test this novel approach on the brightest 249 pulsars from the TPA sample, of which 177 yield sufficient highly polarized samples to be amenable to our method. Remarkably, only 9 of these pulsars (5%) now fail to fit the RVM as opposed to 59% from the original analysis. This result favours the paradigm that the underlying mechanism is coherent curvature radiation., Comment: Delayed interminably by MNRAS

Creative coding is an experimentation-heavy activity that requires translating high-level visual ideas into code. However, most languages and libraries for creative coding may not be adequately intuitive for beginners. In this paper, we present AniFrame, a domain-specific language for drawing and animation. Designed for novice programmers, it (i) features animation-specific data types, operations, and built-in functions to simplify the creation and animation of composite objects, (ii) allows for fine-grained control over animation sequences through explicit specification of the target object and the start and end frames, (iii) reduces the learning curve through a Python-like syntax, type inferencing, and a minimal set of control structures and keywords that map closely to their semantic intent, and (iv) promotes computational expressivity through support for common mathematical operations, built-in trigonometric functions, and user-defined recursion. Our usability test demonstrates AniFrame's potential to enhance readability and writability for multiple creative coding use cases. AniFrame is open-source, and its implementation and reference are available at https://github.com/memgonzales/aniframe-language., Comment: Accepted for paper presentation at the 24th Philippine Computing Science Congress (PCSC 2024), held in Laguna, Philippines

Developing domain models is one of the few remaining places that require manual human labor in AI planning. Thus, in order to make planning more accessible, it is desirable to automate the process of domain model generation. To this end, we investigate if large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. Specifically, we introduce a framework for automated evaluation of LLM-generated domains by comparing the sets of plans for domain instances. Finally, we perform an empirical analysis of 7 large language models, including coding and chat models across 9 different planning domains, and under three classes of natural language domain descriptions. Our results indicate that LLMs, particularly those with high parameter counts, exhibit a moderate level of proficiency in generating correct planning domains from natural language descriptions. Our code is available at https://github.com/IBM/NL2PDDL., Comment: Published at ICAPS 2024

We report here on the timing of 597 pulsars over the last four years with the MeerKAT telescope. We provide Times-of-Arrival, pulsar ephemeris files and per-epoch measurements of the flux density, dispersion measure (DM) and rotation measure (RM) for each pulsar. In addition we use a Gaussian process to model the timing residuals to measure the spin frequency derivative at each epoch. We also report the detection of 11 glitches in 9 individual pulsars. We find significant DM and RM variations in 87 and 76 pulsars respectively. We find that the DM variations scale approximately linearly with DM, which is broadly in agreement with models of the ionised interstellar medium. The observed RM variations seem largely independent of DM, which may suggest that the RM variations are dominated by variations in the interstellar magnetic field on the line of sight, rather than varying electron density. We also find that normal pulsars have around 5 times greater amplitude of DM variability compared to millisecond pulsars, and surmise that this is due to the known difference in their velocity distributions., Comment: This is a pre-copyedited, author-produced PDF of an article accepted for publication in MNRAS following peer review. 12 pages 7 figures

Although large language models (LLMs) have demonstrated remarkable proficiency in modeling text and generating human-like text, they may exhibit biases acquired from training data in doing so. Specifically, LLMs may be susceptible to a common cognitive trap in human decision-making called the representativeness heuristic. This is a concept in psychology that refers to judging the likelihood of an event based on how closely it resembles a well-known prototype or typical example, versus considering broader facts or statistical evidence. This research investigates the impact of the representativeness heuristic on LLM reasoning. We created ReHeAT (Representativeness Heuristic AI Testing), a dataset containing a series of problems spanning six common types of representativeness heuristics. Experiments reveal that four LLMs applied to ReHeAT all exhibited representativeness heuristic biases. We further identify that the model's reasoning steps are often incorrectly based on a stereotype rather than on the problem's description. Interestingly, the performance improves when adding a hint in the prompt to remind the model to use its knowledge. This suggests the uniqueness of the representativeness heuristic compared to traditional biases. It can occur even when LLMs possess the correct knowledge while falling into a cognitive trap. This highlights the importance of future research focusing on the representativeness heuristic in model reasoning and decision-making and on developing solutions to address it., Comment: Published as a conference paper at COLM 2024

Recent advancements in RGB-only dense Simultaneous Localization and Mapping (SLAM) have predominantly utilized grid-based neural implicit encodings and/or struggle to efficiently realize global map and pose consistency. To this end, we propose an efficient RGB-only dense SLAM system using a flexible neural point cloud scene representation that adapts to keyframe poses and depth updates, without needing costly backpropagation. Another critical challenge of RGB-only SLAM is the lack of geometric priors. To alleviate this issue, with the aid of a monocular depth estimator, we introduce a novel DSPO layer for bundle adjustment which optimizes the pose and depth of keyframes along with the scale of the monocular depth. Finally, our system benefits from loop closure and online global bundle adjustment and performs either better or competitive to existing dense neural RGB SLAM methods in tracking, mapping and rendering accuracy on the Replica, TUM-RGBD and ScanNet datasets. The source code is available at https://github.com/zhangganlin/GlOIRE-SLAM

We study ancestral lineages of individuals of a stationary discrete-time branching annihilating random walk (BARW) on the $d$-dimensional lattice $\mathbb{Z}^d$. Each individual produces a Poissonian number of offspring with mean $\mu$ which then jump independently to a uniformly chosen site with a fixed distance $R$ of their parent. By interpreting the ancestral lineage of such an individual as a random walk in a dynamical random environment, we obtain a law of large numbers and a functional central limit theorem for the ancestral lineage., Comment: 21 pages, 1 figure

The landscape of deep learning research is moving towards innovative strategies to harness the true potential of data. Traditionally, emphasis has been on scaling model architectures, resulting in large and complex neural networks, which can be difficult to train with limited computational resources. However, independently of the model size, data quality (i.e. amount and variability) is still a major factor that affects model generalization. In this work, we propose a novel technique to exploit available data through the use of automatic data augmentation for the tasks of image classification and semantic segmentation. We introduce the first Differentiable Augmentation Search method (DAS) to generate variations of images that can be processed as videos. Compared to previous approaches, DAS is extremely fast and flexible, allowing the search on very large search spaces in less than a GPU day. Our intuition is that the increased receptive field in the temporal dimension provided by DAS could lead to benefits also to the spatial receptive field. More specifically, we leverage DAS to guide the reshaping of the spatial receptive field by selecting task-dependant transformations. As a result, compared to standard augmentation alternatives, we improve in terms of accuracy on ImageNet, Cifar10, Cifar100, Tiny-ImageNet, Pascal-VOC-2012 and CityScapes datasets when plugging-in our DAS over different light-weight video backbones.

In this research, we introduce a novel methodology for assessing Emotional Mimicry Intensity (EMI) as part of the 6th Workshop and Competition on Affective Behavior Analysis in-the-wild. Our methodology utilises the Wav2Vec 2.0 architecture, which has been pre-trained on an extensive podcast dataset, to capture a wide array of audio features that include both linguistic and paralinguistic components. We refine our feature extraction process by employing a fusion technique that combines individual features with a global mean vector, thereby embedding a broader contextual understanding into our analysis. A key aspect of our approach is the multi-task fusion strategy that not only leverages these features but also incorporates a pre-trained Valence-Arousal-Dominance (VAD) model. This integration is designed to refine emotion intensity prediction by concurrently processing multiple emotional dimensions, thereby embedding a richer contextual understanding into our framework. For the temporal analysis of audio data, our feature fusion process utilises a Long Short-Term Memory (LSTM) network. This approach, which relies solely on the provided audio data, shows marked advancements over the existing baseline, offering a more comprehensive understanding of emotional mimicry in naturalistic settings, achieving the second place in the EMI challenge.

Current multimodal models leveraging contrastive learning often face limitations in developing fine-grained conceptual understanding. This is due to random negative samples during pretraining, causing almost exclusively very dissimilar concepts to be compared in the loss function. Consequently, the models struggle with fine-grained semantic differences. To address this problem, we introduce a novel pretraining method incorporating synthetic hard negative text examples. The hard negatives permute terms corresponding to visual concepts, leading to a more fine-grained visual and textual concept alignment. Further, we introduce InpaintCOCO, a new challenging dataset for assessing the fine-grained alignment of colors, objects, and sizes in vision-language models. We created the dataset using generative inpainting from COCO images by changing the visual concepts so that the images no longer match their original captions. Our results show significant improvements in fine-grained concept understanding across a wide range of vision-language datasets, including our InpaintCOCO dataset.

Recent research has demonstrated that transformers, particularly linear attention models, implicitly execute gradient-descent-like algorithms on data provided in-context during their forward inference step. However, their capability in handling more complex problems remains unexplored. In this paper, we prove that each layer of a linear transformer maintains a weight vector for an implicit linear regression problem and can be interpreted as performing a variant of preconditioned gradient descent. We also investigate the use of linear transformers in a challenging scenario where the training data is corrupted with different levels of noise. Remarkably, we demonstrate that for this problem linear transformers discover an intricate and highly effective optimization algorithm, surpassing or matching in performance many reasonable baselines. We analyze this algorithm and show that it is a novel approach incorporating momentum and adaptive rescaling based on noise levels. Our findings show that even linear transformers possess the surprising ability to discover sophisticated optimization strategies.

Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges.

Neural RGBD SLAM techniques have shown promise in dense Simultaneous Localization And Mapping (SLAM), yet face challenges such as error accumulation during camera tracking resulting in distorted maps. In response, we introduce Loopy-SLAM that globally optimizes poses and the dense 3D model. We use frame-to-model tracking using a data-driven point-based submap generation method and trigger loop closures online by performing global place recognition. Robust pose graph optimization is used to rigidly align the local submaps. As our representation is point based, map corrections can be performed efficiently without the need to store the entire history of input frames used for mapping as typically required by methods employing a grid based mapping structure. Evaluation on the synthetic Replica and real-world TUM-RGBD and ScanNet datasets demonstrate competitive or superior performance in tracking, mapping, and rendering accuracy when compared to existing dense neural RGBD SLAM methods. Project page: notchla.github.io/Loopy-SLAM.

A number of pulsars are known to have profile evolution on timescales of months, often correlated with spin-down rate changes. Here, we present the first result from 3 years of monitoring observations from MeerKAT as part of the Thousand Pulsar Array programme. This programme obtains high-fidelity pulse profiles for $\sim$ 500 pulsars, which enabled the detection of subtle changes in seven sources not previously known to exhibit long-term profile evolution. A 2D Gaussian convolution is used to highlight correlated emission variability in both the pulse phase and observing epoch direction. Simulations show that for one additional source the observed profile variability is likely to originate from stochastic single-pulse shape variability (jitter). We find that it is common for long-term profile variability to be associated with changes in polarization fractions, but not with polarisation position angle (PA) changes. PA changes are expected if emission height changes or precession is responsible for the profile variability. PSR J1741$-$3927 is the only pulsar in our sample that shows correlated PA variability, and this is associated with orthogonal polarization mode activity. For the six %the rest, without correlated PA variability, other pulsars limits on possible emission height changes and impact angle changes are derived. These limits are consistent with the small changes in the total intensity profile shape. None of the sources show detectable spin-down variability correlated with the emission changes, which are thought to be driven by magnetospheric current fluctuations. Therefore the absence of correlated spin-down rate variability allows upper limits to be placed on changes in the magnetospheric charge density., Comment: 19 pages, 12 figures, 3 Tables. Accepted for publication in MNRAS