Search

At the risk of overstating the case, connectionist approaches to machine learning, i.e. neural networks, are enjoying a small vogue right now. However, these methods require large volumes of data and produce models that are uninterpretable to humans. An alternative framework that is compatible with neural networks and gradient-based learning, but explicitly models compositionality, is Vector Symbolic Architectures (VSAs). VSAs are a family of algebras on high-dimensional vector representations. They arose in cognitive science from the need to unify neural processing and the kind of symbolic reasoning that humans perform. While machine learning methods have benefited from category theoretical analyses, VSAs have not yet received similar treatment. In this paper, we present a first attempt at applying category theory to VSAs. Specifically, we conduct a brief literature survey demonstrating the lacking intersection of these two topics, provide a list of desiderata for VSAs, and propose that VSAs may be understood as a (division) rig in a category enriched over a monoid in Met (the category of Lawvere metric spaces). This final contribution suggests that VSAs may be generalised beyond current implementations. It is our hope that grounding VSAs in category theory will lead to more rigorous connections with other research, both within and beyond, learning and cognition., Comment: 13 pages, no figures, 2 tables, one appendix

Strong anomalous diffusion is {often} characterized by a piecewise-linear spectrum of the moments of displacement. The spectrum is characterized by slopes $\xi$ and $\zeta$ for small and large moments, respectively, and by the critical moment $\alpha$ of the crossover. The exponents $\xi$ and $\zeta$ characterize the asymptotic scaling of the bulk and the tails of the probability distribution function of displacements, respectively. Here, we adopt asymptotic theory to match the behaviors at intermediate scales. The resulting constraint explains how distributions with algebraic tails imply strong anomalous diffusion, and it relates $\alpha$ to the corresponding power law. Our theory provides novel relations between exponents characterizing strong anomalous diffusion, and it yields explicit expressions for the leading-order corrections to the asymptotic power-law behavior of the moments of displacement. They provide the time scale that must be surpassed to clearly discriminate the leading-order power law from its sub-leading corrections. This insight allows us to point out sources of systematic errors in their numerical estimates. Rather than separately fitting an exponent for each moment we devise a robust scheme to determine $\xi$, $\zeta$ and $\alpha$. The findings are supported by numerical and analytical results on five different models exhibiting strong anomalous diffusion.

High-dimensional vectors have been proposed as a neural method for representing information in the brain using Vector Symbolic Algebras (VSAs). While previous work has explored decoding and cleaning up these vectors under the noise that arises during computation, existing methods are limited. Cleanup methods are essential for robust computation within a VSA. However, cleanup methods for continuous-value encodings are not as effective. In this paper, we present an iterative optimization method to decode and clean up Fourier Holographic Reduced Representation (FHRR) vectors that are encoding continuous values. We combine composite likelihood estimation (CLE) and maximum likelihood estimation (MLE) to ensure convergence to the global optimum. We also demonstrate that this method can effectively decode FHRR vectors under different noise conditions, and show that it outperforms existing methods., Comment: 9 pages

An adversarial example is a modified input image designed to cause a Machine Learning (ML) model to make a mistake; these perturbations are often invisible or subtle to human observers and highlight vulnerabilities in a model's ability to generalize from its training data. Several adversarial attacks can create such examples, each with a different perspective, effectiveness, and perceptibility of changes. Conversely, defending against such adversarial attacks improves the robustness of ML models in image processing and other domains of deep learning. Most defence mechanisms require either a level of model awareness, changes to the model, or access to a comprehensive set of adversarial examples during training, which is impractical. Another option is to use an auxiliary model in a preprocessing manner without changing the primary model. This study presents a practical and effective solution -- using predictive coding networks (PCnets) as an auxiliary step for adversarial defence. By seamlessly integrating PCnets into feed-forward networks as a preprocessing step, we substantially bolster resilience to adversarial perturbations. Our experiments on MNIST and CIFAR10 demonstrate the remarkable effectiveness of PCnets in mitigating adversarial examples with about 82% and 65% improvements in robustness, respectively. The PCnet, trained on a small subset of the dataset, leverages its generative nature to effectively counter adversarial efforts, reverting perturbed images closer to their original forms. This innovative approach holds promise for enhancing the security and reliability of neural network classifiers in the face of the escalating threat of adversarial attacks.

Achieving personalized intelligence at the edge with real-time learning capabilities holds enormous promise in enhancing our daily experiences and helping decision making, planning, and sensing. However, efficient and reliable edge learning remains difficult with current technology due to the lack of personalized data, insufficient hardware capabilities, and inherent challenges posed by online learning. Over time and across multiple developmental stages, the brain has evolved to efficiently incorporate new knowledge by gradually building on previous knowledge. In this work, we emulate the multiple stages of learning with digital neuromorphic technology that simulates the neural and synaptic processes of the brain using two stages of learning. First, a meta-training stage trains the hyperparameters of synaptic plasticity for one-shot learning using a differentiable simulation of the neuromorphic hardware. This meta-training process refines a hardware local three-factor synaptic plasticity rule and its associated hyperparameters to align with the trained task domain. In a subsequent deployment stage, these optimized hyperparameters enable fast, data-efficient, and accurate learning of new classes. We demonstrate our approach using event-driven vision sensor data and the Intel Loihi neuromorphic processor with its plasticity dynamics, achieving real-time one-shot learning of new classes that is vastly improved over transfer learning. Our methodology can be deployed with arbitrary plasticity models and can be applied to situations demanding quick learning and adaptation at the edge, such as navigating unfamiliar environments or learning unexpected categories of data through user engagement., Comment: 17 page journal article. Submitted to IOP NCE

We study the evidence for dark energy (DE) evolution at low redshift, using baryonic acoustic oscillations (BAOs) from the DESI Early Data Release, Pantheon+ Type Ia supernovae (SNe-Ia), and redshift space distortions (RSDs) to constrain cosmological parameters. Furthermore, we make use of the angular acoustic scale to analyse the effect of introducing condensed CMB information on the cosmological parameters informing DE evolution. The analysis is divided into cases based on the variability of priors inferred from early-time physics. Using a quadratic parametrisation, $X(z)$, for DE density, we find evidence for DE evolution in all cases, both with and without the angular acoustic scale. We reconstruct $X(z)$ using best fit parameters and find that DE density starts to exhibit dynamical behaviour at $z \sim 0.5 $, assuming negative values beyond $z\sim 1.5$. The data show no significant preference for $X(z)$CDM over a $\Lambda$CDM, with both models performing equally well according to our chosen metrics of reduced $\chi^2$ and the Durbin-Watson statistic.

Neuroscience and Artificial Intelligence (AI) have progressed in tandem, each contributing to our understanding of the brain, and inspiring recent developments in biologically-plausible neural networks (NNs) and learning rules. Predictive coding (PC), and its learning rule, have been shown to approximate error backpropagation in a biologically relevant manner, with local weight updates that depend only on the activity of the pre- and post-synaptic neurons. Unlike traditional feedforward NNs where the flow of information goes in one direction, PC models mimic the brain more accurately by passing information bidirectionally: prediction in one direction, and correction/error in the other. PC models learn by clamping some neurons to target values and running the network to equilibrium. At equilibrium, the network calculates its own error gradients right at the location where they are used for weight updates. Traditional backprop requires the computation graph to be feedforward. However, the PC version of backprop does not have this requirement. Amazingly, no one has demonstrated the application of PC learning directly to recurrent neural networks (RNNs). Hopfield networks (HNs) are RNNs that implement a content-addressable memory, learning patterns (or ``memories'') that can be retrieved from partial or corrupted patterns. In this paper, we show that a HN can be trained using the PC learning rules without modification. To our knowledge, this is the first time PC learning has been applied directly to train a RNN, without the need to unroll it in time. Our results indicate that the PC-trained HNs behave like classical HNs., Comment: 9 pages, 6 figures

Recent work conceptualized root cause analysis (RCA) of anomalies via quantitative contribution analysis using causal counterfactuals in structural causal models (SCMs).The framework comes with three practical challenges: (1) it requires the causal directed acyclic graph (DAG), together with an SCM, (2) it is statistically ill-posed since it probes regression models in regions of low probability density, (3) it relies on Shapley values which are computationally expensive to find. In this paper, we propose simplified, efficient methods of root cause analysis when the task is to identify a unique root cause instead of quantitative contribution analysis. Our proposed methods run in linear order of SCM nodes and they require only the causal DAG without counterfactuals. Furthermore, for those use cases where the causal DAG is unknown, we justify the heuristic of identifying root causes as the variables with the highest anomaly score. To this end, we prove that anomalies with small scores are unlikely to cause those with large scores and show upper bounds for the likelihood of causal pathways with non-monotonic anomaly scores.

Despite an abundance of proposed systems, the verification of units-of-measure within programs remains rare in scientific computing. We attempt to address this issue by providing a lightweight static verification system for units-of-measure in Fortran programs which supports incremental annotation of large projects. We take the opposite approach to the most mainstream existing deployment of units-of-measure typing (in F#) and generate a global, rather than local, constraints system for a program. We show that such a system can infer (and check) polymorphic units specifications for under-determined parts of the program. Not only does this ability allow checking of partially annotated programs but it also allows the global constraint problem to be partitioned. This partitioning means we can scale to large programs by solving constraints for each program module independently and storing inferred units at module boundaries (separate verification). We provide an implementation of our approach as an extension to an open-source Fortran analysis tool., Comment: Please note that this article was written in 2018 and has been relocated to arXiv simply for archival purposes

Polygonal billiards exhibit a rich and complex dynamical behavior. In recent years polygonal billiards have attracted great attention due to their application in the understanding of anomalous transport, but also at the fundamental level, due to its connections with diverse fields in mathematics. We explore this complexity and its consequences on the properties of particle transport in infinitely long channels made of the repetitions of an elementary open polygonal cell. Borrowing ideas from the Zemlyakov-Katok construction, we construct an interval exchange transformation classified by the singular directions of the discontinuities of the billiard flow over the translation surface associated to the elementary cell. From this, we derive an exact expression of a scattering map of the cell connecting the outgoing flow of trajectories with the unconstrained incoming flow. The scattering map is defined over a partition of the coordinate space, characterized by different families of trajectories. Furthermore, we obtain an analytical expression for the average speed of propagation of ballistic modes, describing with high accuracy the speed of propagation of ballistic fronts appearing in the tails of the distribution of the particle displacement. The symbolic hierarchy of the trajectories forming these ballistic fronts is also discussed.

The ability to detect when a system undergoes an incipient fault is of paramount importance in preventing a critical failure. In this work, we propose an information-driven fault detection method based on a novel concept drift detector. The method is tailored to identifying drifts in input-output relationships of additive noise models (i.e., model drifts) and is based on a distribution-free mutual information (MI) estimator. Our scheme does not require prior faulty examples and can be applied distribution-free over a large class of system models. Our core contributions are twofold. First, we demonstrate the connection between fault detection, model drift detection, and testing independence between two random variables. Second, we prove several theoretical properties of the proposed MI-based fault detection scheme: (i) strong consistency, (ii) exponentially fast detection of the non-faulty case, and (iii) control of both significance levels and power of the test. To conclude, we validate our theory with synthetic data and the benchmark dataset N-CMAPSS of aircraft turbofan engines. These empirical results support the usefulness of our methodology in many practical and realistic settings, and the theoretical results show performance guarantees that other methods cannot offer., Comment: 28 pages, 11 figures

Charts, figures, and text derived from data play an important role in decision making, from data-driven policy development to day-to-day choices informed by online articles. Making sense of, or fact-checking, outputs means understanding how they relate to the underlying data. Even for domain experts with access to the source code and data sets, this poses a significant challenge. In this paper we introduce a new program analysis framework which supports interactive exploration of fine-grained I/O relationships directly through computed outputs, making use of dynamic dependence graphs. Our main contribution is a novel notion in data provenance which we call related inputs, a relation of mutual relevance or "cognacy" which arises between inputs when they contribute to common features of the output. Queries of this form allow readers to ask questions like "What outputs use this data element, and what other data elements are used along with it?". We show how Jonsson and Tarski's concept of conjugate operators on Boolean algebras appropriately characterises the notion of cognacy in a dependence graph, and give a procedure for computing related inputs over such a graph.

Graded modal logics generalise standard modal logics via families of modalities indexed by an algebraic structure whose operations mediate between the different modalities. The graded "of-course" modality $!_r$ captures how many times a proposition is used and has an analogous interpretation to the of-course modality from linear logic; the of-course modality from linear logic can be modelled by a linear exponential comonad and graded of-course can be modelled by a graded linear exponential comonad. Benton showed in his seminal paper on Linear/Non-Linear logic that the of-course modality can be split into two modalities connecting intuitionistic logic with linear logic, forming a symmetric monoidal adjunction. Later, Fujii et al. demonstrated that every graded comonad can be decomposed into an adjunction and a `strict action'. We give a similar result to Benton, leveraging Fujii et al.'s decomposition, showing that graded modalities can be split into two modalities connecting a graded logic with a graded linear logic. We propose a sequent calculus, its proof theory and categorical model, and a natural deduction system which we show is isomorphic to the sequent calculus system. Interestingly, our system can also be understood as Linear/Non-Linear logic composed with an action that adds the grading, further illuminating the shared principles between linear logic and a class of graded modal logics., Comment: Published in 33rd EACSL Annual Conference on Computer Science Logic (CSL 2025)

In this paper we study the joint determination of source and background flux for point sources as observed by digital array detectors. We explicitly compute the two-dimensional Cram\'er-Rao absolute lower bound (CRLB) as well as the performance bounds for high-dimensional implicit estimators from a generalized Taylor expansion. This later approach allows us to obtain computable prescriptions for the bias and variance of the joint estimators. We compare these prescriptions with empirical results from numerical simulations in the case of the weighted least squares estimator (introducing an improved version, denoted stochastic weighted least-squares) as well as with the maximum likelihood estimator, finding excellent agreement. We demonstrate that these estimators provide quasi-unbiased joint estimations of the flux and background, with a variance that approaches the CRLB very tightly and are, hence, optimal, unlike the case of sequential estimation used commonly in astronomical photometry which is sub-optimal. We compare our predictions with numerical simulations of realistic observations, as well as with observations of a bona-fide non-variable stellar source observed with TESS, and compare it to the results from the sequential estimation of background and flux, confirming our theoretical expectations. Our practical estimators can be used as benchmarks for general photometric pipelines, or for applications that require maximum precision and accuracy in absolute photometry., Comment: Accepted by PASP. 35 pages, 12 figures, 5 appendices

Identifying root causes for unexpected or undesirable behavior in complex systems is a prevalent challenge. This issue becomes especially crucial in modern cloud applications that employ numerous microservices. Although the machine learning and systems research communities have proposed various techniques to tackle this problem, there is currently a lack of standardized datasets for quantitative benchmarking. Consequently, research groups are compelled to create their own datasets for experimentation. This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system. We showcase how this dataset can be used to evaluate the accuracy of a variety of methods spanning different causal and non-causal characterisations of the root cause analysis problem. We hope the new dataset, available at https://github.com/amazon-science/petshop-root-cause-analysis/ enables further development of techniques in this important area., Comment: 22 pages, 6 figures, 10 tables, for associated git repo see https://github.com/amazon-science/petshop-root-cause-analysis/, to be published in Proceedings of Machine Learning Research vol 236, 2024, 3rd Conference on Causal Learning and Reasoning

Ownership and borrowing systems, designed to enforce safe memory management without the need for garbage collection, have been brought to the fore by the Rust programming language. Rust also aims to bring some guarantees offered by functional programming into the realm of performant systems code, but the type system is largely separate from the ownership model, with type and borrow checking happening in separate compilation phases. Recent models such as RustBelt and Oxide aim to formalise Rust in depth, but there is less focus on integrating the basic ideas into more traditional type systems. An approach designed to expose an essential core for ownership and borrowing would open the door for functional languages to borrow concepts found in Rust and other ownership frameworks, so that more programmers can enjoy their benefits. One strategy for managing memory in a functional setting is through uniqueness types, but these offer a coarse-grained view: either a value has exactly one reference, and can be mutated safely, or it cannot, since other references may exist. Recent work demonstrates that linear and uniqueness types can be combined in a single system to offer restrictions on program behaviour and guarantees about memory usage. We develop this connection further, showing that just as graded type systems like those of Granule and Idris generalise linearity, Rust's ownership model arises as a graded generalisation of uniqueness. We combine fractional permissions with grading to give the first account of ownership and borrowing that smoothly integrates into a standard type system alongside linearity and graded types, and extend Granule accordingly with these ideas., Comment: 27 pages + references. In submission (minor revision)

Spiking neural networks (SNNs) promise ultra-low-power applications by exploiting temporal and spatial sparsity. The number of binary activations, called spikes, is proportional to the power consumed when executed on neuromorphic hardware. Training such SNNs using backpropagation through time for vision tasks that rely mainly on spatial features is computationally costly. Training a stateless artificial neural network (ANN) to then convert the weights to an SNN is a straightforward alternative when it comes to image recognition datasets. Most conversion methods rely on rate coding in the SNN to represent ANN activation, which uses enormous amounts of spikes and, therefore, energy to encode information. Recently, temporal conversion methods have shown promising results requiring significantly fewer spikes per neuron, but sometimes complex neuron models. We propose a temporal ANN-to-SNN conversion method, which we call Quartz, that is based on the time to first spike (TTFS). Quartz achieves high classification accuracy and can be easily implemented on neuromorphic hardware while using the least amount of synaptic operations and memory accesses. It incurs a cost of two additional synapses per neuron compared to previous temporal conversion methods, which are readily available on neuromorphic hardware. We benchmark Quartz on MNIST, CIFAR10, and ImageNet in simulation to show the benefits of our method and follow up with an implementation on Loihi, a neuromorphic chip by Intel. We provide evidence that temporal coding has advantages in terms of power consumption, throughput, and latency for similar classification accuracy. Our code and models are publicly available.

Graded type systems, such as the one underlying the Granule programming language, allow various different properties of a program's behaviour to be tracked via annotating types with additional information, which we call grades. One example of such a property, often used as a case study in prior work on graded types, is information flow control, in which types are graded by a lattice of security levels allowing noninterference properties to be automatically verified and enforced. These typically focus on one particular aspect of security, however, known as confidentiality; public outputs are prohibited from depending on private inputs. Integrity, a property specifying that trusted outputs must not depend on untrusted inputs, has not been examined in this context. This short paper aims to remedy this omission. It is well-known that confidentiality and integrity are in some sense dual properties, but simply reversing the ordering of the security lattice turns out to be unsatisfactory for the purpose of combining both kinds of property in a single system, at least in our setting. We analogize the situation to recent work on embedding both linear and uniqueness types in a graded framework, and use this framing to demonstrate that we can enforce both integrity and confidentiality alongside one another. The main idea is to add an additional flavour of modality annotated for integrity, such that the existing graded comonad for tracking confidentiality now also acts as a relative monad over the new modality, with rules allowing information to flow from trusted to public to private., Comment: 3 pages. Originally presented as a short paper at PLAS 2022

Brand and Zafiropulo's notion of Communicating Finite-State Machines (CFSMs) provides a succinct and powerful model of message-passing concurrency, based around channels. However, a major variant of message-passing concurrency is not readily captured by CFSMs: the actor model. In this work, we define a variant of CFSMs, called Communicating Actor Automata, to capture the actor model of concurrency as provided by Erlang: with mailboxes, from which messages are received according to repeated application of pattern matching. Furthermore, this variant of CFSMs supports dynamic process topologies, capturing common programming idioms in the context of actor-based message-passing concurrency. This gives a new basis for modelling, specifying, and verifying Erlang programs. We also consider a class of CAAs that give rise to freedom from race conditions., Comment: In Proceedings PLACES 2023, arXiv:2304.05439

This paper draws correlations between several challenges and opportunities within the area of team sports analytics and key research areas within multiagent systems (MAS). We specifically consider invasion games, defined as sports where players invade the opposing team's territory and can interact anywhere on a playing surface such as ice hockey, soccer, and basketball. We argue that MAS is well-equipped to study invasion games and will benefit both MAS and sports analytics fields. Our discussion highlights areas for MAS implementation and further development along two axes: short-term in-game strategy (coaching) and long-term team planning (management)., Comment: 5 pages, 1 figure, In Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)

A critical enabler for progress in neuromorphic computing research is the ability to transparently evaluate different neuromorphic solutions on important tasks and to compare them to state-of-the-art conventional solutions. The Intel Neuromorphic Deep Noise Suppression Challenge (Intel N-DNS Challenge), inspired by the Microsoft DNS Challenge, tackles a ubiquitous and commercially relevant task: real-time audio denoising. Audio denoising is likely to reap the benefits of neuromorphic computing due to its low-bandwidth, temporal nature and its relevance for low-power devices. The Intel N-DNS Challenge consists of two tracks: a simulation-based algorithmic track to encourage algorithmic innovation, and a neuromorphic hardware (Loihi 2) track to rigorously evaluate solutions. For both tracks, we specify an evaluation methodology based on energy, latency, and resource consumption in addition to output audio quality. We make the Intel N-DNS Challenge dataset scripts and evaluation code freely accessible, encourage community participation with monetary prizes, and release a neuromorphic baseline solution which shows promising audio quality, high power efficiency, and low resource consumption when compared to Microsoft NsNet2 and a proprietary Intel denoising model used in production. We hope the Intel N-DNS Challenge will hasten innovation in neuromorphic algorithms research, especially in the area of training tools and methods for real-time signal processing. We expect the winners of the challenge will demonstrate that for problems like audio denoising, significant gains in power and resources can be realized on neuromorphic devices available today compared to conventional state-of-the-art solutions., Comment: 13 pages, 4 figures, 1 table

Vector Symbolic Architectures (VSAs) are a powerful framework for representing compositional reasoning. They lend themselves to neural-network implementations, allowing us to create neural networks that can perform cognitive functions, like spatial reasoning, arithmetic, symbol binding, and logic. But the vectors involved can be quite large, hence the alternative label Hyperdimensional (HD) computing. Advances in neuromorphic hardware hold the promise of reducing the running time and energy footprint of neural networks by orders of magnitude. In this paper, we extend some pioneering work to run VSA algorithms on a substrate of spiking neurons that could be run efficiently on neuromorphic hardware.

The optimal instant of observation of astrophysical phenomena for objects that vary on human time-sales is an important problem, as it bears on the cost-effective use of usually scarce observational facilities. In this paper we address this problem for the case of tight visual binary systems through a Bayesian framework based on the maximum entropy sampling principle. Our proposed information-driven methodology exploits the periodic structure of binary systems to provide a computationally efficient estimation of the probability distribution of the optimal observation time. We show the optimality of the proposed sampling methodology in the Bayes sense and its effectiveness through direct numerical experiments. We successfully apply our scheme to the study of two visual-spectroscopic binaries, and one purely astrometric triple hierarchical system. We note that our methodology can be applied to any time-evolving phenomena, a particularly interesting application in the era of dedicated surveys, where a definition of the cadence of observations can have a crucial impact on achieving the science goals., Comment: Accepted for publication to PASP. 23 pages, 2 Tables, 9 Figures, 2 Appendices

Quantum gravity in AdS$_7 \times$S$^4$ is dual to a 6d superconformal field theory, known as the 6d $(2,0)$ theory, which is very challenging to describe because it lacks a conventional Lagrangian description. On the other hand, certain null reductions of the 6d $(2,0)$ theory give rise to 5d Lagrangian theories with $SU(1,3)$ spacetime symmetry, $SO(5)$ R-symmetry, and 24 supercharges. This appears to be closely related to the superconformal group of a 3d superconformal Chern-Simons theory known as the ABJM theory, which is believed to be integrable in the planar limit, if one exchanges the role of conformal and R-symmetry. In this note, we construct a representation of the 5d superconformal group using 6d supertwistors and show that it admits an infinite dimensional extension known as Yangian symmetry, which opens up the possiblity that these 5d theories are exactly solvable in the planar limit., Comment: v3. published in PRL