Search

We study hydrodynamic limits of the cluster coagulation model; a coagulation model introduced by Norris [$\textit{Comm. Math. Phys.}$, 209(2):407-435 (2000)]. In this process, pairs of particles $x,y$ in a measure space $E$, merge to form a single new particle $z$ according to a transition kernel $K(x, y, \mathrm{d} z)$, in such a manner that a quantity, one may regard as the total mass of the system, is conserved. This model is general enough to incorporate various inhomogeneities in the evolution of clusters, for example, their shape, or their location in space. We derive sufficient criteria for trajectories associated with this process to concentrate among solutions of a generalisation of the $\textit{Flory equation}$, and, in some special cases, by means of a uniqueness result for solutions of this equation, prove a weak law of large numbers. This multi-type Flory equation is associated with $\textit{conserved quantities}$ associated with the process, which may encode different information to conservation of mass (for example, conservation of centre of mass in spatial models). We also apply criteria for gelation in this process to derive sufficient criteria for this equation to exhibit gelling solutions. When this occurs, this multi-type Flory equation encodes, via the associated conserved property, the interaction between the gel and the finite size sol particles., Comment: 27 pages; some text overlap with arXiv:2308.10232v1, as this version has been separated into two parts

Transfer learning has gained significant attention in recent deep learning research due to its ability to accelerate convergence and enhance performance on new tasks. However, its success is often contingent on the similarity between source and target data, and training on numerous datasets can be costly, leading to blind selection of pretrained models with limited insight into their effectiveness. To address these challenges, we introduce D2NWG, a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning, conditioned on the target dataset. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation, learning the weight distributions of models pretrained on various datasets. This allows for automatic generation of weights that generalize well across both seen and unseen tasks, outperforming state-of-the-art meta-learning methods and pretrained models. Moreover, our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques that rely on task-specific model collections or access to original training data. By modeling the parameter distribution of LLMs, D2NWG enables task-specific parameter generation without requiring additional fine-tuning or large collections of model variants. Extensive experiments show that our method consistently enhances the performance of diverse base models, regardless of their size or complexity, positioning it as a robust solution for scalable transfer learning., Comment: 32 pages

We study a spatial Markovian particle system with pairwise coagulation, a spatial version of the Marcus--Lushnikov process: according to a coagulation kernel $K$, particle pairs merge into a single particle, and their masses are united. We introduce a statistical-mechanics approach to the study of this process. We derive an explicit formula for the empirical process of the particle configuration at a given fixed time $T$ in terms of a reference Poisson point process, whose points are trajectories that coagulate into one particle by time $T$. The non-coagulation between any two of them induces an exponential pair-interaction, which turns the description into a many-body system with a Gibbsian pair-interaction. Based on this, we first give a large-deviation principle for the joint distribution of the particle histories (conditioning on an upper bound for particle sizes), in the limit as the number $N$ of initial atoms diverges and the kernel scales as $\frac 1N K$. We characterise the minimiser(s) of the rate function, we give criteria for its uniqueness and prove a law of large numbers (unconditioned). Furthermore, we use the unique minimiser to construct a solution of the Smoluchowski equation and give a criterion for the occurrence of a gelation phase transition., Comment: 60 pages, 1 figure

We study the near-critical behavior of the sparse Erd\H{o}s-R\'enyi random graph $\mathcal{G}(n,p)$ on $n\gg1$ vertices, where the connection probability $p$ satisfies $np = 1+\theta(b_n^2/n)^{1/3}$, with $n^{3/10}\ll {b_n}\ll n^{1/2}$, and $\theta\in\mathbb{R}$. To this end, we introduce an empirical measure that describes connected components of $\mathcal{G}(n,p)$ of mesoscopic size $\propto (nb_n)^{2/3}$, and we characterize its large deviation behavior. The proof hinges on detailed combinatorial estimates and optimization procedures. In particular, we give precise estimates for the probability that the graph has no connected component of mesoscopic size or larger. We argue that these are a stepping stone for the analysis of more general inhomogeneous random graphs. Our proof strategy gives new and accurate estimates of the probability that the sparse Erd\H{o}s-R\'enyi graph is connected.

The surge of deep-space probes makes it unsustainable to navigate them with standard radiometric tracking. Self-driving interplanetary satellites represent a solution to this problem. In this work, a full vision-based navigation algorithm is built by combining an orbit determination method with an image processing pipeline suitable for interplanetary transfers of autonomous platforms. To increase the computational efficiency of the algorithm, a non-dimensional extended Kalman filter is selected as state estimator, fed by the positions of the planets extracted from deep-space images. An enhancement of the estimation accuracy is performed by applying an optimal strategy to select the best pair of planets to track. Moreover, a novel analytical measurement model for deep-space navigation is developed providing a first-order approximation of the light-aberration and light-time effects. Algorithm performance is tested on a high-fidelity, Earth--Mars interplanetary transfer, showing the algorithm applicability for deep-space navigation.

We consider the problem of gelation in the cluster coagulation model introduced by Norris [$\textit{Comm. Math. Phys.}$, 209(2):407-435 (2000)], where pairs of clusters of types $(x,y)$ taking values in a measure space $E$, merge to form a new particle of type $z\in E$ according to a transition kernel $K(x,y, \mathrm{d} z)$.This model possesses enough generality to accommodate inhomogenieties in the evolution of clusters, including variations in their shape or spatial distribution. We derive general, sufficient criteria for stochastic gelation and strong gelation in this model. As particular cases, we extend results related to the classical Marcus--Lushnikov coagulation process, showing that reasonable `homogenous' coagulation processes with exponent $\gamma>1$ yield gelation; and also, coagulation processes with kernel $\bar{K}(m,n)~\geq~(m \wedge n) \log{(m \wedge n)}^{3 +\epsilon}$ for $\epsilon>0$. Finally, we use coupling arguments with inhomogeneous random graphs to deduce sufficient criterion for strong gelation (the emergence of a particle of size $O(N)$)., Comment: 24 pages; this paper is a shortened version of arXiv:2308.10232v1, since version 1 will be split into two parts

We propose a neural network weight encoding method for network property prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network parameters. Our approach is capable of encoding neural networks in a model zoo of mixed architecture and different parameter sizes as opposed to previous approaches that require custom encoding models for different architectures. Furthermore, our \textbf{S}et-based \textbf{N}eural network \textbf{E}ncoder (SNE) takes into consideration the hierarchical computational structure of neural networks. To respect symmetries inherent in network weight space, we utilize Logit Invariance to learn the required minimal invariance properties. Additionally, we introduce a \textit{pad-chunk-encode} pipeline to efficiently encode neural network layers that is adjustable to computational and memory constraints. We also introduce two new tasks for neural network property prediction: cross-dataset and cross-architecture. In cross-dataset property prediction, we evaluate how well property predictors generalize across model zoos trained on different datasets but of the same architecture. In cross-architecture property prediction, we evaluate how well property predictors transfer to model zoos of different architecture not seen during training. We show that SNE outperforms the relevant baselines on standard benchmarks., Comment: 23 pages

A new era of space exploration and exploitation is fast approaching. A multitude of spacecraft will flow in the future decades under the propulsive momentum of the new space economy. Yet, the flourishing proliferation of deep-space assets will make it unsustainable to pilot them from ground with standard radiometric tracking. The adoption of autonomous navigation alternatives is crucial to overcoming these limitations. Among these, optical navigation is an affordable and fully ground-independent approach. Probes can triangulate their position by observing visible beacons, e.g., planets or asteroids, by acquiring their line-of-sight in deep space. To do so, developing efficient and robust image processing algorithms providing information to navigation filters is a necessary action. This paper proposes an innovative pipeline for unresolved beacon recognition and line-of-sight extraction from images for autonomous interplanetary navigation. The developed algorithm exploits the k-vector method for the non-stellar object identification and statistical likelihood to detect whether any beacon projection is visible in the image. Statistical results show that the accuracy in detecting the planet position projection is independent of the spacecraft position uncertainty. Whereas, the planet detection success rate is higher than 95% when the spacecraft position is known with a 3sigma accuracy up to 10^5 km., Comment: 26 pages, 7 figures

We investigate the emergence of a collective periodic behavior in a frustrated network of interacting diffusions. Particles are divided into two communities depending on their mutual couplings. On the one hand, both intra-population interactions are positive; each particle wants to conform to the average position of the particles in its own community. On the other hand, inter-population interactions have different signs: the particles of one population want to conform to the average position of the particles of the other community, while the particles in the latter want to do the opposite. We show that this system features the phenomenon of noise-induced periodicity: in the infinite volume limit, in a certain range of interaction strengths, although the system has no periodic behavior in the zero-noise limit, a moderate amount of noise may generate an attractive periodic law., Comment: 32 pages, 8 figures

Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaranteeing the same output for all partitions. However, existing constraints on MBC architectures lead to models with limited expressive power. Additionally, prior work has not addressed how to deal with large sets during training when the full set gradient is required. To address these issues, we propose a Universally MBC (UMBC) class of set functions which can be used in conjunction with arbitrary non-MBC components while still satisfying MBC, enabling a wider range of function classes to be used in MBC settings. Furthermore, we propose an efficient MBC training algorithm which gives an unbiased approximation of the full set gradient and has a constant memory overhead for any set size for both train- and test-time. We conduct extensive experiments including image completion, text classification, unsupervised clustering, and cancer detection on high-resolution images to verify the efficiency and efficacy of our scalable set encoding framework. Our code is available at github.com/jeffwillette/umbc, Comment: ICML 2023

Real-time video segmentation is a crucial task for many real-world applications such as autonomous driving and robot control. Since state-of-the-art semantic segmentation models are often too heavy for real-time applications despite their impressive performance, researchers have proposed lightweight architectures with speed-accuracy trade-offs, achieving real-time speed at the expense of reduced accuracy. In this paper, we propose a novel framework to speed up any architecture with skip-connections for real-time vision tasks by exploiting the temporal locality in videos. Specifically, at the arrival of each frame, we transform the features from the previous frame to reuse them at specific spatial bins. We then perform partial computation of the backbone network on the regions of the current frame that captures temporal differences between the current and previous frame. This is done by dynamically dropping out residual blocks using a gating mechanism which decides which blocks to drop based on inter-frame distortion. We validate our Spatial-Temporal Mask Generator (STMG) on video semantic segmentation benchmarks with multiple backbone networks, and show that our method largely speeds up inference with minimal loss of accuracy.

Meta-learning approaches enable machine learning systems to adapt to new tasks given few examples by leveraging knowledge from related tasks. However, a large number of meta-training tasks are still required for generalization to unseen tasks during meta-testing, which introduces a critical bottleneck for real-world problems that come with only few tasks, due to various reasons including the difficulty and cost of constructing tasks. Recently, several task augmentation methods have been proposed to tackle this issue using domain-specific knowledge to design augmentation techniques to densify the meta-training task distribution. However, such reliance on domain-specific knowledge renders these methods inapplicable to other domains. While Manifold Mixup based task augmentation methods are domain-agnostic, we empirically find them ineffective on non-image domains. To tackle these limitations, we propose a novel domain-agnostic task augmentation method, Meta-Interpolation, which utilizes expressive neural set functions to densify the meta-training task distribution using bilevel optimization. We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains such as image classification, molecule property prediction, text classification and speech recognition. Experimentally, we show that Meta-Interpolation consistently outperforms all the relevant baselines. Theoretically, we prove that task interpolation with the set function regularizes the meta-learner to improve generalization., Comment: First two authors contributed equally. Name order decided by a coin toss

We study an inhomogeneous sparse random graph on [N] = {1, . . . , N } as introduced in a seminal paper by Bollobas, Janson and Riordan (2007): vertices have a type (here in a compact metric space S), and edges between different vertices occur randomly and independently over all vertex pairs, with a probability depending on the two vertex types. In the limit N to infinity, we consider the sparse regime, where the average degree is O(1). We prove a large-deviations principle with explicit rate function for the statistics of the collection of all the connected components, registered according to their vertex type sets, and distinguished according to being microscopic (of finite size) or macroscopic (of size proportional to N). In doing so, we derive explicit logarithmic asymptotics for the probability that GN is connected. We present a full analysis of the rate function including its minimizers. From this analysis we deduce a number of limit laws, conditional and unconditional, which provide comprehensive information about all the microscopic and macroscopic components of the graph. In particular, we recover the criterion for the existence of the phase transition given in [5].

Most existing set encoding algorithms operate under the implicit assumption that all the set elements are accessible, and that there are ample computational and memory resources to load the set into memory during training and inference. However, both assumptions fail when the set is excessively large such that it is impossible to load all set elements into memory, or when data arrives in a stream. To tackle such practical challenges in large-scale set encoding, the general set-function constraints of permutation invariance and equivariance are not sufficient. We introduce a new property termed Mini-Batch Consistency (MBC) that is required for large scale mini-batch set encoding. Additionally, we present a scalable and efficient attention-based set encoding mechanism that is amenable to mini-batch processing of sets, and capable of updating set representations as data arrives. The proposed method adheres to the required symmetries of invariance and equivariance as well as maintaining MBC for any partition of the input set. We perform extensive experiments and show that our method is computationally efficient and results in rich set encoding representations for set-structured data., Comment: 16 pages