Search

The Mixture-of-Experts (MoE) architecture enables a significant increase in the total number of model parameters with minimal computational overhead. However, it is not clear what performance tradeoffs, if any, exist between MoEs and standard dense transformers. In this paper, we show that as we increase the number of experts (while fixing the number of active parameters), the memorization performance consistently increases while the reasoning capabilities saturate. We begin by analyzing the theoretical limitations of MoEs at reasoning. We prove that there exist graph problems that cannot be solved by any number of experts of a certain width; however, the same task can be easily solved by a dense model with a slightly larger width. On the other hand, we find that on memory-intensive tasks, MoEs can effectively leverage a small number of active parameters with a large number of experts to memorize the data. We empirically validate these findings on synthetic graph problems and memory-intensive closed book retrieval tasks. Lastly, we pre-train a series of MoEs and dense transformers and evaluate them on commonly used benchmarks in math and natural language. We find that increasing the number of experts helps solve knowledge-intensive tasks, but fails to yield the same benefits for reasoning tasks.

It is known that a properly rescaled version of Sinai's random walk converges in distribution to Brox's diffusion. In this article we quantify this convergence by considering a specific coupling between Sinai's walk and Brox's diffusion. Our method relies on convergence results for martingale problems considered in the rough path setting.

We give a uniform estimate and an inequality for solutions of an equation with Dirichlet boundary condition.

Wearable sensors have become ubiquitous thanks to a variety of health tracking features. The resulting continuous and longitudinal measurements from everyday life generate large volumes of data; however, making sense of these observations for scientific and actionable insights is non-trivial. Inspired by the empirical success of generative modeling, where large neural networks learn powerful representations from vast amounts of text, image, video, or audio data, we investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM, a multimodal foundation model built on the largest wearable-signals dataset with the most extensive range of sensor modalities to date. Our results establish the scaling laws of LSM for tasks such as imputation, interpolation and extrapolation, both across time and sensor modalities. Moreover, we highlight how LSM enables sample-efficient downstream learning for tasks like exercise and activity recognition.

Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally averages LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks., Comment: 9 pages plus references and appendices

Modern vision models have achieved remarkable success in benchmarks where local features provide critical information about the target. There is now a growing interest in solving tasks that require more global reasoning, where local features offer no significant information. These tasks are reminiscent of the connectivity tasks discussed by Minsky and Papert in 1969, which exposed the limitations of the perceptron model and contributed to the first AI winter. In this paper, we revisit such tasks by introducing four global visual benchmarks involving path findings and mazes. We show that: (1) although today's large vision models largely surpass the expressivity limitations of the early models, they still struggle with the learning efficiency; we put forward the "globality degree" notion to understand this limitation; (2) we then demonstrate that the picture changes and global reasoning becomes feasible with the introduction of "visual scratchpads"; similarly to the text scratchpads and chain-of-thoughts used in language models, visual scratchpads help break down global tasks into simpler ones; (3) we finally show that some scratchpads are better than others, in particular, "inductive scratchpads" that take steps relying on less information afford better out-of-distribution generalization and succeed for smaller model sizes.

Implicit networks are a class of neural networks whose outputs are defined by the fixed point of a parameterized operator. They have enjoyed success in many applications including natural language processing, image processing, and numerous other applications. While they have found abundant empirical success, theoretical work on its generalization is still under-explored. In this work, we consider a large family of implicit networks defined parameterized contractive fixed point operators. We show a generalization bound for this class based on a covering number argument for the Rademacher complexity of these architectures.

Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions. While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities have genuinely advanced, raising questions about the reliability of the reported metrics. To address these concerns, we conduct a large-scale study on several SOTA open and closed models. To overcome the limitations of existing evaluations, we introduce GSM-Symbolic, an improved benchmark created from symbolic templates that allow for the generation of a diverse set of questions. GSM-Symbolic enables more controllable evaluations, providing key insights and more reliable metrics for measuring the reasoning capabilities of models.Our findings reveal that LLMs exhibit noticeable variance when responding to different instantiations of the same question. Specifically, the performance of all models declines when only the numerical values in the question are altered in the GSM-Symbolic benchmark. Furthermore, we investigate the fragility of mathematical reasoning in these models and show that their performance significantly deteriorates as the number of clauses in a question increases. We hypothesize that this decline is because current LLMs cannot perform genuine logical reasoning; they replicate reasoning steps from their training data. Adding a single clause that seems relevant to the question causes significant performance drops (up to 65%) across all state-of-the-art models, even though the clause doesn't contribute to the reasoning chain needed for the final answer. Overall, our work offers a more nuanced understanding of LLMs' capabilities and limitations in mathematical reasoning., Comment: preprint

Recent work has suggested that certain neural network architectures-particularly recurrent neural networks (RNNs) and implicit neural networks (INNs) are capable of logical extrapolation. That is, one may train such a network on easy instances of a specific task and then apply it successfully to more difficult instances of the same task. In this paper, we revisit this idea and show that (i) The capacity for extrapolation is less robust than previously suggested. Specifically, in the context of a maze-solving task, we show that while INNs (and some RNNs) are capable of generalizing to larger maze instances, they fail to generalize along axes of difficulty other than maze size. (ii) Models that are explicitly trained to converge to a fixed point (e.g. the INN we test) are likely to do so when extrapolating, while models that are not (e.g. the RNN we test) may exhibit more exotic limiting behaviour such as limit cycles, even when they correctly solve the problem. Our results suggest that (i) further study into why such networks extrapolate easily along certain axes of difficulty yet struggle with others is necessary, and (ii) analyzing the dynamics of extrapolation may yield insights into designing more efficient and interpretable logical extrapolators.

We introduce an algorithm for the uniform generation of infinite traces, i.e., infinite words up to commutation of some letters. The algorithm outputs on-the-fly approximations of a theoretical infinite trace, the latter being distributed according to the exact uniform probability measure. The average size of the approximation grows linearly with the time of execution of the algorithm, hence its output can be effectively used while running. Two versions of the algorithm are given. A version without rejection has a good production speed, provided that some precomputations have been done, but these may be costly. A version with rejection requires much fewer computations, at the expense of a production speed that can be small. We also show that, for some particular trace monoids, one or the other version of the algorithm can actually be very good: few computations for a good production speed., Comment: 21 pages

We study the convergence of distributions on finite paths of weighted digraphs, namely the family of Boltzmann distributions and the sequence of uniform distributions. Targeting applications to the convergence of distributions on paths, we revisit some known results from reducible nonnegative matrix theory and obtain new ones, with a systematic use of tools from analytic combinatorics. In several fields of mathematics, computer science and system theory, including concurreny theory, one frequently faces non strongly connected weighted digraphs encoding the elements of combinatorial structures of interest; this motivates our study., Comment: 24 pages, proofs at the end

Control Barrier Functions (CBFs) are an effective methodology to ensure safety and performative efficacy in real-time control applications such as power systems, resource allocation, autonomous vehicles, robotics, etc. This approach ensures safety independently of the high-level tasks that may have been pre-planned offline. For example, CBFs can be used to guarantee that a vehicle will remain in its lane. However, when the number of agents is large, computation of CBFs can suffer from the curse of dimensionality in the multi-agent setting. In this work, we present Mean-field Control Barrier Functions (MF-CBFs), which extends the CBF framework to the mean-field (or swarm control) setting. The core idea is to model a population of agents as probability measures in the state space and build corresponding control barrier functions. Similar to traditional CBFs, we derive safety constraints on the (distributed) controls but now relying on the differential calculus in the space of probability measures.

The rolling tachyon is a non-canonical scalar field model well motivated in string theory which naturally predicts variations of the fine-structure constant. Such variations can in principle lead to interesting observable consequences, but they can also lead to extremely tight constraints on these kinds of models. In this work we subject the rolling tachyon model evolving in a variety of potentials to current data and show that most cosmologically interesting evolutions are already strongly excluded. We find $|1+w_0| < 10^{-3}$ from cosmological data and $|1+w_0| < 10^{-9}$ from fine-structure data, leaving the rolling tachyon to either play a role almost entirely equivalent to a cosmological constant or that of a test field. We also find that in most of the allowed parameter space the field evolves very slowly, allowing its evolution to be approximated as an equivalent canonical scalar field., Comment: 30 pages, 18 figures, 1 table

In this study, we carried out equilibrium molecular dynamics (EMD) simulations of the liquid-liquid interface between two different Lennard-Jones components with varying miscibility, where we examined the relation between the interfacial tension and isolation free energy using both a mechanical and thermodynamic approach. Using the mechanical approach, we obtained a stress distribution around a quasi-one-dimensional (1D) EMD systems with a flat LL interface. From the stress distribution, we calculated the liquid-liquid interfacial tension based on Bakker's equation, which uses the stress anisotropy around the interface, and measures how it varies with miscibility. The second approach uses thermodynamic integration by enforcing quasi-static isolation of the two liquids to calculate the free energy. This uses the same EMD systems as the mechanical approach, with both extended dry-surface and phantom-wall (PW) schemes applied. When the two components were immiscible, the interfacial tension and isolation free energy were in good agreement, provided all kinetic and interaction contributions were included in the stress. When the components were miscible, the values were significantly different. From the result of PW for the case of completely mixed liquids, the difference was attributed to the additional free energy required to separate the binary mixture into single components against the osmotic pressure prior to the complete detachment of the two components, i.e., the free energy of mixing., Comment: 5 figures

Hand-Object Interaction (HOI) is gaining significant attention, particularly with the creation of numerous egocentric datasets driven by AR/VR applications. However, third-person view HOI has received less attention, especially in terms of datasets. Most third-person view datasets are curated for action recognition tasks and feature pre-segmented clips of high-level daily activities, leaving a gap for in-the-wild datasets. To address this gap, we propose ChildPlay-Hand, a novel dataset that includes person and object bounding boxes, as well as manipulation actions. ChildPlay-Hand is unique in: (1) providing per-hand annotations; (2) featuring videos in uncontrolled settings with natural interactions, involving both adults and children; (3) including gaze labels from the ChildPlay-Gaze dataset for joint modeling of manipulations and gaze. The manipulation actions cover the main stages of an HOI cycle, such as grasping, holding or operating, and different types of releasing. To illustrate the interest of the dataset, we study two tasks: object in hand detection (OiH), i.e. if a person has an object in their hand, and manipulation stages (ManiS), which is more fine-grained and targets the main stages of manipulation. We benchmark various spatio-temporal and segmentation networks, exploring body vs. hand-region information and comparing pose and RGB modalities. Our findings suggest that ChildPlay-Hand is a challenging new benchmark for modeling HOI in the wild.

We introduce a hybrid filter that incorporates a mathematically accurate moment-based filter with a data driven filter for discontinuous Galerkin approximations to PDE solutions that contain discontinuities. Numerical solutions of PDEs suffer from an $\mathcal{O}(1)$ error in the neighborhood about discontinuities, especially for shock waves that arise in inviscid compressible flow problems. While post-processing filters, such as the Smoothness-Increasing Accuracy-Conserving (SIAC) filter, can improve the order of error in smooth regions, the $\mathcal{O}(1)$ error in the vicinity of a discontinuity remains. To alleviate this, we combine the SIAC filter with a data-driven filter based on Convolutional Neural Networks. The data-driven filter is specifically focused on improving the errors in discontinuous regions and therefore {\it only includes top-hat functions in the training dataset.} For both filters, a consistency constraint is enforced, while the SIAC filter additionally satisfies $r-$moments. This hybrid filter approach allows for maintaining the accuracy guaranteed by the theory in smooth regions while the hybrid SIAC-data-driven approach reduces the $\ell_2$ and $\ell_\infty$ errors about discontinuities. Thus, overall the global errors are reduced. We examine the performance of the hybrid filter about discontinuities for the one-dimensional Euler equations for the Lax, Sod, and sine-entropy (Shu-Osher) shock-tube problems.

Ptychography is a popular imaging technique that combines diffractive imaging with scanning microscopy. The technique consists of a coherent beam that is scanned across an object in a series of overlapping positions, leading to reliable and improved reconstructions. Ptychographic microscopes allow for large fields to be imaged at high resolution at additional computational expense. In this work, we explore the use of the fast Partial Fourier Transforms (PFTs), which efficiently compute Fourier coefficients corresponding to low frequencies. The core idea is to use the PFT in a plug-and-play manner to warm-start existing ptychography algorithms such as the ptychographic iterative engine (PIE). This approach reduces the computational budget required to solve the ptychography problem. Our numerical results show that our scheme accelerates the convergence of traditional solvers without sacrificing quality of reconstruction.

The ultra high-energy (UHE) Galactic diffuse gamma-ray emission holds important information on the propagation of cosmic rays in the Galaxy. However, its measurements suffer from a contamination from unresolved sources whose contribution remains unclear. In this Letter, we propose a novel data-driven estimate of the contribution of unresolved pulsar wind nebulae and TeV halos based on the information present in the ATNF and the LHAASO catalogs. We find that in the inner Galaxy, this contribution is limited to $\sim38\%\pm10\%$ of the diffuse flux measured by LHAASO at $\sim20\,\rm{TeV}$ in the case where all sources associated to pulsars contribute as unresolved sources, and this fraction drops with energy to less than $21\%\pm6\%$ above $100\,\rm{TeV}$. In the outer Galaxy, this contribution is always subdominant. In particular, it reaches at most $\sim18\%\pm2\%$ at $10\,\rm{TeV}$ and is less than $\sim7\%\pm1\%$ above $\sim25\,\rm{TeV}$. We conclude that the UHE Galactic diffuse gamma-ray emission cannot be dominated by unresolved pulsar sources above a few tens of $\rm{TeV}$., Comment: Published in ApJL, 8 pages, 4 figures

Thermal management at silicon-diamond interface is critical for advancing high-performance electronic and optoelectronic devices. In this study, we calculate the interfacial thermal conductance between silicon and diamond using machine learning (ML) interatomic potentials trained on density functional theory (DFT) data. Using non-equilibrium molecular dynamics (NEMD) simulations, we compute the interfacial thermal conductance (ITC) for various system sizes. Our results show a closer agreement with experimental data than those obtained using traditional semi-empirical potentials such as Tersoff and Brenner which overestimate ITC by a factor of about 3. In addition, we analyze the frequency-dependent heat transfer spectrum, providing insights into the contributions of different phonon modes to the interfacial thermal conductance. The ML potential accurately captures the phonon dispersion relations and lifetimes, in good agreement with DFT calculations and experimental observations. It is shown that the Tersoff potential predicts higher phonon group velocities and phonon lifetimes compared to the DFT results. Furthermore, it predicts higher interfacial bonding strength, which is consistent with higher interfacial thermal conductance as compared to the ML potential. This study highlights the use of the ML interatomic potential to improve the accuracy and computational efficiency of thermal transport simulations in complex material systems.

Commercial large language models (LLMs), like OpenAI's GPT-4 powering ChatGPT and Anthropic's Claude 3 Opus, have dominated natural language processing (NLP) benchmarks across different domains. New competing Open-Source alternatives like Mixtral 8x7B or Llama 3 have emerged and seem to be closing the gap while often offering higher throughput and being less costly to use. Open-Source LLMs can also be self-hosted, which makes them interesting for enterprise and clinical use cases where sensitive data should not be processed by third parties. We participated in the 12th BioASQ challenge, which is a retrieval augmented generation (RAG) setting, and explored the performance of current GPT models Claude 3 Opus, GPT-3.5-turbo and Mixtral 8x7b with in-context learning (zero-shot, few-shot) and QLoRa fine-tuning. We also explored how additional relevant knowledge from Wikipedia added to the context-window of the LLM might improve their performance. Mixtral 8x7b was competitive in the 10-shot setting, both with and without fine-tuning, but failed to produce usable results in the zero-shot setting. QLoRa fine-tuning and Wikipedia context did not lead to measurable performance gains. Our results indicate that the performance gap between commercial and open-source models in RAG setups exists mainly in the zero-shot setting and can be closed by simply collecting few-shot examples for domain-specific use cases. The code needed to rerun these experiments is available through GitHub., Comment: Version as accepted at the BioASQ Lab at CLEF 2024

Under some conditions we give a blow-up analysis for solutions of an equation with Dirichlet boundary condition.

Enterprise chatbots, powered by generative AI, are emerging as key applications to enhance employee productivity. Retrieval Augmented Generation (RAG), Large Language Models (LLMs), and orchestration frameworks like Langchain and Llamaindex are crucial for building these chatbots. However, creating effective enterprise chatbots is challenging and requires meticulous RAG pipeline engineering. This includes fine-tuning embeddings and LLMs, extracting documents from vector databases, rephrasing queries, reranking results, designing prompts, honoring document access controls, providing concise responses, including references, safeguarding personal information, and building orchestration agents. We present a framework for building RAG-based chatbots based on our experience with three NVIDIA chatbots: for IT/HR benefits, financial earnings, and general content. Our contributions are three-fold: introducing the FACTS framework (Freshness, Architectures, Cost, Testing, Security), presenting fifteen RAG pipeline control points, and providing empirical results on accuracy-latency tradeoffs between large and small LLMs. To the best of our knowledge, this is the first paper of its kind that provides a holistic view of the factors as well as solutions for building secure enterprise-grade chatbots.", Comment: 8 pages, 6 figures, 2 tables, Preprint submission to ACM CIKM 2024

A framework with sets of attacking arguments (SETAF) is an extension of the well-known Dung's Abstract Argumentation Frameworks (AAFs) that allows joint attacks on arguments. In this paper, we provide a translation from Normal Logic Programs (NLPs) to SETAFs and vice versa, from SETAFs to NLPs. We show that there is pairwise equivalence between their semantics, including the equivalence between L-stable and semi-stable semantics. Furthermore, for a class of NLPs called Redundancy-Free Atomic Logic Programs (RFALPs), there is also a structural equivalence as these back-and-forth translations are each other's inverse. Then, we show that RFALPs are as expressive as NLPs by transforming any NLP into an equivalent RFALP through a series of program transformations already known in the literature. We also show that these program transformations are confluent, meaning that every NLP will be transformed into a unique RFALP. The results presented in this paper enhance our understanding that NLPs and SETAFs are essentially the same formalism. Under consideration in Theory and Practice of Logic Programming (TPLP)., Comment: 44 pages, 5 figures. Under consideration in Theory and Practice of Logic Programming (TPLP)

Length generalization refers to the ability to extrapolate from short training sequences to long test sequences and is a challenge for current large language models. While prior work has proposed some architecture or data format changes to achieve length generalization, these proposals typically apply to a limited set of tasks. Building on prior scratchpad and Chain-of-Thought (CoT) techniques, we propose Turing Programs, a novel CoT strategy that decomposes an algorithmic task into steps mimicking the computation of a Turing Machine. This framework is both universal, as it can accommodate any algorithmic task, and simple, requiring only copying text from the context with small modifications. We show that by using Turing Programs, we obtain robust length generalization on a range of algorithmic tasks: addition, multiplication and in-context SGD. We then demonstrate that transformers achieve length generalization on random Turing Programs, suggesting that length generalization is possible for any algorithmic task. Finally, we theoretically prove that transformers can implement Turing Programs, constructing a simple RASP (Weiss et al.) program that simulates an arbitrary Turing machine.

Overparameterization, the condition where models have more parameters than necessary to fit their training loss, is a crucial factor for the success of deep learning. However, the characteristics of the features learned by overparameterized networks are not well understood. In this work, we explore this question by comparing models with the same architecture but different widths. We first examine the expressivity of the features of these models, and show that the feature space of overparameterized networks cannot be spanned by concatenating many underparameterized features, and vice versa. This reveals that both overparameterized and underparameterized networks acquire some distinctive features. We then evaluate the performance of these models, and find that overparameterized networks outperform underparameterized networks, even when many of the latter are concatenated. We corroborate these findings using a VGG-16 and ResNet18 on CIFAR-10 and a Transformer on the MNLI classification dataset. Finally, we propose a toy setting to explain how overparameterized networks can learn some important features that the underparamaterized networks cannot learn.

We present our new model for the description of the very high energy Galactic gamma-ray emission based on a discrete injection of cosmic rays by individual sources. We investigate the morphology of the very high energy gamma-ray sky, the detectability of cosmic-ray sources and the clumpiness of the diffuse gamma-ray flux, assuming two different scenarios for cosmic-ray propagation. Namely, a standard isotropic and homogeneous diffusion process and an isotropic and inhomogeneous diffusion process. We notably formulate a possible explanation to the small number of hadronic PeVatrons recently detected by LHAASO. In the case of the inhomogeneous diffusion process, we constrain the number of hadronic PeVatrons to be small. Finally, we give an argument that may explain the discrepancy between the interstellar gas density distribution and the very high energy diffuse gamma-ray flux., Comment: 19 pages, 9 figures

Can Transformers predict new syllogisms by composing established ones? More generally, what type of targets can be learned by such models from scratch? Recent works show that Transformers can be Turing-complete in terms of expressivity, but this does not address the learnability objective. This paper puts forward the notion of 'globality degree' of a target distribution to capture when weak learning is efficiently achievable by regular Transformers, where the latter measures the least number of tokens required in addition to the tokens histogram to correlate nontrivially with the target. As shown experimentally and theoretically under additional assumptions, distributions with high globality cannot be learned efficiently. In particular, syllogisms cannot be composed on long chains. Furthermore, we show that (i) an agnostic scratchpad cannot help to break the globality barrier, (ii) an educated scratchpad can help if it breaks the globality at each step, however not all such scratchpads can generalize to out-of-distribution (OOD) samples, (iii) a notion of 'inductive scratchpad', that composes the prior information more efficiently, can both break the globality barrier and improve the OOD generalization. In particular, some inductive scratchpads can achieve length generalizations of up to 6x for some arithmetic tasks depending on the input formatting., Comment: 38 pages, 11 figures; terminology updated

Reconfigurable intelligent surfaces (RISs) have emerged as a spectrum- and energy-efficient technology to enhance the coverage of wireless communications within the upcoming 6G networks. Recently, novel extensions of this technology, referred to as multi-sector beyond diagonal RIS (BD-RIS), have been proposed, where the configurable elements are divided into $L$ sectors $(L \geq 2)$ and arranged as a polygon prism, with each sector covering $1/L$ space. This paper presents a performance analysis of a multi-user communication system assisted by a multi-sector BD-RIS operating in time-switching (TS) mode. Specifically, we derive closed-form expressions for the moment-generating function (MGF), probability density function (PDF), and cumulative density function (CDF) of the signal-to-noise ratio (SNR) per user. Furthermore, closed-form expressions for the outage probability, achievable spectral and energy efficiency, symbol error probability, and diversity order for the proposed system model are derived. Moreover, a comparison is performed with the simultaneously transmitting and reflecting (STAR)-RISs, a special case of multi-sector BD-RIS with two sectors. Our analysis shows that for a fixed number of elements, increasing the sectors improves outage performance at the expense of reduced diversity order compared to STAR-RIS. This trade-off is influenced by the Rician factors of the cascaded channel and the number of configurable elements per sector. However, this superiority in slope is observed at outage probability values below $10^{-5}$, which remains below practical operating ranges of communication systems. Additionally, simulations are provided to validate the accuracy of our theoretical analyses showing a notable $182\%$ increase in spectral efficiency and a $238\%$ increase in energy efficiency when transitioning from a 2-sector to a 6-sector configuration.

We study approximations to the Moreau envelope -- and infimal convolutions more broadly -- based on Laplace's method, a classical tool in analysis which ties certain integrals to suprema of their integrands. We believe the connection between Laplace's method and infimal convolutions is generally deserving of more attention in the study of optimization and partial differential equations, since it bears numerous potentially important applications, from proximal-type algorithms to solving Halmiton-Jacobi equations.

Mean-field control (MFC) problems aim to find the optimal policy to control massive populations of interacting agents. These problems are crucial in areas such as economics, physics, and biology. We consider the non-local setting, where the interactions between agents are governed by a suitable kernel. For $N$ agents, the interaction cost has $\mathcal{O}(N^2)$ complexity, which can be prohibitively slow to evaluate and differentiate when $N$ is large. To this end, we propose an efficient primal-dual algorithm that utilizes basis expansions of the kernels. The basis expansions reduce the cost of computing the interactions, while the primal-dual methodology decouples the agents at the expense of solving for a moderate number of dual variables. We also demonstrate that our approach can further be structured in a multi-resolution manner, where we estimate optimal dual variables using a moderate $N$ and solve decoupled trajectory optimization problems for large $N$. We illustrate the effectiveness of our method on an optimal control of 5000 interacting quadrotors.

This paper seeks whether Alfv\'en waves (AW) can be produced in laboratory-scale liquid metal experiments, \emph{i.e.} at low-magnetic Reynolds Number ($R\!m$). AW are incompressible waves propagating along magnetic fields typically found geo and astrophysical systems. Until now, only faint linear waves have been experimentally produced in liquid metals because of the large magnetic dissipation they undergo when $R\!m\ll1$. Yet, controlling laboratory AW could emulate such far remote processes as anomalous heating in the solar corona, oscillations of the Earth inner core or turbulence in the solar wind. To answer this question, we force AW with an AC electric current in a liquid metal channel in a transverse magnetic field. We derive a wave-bearing extension of the usual low$-R\!m$ MHD approximation to identify two linear regimes: The purely diffusive regime exists when $N_\omega$, the ratio of the oscillation period to the timescale of diffusive two-dimensionalisation by the Lorentz force, is small. The propagative regime is governed by the ratio of the forcing period to the AW propagation timescale which, we call the Jameson number $J\!a$ after Jameson (1964), JFM. In this regime, AW are dissipative and dispersive as they propagate more slowly where velocity gradients are higher. Both regimes are recovered in the FLOWCUBE experiment, in excellent agreement with the model up to $J\!a \lesssim 0.85$ but near the $J\!a=1$ resonance, high amplitude waves become clearly nonlinear. Hence, in electrically driving AW, we were able to produce some of the propagative, diffusive and nonlinear processes of astro and geophysical AW., Comment: 36 pages, 18 figures

The field of neuro-symbolic artificial intelligence (NeSy), which combines learning and reasoning, has recently experienced significant growth. There now are a wide variety of NeSy frameworks, each with its own specific language for expressing background knowledge and how to relate it to neural networks. This heterogeneity hinders accessibility for newcomers and makes comparing different NeSy frameworks challenging. We propose a unified language for NeSy, which we call ULLER, a Unified Language for LEarning and Reasoning. ULLER encompasses a wide variety of settings, while ensuring that knowledge described in it can be used in existing NeSy systems. ULLER has a neuro-symbolic first-order syntax for which we provide example semantics including classical, fuzzy, and probabilistic logics. We believe ULLER is a first step towards making NeSy research more accessible and comparable, paving the way for libraries that streamline training and evaluation across a multitude of semantics, knowledge bases, and NeSy systems., Comment: Pre-review version. Final version accepted at NeSy 2024

Thermo-osmotic flows, generated by applying a thermal gradient along a liquid-solid interface, could be harnessed to convert waste heat into electricity. While this phenomenon has been known for almost a century, there is a crucial need to gain a better understanding of the molecular origins of thermo-osmosis. In this paper, we start by detailing the multiple contributions to thermo-osmosis. We then showcase three approaches to compute the thermo-osmotic coefficient using molecular dynamics; a first method based on the computation of the interfacial enthalpy excess and Derjaguin's theoretical framework, a second approach based on the computation of the interfacial entropy excess using the so-called dry-surface method, and a novel non-equilibrium method to compute the thermo-osmotic coefficient in a periodic channel. We show that the three methods align with each other, in particular for smooth surfaces. In addition, for a polarized graphene-water interface, we observe large variations of thermo-osmotic responses, and multiple changes in flow direction with increasing surface charge. Overall, this study showcases the versatility of osmotic flows and calls for experimental investigation of thermo-osmotic behavior in the vicinity of charged surfaces.

Gaze following and social gaze prediction are fundamental tasks providing insights into human communication behaviors, intent, and social interactions. Most previous approaches addressed these tasks separately, either by designing highly specialized social gaze models that do not generalize to other social gaze tasks or by considering social gaze inference as an ad-hoc post-processing of the gaze following task. Furthermore, the vast majority of gaze following approaches have proposed static models that can handle only one person at a time, therefore failing to take advantage of social interactions and temporal dynamics. In this paper, we address these limitations and introduce a novel framework to jointly predict the gaze target and social gaze label for all people in the scene. The framework comprises of: (i) a temporal, transformer-based architecture that, in addition to image tokens, handles person-specific tokens capturing the gaze information related to each individual; (ii) a new dataset, VSGaze, that unifies annotation types across multiple gaze following and social gaze datasets. We show that our model trained on VSGaze can address all tasks jointly, and achieves state-of-the-art results for multi-person gaze following and social gaze prediction.

Penta-NiN2, a novel pentagonal 2D sheet with potential nanoelectronic applications, is investigated in terms of its lattice thermal conductivity, stability, and mechanical behavior. A deep learning interatomic potential (DLP) is firstly generated from ab-initio molecular dynamics (AIMD) data and then utilized for classical molecular dynamics simulations. The DLP's accuracy is verified, showing strong agreement with AIMD results. The dependence of thermal conductivity on size, temperature, and tensile strain, reveals important insights into the material's thermal properties. Additionally, the mechanical response of penta-NiN2 under uniaxial loading is examined, yielding a Young's modulus of approximately 368 GPa. The influence of vacancy defects on mechanical properties is analyzed, demonstrating significant reduction in modulus, fracture stress, and ultimate strength. This study also investigates the influence of strain on phonon dispersion relations and phonon group velocity in penta-NiN2, shedding light on how alterations in the atomic lattice affect the phonon dynamics and, consequently, impact the thermal conductivity. This investigation showcases the ability of deep learning based interatomic potentials in studying the properties of 2D Penta-NiN2.