Search

Diffusion models based on Multi-Head Attention (MHA) have become ubiquitous to generate high quality images and videos. However, encoding an image or a video as a sequence of patches results in costly attention patterns, as the requirements both in terms of memory and compute grow quadratically. To alleviate this problem, we propose a drop-in replacement for MHA called the Polynomial Mixer (PoM) that has the benefit of encoding the entire sequence into an explicit state. PoM has a linear complexity with respect to the number of tokens. This explicit state also allows us to generate frames in a sequential fashion, minimizing memory and compute requirement, while still being able to train in parallel. We show the Polynomial Mixer is a universal sequence-to-sequence approximator, just like regular MHA. We adapt several Diffusion Transformers (DiT) for generating images and videos with PoM replacing MHA, and we obtain high quality samples while using less computational resources. The code is available at https://github.com/davidpicard/HoMM.

This work addresses the problem of exact schedulability assessment in uniprocessor mixed-criticality real-time systems with sporadic task sets. We model the problem by means of a finite automaton that has to be explored in order to check for schedulability. To mitigate the state explosion problem, we provide a generic algorithm which is parameterised by several techniques called oracles and simulation relations. These techniques leverage results from the scheduling literature as "plug-ins" that make the algorithm more efficient in practice. Our approach achieves up to a 99.998% reduction in the search space required for exact schedulability testing, making it practical for a range of task sets, up to 8 tasks or maximum periods of 350. This method enables to challenge the pessimism of an existing schedulability test and to derive a new dynamic-priority scheduler, demonstrating its good performance. This is the full version of an RTNS 2024 paper., Comment: In the proceedings of 32nd International Conference on Real-Time Networks and Systems, RTNS24, ACM, 2024

If primordial scalar or tensor perturbations are enhanced on short scales, it may lead to the production of observable gravitational wave signals. These waves may be sourced by scalar-scalar, scalar-tensor or tensor-tensor interactions. Typically, models of inflation capable of producing large peaks in the scalar primordial power spectrum also generate sizeable scalar non-Gaussianity. Previous studies have investigated the possible effects of this on the scalar-scalar induced gravitational wave spectrum by assuming a local expansion in terms of the parameters $F_{\textrm{NL}}$, $G_{\textrm{NL}}$ and so on. We extend this approach to the case of scalar-tensor induced gravitational waves, introducing a local expansion for scalar non-Gaussianity into the scalar-tensor sector equations. We compute the contribution to the gravitational wave spectrum from the resulting new term and analyse its distinguishing features., Comment: 29 pages, 5 figures, comments welcome

PAC-Bayes learning is a comprehensive setting for (i) studying the generalisation ability of learning algorithms and (ii) deriving new learning algorithms by optimising a generalisation bound. However, optimising generalisation bounds might not always be viable for tractable or computational reasons, or both. For example, iteratively querying the empirical risk might prove computationally expensive. In response, we introduce a novel principled strategy for building an iterative learning algorithm via the optimisation of a sequence of surrogate training objectives, inherited from PAC-Bayes generalisation bounds. The key argument is to replace the empirical risk (seen as a function of hypotheses) in the generalisation bound by its projection onto a constructible low dimensional functional space: these projections can be queried much more efficiently than the initial risk. On top of providing that generic recipe for learning via surrogate PAC-Bayes bounds, we (i) contribute theoretical results establishing that iteratively optimising our surrogates implies the optimisation of the original generalisation bounds, (ii) instantiate this strategy to the framework of meta-learning, introducing a meta-objective offering a closed form expression for meta-gradient, (iii) illustrate our approach with numerical experiments inspired by an industrial biochemical problem.

Social-emotional learning (SEL) skills are essential for children to develop to provide a foundation for future relational and academic success. Using art as a medium for creation or as a topic to provoke conversation is a well-known method of SEL learning. Similarly, social robots have been used to teach SEL competencies like empathy, but the combination of art and social robotics has been minimally explored. In this paper, we present a novel child-robot interaction designed to foster empathy and promote SEL competencies via a conversation about art scaffolded by a social robot. Participants (N=11, age range: 7-11) conversed with a social robot about emotional and neutral art. Analysis of video and speech data demonstrated that this interaction design successfully engaged children in the practice of SEL skills, like emotion recognition and self-awareness, and greater rates of empathetic reasoning were observed when children engaged with the robot about emotional art. This study demonstrated that art-based reflection with a social robot, particularly on emotional art, can foster empathy in children, and interactions with a social robot help alleviate discomfort when sharing deep or vulnerable emotions., Comment: Copyright protected by IEEE, 8 pages, 7 figures, 1 table, in proceedings of 33rd IEEE International Conference on Robot & Human Interactive Communication (RO-MAN 2024)

An automata network with $n$ components over a finite alphabet $Q$ of size $q$ is a discrete dynamical system described by the successive iterations of a function $f:Q^n\to Q^n$. In most applications, the main parameter is the interaction graph of $f$: the digraph with vertex set $[n]$ that contains an arc from $j$ to $i$ if $f_i$ depends on input $j$. What can be said on the set $\mathbb{G}(f)$ of the interaction graphs of the automata networks isomorphic to $f$? It seems that this simple question has never been studied. In a previous paper, we prove that the complete digraph $K_n$, with $n^2$ arcs, is universal in that $K_n\in \mathbb{G}(f)$ whenever $f$ is not constant nor the identity (and $n\geq 5$). In this paper, taking the opposite direction, we prove that there exists universal automata networks $f$, in that $\mathbb{G}(f)$ contains all the digraphs on $[n]$, excepted the empty one. Actually, we prove that the presence of only three specific digraphs in $\mathbb{G}(f)$ implies the universality of $f$, and we prove that this forces the alphabet size $q$ to have at least $n$ prime factors (with multiplicity). However, we prove that for any fixed $q\geq 3$, there exists almost universal functions, that is, functions $f:Q^n\to Q^n$ such that the probability that a random digraph belongs to $\mathbb{G}(f)$ tends to $1$ as $n\to\infty$. We do not know if this holds in the binary case $q=2$, providing only partial results., Comment: 28 pages

We consider the problem of estimating the direction-of-arrival (DoA) of a desired source located in a known region of interest in the presence of interfering sources and multipath. We propose an approach that precedes the DoA estimation and relies on generating a set of reference steering vectors. The steering vectors' generative model is a free space model, which is beneficial for many DoA estimation algorithms. The set of reference steering vectors is then used to compute a function that maps the received signals from the adverse environment to a reference domain free from interfering sources and multipath. We show theoretically and empirically that the proposed map, which is analogous to domain adaption, improves DoA estimation by mitigating interference and multipath effects. Specifically, we demonstrate a substantial improvement in accuracy when the proposed approach is applied before three commonly used beamformers: the delay-and-sum (DS), the minimum variance distortionless response (MVDR), and the Multiple Signal Classification (MUSIC).

We study the cohomology of an elliptic differential complex arising from the infinitesimal moduli of heterotic string theory. We compute these cohomology groups at the standard embedding, and show that they decompose into a direct sum of cohomologies. While this is often assumed in the literature, it had not been explicitly demonstrated. Given a stable gauge bundle over a complex threefold with trivial canonical bundle and no holomorphic vector fields, we also show that the Euler characteristic of this differential complex is zero. This points towards a perfect obstruction theory for the heterotic moduli problem, at least for the most physically relevant compactifications., Comment: v2: 19 pages, minor changes to the presentation, references updated

We introduce a novel approach for 3D whole-body pose estimation, addressing the challenge of scale- and deformability- variance across body parts brought by the challenge of extending the 17 major joints on the human body to fine-grained keypoints on the face and hands. In addition to addressing the challenge of exploiting motion in unevenly sampled data, we combine stable diffusion to a hierarchical part representation which predicts the relative locations of fine-grained keypoints within each part (e.g., face) with respect to the part's local reference frame. On the H3WB dataset, our method greatly outperforms the current state of the art, which fails to exploit the temporal information. We also show considerable improvements compared to other spatiotemporal 3D human-pose estimation approaches that fail to account for the body part specificities. Code is available at https://github.com/valeoai/PAFUSE.

Quantum computation (QC) and simulation rely on long-lived qubits with controllable interactions. Early work in quantum computing made use of molecules because of their readily available intramolecular nuclear spin coupling and chemical shifts, along with mature nuclear magnetic resonance techniques. Subsequently, the pursuit of many physical platforms has flourished. Trapped polar molecules have been proposed as a promising quantum computing platform, offering scalability and single-particle addressability while still leveraging inherent complexity and strong couplings of molecules. Recent progress in the single quantum state preparation and coherence of the hyperfine-rotational states of individually trapped molecules allows them to serve as promising qubits, with intermolecular dipolar interactions creating entanglement. However, universal two-qubit gates have not been demonstrated with molecules. Here, we harness intrinsic molecular resources to implement a two-qubit iSWAP gate using individually trapped $X^{1}\Sigma^{+}$ NaCs molecules. We characterize the innate dipolar interaction between rotational states and control its strength by tuning the polarization of the traps. By allowing the molecules to interact for 664 $\mu$s at a distance of 1.9 $\mu$m, we create a maximally entangled Bell state with a fidelity of 94(3)\%, following postselection to remove trials with empty traps. Using motion-rotation coupling, we measure residual excitation of the lowest few motional states along the axial trapping direction and find them to be the primary source of decoherence. Finally, we identify two non-interacting hyperfine states within the ground rotational level in which we encode a qubit. The interaction is toggled by transferring between interacting and non-interacting states to realize an iSWAP gate. We verify the gate performance by measuring its logical truth table.

Conditional diffusion models are powerful generative models that can leverage various types of conditional information, such as class labels, segmentation masks, or text captions. However, in many real-world scenarios, conditional information may be noisy or unreliable due to human annotation errors or weak alignment. In this paper, we propose the Coherence-Aware Diffusion (CAD), a novel method that integrates coherence in conditional information into diffusion models, allowing them to learn from noisy annotations without discarding data. We assume that each data point has an associated coherence score that reflects the quality of the conditional information. We then condition the diffusion model on both the conditional information and the coherence score. In this way, the model learns to ignore or discount the conditioning when the coherence is low. We show that CAD is theoretically sound and empirically effective on various conditional generation tasks. Moreover, we show that leveraging coherence generates realistic and diverse samples that respect conditional information better than models trained on cleaned datasets where samples with low coherence have been discarded., Comment: Accepted at CVPR 2024 as a Highlight. Project page: https://nicolas-dufour.github.io/cad.html

Uncertainty quantification is critical for ensuring adequate predictive power of computational models used in biology. Focusing on two anaerobic digestion models, this article introduces a novel generalized Bayesian procedure, called VarBUQ, ensuring a correct tradeoff between flexibility and computational cost. A benchmark against three existing methods (Fisher's information, bootstrapping and Beale's criteria) was conducted using synthetic data. This Bayesian procedure offered a good compromise between fitting ability and confidence estimation, while the other methods proved to be repeatedly overconfident. The method's performances notably benefitted from inductive bias brought by the prior distribution, although it requires careful construction. This article advocates for more systematic consideration of uncertainty for anaerobic digestion models and showcases a new, computationally efficient Bayesian method. To facilitate future implementations, a Python package called 'aduq' is made available.

Kernel-based testing has revolutionized the field of non-parametric tests through the embedding of distributions in an RKHS. This strategy has proven to be powerful and flexible, yet its applicability has been limited to the standard two-sample case, while practical situations often involve more complex experimental designs. To extend kernel testing to any design, we propose a linear model in the RKHS that allows for the decomposition of mean embeddings into additive functional effects. We then introduce a truncated kernel Hotelling-Lawley statistic to test the effects of the model, demonstrating that its asymptotic distribution is chi-square, which remains valid with its Nystrom approximation. We discuss a homoscedasticity assumption that, although absent in the standard two-sample case, is necessary for general designs. Finally, we illustrate our framework using a single-cell RNA sequencing dataset and provide kernel-based generalizations of classical diagnostic and exploration tools to broaden the scope of kernel testing in any experimental design., Comment: 9 pages, 2 figures

We introduce a novel statistical framework for the analysis of replicated point processes that allows for the study of point pattern variability at a population level. By treating point process realizations as random measures, we adopt a functional analysis perspective and propose a form of functional Principal Component Analysis (fPCA) for point processes. The originality of our method is to base our analysis on the cumulative mass functions of the random measures which gives us a direct and interpretable analysis. Key theoretical contributions include establishing a Karhunen-Lo\`{e}ve expansion for the random measures and a Mercer Theorem for covariance measures. We establish convergence in a strong sense, and introduce the concept of principal measures, which can be seen as latent processes governing the dynamics of the observed point patterns. We propose an easy-to-implement estimation strategy of eigenelements for which parametric rates are achieved. We fully characterize the solutions of our approach to Poisson and Hawkes processes and validate our methodology via simulations and diverse applications in seismology, single-cell biology and neurosiences, demonstrating its versatility and effectiveness. Our method is implemented in the pppca R-package.

Classifier-Free Guidance (CFG) enhances the quality and condition adherence of text-to-image diffusion models. It operates by combining the conditional and unconditional predictions using a fixed weight. However, recent works vary the weights throughout the diffusion process, reporting superior results but without providing any rationale or analysis. By conducting comprehensive experiments, this paper provides insights into CFG weight schedulers. Our findings suggest that simple, monotonically increasing weight schedulers consistently lead to improved performances, requiring merely a single line of code. In addition, more complex parametrized schedulers can be optimized for further improvement, but do not generalize across different models and tasks.

We study the geometry of Calabi-Yau conifold transitions. This deformation process is known to possibly connect a K\"ahler threefold to a non-K\"ahler threefold. We use balanced and Hermitian-Yang-Mills metrics to geometrize the conifold transition and show that the whole operation is continuous in the Gromov-Hausdorff topology., Comment: 46 pages

Bayesian Optimization (BO) is a foundational strategy in the field of engineering design optimization for efficiently handling black-box functions with many constraints and expensive evaluations. This paper introduces a fast and accurate BO framework that leverages Pre-trained Transformers for Bayesian Optimization (PFN4sBO) to address constrained optimization problems in engineering. Unlike traditional BO methods that rely heavily on Gaussian Processes (GPs), our approach utilizes Prior-data Fitted Networks (PFNs), a type of pre-trained transformer, to infer constraints and optimal solutions without requiring any iterative retraining. We demonstrate the effectiveness of PFN-based BO through a comprehensive benchmark consisting of fifteen test problems, encompassing synthetic, structural, and engineering design challenges. Our findings reveal that PFN-based BO significantly outperforms Constrained Expected Improvement and Penalty-based GP methods by an order of magnitude in speed while also outperforming them in accuracy in identifying feasible, optimal solutions. This work showcases the potential of integrating machine learning with optimization techniques in solving complex engineering challenges, heralding a significant leap forward for optimization methodologies, opening up the path to using PFN-based BO to solve other challenging problems, such as enabling user-guided interactive BO, adaptive experiment design, or multi-objective design optimization. Additionally, we establish a benchmark for evaluating BO algorithms in engineering design, offering a robust platform for future research and development in the field. This benchmark framework for evaluating new BO algorithms in engineering design will be published at https://github.com/rosenyu304/BOEngineeringBenchmark., Comment: Under review of Structural and Multidisciplinary Optimization

Covariance fitting (CF) is a comprehensive approach for direction of arrival (DoA) estimation, consolidating many common solutions. Standard practice is to use Euclidean criteria for CF, disregarding the intrinsic Hermitian positive-definite (HPD) geometry of the spatial covariance matrices. We assert that this oversight leads to inherent limitations. In this paper, as a remedy, we present a comprehensive study of the use of various Riemannian metrics of HPD matrices in CF. We focus on the advantages of the Affine-Invariant (AI) and the Log-Euclidean (LE) Riemannian metrics. Consequently, we propose a new practical beamformer based on the LE metric and derive analytically its spatial characteristics, such as the beamwidth and sidelobe attenuation, under noisy conditions. Comparing these features to classical beamformers shows significant advantage. In addition, we demonstrate, both theoretically and experimentally, the LE beamformer's robustness in scenarios with small sample sizes and in the presence of noise, interference, and multipath channels.

This paper addresses a new active learning strategy for regression problems. The presented Wasserstein active regression model is based on the principles of distribution-matching to measure the representativeness of the labeled dataset. The Wasserstein distance is computed using GroupSort Neural Networks. The use of such networks provides theoretical foundations giving a way to quantify errors with explicit bounds for their size and depth. This solution is combined with another uncertainty-based approach that is more outlier-tolerant to complete the query strategy. Finally, this method is compared with other classical and recent solutions. The study empirically shows the pertinence of such a representativity-uncertainty approach, which provides good estimation all along the query procedure. Moreover, the Wasserstein active regression often achieves more precise estimations and tends to improve accuracy faster than other models.

This is a survey on the Strominger system and a geometric flow known as the anomaly flow. We will discuss various aspects of non-K\"ahler geometry on Calabi-Yau threefolds. Along the way, we discuss balanced metrics and balanced classes, the Aeppli cohomology class associated to a solution to the Strominger system, the equations of motion of heterotic supergravity, and a version of Ricci flow in this special geometry., Comment: To appear in Surveys in Differential Geometry; v3 minor changes. 44 pages

As soon as a new technology emerges, the education community explores its affordances and the possibilities to apply it in education. In this paper, we analyze sessions with ChatGPT around topics in basic Linear Algebra. We reflect the process undertaken by the ChatGPT along the recent year in our area of interest, emphasising the vast improvement that has been done in grappling with Linear Algebra problems. In particular, the question whether this software can be a teaching assistant or even somehow replace the human teacher, is addressed. As of the time this paper is written, the answer is generally negative. For the small part where the answer can be positive, some reflections about an original instrumental genesis are given. Communication with the software gives the impression to talk to a human, and sometimes the question is whether the software understands the question or not. Therefore, the reader's attention is drawn to the fact that ChatGPT works on a statistical basis and not according to reflection and understanding., Comment: arXiv admin note: text overlap with arXiv:2306.16282

We consider N=1, d=4 vacua of heterotic theories in the large radius limit in which alpha' << 1. We construct a real differential operator $\mathcal{D}= D+\bar{D}$ on an extension bundle $(Q, \mathcal{D})$ with underlying topology $Q=(T^{1,0}X)^* \oplus {\rm End} \, E \oplus T^{1,0} X$ whose curvature is holomorphic and Hermitian-Yang-Mills with respect to the complex structure and metric on the underlying non-Kahler complex 3-fold X if and only if the heterotic supersymmetry equations and Bianchi identity are satisfied. This is suggestive of an analogue of the Donaldson--Uhlenbeck--Yau correspondence for heterotic vacua of this type., Comment: 34 pages; v2 references updated, typos fixed

A Boolean network is a function $f:\{0,1\}^n\to\{0,1\}^n$ from which several dynamics can be derived, depending on the context. The most classical ones are the synchronous and asynchronous dynamics. Both are digraphs on $\{0,1\}^n$, but the synchronous dynamics (which is identified with $f$) has an arc from $x$ to $f(x)$ while the asynchronous dynamics $\mathcal{A}(f)$ has an arc from $x$ to $x+e_i$ whenever $x_i\neq f_i(x)$. Clearly, $f$ and $\mathcal{A}(f)$ share the same information, but what can be said on these objects up to isomorphism? We prove that if $\mathcal{A}(f)$ is only known up to isomorphism then, with high probability, $f$ can be fully reconstructed up to isomorphism. We then show that the converse direction is far from being true. In particular, if $f$ is only known up to isomorphism, very little can be said on the attractors of $\mathcal{A}(f)$. For instance, if $f$ has $p$ fixed points, then $\mathcal{A}(f)$ has at least $\max(1,p)$ attractors, and we prove that this trivial lower bound is tight: there always exists $h\sim f$ such that $\mathcal{A}(h)$ has exactly $\max(1,p)$ attractors. But $\mathcal{A}(f)$ may often have much more attractors since we prove that, with high probability, there exists $h\sim f$ such that $\mathcal{A}(h)$ has $\Omega(2^n)$ attractors., Comment: 30p, submitted

Polar molecules are a quantum resource with rich internal structure that can be coherently controlled. The structure, however, also makes the state preparation and measurement (SPAM) of molecules challenging. We advance the SPAM of individual molecules assembled from constituent atoms trapped in optical tweezer arrays. Sites without NaCs molecules are eliminated using high-fidelity Cs atom detection, increasing the peak molecule filling fraction of the array threefold. We site-selectively initialize the array in a rotational qubit subspace that is insensitive to differential AC Stark shifts from the optical tweezer. Lastly, we detect multiple rotational states per experimental cycle by imaging atoms after sequential state-selective dissociations. These demonstrations extend the SPAM capabilities of molecules for quantum information, simulation, and metrology.

We present simple models of trajectories in space, both in 2D and in 3D. The first examples, which model bicircular moves in the same direction, are classical curves (epicycloids, etc.). Then, we explore bicircular moves in reverse direction and tricircular moves in 2D and 3D, to explore complex visualisations of extraplanetary movements. These moves are studied in a plane setting. Then, adding increasing complexity, we explore them in a non planar setting (which is a closer model of the real situation). The exploration is followed by using these approaches for creating mathematical art in 2D and 3D printed objects, providing new ways of mathematical representations. Students' activities are organized around this exploration., Comment: In Proceedings ADG 2023, arXiv:2401.10725

In this paper, we fix the complex structure and explore the moduli space of the heterotic system by considering two different yet "dual" deformation paths starting from a K\"ahler solution. They correspond to deformation along the Bott-Chern cohomology class and the Aeppli cohomology class respectively. Using the implicit function theorem, we prove the local existence of heterotic solutions along these two paths and hence establish an initial step to construct local moduli coordinates around a K\"ahler solution., Comment: 35 pages

We provide certain compatibility conditions for m-accretive operators such that the adjoint of the sum is given by the closure of the sum of the respective adjoint. We revisit the proof of well-posedness of the abstract class of partial differential-algebraic equations known as evolutionary equations. We show that the general mechanism provided here can be applied to establish well-posedness for non-autonomous evolutionary equations with $L_{\infty}$-coefficients thus not only generalising known results but opening up new directions other methods such as evolution families have a hard time to come by., Comment: 13 pages

Ensuring fairness in NLP models is crucial, as they often encode sensitive attributes like gender and ethnicity, leading to biased outcomes. Current concept erasure methods attempt to mitigate this by modifying final latent representations to remove sensitive information without retraining the entire model. However, these methods typically rely on linear classifiers, which leave models vulnerable to non-linear adversaries capable of recovering sensitive information. We introduce Targeted Concept Erasure (TaCo), a novel approach that removes sensitive information from final latent representations, ensuring fairness even against non-linear classifiers. Our experiments show that TaCo outperforms state-of-the-art methods, achieving greater reductions in the prediction accuracy of sensitive attributes by non-linear classifier while preserving overall task performance. Code is available on https://github.com/fanny-jourdan/TaCo.