Search

The classical Armijo backtracking algorithm achieves the optimal complexity for smooth functions like gradient descent but without any hyperparameter tuning. However, the smoothness assumption is not suitable for Deep Learning optimization. In this work, we show that some variants of the Armijo optimizer achieves acceleration and optimal complexities under assumptions more suited for Deep Learning: the (L 0 , L 1 ) smoothness condition and analyticity. New dependences on the smoothness constants and the initial gap are established. The results theoretically highlight the powerful efficiency of Armijo-like conditions for highly non-convex problems.

Recent advances in image editing, driven by image diffusion models, have shown remarkable progress. However, significant challenges remain, as these models often struggle to follow complex edit instructions accurately and frequently compromise fidelity by altering key elements of the original image. Simultaneously, video generation has made remarkable strides, with models that effectively function as consistent and continuous world simulators. In this paper, we propose merging these two fields by utilizing image-to-video models for image editing. We reformulate image editing as a temporal process, using pretrained video models to create smooth transitions from the original image to the desired edit. This approach traverses the image manifold continuously, ensuring consistent edits while preserving the original image's key aspects. Our approach achieves state-of-the-art results on text-based image editing, demonstrating significant improvements in both edit accuracy and image preservation.

In this short article, we will reconstruct the KP equation from Plucker relations and provide some generalizations on this topic. Additionally, in the final section, we define the discrete function $\tau$ in a similar manner, leading to the construction of an integer sequence that has not yet been listed in the OEIS. Furthermore, this approach allows us to construct many other sequences that are not listed in the OEIS.

Both accelerated and adaptive gradient methods are among state of the art algorithms to train neural networks. The tuning of hyperparameters is needed to make them work efficiently. For classical gradient descent, a general and efficient way to adapt hyperparameters is the Armijo backtracking. The goal of this work is to generalize the Armijo linesearch to Momentum and RMSProp, two popular optimizers of this family, by means of stability theory of dynamical systems. We establish convergence results, under the Lojasiewicz assumption, for these strategies. As a direct result, we obtain the first guarantee on the convergence of the iterates for RMSProp, in the non-convex setting without the classical bounded assumptions.

Recently, many machine learning optimizers have been analysed considering them as the asymptotic limit of some differential equations when the step size goes to zero. In other words, the optimizers can be seen as a finite difference scheme applied to a continuous dynamical system. But the major part of the results in the literature concerns constant step size algorithms. The main aim of this paper is to investigate the guarantees of the adaptive step size counterpart. In fact, this dynamical point of view can be used to design step size update rules, by choosing a discretization of the continuous equation that preserves its most relevant features. In this work, we analyse this kind of adaptive optimizers and prove their Lyapunov stability and convergence properties for any choice of hyperparameters. At the best of our knowledge, this paper introduces for the first time the use of continuous selection theory from general topology to overcome some of the intrinsic difficulties due to the non constant and non regular step size policies. The general framework developed gives many new results on adaptive and constant step size Momentum/Heavy-Ball and p-GD algorithms.

We show that there exists a quasi-isometric embedding of the product of $n$ copies of $\mathbb{H}_{\mathbb{R}}^2$ into any symmetric space of non-compact type of rank $n$, and there exists a bi-Lipschitz embedding of the product of $n$ copies of the $3$-regular tree $T_3$ into any thick Euclidean building of rank $n$ with co-compact affine Weyl group. This extends a previous result of Fisher--Whyte. The proof is purely geometrical, and the result also applies to the non Bruhat--Tits buildings., Comment: 23 pages, 6 figures. Comments are welcome!

In this article, we introduce and study a natural notion of coarse separation for metric spaces, with an emphasis on coarse separation by subspaces of polynomial or subexponential growth. For instance, we show that symmetric spaces of non-compact type different from $\mathbb{H}_\mathbb{R}^2$ and thick Euclidean buildings of rank $\geq 2$ cannot be coarsely separated by subspaces of subexponential growth; and that a connected nilpotent Lie group of growth degree $D \geq 2$ cannot be coarsely separated by a subspace of polynomial degree $\leq D-2$. We apply these results to the large-scale geometry of amalgamated free products and wreath products. The latter application is based on an Embedding Theorem that generalises previous work of the last two authors, and which is of independent interest. We also discuss some further applications to the distorsion of coarse embeddings between certain metric spaces., Comment: 53 pages, 8 figures. Comments are welcome!

In recent years, the rapid evolution of computer vision has seen the emergence of various foundation models, each tailored to specific data types and tasks. In this study, we explore the adaptation of these models for few-shot semantic segmentation. Specifically, we conduct a comprehensive comparative analysis of four prominent foundation models: DINO V2, Segment Anything, CLIP, Masked AutoEncoders, and of a straightforward ResNet50 pre-trained on the COCO dataset. We also include 5 adaptation methods, ranging from linear probing to fine tuning. Our findings show that DINO V2 outperforms other models by a large margin, across various datasets and adaptation methods. On the other hand, adaptation methods provide little discrepancy in the obtained results, suggesting that a simple linear probing can compete with advanced, more computationally intensive, alternatives

In this paper, we introduce various machine learning solvers for (coupled) forward-backward systems of stochastic differential equations (FBSDEs) driven by a Brownian motion and a Poisson random measure. We provide a rigorous comparison of the different algorithms and demonstrate their effectiveness in various applications, such as cases derived from pricing with jumps and mean-field games. In particular, we show the efficiency of the deep-learning algorithms to solve a coupled multi-dimensional FBSDE system driven by a time-inhomogeneous jump process with stochastic intensity, which describes the Nash equilibria for a specific mean-field game (MFG) problem for which we also provide the complete theoretical resolution. More precisely, we develop an extension of the MFG model for smart grids introduced in Alasseur, Campi, Dumitrescu and Zeng (Annals of Operations Research, 2023) to the case when the random jump times correspond to the jump times of a doubly Poisson process. We first provide an existence result of an equilibria and derive its semi-explicit characterization in terms of a system of FBSDEs in the linear-quadratic setting. We then compare the MFG solution to the optimal strategy of a central planner and provide several numerical illustrations using the deep-learning solvers presented in the first part of the paper.

The analysis of trail-running performance appears to be complex and cardio-respiratory and muscular factors could have a variable importance depending on the inclination. Our study aims to determine the role of these parameters in performance. 13 subjects with heterogeneous levels participated in the study. They carried out 7 visits including 3 maximal aerobic speed (MAS) test at 1, 10 and 25% slope on treadmill, 3 endurance tests at 100% of the MAS reached at 1, 10 and 25% and an evaluation on isokinetic ergometer at different speeds (60-180-240 {\textdegree}/s). Gas exchange measured during the incremental tests. We were able to identify 2 groups, a performance and a recreational group. We observe a difference in VO2max, MAS at 1 and 10%, and maximal aerobic ascensional speed (MAaS) at 25%, between the 2 groups but no difference in VO2max and exhaustion time at 100% MAS between the different conditions (1-10-25%). Interestingly, at ventilatory thresholds the metabolic parameters, expressed as absolute or relative values, are similar between conditions (10-25%) while the ascensional speed are different. This study suggests that the measurement of ascensional speed is not as relevant as heart rate for controlling intensity given the variety of slope gradients in trail-running races.

The advent of vision-language pre-training techniques enhanced substantial progress in the development of models for image captioning. However, these models frequently produce generic captions and may omit semantically important image details. This limitation can be traced back to the image-text datasets; while their captions typically offer a general description of image content, they frequently omit salient details. Considering the magnitude of these datasets, manual reannotation is impractical, emphasizing the need for an automated approach. To address this challenge, we leverage existing captions and explore augmenting them with visual details using "frozen" vision experts including an object detector, an attribute recognizer, and an Optical Character Recognizer (OCR). Our proposed method, FuseCap, fuses the outputs of such vision experts with the original captions using a large language model (LLM), yielding comprehensive image descriptions. We automatically curate a training set of 12M image-enriched caption pairs. These pairs undergo extensive evaluation through both quantitative and qualitative analyses. Subsequently, this data is utilized to train a captioning generation BLIP-based model. This model outperforms current state-of-the-art approaches, producing more precise and detailed descriptions, demonstrating the effectiveness of the proposed data-centric approach. We release this large-scale dataset of enriched image-caption pairs for the community.

Reading text in real-world scenarios often requires understanding the context surrounding it, especially when dealing with poor-quality text. However, current scene text recognizers are unaware of the bigger picture as they operate on cropped text images. In this study, we harness the representative capabilities of modern vision-language models, such as CLIP, to provide scene-level information to the crop-based recognizer. We achieve this by fusing a rich representation of the entire image, obtained from the vision-language model, with the recognizer word-level features via a gated cross-attention mechanism. This component gradually shifts to the context-enhanced representation, allowing for stable fine-tuning of a pretrained recognizer. We demonstrate the effectiveness of our model-agnostic framework, CLIPTER (CLIP TExt Recognition), on leading text recognition architectures and achieve state-of-the-art results across multiple benchmarks. Furthermore, our analysis highlights improved robustness to out-of-vocabulary words and enhanced generalization in low-data regimes., Comment: Accepted for publication by ICCV 2023

Context. Filaments are ubiquitous in the Galaxy, and they host star formation. Detecting them in a reliable way is therefore key towards our understanding of the star formation process. Aims. We explore whether supervised machine learning can identify filamentary structures on the whole Galactic plane. Methods. We used two versions of UNet-based networks for image segmentation.We used H2 column density images of the Galactic plane obtained with Herschel Hi-GAL data as input data. We trained the UNet-based networks with skeletons (spine plus branches) of filaments that were extracted from these images, together with background and missing data masks that we produced. We tested eight training scenarios to determine the best scenario for our astrophysical purpose of classifying pixels as filaments. Results. The training of the UNets allows us to create a new image of the Galactic plane by segmentation in which pixels belonging to filamentary structures are identified. With this new method, we classify more pixels (more by a factor of 2 to 7, depending on the classification threshold used) as belonging to filaments than the spine plus branches structures we used as input. New structures are revealed, which are mainly low-contrast filaments that were not detected before.We use standard metrics to evaluate the performances of the different training scenarios. This allows us to demonstrate the robustness of the method and to determine an optimal threshold value that maximizes the recovery of the input labelled pixel classification. Conclusions. This proof-of-concept study shows that supervised machine learning can reveal filamentary structures that are present throughout the Galactic plane. The detection of these structures, including low-density and low-contrast structures that have never been seen before, offers important perspectives for the study of these filaments., Comment: 27 pages, 22 figures, accepted by Astronomy & Astrophysics

Let $(\tau,V_\tau)$ be a spinor representation of $\mathrm{Spin}(n)$ and let $(\sigma,V_\sigma)$ be a spinor representation of $\mathrm{Spin}(n-1)$ that occurs in the restriction $\tau_{\mid \mathrm{Spin}(n-1)}$. We consider the real hyperbolic space $H^n(\mathbb R)$ as the rank one homogeneous space $\mathrm{Spin}_0(1,n)/\mathrm{Spin}(n)$ and the spinor bundle $\Sigma H^n(\mathbb R)$ over $H^n(\mathbb R)$ as the homogeneous bundle $\mathrm{Spin}_0(1,n)\times_{\mathrm{Spin}(n)} V_\tau$. Our aim is to characterize eigenspinors of the algebra of invariant differential operators acting on $\Sigma H^n(\mathbb R)$ which can be written as the Poisson transform of $L^p$-sections of the bundle $\mathrm{Spin}(n)\times_{\mathrm{Spin}(n-1)} V_\sigma$ over the boundary $S^{n-1}\simeq \mathrm{Spin}(n)/\mathrm{Spin}(n-1)$ of $H^n(\mathbb R)$, for $1

Evaluating the similarity of non-rigid shapes with significant partiality is a fundamental task in numerous computer vision applications. Here, we propose a novel axiomatic method to match similar regions across shapes. Matching similar regions is formulated as the alignment of the spectra of operators closely related to the Laplace-Beltrami operator (LBO). The main novelty of the proposed approach is the consideration of differential operators defined on a manifold with multiple metrics. The choice of a metric relates to fundamental shape properties while considering the same manifold under different metrics can thus be viewed as analyzing the underlying manifold from different perspectives. Specifically, we examine the scale-invariant metric and the corresponding scale-invariant Laplace-Beltrami operator (SI-LBO) along with the regular metric and the regular LBO. We demonstrate that the scale-invariant metric emphasizes the locations of important semantic features in articulated shapes. A truncated spectrum of the SI-LBO consequently better captures locally curved regions and complements the global information encapsulated in the truncated spectrum of the regular LBO. We show that matching these dual spectra outperforms competing axiomatic frameworks when tested on standard benchmarks. We introduced a new dataset and compare the proposed method with the state-of-the-art learning based approach in a cross-database configuration. Specifically, we show that, when trained on one data set and tested on another, the proposed axiomatic approach which does not involve training, outperforms the deep learning alternative.

Introduced by Gromov in the 80's, coarse embeddings are a generalization of quasi-isometric embeddings when the control functions are not necessarily affine. In this paper, we will be particularly interested in coarse embeddings between symmetric spaces and Euclidean buildings. The quasi-isometric case is very well understood thanks to the rigidity results for symmetric spaces and buildings of higher rank by Anderson-Schroeder, Kleiner, Kleiner-Leeb, Eskin-Farb and Fisher-Whyte. In particular, it is well known that the rank of these spaces is monotonous under quasi-isometric embeddings. This is no longer the case for coarse embeddings as shown by horospherical embeddings. However, we show that in the absence of a Euclidean factor in the domain, the rank is monotonous under coarse embeddings. This answers a question by David Fisher and Kevin Whyte. This still holds when we replace the target space by a proper cocompact CAT(0) space or by a mapping class group. Between symmetric spaces and Euclidean buildings, we can also relax the condition on the domain by allowing it to contain a Euclidean factor of dimension 1, answering a question by Gromov., Comment: 63 pages, 6 figures. We extend the previous results to CAT(0) spaces and mapping class groups in the target space

This paper is concerned with the Poisson transform of differential forms on the hyperbolic space $H^n(\mathbb R)$. Consider an integer $p$ such that $1\leqslant p\leqslant n$ and let $q$ be either $p-1$ or $p$. For $1

Depth estimation is a cornerstone of a vast number of applications requiring 3D assessment of the environment, such as robotics, augmented reality, and autonomous driving to name a few. One prominent technique for depth estimation is stereo matching which has several advantages: it is considered more accessible than other depth-sensing technologies, can produce dense depth estimates in real-time, and has benefited greatly from the advances of deep learning in recent years. However, current techniques for depth estimation from stereoscopic images still suffer from a built-in drawback. To reconstruct depth, a stereo matching algorithm first estimates the disparity map between the left and right images before applying a geometric triangulation. A simple analysis reveals that the depth error is quadratically proportional to the object's distance. Therefore, constant disparity errors are translated to large depth errors for objects far from the camera. To mitigate this quadratic relation, we propose a simple but effective method that uses a refinement network for depth estimation. We show analytical and empirical results suggesting that the proposed learning procedure reduces this quadratic relation. We evaluate the proposed refinement procedure on well-known benchmarks and datasets, like Sceneflow and KITTI datasets, and demonstrate significant improvements in the depth accuracy metric.

Storytelling is an open-ended task that entails creative thinking and requires a constant flow of ideas. Natural language generation (NLG) for storytelling is especially challenging because it requires the generated text to follow an overall theme while remaining creative and diverse to engage the reader. In this work, we introduce a system and a web-based demo, FairyTailor, for human-in-the-loop visual story co-creation. Users can create a cohesive children's fairytale by weaving generated texts and retrieved images with their input. FairyTailor adds another modality and modifies the text generation process to produce a coherent and creative sequence of text and images. To our knowledge, this is the first dynamic tool for multimodal story generation that allows interactive co-formation of both texts and images. It allows users to give feedback on co-created stories and share their results., Comment: visit https://fairytailor.org/ and https://github.com/EdenBD/MultiModalStory-demo for web demo and source code