Search

Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural Architecture Search (NAS) methods. It drastically reduces search cost by resorting to weight-sharing. However, it also dramatically reduces the search space, thus excluding potential promising architectures. In this article, we propose D-DARTS, a solution that addresses this problem by nesting neural networks at the cell level instead of using weight-sharing to produce more diversified and specialized architectures. Moreover, we introduce a novel algorithm that can derive deeper architectures from a few trained cells, increasing performance and saving computation time. In addition, we also present an alternative search space (DARTOpti) in which we optimize existing handcrafted architectures (e.g., ResNet) rather than starting from scratch. This approach is accompanied by a novel metric that measures the distance between architectures inside our custom search space. Our solution reaches competitive performance on multiple computer vision tasks. Code and pretrained models can be accessed at https://github.com/aheuillet/D-DARTS., Submitted to Pattern Recognition Letters

International audience; Human pose estimation and action recognition are related tasks since both problems are strongly dependent on the human body representation and analysis. Nonetheless, most recent methods in the literature handle the two problems separately. In this work, we propose a multi-task framework for jointly estimating 2D or 3D human poses from monocular color images and classifying human actions from video sequences. We show that a single architecture can be used to solve both problems in an efficient way and still achieves state-of-the-art or comparable results at each task while running with a throughput of more than 100 frames per second. The proposed method benefits from high parameters sharing between the two tasks by unifying still images and video clips processing in a single pipeline, allowing the model to be trained with data from different categories simultaneously and in a seamlessly way. Additionally, we provide important insights for end-to-end training the proposed multi-task model by decoupling key prediction parts, which consistently leads to better accuracy on both tasks. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU RGB+D) demonstrate the effectiveness of our method on the targeted tasks. Our source code and trained weights are publicly available at https://github.com/dluvizon/deephar.

International audience; Emotions taint our life and allow expressing the different facets of the personality. Among the expressions of the human body, facial ones are the most representative of the mindscape of a person. Several works are devoted to it and applications have already been developed. The latter, based on computer vision, are nevertheless facing some limitations and difficulties that are related to the point of view, lighting, occlusions, etc. Artificial Neural Networks (ANN) have been introduced to solve some of these limitations. The latter give satisfactory results, but still have not solved all the problems such as camera angle, the position of the head and, the occlusions, etc. In this paper, we review models of neural networks used in the field of recognition of facial emotions. We also propose an architecture based on the bilinear pooling in order to improve the results obtained by previous works and to provide solutions to solve these recurring constraints. This technique greatly improves the results obtained by architectures based on conventional CNNs.

International audience; Emotions taint our life and allow expressing the different facets of the personality. Among the expressions of the human body, facial ones are the most representative of the mindscape of a person. Several works are devoted to it and applications have already been developed. The latter, based on computer vision, are nevertheless facing some limitations and difficulties that are related to the point of view, lighting, occlusions, etc. Artificial Neural Networks (ANN) have been introduced to solve some of these limitations. The latter give satisfactory results, but still have not solved all the problems such as camera angle, the position of the head and, the occlusions, etc. In this paper, we review models of neural networks used in the field of recognition of facial emotions. We also propose an architecture based on the bilinear pooling in order to improve the results obtained by previous works and to provide solutions to solve these recurring constraints. This technique greatly improves the results obtained by architectures based on conventional CNNs.

International audience; Dans cet article, nous présentons Deepzzle, une méthode de résolution de puzzles reposant sur l'apprentissage automatique, adaptée au réassemblage d'objets patrimoniaux. En effet, ceux-ci subissent des contraintes particulières : érosion, fragments manquants, morceaux mélangés entre objets, etc. Deepzzle apporte une solution à ces problèmes. Abstract-In this paper, we present Deepzzle, a puzzle-solving method based on deep learning. Deepzzle is able to handle efficiently cultural heritage constraints: heritage collections suffer from erosion, missing fragments and mixed puzzles.

In this work, we present a unified multimodal neural network for pose estimation from RGB images and action recognition from video sequences. We show that a multimodal approach benefits 3D pose estimation by mixing high precision 3D data and “in the wild” 2D annotated images, while action recognition also benefits from better visual features. Furthermore, we demonstrate by our experiments that end-to-end optimization results in better performance for action recognition than separated learning. We reported state-of-the-art results on 3D pose estimation and action recognition respectively on Human3.6M and NTU RGB+D datasets., Dans cet article, nous présentons un réseaux de neurones multimodal pour l’estimation de pose et la reconnaissance multimodale favorise l’estimation de pose en combinant des données 3D très précises et des données 2D capturées dans de conditions réelles, ce que favorise aussil’apprentissage de caractéristiques visuelles génériques. Nous montrons que l’optimisation multi-objectif et multimodale du réseau entraîne de meilleurs résultats que l’optimisation séparée de chaque objectif mono-modal. Ceci nous permet de rapporter des résultats au niveau de l’état de l’art pour l’estimation de pose (Human3.6M) et pour la reconnaissance d’action (NTU RGB+D).

International audience; The need for efficient tools to index and retrieve 3D content becomes even more acute. This paper presents a fully automatic 3D-object retrieval method. It consists of two main steps namely shape signature extraction to describe the shape of objects, and similarity computing to compute similarity between objects. In the first step (signature extraction), we use a shape descriptor called geodesic cords. This descriptor can be seen as a probability distribution sampled from a shape function. In the second step (similarity computing), a global distance, based on belief function theory, is computed between each pairwise of descriptors corresponding respectively to an object query and an object from a given database. Experiments on commonly-used benchmarks demonstrate that our method obtains competitive performance compared to 3D-object retrieval methods from the state-of-the-art.

With the increasing usage of 3D models, the need for efficient shape retrieval techniques became mandatory. In this chapter, several approaches for 3D retrieval and shape recognition are presented. Invariant descriptor issues for 3D static mesh are exposed, before addressing action and face recognition in 3D videos. Then classical 3D-search engine performance evaluation tools are thoroughly explained. Finally, examples of significant applications using 3D retrieval techniques are given.

International audience; During the last decade, a significant attention has been paid, by the computer vision and the computer graphics communities, to three dimensional (3D) object retrieval. Shape retrieval methods can be divided into three main steps: the shape descriptors extraction, the shape signatures and their associated similarity measures, and the machine learning relevance functions. While the first and the last points have vastly been addressed in recent years, in this paper, we focus on the second point; presenting a new 3D object retrieval method using a new coding/pooling technique and powerful 3D shape descriptors extracted from 2D views. For a given 3D shape, the approach extracts a very large and dense set of local descriptors. From these descriptors, we build a new shape signature by aggregating tensor products of visual descriptors. The similarity between 3D models can then be efficiently computed with a simple dot product. We further improve the compactness and discrimination power of the descriptor using local Principal Component Analysis on each cluster of descriptors. Experiments on the SHREC 2012 and the McGill benchmarks show that our approach outperforms the state-of-the-art techniques, including other BoF methods, both in compactness of the representation and in the retrieval performance.

Three dimensional object representations have become an integral part of modern computer graphic applications such as computer-aided design, game development and audio-visual production. At the Meanwhile, the 3D data has also become extremely common in fields such as computer vision, computation geometry, molecular biology and medicine. This is due to the rapid evolution of graphics hardware and software development, particularly the availability of low cost 3D scanners which has greatly facilitated 3D model acquisition, creation and manipulation. Content-based search is a necessary solution for structuring, managing these multimedia data, and browsing within these data collections. In this context, we are looking for a system that can automatically retrieve the 3D-models visually similar to a requested 3D-object. Existing solutions for 3D-shape retrieval and classification suffer from high variability towards shape-preserving transformations like affine or isometric transformations (non-rigid transformations). In this context, the aim of my research is to develop a system that can automatically retrieve quickly and with precision 3D models visually similar to a 3D-object query. The system has to be robust to non-rigid transformation that a shape can undergo.During my PhD thesis: We havedeveloped a novel approach to match 3D objects in the presence of non-rigid transformation and partially similar models. We have proposed to use a new representation of 3D-surfaces using 3D curves extracted around feature points. Tools from shape analysis of curves are applied to analyze and to compare curves of two 3D-surfaces. We have used the belief functions, as fusion technique, to define a global distance between 3D-objects. We have also experimented this technique in the retrieval and classification tasks. We have proposed the use of Bag of Feature techniques in 3D-object retrieval and classification.; Une nouvelle approche pour la mise en correspondance des objets 3D en présence des transformations non-rigides et des modèles partiellement similaires est proposée dans le cadre de cette thèse. L'approche est composée de deux phases. Une première phase pour la description d'objets et une deuxième phase de mesure de similarité. Pour décrire un objet 3D, nous avons choisi une méthode basée sur des descripteurs locaux. La méthode consiste à extraire d'un objet 3D un ensemble de points caractéristiques pour lesquels deux descripteurs locaux sont calculés. Le premier descripteur Geodesic cord descriptor représente la distribution des distances géodésiques entre un point caractéristique et l'ensemble des points de la surface de l'objet 3D. Le deuxième descripteur Curve based descriptor permet de représenter la surface 3D de l'objet par un ensemble de courbes. La forme de ces courbes est analysée à l'aide d'outils issus de la géométrie Riemannienne. Pour mesurer la similarité entre les objets 3D, nous avons utilisé deux techniques différentes dont l'une est basée sur les fonctions de croyance et l'autre est basée sur les sac-de-mots. Afin de valider notre approche nous l'avons adaptée à deux applications différentes à savoir la recherche et la classification d'objets 3D. Les résultats obtenus sur différent benchmarks montrent une efficacité et une pertinence comparés avec les autres méthodes de l'état-de-l'art.

We propose an algorithm for generating novel 3D tree model variations from existing ones via geometric and structural blending. Our approach is to treat botanical trees as elements of a tree-shape space equipped with a proper metric that quantifies geometric and structural deformations. Geodesics, or shortest paths under the metric, between two points in the tree-shape space correspond to optimal deformations that align one tree onto another, including the possibility of expanding, adding, or removing branches and parts. Central to our approach is a mechanism for computing correspondences between trees that have different structures and a different number of branches. The ability to compute geodesics and their lengths enables us to compute continuous blending between botanical trees, which, in turn, facilitates statistical analysis, such as the computation of averages of tree structures. We show a variety of 3D tree models generated with our approach from 3D trees exhibiting complex geometric and structural differences. We also demonstrate the application of the framework in reflection symmetry analysis and symmetrization of botanical trees. Refereed/Peer-reviewed

International audience; State-of-the-art 3D shape classification and retrieval algorithms, hereinafter referred to as shape analysis, are often based on comparing signatures or descriptors that capture the main geometric and topological properties of 3D objects. None of the existing descriptors, however, achieve best performance on all shape classes. In this article, we explore, for the first time, the usage of covariance matrices of descriptors, instead of the descriptors themselves, in 3D shape analysis. Unlike histogram -based techniques, covariance-based 3D shape analysis enables the fusion and encoding of different types of features and modalities into a compact representation. Covariance matrices, however, are elements of the non-linear manifold of symmetric positive definite (SPD) matrices and thus \BBL2 metrics are not suitable for their comparison and clustering. In this article, we study geodesic distances on the Riemannian manifold of SPD matrices and use them as metrics for 3D shape matching and recognition. We then: (1) introduce the concepts of bag of covariance (BoC) matrices and spatially-sensitive BoC as a generalization to the Riemannian manifold of SPD matrices of the traditional bag of features framework, and (2) generalize the standard kernel methods for supervised classification of 3D shapes to the space of covariance matrices. We evaluate the performance of the proposed BoC matrices framework and covariance -based kernel methods and demonstrate their superiority compared to their descriptor-based counterparts in various 3D shape matching, retrieval, and classification setups.

International audience; The intent of 3D-models classification is to find categories of similar objects according to their shapes. This task is a challenging and important problem in 3D-mining and shape processing. In this paper, we present a novel method to categorize 3D-objects based on view-based descriptors. The proposed method goes into two stages. The first stage corresponds to the training in which 3D-objects in the same category are processed and a set of representative 2D views is selected, The second stage corresponds to the labelling in which unknown objects are classified using a belief based classifier. The experimental results obtained on the Shrec07 datasets show that the system efficiently performs in categorizing 3D-models.

International audience; Several descriptors have been proposed in the past for 3D shape analysis, yet none of them achieves best perfor- mance on all shape classes. In this paper we propose a novel method for 3D shape analysis using the covariance matrices of the descriptors rather than the descriptors them- selves. Covariance matrices enable efficient fusion of dif- ferent types of features and modalities. They capture, using the same representation, not only the geometric and the spa- tial properties of a shape region but also the correlation of these properties within the region. Covariance matrices, however, lie on the manifold of Symmetric Positive Defi- nite (SPD) tensors, a special type of Riemannian manifolds, which makes comparison and clustering of such matrices challenging. In this paper we study covariance matrices in their native space and make use of geodesic distances on the manifold as a dissimilarity measure. We demonstrate the performance of this metric on 3D face matching and recognition tasks. We then generalize the Bag of Features paradigm, originally designed in Euclidean spaces, to the Riemannian manifold of SPD matrices. We propose a new clustering procedure that takes into account the geometry of the Riemannian manifold. We evaluate the performance of the proposed Bag of Covariance Matrices framework on 3D shape matching and retrieval applications and demonstrate its superiority compared to descriptor-based techniques.

BMVC 2013; International audience; Elastic shape analysis on non-linear Riemannian manifolds provides an efficient and elegant way for simultaneous comparison and registration of non-rigid shapes. In such formulation, shapes become points on some high dimensional shape space. A geodesic between two points corresponds to the optimal deformation needed to register one shape onto another. The length of the geodesic provides a proper metric for shape comparison. However, the computation of geodesics, and therefore the metric, is computationally very expensive as it involves a search over the space of all possible rotations and re- parameterization. This problem is even more important in shape retrieval scenarios where the query shape is compared to every element in the collection to search. In this paper, we propose a new procedure for metric approximation using the framework of kernel functions. We will demonstrate that this provides a fast approximation of the metric while preserving its invariance properties.