Search

Your search keyword '"Parashar, Shaifali"' showing total 25 results
Start Over Author "Parashar, Shaifali"
25 results on '"Parashar, Shaifali"'

1. GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping

Author

Chen, Ruochen, Chen, Liming, and Parashar, Shaifali

Subjects
Computer Science - Computer Vision and Pattern Recognition, Computer Science - Graphics, Computer Science - Machine Learning
Abstract

Recent neural, physics-based modeling of garment deformations allows faster and visually aesthetic results as opposed to the existing methods. Material-specific parameters are used by the formulation to control the garment inextensibility. This delivers unrealistic results with physically implausible stretching. Oftentimes, the draped garment is pushed inside the body which is either corrected by an expensive post-processing, thus adding to further inconsistent stretching; or by deploying a separate training regime for each body type, restricting its scalability. Additionally, the flawed skinning process deployed by existing methods produces incorrect results on loose garments. In this paper, we introduce a geometrical constraint to the existing formulation that is collision-aware and imposes garment inextensibility wherever possible. Thus, we obtain realistic results where draped clothes stretch only while covering bigger body regions. Furthermore, we propose a geometry-aware garment skinning method by defining a body-garment closeness measure which works for all garment types, especially the loose ones.

Published
2023

2. Temporally-Consistent Surface Reconstruction using Metrically-Consistent Atlases

Author

Bednarik, Jan, Aigerman, Noam, Kim, Vladimir G., Chaudhuri, Siddhartha, Parashar, Shaifali, Salzmann, Mathieu, and Fua, Pascal

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

We propose a method for unsupervised reconstruction of a temporally-consistent sequence of surfaces from a sequence of time-evolving point clouds. It yields dense and semantically meaningful correspondences between frames. We represent the reconstructed surfaces as atlases computed by a neural network, which enables us to establish correspondences between frames. The key to making these correspondences semantically meaningful is to guarantee that the metric tensors computed at corresponding points are as similar as possible. We have devised an optimization strategy that makes our method robust to noise and global motions, without a priori correspondences or pre-alignment steps. As a result, our approach outperforms state-of-the-art ones on several challenging datasets. The code is available at https://github.com/bednarikjan/temporally_coherent_surface_reconstruction., Comment: 21 pages. arXiv admin note: substantial text overlap with arXiv:2104.06950

Published
2021

3. Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases

Author

Bednarik, Jan, Kim, Vladimir G., Chaudhuri, Siddhartha, Parashar, Shaifali, Salzmann, Mathieu, Fua, Pascal, and Aigerman, Noam

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

We propose a method for the unsupervised reconstruction of a temporally-coherent sequence of surfaces from a sequence of time-evolving point clouds, yielding dense, semantically meaningful correspondences between all keyframes. We represent the reconstructed surface as an atlas, using a neural network. Using canonical correspondences defined via the atlas, we encourage the reconstruction to be as isometric as possible across frames, leading to semantically-meaningful reconstruction. Through experiments and comparisons, we empirically show that our method achieves results that exceed that state of the art in the accuracy of unsupervised correspondences and accuracy of surface reconstruction., Comment: ICCV 2021

Published
2021

4. A Closed-Form Solution to Local Non-Rigid Structure-from-Motion

Author

Parashar, Shaifali, Long, Yuxuan, Salzmann, Mathieu, and Fua, Pascal

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

A recent trend in Non-Rigid Structure-from-Motion (NRSfM) is to express local, differential constraints between pairs of images, from which the surface normal at any point can be obtained by solving a system of polynomial equations. The systems of equations derived in previous work, however, are of high degree, having up to five real solutions, thus requiring a computationally expensive strategy to select a unique solution. Furthermore, they suffer from degeneracies that make the resulting estimates unreliable, without any mechanism to identify this situation. In this paper, we show that, under widely applicable assumptions, we can derive a new system of equation in terms of the surface normals whose two solutions can be obtained in closed-form and can easily be disambiguated locally. Our formalism further allows us to assess how reliable the estimated local normals are and, hence, to discard them if they are not. Our experiments show that our reconstructions, obtained from two or more views, are significantly more accurate than those of state-of-the-art methods, while also being faster.

Published
2020

5. Robust Isometric Non-Rigid Structure-from-Motion

Author

Parashar, Shaifali, Bartoli, Adrien, and Pizarro, Daniel

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Non-Rigid Structure-from-Motion (NRSfM) reconstructs a deformable 3D object from the correspondences established between monocular 2D images. Current NRSfM methods lack statistical robustness, which is the ability to cope with correspondence errors.This prevents one to use automatically established correspondences, which are prone to errors, thereby strongly limiting the scope of NRSfM. We propose a three-step automatic pipeline to solve NRSfM robustly by exploiting isometry. Step 1 computes the optical flow from correspondences, step 2 reconstructs each 3D point's normal vector using multiple reference images and integrates them to form surfaces with the best reference and step 3 rejects the 3D points that break isometry in their local neighborhood. Importantly, each step is designed to discard or flag erroneous correspondences. Our contributions include the robustification of optical flow by warp estimation, new fast analytic solutions to local normal reconstruction and their robustification, and a new scale-independent measure of 3D local isometric coherence. Experimental results show that our robust NRSfM method consistently outperforms existing methods on both synthetic and real datasets., Comment: Accepted in TPAMI 2021

Published
2020

6. GarNet++: Improving Fast and Accurate Static3D Cloth Draping by Curvature Loss

Author

Gundogdu, Erhan, Constantin, Victor, Parashar, Shaifali, Seifoddini, Amrollah, Dang, Minh, Salzmann, Mathieu, and Fua, Pascal

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

In this paper, we tackle the problem of static 3D cloth draping on virtual human bodies. We introduce a two-stream deep network model that produces a visually plausible draping of a template cloth on virtual 3D bodies by extracting features from both the body and garment shapes. Our network learns to mimic a Physics-Based Simulation (PBS) method while requiring two orders of magnitude less computation time. To train the network, we introduce loss terms inspired by PBS to produce plausible results and make the model collision-aware. To increase the details of the draped garment, we introduce two loss functions that penalize the difference between the curvature of the predicted cloth and PBS. Particularly, we study the impact of mean curvature normal and a novel detail-preserving loss both qualitatively and quantitatively. Our new curvature loss computes the local covariance matrices of the 3D points, and compares the Rayleigh quotients of the prediction and PBS. This leads to more details while performing favorably or comparably against the loss that considers mean curvature normal vectors in the 3D triangulated meshes. We validate our framework on four garment types for various body shapes and poses. Finally, we achieve superior performance against a recently proposed data-driven method., Comment: Accepted to be published in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), July 2020. arXiv admin note: text overlap with arXiv:1811.10983

Published
2020

7. Shape Reconstruction by Learning Differentiable Surface Representations

Author

Bednarik, Jan, Parashar, Shaifali, Gundogdu, Erhan, Salzmann, Mathieu, and Fua, Pascal

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the deformations of the surface patches that form the ensemble and thus fail to prevent them from either overlapping or collapsing into single points or lines. As a consequence, computing shape properties such as surface normals and curvatures becomes difficult and unreliable. In this paper, we show that we can exploit the inherent differentiability of deep networks to leverage differential surface properties during training so as to prevent patch collapse and strongly reduce patch overlap. Furthermore, this lets us reliably compute quantities such as surface normals and curvatures. We will demonstrate on several tasks that this yields more accurate surface reconstructions than the state-of-the-art methods in terms of normals estimation and amount of collapsed and overlapped patches., Comment: 14 pages

Published
2019

8. DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences

Author

Lamarca, Jose, Parashar, Shaifali, Bartoli, Adrien, and Montiel, J. M. M.

Subjects
Computer Science - Computer Vision and Pattern Recognition, Electrical Engineering and Systems Science - Image and Video Processing
Abstract

Monocular SLAM algorithms perform robustly when observing rigid scenes, however, they fail when the observed scene deforms, for example, in medical endoscopy applications. We present DefSLAM, the first monocular SLAM capable of operating in deforming scenes in real-time. Our approach intertwines Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) techniques to deal with the exploratory sequences typical of SLAM. A deformation tracking thread recovers the pose of the camera and the deformation of the observed map, at frame rate, by means of SfT processing a template that models the scene shape-at-rest. A deformation mapping thread runs in parallel with the tracking to update the template, at keyframe rate, by means of an isometric NRSfM processing a batch of full perspective keyframes. In our experiments, DefSLAM processes close-up sequences of deforming scenes, both in a laboratory controlled experiment and in medical endoscopy sequences, producing accurate 3D models of the scene with respect to the moving camera., Comment: Experiments results: https://www.youtube.com/watch?v=6mmhD2_t6Gs ; More Results: https://www.youtube.com/playlist?list=PLKBuKNhAV30SlKGJ9eaMlAExdWRypUy-K

Published
2019

10. Self-Calibrating Isometric Non-Rigid Structure-from-Motion

Author

Parashar, Shaifali, Bartoli, Adrien, Pizarro, Daniel, Hutchison, David, Series Editor, Kanade, Takeo, Series Editor, Kittler, Josef, Series Editor, Kleinberg, Jon M., Series Editor, Mattern, Friedemann, Series Editor, Mitchell, John C., Series Editor, Naor, Moni, Series Editor, Pandu Rangan, C., Series Editor, Steffen, Bernhard, Series Editor, Terzopoulos, Demetri, Series Editor, Tygar, Doug, Series Editor, Weikum, Gerhard, Series Editor, Ferrari, Vittorio, editor, Hebert, Martial, editor, Sminchisescu, Cristian, editor, and Weiss, Yair, editor

Published
2018
Full Text
View/download PDF

19. 3DVFX: 3D Video Editing using Non-Rigid Structure-from-Motion

Author

Parashar, Shaifali and Bartoli, Adrien

Subjects
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION, Mixed / augmented reality, Reconstruction, Texturing, Computing methodologies, ComputingMethodologies_COMPUTERGRAPHICS
Abstract

Numerous video post-processing techniques can add or remove objects to the observed scene in the video. Most of these techniques rely on 2D image points to perform the desired changes. Structure-from-Motion (SfM) has allowed the use of 3D points, however only for the objects that remain rigid in the scene. We propose to use both 2D image points and 3D points to modify the scene's deformable objects using Non-Rigid Structure-from-Motion (NRSfM). We rely on a recent effective NRSfM solution to develop a complete pipeline including manual 3D editing of an image and automatic 3D transfer of the edits. We perform object manipulation tasks such as retexturing a real deforming object., Eurographics 2019 - Short Papers, Image and Video, 29, 32, Shaifali Parashar and Adrien Bartoli, CCS Concepts: Computing methodologies --> Reconstruction; Texturing; Mixed / augmented reality

Published
2019
Full Text
View/download PDF

20. Image-based deformable 3D reconstruction using differential geometry and cartan's connections

Author

Parashar, Shaifali, parashar, shaifali, Institut Pascal (IP), SIGMA Clermont (SIGMA Clermont)-Université Clermont Auvergne [2017-2020] (UCA [2017-2020])-Centre National de la Recherche Scientifique (CNRS), Université Clermont Auvergne, Adrien Bartoli, Daniel Pizarro (co-encadrant), and Université Clermont Auvergne [2017-2020]

Subjects
Vision par Ordinateur, Géométrie différentielle, Reconstruction 3D, Connections, Connexions, [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], Cartan, computer vision, NRSfM, [INFO.INFO-CV] Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], differential geometry, Cartan connection, SfT, 3D Reconstruction
Abstract

Reconstructing the 3D shape of objects from multiple images is an important goal in computer vision and has been extensively studied for both rigid and non-rigid (or deformable) objects. Structure-from-Motion (SfM) is an algorithm that performs the 3D reconstruction of rigid objects using the inter-image visual motion from multiple images obtained from a moving camera. SfM is a very accurate and stable solution. Deformable 3D reconstruction, however, has been widely studied for monocular images (obtained from a single camera) and still emains an open research problem. The current methods exploit visual cues such as the inter-image visual motion and shading in order to formalise a reconstruction algorithm. This thesis focuses on the use of the inter-image visual motion for solving this problem. Twotypes of scenarios exist in the literature: 1) Non-Rigid Structure-from-Motion (NRSfM) and 2) Shape-from-Template (SfT). The goal of NRSfM is to reconstruct multiple shapes of a deformable object as viewed in multiple images while SfT (also referred to as template-based reconstruction) uses a single image of a deformed object and its 3D template (a textured 3D shape of the object in one configuration) to recover the deformed shape of the object.We propose an NRSfM method to reconstruct the deformable surfaces undergoing isometric deformations (the objects do not stretch or shrink under an isometric deformation) using Riemannian geometry. This allows NRSfM to be expressed in terms of Partial Differential Equations (PDE) and to be solved algebraically. We show that the problem has linear complexity and the reconstruction algorithm has a very low computational cost compared to existing NRSfM methods. This work motivated us to use differential geometry and Cartan’s theory of connections to model NRSfM, which led to the possibility of extending the solution to deformations other than isometry. In fact, this led to a unified theoretical framework for modelling and solving both NRSfM and SfT for various types of deformations. In addition, italso makes it possible to have a solution to SfT which does not require an explicit modelling of deformation. An important point is that most of the NRSfM and SfT methods reconstruct the thin-shell surface of the object. The reconstruction of the entire volume (the thin-shell surface and the interior) has not been explored yet. We propose the first SfT method that reconstructs the entire volume of a deformable object., La reconstruction 3D d’objets à partir de plusieurs images est un objectif important de la vision par ordinateur. Elle a été largement étudiée pour les objets rigides et non rigides (ou déformables). Le Structure-from-Motion (SfM) est un algorithme qui effectue la reconstruction 3D d’objets rigides en utilisant le mouvement visuel entre plusieurs images obtenues à l’aide d’une caméra en mouvement. Le SfM est une solution très précise et stable. La reconstruction 3D déformable a été largement étudiée pour les images monoculaires (obtenues à partir d’une seule caméra) mais reste un problème ouvert. Les méthodes actuelles exploitent des indices visuels tels que le mouvement visuel inter-image et l’ombrage afin de construire un algorithme de reconstruction. Cette thèse se concentre sur l’utilisation du mouvement visuel inter-image pour résoudre ce problème. Deux types de scénarios existent dans la littérature: 1) le Non-Rigid Structure-from-Motion (NRSfM) et 2) le Shape-from-Template (SfT). L’objectif du NRSfM est de reconstruire plusieurs formes d’un objet déformable tel qu’il apparaı̂t dans plusieurs images, alors que le SfT (également appelé reconstruction à partir d’un modèle de référence) utilise une seule image d’un objet déformé et son modèle 3D de référence (une forme 3D texturée de l’objet dans une configuration) pour estimer la forme déformée de l’objet.Nous proposons une méthode de NRSfM pour reconstruire les surfaces déformables soumises à des déformations isométriques (les objets ne s’étirent pas ou ne se contractent pas sous une déformation isométrique) en utilisant la géométrie riemannienne. Cela permet d’exprimer le NRSfM en termes d’équations aux dérivées partielles et de le résoudre algébriquement. Nous montrons que le problème a une complexité linéaire et que l’algorithme de reconstruction proposé a un coût de calcul très bas comparé aux méthodes existantes de NRSfM. Ce travail nous a motivé à utiliser la géométrie différentielle et la théorie des connexions de Cartan pour modéliser le NRSfM, ce qui nous a permis d’étendre la solution à des déformations autres que l’isométrie. En fait, cela a conduit à un cadre théorique unifié pour modéliser et résoudre le NRSfM et le SfT pour différents types de déformations. Ce cadre permet également d’avoir une solution au SfT qui ne nécessite pas de modélisation explicite de la déformation. Un point important est que la plupart des méthodes de NRSfM et de SfT reconstruisent la surface de l’objet (hypothèse coque mince). La reconstruction de l’ensemble d’un volume (la surface et l’intérieur d’un objet) n’avait pas encore été explorée. Nous proposons la première méthode de SfT qui reconstruit le volume complet d’un objet déformable.

Published
2017

21. Reconstruction 3D déformable basée sur l'image utilisant la géométrie différentielle et les connexions de cartan

Author

Parashar, Shaifali, Institut Pascal (IP), SIGMA Clermont (SIGMA Clermont)-Université Clermont Auvergne [2017-2020] (UCA [2017-2020])-Centre National de la Recherche Scientifique (CNRS), Université Clermont Auvergne [2017-2020], and Adrien Bartoli

Subjects
NRSfM, Géométrie différentielle, Vision par ordinateur, Reconstruction 3D, Connections, Connexions, [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], Differential geometry, Computer vision, 3D reconstruction, SfT, Cartan
Abstract

Reconstructing the 3D shape of objects from multiple images is an important goal in computer vision and has been extensively studied for both rigid and non-rigid (or deformable) objects. Structure-from-Motion (SfM) is an algorithm that performs the 3D reconstruction of rigid objects using the inter-image visual motion from multiple images obtained from a moving camera. SfM is a very accurate and stable solution. Deformable 3D reconstruction, however, has been widely studied for monocular images (obtained from a single camera) and still remains an open research problem. The current methods exploit visual cues such as the inter-image visual motion and shading in order to formalise a reconstruction algorithm. This thesis focuses on the use of the inter-image visual motion for solving this problem. Two types of scenarios exist in the literature: 1) Non-Rigid Structure-from-Motion (NRSfM) and 2) Shape-from-Template (SfT). The goal of NRSfM is to reconstruct multiple shapes of a deformable object as viewed in multiple images while SfT (also referred to as template-based reconstruction) uses a single image of a deformed object and its 3D template (a textured 3D shape of the object in one configuration) to recover the deformed shape of the object. We propose an NRSfM method to reconstruct the deformable surfaces undergoing isometric deformations (the objects do not stretch or shrink under an isometric deformation) using Riemannian geometry. This allows NRSfM to be expressed in terms of Partial Differential Equations (PDE) and to be solved algebraically. We show that the problem has linear complexity and the reconstruction algorithm has a very low computational cost compared to existing NRSfM methods. This work motivated us to use differential geometry and Cartan’s theory of connections to model NRSfM, which led to the possibility of extending the solution to deformations other than isometry. In fact, this led to a unified theoretical framework for modelling and solving both NRSfM and SfT for various types of deformations. In addition, it also makes it possible to have a solution to SfT which does not require an explicit modelling of deformation. An important point is that most of the NRSfM and SfT methods reconstruct the thin-shell surface of the object. The reconstruction of the entire volume (the thin-shell surface and the interior) has not been explored yet. We propose the first SfT method that reconstructs the entire volume of a deformable object.; La reconstruction 3D d’objets à partir de plusieurs images est un objectif important de la vision par ordinateur. Elle a été largement étudiée pour les objets rigides et non rigides (ou déformables). Le Structure-from-Motion (SfM) est un algorithme qui effectue la reconstruction 3D d’objets rigides en utilisant le mouvement visuel entre plusieurs images obtenues à l’aide d’une caméra en mouvement. Le SfM est une solution très précise et stable. La reconstruction 3D déformable a été largement étudiée pour les images monoculaires (obtenues à partir d’une seule caméra) mais reste un problème ouvert. Les méthodes actuelles exploitent des indices visuels tels que le mouvement visuel inter-image et l’ombrage afin de construire un algorithme de reconstruction. Cette thèse se concentre sur l’utilisation du mouvement visuel inter-image pour résoudre ce problème. Deux types de scénarios existent dans la littérature : 1) le Non-Rigid Structure-from-Motion (NRSfM) et 2) le Shape-from-Template (SfT). L’objectif du NRSfM est de reconstruire plusieurs formes d’un objet déformable tel qu’il apparaît dans plusieurs images, alors que le SfT (également appelé reconstruction à partir d’un modèle de référence) utilise une seule image d’un objet déformé et son modèle 3D de référence (une forme 3D texturée de l’objet dans une configuration) pour estimer la forme déformée de l’objet. (...)

Published
2017

25. A Closed-Form, Pairwise Solution to Local Non-Rigid Structure-From-Motion.

Author

Parashar S, Long Y, Salzmann M, and Fua P

Abstract

A recent trend in Non-Rigid Structure-from-Motion (NRSfM) is to express local, differential constraints between pairs of images, from which the surface normal at any point can be obtained by solving a system of polynomial equations. While this approach is more successful than its counterparts relying on global constraints, the resulting methods face two main problems: First, most of the equation systems they formulate are of high degree and must be solved using computationally expensive polynomial solvers. Some methods use polynomial reduction strategies to simplify the system, but this adds some phantom solutions. In any event, an additional mechanism is employed to pick the best solution, which adds to the computation without any guarantees on the reliability of the solution. Second, these methods formulate constraints between a pair of images. Even if there is enough motion between them, they may suffer from local degeneracies that make the resulting estimates unreliable without any warning mechanism. In this paper, we solve these problems for isometric/conformal NRSfM. We show that, under widely applicable assumptions, we can derive a new system of equations in terms of the surface normals, whose two solutions can be obtained in closed-form and can easily be disambiguated locally. Our formalism also allows us to assess how reliable the estimated local normals are and to discard them if they are not. Our experiments show that our reconstructions, obtained from two or more views, are significantly more accurate than those of state-of-the-art methods, while also being faster.

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results