Search

Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem. The gap between the Euclidean network output space and the non-Euclidean SO(3) manifold imposes a severe challenge for neural network learning in both forward and backward passes. While several works have proposed different regression-friendly rotation representations, very few works have been devoted to improving the gradient backpropagating in the backward pass. In this paper, we propose a manifold-aware gradient that directly backpropagates into deep network weights. Leveraging Riemannian optimization to construct a novel projective gradient, our proposed regularized projective manifold gradient (RPMG) method helps networks achieve new state-of-the-art performance in a variety of rotation estimation tasks. Our proposed gradient layer can also be applied to other smooth manifolds such as the unit sphere. Our project page is at https://jychen18.github.io/RPMG., CVPR2022

We present a novel, end-to-end learnable, multiview 3D point cloud registration algorithm. Registration of multiple scans typically follows a two-stage pipeline: the initial pairwise alignment and the globally consistent refinement. The former is often ambiguous due to the low overlap of neighboring point clouds, symmetries and repetitive scene parts. Therefore, the latter global refinement aims at establishing the cyclic consistency across multiple scans and helps in resolving the ambiguous cases. In this paper we propose, to the best of our knowledge, the first end-to-end algorithm for joint learning of both parts of this two-stage problem. Experimental evaluation on well accepted benchmark datasets shows that our approach outperforms the state-of-the-art by a significant margin, while being end-to-end trainable and computationally less costly. Moreover, we present detailed analysis and an ablation study that validate the novel components of our approach. The source code and pretrained models are publicly available under https://github.com/zgojcic/3D_multiview_reg., 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ISBN:978-1-7281-7168-5, ISBN:978-1-7281-7169-2

In this survey we carefully analyze the characteristics of higher order rigid body motion interpolation methods to obtain a continuous trajectory from a discrete set of poses. We first discuss the tradeoff between continuity, local control and approximation of classical Euclidean interpolation schemes such as Bezier and B-splines. The benefits of the manifold of unit quaternions SU(2), a double-cover of rotation matrices SO(3), as rotation parameterization are presented, which allow for an elegant formulation of higher order orientation interpolation with easy analytic derivatives, made possible through the Lie Algebra su(2) of pure quaternions and the cumulative form of cubic B-splines. The same construction scheme is then applied for joint interpolation in the full rigid body pose space, which had previously been done for the matrix representation SE(3) and its twists, but not for the more efficient unit dual quaternion DH1 and its screw motions. Both suffer from the effects of coupling translation and rotation that have mostly been ignored by previous work. We thus conclude that split interpolation in R3 × SU(2) is preferable for most applications. Our final runtime experiments show that joint interpolation in SE(3) is 2 times and in DH1 1.3 times slower - which furthermore justifies our suggestion from a practical point of view.

This paper proposes a segmentation-free, automatic and efficient procedure to detect general geometric quadric forms in point clouds, where clutter and occlusions are inevitable. Our everyday world is dominated by man-made objects which are designed using 3D primitives (such as planes, cones, spheres, cylinders, etc.). These objects are also omnipresent in industrial environments. This gives rise to the possibility of abstracting 3D scenes through primitives, thereby positions these geometric forms as an integral part of perception and high level 3D scene understanding. As opposed to state-of-the-art, where a tailored algorithm treats each primitive type separately, we propose to encapsulate all types in a single robust detection procedure. At the center of our approach lies a closed form 3D quadric fit, operating in both primal & dual spaces and requiring as low as 4 oriented-points. Around this fit, we design a novel, local null-space voting strategy to reduce the 4-point case to 3. Voting is coupled with the famous RANSAC and makes our algorithm orders of magnitude faster than its conventional counterparts. This is the first method capable of performing a generic cross-type multi-object primitive detection in difficult scenes. Results on synthetic and real datasets support the validity of our method., Accepted for publication at CVPR 2018

We present PPFNet - Point Pair Feature NETwork for deeply learning a globally informed 3D local feature descriptor to find correspondences in unorganized point clouds. PPFNet learns local descriptors on pure geometry and is highly aware of the global context, an important cue in deep learning. Our 3D representation is computed as a collection of point-pair-features combined with the points and normals within a local vicinity. Our permutation invariant network design is inspired by PointNet and sets PPFNet to be ordering-free. As opposed to voxelization, our method is able to consume raw point clouds to exploit the full sparsity. PPFNet uses a novel $\textit{N-tuple}$ loss and architecture injecting the global information naturally into the local descriptor. It shows that context awareness also boosts the local feature representation. Qualitative and quantitative evaluations of our network suggest increased recall, improved robustness and invariance as well as a vital step in the 3D descriptor extraction performance., Comment: Accepted for publication at CVPR 2018

With the profiliferation of 3D sensors, 3D data analysis has become an important tool. Due to its capability of data summarization, sampling has taken a critical part of this analysis. In this paper, we argue that finding a universally good sampling is difficult, if not impossible and rather propose a task-tailored solution. In the scope of this work, we address the task of Iterative Closest Point (ICP) registration and cast the point selection procedure as an optimization problem over the samples, minimizing the ICP energy. Because the parametrization over 3D points results in a discrete and large optimization problem, we benefit from genetic algorithms, which are proven to evolve into better solutions, at each time-step. The quantitative and qualitative results clearly demonstrate that we are able to obtain better results than the state-of-the-art.

Time-varying, smooth trajectory estimation is of great interest to the vision community for accurate and well behaving 3D systems. In this paper, we propose a novel principal component local regression filter acting directly on the Riemannian manifold of unit dual quaternions $\mathbb{D} \mathbb{H}_1$. We use a numerically stable Lie algebra of the dual quaternions together with $\exp$ and $\log$ operators to locally linearize the 6D pose space. Unlike state of the art path smoothing methods which either operate on $SO\left(3\right)$ of rotation matrices or the hypersphere $\mathbb{H}_1$ of quaternions, we treat the orientation and translation jointly on the dual quaternion quadric in the 7-dimensional real projective space $\mathbb{R}\mathbb{P}^7$. We provide an outlier-robust IRLS algorithm for generic pose filtering exploiting this manifold structure. Besides our theoretical analysis, our experiments on synthetic and real data show the practical advantages of the manifold aware filtering on pose tracking and smoothing.

We present a 3D mesh re-sampling algorithm, carefully tailored for 3D object detection using point pair features (PPF). Computing a sparse representation of objects is critical for the success of state-of-the-art object detection, recognition and pose estimation methods. Yet, sparsity needs to preserve fidelity. To this end, we develop a simple, yet very effective point sampling strategy for detection of any CAD model through geometric hashing. Our approach relies on rendering the object coordinates from a set of views evenly distributed on a sphere. Actual sampling takes place on 2D domain over these renderings; the resulting samples are efficiently merged in 3D with the aid of a special voxel structure and relaxed with Lloyd iterations. The generated vertices are not concentrated only on critical points, as in many keypoint extraction algorithms, and there is even spacing between selected vertices. This is valuable for quantization based detection methods, such as geometric hashing of point pair features. The algorithm is fast and can easily handle the elongated/acute triangles and sharp edges typically existent in industrial CAD models, while automatically pruning the invisible structures. We do not introduce structural changes such as smoothing or interpolation and sample the normals on the original CAD model, achieving the maximum fidelity. We demonstrate the strength of this approach on 3D object detection in comparison to similar sampling algorithms.

In this paper we design a novel planar 2D fiducial marker and develop fast detection algorithm aiming easy camera calibration and precise 3D reconstruction at the marker locations via the bundle adjustment. Even though an abundance of planar fiducial markers have been made and used in various tasks, none of them has properties necessary to solve the aforementioned tasks. Our marker, X-tag, enjoys a novel design, coupled with very efficient and robust detection scheme, resulting in a reduced number of false positives. This is achieved by constructing markers with random circular features in the image domain and encoding them using two true perspective invariants: cross-ratios and intersection preservation constraints. To detect the markers, we developed an effective search scheme, similar to Geometric Hashing and Hough Voting, in which the marker decoding is cast as a retrieval problem. We apply our system to the task of camera calibration and bundle adjustment. With qualitative and quantitative experiments, we demonstrate the robustness and accuracy of X-tag in spite of blur, noise, perspective and radial distortions, and showcase camera calibration, bundle adjustment and 3d fusion of depth data from precise extrinsic camera poses.

The raising standards in manufacturing demands reliable and fast industrial quality control mechanisms. This paper proposes an accurate, yet easy to install multi-view, close range optical metrology system, which is suited to online operation. The system is composed of multiple static, locally overlapping cameras forming a network. Initially, these cameras are calibrated to obtain a global coordinate frame. During run-time, the measurements are performed via a novel geometry extraction techniques coupled with an elegant projective registration framework, where 3D to 2D fitting energies are minimized. Finally, a non-linear regression is carried out to compensate for the uncontrollable errors. We apply our pipeline to inspect various geometrical structures found on automobile parts. While presenting the implementation of an involved 3D metrology system, we also demonstrate that the resulting inspection is as accurate as 0.2 mm, repeatable and much faster, compared to the existing methods such as coordinate measurement machines (CMM) or ATOS.

We present a revised pipe-line of the existing 3D object detection and pose estimation framework based on point pair feature matching. This framework proposed to represent 3D target object using self-similar point pairs, and then matching such model to 3D scene using efficient Hough-like voting scheme operating on the reduced pose parameter space. Even though this work produces great results and motivated a large number of extensions, it had some general shortcoming like relatively high dimensionality of the search space, sensitivity in establishing 3D correspondences, having performance drops in presence of many outliers and low density surfaces. In this paper, we explain and address these drawbacks and propose new solutions within the existing framework. In particular, we propose to couple the object detection with a coarse-to-fine segmentation, where each segment is subject to disjoint pose estimation. During matching, we apply a weighted Hough voting and an interpolated recovery of pose parameters. Finally, all the generated hypothesis are tested via an occlusion-aware ranking and sorted. We argue that such a combined pipeline simultaneously boosts the detection rate and reduces the complexity, while improving the accuracy of the resulting pose. Thanks to such enhanced pose retrieval, our verification doesn't necessitate ICP and thus achieves better compromise of speed vs. accuracy. We demonstrate our method on existing datasets as well as on our scenes. We conclude that via the new pipe-line, point pair features can now be used in more challenging scenarios.

Both consumer market and manufacturing industry makes heavy use of 1D (linear) barcodes. From helping the visually impaired to identifying the products to industrial automated industry management, barcodes are the prevalent source of item tracing technology. Because of this ubiquitous use, in recent years, many algorithms have been proposed targeting barcode decoding from high-accessibility devices such as cameras. However, the current methods have at least one of the two major problems: 1) they are sensitive to blur, perspective/ lens distortions, and non-linear deformations, which often occur in practice, 2) they are specifically designed for a specific barcode symbology (such as UPC-A) and cannot be applied to other symbologies. In this paper, we aim to address these problems and present a dynamic Bayesian network in order to robustly model all kinds of linear progressive barcodes. We apply our method on various barcode datasets and compare the performance with the state-of-the-art. Our experiments show that, as well as being applicable to all progressive barcode types, our method provides competitive results in clean UPC-A datasets and outperforms the state-of-the-art in difficult scenarios.

Along with the growth of the Internet, automatic recommender systems have become popular. Due to being intuitive and useful, factorization based models, including the Nonnegative Matrix Factorization (NMF) model, are one of the most common approachs for building recommender systems. In this study, we focus on how a recommender system can be built for online services and how the parameters of an NMF model should be selected in a recommender system setting. We first present a general system architecture in which any kind of factorization model can be used. Then, in order to see how accurate the NMF model fits the data, we randomly erase some parts of a real data set that is gathered from an online food ordering service, and we reconstruct the erased parts by using the NMF model. We report the mean squared errors for different parameter settings and different divergences.