Search

Visual Relocalization is a key technology in many computer vision applications. Traditional visual relocalization is mainly achieved through geometric methods, while PoseNet introduces convolutional neural network in visual relocalization for the first time to realize real-time camera pose estimation based on a single image. Aiming at the problem of accuracy and robustness of the current PoseNet algorithm in complex environment, this paper proposes and implements a new high-precision robust camera pose calculation method (LRF-PoseNet). This method directly adjusts the size of the input image without cropping, so as to increase the receptive field of the training image. Then, the image and the corresponding pose tags are input into the improved LSTM-based PoseNet network for training, and the Adam optimizer is used to optimize the network. Finally, the trained network is used to estimate the camera pose. Experimental results on open RGB dataset show that the proposed method in this paper can obtain more accurate camera pose compared with the existing CNN-based methods., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B2-2021, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

ISPRS Technical Commission II focuses on geometric, radiometric and multitemporal aspects of image- and range-based 3D surveying and modeling. Specifically, Commission II focuses on research and developments on image orientation, point cloud generation and processing, 3D feature extraction, dynamic and static scene analysis, sensor fusion, data integration, and machine learning for geospatial data analysis. TC II considers the applications of these focus areas in the fields of mapping, infrastructure monitoring, heritage studies, space exploration, underwater photogrammetry and environmental engineering., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B2-2022, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Deep learning models achieve excellent semantic segmentation results for airborne laser scanning (ALS) point clouds, if sufficient training data are provided. Increasing amounts of annotated data are becoming publicly available thanks to contributors from all over the world. However, models trained on a specific dataset typically exhibit poor performance on other datasets. I.e., there are significant domain shifts, as data captured in different environments or by distinct sensors have different distributions. In this work, we study this domain shift and potential strategies to mitigate it, using two popular ALS datasets: the ISPRS Vaihingen benchmark from Germany and the LASDU benchmark from China. We compare different training strategies for cross-city ALS point cloud semantic segmentation. In our experiments, we analyse three factors that may lead to domain shift and affect the learning: point cloud density, LiDAR intensity, and the role of data augmentation. Moreover, we evaluate a well-known standard method of domain adaptation, deep CORAL (Sun and Saenko, 2016). In our experiments, adapting the point cloud density and appropriate data augmentation both help to reduce the domain gap and improve segmentation accuracy. On the contrary, intensity features can bring an improvement within a dataset, but deteriorate the generalisation across datasets. Deep CORAL does not further improve the accuracy over the simple adaptation of density and data augmentation, although it can mitigate the impact of improperly chosen point density, intensity features, and further dataset biases like lack of diversity., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B2-2021, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

In order to reach the goal of reliably solving Earth monitoring tasks, automated and efficient machine learning methods are necessary for large-scale scene analysis and interpretation. A typical bottleneck of supervised learning approaches is the availability of accurate (manually) labeled training data, which is particularly important to train state-of-the-art (deep) learning methods. We present SemCity Toulouse, a publicly available, very high resolution, multi-spectral benchmark data set for training and evaluation of sophisticated machine learning models. The benchmark acts as test bed for single building instance segmentation which has been rarely considered before in densely built urban areas. Additional information is provided in the form of a multi-class semantic segmentation annotation covering the same area plus an adjacent area 3 times larger. The data set addresses interested researchers from various communities such as photogrammetry and remote sensing, but also computer vision and machine learning.

International audience; We present an algorithm for extracting and vectorizing objects in images with polygons. Departing from a polygonal partition that oversegments an image into convex cells, the algorithm refines the geometry of the partition while labeling its cells by a semantic class. The result is a set of polygons, each capturing an object in the image. The quality of a configuration is measured by an energy that accounts for both the fidelity to input data and the complexity of the output polygons. To efficiently explore the configuration space, we perform splitting and merging operations in tandem on the cells of the polygonal partition. The exploration mechanism is controlled by a priority queue that sorts the operations most likely to decrease the energy. We show the potential of our algorithm on different types of scenes, from organic shapes to man-made objects through floor maps, and demonstrate its efficiency compared to existing vectorization methods.

International audience; Converting point clouds generated by Laser scanning, multiview stereo imagery or depth cameras into compact polygon meshes is a challenging problem in vision. Existing methods are either robust to imperfect data or scalable, but rarely both. In this paper, we address this issue with an hybrid method that successively connects and slices planes detected from 3D data. The core idea consists in constructing an efficient and compact partitioning data structure. The later is i) spatially-adaptive in the sense that a plane slices a restricted number of relevant planes only, and ii) composed of components with different structural meaning resulting from a preliminary analysis of the plane connec-tivity. Our experiments on a variety of objects and sensors show the versatility of our approach as well as its competitiveness with respect to existing methods.

We introduce a pipeline that reconstructs buildings of urban environments as concise polygonal meshes from airborne LiDAR scans. It consists of three main steps: classification, building contouring, and building reconstruction, the two last steps being achieved using computational geometry tools. Our algorithm demonstrates its robustness, flexibility and scalability by producing accurate and compact 3D models over large and varied urban areas in a few minutes only.

International audience; The generation of 3D building models from Very High Resolution geospatial data is now an automatized procedure. However , urban areas are very complex and practitioners still have to visually assess the correctness of these models and detect reconstruction errors. We proposed an approach for automatically evaluating the quality of 3D building models. It is cast as a supervised classification task based on a hierarchical taxon-omy and multimodal handcrafted features (building geometry, optical images, height data). In this paper, we evaluate how the urban area composition impacts prediction transferability and scalability of our framework to unseen scenes. This allows to define minimal feature and training sets for a problem where no benchmark data has been released so far.

Analyzing and extracting geometric features from 3D data is a fundamental step in 3D scene understanding. Recent works demonstrated that deep learning architectures can operate directly on raw point clouds, i.e. without the use of intermediate grid-like structures. These architectures are however not designed to encode contextual information in-between objects efficiently. Inspired by a global feature aggregation algorithm designed for images (Zhao et al., 2017), we propose a 3D pyramid module to enrich pointwise features with multi-scale contextual information. Our module can be easily coupled with 3D semantic segmentation methods operating on 3D point clouds. We evaluated our method on three large scale datasets with four baseline models. Experimental results show that the use of enriched features brings significant improvements to the semantic segmentation of indoor and outdoor scenes.

We present a supervised machine learning approach for classification of objects from sampled point data. The main idea consists in first abstracting the input object into planar parts at several scales, then discriminate between the different classes of objects solely through features derived from these planar shapes. Abstracting into planar shapes provides a means to both reduce the computational complexity and improve robustness to defects inherent to the acquisition process. Measuring statistical properties and relationships between planar shapes offers invariance to scale and orientation. A random forest is then used for solving the multiclass classification problem. We demonstrate the potential of our approach on a set of indoor objects from the Princeton shape benchmark and on objects acquired from indoor scenes and compare the performance of our method with other point-based shape descriptors.; Nous introduisons une approche par apprentissage supervisée pour classifier des objetsà partir de points échantillonnés dans l’espace. L’idée principale consiste à approximer l’objet initialen parties planaires à différentes échelles, pour ensuite distinguer les différentes classes d’objets sanstenir compte des points échantillonnées. L’abstraction en formes planaires est un moyen à la fois deréduire la complexité algorithmique de l’analyse, et d’améliorer la robustesse aux défauts de mesuresdans le processus d’acquisition des données. Mesurer des propriétés statistiques et des relations entreformes planaires offre une invariance à l’échelle et à l’orientation. L’algorithme Random Forest estutilisé pour résoudre le problème de classification multi-classe. Nous démontrons le potentiel de notreapproche sur un ensemble d’objet de scène d’intérieur en utilisant plusieurs benchmarks et en comparantles performances avec des méthodes basées sur des descripteurs locaux de points.

International audience; Recent works showed that floating polygons can be an interesting alternative to traditional superpixels, especially for analyzing scenes with strong geometric signatures , as man-made environments. Existing algorithms produce homogeneously-sized polygons that fail to capture thin geometric structures and over-partition large uniform areas. We propose a kinetic approach that brings more flexibility on polygon shape and size. The key idea consists in progressively extending pre-detected line-segments until they meet each other. Our experiments demonstrate that output partitions both contain less polygons and better capture geometric structures than those delivered by existing methods. We also show the applicative potential of the method when used as preprocessing in object contouring.

International audience; Automatic city modeling from satellite imagery is one of the biggest challenges in urban reconstruction. Existing methods produce at best rough and dense Digital Surface Models. Inspired by recent works on semantic 3D reconstruction and region-based stereovision, we propose a method for producing compact , semantic-aware and geometrically accurate 3D city models from stereo pair of satellite images. Our approach relies on two key ingredients. First, geometry and semantics are retrieved simultaneously bringing robustness to occlusions and to low image quality. Second, we operate at the scale of geometric atomic region which allows the shape of urban objects to be well preserved, and a gain in scalability and efficiency. We demonstrate the potential of our algorithm by reconstructing different cities around the world in a few minutes.

International audience; In this paper we present an update on the geometric modeling of urban scenes from physical measurements. This field of research has been studied for more than thirty years, but remains an important challenge in many scientific communi-ties as photogrammetry, computer vision, robotics or computer graphics. After introducing the objectives and difficulties of urban reconstruction, we present an non-exhaustive overview of the approaches and trends that have inspired the research communities so far. We also propose some new research directions that might be worth investigating in the coming years.

International audience; Many design tasks involve the creation of new objects in the context of an existing scene. Existing work in computer vision only provides partial support for such tasks. On the one hand, multi-view stereo algorithms allow the reconstruction of real-world scenes, while on the other hand algorithms for line-drawing interpretation do not take context into account. Our work combines the strength of these two domains to interpret line drawings of imaginary objects drawn over photographs of an existing scene. The main challenge we face is to identify the existing 3D structure that correlates with the line drawing while also allowing the creation of new structure that is not present in the real world. We propose a labeling algorithm to tackle this problem , where some of the labels capture dominant orientations of the real scene while a free label allows the discovery of new orientations in the imaginary scene. We illustrate our algorithm by interpreting line drawings for urban planing, home remodeling, furniture design and cultural heritage.

International audience; We present a novel and robust method for modeling cities from 3D-point data. Our algorithm pro- vides a more complete description than existing ap- proaches by reconstructing simultaneously buildings, trees and topologically complex grounds. A major con- tribution of our work is the original way of model- ing buildings which guarantees a high generalization level while having semantized and compact represen- tations. Geometric 3D-primitives such as planes, cylin- ders, spheres or cones describe regular roof sections, and are combined with mesh-patches that represent irregu- lar roof components. The various urban components in- teract through a non-convex energy minimization prob- lem in which they are propagated under arrangement constraints over a planimetric map. Our approach is ex- perimentally validated on complex buildings and large urban scenes of millions of points, and is compared to state-of-the-art methods.

International audience; We present a robust method for modeling cities from unstructured point data. Our algorithm provides a more complete description than existing approaches by reconstructing simultaneously buildings, trees and topologically complex grounds. Buildings are modeled by an original approach which guarantees a high generalization level while having semantized and compact representations. Geometric 3D-primitives such as planes, cylinders, spheres or cones describe regular roof sections, and are combined with mesh-patches that represent irregular roof components. The various urban components interact through a non-convex energy minimization problem in which they are propagated under arrangement constraints over a planimetric map. We experimentally validate the approach on complex urban structures and large urban scenes of millions of points.

We introduce a novel approach that reconstructs 3D urban scenes in the form of levels of detail (LODs). Starting from raw datasets such as surface meshes generated by multiview stereo systems, our algorithm proceeds in three main steps: classification, abstraction, and reconstruction. From geometric attributes and a set of semantic rules combined with a Markov random field, we classify the scene into four meaningful classes. The abstraction step detects and regularizes planar structures on buildings, fits icons on trees, roofs, and facades, and performs filtering and simplification for LOD generation. The abstracted data are then provided as input to the reconstruction step which generates watertight buildings through a min-cut formulation on a set of 3D arrangements. Our experiments on complex buildings and large-scale urban scenes show that our approach generates meaningful LODs while being robust and scalable. By combining semantic segmentation and abstraction, it also outperforms general mesh approximation approaches at preserving urban structures.