Cross domain matching for semantic point cloud segmentation based on image segmentation and geometric reasoning.

Many infrastructure assets in transportation such as roads and bridges represent challenges for inspection and maintenance due to advanced age, structural deficiencies and modifications. Concepts such as Building Information Modelling (BIM) aim to alleviate the problem of health monitoring and asset management by providing digital building models constructed from survey data to all stakeholders. Ageing and oftentimes poorly-documented infrastructure objects such as bridges in particular benefit from a continuous integration of changes to form a digital twin which reflects the asset's as-is state. However, the process of reconstructing geometric–semantic models from survey data is a manual and labour-intensive process and makes continuously updating the models a difficult task. To automate this process, a cross-domain approach using an artificial neural network is presented which performs semantic segmentation in the image domain and transfers the results over to the point cloud. For the following fine segmentation, geometric knowledge in the 3D domain is used for post-processing and filtering via geometric reasoning. Using this method, a 3D semantic segmentation is achieved which does not require any 3D point cloud training data and only a low amount of image training data. • Combination of image-based and point cloud-based segmentation techniques. • Novel method for image-based semantic segmentation of point clouds for scenarios without 3D training data. • Robust transfer of labels from image to point cloud domain respecting the underlying 3D geometry. • Point Cloud fine segmentation based on algorithmic geometry and reasoning for dealing with segmentation errors, outliers and occluded regions. [ABSTRACT FROM AUTHOR]