3D Modeling of Façade Elements Using Multi-View Images from Mobile Scanning Systems

There is a growing demand for detailed building façade models (Level-of-Detail 3: LoD 3) in a variety of applications. Despite the increasing number of papers addressing this issue in the literature, occlusions are still a significant problem when processing building façade elements. Conversely, depending on the view angle of the images, the detected elements might not be projected to their accurate locations causing uncertainties in their 3D locations. In this paper, we address the aforementioned issues utilizing multi-view images. Using a building footprint layer, we first locate the points belonging to buildings. We then detect the 2D windows and doors in images by combining Faster R-CNN and Segment Anything (SAM) deep learning models. The 2D borders are projected into the 3D object space using a pinhole camera model and collinearity equations. Utilizing the multi-view capabilities of mobile scanning systems, this method effectively mitigates uncertainties associated with occlusion and exterior orientation parameters (EOP). This study provides a comprehensive evaluation of 3D spatial accuracy, achieving an average of 84% Intersection over Union (IoU) accuracy for 12 different single-sided façades over 750 multi-view images for 312 windows and doors of various sizes with rectangular and curved shapes.