Search

International audience

The task of semantic segmentation is an important one in the context of 3D building modelling. Indeed, developments in 3D generation techniques have rendered the point cloud ubiquitous. However pure data acquisition only captures geometric information and semantic classification remains to be performed, often manually, in order to give a tangible sense to the 3D data. Recently progress in computing power also opened the way for massive application of deep learning methods, including for semantic segmentation purposes. Although well established in the processing of 2D images, deep learning solutions remain an open question for 3D data. In this study, we aim to benefit from the vastly more developed 2D semantic segmentation by performing transfer learning on a photogrammetric orthoimage. The neural network was trained using labelled and rectified images of building façades. Another programme was then written to permit the passage between 2D orthoimage and 3D point cloud. Results show that the approach worked well and presents an alternative to help the automation process for point cloud semantic segmentation, at least in the case of photogrammetric data.

In the context of building renovation, energy audits of buildings are performed in order to reduce energy consumption and greenhouse gas emissions. Infrared cameras allow analyzing precisely the energetic performances and the thermal comfort of a building and the analysis of thermal images of buildings represents a key step on the way to the reduction of energy consumption. In the last few years, the combination of thermal and metric information has been increasingly studied. The geometry provides the exact location of defects and enables accurate energy analysis. This paper focuses on the automatic analysis of facades point clouds with thermal attributes. The aim of our work is to extract windows and thermal bridges by considering both geometric and thermal features. The extraction of objects or zones of interest in thermal data consists in the detection of significant temperature variations. The developed approach, mainly based on the automatic analysis of histograms, is applied to a point cloud acquired by a Mobile Mapping Unit.

Developments in the field of artificial intelligence have made great strides in the field of automatic semantic segmentation, both in the 2D (image) and 3D spaces. Within the context of 3D recording technology it has also seen application in several areas, most notably in creating semantically rich point clouds which is usually performed manually. In this paper, we propose the introduction of deep learning-based semantic image segmentation into the photogrammetric 3D reconstruction and classification workflow. The main objective is to be able to introduce semantic classification at the beginning of the classical photogrammetric workflow in order to automatically create classified dense point clouds by the end of the said workflow. In this regard, automatic image masking depending on pre-determined classes were performed using a previously trained neural network. The image masks were then employed during dense image matching in order to constraint the process into the respective classes, thus automatically creating semantically classified point clouds as the final output. Results show that the developed method is promising, with automation of the whole process feasible from input (images) to output (labelled point clouds). Quantitative assessment gave good results for specific classes e.g., building facades and windows, with IoU scores of 0.79 and 0.77 respectively.

Creating three-dimensional as-built models from point clouds is still a challenging task in the Cultural Heritage environment. Nowadays, performing such task typically requires the quite time-consuming manual intervention of an expert operator, in particular to deal with the complexities and peculiarities of heritage buildings. Motivated by these considerations, the development of automatic or semi-automatic tools to ease the completion of such task has recently became a very hot topic in the research community. Among the tools that can be considered to such aim, the use of deep learning methods for the semantic segmentation and classification of 2D and 3D data seems to be one of the most promising approaches. Indeed, these kinds of methods have already been successfully applied in several applications enabling scene understanding and comprehension, and, in particular, to ease the process of geometrical and informative model creation. Nevertheless, their use in the specific case of heritage buildings is still quite limited, and the already published results not completely satisfactory. The quite limited availability of dedicated benchmarks for the considered task in the heritage context can also be one of the factors for the not so satisfying results in the literature.Hence, this paper aims at partially reducing the issues related to the limited availability of benchmarks in the heritage context by presenting a new dataset for semantic segmentation of heritage buildings. The dataset is composed by both images and point clouds of the considered buildings, in order to enable the implementation, validation and comparison of both point-based and multiview-based semantic segmentation approaches. Ground truth segmentation is provided, for both the images and point clouds related to each building, according to the class definition used in the ARCHdataset, hence potentially enabling also the integration and comparison of the results obtained on such dataset.

The Damped Bundle Adjustment Toolbox (DBAT) is a free, open-source, toolbox for bundle adjustment. The purpose of DBAT is to provide an independent, open-source toolkit for statistically rigorous bundle adjustment computations. The capabilities include bundle adjustment, network analysis, point filtering, forward intersection, spatial intersection, plotting functions, and computations of quality indicators such as posterior covariance estimates and parameter correlations. DBAT is written in the high-level Matlab language and includes several processing example files. The input formats have so far been restricted to PhotoModeler export files and Photoscan (Metashape) native files. Fine-tuning of the processing has so far required knowledge of the Matlab language.This paper describes the development of a scripting language based on the XML (eXtensible Markup Language) language that allow the user a fine-grained control over what operations are applied to the input data, while keeping the needed programming skills at a minimum. Furthermore, the scripting language allows a wide range of input formats. Additionally, the XML format allows simple extension of the script file format both in terms of adding new operations, file formats, or adding parameters to existing operations. Overall, the script files will in principle allow DBAT to process any kind of photogrammetric input and should extend the usability of DBAT as a scientific and teaching tool for photogrammetric computations.

In the context of building renovation, infrared (IR) cameras are widely used to perform the energy audit of buildings. They allow analysing precisely the energetic performances of existing buildings and thermal analyses represent a key step for the reduction of energy consumption. They are also used to assess the thermal comfort of people living or working in a building. Building Information Models (BIM) are widespread to plan the rehabilitation of existing buildings and laser scanning is now commonly used to capture the geometry of buildings for as-built BIM creation. The combination of thermographic and geometric data presents a high number and variety of applications (Lagüela and Díaz-Vilariño, 2016). However, geometric and thermal information are generally acquired separately by different building stakeholders and thermal analyses are performed with independence of geometry. In this paper, the combination of thermal and geometric information is investigated for indoor of buildings. The aim of the project is to create 3D thermographic point clouds based on data acquired by a laser scanner and a thermal camera. Based on these point clouds, BIM models might be enriched with thermal information through the scan-to-BIM process.

The paper explores the possibilities of using old images for 2D and 3D documentation of archaeological monuments using open source, free and commercial photogrammetric software. The available images represent the external façade of the Western gate and Al Omari Mosque in the city of Bosra al-Sham in Syria, which were severely damaged during the recent war. The images were captured using consumer camera and they were originally used to achieve 2D documentation for each part of the gate separately. 2D control points were used to scale the digital photomosaic and reference distances were applied for the scaling of the 3D models. Archive images were used to produce a 2D digital photomosaic of the monument by image rectification and 3D dense point clouds by applying Structure from Motion (SfM) techniques. The geometric accuracy of the results has been assessed.

The segmentation of a point cloud presents an important step in the 3D modelling process of heritage structures. This is true in many scale levels, including the segmentation, identification, and classification of architectural elements from the point cloud of a building. In this regard, historical buildings often present complex elements which render the 3D modelling process longer when performed manually. The aim of this paper is to explore approaches based on certain common geometric rules in order to segment, identify, and classify point clouds into architectural elements. In particular, the detection of attics and structural supports (i.e. columns and piers) will be addressed. Results show that the developed algorithm manages to detect supports in three separate data sets representing three different types of architecture. The algorithm also managed to identify the type of support and divide them into two groups: columns and piers. Overall, the developed method provides a fast and simple approach to classify point clouds automatically into several classes, with a mean success rate of 81.61 % and median success rate of 85.61 % for three tested data sets.

In the context of architectural heritage preservation, constructing building information models is an important task. However, conceiving a pertinent model is a difficult, time consuming and user-dependent task. Our laboratory has been researching methods to decrease the time and errors inferred by manual segmentation of point clouds. In the perspective of automatization of the process, we implemented an automated registration method that used only keypoints. Keypoints are special points that hold more information about the global structure of the cloud. In order to detect keypoints, we used the Point Cloud Library (PCL) toolbox. The pertinence of the method was evaluated by registering more than 300 clouds of the zoological museum of Strasbourg. The quality of the keypoint detection was first verified on geo-referenced indoor point clouds. Then we applied this method to register the indoor and outdoor point clouds that have much less area in common; those common points being generally the doors and windows of the façade. The registrations of indoor point clouds were satisfying, with mean distances to the ground truth inferior to 20 cm. While the first result for joint indoor/outdoor registration are promising, it may be improved in future works.

3D recording is an important procedure in the conservation of heritage sites. This past decade, a myriad of 3D sensors has appeared in the market with different advantages and disadvantages. Most notably, the laser scanning and photogrammetry methods have become some of the most used techniques in 3D recording. The integration of these different sensors is an interesting topic, one which will be discussed in this paper. Integration is an activity to combine two or more data with different characteristics to produce a 3D model with the best results. The discussion in this study includes the process of acquisition, processing, and analysis of the geometric quality from the results of the 3D recording process; starting with the acquisition method, registration and georeferencing process, up to the integration of laser scanning and photogrammetry 3D point clouds. The final result of the integration of the two point clouds is the 3D point cloud model that has become a single entity. Some detailed parts of the object of interest draw both geometric and textural information from photogrammetry, while laser scanning provided a point cloud depicting the overall overview of the building. The object used as our case study is Sari Temple, located in Special Region of Yogyakarta, Indonesia.

This paper presents the work accomplished in order to digitize and model the Winter Garden of the Earthenware Museum at Sarreguemines (Moselle, France). The objectives were to create a digital archive of this cultural heritage place and to find appropriate ways of promoting it. Topographic, photogrammetric and lasergrammetric methods were used to model the Winter Garden. Different virtual reality tools enabled us to spotlight some of its parts and to offer an immersive visualization. Several challenges had to be taken up during this project. We first had to find out the best processing workflow to model earthenware, which is a highly reflecting material. Image processing was also needed for aesthetic reasons and we finally had to find methods to reduce the size of the models.

As the use of building information model (BIM) for architectural heritage becomes more relevant, this paper explores different solutions to further automatize the modelling process. The scan-to-BIM process still requires manual intervention that is time consuming, subject to errors and user-dependent. In this paper, the main focus is the automated segmentation of windows. In the first part of our paper, we will review and compare several state-of-the-art methods for automatic detection and segmentation of openings in a point cloud. Based on the most pertinent aspects of those methods, a new algorithm focusing on indoor point clouds is proposed. After walls are already detected, they are converted in 2D binary images. Holes in those images correspond to openings. We submit each opening to an energy function with two terms: data and coherence. The data term depends on the shape of the opening. The coherence term considers the position of the opening in the scene. Those function let us determine if an opening in the point cloud is due to a window/door or an object obstructing the acquisition. In the third part we discuss the results obtained by applying the method to different datasets.

Heritage documentation has benefitted greatly from significant developments in sensor technology during the past two decades. Miniaturisation of sensors is also an important aspect in the development of low cost sensors, always interesting in heritage projects where budgetary constraints are often present. Among these sensors, the solid-state lidar has begun to attract attention, partly due to its integration in Apple Inc.’s latest version of the iPhone and iPad series. We hypothesise that this type of sensor will see a lot of use in the near future; however, the question remains whether they are sufficient for heritage documentation purposes. In this paper, results from the 2020 iPad Pro SSL point cloud will be assessed and compared to more traditional techniques for 3D scanning (photogrammetry and terrestrial laser scanning). While understandably the geometric quality of benchmark-level techniques such as these remain undeniably better, at least for the moment, the paper concludes that SSL sensors may nevertheless be sufficient for some lower-precision applications.

In recent years, the numerous advantages introduced by Building Information modelling (BIM) have led in its application on the heritage environment and giving birth to the concept of H-BIM (Heritage BIM). The resulting demand in heritage survey data processing has focused this research on the development of strategies and methods to improve the construction of three-dimensional and informative models starting from 3D point clouds. The implementation of an automated procedure is fundamental for easing and speeding up the survey data processing and one of the most challenging tasks to achieve this purpose is the problem of semantic segmentation. The research presented in this paper aims at testing already existing methods and exploring new strategies for 3D point cloud semantic segmentation on heritage scenarios focusing on deep learning and neural network techniques.

International audience; Producing accurate and rapid geomorphic surveys is a key issue for the growing scientific and operational area of physical river restoration. A geodetic survey using 3-D modelling (lasergrammetry, photogrammetry, tacheometry) was combined with a geomorphic survey (RFID bedload tracking, survey of grain size, geomorphic units evolution and geometrical changes) in order to monitor morphodynamic adjustments in a restored anastomosing channel of the Upper Rhine. On this basis, functioning indicators were developed to survey morphological changes at different spatio-temporal scales. Because Structure from Motion (SfM)-photogrammetry is a fast and low-cost method able to produce high-resolution point clouds, the method is particularly well-suited for monitoring complex fluvial environments that have been subject to rapid and intense changes. A hybrid method was developed to complement bank point-clouds with bathymetric data obtained by Total Station leveling. By using the Multiscale Model to Model Cloud Comparison method (M3C2), volumes of erosion-deposition were determined for all surveys. Combining geodetic and geomorphic approaches resulted in a detailed assessment of channel adjustments. This was achieved by creating a set of indicators (e.g., related to vertical and longitudinal evolutions, sediment budget, etc.) that allowed us to both characterize geomorphic adjustments and identify morphodynamic limiting factors. These indicators may be used in the future in a wide range of restoration surveys.

Documentation of heritage objects has advanced significantly with the invention and development of many 3D sensors and processing algorithms. These technical advancements are of the utmost interest for cultural heritage stakeholders; knowledge transfer between key players is therefore essential and is indeed the main philosophy behind the CIPA Heritage Documentation as an organisation. In this paper we present a report on the CIPA Tropical School in the Candi Sewu temple complex in Indonesia. Organisational aspects will be described, as well as some technical results of the workshop. As public interest in the tropical school was quite high, we argue that this type of limited participant training event involving international partners and cross-disciplinary experts is a very powerful tool in knowledge dissemination. Furthermore, we encourage the organisation of such events in other places around the world.

The rapid development of 3D scanning technology is a welcome progress in the field of tangible cultural heritage documentation. While active sensors such as handheld Time-of-Flight (ToF) cameras and lidar have recently generated much hype, developments in low-cost imaging sensors have also seen long strides in recent decades. This paper aims to see the potential of videogrammetry for the purposes of heritage documentation. This technique has existed for decades, but we argue that when combined with modern smartphone sensors and proper photogrammetric processing workflow it may present an interesting low-cost solution for 3D scanning. Furthermore, the paper wishes to address the requirement for a certain geometric quality in heritage documentation and how the proposed method may fulfil them. For this reason, comparisons between the videogrammetric result and traditional DSLR close range photogrammetry will be described to determine its suitability for heritage documentation. Results show that using modern low-cost smartphone imaging sensors, a good compromise between geometric quality and overall cost in the context of cultural heritage recording is possible to achieve.

International audience; Built heritage documentation involves the 3D modelling of the geometry (typically using 3D computer graphics, photogrammetry and laser scanning techniques) and information management of semantic knowledge (i.e., using Geographic Information System (GIS) and ontology tools). The recent developed Building Information Modelling (BIM) technique combines 3D modelling and information management. One of its modern application is heritage documentation and has generated a new concept of Historic/Heritage Building Information Modelling (HBIM). This paper summarises the applications of these information techniques on the built heritage documentation. We utilise Web of Science Collection to monitor the publications on built heritage documentation. We analyse the research trend in heritage modelling by comparing the attention paid by researchers before and during the 2010s. The results show that photogrammetry is always the most popular method in heritage modelling. More and more works in heritage modelling have begun to use laser scanning, computer science, GIS and especially BIM techniques. Ontologies and 3D computer graphics are traditional ways for heritage documentation. Moreover, we pay attention to the roles of BIM on heritage documentation and conduct a detailed discussion on how to extend the HBIM capabilities by integrating with other techniques. The integration provides possible enhanced functions in HBIM, including accurate parametric modelling from computer graphics, automatic semantic segmentation of 3D point cloud from reality-based modelling, spatial information management and analysis by GIS, and knowledge modelling by ontology.

One of the major quality control parameters in bundle adjustment are the posterior estimates of the covariance of the estimated parameters. Posterior covariance computations have been part of the open source Damped Bundle Adjustment Toolbox in Matlab (DBAT) since its first public release. However, for large projects, the computation of especially the posterior covariances of object points have been time consuming.The non-zero structure of the normal matrix depends on the ordering of the parameters to be estimated. For some algorithms, the ordering of the parameters highly affect the computational effort needed to compute the results. If the parameters are ordered to have the object points first, the non-zero structure of the normal matrix forms an arrowhead.In this paper, the legacy DBAT posterior computation algorithm was compared to three other algorithms: The Classic algorithm based on the reduced normal equation, the Sparse Inverse algorithm by Takahashi, and the novel Inverse Cholesky algorithm. The Inverse Cholesky algorithm computes the explicit inverse of the Cholesky factor of the normal matrix in arrowhead ordering.The algorithms were applied to normal matrices of ten data sets of different types and sizes. The project sizes ranged from 21 images and 100 object points to over 900 images and 400,000 object points. Both self-calibration and non-self-calibration cases were investigated. The results suggest that the Inverse Cholesky algorithm is the fastest for projects up to about 300 images. For larger projects, the Classic algorithm is faster. Compared to the legacy DBAT implementation, the Inverse Cholesky algorithm provides a performance increase by one to two orders of magnitude. The largest data set was processed in about three minutes on a five year old workstation.The legacy and Inverse Cholesky algorithms were implemented in Matlab. The Classic and Sparse Inverse algorithms included code written in C. For a general toolbox as DBAT, a pure Matlab implementation is advantageous, as it removes any dependencies on, e.g., compilers. However, for a specific lab with mostly large projects, compiling and using the classic algorithm will most likely give the best performance. Nevertheless, the Inverse Cholesky algorithm is a significant addition to DBAT as it enables a relatively rapid computation of more statistical metrics, further reinforcing its application for reprocessing bundle adjustment results of black-box solutions.

Point cloud segmentation is an important first step in categorising a raw point cloud data. This step is necessary in order to better manage the data and generate other derivative products, e.g. 3D GIS or HBIM. The idea presented in this paper involves the use of 2D GIS to help in the segmentation, classification, as well as (early) semantic annotation of the point cloud. This derives from the fact that in the case of heritage complex sites, often times the site has been previously documented in a 2D GIS often with attributes and entities. We used this 2D data to help in the segmentation of a 3D point cloud, with the added benefit of automatic extraction and annotation of the related semantic information directly to the segmented clusters. Results show that the developed algorithm performs well with TLS data of spread out heritage sites, with a median success rate of 93% and an average rate of 86%. While manual intervention is still inevitable in some parts of the workflow (e.g. creation of the base shapefiles and choice of object segmentation order), the developed algorithm has shown to significantly reduce overall processing time and resources required in terms of segmentation and semantic annotation of a point cloud in the case of heritage complexes.

Building Information Modelling (BIM) technique has been widely utilized in heritage documentation and comes to a general term Historical/Heritage BIM (HBIM). The current HBIM project mostly employs the scan-to-BIM process to manually create the geometric model from the point cloud. This paper explains how it is possible to shape from the mesh geometry with reduced human involvement during the modelling process. Aiming at unbuilt heritage, two case studies are handled in this study, including a ruined Roman stone architectural and a severely damaged abbey. The pipeline consists of solid element modelling based on documentation data using Autodesk Revit, a common BIM platform, and the successive modelling from these geometric primitives using Autodesk Dynamo, a visual programming built-in plugin tool in Revit. The BIM-based reconstruction enriches the classic visual model from computer graphics approaches with measurement, semantic and additional information. Dynamo is used to develop a semi-automated function to reduce the manual process, which builds the final BIM model from segmented parametric elements directly. The level of detail (LoD) of the final models is dramatically relevant with the manual involvement in the element creation. The proposed outline also presents two potential issues in the ongoing work: combining the ontology semantics with the parametric BIM model, and introducing the proposed pipeline into the as-built HBIM process.

Photogrammetry has recently seen a rapid increase in many applications, thanks to developments in computing power and algorithms. Furthermore with the democratisation of UAVs (Unmanned Aerial Vehicles), close range photogrammetry has seen more and more use due to the easier capability to acquire aerial close range images. In terms of photogrammetric processing, many commercial software solutions exist in the market that offer results from user-friendly environments. However, in most commercial solutions, a black-box approach to photogrammetric calculations is often used. This is understandable in light of the proprietary nature of the algorithms, but it may pose a problem if the results need to be validated in an independent manner. In this paper, the Damped Bundle Adjustment Toolbox (DBAT) developed for Matlab was used to reprocess some photogrammetric projects that were processed using the commercial software Agisoft Photoscan. Several scenarios were experimented on in order to see the performance of DBAT in reprocessing terrestrial and UAV close range photogrammetric projects in several configurations of self-calibration setting. Results show that DBAT managed to reprocess PS projects and generate metrics which can be useful for project verification.

International audience; Three-dimensional (3D) documentation of heritage buildings has long employed both image-based and range-based techniques. Unmanned aerial vehicles (UAVs) provide a particular advantage for image-based techniques in acquiring aerial views, which are difficult to attain using classical terrestrial-based methods. The technological development of optical sensors and dense matching algorithms also complement existing photogrammetric workflows for the documentation of heritage objects. In this paper, fundamental concepts in photogrammetry and 3D reconstruction based on structure from motion (SfM) will be briefly reviewed. Two case studies were performed using two types of UAVs, one being a state-of-the-art platform dedicated to obtaining close-range images. Comparisons with laser scanning data were performed and several issues regarding the aerial triangulation and dense matching results were assessed. The results show that although the dense matching of these UAV images may generate centimetre-level precision, a further increase in precision is often hampered by the quality of the onboard sensor.

This paper is dedicated to the digitization of blocks and virtual anastylosis of an antique façade in Pont-Sainte-Maxence (France). In 2014 during the construction of a shopping center, the National Institute for Preventive Archaeological Research (INRAP) discovered a Gallo-Roman site from the 2nd century AD. The most interesting part of the site for the study is a façade of 70 meters long by nearly 10 meters high. The state of the conservation of the blocks of the façade makes them exceptional due to the question raised by the collapse. Representative and symbolic blocks of this building have been selected for a virtual anastylosis study. The blocks discovered belong to different types: decorated architectural blocks, monumental statuary elements and details of very fine decorations. The digital reproduction of the façade will facilitate the formulation of hypothesis for the collapse of the structure. The Photogrammetry and Geomatics Group from INSA Strasbourg is in charge of the digitization, the anastylosis and the development of exploratory methods for understanding the ruin of the façade. To develop the three-dimensional model of the facade, approximately 70 blocks of various dimensions were chosen by the archaeologists. The choice of the digitization technique is made according to the following pragmatic criterion: the movable objects are acquired with a scan-arm or a hand-held scanner in the laboratory and the largest blocks are recorded by photogrammetry at the repository near Paris. The expected types of deliverables are multiple: very accurate 3D models with the most faithful representation to document the objects in the best way and with optimized size model allowing easy handling during anastylosis tests. The visual aspect of the models is also a very important issue. Indeed, textures from photos are an excellent way to bring about the realism of the virtual model, but fine details of the object are sometimes blurred by the uniformity of the color of the original material. Acquisition by hand-held scanner does not provide the textures (they must be acquired according to a complementary process). The data types are therefore different depending on the acquisition. The type of rendering of the models depends therefore on precise choices to be defined optimally. After the acquisition, hypothesis for the construction of the façade must be validated and / or adapted by the anastylosis of the digitized blocks. Different cases must be taken into account. First, the reconstruction of broken blocks is done by adjusting the recovered fragments. If all the fragments discovered are close to the initial shape of the block, the process is assimilated to a puzzle of complex surfaces. If the fragments have no contact but are an integral part of the block, the proportion of hypotheses in relation to the contact pieces is changed. And finally, if the blocks are to be assembled together by superposition and thanks to a common plan, as assumed during the construction, the restitution could be based on the positions of discoveries and hypotheses based on the architectural knowledge of this period. Each of these three methods of reconstruction involves different processes. The three-dimensional model will be validated by the positioning of the blocks and extended according to the actual dimensions of the façade. Different collapse scenarios will result from this study.