Search

Current procedures for the rapid inspection of buildings and infrastructure are subjective, time-consuming, and cumbersome to document, necessitating new technologies to automate the process and eliminate these shortcomings. Fortunately, recent developments in imaging devices and artificial intelligence, such as computer vision, provide the necessary tools for this, though they are not yet integrated into infrastructure applications. In this paper, we propose an end-to-end pipeline that generates damage-augmented digital twins for buildings at LOD3, including geometrical information as well as data pertaining to damage condition and its characterization. Our framework incorporates multiple-view images to (1) create a level of detail model, (2) segment damage information, and (3) characterize damage. The core of the method is the structure from motion, which is used to reconstruct the building scene, and machine-learning models that segment and characterize damage. In contrast to current practices, our method does not require manual intervention, generates lightweight models, and can be applied to a wide range of assets. The results generated with our pipeline represent a significant step towards an automated infrastructure damage assessment. We intend to expand our work in the future to include real-time applications and applications to other types of infrastructure. Codes and data sets are publicly available (https://github.com/eesd-epfl/DADT%5fbuildings and https://doi.org/10.5281/zenodo.7767478). • A novel framework to generate Damage Augmented Digital Twins (DADT) of buildings. • State-of-the-art solutions for 3D models, crack segmentation and characterization. • Bench-marking data set for evaluating the automated generation of DADT of buildings. [ABSTRACT FROM AUTHOR]

• Novel pipeline for automatically reconstruct LOD3 models of free-standing buildings. • New metrics for evaluating the performance of a LOD reconstruction. • Benchmark dataset for evaluating the reconstruction of SfM as well as LOD3 models. • Deep learning models for facade and opening segmentation of free-standing masonry buildings. This paper describes a pipeline for automatically generating level of detail (LOD) models (digital twins), specifically LOD2 and LOD3, from free-standing buildings. Our approach combines structure from motion (SfM) with deep-learning-based segmentation techniques. Given multiple-view images of a building, we compute a three-dimensional (3D) planar abstraction (LOD2 model) of its point cloud using SfM techniques. To obtain LOD3 models, we use deep learning to perform semantic segmentation of the openings in the two-dimensional (2D) images. Unlike existing approaches, we do not rely on complex input, pre-defined 3D shapes or manual intervention. To demonstrate the robustness of our method, we show that it can generate 3D building shapes from a collection of building images with no further input. For evaluating reconstructions, we also propose two novel metrics. The first is a Euclidean–distance-based correlation of the 3D building model with the point cloud. The second involves re-projecting 3D model facades onto source photos to determine dice scores with respect to the ground-truth masks. Finally, we make the code, the image datasets, SfM outputs, and digital twins reported in this work publicly available in github.com/eesd-epfl/LOD3_buildings and doi.org/10.5281/zenodo.6651663. With this work we aim to contribute research in applications such as construction management, city planning, and mechanical analysis, among others. [ABSTRACT FROM AUTHOR]

• Novel framework to generate Geometrical Digital Twins of stone masonry elements. • 3D registration algorithm based on least squares and 2D image point features. • Benchmark dataset to assess the generation of GDT for stone masonry elements. We present an image-based pipeline for generating geometrical digital twins (GDTs) of stone masonry elements with detail down to the stone level. For this purpose, we acquire RGB images of the individual stones and of the wall during the construction phase. In our framework, we use structure from motion (SfM) to first generate 3D source and destination models, which are then registered to form the GDT through non-linear least squares and 2D point feature correspondences detected on the SfM images. This method contrasts with traditional techniques that register point clouds using 3D point descriptors. Because of the robustness of image feature descriptors, we found that using 2D instead of 3D point features facilitates the automation of the GDT generation. To benchmark our algorithm, we compared the results through an Euclidean–distance-based proposed metric with a known 3D textured model from which images were synthetically generated. We show the robustness and feasibility of our method for full size elements, wherein GDTs were generated for dry-stone and stone-mortar systems. This study allows researchers to produce accurate representations of the 3D geometry of walls built for experimental research, reducing therefore uncertainties related to the stone size, shape and arrangement to a minimum when comparing 3D numerical simulations of these walls to experimental results. Codes and data sets are publicly available (https://github.com/eesd-epfl/stone%5fmasonry%5fGDT and https://doi.org/10.5281/zenodo.7266587). [ABSTRACT FROM AUTHOR]

Determining the relationship between the cause of damage and the subsequent structural behavior of infrastructure systems requires an accurate characterization of the propagation of cracks, which represents the evolution of the damage state. When no information about the cause of damage is available, kinematic approaches can be used to describe the motion of crack contours. Current image-based approaches to derive crack kinematics use digital image correlation (DIC) on a set of sequential images as the crack propagates. However, DIC is invasive in that the structure surfaces must be painted with random speckle patterns, limiting its use primarily to controlled experiments. In this paper, we propose a novel image-based methodology for computing crack opening in Mode I or Mode II. As an input, this method takes a binary image from a semantic segmentation of an image of a crack pattern. This binary image is used to detect the opposite edges along the crack, which are then registered using an optimization algorithm based on the Euclidean transformation model and non-linear least squares. As a final output, this method produces displacement maps in the tangential and normal directions to the crack skeleton. To demonstrate its performance, we validate our methodology first with synthetic crack patterns and then with real crack patterns. Because this methodology for determining crack openings requires only simple data (just a binary crack pattern image), it is straightforward, robust, and adaptable, thus contributing to the development of structural image-based damage assessments. The computational codes and datasets are available to the public for future research and benchmarking on https://github.com/eesd-epfl/crack%5fkinematics and https://doi.org/10.5281/zenodo.6632071. • A novel method for determining crack opening in Mode I and Mode II from images. • A crack kinematics formulation as a registration problem of the two crack edges. • A simple algorithm for registering 2D point sets based on non-linear least squares. • A dataset of binary crack patterns with known displacements for benchmarking. [ABSTRACT FROM AUTHOR]

Determining the relationship between the cause of damage and the subsequent structural behavior of infrastructure systems requires an accurate characterization of the propagation of cracks, which represents the evolution of the damage state. When no information about the cause of damage is available, kinematic approaches can be used to describe the motion of crack contours. Current image-based approaches to derive crack kinematics use digital image correlation (DIC) on a set of sequential images as the crack propagates. However, DIC is invasive in that the structure surfaces must be painted with random speckle patterns, limiting its use primarily to controlled experiments. In this paper, we propose a novel image-based methodology for computing crack opening in Mode I or Mode II. As an input, this method takes a binary image from a semantic segmentation of an image of a crack pattern. This binary image is used to detect the opposite edges along the crack, which are then registered using an optimization algorithm based on the Euclidean transformation model and non-linear least squares. As a final output, this method produces displacement maps in the tangential and normal directions to the crack skeleton. To demonstrate its performance, we validate our methodology first with synthetic crack patterns and then with real crack patterns. Because this methodology for determining crack openings requires only simple data (just a binary crack pattern image), it is straightforward, robust, and adaptable, thus contributing to the development of structural image-based damage assessments. The computational codes and datasets are available to the public for future research and benchmarking on https://github.com/eesd-epfl/crack%5fkinematics and https://doi.org/10.5281/zenodo.6632071. • A novel method for determining crack opening in Mode I and Mode II from images. • A crack kinematics formulation as a registration problem of the two crack edges. • A simple algorithm for registering 2D point sets based on non-linear least squares. • A dataset of binary crack patterns with known displacements for benchmarking. [ABSTRACT FROM AUTHOR]

We present a method for segmenting cracks in images of masonry buildings damaged by earthquakes. Existing methods of crack detection fail to preserve the continuity of cracks, and their performance deteriorates with imprecise training labels. We address these problems by adapting an approach previously proposed for reconstructing roads in aerial images, in which a Convolutional Neural Network is trained with a loss function specifically designed to encourage the continuity of thin structures and to accommodate imprecise annotations. We evaluate combinations of three loss functions (the Mean Squared Error, the Dice loss and the new connectivity-oriented loss) on two datasets using TernausNet, a deep network shown to attain state-of-the-art accuracy in crack detection. We herein show that combining these three losses significantly improves the topology of the predictions quantitatively and qualitatively. We also propose a new continuity metric, named Cracks Per Patch (CPP), and share a new dataset of images of earthquake-affected urban scenes accompanied by crack annotations. The dataset and implementations are publicly available for future studies and benchmarking (https://github.com/eesd-epfl/topo%5fcrack%5fdetection and https://doi.org/10.5281/zenodo.6769028). • Evaluation of a loss function for crack detection using deep learning that emphasize the correct representation of crack topology. • A solution for continuity crack detection problems using deep learning that does not require pixel-precise labels. • New metric to assess continuity preservation in crack prediction. • New training dataset of real-world post-earthquake building images containing labeled cracks. [ABSTRACT FROM AUTHOR]