Search

Geometric modelling from point cloud data is a fundamental step of the digital twinning process for rail infrastructure. Currently, this onerous procedure outweighs the anticipated benefits of the resulting model and expends 74% of the modellers' effort on converting point cloud data to a model. This is particularly true for rail infrastructure because railway digital twin generation could be an efficient means for maintaining and retrofitting the fundamental mode of the nation's transport. Recent statistics illustrate that the UK increased £5.4 billion of expenditure on railway maintenance per year. This expenditure could go up to £7.2 billion increments per year per incident for railway closures due to unplanned maintenance. Better documentation of current conditions enables tracking and detecting defects in existing railways, potentially avoiding irreversible harm without impeding the national economy. This explains why there is a huge market demand for less labour-intensive railway maintenance techniques that can efficiently boost railway operations and productivity. Railway geometric modelling in the literature has focused mostly on rail detection using laser scanner profile information, with state-of-the-art methods achieving a 99% F1 score for rail detection. However, the existing methods cannot offer large-scale digital twinning required over kilometres without forfeiting precision and manual cost. They only work well in much shorter track segments (on average 300 m) or simplified cases (without complex geometries such as bridges, tunnels, crossings). This is because the search space for railway elements is much longer in multi-kilometre segments and has thousands of element types that are relatively small and thin compared to the asset as a whole. Besides, the real-railway point clouds that stretch over kilometres on the ground often encounter huge challenges, such as occlusions caused by vegetation around the track and unevenly distributed points. Varying horizontal, vertical elevations and cross-sections define the railway geometries. These characteristics complicate the modelling, which is why none of the existing methods can manage them reliably. The aim of this PhD research is to devise, implement and benchmark a novel framework that can accurately generate individually labelled geometric objects of existing rail infrastructure comprising railway track structure and overhead catenary system elements in an established data format [i.e. Industry Foundation Classes (IFC)]. The research provided in this thesis first automatically and effectively segments labelled point clusters of railway elements in point cloud data. It then automatically reconstructs the 3D geometry of the segmented point clusters in IFC format to achieve this objective. The author formed five research questions to answer the segmentation task: (1) how to automatically remove vegetation and other noise surrounding railways without using any additional prior information such as neighbourhood structures, scanning geometries and intensities of the input data? (2) how to automatically segment masts in the form of point clusters by differentiating masts from other pole-like objects in imperfect railway Point Cloud Dataset (PCD)s where occlusions, data gaps and varying point densities exist? (3) how to automatically segment Overhead Line Equipment (OLE) elements in the form of point clusters from real-world railway PCDs with complex railway geometries while occlusions, data gaps and varying point densities are present? (4) how to automatically segment railway track structure elements in the form of point clusters from real-world railway PCDs with varying horizontal and vertical elevations and complex railway geometries?, and (5) how to automatically separate rails from other linear elements adjacent to the railway corridor without relying on prior knowledge such as scanning geometry? The research presented in this thesis exploits standard railway design guidelines to automatically reconstruct the 3D geometry of the segmented point clusters as 3D solid models. The author formed two further research questions to answer the 3D solid model generation task: (1) how to automatically reconstruct labelled point clusters into 3D IFC objects for the railway domain?, and (2) how to evaluate the accuracy of a railway GDT reconstructed from a PCD?. Railways are a linear asset type; their geometric relations stay roughly unchanged, often over very long distances. The proposed framework uses the knowledge of the highly regulated and standardised railway topology and railway design engineering knowledge to segment and model railway elements in point clouds. This framework directly segments railway track structure and overhead catenary system elements and then models them without generating low-level shape primitives and without using any prior information such as user inputs, intensities of input data and scanning geometries. Experiments reveal that the proposed framework can perform quickly and reliably with complex and incomplete real-world railway PCDs featuring occlusions, extreme vegetation around the track, and local variable densities of points. Experiments on 18 km railway PCDs yield an average segmentation F1 score of 88%, an average modelling accuracy below 6 cm Root Mean Square Error (RMSE). The proposed framework can realise an estimated time savings of 94% on average compared to the current manual geometric twinning practice. The proposed framework is the first of its kind to achieve such high and reliable performance of geometric digital twin generation of existing rail infrastructure. Contributions. This PhD research provides the unprecedented ability to rapidly and intelligently model geometric railway track structure and overhead catenary system elements based on quantitative measurements. This is a huge leap over the current practice and a significant step towards the automated generation of Railway Digital Twins.

Shape segmentation from point cloud data is a core step of the digital twinning process for industrial facilities. However, this process is labour-intensive with 90% of the cost being spent on converting point cloud data to a model. This counteracts the perceived value of the resulting model in managing and retrofitting the facilities and motivates the use of automation to reduce this cost. In the US alone, unplanned factory shutdowns due to maintenance cost $50 billion per year. Better documenting the existing conditions can significantly circumvent irreversible damages and decrease the frequency of shutdowns, thus boosting the productivity of industrial assets. This explains why there is a huge market demand for less labour-intensive industrial documentation. Shape segmentation in the literature has so far mostly focused on cylinders, with state-of- the-art methods achieving 60-70% precision and recall for cylinder detection. Such performance is promising, but far from drastically eliminating the manual labour cost, as all other shapes have to be segmented manually. Yet the search space is massive; industrial facilities contain thousands of object types, making automated detection an impossible problem. Hence, there is a direct need to prioritise the most tedious to model objects. The objective of this PhD research is to devise, implement and benchmark a novel framework that can accurately generate individual labelled point clusters of the most important shapes of existing industrial facilities with minimal manual effort in a generic point-level format. This is addressed by first identifying the most important shapes to be modelled and then developing algorithms to efficiently detect those shapes. The former is achieved by answering the following three general research questions: a) what are the most frequent industrial object types?, b) what is the time to model the most frequent object types in state-of-the-art commercial software? and c) what is the performance of state-of-the-art tools in terms of automated object detection? The proposed methodology employs a statistical analysis to identify the most frequent industrial object types and then manually models those to estimate the average man-hours needed for each type. Then, it evaluates the state-of-the-art automated cylinder extraction tool and concludes a 64% reduction in manual modelling time of cylinders. This leads to focus on reducing the remaining man-hours for cylinder modelling as well as for manual modelling of the remaining industrial objects, which are still substantial. This is achieved by answering the following technical research questions: (1) how to automatically segment the most important industrial shapes from point cloud data with varying point densities and occlusions without relying on prior knowledge? (2) how to minimise the time for manually assigning class labels to points? and (3) how to automatically segment instance point clusters with less manual labour compared to the state-of-the-art? The proposed framework employs a combination of deep learning and geometric methods to segment the points into classes and individual instances. Along the way, the author generates the largest to-date dataset of laser scanned industrial facilities used for training and evaluation. Experiments reveal that the method can work reliably in complex and incomplete point clouds of industrial facilities, yielding 82% class segmentation accuracy. Compared to the current state-of-practice, the proposed framework can realise estimated time-savings of 30% on average. Contributions. This PhD research provides the unprecedented ability to rapidly and intelligently segment point clusters based on quantitative measurements. This is a huge leap over the current practice and a significant step towards the automated generation of industrial Digital Twins. As a result, the knowledge created in this PhD research will enable the future development of novel, automated applications for real-time factory maintenance, planned and unplanned downtime reduction.

Bridges are a vitally important part of modern infrastructure. Their condition needs to be monitored on a continuous basis in order to ensure their safety and functionality. Teams of engineers visually inspect more than half a million bridges per year in the US and the EU. There is clear evidence to suggest that they are not able to meet all bridge inspection guideline requirements. In addition, the format and storage of inspection reports varies considerably across authorities because of the lack of standardisation. The availability of a comprehensive and open digital representation of the data involved in and required for bridge inspection is an indispensable necessity for exploiting the full potential of modern digital technologies like big data exploration, artificial intelligence and database technologies. A thorough understanding of bridge inspection information requirements for reinforced concrete bridges is needed as basis for overcoming the stated problem. This work starts with a bridge inspection guideline analysis, from which an information model and a candidate binding to Industry Foundation Classes (IFC) is developed. The resulting bridge model can fully store inspection information in a standardised way which makes it easily shareable and comparable between users and standards. Then, two inspection stages for locating and classifying visual concrete defects are devised, implemented and benchmarked to support the bridge inspection process: In a first stage, healthy concrete surfaces are located and disregarded for further inspection. In a second hierarchical classification stage, each of the remaining potentially unhealthy surface areas is classified into a specific defect type in accordance with bridge inspection guidelines. The first stage achieves a search space reduction for a subsequent defect type classification of over 90% with a risk of missing a defect patch of less than 10%. The second stage identifies the correct defect type to a potentially unhealthy surface area with a probability of 85%. A prototypical implementation serves as a proof of concept. This work closes the gap between requirements arising from established inspection guidelines, the demand for holistic data models which has recently become known as "digital twin", and methods for automatically identifying and measuring specific defect classes on small scale images. It is of great significance for bridge inspectors, bridge owners and authorities as they now have more suitable data models at hand to store, view and manage maintenance information on bridges including defect location and defect types which are being retrieved automatically. With these developments, a foundation is available for a complete revision of bridge inspection processes on a modern, digital basis.

The cost and effort of modelling existing bridges from point clouds currently outweighs the perceived benefits of the resulting model. The time required for generating a geometric Bridge Information Model, a holistic data model which has recently become known as a "Digital Twin", of an existing bridge from Point Cloud Data is roughly ten times greater than laser scanning it. There is a pressing need to automate this process. This is particularly true for the highway infrastructure sector because Bridge Digital Twin Generation is an efficient means for documenting bridge condition data. Based on a two-year inspection cycle, there is a need for at least 315,000 bridge inspections per annum across the United States and the United Kingdom. This explains why there is a huge market demand for less labour-intensive bridge documentation techniques that can efficiently boost bridge management productivity. Previous research has achieved the automatic generation of surface primitives combined with rule-based classification to create labelled cuboids and cylinders from point clouds. While existing methods work well in synthetic datasets or simplified cases, they encounter huge challenges when dealing with real-world bridge point clouds, which are often unevenly distributed and suffer from occlusions. In addition, real bridge topology is much more complicated than idealized cases. Real bridge geometries are defined with curved horizontal alignments, and varying vertical elevations and cross-sections. These characteristics increase the modelling difficulties, which is why none of the existing methods can handle reliably. The objective of this PhD research is to devise, implement, and benchmark a novel framework that can reasonably generate labelled geometric object models of constructed bridges comprising concrete elements in an established data format (i.e. Industry Foundation Classes). This objective is achieved by answering the following research questions: (1) how to effectively detect reinforced concrete bridge components in Point Cloud Data? And (2) how to effectively fit 3D solid models in the format of Industry Foundation Classes to the detected point clusters? The proposed framework employs bridge engineering knowledge that mimics the intelligence of human modellers to detect and model reinforced concrete bridge objects in point clouds. This framework directly extracts structural bridge components and then models them without generating low-level shape primitives. Experimental results suggest that the proposed framework can perform quickly and reliably with complex and incomplete real-world bridge point clouds encounter occlusions and unevenly distributed points. The results of experiments on ten real-world bridge point clouds indicate that the framework achieves an overall micro-average detection F1-score of 98.4%, an average modelling accuracy of (C2C) ̅_Auto 7.05 cm, and the average modelling time of merely 37.8 seconds. Compared to the laborious and time-consuming manual practice, the proposed framework can realize a direct time-savings of 95.8%. This is the first framework of its kind to achieve such high and reliable performance of geometric digital twin generation of existing bridges. Contributions. This PhD research provides the unprecedented ability to rapidly model geometric bridge concrete elements, based on quantitative measurements. This is a huge leap over the current practice of Bridge Digital Twin Generation, which performs this operation manually. The presented research activities will create the foundations for generating meaningful digital twins of existing bridges that can be used over the whole lifecycle of a bridge. As a result, the knowledge created in this PhD research will enable the future development of novel, automated applications for real-time condition assessment and retrofit engineering.

Many older bridges and tunnels were constructed using standards by now many decades out-of-date, at a time when trucks and other large vehicles were smaller. A bridge or tunnel strike is an incidence in which a vehicle, typically a lorry (truck) or double-decker bus, tries to pass under a bridge or tunnel that is lower than its height, subsequently colliding with the structure. These strikes lead to an increased cost of bridge repairs, clogged up roadways and increased potential for catastrophic events: hazardous spillage and/or total collapse. Today, Network Rail reports on average a strike every 4.5 hours. There are a number of reasons why strikes occur, and why drivers of heavy goods vehicles sometimes fail to recognise the warning signs, consequently striking the bridge or tunnel. At first glance, it may seem like the problem is a fairly easy one to solve; however, no matter how well planned the road system, human error is an ever-present risk. The research proposes to address the problem of bridge and tunnel strike prevention and management. The intent of the research is to develop an affordable, reliable and robust early warning over-height detection system bridge-owners can implement at locations with high strike occurrences. The research aims to test and validate a novel vision-based system using a single camera to accurately detect over-height vehicles using a set of optimised parameters. The system uses a camera installed at the offending height, which acts as an “over-height plane” formed by the averages of the maximum allowable heights across all lanes in a given traffic direction. Any vehicle exceeding this plane is analysed within a region of interest using a trigger-based approach for accurate detection and driver warning. If the vehicle is deemed to be over-height, a warning is issued to the driver. As a result, prolonging life expectancy of structures while decreasing the cost of repairs, maintenance and inspections.

Over the past decades, the construction industry lags further and further behind the manufacturing sector when productivity is considered. This is due to internal factors that take place on-site. Almost all of them are directly related to the way that productivity is monitored. Current practices for monitoring labour productivity are labour intensive, time - cost consuming and error prone. They are mainly reactive processes initiated after the detection of a negatively influencing factor. Although research studies have been performed towards leveraging these limitations, a gap still exists in monitoring labour productivity of multiple workers at the same time accurately, unobtrusively, cost and time efficiently. This thesis proposes a framework to address this gap. It hypothesizes that task productivity of construction workers can be monitored through their trajectory data. The proposed framework uses as input, video data streamed from cameras with overlapping field of view. It consists of two main methods. The output of the first is the input of the second. The first method tracks the location of workers across the range of a jobsite over time and returns their 4D trajectories. Such type of tracking requires that workers are matched under a unique ID not only across successive frames of a single camera (intra tracking) but also across multiple cameras (inter tracking). Existing tag-less studies fail to track construction workers due to the challenging nature of their working environments. Therefore, two novel computer vision-based algorithms are developed to perform both the intra and the inter camera tracking. The second method of the proposed framework converts the 4D trajectories of workers into productivity information. These trajectories are clustered into work cycles with an accuracy of 95%, recall of 76% and precision of 76%. Such work cycles depict the actual execution of tasks. The overall proposed framework features an average accuracy of 95% in terms of determining the total time workers spend on construction-related tasks.

Construction progress monitoring allows schedule and/or cost deviations to be identified early enough to effectively implement corrective actions. At least 77% of transportation projects experience cost overruns, and as much as 75% of these overruns have been attributed to “real” construction management factors like progress monitoring. Progress is measured on road construction sites in terms of completion percentages at various activity and work package levels. This percentage is then used to identify schedule deviations and support the earned value analysis often used as the baseline for contractor progress payments. Unfortunately, the current methods for producing these completion percentages are not as correct or time efficient as they should be to enable effective project control. The objective of this research is to develop, test, and validate a novel solution for automatically producing completion percentages and progress status determinations that are more correct and time efficient than those generated in current practice. The proposed solution seeks to automatically detect incremental progress on road design layers in 3D as-built point cloud data generated using unmanned aerial photogrammetry and a novel data simulation approach. A parallel as-planned progress estimate is also automatically prepared using 4D information, and the progress status determinations are made by comparing the two results. This solution was tested on 15 datasets (13 simulated and 2 real-world) representing a variety of road designs and progress conditions. The method achieved an average 95% F1 score in layer detection on the real-world data, and mostly outperformed current practice in correctness. The automated processing of as-built and as-planned data to produce the progress estimate took 12 seconds for the real world data, which was indeed faster than the current practice equivalent. Although the research objectives were met, there remains room for further improvement, particularly in regards to the solution’s robustness to occlusions on the monitored surfaces.

This thesis presents a real-time automated building progress monitoring solution for indoor environments using a mobile device. Such a system could prompt accurate and timely assessment of work progress that would allow managers to make adjustments and minimise both time and cost overrun when deviations from the schedule occur. Although many researchers have proposed approaches for progress monitoring in outdoor scenes, these cannot perform in real-time and shift into the complex interior environment. Research efforts for indoor environments are not fully automated and lead to errors in more complex scenes. Systems based on mobile devices could potentially enhance the inspection process and reduce the required time by allowing the inspector to acquire progress data by simply walking around the site. The main challenge of these systems is the tracking of the pose of the camera to achieve accurate alignment between the 3D design model and the real-world scene. Methods for estimating the user’s pose rely on a) tags on each target of interest, which require additional time and cost for installation and maintenance; b) pre-selected user locations, which restricts the user to those locations only; or c) GPS on the augmented reality headset, which only applies to outdoor inspections. Additionally, current mobile-based inspection systems do not perform any comparison between the captured as-built and the as-planned data. In this research, different marker-less Augmented Reality (AR) potential methods were implemented and tested for finding the most robust tracking solution. The Microsoft HoloLens was found to be the top performer for tracking the user’s pose and for overall user-experience. Next, a semi-automated method was developed for initially registering the 3D model to the real environment by exploiting information from detected floor and wall surfaces. Results showed that this method reduces the time of the initial registration by 58%. Having the 3D model aligned to the real environment and knowing the pose of the camera at every moment, an automated method was developed that exploits the captured as-built surface mesh data from the mobile device, compares it against the 3D design model and identifies in real-time whether an object has been built according to plan. Different parameters were tested for finding the optimum combination based on the current quality of mesh data. If quality of mesh data changes, then new parameters should be explored. Finally, the proposed solution was tested in real site conditions resulting in 76.6% precision, 100.0% recall, and 83.5% accuracy.

As-Is Building Information Models aid in the management, maintenance and renovation of existing buildings. However, most existing buildings do not have an accurate geometric depiction of their As-Is conditions. The process of generating As-Is models of existing structures involves practitioners, who manually convert Point Cloud Data (PCD) into semantically meaningful 3D models. This process requires a significant amount of manual effort and time. Previous research has been able to model objects by segmenting the point clouds into planes and classifying each one separately into classes, such as walls, floors and ceilings; this is insufficient for modelling, as BIM objects are comprised of multiple planes that form volumetric objects. This thesis introduces a novel method that focuses on the geometric creation of As-Is BIMs with enriched information. It tackles the problem by detecting objects, modelling them and enriching the model with spaces and object adjacencies from PCD. The first step of the proposed method detects objects by exploiting the relationships the segments should satisfy to be grouped into one object. It further proposes a method for detecting slabs with variations in height by finding local maxima in the point density. The second step models the geometry of walls and finally enriches the model with closed spaces encoded in the Industry Foundation Classes (IFC) standard. The method uses the point cloud density of detected walls to determine their width by projecting the wall into two directions and finding the edges with the highest density. It identifies adjacent walls by finding gaps or intersections between walls and exploits walls adjacency for correcting their boundaries, creating an accurate 3D geometry of the model. Finally, the method detects closed spaces by using a shortest-path algorithm. The method was tested on three original PCD which represent office floors. The method detects objects of class walls, floors and ceilings in PCD with an accuracy of approximately 96%. The precision and recall for the room detection were found to be 100%.