Search

This survey provides a comprehensive overview of traditional techniques and deep learning-based methodologies for monocular visual odometry (VO), with a focus on displacement measurement applications. This paper outlines the fundamental concepts and general procedures for VO implementation, including feature detection, tracking, motion estimation, triangulation, and trajectory estimation. This paper also explores the research challenges inherent in VO implementation, including scale estimation and ground plane considerations. The scientific literature is rife with diverse methodologies aiming to overcome these challenges, particularly focusing on the problem of accurate scale estimation. This issue has been typically addressed through the reliance on knowledge regarding the height of the camera from the ground plane and the evaluation of feature movements on that plane. Alternatively, some approaches have utilized additional tools, such as LiDAR or depth sensors. This survey of approaches concludes with a discussion of future research challenges and opportunities in the field of monocular visual odometry. [ABSTRACT FROM AUTHOR]

Environmental effects may lead to cracking, stiffness loss, brace damage, and other damages in bridges, frame structures, buildings, etc. Structural Health Monitoring (SHM) technology could prevent catastrophic events by detecting damage early. In recent years, Deep Learning (DL) has developed rapidly and has been applied to SHM to detect, localize, and evaluate diverse damages through efficient feature extraction. This paper analyzes 337 articles through a systematic literature review to investigate the application of DL for SHM in the operation and maintenance phase of facilities from three perspectives: data, DL algorithms, and applications. Firstly, the data types in SHM and the corresponding collection methods are summarized and analyzed. The most common data types are vibration signals and images, accounting for 80% of the literature studied. Secondly, the popular DL algorithm types and application areas are reviewed, of which CNN accounts for 60%. Then, this article carefully analyzes the specific functions of DL application for SHM based on the facility's characteristics. The most scrutinized study focused on cracks, accounting for 30 percent of research papers. Finally, challenges and trends in applying DL for SHM are discussed. Among the trends, the Structural Health Monitoring Digital Twin (SHMDT) model framework is suggested in response to the trend of strong coupling between SHM technology and Digital Twin (DT), which can advance the digitalization, visualization, and intelligent management of SHM. [ABSTRACT FROM AUTHOR]

This review paper addresses the critical need for structural prognostics and health management (SPHM) in aircraft maintenance, highlighting its role in identifying potential structural issues and proactively managing aircraft health. With a comprehensive assessment of various SPHM techniques, the paper contributes by comparing traditional and modern approaches, evaluating their limitations, and showcasing advancements in data-driven and model-based methodologies. It explores the implementation of machine learning and deep learning algorithms, emphasizing their effectiveness in improving prognostic capabilities. Furthermore, it explores model-based approaches, including finite element analysis and damage mechanics, illuminating their potential in the diagnosis and prediction of structural health issues. The impact of digital twin technology in SPHM is also examined, presenting real-life case studies that demonstrate its practical implications and benefits. Overall, this review paper will inform and guide researchers, engineers, and maintenance professionals in developing effective strategies to ensure aircraft safety and structural integrity. [ABSTRACT FROM AUTHOR]

Digital twins (DTs) are becoming increasingly popular in various industries, and their potential for healthcare in the metaverse continues to attract attention. The metaverse is a virtual world where individuals interact with digital replicas of themselves and the environment. This paper focuses on personalized and precise medicine and examines the current application of DTs in healthcare within the metaverse. Healthcare practitioners may use immersive virtual worlds to replicate medical scenarios, improve teaching experiences, and provide personalized care to patients. However, the integration of DTs in the metaverse poses technical, regulatory, and ethical challenges that need to be addressed, including data privacy, standards, and accessibility. Through this examination, we aim to provide insights into the transformative potential of DTs in healthcare within the metaverse and encourage further research and development in this exciting domain. [ABSTRACT FROM AUTHOR]

Digital twin is the digital representation of an entity, and it drives Industry 4.0. This paper presents a bibliometric analysis of digital twin in the supply chain to help researchers, industry practitioners, and academics to understand the trend, development, and focus of the areas of digital twin in the supply chain. This paper found several key clusters of research, including the designing of a digital twin model, integration of a digital twin model, application of digital twin in quality control, and digital twin in digitalization. In the embryonic stage of research, digital twin was tested in the production line with limited optimization. In the development stage, the importance of digital twin in Industry 4.0 was observed, as big data, machine learning, Industrial Internet of Things, blockchain, edge computing, and cloud-based systems complemented digital twin models. Digital twin was applied to improve sustainability in manufacturing and production logistics. In the current prosperity stage with high annual publications, the recent trends of this topic focus on the integration of deep learning, data models, and artificial intelligence for digitalization. This bibliometric analysis also found that the COVID-19 pandemic drove the start of the prosperity stage of digital twin research in the supply chain. Researchers in this field are slowly moving towards applying digital twin for human-centric systems and mass personalization to prepare to transit to Industry 5.0. [ABSTRACT FROM AUTHOR]

Intelligent digital twins closely resemble their real-life counterparts. In health and medical care, they enable the real-time monitoring of patients, whereby large amounts of data can be collected to produce actionable information. These powerful tools are constructed with the aid of artificial intelligence, machine learning, and deep learning; the Internet of Things; and cloud computing to collect a diverse range of digital data (e.g., from digital patient journals, wearable sensors, and digitized monitoring equipment or processes), which can provide information on the health conditions and therapeutic responses of their physical twins. Intelligent digital twins can enable data-driven clinical decision making and advance the realization of personalized care. Migraines are a highly prevalent and complex neurological disorder affecting people of all ages, genders, and geographical locations. It is ranked among the top disabling diseases, with substantial negative personal and societal impacts, but the current treatment strategies are suboptimal. Personalized care for migraines has been suggested to optimize their treatment. The implementation of intelligent digital twins for migraine care can theoretically be beneficial in supporting patient-centric care management. It is also expected that the implementation of intelligent digital twins will reduce costs in the long run and enhance treatment effectiveness. This study briefly reviews the concept of digital twins and the available literature on digital twins for health disorders such as neurological diseases. Based on these, the potential construction and utility of digital twins for migraines will then be presented. The potential and challenges when implementing intelligent digital twins for the future management of migraines are also discussed. [ABSTRACT FROM AUTHOR]

Output-only structural health monitoring is a highly active research direction because it is a promising methodology for building digital twin applications providing near-real-time monitoring results of the structure. However, one of the technical bottlenecks is how to work effectively with multiple high-dimensional vibration signals. To address this question, this study develops a two-stage data-driven framework based on various advanced techniques, such as time-series feature extractions, self-learning, graph neural network, and machine learning algorithms. At first, multiple features in statistical, time, and spectral domains, are extracted from raw vibration data; then, they subsequently enter a graph convolution network to account for the spatial correlation of sensor locations. After that, the high-performance adaptive boosting machine learning algorithm is leveraged to assess structures' health states. This method allows for learning a lower-dimensional yet informative representation of vibration data; thus, the subsequent monitoring tasks could be performed with reduced time complexity and economical computational resources. The performance of the proposed method is qualitatively and quantitatively demonstrated through two examples involving both numerical and experimental structural data. Furthermore, comparison and robustness studies are carried out, showing that the proposed approach outperforms various machine learning/deep learning-based methods in terms of accuracy and noise/missing-robustness. [ABSTRACT FROM AUTHOR]

Railway track maintenance plays an important role in enabling safe, reliable, and seamless train operations and passenger comfort. Due to the increasing rail transportation, rolling stocks tend to run faster and the load tends to increase continuously. As a result, the track deteriorates quicker, and maintenance needs to be performed more frequently. However, more frequent maintenance activities do not guarantee a better overall performance of the railway system. It is crucial for rail infrastructure managers to optimize predictive and preventative maintenance. This study is the world's first to develop deep machine learning models using three-dimensional recurrent neural network-based co-simulation models to predict track geometry parameters in the next year. Different recurrent neural network-based techniques are used to develop predictive models. In addition, a building information modeling (BIM) model is developed to integrate and cross-functionally co-simulate the track geometry measurement with the prediction for predictive and preventative maintenance purposes. From the study, the developed BIM models can be used to exchange information for predictive maintenance. Machine learning models provide the average R2 of 0.95 and the average mean absolute error of 0.56 mm. The insightful breakthrough demonstrates the potential of machine learning and BIM for predictive maintenance, which can promote the safety and cost effectiveness of railway maintenance. [ABSTRACT FROM AUTHOR]

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics. [ABSTRACT FROM AUTHOR]

Large volumes of 3D parametric datasets, such as building information modeling (BIM), are the foundation for developing and applying smart city and digital twin technologies. Those datasets are also considered valuable tools for efficiently managing rebuilt structures during the operation and maintenance stages. Nevertheless, current approaches developed for the scan-to-BIM process rely on manual or semi-automatic procedures and insufficiently leverage semantic data in point clouds. These methods struggle to accurately represent large-scale indoor complex layouts and extract details from irregular-shaped unstructured elements, causing inefficiencies in BIM model generation. To address these issues, we propose an innovative scan-to-BIM framework based on deep learning algorithms and raw point cloud data, enabling the automatic generation of 3D models for both structured and unstructured indoor elements. Initially, we propose an enhanced deep learning neural network to improve the point clouds' semantic segmentation accuracy. Subsequently, an efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. The proposed workflow can reconstruct large-scale data with multiple room layouts of Manhattan or non-Manhattan structures and reconstruct 3D models automatically by using a BIM parametric algorithm implemented in Revit software. Moreover, we introduce a robust method for unstructured elements to automatically generate corresponding 3D BIM models, even when the incorporating semantic information is incomplete. The proposed approach was evaluated on synthetic and real data for different scales and complexities of indoor scenes. The results of the experiments demonstrate that the improved model significantly enhances the overall semantic segmentation accuracy compared to the baseline models. The proposed scan-to-BIM framework is efficient for indoor element 3D reconstruction, achieving precision, recall, and F-score values ranging from 96% to 99%. The generated BIM models are competitive with traditional methods regarding model completeness and geometric accuracy. • We propose an innovative scan-to-BIM framework based on deep learning algorithms: • We propose an enhanced deep-learning neural network to improve the point clouds' semantic segmentation accuracy. • A novel and efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. It can reconstruct large-scale data with multiple room layouts in 3D model automatically by using a BIM parametric • For unstructured elements, we propose a robust method to automatically generate 3D BIM models, even when the semantic information is incomplete. [ABSTRACT FROM AUTHOR]

• Numerical simulation modeling is complex and computationally intensive. • Digital twin deployment requires lightweight and responsive systems. • Process sensor data drives deep learning algorithms to implement digital twins. • Generative adversarial networks enable monitoring of heat distribution changes. • Results contribute to weld quality assurance and blade structure optimization. In this paper, a novel method of using deep learning to construct a digital twin of the thermos-compression bonding blade's temperature field is presented. Usually, a heat transfer model based on numerical analysis can calculate the heat distribution in the welding process, but its complex modeling process and heavy computational effort are not appropriate for the deployment of the digital twin concept. The approach proposed in this paper is distinctive because it directly converts the collected process sensor data into the bonding blade's temperature field images and the bonding surface's temperature distribution curves for monitoring the welding process. This is achieved by training the generator and discriminator of the conditional depth convolutional generative adversarial network separately. The network is then capable of predicting the bonding blade's heat distribution within a certain range of welding parameters. Compared to methods based on numerical simulation, this approach is far more effective. Training and testing datasets were created to assess the suggested method's efficacy. Multi-scale structural similarity index, structural similarity index, peak signal-to-noise ratio, difference hash and mean absolute error were used to evaluate the simulations' quality. The results show that the simulation's results and the actual data agree quite well. The temperature changes that occur during the welding process may be quickly and accurately simulated using the proposed technology. [ABSTRACT FROM AUTHOR]

Decoupling the physical world and providing standardized service interfaces is still challenging when developing Location Based Services (LBS). This lack also hinders the possibility of developing Intelligent services on top of LBS architectures. In this paper, we propose a multi-layer Digital Twin-based architecture that aims to enable the development of machine learning-based Intelligent LBS (I-LBS) that are able to adapt, evolve, and perform Continual Learning (CL). The platform uses Digital Twins to ensure physical abstraction and provide cyber–physical knowledge to the I-LBSs, which is defined as an execution graph of operation modules. Finally, we simulated a use-case for this platform in the complex scenario of Healthcare organization and management where the I-LBS classifies allowed/not allowed trajectories of users inside a real-existing hospital scenario depending on their role in the organization. The use case is implemented as a Deep Learning-based reconstruction task of high-resolution trajectories processed by the DT architecture that also deploys the I-LBS. The platform is evaluated in terms of physical complexity and computational time on the DT side and on both a traditional machine learning setting and a replay-based CL one for the intelligence side to demonstrate the flexibility and adaptability features introduced by the components for dynamic or unseen scenarios. [ABSTRACT FROM AUTHOR]

Digital twins (DTs) have attracted widespread attention in academia and industry in recent years. It can accurately reflect the physical world in real-time, enabling online monitoring, control, and prediction operations. Their foundation is super-real-time computing and high data representation capabilities. However, current DTs do not achieve 3D super-real-time computing. This study proposes a novel 3D computational method for solving fluid–solid coupling problems in a super-real-time. The method is based on a mixed solution framework that combines traditional numerical methods with deep learning operators. Specifically, the method employs multi-core CPU parallel acceleration to solve the solid equations while leveraging the computing power of GPU to solve the fluid equations. The fluid–solid coupling is achieved through information exchange between the GPU and the multi-core CPU. In addition, the proposed method introduces a new deep learning operator framework based on the DeepONET. The framework is accompanied by a database structure that facilitates model training and validation and a loss function that guides the training. The space nuclear reactor, an improved TOPAZ-II system, was selected to demonstrate its feasibility. Four non-training transient conditions were simulated to test the generalization performance. The results show that the proposed method achieves an average error between the calculated results and reference values below 2.5%, with the average error of thermodynamic parameters below 1.5%. The average deviation between system parameter peak values during the transient process and the reference value was less than 5 s. The result meets the acceptable error level and satisfies the super-real-time requirements with a time acceleration ratio of approximately 1.17, which is 60 times faster than traditional numerical methods. The results demonstrate the accuracy and efficiency of the proposed method for DT. [ABSTRACT FROM AUTHOR]

Digital twins are undergoing growth that enables highly informative and scaleable interaction between physical objects and virtual twins, which is of great significance to the life cycle analysis of composites. Real-time fine evolution is challenging due to vast combinations of input features and high-resolution calculated variables. Here, we systematically demonstrate an AI-based methodology for digital twinning of complex composite structures. First, three types of deep neural networks are created with optionally used autoencoders as surrogate models, with architectures and data processing inspired by the rule-of-mixture of composites. Second, the prediction accuracy and efficiency are evaluated quantitatively and qualitatively, demonstrating the feasibility of predicting 3D displacement and stress fields directly from sensing data of temperature, pressure and loading displacement, and the optimal architecture is selected to be the evolving digital twin. Finally, the real-time interactive experiments are relayed to the digital twin and demonstrated that it can interact with physical objects and evolve online with high accuracy. These results indicate that the computational time can be reduced by 3∼6 orders of magnitude with high information intensity and scalability compared with conventional numerical and experimental methods, which opens up the avenues for the cost-effective and efficient development of digital twin services for composites. [Display omitted] [ABSTRACT FROM AUTHOR]

Finite element method (FEM) based high-fidelity simulation can be computationally demanding and time-consuming as engineering problems become more complicated. It is thus necessary to develop a surrogate model that only requires a small amount of computational time but retains sufficient accuracy. A graph neural network (GNN) based framework is proposed as a general surrogate model for FEM to simulate the Von Mises stress distribution. The mesh body is embedded to a graph and a novel global attribute representation is introduced to capture the geometry and boundary conditions while overcoming the common problem of over smoothing in graph deep learning. The challenge to deal with varying geometry and boundary conditions is overcome by the proposed model and thus it outperforms existing methods such as proper orthogonal decomposition (POD) and greedy algorithm in terms of generalization. Numerical experiments are given to demonstrate the capability of the model developed and the results show that the proposed model not only accurately predicts the stress distribution but also speed-ups hundreds of times faster compared to a FEM-based simulator, enabling real-time structural response analysis for the application of digital twin and structural health monitoring. It is indicated that GNNs can be a powerful tool for resolving complex physical problems, thereby assisting in advancing science and enhancing engineering productivity. • A surrogate model is developed to accelerate the numerical simulation. • Global attributes are proposed to mitigate potential issues with graph deep learning. • Accurate prediction of physical quantities of interest is achieved. • Significant improvements in prediction speed over numerical simulation are achieved. • Powerful generalizability is obtained through the use of graph neural networks. [ABSTRACT FROM AUTHOR]

Current procedures for the rapid inspection of buildings and infrastructure are subjective, time-consuming, and cumbersome to document, necessitating new technologies to automate the process and eliminate these shortcomings. Fortunately, recent developments in imaging devices and artificial intelligence, such as computer vision, provide the necessary tools for this, though they are not yet integrated into infrastructure applications. In this paper, we propose an end-to-end pipeline that generates damage-augmented digital twins for buildings at LOD3, including geometrical information as well as data pertaining to damage condition and its characterization. Our framework incorporates multiple-view images to (1) create a level of detail model, (2) segment damage information, and (3) characterize damage. The core of the method is the structure from motion, which is used to reconstruct the building scene, and machine-learning models that segment and characterize damage. In contrast to current practices, our method does not require manual intervention, generates lightweight models, and can be applied to a wide range of assets. The results generated with our pipeline represent a significant step towards an automated infrastructure damage assessment. We intend to expand our work in the future to include real-time applications and applications to other types of infrastructure. Codes and data sets are publicly available (https://github.com/eesd-epfl/DADT%5fbuildings and https://doi.org/10.5281/zenodo.7767478). • A novel framework to generate Damage Augmented Digital Twins (DADT) of buildings. • State-of-the-art solutions for 3D models, crack segmentation and characterization. • Bench-marking data set for evaluating the automated generation of DADT of buildings. [ABSTRACT FROM AUTHOR]

Earthwork operations are crucial parts of most construction projects. Heavy construction equipment and workers are often required to work in limited workspaces simultaneously. Struck-by accidents resulting from poor worker and equipment interactions account for a large proportion of accidents and fatalities on construction sites. The emerging technologies based on computer vision and artificial intelligence offer an opportunity to enhance construction safety through advanced monitoring utilizing site cameras. A crucial pre-requisite to the development of safety monitoring applications is the ability to identify accurately and localize the position of the equipment and its critical components in 3D space. This study proposes a workflow for excavator 3D pose estimation based on deep learning using RGB images. In the proposed workflow, an articulated 3D digital twin of an excavator is used to generate the necessary data for training a 3D pose estimation model. In addition, a method for generating hybrid datasets (simulation and laboratory) for adapting the 3D pose estimation model for various scenarios with different camera parameters is proposed. Evaluations prove the capability of the workflow in estimating the 3D pose of excavators. The study concludes by discussing the limitations and future research opportunities. [ABSTRACT FROM AUTHOR]

By constructing digital twins (DT) of an integrated energy system (IES), one can benefit from DT's predictive capabilities to improve coordinations among various energy converters, hence enhancing energy efficiency, cost savings and carbon emission reduction. This paper is motivated by the fact that practical IESs suffer from multiple uncertainty sources, and complicated surrounding environment. To address this problem, a novel DT-based day-ahead scheduling method is proposed. The physical IES is modelled as a multi-vector energy system in its virtual space that interacts with the physical IES to manipulate its operations. A deep neural network is trained to make statistical cost-saving scheduling by learning from both historical forecasting errors and day-ahead forecasts. Case studies of IESs show that the proposed DT-based method is able to reduce the operating cost of IES by 63.5%, comparing to the existing forecast-based scheduling methods. It is also found that both electric vehicles and thermal energy storages play proactive roles in the proposed method, highlighting their importance in future energy system integration and decarbonisation. • A deep learning and digital twins (DT) based scheduling method is proposed. • Physical IES performance is addressed by its virtual replica's predictive capability. • A machine learning method is proposed to address multiple uncertainty challenges. • Historical U.K. datasets are used for performance evaluations. • Results demonstrate the Digital Twin model reduces the extra operating cost by 63.5%. [ABSTRACT FROM AUTHOR]