Search

The accidental leakage of gaseous hydrogen from hydrogen refueling station facilities has the potential to result in a large fire and explosion catastrophe. The ability to forecast the spatiotemporal evolution of hydrogen plumes in real-time is crucial for monitoring the distribution of plume concentrations and mitigating the risk of fire and explosion. The utilization of deep learning has been extensively employed in a diverse range of spatiotemporal forecasting tasks. However, these approaches encounter limitations in accurately quantifying uncertainty when predicting spatiotemporal concentrations and plume boundaries. This research paper introduces a novel model called DPSTNN_H 2 Evolution, which is a deep probabilistic spatial-temporal neural network designed for forecasting the spatiotemporal evolution of hydrogen plumes. The benchmark dataset is constructed by the implementation of numerical simulations pertaining to the inadvertent leak of hydrogen within a hydrogen refueling station. The findings of the study indicate that incorporating quantified uncertainty information might enhance the precision of plume boundaries and the resilience of plume concentrations when predicting the spatiotemporal development of hydrogen. By employing Monte Carlo sampling with a sample size of m = 100 and a dropout rate of p = 0.1 , the model demonstrates the ability to provide real-time inference for 10 instances of plumes, while maintaining a high level of accuracy with R2 = 0.9829. In comparison to the current state-of-the-art model, the proposed model demonstrates superior accuracy and robustness in forecasting the spatiotemporal evolution of hydrogen plumes. In general, our proposed model has the potential to serve as a dependable option for the development of a digital twin for emergency management of hydrogen refueling stations. • Probabilistic spatial-temporal neural network model for hydrogen plume evolutions is proposed. • Normalized uncertainty contours and uncertainty intervals to improve accuracy and robustness. • Numerical experiment of hydrogen leakage and dispersion in refueling station are conducted. • Optimal hyper-parameters of the proposed model are determined • Comparison between the proposed model with point-estimation based model is conducted [ABSTRACT FROM AUTHOR]

The benefits of having digital twins of urban buildings are numerous. However, a major difficulty encountered in their creation from airborne LiDAR point clouds is the effective means of accurately reconstructing significant occlusions amidst point density variations and noise. To bridge the noise/sparsity/occlusion gap and generate high fidelity 3D building models, we propose APC2Mesh which integrates point completion into a 3D reconstruction pipeline, enabling the learning of dense geometrically accurate representation of buildings. Specifically, we leveraged complete points generated from occluded ones as input to a linearized skip attention-based deformation network for 3D mesh reconstruction. In our experiments, conducted on 3 different scenes, we demonstrate that: (1) Compared to SoTA methods like NDF, Points2Poly, Point2Mesh, and a few others, APC2Mesh ranked second in positional RMSE and first in directional RSME, with error magnitudes of 0.0134 m and 0.1581, respectively. This indicates the efficacy of APC2Mesh in handling the challenges of airborne building points of diverse styles and complexities. (2) The combination of point completion with typical deep learning-based 3D point cloud reconstruction methods offers a direct and effective solution for reconstructing significantly occluded airborne building points. As such, this neural integration holds promise for advancing the creation of digital twins for urban buildings with greater accuracy and fidelity. Our source code is available at https://github.com/geospatial-lab/APC2Mesh. [ABSTRACT FROM AUTHOR]

Hydrogen release and dispersion from hydrogen refuelling stations have the potential to cause explosion disaster and bring significant causalities and economic losses to the surroundings. Real-time spatial hydrogen plume concentration prediction is essential for the quick emergency response planning to dissipate such flammable vapor cloud and prevent explosion disaster. Deep learning approaches have recently been applied to real-time gas release and dispersion modeling, however, are 'over-confident' for spatial plume concentration and boundary estimation, which could not support the robust decision-makings. This study proposes a hybrid deep probability learning-based spatial hydrogen plume concentration prediction model, namely DPL_H 2 Plume by integrating deep learning and Variational Bayesian Inference. Numerical model of hydrogen release and dispersion from hydrogen refuelling station is built to construct the benchmark dataset. By using such dataset, two pre-defined parameters, namely Monte Carlo sampling number m = 300 and dropout probability p = 0.1 are determined to ensure the model's tradeoff between inference accuracy and efficiency. Comparison between our proposed model and the state-of-the-art model is also conducted. The results demonstrate that our model exhibits a competitive accuracy of R 2 = 0.97 as well as an inference time 3.32 s. In addition, our model gives the comprehensive estimations including not only spatial hydrogen plume concentration but also its uncertainty. Also, our model provides the more accurate estimation at plume boundary compared to the state-of-the-art model. Overall, our proposed model could provide reliable alternative for constructing a digital twin for emergency management of hydrogen refuelling station. • Deep probabilistic learning-based hydrogen plume prediction model is proposed. • Optimal hyperparameters of variational Bayesian inference are determined. • Model accurately predicts real-time spatial hydrogen plume concentration given scenario-related parameters. • Model predicts additional uncertainty information for comprehensive emergency decision-makings. [ABSTRACT FROM AUTHOR]

Industrial collaborative robots play an essential role in smart manufacturing because they improve productivity while also ensuring workplace safety. However, the development of prognostic and health management systems to ensure the reliability of these robots has been a major challenge due to the lack of fault data. This paper proposed a digital twin scheme based on the fusion of the robot kinematic and dynamic models' information down to the powertrains (i.e., the joints motor, and gear) along with the control algorithms and uncertainty accommodation based upon deep learning. The presented digital twin concept has the potential to propel simulation-based fault prediction. We also highlight and discuss challenges and opportunities around the development of the hybrid digital twin for condition monitoring of industrial collaborative robots. [ABSTRACT FROM AUTHOR]

The absence of effective quality prediction methods for functional composite materials (FCMs) produced by laser additive remanufacturing (LARM) hampers their application due to the complex cross-scale defects, including surface cracks and thermal damage to the internal reinforcement phase. This paper presents a multi-task deep learning-empowered digital twin for predicting visible and invisible defects in the fabricating process of FCMs. The dimensions of FCM trajectory, thermal damage to the reinforcement phase, and forming cracks were predicted via a parallel multi-task deep learning model. The dynamic visualization of the digital twin is realized through cross-sectional modeling and provides an intuitive and effective perception for monitoring the process. [ABSTRACT FROM AUTHOR]

The meticulous reconstruction of three-dimensional (3D) braided composite materials serves as a crucial foundation for achieving high-fidelity simulations. Nonetheless, the transition from tomographic images to a 3D mesh entails a laborious and time-intensive process. To address this, an integrated procedure based on artificial intelligence is proposed for reconstructing meshes from tomograms. The initial stage of the process involves employing artificial intelligence techniques to segment complex contours and optimize high-dimensional contours. This facilitates the input of high-quality images needed to reconstruct accurate digital twins with strong convergence. The subsequent reconstruction phase integrates various calculations, including shape interpolation, contour extraction, 3D surface reconstruction, 3D mesh reconstruction, and element data interpolation. During this process, optimization objectives are set to minimize the deviation between the digital twin's surface and the actual surface, as well as to optimize the aspect ratio of the element mesh. Upon completion of the aforementioned steps, high-quality input files suitable for finite element calculations are directly generated. Ultimately, the proposed method utilizes the reconstructed finite element model for mechanical analysis, and the results are found to be in good agreement with experimental tests. This method offers an efficient and rapid way to achieve high-quality reconstruction of complex digital twins. [Display omitted] • The proposed procedure to reconstruct meshes from tomograms is integrated, enabling batch processing. • The deep learning models are employed to segment the intricate yarns contours of 3D5D braided composites. • The Ramer–Douglas–Peucker algorithm is used to capture high-dimensional contour features and improve convergence. • A decimating filter employing the quadric-based edge-collapsed strategy improves volumetric meshing. [ABSTRACT FROM AUTHOR]

Generalization remains a significant challenge in the application of deep learning methods, particularly in predicting mooring line tension for moored floating structures. The dynamic and complex nature of marine environments makes accurate prediction of mooring line tension a formidable task. Existing deep learning models, constrained by the data distribution of their training sets, typically perform well within the scope of these datasets. However, their limited generalization capability becomes apparent when confronted with unknown and changing marine conditions, resulting in a decline in prediction accuracy. To address this issue, this paper proposes a Domain Generalization (DG) model approach, which incorporates a distribution matching regularization term to the model's loss function, aimed at reducing distributional disparities between data from different domains. The proposed method was compared with existing Deep Neural Network (DNN) model method through validation with engineering case studies. The results indicate that the DG model method significantly surpasses the DNN model in terms of generalization capabilities. The mooring line tension prediction model developed herein holds potential for constructing digital twins, offering near real-time data support for moored floating engineering projects. This facilitates timely and accurate decision-making in response to environmental changes. • Develops a Domain Generalization model to predict mooring tension in unknown sea conditions. • Demonstrates improvements in model generalization, surpassing traditional Deep Neural Network (DNN) methods. • The effectiveness of the model was validated through a practical engineering case study of a floating wind turbine. [ABSTRACT FROM AUTHOR]

Irregular ocean waves contribute to random and non-linear hydrodynamic forces on marine and offshore platforms, in particular those made of slender bodies such as jack-ups. Keeping track of the structural fatigue, integrity, and reliability renders benefits to these platforms such as enabling predictive maintenance and risk-based decision support. While computational techniques are useful in providing insights to deepen our understanding of the physics involved, these are resource-intensive and not suited for predictive capabilities. Here we introduce an effective deep learning method based on Long Short Time Memory (LSTM) to predict unsteady hydrodynamic forces on jack-up legs, which are mainly governed by non-linear drag forces, as well as dynamic responses of the jack-up in terms of reaction forces and hull motion. The numerical OF-ABAQUS solver (one-way fluid-structure coupling) designed for jack-up by Li et al. (2022, 2023) is used to generate a database for the deep neural network, which takes in wave elevation time series as the input. A leaky non-linear rectification is used to trace the relationship between wave elevation (as the input) and hydrodynamic forces as well as structural responses (as the outputs). The LSTM model is trained using a stochastic gradient descent method, and the results are compared with the full-order numerical simulations. Once the hyper-parameters of the LSTM algorithm are identified, further tuning for each sea-state is not required. Instead, the same LSTM algorithm can be used for different sea-states. The predictions made with our LSTM model are real-time, 96,000 times faster than the full-order simulations, within a 6% error threshold, and use a negligible amount of central processing unit (CPU) power. This powerful tool can be used to obtain real-time predictions using computers on offshore platforms without high-performance computers inland. Further, the proposed LSTM-based method can be applied to create digital twins for jack-up platforms. [ABSTRACT FROM AUTHOR]

• Integrated digital twins Framework for revolutionizes defect diagnosis for pumps. • Domain adaptation boosts diagnostic model accuracy. • Promising potential for scalable industrial applications. Bearing defects in centrifugal pumps represent a prevalent source of equipment failure, often resulting in significant downtime and maintenance expenses. Deep learning algorithms can be used to detect defects. However, in the real world, there is always a scarcity of labeled data. To address this challenge, we propose a framework integrating digital twin technology with domain adaptation for accurate diagnosis of bearing defects. The proposed framework leverages the concept of digital twins to create a virtual representation of the pump bearing, enabling real-time monitoring and simulation of operating conditions. Domain adaptation techniques are then applied to transfer knowledge from synthetic data generated by the digital twin to the actual operating environment, overcoming the domain gap between synthetic and real-world data. The results highlight the potential of digital twin-assisted combined with domain adaptation techniques for enhancing predictive maintenance strategies in industrial applications. [ABSTRACT FROM AUTHOR]

Reliable traffic infrastructure is a key factor for any country's economy. However, aging bridges often require renovation or reconstruction. Promising approaches for enhancing asset management and cost reduction are digital twins and predictive maintenance strategies. However, the creation of geometric-semantic as-is models as a basis for digital twins currently involves labor-intensive manual data capture and modeling. Modern deep learning models, such as Kernel Point Convolution (KPConv) show promising results in reducing the time needed to create digital twins by semantically segmenting point cloud data but have so far been hindered by the lack of a reliable quality measure, which can predict when the model's prediction can be trusted. This is especially viable in the construction industry, where objects and sensors may vary widely between projects and contractors. In this work, we present Bayesian neural networks, implemented through Variational Inference and Monte-Carlo dropout as approaches to conducting inference with KPConv, which show improvements in the confidence estimation and out-of-domain (OOD) detection in the scans of typical infrastructure point clouds. When confronted with different domain shifts to the test data, such as a change of scanning device and introduction of unseen classes the proposed model showed a 25.3% decrease in expected calibration error (ECE) and a 4.82 increase in OOD detection in terms of outlier-intersection-over-union (O-IoU) on average with respect to a deterministic baseline. • Bayesian neural networks applied in point cloud semantic segmentation. • Training stochastic models with Variational Inference improve calibration. • Out of domain detection was improved with a single stochastic layer. • Model robustness to domain shift from mobile to terrestrial laser scanner. [ABSTRACT FROM AUTHOR]

The thermal errors in ball screws present complex spatial-temporal characteristics and exhibit persistent long-term memory, significantly affecting the machining accuracy of the whole machine tool in applications. Deep learning has emerged as a promising approach to predict these errors using thermal data. However, the scarcity of extensive thermal datasets, typically due to the prohibitive costs and time required for experimental acquisition, hampers the comprehensive exploitation of spatial-temporal and long-term memory attributes. Moreover, the current thermal error compensation system does not support the deployment of deep learning-based thermal error models due to its weak real-time performance. These limitations lead to suboptimal prediction accuracy and inadequate error compensation. Addressing this challenge, a novel spatial-temporal interactive integration network is proposed, and this is a sophisticated model that synergistically blends a time memory gate and a spatial-temporal fusion gate within an advanced attention-based spatial-temporal graph convolutional framework. Our network ingeniously leverages spatial data to anchor long-term memory while employing temporal data for selective memory feature extraction, facilitating a refined integration of spatial-temporal features. Concurrently, the integration of gated recurrent units and a time attention layer meticulously extracts and enhances temporal features, bolstered by our innovative time memory gate, which is adept at handling small-sample scenarios and refining thermal error predictions. Critically, the digital twin system for thermal error compensation is constructed to improve the system's real-time performance. The integration of this network into the digital twin system marks a pivotal advancement in thermal error compensation. Our empirical results underscore the system's remarkable robustness and superior predictive accuracy, demonstrating a significant reduction in positioning and machining errors (over 90% and 80% respectively) even with constrained thermal data inputs. These advancements not only delineate a substantial leap in predictive accuracy but also underscore our contribution to reducing operational costs and enhancing the efficacy of machine tools. [Display omitted] • Spatial-temporal fusion gate is proposed to extract spatiotemporal features. • STIIN is proposed to extract spatiotemporal features of thermal error. • Digital twin system for thermal error compensation is established. • STIIN model is embedded into digital twin system. • Robust compensation is realized with small-volume thermal information data. [ABSTRACT FROM AUTHOR]

Health monitoring of structures using ultrasonic guided waves is an evolving technology with potential applications in monitoring pipelines, civil bridges, and aircraft components. However, the sensitivity of guided waves to external parameters affects the reliability of monitoring systems based on them. These influencing factors and experimental related factors cannot be perfectly modeled, which give rise to the discrepancy between numerical simulations and experimental measurements. Therefore, it is important to address this inevitable discrepancy and generate close-to-experiment simulations. In this work, we present a deep learning-based Digital Twin framework containing multi-fidelity modeling to reduce the discrepancy between measurements and simulations and a deep generative model to generate close-to-experiment guided wave responses by harnessing the vital characteristics of the two sources. These realistic simulations (close to experiment) can then be used in assessing the reliability of health monitoring system by generating probability of detection curves. Furthermore, they can also be used for augmenting the training data for a machine learning algorithm. We use a measurement dataset corresponding to crack propagation and simulations to validate the proposed framework. The results show that the discrepancy is indeed reduced to a great extent, furthermore, we also show that this framework enables the computation of probability of detection from close-to-experiment data as a direct consequence of rapid generation of realistic simulations. • Structural health monitoring based on ultrasonic guided waves. • Discrepancy reduction between simulations and experiments. • A novel unifed deep neural network model blending simulations and experiments. • Enhanced ultrasonic guided waves signals generation. • Discrepancy quantification through phase and amplitude based misfit metrics. • Metamodel based computation of probability of detection. [ABSTRACT FROM AUTHOR]

In this article, we present a deep learning-based surrogate model for spatio-temporal prediction of cavitating fluid flow. Specifically, we introduce a finite element-inspired rotation equivariant hypergraph neural network for inferring and predicting dynamical behaviors of cavitating flow. We generate ground-truth spatial–temporal data by simulating a full-order variational system based on homogeneous mixture-based cavitation theory. We consider the flow past a NACA0012 hydrofoil to examine the predictive ability of the proposed graph neural network for cavitation dynamics. Results demonstrate that the network achieves stabilized and accurate temporal predictions of the system states, successfully forecasting the evolution patterns of individual cavitation events. Additionally, comparisons of predicted fluid loading coefficients are in good agreements with the ground-truth values. We also discuss some challenges encountered in the long-term prediction of flow patterns across multiple cavitation events. The proposed framework has implications for design optimization, active control and development of a physics-based digital twin of a cavitating marine propeller. [Display omitted] • Finite element-inspired graph neural network for cavitation dynamics. • Stabilized and accurate spatio-temporal predictions of system states. • Successful forecasting of individual cavitation events. • Good comparison of predicted force coefficients with ground truth values. [ABSTRACT FROM AUTHOR]

The influence of weather conditions on sea state, and in particular on the dynamic evolution of waves, is an important issue that affects several areas, including maritime traffic and the planning of coastal works. To collect relevant data, buoys are used to set up distributed sensor networks along coastal areas. However, unfavourable weather conditions can lead to downtime, which can be extended due to maintenance issues. The ability to improve the robustness of these sensor systems using predictive models, i.e. digital twins, to interpolate and extrapolate missing data is an important and growing area of research. To accomplish such a task, models must be found that can account for both the spatial and temporal dynamics of the input data to correctly estimate the variables of interest. In this work, a deep learning architecture is proposed to realize a digital twin for the monitoring buoy for significant wave height estimation using spatial and temporal information about the wind field in the area of interest. The proposed methodology was applied to a case study using wave height data from an Italian Sea Monitoring Network buoy installed near the coast of Sicily and wind field data from the Copernicus Climate Change Service ERA5 reanalysis. The reported results show that the use of a multi-block hybrid deep neural network consisting of convolutional layers for spatial feature extraction and short-term memory layers for modelling the involved dynamics, which takes into account the buoy surrounding area, outperforms other empirical, numerical, machine learning and deep learning methods used in the literature. • The wind field and the significant wave height relation are modelled with a hybrid neural network. • Single and multiple CNN-LSTM layers are compared. • Peak estimation is addressed through dataset balancing strategy. • Specific performance analysis on extreme events has been reported [ABSTRACT FROM AUTHOR]

Large volumes of 3D parametric datasets, such as building information modeling (BIM), are the foundation for developing and applying smart city and digital twin technologies. Those datasets are also considered valuable tools for efficiently managing rebuilt structures during the operation and maintenance stages. Nevertheless, current approaches developed for the scan-to-BIM process rely on manual or semi-automatic procedures and insufficiently leverage semantic data in point clouds. These methods struggle to accurately represent large-scale indoor complex layouts and extract details from irregular-shaped unstructured elements, causing inefficiencies in BIM model generation. To address these issues, we propose an innovative scan-to-BIM framework based on deep learning algorithms and raw point cloud data, enabling the automatic generation of 3D models for both structured and unstructured indoor elements. Initially, we propose an enhanced deep learning neural network to improve the point clouds' semantic segmentation accuracy. Subsequently, an efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. The proposed workflow can reconstruct large-scale data with multiple room layouts of Manhattan or non-Manhattan structures and reconstruct 3D models automatically by using a BIM parametric algorithm implemented in Revit software. Moreover, we introduce a robust method for unstructured elements to automatically generate corresponding 3D BIM models, even when the incorporating semantic information is incomplete. The proposed approach was evaluated on synthetic and real data for different scales and complexities of indoor scenes. The results of the experiments demonstrate that the improved model significantly enhances the overall semantic segmentation accuracy compared to the baseline models. The proposed scan-to-BIM framework is efficient for indoor element 3D reconstruction, achieving precision, recall, and F-score values ranging from 96% to 99%. The generated BIM models are competitive with traditional methods regarding model completeness and geometric accuracy. • We propose an innovative scan-to-BIM framework based on deep learning algorithms: • We propose an enhanced deep-learning neural network to improve the point clouds' semantic segmentation accuracy. • A novel and efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. It can reconstruct large-scale data with multiple room layouts in 3D model automatically by using a BIM parametric • For unstructured elements, we propose a robust method to automatically generate 3D BIM models, even when the semantic information is incomplete. [ABSTRACT FROM AUTHOR]

• A digital twin-assisted dual transfer method was proposed with information and model adaptation for bearing fault diagnosis. • DTa-DT aims to simultaneously synthesize information transfer and model transfer for domain transfer error minimization. • DTd-IT method was proposed to information transfer with low distribution difference of twin data adapted. • DAd-MT method, composed of DBTN, was proposed to model transfer with feature distribution difference of domain-model adapted. • Experiments and ablation study verified that the proposed DTa-DT method can be well applied to small sample learning. Rolling bearing fault diagnosis is of great importance to the safety management of mechanical equipment. The scarcity of labelled fault data makes it difficult to adequately perform the training process of intelligent diagnosis models, and this will result in these intelligent models not being effectively and widely used in practice. Although some recent studies have verified that the addition of dynamic model response to the training process will greatly improve the ability of the model with low cost and high efficiency, it is still stuck in poor effect caused by large information distribution difference between dynamic model response and real measured data. Focusing on this issue, a digital twin-assisted dual transfer (DTa-DT) method with information and model adaptation was proposed for rolling bearing fault diagnosis. Different from the traditional digital-analogue driven transfer methods, the proposed DTa-DT aims to simultaneously synthesize data information transfer and feature model transfer together with domain transfer error minimization. In particular, it should be noted that the DTa-DT architecture consists of a dual transfer learning process, including digital twin-driven information transfer (DTd-IT) and digital-analogue-driven model transfer (DAd-MT), where the information is collaborated with the model to improve the integrated transfer diagnosis effect under sampling. On one aspect, with the employment of bearing dynamic model responses, DTd-IT is innovatively designed to establish the transfer of dynamic information and measured information. The information distribution difference between these twin data and real measured data is effetely adjusted with the introduced actual inference components, where the twin data with low information distribution difference can be well fusion generated by the information transfer digital twin (ITDT) model. On the other aspect, considering the truth that there are still small sample cases of real measured data and information distribution differences will affect the quality of the twin data, a digital-analogue driven model transfer (DAd-MT) method is further proposed, where the deep branch transfer network (DBTN) model with improved convolutional neural network (CNN) is used to achieve an accurate fault diagnosis effect with the help of digital twin data. Experiments and wear analysis verified that the proposed DTa-DT can significantly reduce the distribution difference between the dynamic model response and the real measured data, thus achieving low-cost and efficient rolling bearing transfer diagnosis compared to other ten state-of-the-art deep learning models. It can be predicted that the proposed dual transfer architecture provides more opportunities for the practical application of intelligent fault diagnosis under small sample sizes. [ABSTRACT FROM AUTHOR]

Optimizing the combustion process by predicting combustion parameters during prolonged engine operation is crucial for engine maintenance. This study presents a zero-dimensional (0-D) prediction model that integrates the advantages of model-driven and data-driven approaches. Initially, the snake optimization algorithm (SO) is employed to address the challenges related to low parameter fitting accuracy and multiple solutions in calculating Wiebe parameters. Subsequently, a convolutional neural network-bidirectional long short-term memory neural network (CNN–Bi-LSTM) is devised to establish a nonlinear correlation between operating parameters and Wiebe parameters. The structural parameters of CNN–Bi-LSTM are then optimized using the SO algorithm (SO–CNN–Bi-LSTM). Ultimately, a 0-D prediction combustion model is formulated by amalgamating the Wiebe function with the neural network, enabling real-time prediction of combustion results and generalization analysis of prediction performance under non-calibrated conditions. The findings demonstrate that the combustion model exhibits heightened accuracy, thereby establishing a robust technical foundation for the development of a digital twin in the engine combustion process. • The SO algorithm for calculating Wiebe parameters was applied. • A parameter identification model based on SO–CNN–Bi-LSTM was proposed. • A 0-D prediction model was constructed by combining Wiebe function with neural network. • Based on the prediction model, the combustion process was simplified and reconstructed. [ABSTRACT FROM AUTHOR]

Digital twin has become a critical technical solution for satellite in-orbit service and maintenance. The majority of existing research in this field has focused on constructing system-level digital twins, with various typical applications that primarily describe the operating condition of satellite subsystems. However, achieving a more meticulous portrayal of in-orbit satellites necessitates the inclusion of physical field information, e.g., the temperature field serves as a fundamental basis for characterizing the thermodynamic state and plays a crucial role in precise thermal management of a satellite. The main challenge for realizing thorough state detection and real-time response lies in the cross-level model fusion of system simulation and physical field calculation. To overcome this problem, this paper proposes a Hybrid Digital Twin (HDT) framework that seamlessly integrates the digital twin at the system-level with its physical field-level counterpart, enabling a comprehensive and detailed representation of modeling objects. Specifically, at the physical field-level, deep learning methods are utilized to construct an accurate surrogate model, offering a lightweight, computationally efficient and accelerated execution alternative to the Finite Element Analysis models. At the system-level, multi-domain unified modeling technology is employed to ensure both efficiency and accuracy of coupling simulation. Moreover, a co-simulation approach is developed on the basis of the Functional Mock-up Interface, overcoming the bottleneck of cross-level information interaction and computing time synchronization. The practical utility of the proposed framework is verified by a real-world engineering problem, i.e., satellite temperature field perception and attitude control in the context of the harsh space environment. Whole process engineering experiments demonstrate the effectiveness of the HDT, with mean absolute errors below 1 K for temperature field reconstruction and within 1° for attitude control. [ABSTRACT FROM AUTHOR]

Given the pervasive use of deep learning models across various domains, ensuring model security has emerged as a critical concern. This paper examines backdoor attacks, a form of security threat that compromises model output by poisoning the training data. Our investigation specifically addresses backdoor attacks on object detection models, vital for security-sensitive applications like autonomous driving and smart city systems. Consequently, such attacks on object detection models could pose significant risks to human life and property. Consequently, backdoor attacks on object detection could pose serious threats to human life and property. To elucidate this security risk, we propose and experimentally evaluate five backdoor attack methods for object detection models. The key findings are: (1) Unnecessary Object Generation : a globally embedded trigger creating false objects in the target class; (2) Partial Misclassification : a trigger causing specific class misclassification; (3) Global Misclassification : a trigger reclassifying all objects into the target class; (4) Specific Object Vanishing : a trigger causing non-detection of certain objects; (5) Object Position Shifting : a trigger causing bounding box shifts for a specific class. To assess attack effectiveness, we introduced the Attack Success Rate (ASR), which can surpass 1 in object detection tasks, thus providing a more accurate reflection of the attack impact. Experimental outcomes indicate that the ASR values of these varied backdoor attacks frequently approach or surpass 1, demonstrating our method's capacity to impact multiple objects simultaneously. Additionally, to augment trigger stealth, we introduce Backdoor Attack with Wavelet Embedding (BAWE), which discreetly embeds triggers as image watermarks in training data. This embedding method yields more natural triggers with enhanced stealth. Highly stealthy triggers are less detectable, significantly increasing the likelihood of attack success and efficacy. We have developed a Transformer-based network architecture, diverging from traditional neural network frameworks. Our experiments across various object detection datasets highlight the susceptibility of these models and the high success rate of our approaches. This vulnerability poses significant risks to digital twin systems utilizing object detection technology. Our methodology not only enhances trigger stealth but also suits dense predictive tasks and circumvents current neural network backdoor attack detection methods. The experimental findings expose key challenges in the security of object detection models, particularly when integrated with digital twins, offering new avenues for backdoor attack research and foundational insights for devising defense strategies against these attacks. • We embed watermarks as invisible triggers in training data using wavelet transform. • Our method is suited for dense prediction tasks like object detection. • We use watermarks as triggers and are not limited to standard neural network. [ABSTRACT FROM AUTHOR]

Accurate and efficient inspection of rebar dimensions has proven to be a persistent challenge for researchers and practitioners. This paper introduces a semantically enriched 3D model-based system that employs computer vision and deep learning for location-aware identification and tracking of rebar issues. The system comprises four modules: (A) digital twin generation, (B) segmentation, (C) inspection, and (D) issue identification and tracking. The generation module constructs 3D models from rebar structures. The segmentation and inspection modules analyze the 3D models, enriching them with semantic information. The issue identification and tracking module exchanges information between the semantically enriched 3D models and the building information models across time. An experiment on a column rebar cage is conducted. A precision of over 90% and a recall of over 97% are reported in 3D instance segmentation. Diameter inspection achieves an accuracy of 95.5% for large-size rebars. Spacing inspection achieves a mean relative error of 0.98%. The defective spacing is identified and tracked. • Fuses deep learning and computer vision to construct 3D semantic rebar model. • BIM- and digital-twin-based information system fulfills rebar quality control. • Autonomous rebar inspection pipeline directly generates issue reports from images. [ABSTRACT FROM AUTHOR]

The significance of understanding the flow past a bluff body (BB) lies in its relevance to ocean, structural, and environmental applications. Capturing the transient flow behaviors with fine details requires extensive computational power. To address this, the present study develops an improved method for modeling the complex flow dynamics around a BB under steady and unsteady conditions. It is a deep learning (DL)-enhanced reduced-order model (ROM) that leverages the strengths of proper orthogonal decomposition (POD) for model reduction, convolutional neural network-long short-term memory (CNN-LSTM) for feature extraction and temporal modeling, and Bayesian optimization for hyperparameter tuning. The model starts with dimensionality reduction, followed by DL optimization and forecasting, and terminates with flow field reconstruction by combining dominant POD modes and predicted amplitudes. The goal is to establish a DL-driven ROM for fast and accurate modeling of the flow evolution. Based on the comparison of millions of data samples, the predictions from the ROM and CFD are considerably consistent, with a coefficient of determination of 0.99. Furthermore, the ROM is ∼10 times faster than the CFD and exhibits a robust noise resistance capability. This study contributes a novel modeling approach for complex flows, enabling rapid decision-making and interactive visualization in various applications, e.g., digital twins and predictive maintenance. • Developing a deep learning-enhanced reduced-order model for transient flow modeling. • Establishing a Bayesian-optimized CNN-LSTM for accurate flow forecasting. • Investigating spatiotemporal bluff body flows under steady and unsteady conditions. [ABSTRACT FROM AUTHOR]

Blow-outs occurred on offshore platform and associated fires have been recurrent during the previous few decades, and poses a potential safety hazard to humans, property and the surrounding environment. Although the real-time forecast based on deep learning have shown promise in the fields of fire modelling and hazardous area evaluations, jet fire spatio-temporal modelling has not yet undergone sufficient investigation in complex ocean engineering cases like offshore platforms. This research therefore proposes a deep learning-based framework for jet fire spatio-temporal probabilistic real-time forecast by developing the Hybrid-VB-ConvSTnn model, which integratesConvGRU and variational Bayesian inference. And the significant hyperparameters were locally optimized through sensitivity analysis and finally identified as Monte Carlo (MC) sampling number m = 100 and dropout probability p = 0.1. By performance comparison with different models, the Hybrid-VB-ConvSTnn model shows competitive spatio-temporal forecasting capabilities in terms of both real-time (Inference time = 0.83s) and accuracy (R 2 = 0.982). Moreover, the Hybrid-VB-ConvSTnn model could provide the additional uncertainty inferences based on the probability density of the Bernoulli distribution, which avoids the inherent shortcomings of "overconfidence" for traditional point-estimate models and lends credibility to flame boundary identification. The proposed framework could support the digital twin-based fire emergency management on offshore platforms by more comprehensive and robust risk evaluation. • Deep learning-based advanced approach framework for natural gas jet fire spatiotemporal probabilistic forecast is developed. • Uncertainty inference in flame boundary forecast can be determined with variational Bayesian inference. • Hyperparameters that determine model performance are locally optimized through sensitivity analysis. • Real-time forecast provides an alternative for digital twin model of jet fire accidents management on offshore platform. [ABSTRACT FROM AUTHOR]

High-definition (HD) maps provide a complementary source of information for Advanced Driver Assistance Systems (ADAS), allowing them to better understand the vehicle's surroundings and make more informed decisions. HD maps are also largely employed in virtual testing phases to evaluate the behavior of ADAS components under simulated conditions. With the advent of autonomous sensorized vehicles, raw machine-oriented data will be increasingly available. The proposed pipeline aims to provide a high-level semantic interpretation of raw vehicle sensory data to derive, in an automated fashion, lane-oriented HD maps of the environment. We first present RoadStarNet, a deep learning architecture designed to extract and classify road line markings from imagery data. We show how to obtain a semantic Bird's-Eye View (BEV) mapping of the extracted road line markings by exploiting frame-by-frame localization information. Then, we present how to progress to a graph-based representation that allows modeling complex road line markings' structures practically, as this representation can be leveraged to produce a Lanelet2 format HD map. Lastly, we experimentally evaluate the proposed approach in real-world scenarios in terms of accuracy and coverage performance. • Data interpretation of survey vehicles for lane-level HD map generation. • Pixel-level mapping and graph-based reconstruction of road line markings. • Experimental evaluation in urban and extra-urban scenarios. [ABSTRACT FROM AUTHOR]

This paper establishes a digital twin (DT) enabled model that aims to achieve real-time monitoring and control to enhance the overall performance of tunnel boring machines (TBM) during excavation. Through the Internet of Things (IoT) sensors that are installed on TBM, the actual operating status is transferred from the physical object to the digital model. Deep learning models that integrated graph convolutional network (GCN) and long short-term memory (LSTM) networks are employed for reliable TBM performance estimation. An online multi-objective optimization (MOO) approach is then proposed to determine the optimal operating parameters for advanced control. Real data from Singapore's Circle Line 6 C885 project is utilized as a case study to examine the applicability of the developed digital twin model. TBM's penetration rate, over-excavation ratio, energy consumption, and tool wear are selected as the objectives (O 1 to O 4) for optimization. The results indicate that (1) The digital twin model can provide a reliable estimation of TBM's performance indicators with a high coefficient of determination (R 2) values. (2) TBM's overall performance can be effectively improved through the proposed online optimization method, with an overall improvement of 21.12%. (3) The proposed online optimization method is more practical and creates better improvement than the conventional MOO approach. This study addresses the existing research gaps of lacking a reliable method to guide TBM operation with enhanced performances for the major objectives. It also contributes to proposing a novel intelligent approach that integrates DT, deep learning, and an online-NSGA-II method for TBM operation optimization during tunnel excavation. • A digital twin enabled approach is proposed for real-time intelligent control for TBM operation. • An MOO approach that integrates deep-learning and online optimization is established. • Data from a realistic tunnel project in Singapore is used to demonstrate as a case study. • TBM's efficiency, safety, energy consumption and tool wear are considered as objectives. • An overall improvement of 21.12% is achieved by implementing the proposed method. [ABSTRACT FROM AUTHOR]

• Numerical simulation modeling is complex and computationally intensive. • Digital twin deployment requires lightweight and responsive systems. • Process sensor data drives deep learning algorithms to implement digital twins. • Generative adversarial networks enable monitoring of heat distribution changes. • Results contribute to weld quality assurance and blade structure optimization. In this paper, a novel method of using deep learning to construct a digital twin of the thermos-compression bonding blade's temperature field is presented. Usually, a heat transfer model based on numerical analysis can calculate the heat distribution in the welding process, but its complex modeling process and heavy computational effort are not appropriate for the deployment of the digital twin concept. The approach proposed in this paper is distinctive because it directly converts the collected process sensor data into the bonding blade's temperature field images and the bonding surface's temperature distribution curves for monitoring the welding process. This is achieved by training the generator and discriminator of the conditional depth convolutional generative adversarial network separately. The network is then capable of predicting the bonding blade's heat distribution within a certain range of welding parameters. Compared to methods based on numerical simulation, this approach is far more effective. Training and testing datasets were created to assess the suggested method's efficacy. Multi-scale structural similarity index, structural similarity index, peak signal-to-noise ratio, difference hash and mean absolute error were used to evaluate the simulations' quality. The results show that the simulation's results and the actual data agree quite well. The temperature changes that occur during the welding process may be quickly and accurately simulated using the proposed technology. [ABSTRACT FROM AUTHOR]

• Deep features are extracted from the CT images using deep convolutional neural network model with dense layers. • Hyper edges are constructed for each images by using extracted deep features. A novel hypergraph convolutional neural network is constructed and trained by representational learning. • Evaluated the performance of the trained neural network with test data and attained ideal accuracy, precision, sensitivity, specificity, recall and F1-score. Kidney failure, also called as end-stage renal disease is the final stage of chronic kidney disease(CKD). Kidney failure cannot be revered and is the life-threatening if left untreated. Hence there is a demand for the early and automatic detection of CKD from radiology images based on deep learning techniques. This work focuses on the diagnosis of the predominant CKDs such as kidney stone, cyst and tumor from CT scan images. CT scan abdomen and urogram images are used to construct the digital-twin for the diagnosis of CKD in the robust manner. The proposed work is based on the dataset which contains 12,446 unique CT scan images, within it in which the cyst contains 3,709, normal 5,077, stone 1,377, and tumor 2,283. Deep features are extracted from these images and generated hyper edges from these features. These features are used to construct hypergraph representing the renal images. These hypergraphs are used in hypergraph convolutional neural network for representational learning. Finally, the model is validated by hold-out dataset. Deep learning metrics including precision, recall, accuracy, and FI score are used to validate the proposed model. It outperforms compared to other state of art algorithms with a superior validation accuracy of 99.71. The proposed model is a robust digital-twin for the early diagnosis of kidney diseases. It will assist the nephrologist for the better prognosis of kidney related abnormalities. [ABSTRACT FROM AUTHOR]

Engineering has many necessary fields, and Structural Health Monitoring (SHM) is one of the most important of them. Sometimes in industrial environments, it is difficult and even impossible to collect data containing different real damages. Therefore, the problem of data acquisition represents a primary challenge in designing damage detection systems. The application of digital twin methods based on simulated models and/or Machine Learning (ML) models is a practical way to solve this problem. In this approach, a digital twin is generated for a compromised structure, utilizing a physics-based model to analyze diverse damage scenarios. Subsequently, an ML model is trained using data extracted from the physics-based model, functioning as the digital twin. This research proposes a method based on a digital twin for detecting damages in structures. The data produced from a Floating Wind Turbine (FWT) model was used to evaluate the performance of the proposed digital twin-based method. For this purpose, the FWT structure was simulated using a numerical model to address the data collection problem in the face of various uncertainties, such as changing loading conditions. In line with the concept of digital twin and to reduce the computational time, a Deep Convolution Long Short-Term Memory Neural Network (DCLSTMNN) model was designed and trained only with the frequency data of various scenarios of the simulated FWT model under constant loads (deterministic loads, including constant wind speed and airy wave model) to learn the damage-sensitive features. Then, to demonstrate the robustness of the proposed model under different uncertainties, the DCLSTMNN model was evaluated using the frequency data of the simulated FWT model under variable loading conditions (including Kaimal wind model and JONSWAP wave theory). Some vibration response components unrelated to the nature of the FWT model were removed using the Complete Ensemble Empirical Mode Decomposition (CEEMD) method. Then, the reconstructed vibration responses were used to create the frequency data using the Frequency Domain Decomposition (FDD) technique. The study results show that the proposed digital twin-based method can detect the location and severity of damage more accurately than other comparable methods despite various uncertainties. • A novel digital twin-based method is proposed for learning damage-sensitive features from data of various scenarios of the simulated simple FWT model to detect damage in the complex FWT model. • A Deep Convolution Long Short-Term Memory Neural Network (DCLSTMNN) model is designed to learn features and damage detection of the complex FWT model. • The proposed DCLSTMNN is trained with extracted data from the simulated simple FWT model and then tested with extracted data from the complex FWT model. [ABSTRACT FROM AUTHOR]

The digital twin concept represents an appealing opportunity to advance condition-based and predictive maintenance paradigms for civil engineering systems, thus allowing reduced lifecycle costs, increased system safety, and increased system availability. This work proposes a predictive digital twin approach to the health monitoring, maintenance, and management planning of civil engineering structures. The asset-twin coupled dynamical system is encoded employing a probabilistic graphical model, which allows all relevant sources of uncertainty to be taken into account. In particular, the time-repeating observations-to-decisions flow is modeled using a dynamic Bayesian network. Real-time structural health diagnostics are provided by assimilating sensed data with deep learning models. The digital twin state is continually updated in a sequential Bayesian inference fashion. This is then exploited to inform the optimal planning of maintenance and management actions within a dynamic decision-making framework. A preliminary offline phase involves the population of training datasets through a reduced-order numerical model and the computation of a health-dependent control policy. The strategy is assessed on two synthetic case studies, involving a cantilever beam and a railway bridge, demonstrating the dynamic decision-making capabilities of health-aware digital twins. • We present a digital twin framework for the SHM of civil structures. • The asset-twin dynamical system is encoded through a probabilistic graphical model. • Sensor recordings are assimilated with artificial neural networks. • A reduced-order model is employed offline to generate training vibration responses. • Our strategy allows predictive maintenance and management planning of structures. [ABSTRACT FROM AUTHOR]

Decoupling the physical world and providing standardized service interfaces is still challenging when developing Location Based Services (LBS). This lack also hinders the possibility of developing Intelligent services on top of LBS architectures. In this paper, we propose a multi-layer Digital Twin-based architecture that aims to enable the development of machine learning-based Intelligent LBS (I-LBS) that are able to adapt, evolve, and perform Continual Learning (CL). The platform uses Digital Twins to ensure physical abstraction and provide cyber–physical knowledge to the I-LBSs, which is defined as an execution graph of operation modules. Finally, we simulated a use-case for this platform in the complex scenario of Healthcare organization and management where the I-LBS classifies allowed/not allowed trajectories of users inside a real-existing hospital scenario depending on their role in the organization. The use case is implemented as a Deep Learning-based reconstruction task of high-resolution trajectories processed by the DT architecture that also deploys the I-LBS. The platform is evaluated in terms of physical complexity and computational time on the DT side and on both a traditional machine learning setting and a replay-based CL one for the intelligence side to demonstrate the flexibility and adaptability features introduced by the components for dynamic or unseen scenarios. [ABSTRACT FROM AUTHOR]

This work proposes a new method for the detection, localization, and classification of grid-connected photovoltaic (PV) array faults. Line-to-line, open-module, shorted-module, open-string, and shorted-string faults as well as partial shading conditions are studied. The proposed method has two stages, which are (1) detection and localization of faults and (2) classification of faults. In the first stage, detection and localization are performed using a digital twin (DT) by analyzing the current ratio of each PV array. The measured DC (direct current) power after the operation of the DC/DC boost converter in the physical object is firstly converted into a 2-dimensional image using a recurrence plot (RP) and is then inputted to the second stage. A deep learning-based shifted windows (swin) transformer optimized by particle swarm optimization (PSO) is used in the classification stage, eliminating the need for model tuning by trial-and-error. A PV system of ten arrays with 49 kW is studied. The coefficients of determination (R2) between the results of the digital object (digital twin) and the physical object for different scenarios demonstrate the accuracy and success of the digital twin. R2 values of 0.99988 for varying irradiation with constant temperature and 0.97923 for constant irradiation with varying temperature indicate strong correlations between the digital and physical objects, further confirming the applicability of the digital twin in PV fault detection. The comparative evaluation of the PSO-optimized swin transformer against classical machine learning algorithms and state-of-the-art convolutional neural networks reveals the superior performance of the proposed method. It achieves an outstanding classification accuracy of 98.55%, demonstrating its ability to effectively classify various types of PV faults. The results of the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, which measures the trade-off between true positive rate (TPR) and false positive rate (FPR), further illustrate the effectiveness of the proposed method for PV fault classification. • A model-based digital twin is used to detect and locate faults in a PV system. • Shifted Window Transformer with few trained variables is used to classify PV faults. • Topology of deep learning model is optimized by particle swarm optimization. [ABSTRACT FROM AUTHOR]