Search

In recent years, the detection performance of SAR-GMTI (synthetic aperture radar-ground moving target indication) algorithm based on deep learning has always been limited by insufficient measured data due to the heavy operation complexity and high cost of real SAR systems. To solve this problem, this paper proposes an overall DT-based implementation framework for SAR ground moving target intelligent detection tasks. In particular, by virtue of a SAR imaging algorithm, a high-fidelity twin replica of SAR moving targets is established in digital space through parameter traversal based on the prior target characteristics of the obtained measured datasets. Then, the constructed SAR twin datasets is fed into the neural network model to train an intelligent detector by fully learning features of the moving targets and preset the SAR scene in the twin space, which can realize the robust detection of ground moving targets in related practical scenarios with no need for multiple and complex field experiments. Moreover, the effectiveness of the proposed framework is verified on the MiniSAR measured system, and a comparison with traditional CFAR detection method is given simultaneously. [ABSTRACT FROM AUTHOR]

This review focuses on the definitions, modalities, applications, and performance of various aspects of digital twins (DTs) in the context of transmission and industrial machinery. In this regard, the context around Industry 4.0 and even aspirations for Industry 5.0 are discussed. The many definitions and interpretations of DTs in this domain are first summarized. Subsequently, their adoption and performance levels for rotating and industrial machineries for manufacturing and lifetime performance are observed, along with the type of validations that are available. A significant focus on integrating fundamental operations of the system and scenarios over the lifetime, with sensors and advanced machine or deep learning, along with other statistical or data-driven methods are highlighted. This review summarizes how individual aspects around DTs are extremely helpful for lifetime design, manufacturing, or decision making even when a DT can remain incomplete or limited. [ABSTRACT FROM AUTHOR]

This article examines the incorporation of the Shopping Assistance Automatic Suggestion (SAAS) model into Virtual Reality (VR) environments in order to improve the online shopping experience. The SAAS model employs sophisticated deep learning methods to offer customized product recommendations, which are conveyed by non-player characters (NPCs) via voice-based interactions. Our goal is to develop an interactive shopping experience that replicates real-life interactions by integrating AI-powered recommendations with immersive VR technology. We gather and standardize data from several open commerce databases, such as Amazon Product and Customer Reviews. The SAAS model, in conjunction with GPT-3, BERT, and T5, undergoes training and testing to evaluate its effectiveness across multiple criteria. The results demonstrate that the SAAS model surpasses other models in delivering contextually aware and pertinent recommendations. The integration process outlines the specific steps involved in capturing, processing, and transforming user interactions in virtual reality (VR) into vocal suggestions provided by non-player characters (NPCs). This strategy improves customization and utilizes the immersive features of virtual reality to effectively engage people. The results of our research establish a higher standard for e-commerce, with the goal of enhancing the user experience of online purchasing by making it more instinctive, engaging, and pleasurable. [ABSTRACT FROM AUTHOR]

Digital transformation is omnipresent in our daily lives and its impact is noticeable through new technologies, like smart devices, AI-Chatbots or the changing work environment. This digitalization also takes place in product development, with the integration of many technologies, such as Industry 4.0, digital twins or data-driven methods, to improve the quality of new products and to save time and costs during the development process. Therefore, the use of data-driven methods reusing existing data has great potential. However, data from product design are very diverse and strongly depend on the respective development phase. One of the first few product representations are sketches and drawings, which represent the product in a simplified and condensed way. But, to reuse the data, the existing sketches must be found with an automated approach, allowing the contained information to be utilized. One approach to solve this problem is presented in this paper, with the detection of principle sketches in the early phase of the development process. The aim is to recognize the symbols in these sketches automatically with object detection models. Therefore, existing approaches were analyzed and a new procedure developed, which uses synthetic training data generation. In the next step, a total of six different data generation types were analyzed and tested using six different one- and two-stage detection models. The entire procedure was then evaluated on two unknown test datasets, one focusing on different gearbox variants and a second dataset derived from CAD assemblies. In the last sections the findings are discussed and a procedure with high detection accuracy is determined. [ABSTRACT FROM AUTHOR]

Gear fault detection and remaining useful life estimation are important tasks for monitoring the health of rotating machinery. In this study, a new benchmark for endurance gear vibration signals is presented and made publicly available. The new dataset was used in the HUMS 2023 conference data challenge to test anomaly detection algorithms. A survey of the suggested techniques is provided, demonstrating that traditional signal processing techniques interestingly outperform deep learning algorithms in this case. Of the 11 participating groups, only those that used traditional approaches achieved good results on most of the channels. Additionally, we introduce a signal processing anomaly detection algorithm and meticulously compare it to a standard deep learning anomaly detection algorithm using data from the HUMS 2023 challenge and simulated signals. The signal processing algorithm surpasses the deep learning algorithm on all tested channels and also on simulated data where there is an abundance of training data. Finally, we present a new digital twin that enables the estimation of the remaining useful life of the tested gear from the HUMS 2023 challenge. [ABSTRACT FROM AUTHOR]

The safe and steady operation of hydropower generation systems is crucial for electricity output in the grid. However, hydropower stations have complicated interior structures, making defect detection difficult without disassembly inspections. The application of digital modeling to hydropower stations will effectively promote the intelligent transformation of hydropower stations as well as reduce the maintenance costs of the system. This study provides a model of the power generating and transmission system for hydropower plants, with an emphasis on primary equipment and measured data. The model utilizes PSCAD to digitalize state response in hydropower plants with various short-circuit faults. The fault information is identified and learned using the Adaptive Time–Frequency Memory (AD-TFM) deep learning model. It is demonstrated that our proposed method can effectively obtain the fault information through radio frequency identification (RFID). The accuracy of the traditional method is 0.90, while the results for AD-TFM show a fault classification accuracy of 0.92, which is more than enough to identify multiple fault types compared to the existing methods. [ABSTRACT FROM AUTHOR]

Urban water management faces new challenges due to the rise of digital solutions and abundant data, leading to the development of data-centric tools for decision-making in global water utilities, with AI technologies poised to become a key trend in the sector. This paper proposes a novel methodology for optimal sensor placement aimed at supporting the creation of a digital twin for water infrastructure. A significant innovation in this study is the creation of a metamodel to estimate pressure at consumption nodes in a water supply system. This metamodel guides the optimal sensor configuration by minimizing the difference between estimated and observed pressures. Our methodology was tested on a synthetic case study, showing accurate results. The estimated pressures at each network node exhibited low error and high accuracy across all sensor configurations tested, highlighting the potential for future development of a digital twin for water distribution systems. [ABSTRACT FROM AUTHOR]

Effective monitoring of road fixtures is essential for urban safety and functionality. However, traditional inspections are time-consuming, costly, and error prone, while current automated solutions struggle with high initial setup costs, limited flexibility preventing wide adaptation, and reliance on centralized processing that can delay response times. This study introduces an edge AI-based remote road fixture monitoring system which automatically and continuously updates the information of the road digital twin (DT). The main component is a small-sized edge device consisting of a camera, GPS, and IMU sensors designed to be installed in typical cars. The device captures images, detects the fixture, and estimates their location by employing deep learning and feature matching. This information is transmitted to a dedicated cloud server and represented on a user-friendly user interface. Experiments were conducted to test the system's performance. The results showed that the device could successfully detect the fixture and estimate their global coordinates. Outputs were marked and shown on the road DT, proving the integrated and smooth operation of the whole system. The proposed Edge AI device demonstrated that it could significantly reduce the data size by 80–84% compared to traditional methods. With a satisfactory object detection accuracy of 65%, the system effectively identifies traffic poles, stop signs, and streetlights, integrating these findings into a digital twin for real-time monitoring. The proposed system improves road monitoring by cutting down on maintenance and emergency response times, increasing the ease of data use, and offering a foundation for an overview of urban road fixtures' current state. However, the system's reliance on the quality of data collected under varying environmental conditions suggests potential improvements for consistent performance across diverse scenarios. [ABSTRACT FROM AUTHOR]

To facilitate the integration of autonomous unmanned air vehicles (UAVs) in day-to-day life, it is imperative that safe navigation can be demonstrated in all relevant scenarios. For UAVs using a navigational protocol driven by artificial neural networks, training and testing data from multiple environmental contexts are needed to ensure that bias is minimised. The reduction in predictive capacity when faced with unfamiliar data is a common weak point in trained networks, which worsens the further the input data deviates from the training data. However, training for multiple environmental variables dramatically increases the man-hours required for data collection and validation. In this work, a potential solution to this data availability issue is presented through the generation and evaluation of photo-realistic image datasets from a simulation of 3D-scanned physical spaces which are theoretically linked in a digital twin (DT) configuration. This simulation is then used to generate environmentally varied iterations of the target object in that physical space by two contextual variables (weather and daylight). This results in an expanded dataset of bicycles that contains weather and time-varied components of the same images which are then evaluated using a generic build of the YoloV3 object detection network; the response is then compared to two real image (night and day) datasets as a baseline. The results reveal that the network response remained consistent across the temporal axis, maintaining a measured domain shift of approximately 23% between the two baselines. [ABSTRACT FROM AUTHOR]

Due to various factors such as aging, natural environment erosion, and man-made destruction, architectural heritage has formed various diseases and cracks, especially in pathology cracks, which are the most typical masonry–timber architectural heritages, directly affecting the structural stability of masonry–timber buildings. This paper uses artificial intelligence and architecture and other multi-disciplinary research methods, taking James Jackson Gymnasium, a famous masonry–timber architectural heritage in Wuhan, as an example, using 3D laser scanning technology to obtain disease details and crack data of architectural heritage, using a Mask R-CNN model to detect crack area, using an FCN model to identify and calculate single cracks, and finally summarizing the type, location, and characteristics of cracks, analyzing the causes of cracks, and then putting forward corresponding hierarchical restoration strategies. The research results build a set of detection and repair systems of masonry–timber architectural heritage pathology cracks, which provide a set of accurate and objective pathology cracks data for architectural heritage protection and repair, and provide a reference for architectural heritage repair. [ABSTRACT FROM AUTHOR]

Mechanistic learning refers to the synergistic combination of mechanistic mathematical modeling and data-driven machine or deep learning. This emerging field finds increasing applications in (mathematical) oncology. This review aims to capture the current state of the field and provides a perspective on how mechanistic learning may progress in the oncology domain. We highlight the synergistic potential of mechanistic learning and point out similarities and differences between purely data-driven and mechanistic approaches concerning model complexity, data requirements, outputs generated, and interpretability of the algorithms and their results. Four categories of mechanistic learning (sequential, parallel, extrinsic, intrinsic) of mechanistic learning are presented with specific examples. We discuss a range of techniques including physics-informed neural networks, surrogate model learning, and digital twins. Example applications address complex problems predominantly from the domain of oncology research such as longitudinal tumor response predictions or time-to-event modeling. As the field of mechanistic learning advances, we aim for this review and proposed categorization framework to foster additional collaboration between the data- and knowledge-driven modeling fields. Further collaboration will help address difficult issues in oncology such as limited data availability, requirements of model transparency, and complex input data which are embraced in a mechanistic learning framework. [ABSTRACT FROM AUTHOR]

3D point clouds are a relevant source of information for multiple applications, including digital twins, building modeling, disaster and risk management, forestry, autonomous driving, and many others. Assigning points to the semantic classes is one of the essential data interpretation steps to effectively use them for further analysis. Deep learning models for semantic segmentation, such as RandLA-Net, are state-of-the-art methods for this task. Although the overall accuracy of classification is usually satisfactory,there are still several shortcomings not allowing assigning correct labels across all the classes. For instance, the receptive field of these networks is often too small to correctly classify point clouds in all cases. These networks suffer also from class imbalance, typical in real-world data sets, and tend to oversmooth small classes. Post-processing approaches help to overcome these problems and achieve better classification accuracy. In this work, we investigate the feasibility of improving the deep-learning outputs by introducing prior knowledge. To do this, the output probabilities of point classes obtained using RandLA-Net are post-processed with a workflow based on Markov Random Fields, in which the unary potentials are adjusted to preserve smaller classes while the pairwise potentials take into account. a hand-tailored inter-class reliability matrix. To validate our method, we apply it to the Hessigheim benchmark. Our MRF-based approach further optimizes these prediction results, effectively and efficiently improving the overall accuracy by approximately 1 to 2 percentage points. [ABSTRACT FROM AUTHOR]

The development of novel and robust digital methods to support the maintenance of existing road infrastructure requires a large amount of harmonized data. Especially in the context of automated modelling having a large amount of matching data from different perspectives enables disruptive, new use cases that might largely impact the efficiency in maintenance of the built environment. Unfortunately, such data compositions are tedious to collect in real world applications, due to many influential factors, leading to deviations between multiple data sources and the sheer complexity in the process of creating a digital model. However, for deep learning applications, a large amount of carefully annotated data is necessary for robust estimations. In this contribution, we tackle this problem by presenting a novel procedural modelling and model configuration approach for generating homogeneous data combinations to step towards direct parameter estimation for machine learning approaches utilizing point clouds of roads and end-to-end model generation of digital road models. [ABSTRACT FROM AUTHOR]

Combining remote-sensing data with in-situ observations to achieve a comprehensive 3D reconstruction of the ocean state presents significant challenges for traditional interpolation techniques. To address this, we developed the CLuster Optimal Interpolation Neural Network (CLOINet), which combines the robust mathematical framework of the Optimal Interpolation (OI) scheme with a self-supervised clustering approach. CLOINet efficiently segments remote sensing images into clusters to reveal non-local correlations, thereby enhancing fine-scale oceanic reconstructions. We trained our network using outputs from an Ocean General Circulation Model (OGCM), which also facilitated various testing scenarios. Our Observing System Simulation Experiments aimed to reconstruct deep salinity fields using Sea Surface Temperature (SST) or Sea Surface Height (SSH), alongside sparse in-situ salinity observations. The results showcased a significant reduction in reconstruction error up to 40% and the ability to resolve scales 50% smaller compared to baseline OI techniques. Remarkably, even though CLOINet was trained exclusively on simulated data, it accurately reconstructed an unseen SST field using only glider temperature observations and satellite chlorophyll concentration data. This demonstrates how deep learning networks like CLOINet can potentially lead the integration of modeling and observational efforts in developing an ocean digital twin. [ABSTRACT FROM AUTHOR]

Urban Air Mobility (UAM) emerges as a transformative approach to address urban congestion and pollution, offering efficient and sustainable transportation for people and goods. Central to UAM is the Operational Digital Twin (ODT), which plays a crucial role in real-time management of air traffic, enhancing safety and efficiency. This study introduces a YOLOTransfer-DT framework specifically designed for Artificial Intelligence (AI) training in simulated environments, focusing on its utility for experiential learning in realistic scenarios. The framework's objective is to augment AI training, particularly in developing an object detection system that employs visual tasks for proactive conflict identification and mission support, leveraging deep and transfer learning techniques. The proposed methodology combines real-time detection, transfer learning, and a novel mix-up process for environmental data extraction, tested rigorously in realistic simulations. Findings validate the use of existing deep learning models for real-time object recognition in similar conditions. This research underscores the value of the ODT framework in bridging the gap between virtual and actual environments, highlighting the safety and cost-effectiveness of virtual testing. This adaptable framework facilitates extensive experimentation and training, demonstrating its potential as a foundation for advanced detection techniques in UAM. [ABSTRACT FROM AUTHOR]

Although brain tumours are few, they have one of the highest mortality rates among all types of cancer due to their abnormal growth and proliferation. Brain tumours develop due to the accumulation of abnormal tissues in the brain. Various forms of abnormal tissue exist, however, in the majority of cases, they develop in a regular manner and perish without creating any detrimental effects. Digital twins are occasionally known as digital mirrors, digital mapping, and digital replicas. All of these are synonymous terms for the identical entity. It is a technique for transferring digital or physical information from one realm to another. Image processing involves enhancing or eliminating data from a photograph to achieve a certain objective. Convolutional neural networks are a specific type of neural network that take signals from images as input and produce the image itself or a subset of its elements as output. This research presents a technique for identifying brain cancers using digital replicas and advanced machine learning algorithms by analysing MRI images. Images obtained from MRI machines are stored in a centralised cloud using Internet of Things (IoT) digital devices. The input pictures and other health-related data are then retrieved from cloud storage. The Particle Swarm Optimization approach chooses features. Brain tumor images are classified using machine learning techniques such as convolutional neural networks, support vector machines, and extreme learning machines. The CNN algorithm demonstrates greater accuracy when assessing MRI images for the purpose of identifying brain tumours.

In this work, the authors present a digital twin (DT) to analyse the manufacturing process of structural composites by using resin transfer moulding (RTM). During RTM, a dry textile preform is impregnated with a polymer resin injected in a closed mould. RTM is one of the most used production methods for high-performance structural composites. The DT is focused on detecting in-homogeneous resin flow produced by race-tracking channels that divert resin flow to the outlet gates of the mould, producing dry spots and lack of impregnation. The DT core contains two surrogate models based on encoder/decoder deep learning architectures, providing the fast/accurate response necessary for interrogation during manufacturing. The first surrogate acts as the disturbance detector, providing the on-the-fly representation of the fabric permeability with the only information gathered by a set of five pressure sensors distributed over the mould surface. The second offers real-time representation of a set of quantities of interests (QoI): namely, the flow progress and the pressure field inside the mould. Training of both surrogates was performed with synthetic data generated by high-fidelity multi-physics simulations of the flow progress in a porous preform by following Darcy's law. Errors in the pressure field predictions of the surrogates are lower than 1 % with consultation time <50 ms, enabling encapsulation in the digital twin. The DT performance was evaluated by comparing the response against a set of RTM experiments for different race-tracking scenarios. The two most relevant novelties of the work are the use of the concept of the instantiated DT, which provides information on the current state of the process from data provided by a network of distributed sensors, and that this DT has been trained exclusively with synthetic data from multiphysics simulations while evaluated against experimental data from injection tests. [ABSTRACT FROM AUTHOR]

This survey provides a comprehensive overview of traditional techniques and deep learning-based methodologies for monocular visual odometry (VO), with a focus on displacement measurement applications. This paper outlines the fundamental concepts and general procedures for VO implementation, including feature detection, tracking, motion estimation, triangulation, and trajectory estimation. This paper also explores the research challenges inherent in VO implementation, including scale estimation and ground plane considerations. The scientific literature is rife with diverse methodologies aiming to overcome these challenges, particularly focusing on the problem of accurate scale estimation. This issue has been typically addressed through the reliance on knowledge regarding the height of the camera from the ground plane and the evaluation of feature movements on that plane. Alternatively, some approaches have utilized additional tools, such as LiDAR or depth sensors. This survey of approaches concludes with a discussion of future research challenges and opportunities in the field of monocular visual odometry. [ABSTRACT FROM AUTHOR]

The integration of artificial intelligence (AI) with Digital Twins (DTs) has emerged as a promising approach to revolutionize healthcare, particularly in terms of diagnosis and management of thoracic disorders. This study proposes a comprehensive framework, named Lung-DT, which leverages IoT sensors and AI algorithms to establish the digital representation of a patient's respiratory health. Using the YOLOv8 neural network, the Lung-DT system accurately classifies chest X-rays into five distinct categories of lung diseases, including "normal", "covid", "lung_opacity", "pneumonia", and "tuberculosis". The performance of the system was evaluated employing a chest X-ray dataset available in the literature, demonstrating average accuracy of 96.8%, precision of 92%, recall of 97%, and F1-score of 94%. The proposed Lung-DT framework offers several advantages over conventional diagnostic methods. Firstly, it enables real-time monitoring of lung health through continuous data acquisition from IoT sensors, facilitating early diagnosis and intervention. Secondly, the AI-powered classification module provides automated and objective assessments of chest X-rays, reducing dependence on subjective human interpretation. Thirdly, the twin digital representation of the patient's respiratory health allows for comprehensive analysis and correlation of multiple data streams, providing valuable insights as to personalized treatment plans. The integration of IoT sensors, AI algorithms, and DT technology within the Lung-DT system demonstrates a significant step towards improving thoracic healthcare. By enabling continuous monitoring, automated diagnosis, and comprehensive data analysis, the Lung-DT framework has enormous potential to enhance patient outcomes, reduce healthcare costs, and optimize resource allocation. [ABSTRACT FROM AUTHOR]

Due to the harsh environment of highway tunnels and frequent breakdowns of various detection sensors and surveillance devices, the operational management of highway tunnels lacks effective data support. This paper analyzes the characteristics of operational surveillance data in highway tunnels. It proposes a multimodal information fusion method based on CNN-LSTM-attention and designs and develops a digital twin for highway tunnel operations. The system addresses issues such as insufficient development and coordination of the technical architecture of operation control systems, weak information service capabilities, and insufficient data application capabilities. The system also lacks intelligent decision-making and control capabilities. The developed system achieves closed-loop management of "accurate perception-risk assessment-decision warning-emergency management" for highway tunnel operations based on data-driven approaches. The engineering demonstration application underscores the system's capacity to enhance tunnel traffic safety, diminish tunnel management costs, and elevate tunnel driving comfort. [ABSTRACT FROM AUTHOR]

For a lithium-ion (Li-ion) battery to operate safely and reliably, an accurate state of health (SOH) estimation is crucial. Data-driven models with manual feature extraction are commonly used for battery SOH estimation, requiring extensive expert knowledge to extract features. In this regard, a novel data pre-processing model is proposed in this paper to extract health-related features automatically from battery-discharging data for SOH estimation. In the proposed method, one-dimensional (1D) voltage data are converted to two-dimensional (2D) data, and a new data set is created using a 2D sliding window. Then, features are automatically extracted in the machine learning (ML) training process. Finally, the estimation of the SOH is achieved by forecasting the battery voltage in the subsequent cycle. The performance of the proposed technique is evaluated on the NASA public data set for a Li-ion battery degradation analysis in four different scenarios. The simulation results show a considerable reduction in the RMSE of battery SOH estimation. The proposed method eliminates the need for the manual extraction and evaluation of features, which is an important step toward automating the SOH estimation process and developing battery digital twins. [ABSTRACT FROM AUTHOR]

Crack detection at the pixel level across complex scenarios (structural interference and adverse working conditions) is a critical consideration in the maintenance of ballastless track slab (BTS). Although existing deep learning models achieve acceptable accuracy on cracks with a monotonous background, the ground truth with high labor cost is inevitable and their performance in complex scenarios may fall far below their theoretical bounds or even cause "all-black images." A hybrid algorithm based on synthetic data from digital twin model and weakly supervised style transfer is proposed in this paper for addressing the above challenges. The algorithm uses a region-attention strategy to enable the uncontrolled generative adversarial network (GAN) focusing its attention on weak labels containing crack regions, directly obtaining segmentation results with the same style as the ground truth of the crack forest dataset. In addition, a digital twin model that can simulate the real inspection working conditions is established to generate a synthetic crack dataset, enabling the hybrid algorithm to extract the most discriminative features. The results show that the performance of the hybrid algorithm on inspection images across complex scenarios is nearly 25% higher than that of the DeepLabv3+ network, while the time cost consumed is only 0.5% of the latter. The deployment of the region attention strategy also enables the hybrid algorithm to achieve a mean intersection of union (MIoU) of 79.38%, which is nearly twice as much as that of GAN. It not only eliminates the oversegmentation caused by structures such as rails and fastener systems but also overcomes "all-black images." In addition, synthetic data can greatly enlarge the range, type, and number of discriminative crack features compared with data augmentation based on limited real data, thus enhancing the performance of the hybrid algorithm for uncertain inspection data. Particularly, the fully trained hybrid algorithm based on the synthetic dataset shows good adaptability and generalization to adverse working conditions such as uneven lighting, noise, and blur. [ABSTRACT FROM AUTHOR]

Three-dimensional semantic segmentation is the foundation for automatically creating enriched Digital Twin Cities (DTCs) and their updates. For this task, prior-level fusion approaches show more promising results than other fusion levels. This article proposes a new approach by developing and benchmarking three prior-level fusion scenarios to enhance the outcomes of point cloud-enriched semantic segmentation. The latter were compared with a baseline approach that used the point cloud only. In each scenario, specific prior knowledge (geometric features, classified images, or classified geometric information) and aerial images were fused into the neural network's learning pipeline with the point cloud data. The goal was to identify the one that most profoundly enhanced the neural network's knowledge. Two deep learning techniques, "RandLaNet" and "KPConv", were adopted, and their parameters were modified for different scenarios. Efficient feature engineering and selection for the fusion step facilitated the learning process and improved the semantic segmentation results. Our contribution provides a good solution for addressing some challenges, particularly for more accurate extraction of semantically rich objects from the urban environment. The experimental results have demonstrated that Scenario 1 has higher precision (88%) on the SensatUrban dataset compared to the baseline approach (71%), the Scenario 2 approach (85%), and the Scenario 3 approach (84%). Furthermore, the qualitative results obtained by the first scenario are close to the ground truth. Therefore, it was identified as the efficient fusion approach for point cloud-enriched semantic segmentation, which we have named the efficient prior-level fusion (Efficient-PLF) approach. [ABSTRACT FROM AUTHOR]

Industrial collaborative robots play an essential role in smart manufacturing because they improve productivity while also ensuring workplace safety. However, the development of prognostic and health management systems to ensure the reliability of these robots has been a major challenge due to the lack of fault data. This paper proposed a digital twin scheme based on the fusion of the robot kinematic and dynamic models' information down to the powertrains (i.e., the joints motor, and gear) along with the control algorithms and uncertainty accommodation based upon deep learning. The presented digital twin concept has the potential to propel simulation-based fault prediction. We also highlight and discuss challenges and opportunities around the development of the hybrid digital twin for condition monitoring of industrial collaborative robots. [ABSTRACT FROM AUTHOR]

The rapid advancement of deep learning and computer vision has driven the intelligent evolution of fire detection, quantification and fighting, although most AI models remain opaque black boxes. This work applies explainable deep learning methods to quantify fire power by flame images and aims to elucidate the underlying mechanism of computer-vision fire calorimetry. The process begins with the utilization of a pre-trained fire segmentation model to create a flame image database in various formats: (1) original RGB, (2) background-free RGB, (3) background-free grey, and (4) background-free binary. This diverse database accounts for factors such as background, color, and brightness. Then, the synthetic database is employed to train and test the fire-calorimetry AI model. Results highlight the dominant role of determining flame area in fire calorimetry, while other factors displaying minimal influence. Enhancing the accuracy of flame segmentation significantly reduces error in computer-vision fire calorimetry to less than 20%. Finally, the study incorporates the Gradient-weighted Class Activation Mapping (Grad-CAM) method to visualize the pixel-level contribution to fire image identification. This research deepens the understanding of vision-based fire calorimetry and providing scientific support for future AI applications in fire monitoring, digital twin, and smart firefighting. [ABSTRACT FROM AUTHOR]

This article examines the incorporation of the Shopping Assistance Automatic Suggestion (SAAS) model into Virtual Reality (VR) environments in order to improve the online shopping experience. The SAAS model employs sophisticated deep learning methods to offer customized product recommendations, which are conveyed by non-player characters (NPCs) via voice-based interactions. Our goal is to develop an interactive shopping experience that replicates real-life interactions by integrating AI-powered recommendations with immersive VR technology. We gather and standardize data from several open commerce databases, such as Amazon Product and Customer Reviews. The SAAS model, in conjunction with GPT-3, BERT, and T5, undergoes training and testing to evaluate its effectiveness across multiple criteria. The results demonstrate that the SAAS model surpasses other models in delivering contextually aware and pertinent recommendations. The integration process outlines the specific steps involved in capturing, processing, and transforming user interactions in virtual reality (VR) into vocal suggestions provided by non-player characters (NPCs). This strategy improves customization and utilizes the immersive features of virtual reality to effectively engage people. The results of our research establish a higher standard for e-commerce, with the goal of enhancing the user experience of online purchasing by making it more instinctive, engaging, and pleasurable.

This document discusses the emergence of deep learning as a transformative tool in the field of operations research (OR). Deep learning, a specialized category of machine learning algorithms, uses multiple layers to uncover hidden patterns and insights from large datasets. It has gained popularity due to its superior predictive performance compared to traditional machine learning models. The integration of deep learning applications into OR has the potential to enhance decision-making, productivity, sustainability, and optimization processes. However, there are challenges to address, such as data availability, interpretability, and computational cost. The document highlights the need for practical research and provides future research directions in areas such as supply chain management, demand forecasting, inventory optimization, and decision-making processes. Overall, deep learning has the potential to transform the field of OR, and this document aims to provide new perspectives for further research and practical applications. [Extracted from the article]

The meticulous reconstruction of three-dimensional (3D) braided composite materials serves as a crucial foundation for achieving high-fidelity simulations. Nonetheless, the transition from tomographic images to a 3D mesh entails a laborious and time-intensive process. To address this, an integrated procedure based on artificial intelligence is proposed for reconstructing meshes from tomograms. The initial stage of the process involves employing artificial intelligence techniques to segment complex contours and optimize high-dimensional contours. This facilitates the input of high-quality images needed to reconstruct accurate digital twins with strong convergence. The subsequent reconstruction phase integrates various calculations, including shape interpolation, contour extraction, 3D surface reconstruction, 3D mesh reconstruction, and element data interpolation. During this process, optimization objectives are set to minimize the deviation between the digital twin's surface and the actual surface, as well as to optimize the aspect ratio of the element mesh. Upon completion of the aforementioned steps, high-quality input files suitable for finite element calculations are directly generated. Ultimately, the proposed method utilizes the reconstructed finite element model for mechanical analysis, and the results are found to be in good agreement with experimental tests. This method offers an efficient and rapid way to achieve high-quality reconstruction of complex digital twins. [Display omitted] • The proposed procedure to reconstruct meshes from tomograms is integrated, enabling batch processing. • The deep learning models are employed to segment the intricate yarns contours of 3D5D braided composites. • The Ramer–Douglas–Peucker algorithm is used to capture high-dimensional contour features and improve convergence. • A decimating filter employing the quadric-based edge-collapsed strategy improves volumetric meshing. [ABSTRACT FROM AUTHOR]

Reliable traffic infrastructure is a key factor for any country's economy. However, aging bridges often require renovation or reconstruction. Promising approaches for enhancing asset management and cost reduction are digital twins and predictive maintenance strategies. However, the creation of geometric-semantic as-is models as a basis for digital twins currently involves labor-intensive manual data capture and modeling. Modern deep learning models, such as Kernel Point Convolution (KPConv) show promising results in reducing the time needed to create digital twins by semantically segmenting point cloud data but have so far been hindered by the lack of a reliable quality measure, which can predict when the model's prediction can be trusted. This is especially viable in the construction industry, where objects and sensors may vary widely between projects and contractors. In this work, we present Bayesian neural networks, implemented through Variational Inference and Monte-Carlo dropout as approaches to conducting inference with KPConv, which show improvements in the confidence estimation and out-of-domain (OOD) detection in the scans of typical infrastructure point clouds. When confronted with different domain shifts to the test data, such as a change of scanning device and introduction of unseen classes the proposed model showed a 25.3% decrease in expected calibration error (ECE) and a 4.82 increase in OOD detection in terms of outlier-intersection-over-union (O-IoU) on average with respect to a deterministic baseline. • Bayesian neural networks applied in point cloud semantic segmentation. • Training stochastic models with Variational Inference improve calibration. • Out of domain detection was improved with a single stochastic layer. • Model robustness to domain shift from mobile to terrestrial laser scanner. [ABSTRACT FROM AUTHOR]

The thermal errors in ball screws present complex spatial-temporal characteristics and exhibit persistent long-term memory, significantly affecting the machining accuracy of the whole machine tool in applications. Deep learning has emerged as a promising approach to predict these errors using thermal data. However, the scarcity of extensive thermal datasets, typically due to the prohibitive costs and time required for experimental acquisition, hampers the comprehensive exploitation of spatial-temporal and long-term memory attributes. Moreover, the current thermal error compensation system does not support the deployment of deep learning-based thermal error models due to its weak real-time performance. These limitations lead to suboptimal prediction accuracy and inadequate error compensation. Addressing this challenge, a novel spatial-temporal interactive integration network is proposed, and this is a sophisticated model that synergistically blends a time memory gate and a spatial-temporal fusion gate within an advanced attention-based spatial-temporal graph convolutional framework. Our network ingeniously leverages spatial data to anchor long-term memory while employing temporal data for selective memory feature extraction, facilitating a refined integration of spatial-temporal features. Concurrently, the integration of gated recurrent units and a time attention layer meticulously extracts and enhances temporal features, bolstered by our innovative time memory gate, which is adept at handling small-sample scenarios and refining thermal error predictions. Critically, the digital twin system for thermal error compensation is constructed to improve the system's real-time performance. The integration of this network into the digital twin system marks a pivotal advancement in thermal error compensation. Our empirical results underscore the system's remarkable robustness and superior predictive accuracy, demonstrating a significant reduction in positioning and machining errors (over 90% and 80% respectively) even with constrained thermal data inputs. These advancements not only delineate a substantial leap in predictive accuracy but also underscore our contribution to reducing operational costs and enhancing the efficacy of machine tools. [Display omitted] • Spatial-temporal fusion gate is proposed to extract spatiotemporal features. • STIIN is proposed to extract spatiotemporal features of thermal error. • Digital twin system for thermal error compensation is established. • STIIN model is embedded into digital twin system. • Robust compensation is realized with small-volume thermal information data. [ABSTRACT FROM AUTHOR]

In order to achieve the causal diagnosis of life health of high-speed spindle system, a digital twin (DT)–driven prediction method is proposed based on the real-time monitoring of thermal characteristics. The finite element method is used to achieve the DT for thermal characteristics through real time correcting the thermal boundaries using the correction models. The long short-term memory recurrent neural network (LSTM-RNN) is used to predict the heat generations of bearings and motor; the prediction results are used to diagnose the health status according to the domain and threshold models. A heat change rate–based causal diagnosis model is proposed to judge the fault sources. The spindle wear and fault experiments are carried out to verify the effectiveness of the proposed DT system. The experimental results show that the DT accuracy of thermal characteristics exceeds 95%, and the proposed DT system can successfully monitor the health status of the spindle system. [ABSTRACT FROM AUTHOR]

In this article, we present a deep learning-based surrogate model for spatio-temporal prediction of cavitating fluid flow. Specifically, we introduce a finite element-inspired rotation equivariant hypergraph neural network for inferring and predicting dynamical behaviors of cavitating flow. We generate ground-truth spatial–temporal data by simulating a full-order variational system based on homogeneous mixture-based cavitation theory. We consider the flow past a NACA0012 hydrofoil to examine the predictive ability of the proposed graph neural network for cavitation dynamics. Results demonstrate that the network achieves stabilized and accurate temporal predictions of the system states, successfully forecasting the evolution patterns of individual cavitation events. Additionally, comparisons of predicted fluid loading coefficients are in good agreements with the ground-truth values. We also discuss some challenges encountered in the long-term prediction of flow patterns across multiple cavitation events. The proposed framework has implications for design optimization, active control and development of a physics-based digital twin of a cavitating marine propeller. [Display omitted] • Finite element-inspired graph neural network for cavitation dynamics. • Stabilized and accurate spatio-temporal predictions of system states. • Successful forecasting of individual cavitation events. • Good comparison of predicted force coefficients with ground truth values. [ABSTRACT FROM AUTHOR]

Large volumes of 3D parametric datasets, such as building information modeling (BIM), are the foundation for developing and applying smart city and digital twin technologies. Those datasets are also considered valuable tools for efficiently managing rebuilt structures during the operation and maintenance stages. Nevertheless, current approaches developed for the scan-to-BIM process rely on manual or semi-automatic procedures and insufficiently leverage semantic data in point clouds. These methods struggle to accurately represent large-scale indoor complex layouts and extract details from irregular-shaped unstructured elements, causing inefficiencies in BIM model generation. To address these issues, we propose an innovative scan-to-BIM framework based on deep learning algorithms and raw point cloud data, enabling the automatic generation of 3D models for both structured and unstructured indoor elements. Initially, we propose an enhanced deep learning neural network to improve the point clouds' semantic segmentation accuracy. Subsequently, an efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. The proposed workflow can reconstruct large-scale data with multiple room layouts of Manhattan or non-Manhattan structures and reconstruct 3D models automatically by using a BIM parametric algorithm implemented in Revit software. Moreover, we introduce a robust method for unstructured elements to automatically generate corresponding 3D BIM models, even when the incorporating semantic information is incomplete. The proposed approach was evaluated on synthetic and real data for different scales and complexities of indoor scenes. The results of the experiments demonstrate that the improved model significantly enhances the overall semantic segmentation accuracy compared to the baseline models. The proposed scan-to-BIM framework is efficient for indoor element 3D reconstruction, achieving precision, recall, and F-score values ranging from 96% to 99%. The generated BIM models are competitive with traditional methods regarding model completeness and geometric accuracy. • We propose an innovative scan-to-BIM framework based on deep learning algorithms: • We propose an enhanced deep-learning neural network to improve the point clouds' semantic segmentation accuracy. • A novel and efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. It can reconstruct large-scale data with multiple room layouts in 3D model automatically by using a BIM parametric • For unstructured elements, we propose a robust method to automatically generate 3D BIM models, even when the semantic information is incomplete. [ABSTRACT FROM AUTHOR]

• A digital twin-assisted dual transfer method was proposed with information and model adaptation for bearing fault diagnosis. • DTa-DT aims to simultaneously synthesize information transfer and model transfer for domain transfer error minimization. • DTd-IT method was proposed to information transfer with low distribution difference of twin data adapted. • DAd-MT method, composed of DBTN, was proposed to model transfer with feature distribution difference of domain-model adapted. • Experiments and ablation study verified that the proposed DTa-DT method can be well applied to small sample learning. Rolling bearing fault diagnosis is of great importance to the safety management of mechanical equipment. The scarcity of labelled fault data makes it difficult to adequately perform the training process of intelligent diagnosis models, and this will result in these intelligent models not being effectively and widely used in practice. Although some recent studies have verified that the addition of dynamic model response to the training process will greatly improve the ability of the model with low cost and high efficiency, it is still stuck in poor effect caused by large information distribution difference between dynamic model response and real measured data. Focusing on this issue, a digital twin-assisted dual transfer (DTa-DT) method with information and model adaptation was proposed for rolling bearing fault diagnosis. Different from the traditional digital-analogue driven transfer methods, the proposed DTa-DT aims to simultaneously synthesize data information transfer and feature model transfer together with domain transfer error minimization. In particular, it should be noted that the DTa-DT architecture consists of a dual transfer learning process, including digital twin-driven information transfer (DTd-IT) and digital-analogue-driven model transfer (DAd-MT), where the information is collaborated with the model to improve the integrated transfer diagnosis effect under sampling. On one aspect, with the employment of bearing dynamic model responses, DTd-IT is innovatively designed to establish the transfer of dynamic information and measured information. The information distribution difference between these twin data and real measured data is effetely adjusted with the introduced actual inference components, where the twin data with low information distribution difference can be well fusion generated by the information transfer digital twin (ITDT) model. On the other aspect, considering the truth that there are still small sample cases of real measured data and information distribution differences will affect the quality of the twin data, a digital-analogue driven model transfer (DAd-MT) method is further proposed, where the deep branch transfer network (DBTN) model with improved convolutional neural network (CNN) is used to achieve an accurate fault diagnosis effect with the help of digital twin data. Experiments and wear analysis verified that the proposed DTa-DT can significantly reduce the distribution difference between the dynamic model response and the real measured data, thus achieving low-cost and efficient rolling bearing transfer diagnosis compared to other ten state-of-the-art deep learning models. It can be predicted that the proposed dual transfer architecture provides more opportunities for the practical application of intelligent fault diagnosis under small sample sizes. [ABSTRACT FROM AUTHOR]

Digital twin has become a critical technical solution for satellite in-orbit service and maintenance. The majority of existing research in this field has focused on constructing system-level digital twins, with various typical applications that primarily describe the operating condition of satellite subsystems. However, achieving a more meticulous portrayal of in-orbit satellites necessitates the inclusion of physical field information, e.g., the temperature field serves as a fundamental basis for characterizing the thermodynamic state and plays a crucial role in precise thermal management of a satellite. The main challenge for realizing thorough state detection and real-time response lies in the cross-level model fusion of system simulation and physical field calculation. To overcome this problem, this paper proposes a Hybrid Digital Twin (HDT) framework that seamlessly integrates the digital twin at the system-level with its physical field-level counterpart, enabling a comprehensive and detailed representation of modeling objects. Specifically, at the physical field-level, deep learning methods are utilized to construct an accurate surrogate model, offering a lightweight, computationally efficient and accelerated execution alternative to the Finite Element Analysis models. At the system-level, multi-domain unified modeling technology is employed to ensure both efficiency and accuracy of coupling simulation. Moreover, a co-simulation approach is developed on the basis of the Functional Mock-up Interface, overcoming the bottleneck of cross-level information interaction and computing time synchronization. The practical utility of the proposed framework is verified by a real-world engineering problem, i.e., satellite temperature field perception and attitude control in the context of the harsh space environment. Whole process engineering experiments demonstrate the effectiveness of the HDT, with mean absolute errors below 1 K for temperature field reconstruction and within 1° for attitude control. [ABSTRACT FROM AUTHOR]

Given the pervasive use of deep learning models across various domains, ensuring model security has emerged as a critical concern. This paper examines backdoor attacks, a form of security threat that compromises model output by poisoning the training data. Our investigation specifically addresses backdoor attacks on object detection models, vital for security-sensitive applications like autonomous driving and smart city systems. Consequently, such attacks on object detection models could pose significant risks to human life and property. Consequently, backdoor attacks on object detection could pose serious threats to human life and property. To elucidate this security risk, we propose and experimentally evaluate five backdoor attack methods for object detection models. The key findings are: (1) Unnecessary Object Generation : a globally embedded trigger creating false objects in the target class; (2) Partial Misclassification : a trigger causing specific class misclassification; (3) Global Misclassification : a trigger reclassifying all objects into the target class; (4) Specific Object Vanishing : a trigger causing non-detection of certain objects; (5) Object Position Shifting : a trigger causing bounding box shifts for a specific class. To assess attack effectiveness, we introduced the Attack Success Rate (ASR), which can surpass 1 in object detection tasks, thus providing a more accurate reflection of the attack impact. Experimental outcomes indicate that the ASR values of these varied backdoor attacks frequently approach or surpass 1, demonstrating our method's capacity to impact multiple objects simultaneously. Additionally, to augment trigger stealth, we introduce Backdoor Attack with Wavelet Embedding (BAWE), which discreetly embeds triggers as image watermarks in training data. This embedding method yields more natural triggers with enhanced stealth. Highly stealthy triggers are less detectable, significantly increasing the likelihood of attack success and efficacy. We have developed a Transformer-based network architecture, diverging from traditional neural network frameworks. Our experiments across various object detection datasets highlight the susceptibility of these models and the high success rate of our approaches. This vulnerability poses significant risks to digital twin systems utilizing object detection technology. Our methodology not only enhances trigger stealth but also suits dense predictive tasks and circumvents current neural network backdoor attack detection methods. The experimental findings expose key challenges in the security of object detection models, particularly when integrated with digital twins, offering new avenues for backdoor attack research and foundational insights for devising defense strategies against these attacks. • We embed watermarks as invisible triggers in training data using wavelet transform. • Our method is suited for dense prediction tasks like object detection. • We use watermarks as triggers and are not limited to standard neural network. [ABSTRACT FROM AUTHOR]

Accurate and efficient inspection of rebar dimensions has proven to be a persistent challenge for researchers and practitioners. This paper introduces a semantically enriched 3D model-based system that employs computer vision and deep learning for location-aware identification and tracking of rebar issues. The system comprises four modules: (A) digital twin generation, (B) segmentation, (C) inspection, and (D) issue identification and tracking. The generation module constructs 3D models from rebar structures. The segmentation and inspection modules analyze the 3D models, enriching them with semantic information. The issue identification and tracking module exchanges information between the semantically enriched 3D models and the building information models across time. An experiment on a column rebar cage is conducted. A precision of over 90% and a recall of over 97% are reported in 3D instance segmentation. Diameter inspection achieves an accuracy of 95.5% for large-size rebars. Spacing inspection achieves a mean relative error of 0.98%. The defective spacing is identified and tracked. • Fuses deep learning and computer vision to construct 3D semantic rebar model. • BIM- and digital-twin-based information system fulfills rebar quality control. • Autonomous rebar inspection pipeline directly generates issue reports from images. [ABSTRACT FROM AUTHOR]

This paper establishes a digital twin (DT) enabled model that aims to achieve real-time monitoring and control to enhance the overall performance of tunnel boring machines (TBM) during excavation. Through the Internet of Things (IoT) sensors that are installed on TBM, the actual operating status is transferred from the physical object to the digital model. Deep learning models that integrated graph convolutional network (GCN) and long short-term memory (LSTM) networks are employed for reliable TBM performance estimation. An online multi-objective optimization (MOO) approach is then proposed to determine the optimal operating parameters for advanced control. Real data from Singapore's Circle Line 6 C885 project is utilized as a case study to examine the applicability of the developed digital twin model. TBM's penetration rate, over-excavation ratio, energy consumption, and tool wear are selected as the objectives (O 1 to O 4) for optimization. The results indicate that (1) The digital twin model can provide a reliable estimation of TBM's performance indicators with a high coefficient of determination (R 2) values. (2) TBM's overall performance can be effectively improved through the proposed online optimization method, with an overall improvement of 21.12%. (3) The proposed online optimization method is more practical and creates better improvement than the conventional MOO approach. This study addresses the existing research gaps of lacking a reliable method to guide TBM operation with enhanced performances for the major objectives. It also contributes to proposing a novel intelligent approach that integrates DT, deep learning, and an online-NSGA-II method for TBM operation optimization during tunnel excavation. • A digital twin enabled approach is proposed for real-time intelligent control for TBM operation. • An MOO approach that integrates deep-learning and online optimization is established. • Data from a realistic tunnel project in Singapore is used to demonstrate as a case study. • TBM's efficiency, safety, energy consumption and tool wear are considered as objectives. • An overall improvement of 21.12% is achieved by implementing the proposed method. [ABSTRACT FROM AUTHOR]

• Numerical simulation modeling is complex and computationally intensive. • Digital twin deployment requires lightweight and responsive systems. • Process sensor data drives deep learning algorithms to implement digital twins. • Generative adversarial networks enable monitoring of heat distribution changes. • Results contribute to weld quality assurance and blade structure optimization. In this paper, a novel method of using deep learning to construct a digital twin of the thermos-compression bonding blade's temperature field is presented. Usually, a heat transfer model based on numerical analysis can calculate the heat distribution in the welding process, but its complex modeling process and heavy computational effort are not appropriate for the deployment of the digital twin concept. The approach proposed in this paper is distinctive because it directly converts the collected process sensor data into the bonding blade's temperature field images and the bonding surface's temperature distribution curves for monitoring the welding process. This is achieved by training the generator and discriminator of the conditional depth convolutional generative adversarial network separately. The network is then capable of predicting the bonding blade's heat distribution within a certain range of welding parameters. Compared to methods based on numerical simulation, this approach is far more effective. Training and testing datasets were created to assess the suggested method's efficacy. Multi-scale structural similarity index, structural similarity index, peak signal-to-noise ratio, difference hash and mean absolute error were used to evaluate the simulations' quality. The results show that the simulation's results and the actual data agree quite well. The temperature changes that occur during the welding process may be quickly and accurately simulated using the proposed technology. [ABSTRACT FROM AUTHOR]

• Deep features are extracted from the CT images using deep convolutional neural network model with dense layers. • Hyper edges are constructed for each images by using extracted deep features. A novel hypergraph convolutional neural network is constructed and trained by representational learning. • Evaluated the performance of the trained neural network with test data and attained ideal accuracy, precision, sensitivity, specificity, recall and F1-score. Kidney failure, also called as end-stage renal disease is the final stage of chronic kidney disease(CKD). Kidney failure cannot be revered and is the life-threatening if left untreated. Hence there is a demand for the early and automatic detection of CKD from radiology images based on deep learning techniques. This work focuses on the diagnosis of the predominant CKDs such as kidney stone, cyst and tumor from CT scan images. CT scan abdomen and urogram images are used to construct the digital-twin for the diagnosis of CKD in the robust manner. The proposed work is based on the dataset which contains 12,446 unique CT scan images, within it in which the cyst contains 3,709, normal 5,077, stone 1,377, and tumor 2,283. Deep features are extracted from these images and generated hyper edges from these features. These features are used to construct hypergraph representing the renal images. These hypergraphs are used in hypergraph convolutional neural network for representational learning. Finally, the model is validated by hold-out dataset. Deep learning metrics including precision, recall, accuracy, and FI score are used to validate the proposed model. It outperforms compared to other state of art algorithms with a superior validation accuracy of 99.71. The proposed model is a robust digital-twin for the early diagnosis of kidney diseases. It will assist the nephrologist for the better prognosis of kidney related abnormalities. [ABSTRACT FROM AUTHOR]

The digital twin concept represents an appealing opportunity to advance condition-based and predictive maintenance paradigms for civil engineering systems, thus allowing reduced lifecycle costs, increased system safety, and increased system availability. This work proposes a predictive digital twin approach to the health monitoring, maintenance, and management planning of civil engineering structures. The asset-twin coupled dynamical system is encoded employing a probabilistic graphical model, which allows all relevant sources of uncertainty to be taken into account. In particular, the time-repeating observations-to-decisions flow is modeled using a dynamic Bayesian network. Real-time structural health diagnostics are provided by assimilating sensed data with deep learning models. The digital twin state is continually updated in a sequential Bayesian inference fashion. This is then exploited to inform the optimal planning of maintenance and management actions within a dynamic decision-making framework. A preliminary offline phase involves the population of training datasets through a reduced-order numerical model and the computation of a health-dependent control policy. The strategy is assessed on two synthetic case studies, involving a cantilever beam and a railway bridge, demonstrating the dynamic decision-making capabilities of health-aware digital twins. • We present a digital twin framework for the SHM of civil structures. • The asset-twin dynamical system is encoded through a probabilistic graphical model. • Sensor recordings are assimilated with artificial neural networks. • A reduced-order model is employed offline to generate training vibration responses. • Our strategy allows predictive maintenance and management planning of structures. [ABSTRACT FROM AUTHOR]

Decoupling the physical world and providing standardized service interfaces is still challenging when developing Location Based Services (LBS). This lack also hinders the possibility of developing Intelligent services on top of LBS architectures. In this paper, we propose a multi-layer Digital Twin-based architecture that aims to enable the development of machine learning-based Intelligent LBS (I-LBS) that are able to adapt, evolve, and perform Continual Learning (CL). The platform uses Digital Twins to ensure physical abstraction and provide cyber–physical knowledge to the I-LBSs, which is defined as an execution graph of operation modules. Finally, we simulated a use-case for this platform in the complex scenario of Healthcare organization and management where the I-LBS classifies allowed/not allowed trajectories of users inside a real-existing hospital scenario depending on their role in the organization. The use case is implemented as a Deep Learning-based reconstruction task of high-resolution trajectories processed by the DT architecture that also deploys the I-LBS. The platform is evaluated in terms of physical complexity and computational time on the DT side and on both a traditional machine learning setting and a replay-based CL one for the intelligence side to demonstrate the flexibility and adaptability features introduced by the components for dynamic or unseen scenarios. [ABSTRACT FROM AUTHOR]

This work proposes a new method for the detection, localization, and classification of grid-connected photovoltaic (PV) array faults. Line-to-line, open-module, shorted-module, open-string, and shorted-string faults as well as partial shading conditions are studied. The proposed method has two stages, which are (1) detection and localization of faults and (2) classification of faults. In the first stage, detection and localization are performed using a digital twin (DT) by analyzing the current ratio of each PV array. The measured DC (direct current) power after the operation of the DC/DC boost converter in the physical object is firstly converted into a 2-dimensional image using a recurrence plot (RP) and is then inputted to the second stage. A deep learning-based shifted windows (swin) transformer optimized by particle swarm optimization (PSO) is used in the classification stage, eliminating the need for model tuning by trial-and-error. A PV system of ten arrays with 49 kW is studied. The coefficients of determination (R2) between the results of the digital object (digital twin) and the physical object for different scenarios demonstrate the accuracy and success of the digital twin. R2 values of 0.99988 for varying irradiation with constant temperature and 0.97923 for constant irradiation with varying temperature indicate strong correlations between the digital and physical objects, further confirming the applicability of the digital twin in PV fault detection. The comparative evaluation of the PSO-optimized swin transformer against classical machine learning algorithms and state-of-the-art convolutional neural networks reveals the superior performance of the proposed method. It achieves an outstanding classification accuracy of 98.55%, demonstrating its ability to effectively classify various types of PV faults. The results of the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, which measures the trade-off between true positive rate (TPR) and false positive rate (FPR), further illustrate the effectiveness of the proposed method for PV fault classification. • A model-based digital twin is used to detect and locate faults in a PV system. • Shifted Window Transformer with few trained variables is used to classify PV faults. • Topology of deep learning model is optimized by particle swarm optimization. [ABSTRACT FROM AUTHOR]